2024-09-16T00:08:35.822 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T00:08:35.848 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:08:35.943 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906386' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGbqyZUSid0sw0FwQwPvQuwMmQA/HJUSK4FqXwtHEJgmPhC+dmXkAZiR5YUetj3CuuisbFde2oRz2dy4LSzj1SE= smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+HWEfcpu5SThORc2OVJHRdFLAQ824i1A2Io7rsO8CFv2ADFmNjZk9mdWWjffvORHnpKx8Hb+srarbgzxmNK74= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOogsfbuKQKp9GoQHyFpHDXWMyzOhqd8m5Au5l7q4iSCudWQ+wGiXATCmofk4qddGhmrxDoJQ1lwKIzqesFt/Gs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T00:08:35.943 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-16T00:08:35.944 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-16T00:08:35.945 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T00:08:35.946 INFO:teuthology.task.internal:Checking packages... 2024-09-16T00:08:35.968 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-16T00:08:35.968 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:08:35.968 INFO:teuthology.packaging:ref: None 2024-09-16T00:08:35.968 INFO:teuthology.packaging:tag: None 2024-09-16T00:08:35.968 INFO:teuthology.packaging:branch: squid 2024-09-16T00:08:35.969 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:08:35.969 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-16T00:08:36.206 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-16T00:08:36.207 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T00:08:36.210 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T00:08:36.210 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T00:08:36.265 INFO:teuthology.task.internal:Saving configuration 2024-09-16T00:08:36.286 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T00:08:36.301 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T00:08:36.326 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:03:31.494964', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGbqyZUSid0sw0FwQwPvQuwMmQA/HJUSK4FqXwtHEJgmPhC+dmXkAZiR5YUetj3CuuisbFde2oRz2dy4LSzj1SE='} 2024-09-16T00:08:36.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:03:31.493327', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+HWEfcpu5SThORc2OVJHRdFLAQ824i1A2Io7rsO8CFv2ADFmNjZk9mdWWjffvORHnpKx8Hb+srarbgzxmNK74='} 2024-09-16T00:08:36.368 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:03:31.491863', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOogsfbuKQKp9GoQHyFpHDXWMyzOhqd8m5Au5l7q4iSCudWQ+wGiXATCmofk4qddGhmrxDoJQ1lwKIzqesFt/Gs='} 2024-09-16T00:08:36.368 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T00:08:36.376 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-16T00:08:36.376 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-16T00:08:36.376 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-16T00:08:36.376 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T00:08:36.542 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe279b47be0>, signals=[15]) 2024-09-16T00:08:36.542 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T00:08:36.554 INFO:teuthology.task.internal:Opening connections... 2024-09-16T00:08:36.554 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-09-16T00:08:36.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:08:36.630 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-09-16T00:08:36.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:08:36.703 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-09-16T00:08:36.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:08:36.780 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T00:08:36.786 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-09-16T00:08:36.804 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-09-16T00:08:36.804 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-09-16T00:08:36.861 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream" 2024-09-16T00:08:36.861 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="9" 2024-09-16T00:08:36.861 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2024-09-16T00:08:36.861 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:08:36.861 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="9" 2024-09-16T00:08:36.861 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:08:36.862 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:08:36.862 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2024-09-16T00:08:36.862 INFO:teuthology.orchestra.run.smithi001.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:08:36.862 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:08:36.862 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:08:36.862 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:08:36.862 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:08:36.862 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:08:36.863 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-09-16T00:08:36.891 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-09-16T00:08:36.909 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-09-16T00:08:36.909 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-09-16T00:08:36.968 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-09-16T00:08:36.968 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-09-16T00:08:36.968 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-09-16T00:08:36.968 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:08:36.968 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-09-16T00:08:36.968 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:08:36.968 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:08:36.968 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-09-16T00:08:36.968 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:08:36.969 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:08:36.969 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:08:36.969 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:08:36.969 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:08:36.969 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:08:36.969 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-09-16T00:08:36.990 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-09-16T00:08:37.009 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-09-16T00:08:37.010 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:08:37.069 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-09-16T00:08:37.090 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T00:08:37.109 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T00:08:37.132 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T00:08:37.133 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:08:37.135 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:08:37.137 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:08:37.154 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T00:08:37.165 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T00:08:37.165 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:08:37.192 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:08:37.195 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:08:37.300 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T00:08:37.422 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T00:08:37.510 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-16T00:08:37.510 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-16T00:08:37.511 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-16T00:08:37.511 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:08:37.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:08:37.529 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-16T00:08:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-503.el9.x86_64 2024-09-16T00:08:37.586 INFO:teuthology.task.kernel:Running kernel on smithi001: 5.14.0-503.el9.x86_64 2024-09-16T00:08:37.586 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum install -y kernel 2024-09-16T00:08:41.023 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-16T00:08:43.099 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-16T00:08:47.458 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-16T00:08:49.251 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-16T00:08:50.088 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-16T00:08:55.794 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T00:08:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-09-16T00:08:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-09-16T00:08:57.980 INFO:teuthology.orchestra.run.smithi001.stdout: Package Architecture Version Repository Size 2024-09-16T00:08:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-09-16T00:08:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-09-16T00:08:57.980 INFO:teuthology.orchestra.run.smithi001.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T00:08:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-09-16T00:08:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T00:08:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T00:08:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T00:08:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:08:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-09-16T00:08:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-09-16T00:08:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:Install 4 Packages 2024-09-16T00:08:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:08:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 79 M 2024-09-16T00:08:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 126 M 2024-09-16T00:08:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-09-16T00:08:58.120 INFO:teuthology.orchestra.run.smithi001.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 202 kB/s | 23 kB 00:00 2024-09-16T00:08:58.721 INFO:teuthology.orchestra.run.smithi001.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-16T00:08:59.362 INFO:teuthology.orchestra.run.smithi001.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 25 MB/s | 35 MB 00:01 2024-09-16T00:08:59.630 INFO:teuthology.orchestra.run.smithi001.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-16T00:08:59.630 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:08:59.630 INFO:teuthology.orchestra.run.smithi001.stdout:Total 48 MB/s | 79 MB 00:01 2024-09-16T00:09:00.338 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-09-16T00:09:00.737 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-09-16T00:09:00.737 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-09-16T00:09:01.706 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-09-16T00:09:01.706 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-09-16T00:09:04.284 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-09-16T00:09:04.950 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:09:05.038 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:09:07.828 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:09:13.092 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:09:13.134 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:09:18.281 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:09:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:09:57.473 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:09:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:09:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:09:58.723 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-09-16T00:09:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T00:09:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T00:09:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T00:09:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T00:09:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:09:58.723 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-09-16T00:09:59.064 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:09:59.192 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:09:59.193 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:09:59.193 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T00:09:59.193 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T00:09:59.193 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T00:09:59.193 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-16T00:09:59.194 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T00:09:59.194 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:09:59.253 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:09:59.253 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:09:59.253 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -qi grub2-tools 2024-09-16T00:09:59.290 INFO:teuthology.orchestra.run.smithi001.stdout:Name : grub2-tools 2024-09-16T00:09:59.290 INFO:teuthology.orchestra.run.smithi001.stdout:Epoch : 1 2024-09-16T00:09:59.290 INFO:teuthology.orchestra.run.smithi001.stdout:Version : 2.06 2024-09-16T00:09:59.290 INFO:teuthology.orchestra.run.smithi001.stdout:Release : 82.el9 2024-09-16T00:09:59.290 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: x86_64 2024-09-16T00:09:59.290 INFO:teuthology.orchestra.run.smithi001.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T00:09:59.290 INFO:teuthology.orchestra.run.smithi001.stdout:Group : Unspecified 2024-09-16T00:09:59.290 INFO:teuthology.orchestra.run.smithi001.stdout:Size : 8274670 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:License : GPLv3+ 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:Packager : builder@centos.org 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:Vendor : CentOS 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:Summary : Support tools for GRUB. 2024-09-16T00:09:59.291 INFO:teuthology.orchestra.run.smithi001.stdout:Description : 2024-09-16T00:09:59.292 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:09:59.292 INFO:teuthology.orchestra.run.smithi001.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T00:09:59.292 INFO:teuthology.orchestra.run.smithi001.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T00:09:59.292 INFO:teuthology.orchestra.run.smithi001.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T00:09:59.292 INFO:teuthology.orchestra.run.smithi001.stdout:hardware devices. 2024-09-16T00:09:59.292 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:09:59.292 INFO:teuthology.orchestra.run.smithi001.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T00:09:59.293 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T00:09:59.293 INFO:teuthology.task.kernel:Updating grub on smithi001 to boot 5.14.0-505.el9.x86_64 2024-09-16T00:09:59.294 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T00:09:59.969 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2024-09-16T00:10:01.322 INFO:teuthology.orchestra.run.smithi001.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T00:10:01.364 INFO:teuthology.orchestra.run.smithi001.stderr:done 2024-09-16T00:10:01.366 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-09-16T00:10:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.KSOjKNelLM 2024-09-16T00:10:01.382 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KSOjKNelLM 2024-09-16T00:10:01.461 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.KSOjKNelLM 2024-09-16T00:10:01.609 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.KSOjKNelLM is 6KB 2024-09-16T00:10:01.657 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.KSOjKNelLM 2024-09-16T00:10:01.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T00:10:01.735 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T00:10:01.735 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T00:10:01.735 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T00:10:01.735 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T00:10:01.736 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T00:10:01.885 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2024-09-16T00:10:31.888 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-16T00:10:31.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-16T00:10:31.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:11:31.892 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T00:11:40.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-16T00:11:40.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:11:41.413 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-16T00:11:41.820 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-16T00:11:41.820 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-16T00:11:41.820 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-16T00:11:41.838 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-505.el9.x86_64 2024-09-16T00:11:41.838 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T00:11:41.839 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T00:11:41.839 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T00:11:42.840 DEBUG:teuthology.parallel:result is None 2024-09-16T00:11:42.840 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T00:11:43.023 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T00:11:43.023 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:11:43.026 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:11:43.029 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:11:43.048 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T00:11:43.072 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T00:11:43.203 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T00:11:43.203 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:11:43.205 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:11:43.207 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:11:43.518 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T00:11:43.525 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T00:11:43.525 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:11:43.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:11:43.542 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:11:43.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:11:43.560 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:11:43.562 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:11:43.585 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:11:43.626 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:11:43.632 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:11:43.640 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:11:43.652 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:11:43.659 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:11:43.674 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:11:43.676 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T00:11:43.683 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T00:11:43.684 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:11:43.686 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:11:43.688 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:11:43.716 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T00:11:43.725 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T00:11:43.726 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:11:43.753 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:11:43.757 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:11:43.775 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:11:43.844 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:11:43.908 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:11:43.995 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:11:44.036 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:11:44.036 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:11:44.107 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:11:44.152 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:11:44.200 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:11:44.284 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:11:44.326 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:11:44.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:11:44.395 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:11:44.439 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:11:44.500 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:11:44.544 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:11:44.587 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:11:44.587 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:11:44.655 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-09-16T00:11:44.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-09-16T00:11:44.660 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-09-16T00:11:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:11:44.700 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:11:44.732 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:11:45.142 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T00:11:45.149 INFO:teuthology.task.internal:Starting timer... 2024-09-16T00:11:45.149 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T00:11:45.196 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T00:11:45.228 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-16T00:11:45.228 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2024-09-16T00:11:45.290 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: 2024-09-16T00:11:45.292 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-09-16T00:11:45.359 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-09-16T00:11:45.361 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2024-09-16T00:11:45.412 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2024-09-16T00:11:45.414 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T00:11:45.414 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2024-09-16T00:11:45.432 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2024-09-16T00:11:45.432 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-09-16T00:11:45.465 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-09-16T00:11:45.465 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2024-09-16T00:11:45.490 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2024-09-16T00:11:45.490 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive'} 2024-09-16T00:11:45.490 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:11:45.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:11:45.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:11:45.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:11:45.548 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:11:45.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:11:45.574 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T00:11:45.575 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:11:45.696 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:11:45.731 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:11:45.760 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T00:11:45.770 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T00:11:45.774 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T00:11:45.894 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T00:11:45.905 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T00:11:45.906 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi019.front.sepia.ceph.com,smithi153.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T00:15:50.304 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com')] 2024-09-16T00:15:50.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-16T00:15:50.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:15:50.401 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-16T00:15:50.481 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-16T00:15:50.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-16T00:15:50.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:15:50.609 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-09-16T00:15:50.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-16T00:15:50.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-16T00:15:50.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:15:50.767 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-09-16T00:15:50.851 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-16T00:15:50.852 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T00:15:51.087 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T00:15:51.087 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:15:51.087 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:15:51.090 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:15:51.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:15:51.093 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:15:51.093 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:15:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:15:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:15:51.128 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:15:51.140 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:15:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:15:51.145 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:15:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-09-16T00:15:51.168 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2024-09-16T00:15:51.171 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2024-09-16T00:15:51.178 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-09-16T00:15:51.180 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2024-09-16T00:15:51.183 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2024-09-16T00:15:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:15:51.196 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:15:51.198 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:15:51.208 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:15:51.212 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:15:51.213 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:15:51.266 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:15:51.270 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:15:51.270 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-09-16T00:15:51.270 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.270 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.270 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.270 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.271 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:15:51.273 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:15:51.274 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:15:51.274 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-09-16T00:15:51.274 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.274 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.274 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.274 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.276 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:15:51.276 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-16T00:15:51.276 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.276 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.276 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.277 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:15:51.277 INFO:teuthology.run_tasks:Running task install... 2024-09-16T00:15:51.286 DEBUG:teuthology.task.install:project ceph 2024-09-16T00:15:51.286 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T00:15:51.286 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T00:15:51.286 INFO:teuthology.task.install:Using flavor: default 2024-09-16T00:15:51.292 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T00:15:51.292 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T00:15:51.292 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:15:51.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:15:51.295 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:15:51.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:15:51.296 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:15:51.296 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:15:51.439 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:15:51.440 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:15:51.443 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:15:51.443 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:15:51.447 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:15:51.447 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:15:51.550 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:15:51.550 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:15:51.550 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:15:51.562 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:15:51.562 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:15:51.562 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:15:51.564 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:15:51.565 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:15:51.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:15:51.577 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:15:51.577 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:15:51.590 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:15:51.590 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:15:51.593 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:15:51.594 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:15:51.642 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:15:51.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:15:51.661 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:15:51.717 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:15:51.732 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:15:51.738 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:15:51.781 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2024-09-16T00:15:51.783 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2024-09-16T00:15:51.796 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-09-16T00:15:51.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-09-16T00:15:51.804 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2024-09-16T00:15:51.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2024-09-16T00:15:52.009 INFO:teuthology.orchestra.run.smithi153.stdout:45 files removed 2024-09-16T00:15:52.028 INFO:teuthology.orchestra.run.smithi019.stdout:45 files removed 2024-09-16T00:15:52.035 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:15:52.051 INFO:teuthology.orchestra.run.smithi001.stdout:45 files removed 2024-09-16T00:15:52.056 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:15:52.077 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:15:52.586 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 310 kB/s | 83 kB 00:00 2024-09-16T00:15:52.600 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 321 kB/s | 83 kB 00:00 2024-09-16T00:15:52.652 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 307 kB/s | 83 kB 00:00 2024-09-16T00:15:52.870 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-16T00:15:52.884 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-16T00:15:52.942 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-16T00:15:53.146 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T00:15:53.160 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T00:15:53.217 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-16T00:15:53.777 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-16T00:15:54.084 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.2 MB 00:00 2024-09-16T00:15:54.946 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-16T00:15:55.885 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-16T00:15:56.228 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-16T00:15:56.655 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-16T00:16:00.403 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-16T00:16:00.645 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-16T00:16:01.987 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-16T00:16:02.130 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 19 kB 00:00 2024-09-16T00:16:02.523 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.5 MB 00:02 2024-09-16T00:16:02.875 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T00:16:03.042 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-16T00:16:03.940 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 19 kB 00:00 2024-09-16T00:16:05.078 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-16T00:16:08.607 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T00:16:08.647 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-16T00:16:10.583 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-16T00:16:10.984 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:16:10.985 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:16:10.997 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:16:10.998 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:16:10.999 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:16:11.000 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:16:11.001 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:16:11.004 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:16:11.005 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:16:11.073 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-16T00:16:11.093 INFO:teuthology.orchestra.run.smithi019.stdout:====================================================================================== 2024-09-16T00:16:11.093 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout:====================================================================================== 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:16:11.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:16:11.095 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:16:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:16:11.097 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:16:11.098 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:16:11.099 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:16:11.100 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:16:11.101 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:16:11.102 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:16:11.103 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:16:11.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:16:11.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:16:11.106 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:16:11.107 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-09-16T00:16:11.107 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:16:11.107 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:16:11.107 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-16T00:16:11.107 INFO:teuthology.orchestra.run.smithi019.stdout:====================================================================================== 2024-09-16T00:16:11.107 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-09-16T00:16:11.107 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 8 Packages 2024-09-16T00:16:11.107 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:16:11.108 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 226 M 2024-09-16T00:16:11.108 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-16T00:16:11.111 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:16:11.112 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:16:11.126 INFO:teuthology.orchestra.run.smithi001.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:16:11.127 INFO:teuthology.orchestra.run.smithi001.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:16:11.128 INFO:teuthology.orchestra.run.smithi001.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:16:11.129 INFO:teuthology.orchestra.run.smithi001.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:16:11.131 INFO:teuthology.orchestra.run.smithi001.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:16:11.134 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:16:11.135 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:16:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-09-16T00:16:11.236 INFO:teuthology.orchestra.run.smithi001.stdout:====================================================================================== 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout:====================================================================================== 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:16:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:16:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:16:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-09-16T00:16:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:16:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:16:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:16:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:16:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:16:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:16:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:16:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:16:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:16:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:16:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:16:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:16:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:16:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:16:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:16:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:16:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:16:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:16:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:16:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:16:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:16:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:16:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:16:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:16:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout:Installing weak dependencies: 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout:====================================================================================== 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout:Install 148 Packages 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 8 Packages 2024-09-16T00:16:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:16:11.251 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 226 M 2024-09-16T00:16:11.251 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-09-16T00:16:11.949 INFO:teuthology.orchestra.run.smithi001.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 40 kB/s | 6.5 kB 00:00 2024-09-16T00:16:12.258 INFO:teuthology.orchestra.run.smithi019.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 32 kB/s | 6.5 kB 00:00 2024-09-16T00:16:12.266 INFO:teuthology.orchestra.run.smithi001.stdout:(2/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-16T00:16:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:(3/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 2.4 MB/s | 896 kB 00:00 2024-09-16T00:16:12.333 INFO:teuthology.orchestra.run.smithi001.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 2.1 MB/s | 145 kB 00:00 2024-09-16T00:16:12.534 INFO:teuthology.orchestra.run.smithi001.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-16T00:16:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.8 MB/s | 896 kB 00:00 2024-09-16T00:16:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.7 MB/s | 145 kB 00:00 2024-09-16T00:16:12.852 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:16:12.852 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:16:12.864 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:16:12.865 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:16:12.865 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:16:12.866 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:16:12.868 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:16:12.871 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:16:12.871 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:16:12.900 INFO:teuthology.orchestra.run.smithi001.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-16T00:16:12.934 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================== 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================== 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:16:12.949 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:16:12.950 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:16:12.951 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:16:12.952 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:16:12.953 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-09-16T00:16:12.953 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:16:12.953 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:16:12.953 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:16:12.953 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:16:12.954 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:16:12.955 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:16:12.956 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:16:12.957 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:16:12.958 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:16:12.959 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:16:12.960 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:16:12.961 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout:Installing weak dependencies: 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:16:12.962 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:16:12.963 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-09-16T00:16:12.963 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================== 2024-09-16T00:16:12.963 INFO:teuthology.orchestra.run.smithi153.stdout:Install 148 Packages 2024-09-16T00:16:12.963 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 8 Packages 2024-09-16T00:16:12.963 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:16:12.963 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 226 M 2024-09-16T00:16:12.963 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-09-16T00:16:12.984 INFO:teuthology.orchestra.run.smithi001.stdout:(7/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 3.5 MB/s | 2.4 MB 00:00 2024-09-16T00:16:13.009 INFO:teuthology.orchestra.run.smithi019.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 5.7 MB/s | 5.4 MB 00:00 2024-09-16T00:16:13.326 INFO:teuthology.orchestra.run.smithi019.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-16T00:16:13.393 INFO:teuthology.orchestra.run.smithi001.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 21 MB 00:01 2024-09-16T00:16:13.499 INFO:teuthology.orchestra.run.smithi001.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 239 kB/s | 25 kB 00:00 2024-09-16T00:16:13.501 INFO:teuthology.orchestra.run.smithi153.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 36 kB/s | 6.5 kB 00:00 2024-09-16T00:16:13.659 INFO:teuthology.orchestra.run.smithi019.stdout:(6/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 21 MB 00:01 2024-09-16T00:16:13.709 INFO:teuthology.orchestra.run.smithi019.stdout:(7/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 2.7 MB/s | 2.4 MB 00:00 2024-09-16T00:16:13.852 INFO:teuthology.orchestra.run.smithi001.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 18 MB/s | 17 MB 00:00 2024-09-16T00:16:13.943 INFO:teuthology.orchestra.run.smithi001.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 2.4 MB/s | 228 kB 00:00 2024-09-16T00:16:14.035 INFO:teuthology.orchestra.run.smithi001.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 295 kB/s | 27 kB 00:00 2024-09-16T00:16:14.127 INFO:teuthology.orchestra.run.smithi001.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 346 kB/s | 32 kB 00:00 2024-09-16T00:16:14.201 INFO:teuthology.orchestra.run.smithi019.stdout:(8/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 5.4 MB/s | 4.7 MB 00:00 2024-09-16T00:16:14.209 INFO:teuthology.orchestra.run.smithi153.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 896 kB 00:00 2024-09-16T00:16:14.244 INFO:teuthology.orchestra.run.smithi001.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 6.3 MB/s | 751 kB 00:00 2024-09-16T00:16:14.293 INFO:teuthology.orchestra.run.smithi019.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 275 kB/s | 25 kB 00:00 2024-09-16T00:16:14.336 INFO:teuthology.orchestra.run.smithi001.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.7 MB/s | 163 kB 00:00 2024-09-16T00:16:14.352 INFO:teuthology.orchestra.run.smithi153.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.0 MB/s | 145 kB 00:00 2024-09-16T00:16:14.428 INFO:teuthology.orchestra.run.smithi001.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.4 MB/s | 127 kB 00:00 2024-09-16T00:16:14.537 INFO:teuthology.orchestra.run.smithi001.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 4.6 MB/s | 506 kB 00:00 2024-09-16T00:16:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 14 MB/s | 17 MB 00:01 2024-09-16T00:16:14.977 INFO:teuthology.orchestra.run.smithi019.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 2.1 MB/s | 228 kB 00:00 2024-09-16T00:16:15.069 INFO:teuthology.orchestra.run.smithi019.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 294 kB/s | 27 kB 00:00 2024-09-16T00:16:15.145 INFO:teuthology.orchestra.run.smithi019.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 423 kB/s | 32 kB 00:00 2024-09-16T00:16:15.162 INFO:teuthology.orchestra.run.smithi001.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 8.6 MB/s | 5.3 MB 00:00 2024-09-16T00:16:15.228 INFO:teuthology.orchestra.run.smithi019.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 8.8 MB/s | 751 kB 00:00 2024-09-16T00:16:15.237 INFO:teuthology.orchestra.run.smithi001.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 603 kB/s | 45 kB 00:00 2024-09-16T00:16:15.252 INFO:teuthology.orchestra.run.smithi153.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 2.8 MB/s | 5.4 MB 00:01 2024-09-16T00:16:15.321 INFO:teuthology.orchestra.run.smithi001.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-16T00:16:15.352 INFO:teuthology.orchestra.run.smithi153.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 2.4 MB/s | 2.4 MB 00:01 2024-09-16T00:16:15.387 INFO:teuthology.orchestra.run.smithi019.stdout:(15/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 6.3 MB/s | 11 MB 00:01 2024-09-16T00:16:15.396 INFO:teuthology.orchestra.run.smithi001.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.1 MB/s | 161 kB 00:00 2024-09-16T00:16:15.412 INFO:teuthology.orchestra.run.smithi019.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 889 kB/s | 163 kB 00:00 2024-09-16T00:16:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.1 MB/s | 127 kB 00:00 2024-09-16T00:16:15.480 INFO:teuthology.orchestra.run.smithi001.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-16T00:16:15.496 INFO:teuthology.orchestra.run.smithi019.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-16T00:16:15.563 INFO:teuthology.orchestra.run.smithi001.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.5 MB/s | 300 kB 00:00 2024-09-16T00:16:15.579 INFO:teuthology.orchestra.run.smithi019.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 542 kB/s | 45 kB 00:00 2024-09-16T00:16:15.638 INFO:teuthology.orchestra.run.smithi001.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.3 MB/s | 99 kB 00:00 2024-09-16T00:16:15.661 INFO:teuthology.orchestra.run.smithi153.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 3.7 MB/s | 1.5 MB 00:00 2024-09-16T00:16:15.672 INFO:teuthology.orchestra.run.smithi019.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.4 MB/s | 136 kB 00:00 2024-09-16T00:16:15.714 INFO:teuthology.orchestra.run.smithi001.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-16T00:16:15.756 INFO:teuthology.orchestra.run.smithi019.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.9 MB/s | 161 kB 00:00 2024-09-16T00:16:15.856 INFO:teuthology.orchestra.run.smithi019.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.1 MB/s | 320 kB 00:00 2024-09-16T00:16:15.915 INFO:teuthology.orchestra.run.smithi019.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-16T00:16:15.939 INFO:teuthology.orchestra.run.smithi001.stdout:(26/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 3.6 MB/s | 11 MB 00:02 2024-09-16T00:16:15.982 INFO:teuthology.orchestra.run.smithi019.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.5 MB/s | 99 kB 00:00 2024-09-16T00:16:16.049 INFO:teuthology.orchestra.run.smithi019.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-16T00:16:16.102 INFO:teuthology.orchestra.run.smithi153.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 6.3 MB/s | 4.7 MB 00:00 2024-09-16T00:16:16.132 INFO:teuthology.orchestra.run.smithi019.stdout:(26/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 7.8 MB/s | 5.3 MB 00:00 2024-09-16T00:16:16.199 INFO:teuthology.orchestra.run.smithi019.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-16T00:16:16.291 INFO:teuthology.orchestra.run.smithi019.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-16T00:16:16.317 INFO:teuthology.orchestra.run.smithi019.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 272 kB/s | 32 kB 00:00 2024-09-16T00:16:16.342 INFO:teuthology.orchestra.run.smithi019.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.8 MB/s | 146 kB 00:00 2024-09-16T00:16:16.364 INFO:teuthology.orchestra.run.smithi001.stdout:(27/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 17 MB/s | 49 MB 00:02 2024-09-16T00:16:16.390 INFO:teuthology.orchestra.run.smithi001.stdout:(28/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 379 kB/s | 170 kB 00:00 2024-09-16T00:16:16.457 INFO:teuthology.orchestra.run.smithi001.stdout:(29/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 4.2 MB/s | 3.1 MB 00:00 2024-09-16T00:16:16.482 INFO:teuthology.orchestra.run.smithi001.stdout:(30/156): ceph-grafana-dashboards-19.1.1-326.g8 272 kB/s | 32 kB 00:00 2024-09-16T00:16:16.515 INFO:teuthology.orchestra.run.smithi001.stdout:(31/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.1 MB/s | 146 kB 00:00 2024-09-16T00:16:16.699 INFO:teuthology.orchestra.run.smithi001.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 1.3 MB/s | 249 kB 00:00 2024-09-16T00:16:16.791 INFO:teuthology.orchestra.run.smithi001.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 537 kB/s | 49 kB 00:00 2024-09-16T00:16:16.851 INFO:teuthology.orchestra.run.smithi019.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 6.7 MB/s | 3.6 MB 00:00 2024-09-16T00:16:16.867 INFO:teuthology.orchestra.run.smithi001.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 220 kB/s | 16 kB 00:00 2024-09-16T00:16:16.917 INFO:teuthology.orchestra.run.smithi019.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 3.7 MB/s | 249 kB 00:00 2024-09-16T00:16:16.925 INFO:teuthology.orchestra.run.smithi001.stdout:(35/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 7.6 MB/s | 3.6 MB 00:00 2024-09-16T00:16:16.927 INFO:teuthology.orchestra.run.smithi153.stdout:(8/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9. 13 MB/s | 11 MB 00:00 2024-09-16T00:16:16.976 INFO:teuthology.orchestra.run.smithi019.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 848 kB/s | 49 kB 00:00 2024-09-16T00:16:16.984 INFO:teuthology.orchestra.run.smithi001.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 2.4 MB/s | 285 kB 00:00 2024-09-16T00:16:16.986 INFO:teuthology.orchestra.run.smithi153.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 431 kB/s | 25 kB 00:00 2024-09-16T00:16:17.042 INFO:teuthology.orchestra.run.smithi001.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-16T00:16:17.060 INFO:teuthology.orchestra.run.smithi019.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 10 MB/s | 7.4 MB 00:00 2024-09-16T00:16:17.068 INFO:teuthology.orchestra.run.smithi001.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 218 kB/s | 18 kB 00:00 2024-09-16T00:16:17.085 INFO:teuthology.orchestra.run.smithi019.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 151 kB/s | 16 kB 00:00 2024-09-16T00:16:17.118 INFO:teuthology.orchestra.run.smithi019.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.8 MB/s | 285 kB 00:00 2024-09-16T00:16:17.160 INFO:teuthology.orchestra.run.smithi001.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-16T00:16:17.177 INFO:teuthology.orchestra.run.smithi019.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 313 kB/s | 18 kB 00:00 2024-09-16T00:16:17.211 INFO:teuthology.orchestra.run.smithi019.stdout:(38/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 6.0 MB/s | 765 kB 00:00 2024-09-16T00:16:17.268 INFO:teuthology.orchestra.run.smithi001.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 354 kB/s | 80 kB 00:00 2024-09-16T00:16:17.377 INFO:teuthology.orchestra.run.smithi001.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-16T00:16:17.402 INFO:teuthology.orchestra.run.smithi001.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 790 kB/s | 191 kB 00:00 2024-09-16T00:16:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.1 MB/s | 801 kB 00:00 2024-09-16T00:16:17.461 INFO:teuthology.orchestra.run.smithi001.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-16T00:16:17.478 INFO:teuthology.orchestra.run.smithi019.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 266 kB/s | 80 kB 00:00 2024-09-16T00:16:17.495 INFO:teuthology.orchestra.run.smithi001.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-16T00:16:17.520 INFO:teuthology.orchestra.run.smithi001.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-16T00:16:17.554 INFO:teuthology.orchestra.run.smithi001.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-16T00:16:17.595 INFO:teuthology.orchestra.run.smithi001.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-16T00:16:17.728 INFO:teuthology.orchestra.run.smithi019.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 764 kB/s | 191 kB 00:00 2024-09-16T00:16:17.895 INFO:teuthology.orchestra.run.smithi019.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-16T00:16:17.929 INFO:teuthology.orchestra.run.smithi019.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-16T00:16:17.936 INFO:teuthology.orchestra.run.smithi153.stdout:(10/156): ceph-common-19.1.1-326.g8921c8ef.el9. 4.6 MB/s | 21 MB 00:04 2024-09-16T00:16:18.021 INFO:teuthology.orchestra.run.smithi001.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 251 kB/s | 107 kB 00:00 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi019.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 853 kB/s | 106 kB 00:00 2024-09-16T00:16:18.104 INFO:teuthology.orchestra.run.smithi153.stdout:(11/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 6.9 MB/s | 17 MB 00:02 2024-09-16T00:16:18.129 INFO:teuthology.orchestra.run.smithi153.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.2 MB/s | 228 kB 00:00 2024-09-16T00:16:18.137 INFO:teuthology.orchestra.run.smithi019.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-16T00:16:18.154 INFO:teuthology.orchestra.run.smithi153.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 542 kB/s | 27 kB 00:00 2024-09-16T00:16:18.179 INFO:teuthology.orchestra.run.smithi153.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 631 kB/s | 32 kB 00:00 2024-09-16T00:16:18.205 INFO:teuthology.orchestra.run.smithi019.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-16T00:16:18.229 INFO:teuthology.orchestra.run.smithi153.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 3.2 MB/s | 163 kB 00:00 2024-09-16T00:16:18.254 INFO:teuthology.orchestra.run.smithi001.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-16T00:16:18.271 INFO:teuthology.orchestra.run.smithi153.stdout:(16/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 6.3 MB/s | 751 kB 00:00 2024-09-16T00:16:18.280 INFO:teuthology.orchestra.run.smithi001.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 907 kB/s | 689 kB 00:00 2024-09-16T00:16:18.296 INFO:teuthology.orchestra.run.smithi153.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.9 MB/s | 127 kB 00:00 2024-09-16T00:16:18.305 INFO:teuthology.orchestra.run.smithi001.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 76 kB/s | 22 kB 00:00 2024-09-16T00:16:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-16T00:16:18.355 INFO:teuthology.orchestra.run.smithi153.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-16T00:16:18.363 INFO:teuthology.orchestra.run.smithi019.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-09-16T00:16:18.406 INFO:teuthology.orchestra.run.smithi153.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 903 kB/s | 45 kB 00:00 2024-09-16T00:16:18.472 INFO:teuthology.orchestra.run.smithi153.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.0 MB/s | 136 kB 00:00 2024-09-16T00:16:18.497 INFO:teuthology.orchestra.run.smithi019.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-16T00:16:18.539 INFO:teuthology.orchestra.run.smithi019.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-09-16T00:16:18.540 INFO:teuthology.orchestra.run.smithi153.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.4 MB/s | 161 kB 00:00 2024-09-16T00:16:18.547 INFO:teuthology.orchestra.run.smithi001.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 122 kB/s | 33 kB 00:00 2024-09-16T00:16:18.564 INFO:teuthology.orchestra.run.smithi019.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-16T00:16:18.589 INFO:teuthology.orchestra.run.smithi019.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-16T00:16:18.615 INFO:teuthology.orchestra.run.smithi019.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-16T00:16:18.623 INFO:teuthology.orchestra.run.smithi153.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-16T00:16:18.640 INFO:teuthology.orchestra.run.smithi019.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-16T00:16:18.682 INFO:teuthology.orchestra.run.smithi153.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-16T00:16:18.682 INFO:teuthology.orchestra.run.smithi019.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-16T00:16:18.741 INFO:teuthology.orchestra.run.smithi153.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-16T00:16:18.741 INFO:teuthology.orchestra.run.smithi019.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-16T00:16:18.766 INFO:teuthology.orchestra.run.smithi019.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-16T00:16:18.791 INFO:teuthology.orchestra.run.smithi153.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T00:16:18.791 INFO:teuthology.orchestra.run.smithi019.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-16T00:16:18.814 INFO:teuthology.orchestra.run.smithi001.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 68 kB/s | 18 kB 00:00 2024-09-16T00:16:18.817 INFO:teuthology.orchestra.run.smithi019.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-16T00:16:18.839 INFO:teuthology.orchestra.run.smithi001.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 536 kB/s | 313 kB 00:00 2024-09-16T00:16:18.850 INFO:teuthology.orchestra.run.smithi019.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-16T00:16:18.886 INFO:teuthology.orchestra.run.smithi019.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-16T00:16:18.909 INFO:teuthology.orchestra.run.smithi019.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-16T00:16:18.934 INFO:teuthology.orchestra.run.smithi019.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-16T00:16:18.949 INFO:teuthology.orchestra.run.smithi153.stdout:(26/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 8.2 MB/s | 5.3 MB 00:00 2024-09-16T00:16:18.960 INFO:teuthology.orchestra.run.smithi019.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-16T00:16:18.985 INFO:teuthology.orchestra.run.smithi019.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T00:16:18.997 INFO:teuthology.orchestra.run.smithi001.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 61 kB/s | 11 kB 00:00 2024-09-16T00:16:19.008 INFO:teuthology.orchestra.run.smithi153.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-16T00:16:19.010 INFO:teuthology.orchestra.run.smithi019.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-16T00:16:19.058 INFO:teuthology.orchestra.run.smithi153.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 639 kB/s | 32 kB 00:00 2024-09-16T00:16:19.108 INFO:teuthology.orchestra.run.smithi153.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-16T00:16:19.153 INFO:teuthology.orchestra.run.smithi019.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 30 MB/s | 4.9 MB 00:00 2024-09-16T00:16:19.165 INFO:teuthology.orchestra.run.smithi001.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-09-16T00:16:19.184 INFO:teuthology.orchestra.run.smithi153.stdout:(30/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 7.9 MB/s | 3.1 MB 00:00 2024-09-16T00:16:19.219 INFO:teuthology.orchestra.run.smithi019.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-16T00:16:19.261 INFO:teuthology.orchestra.run.smithi019.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T00:16:19.295 INFO:teuthology.orchestra.run.smithi019.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-16T00:16:19.320 INFO:teuthology.orchestra.run.smithi019.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-16T00:16:19.340 INFO:teuthology.orchestra.run.smithi001.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 132 kB/s | 45 kB 00:00 2024-09-16T00:16:19.365 INFO:teuthology.orchestra.run.smithi001.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 305 kB/s | 160 kB 00:00 2024-09-16T00:16:19.387 INFO:teuthology.orchestra.run.smithi019.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-16T00:16:19.462 INFO:teuthology.orchestra.run.smithi019.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-09-16T00:16:19.479 INFO:teuthology.orchestra.run.smithi019.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 954 kB/s | 87 kB 00:00 2024-09-16T00:16:19.529 INFO:teuthology.orchestra.run.smithi019.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-16T00:16:19.605 INFO:teuthology.orchestra.run.smithi019.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-09-16T00:16:19.630 INFO:teuthology.orchestra.run.smithi019.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-16T00:16:19.632 INFO:teuthology.orchestra.run.smithi001.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 92 kB/s | 24 kB 00:00 2024-09-16T00:16:19.655 INFO:teuthology.orchestra.run.smithi019.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-16T00:16:19.667 INFO:teuthology.orchestra.run.smithi153.stdout:(31/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 18 MB/s | 49 MB 00:02 2024-09-16T00:16:19.681 INFO:teuthology.orchestra.run.smithi019.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-16T00:16:19.708 INFO:teuthology.orchestra.run.smithi019.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-16T00:16:19.717 INFO:teuthology.orchestra.run.smithi153.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 4.9 MB/s | 249 kB 00:00 2024-09-16T00:16:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-16T00:16:19.756 INFO:teuthology.orchestra.run.smithi019.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-16T00:16:19.776 INFO:teuthology.orchestra.run.smithi153.stdout:(33/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 5.3 MB/s | 3.6 MB 00:00 2024-09-16T00:16:19.801 INFO:teuthology.orchestra.run.smithi153.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 592 kB/s | 49 kB 00:00 2024-09-16T00:16:19.807 INFO:teuthology.orchestra.run.smithi019.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-16T00:16:19.807 INFO:teuthology.orchestra.run.smithi001.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.0 MB/s | 662 kB 00:00 2024-09-16T00:16:19.826 INFO:teuthology.orchestra.run.smithi153.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 328 kB/s | 16 kB 00:00 2024-09-16T00:16:19.852 INFO:teuthology.orchestra.run.smithi153.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.5 MB/s | 285 kB 00:00 2024-09-16T00:16:19.891 INFO:teuthology.orchestra.run.smithi019.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-09-16T00:16:19.894 INFO:teuthology.orchestra.run.smithi153.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 441 kB/s | 18 kB 00:00 2024-09-16T00:16:19.916 INFO:teuthology.orchestra.run.smithi019.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-16T00:16:19.933 INFO:teuthology.orchestra.run.smithi001.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 980 kB/s | 581 kB 00:00 2024-09-16T00:16:19.941 INFO:teuthology.orchestra.run.smithi019.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T00:16:19.944 INFO:teuthology.orchestra.run.smithi153.stdout:(38/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi019.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi001.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 138 kB/s | 47 kB 00:00 2024-09-16T00:16:19.991 INFO:teuthology.orchestra.run.smithi019.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-16T00:16:20.017 INFO:teuthology.orchestra.run.smithi019.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T00:16:20.050 INFO:teuthology.orchestra.run.smithi019.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-16T00:16:20.076 INFO:teuthology.orchestra.run.smithi019.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-09-16T00:16:20.101 INFO:teuthology.orchestra.run.smithi019.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T00:16:20.136 INFO:teuthology.orchestra.run.smithi153.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 7.8 MB/s | 7.4 MB 00:00 2024-09-16T00:16:20.151 INFO:teuthology.orchestra.run.smithi019.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-09-16T00:16:20.161 INFO:teuthology.orchestra.run.smithi153.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 299 kB/s | 80 kB 00:00 2024-09-16T00:16:20.193 INFO:teuthology.orchestra.run.smithi019.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-16T00:16:20.228 INFO:teuthology.orchestra.run.smithi153.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-16T00:16:20.235 INFO:teuthology.orchestra.run.smithi019.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-16T00:16:20.250 INFO:teuthology.orchestra.run.smithi001.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 140 kB/s | 38 kB 00:00 2024-09-16T00:16:20.270 INFO:teuthology.orchestra.run.smithi153.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-16T00:16:20.277 INFO:teuthology.orchestra.run.smithi019.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 864 kB/s | 36 kB 00:00 2024-09-16T00:16:20.325 INFO:teuthology.orchestra.run.smithi001.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 565 kB/s | 292 kB 00:00 2024-09-16T00:16:20.329 INFO:teuthology.orchestra.run.smithi153.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-16T00:16:20.379 INFO:teuthology.orchestra.run.smithi153.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-16T00:16:20.421 INFO:teuthology.orchestra.run.smithi153.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 976 kB/s | 253 kB 00:00 2024-09-16T00:16:20.446 INFO:teuthology.orchestra.run.smithi153.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-16T00:16:20.459 INFO:teuthology.orchestra.run.smithi001.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 358 kB/s | 188 kB 00:00 2024-09-16T00:16:20.505 INFO:teuthology.orchestra.run.smithi153.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-09-16T00:16:20.530 INFO:teuthology.orchestra.run.smithi153.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-16T00:16:20.639 INFO:teuthology.orchestra.run.smithi153.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 988 kB/s | 107 kB 00:00 2024-09-16T00:16:20.672 INFO:teuthology.orchestra.run.smithi153.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-16T00:16:20.698 INFO:teuthology.orchestra.run.smithi153.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-16T00:16:20.702 INFO:teuthology.orchestra.run.smithi019.stdout:(96/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 7.6 MB/s | 49 MB 00:06 2024-09-16T00:16:20.728 INFO:teuthology.orchestra.run.smithi019.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 105 kB/s | 47 kB 00:00 2024-09-16T00:16:20.740 INFO:teuthology.orchestra.run.smithi153.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-16T00:16:20.773 INFO:teuthology.orchestra.run.smithi153.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-09-16T00:16:20.807 INFO:teuthology.orchestra.run.smithi153.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-16T00:16:20.833 INFO:teuthology.orchestra.run.smithi153.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-16T00:16:20.858 INFO:teuthology.orchestra.run.smithi153.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-16T00:16:20.899 INFO:teuthology.orchestra.run.smithi153.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-16T00:16:20.903 INFO:teuthology.orchestra.run.smithi019.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-16T00:16:20.925 INFO:teuthology.orchestra.run.smithi153.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-16T00:16:20.928 INFO:teuthology.orchestra.run.smithi019.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 289 kB/s | 58 kB 00:00 2024-09-16T00:16:20.954 INFO:teuthology.orchestra.run.smithi019.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-16T00:16:20.979 INFO:teuthology.orchestra.run.smithi019.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T00:16:20.983 INFO:teuthology.orchestra.run.smithi153.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 9.8 MB/s | 581 kB 00:00 2024-09-16T00:16:21.004 INFO:teuthology.orchestra.run.smithi019.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-09-16T00:16:21.025 INFO:teuthology.orchestra.run.smithi153.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 588 kB/s | 24 kB 00:00 2024-09-16T00:16:21.038 INFO:teuthology.orchestra.run.smithi019.stdout:(103/156): socat-1.7.4.1-6.el9.x86_64.rpm 909 kB/s | 304 kB 00:00 2024-09-16T00:16:21.051 INFO:teuthology.orchestra.run.smithi153.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-16T00:16:21.076 INFO:teuthology.orchestra.run.smithi153.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-16T00:16:21.088 INFO:teuthology.orchestra.run.smithi019.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-16T00:16:21.109 INFO:teuthology.orchestra.run.smithi153.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-16T00:16:21.122 INFO:teuthology.orchestra.run.smithi019.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-16T00:16:21.143 INFO:teuthology.orchestra.run.smithi153.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-09-16T00:16:21.147 INFO:teuthology.orchestra.run.smithi019.stdout:(106/156): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-09-16T00:16:21.167 INFO:teuthology.orchestra.run.smithi001.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 5.4 MB/s | 4.9 MB 00:00 2024-09-16T00:16:21.168 INFO:teuthology.orchestra.run.smithi153.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-16T00:16:21.172 INFO:teuthology.orchestra.run.smithi019.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-09-16T00:16:21.206 INFO:teuthology.orchestra.run.smithi019.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T00:16:21.209 INFO:teuthology.orchestra.run.smithi001.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-09-16T00:16:21.231 INFO:teuthology.orchestra.run.smithi019.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-16T00:16:21.256 INFO:teuthology.orchestra.run.smithi019.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T00:16:21.276 INFO:teuthology.orchestra.run.smithi001.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-09-16T00:16:21.281 INFO:teuthology.orchestra.run.smithi019.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-16T00:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-16T00:16:21.357 INFO:teuthology.orchestra.run.smithi019.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-16T00:16:21.399 INFO:teuthology.orchestra.run.smithi019.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-16T00:16:21.416 INFO:teuthology.orchestra.run.smithi019.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 2.2 MB/s | 32 kB 00:00 2024-09-16T00:16:21.433 INFO:teuthology.orchestra.run.smithi019.stdout:(116/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 576 kB/s | 43 kB 00:00 2024-09-16T00:16:21.450 INFO:teuthology.orchestra.run.smithi019.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 421 kB/s | 14 kB 00:00 2024-09-16T00:16:21.484 INFO:teuthology.orchestra.run.smithi019.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-16T00:16:21.509 INFO:teuthology.orchestra.run.smithi019.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-16T00:16:21.545 INFO:teuthology.orchestra.run.smithi019.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-16T00:16:21.560 INFO:teuthology.orchestra.run.smithi153.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2024-09-16T00:16:21.568 INFO:teuthology.orchestra.run.smithi019.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-16T00:16:21.601 INFO:teuthology.orchestra.run.smithi019.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-16T00:16:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-16T00:16:21.635 INFO:teuthology.orchestra.run.smithi001.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 323 kB/s | 137 kB 00:00 2024-09-16T00:16:21.652 INFO:teuthology.orchestra.run.smithi019.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-16T00:16:21.677 INFO:teuthology.orchestra.run.smithi019.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 938 kB/s | 23 kB 00:00 2024-09-16T00:16:21.702 INFO:teuthology.orchestra.run.smithi019.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-16T00:16:21.728 INFO:teuthology.orchestra.run.smithi019.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-16T00:16:21.811 INFO:teuthology.orchestra.run.smithi019.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-09-16T00:16:21.837 INFO:teuthology.orchestra.run.smithi019.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:16:21.853 INFO:teuthology.orchestra.run.smithi019.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 4.7 MB/s | 79 kB 00:00 2024-09-16T00:16:21.860 INFO:teuthology.orchestra.run.smithi001.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 1.3 MB/s | 910 kB 00:00 2024-09-16T00:16:21.887 INFO:teuthology.orchestra.run.smithi019.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-16T00:16:21.921 INFO:teuthology.orchestra.run.smithi019.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-16T00:16:21.963 INFO:teuthology.orchestra.run.smithi019.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.9 MB/s | 2.0 MB 00:00 2024-09-16T00:16:21.988 INFO:teuthology.orchestra.run.smithi019.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-09-16T00:16:21.994 INFO:teuthology.orchestra.run.smithi153.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-16T00:16:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-16T00:16:22.039 INFO:teuthology.orchestra.run.smithi019.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-16T00:16:22.064 INFO:teuthology.orchestra.run.smithi019.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-16T00:16:22.089 INFO:teuthology.orchestra.run.smithi019.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-16T00:16:22.115 INFO:teuthology.orchestra.run.smithi019.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-16T00:16:22.140 INFO:teuthology.orchestra.run.smithi019.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-16T00:16:22.180 INFO:teuthology.orchestra.run.smithi019.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-16T00:16:22.207 INFO:teuthology.orchestra.run.smithi019.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 809 kB/s | 74 kB 00:00 2024-09-16T00:16:22.232 INFO:teuthology.orchestra.run.smithi019.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-16T00:16:22.252 INFO:teuthology.orchestra.run.smithi001.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 5.9 MB/s | 5.8 MB 00:00 2024-09-16T00:16:22.252 INFO:teuthology.orchestra.run.smithi153.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2024-09-16T00:16:22.257 INFO:teuthology.orchestra.run.smithi019.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-16T00:16:22.274 INFO:teuthology.orchestra.run.smithi019.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-09-16T00:16:22.277 INFO:teuthology.orchestra.run.smithi001.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 210 kB/s | 87 kB 00:00 2024-09-16T00:16:22.304 INFO:teuthology.orchestra.run.smithi153.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 2.9 MB/s | 910 kB 00:00 2024-09-16T00:16:22.308 INFO:teuthology.orchestra.run.smithi019.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T00:16:22.333 INFO:teuthology.orchestra.run.smithi019.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-16T00:16:22.361 INFO:teuthology.orchestra.run.smithi153.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-16T00:16:22.383 INFO:teuthology.orchestra.run.smithi019.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T00:16:22.411 INFO:teuthology.orchestra.run.smithi001.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2024-09-16T00:16:22.612 INFO:teuthology.orchestra.run.smithi153.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.4 MB/s | 4.9 MB 00:01 2024-09-16T00:16:22.628 INFO:teuthology.orchestra.run.smithi001.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 175 kB/s | 61 kB 00:00 2024-09-16T00:16:22.634 INFO:teuthology.orchestra.run.smithi019.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-16T00:16:22.670 INFO:teuthology.orchestra.run.smithi153.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-16T00:16:22.704 INFO:teuthology.orchestra.run.smithi153.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-09-16T00:16:22.709 INFO:teuthology.orchestra.run.smithi019.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 8.4 MB/s | 3.2 MB 00:00 2024-09-16T00:16:22.737 INFO:teuthology.orchestra.run.smithi153.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-16T00:16:22.829 INFO:teuthology.orchestra.run.smithi153.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-16T00:16:22.910 INFO:teuthology.orchestra.run.smithi019.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 204 kB/s | 41 kB 00:00 2024-09-16T00:16:22.920 INFO:teuthology.orchestra.run.smithi001.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 481 kB/s | 245 kB 00:00 2024-09-16T00:16:22.929 INFO:teuthology.orchestra.run.smithi153.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-09-16T00:16:22.963 INFO:teuthology.orchestra.run.smithi153.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 770 kB/s | 26 kB 00:00 2024-09-16T00:16:23.005 INFO:teuthology.orchestra.run.smithi153.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 462 kB/s | 19 kB 00:00 2024-09-16T00:16:23.072 INFO:teuthology.orchestra.run.smithi153.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-16T00:16:23.114 INFO:teuthology.orchestra.run.smithi153.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 838 kB/s | 35 kB 00:00 2024-09-16T00:16:23.168 INFO:teuthology.orchestra.run.smithi019.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 964 kB/s | 249 kB 00:00 2024-09-16T00:16:23.170 INFO:teuthology.orchestra.run.smithi001.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 6.5 MB/s | 6.0 MB 00:00 2024-09-16T00:16:23.187 INFO:teuthology.orchestra.run.smithi001.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 96 kB/s | 26 kB 00:00 2024-09-16T00:16:23.212 INFO:teuthology.orchestra.run.smithi001.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 425 kB/s | 248 kB 00:00 2024-09-16T00:16:23.297 INFO:teuthology.orchestra.run.smithi153.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 5.8 MB/s | 5.8 MB 00:00 2024-09-16T00:16:23.339 INFO:teuthology.orchestra.run.smithi153.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-16T00:16:23.392 INFO:teuthology.orchestra.run.smithi153.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-16T00:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 2.3 MB/s | 2.3 MB 00:01 2024-09-16T00:16:23.432 INFO:teuthology.orchestra.run.smithi153.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-16T00:16:23.437 INFO:teuthology.orchestra.run.smithi001.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 19 kB 00:00 2024-09-16T00:16:23.474 INFO:teuthology.orchestra.run.smithi153.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-16T00:16:23.487 INFO:teuthology.orchestra.run.smithi001.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 126 kB/s | 35 kB 00:00 2024-09-16T00:16:23.516 INFO:teuthology.orchestra.run.smithi153.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-16T00:16:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 2.0 MB/s | 1.8 MB 00:00 2024-09-16T00:16:23.566 INFO:teuthology.orchestra.run.smithi153.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T00:16:23.608 INFO:teuthology.orchestra.run.smithi153.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 6.6 MB/s | 279 kB 00:00 2024-09-16T00:16:23.621 INFO:teuthology.orchestra.run.smithi001.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 396 kB/s | 172 kB 00:00 2024-09-16T00:16:23.650 INFO:teuthology.orchestra.run.smithi153.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-16T00:16:23.692 INFO:teuthology.orchestra.run.smithi153.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-16T00:16:23.777 INFO:teuthology.orchestra.run.smithi019.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 2.2 MB/s | 1.4 MB 00:00 2024-09-16T00:16:24.022 INFO:teuthology.orchestra.run.smithi001.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 469 kB/s | 250 kB 00:00 2024-09-16T00:16:24.025 INFO:teuthology.orchestra.run.smithi153.stdout:(91/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2024-09-16T00:16:24.067 INFO:teuthology.orchestra.run.smithi153.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-16T00:16:24.109 INFO:teuthology.orchestra.run.smithi153.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-16T00:16:24.130 INFO:teuthology.orchestra.run.smithi001.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 437 kB/s | 222 kB 00:00 2024-09-16T00:16:24.143 INFO:teuthology.orchestra.run.smithi019.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.2 MB/s | 2.4 MB 00:00 2024-09-16T00:16:24.150 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:16:24.150 INFO:teuthology.orchestra.run.smithi019.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-16T00:16:24.151 INFO:teuthology.orchestra.run.smithi153.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-16T00:16:24.193 INFO:teuthology.orchestra.run.smithi153.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 864 kB/s | 36 kB 00:00 2024-09-16T00:16:24.235 INFO:teuthology.orchestra.run.smithi153.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-16T00:16:24.285 INFO:teuthology.orchestra.run.smithi153.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-16T00:16:24.297 INFO:teuthology.orchestra.run.smithi001.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.9 MB/s | 5.1 MB 00:00 2024-09-16T00:16:24.327 INFO:teuthology.orchestra.run.smithi153.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-16T00:16:24.378 INFO:teuthology.orchestra.run.smithi153.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.9 MB/s | 96 kB 00:00 2024-09-16T00:16:24.411 INFO:teuthology.orchestra.run.smithi153.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-16T00:16:24.445 INFO:teuthology.orchestra.run.smithi153.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-16T00:16:24.472 INFO:teuthology.orchestra.run.smithi001.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 171 kB/s | 77 kB 00:00 2024-09-16T00:16:24.506 INFO:teuthology.orchestra.run.smithi001.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 112 kB/s | 42 kB 00:00 2024-09-16T00:16:24.562 INFO:teuthology.orchestra.run.smithi153.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-16T00:16:24.762 INFO:teuthology.orchestra.run.smithi153.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-09-16T00:16:24.806 INFO:teuthology.orchestra.run.smithi001.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 529 kB/s | 269 kB 00:00 2024-09-16T00:16:24.812 INFO:teuthology.orchestra.run.smithi153.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-16T00:16:24.854 INFO:teuthology.orchestra.run.smithi153.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-16T00:16:24.880 INFO:teuthology.orchestra.run.smithi153.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-16T00:16:24.907 INFO:teuthology.orchestra.run.smithi001.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 366 kB/s | 159 kB 00:00 2024-09-16T00:16:25.007 INFO:teuthology.orchestra.run.smithi001.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 557 kB/s | 279 kB 00:00 2024-09-16T00:16:25.038 INFO:teuthology.orchestra.run.smithi153.stdout:(107/156): python3-babel-2.9.1-2.el9.noarch.rpm 2.5 MB/s | 6.0 MB 00:02 2024-09-16T00:16:25.114 INFO:teuthology.orchestra.run.smithi153.stdout:(108/156): python3-numpy-1.20.1-5.el9.x86_64.rp 2.5 MB/s | 5.1 MB 00:01 2024-09-16T00:16:25.181 INFO:teuthology.orchestra.run.smithi153.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-09-16T00:16:25.190 INFO:teuthology.orchestra.run.smithi001.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 133 kB/s | 51 kB 00:00 2024-09-16T00:16:25.206 INFO:teuthology.orchestra.run.smithi153.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-09-16T00:16:25.231 INFO:teuthology.orchestra.run.smithi153.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 444 kB/s | 49 kB 00:00 2024-09-16T00:16:25.265 INFO:teuthology.orchestra.run.smithi153.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-16T00:16:25.266 INFO:teuthology.orchestra.run.smithi001.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 149 kB/s | 54 kB 00:00 2024-09-16T00:16:25.332 INFO:teuthology.orchestra.run.smithi153.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-09-16T00:16:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:16:25.383 INFO:teuthology.orchestra.run.smithi153.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T00:16:25.392 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-16T00:16:25.408 INFO:teuthology.orchestra.run.smithi153.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-16T00:16:25.433 INFO:teuthology.orchestra.run.smithi153.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-09-16T00:16:25.458 INFO:teuthology.orchestra.run.smithi001.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 156 kB/s | 42 kB 00:00 2024-09-16T00:16:25.467 INFO:teuthology.orchestra.run.smithi153.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-09-16T00:16:25.492 INFO:teuthology.orchestra.run.smithi153.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-16T00:16:25.525 INFO:teuthology.orchestra.run.smithi153.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-16T00:16:25.551 INFO:teuthology.orchestra.run.smithi153.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-16T00:16:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-16T00:16:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-16T00:16:25.576 INFO:teuthology.orchestra.run.smithi153.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-16T00:16:25.601 INFO:teuthology.orchestra.run.smithi153.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 434 kB/s | 11 kB 00:00 2024-09-16T00:16:25.608 INFO:teuthology.orchestra.run.smithi001.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 129 kB/s | 44 kB 00:00 2024-09-16T00:16:25.627 INFO:teuthology.orchestra.run.smithi153.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-16T00:16:25.652 INFO:teuthology.orchestra.run.smithi153.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-16T00:16:25.677 INFO:teuthology.orchestra.run.smithi153.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-16T00:16:25.702 INFO:teuthology.orchestra.run.smithi153.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-16T00:16:25.728 INFO:teuthology.orchestra.run.smithi153.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-16T00:16:25.753 INFO:teuthology.orchestra.run.smithi153.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:16:25.778 INFO:teuthology.orchestra.run.smithi153.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T00:16:25.804 INFO:teuthology.orchestra.run.smithi153.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-16T00:16:25.837 INFO:teuthology.orchestra.run.smithi153.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-16T00:16:25.871 INFO:teuthology.orchestra.run.smithi153.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-16T00:16:25.875 INFO:teuthology.orchestra.run.smithi001.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 134 kB/s | 36 kB 00:00 2024-09-16T00:16:25.896 INFO:teuthology.orchestra.run.smithi153.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-16T00:16:25.921 INFO:teuthology.orchestra.run.smithi153.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-16T00:16:25.938 INFO:teuthology.orchestra.run.smithi153.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 676 kB/s | 28 kB 00:00 2024-09-16T00:16:25.955 INFO:teuthology.orchestra.run.smithi153.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 923 kB/s | 31 kB 00:00 2024-09-16T00:16:25.972 INFO:teuthology.orchestra.run.smithi153.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-16T00:16:26.050 INFO:teuthology.orchestra.run.smithi001.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 373 kB/s | 221 kB 00:00 2024-09-16T00:16:26.056 INFO:teuthology.orchestra.run.smithi153.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-16T00:16:26.073 INFO:teuthology.orchestra.run.smithi153.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-09-16T00:16:26.098 INFO:teuthology.orchestra.run.smithi153.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-16T00:16:26.132 INFO:teuthology.orchestra.run.smithi153.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-16T00:16:26.158 INFO:teuthology.orchestra.run.smithi153.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-16T00:16:26.208 INFO:teuthology.orchestra.run.smithi153.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-09-16T00:16:26.209 INFO:teuthology.orchestra.run.smithi001.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 142 kB/s | 47 kB 00:00 2024-09-16T00:16:26.232 INFO:teuthology.orchestra.run.smithi153.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-09-16T00:16:26.258 INFO:teuthology.orchestra.run.smithi153.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T00:16:26.283 INFO:teuthology.orchestra.run.smithi153.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-16T00:16:26.333 INFO:teuthology.orchestra.run.smithi153.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T00:16:26.551 INFO:teuthology.orchestra.run.smithi001.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 608 kB/s | 304 kB 00:00 2024-09-16T00:16:26.576 INFO:teuthology.orchestra.run.smithi001.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 158 kB/s | 58 kB 00:00 2024-09-16T00:16:26.625 INFO:teuthology.orchestra.run.smithi153.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 9.3 MB/s | 3.4 MB 00:00 2024-09-16T00:16:26.692 INFO:teuthology.orchestra.run.smithi153.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 7.7 MB/s | 3.2 MB 00:00 2024-09-16T00:16:26.809 INFO:teuthology.orchestra.run.smithi153.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 349 kB/s | 41 kB 00:00 2024-09-16T00:16:26.885 INFO:teuthology.orchestra.run.smithi153.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 4.2 MB/s | 2.3 MB 00:00 2024-09-16T00:16:26.943 INFO:teuthology.orchestra.run.smithi001.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 174 kB/s | 64 kB 00:00 2024-09-16T00:16:26.951 INFO:teuthology.orchestra.run.smithi153.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 1.7 MB/s | 249 kB 00:00 2024-09-16T00:16:27.043 INFO:teuthology.orchestra.run.smithi153.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.3 MB/s | 1.8 MB 00:00 2024-09-16T00:16:27.110 INFO:teuthology.orchestra.run.smithi001.stdout:(101/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.7 MB/s | 16 MB 00:02 2024-09-16T00:16:27.135 INFO:teuthology.orchestra.run.smithi001.stdout:(102/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 164 kB/s | 96 kB 00:00 2024-09-16T00:16:27.210 INFO:teuthology.orchestra.run.smithi001.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-16T00:16:27.286 INFO:teuthology.orchestra.run.smithi001.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-16T00:16:27.327 INFO:teuthology.orchestra.run.smithi001.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-16T00:16:27.353 INFO:teuthology.orchestra.run.smithi001.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 776 kB/s | 19 kB 00:00 2024-09-16T00:16:27.386 INFO:teuthology.orchestra.run.smithi001.stdout:(107/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 101 kB/s | 45 kB 00:00 2024-09-16T00:16:27.437 INFO:teuthology.orchestra.run.smithi001.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-16T00:16:27.450 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-16T00:16:27.451 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-16T00:16:27.462 INFO:teuthology.orchestra.run.smithi001.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T00:16:27.487 INFO:teuthology.orchestra.run.smithi001.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T00:16:27.521 INFO:teuthology.orchestra.run.smithi001.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-16T00:16:27.546 INFO:teuthology.orchestra.run.smithi001.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-16T00:16:27.552 INFO:teuthology.orchestra.run.smithi153.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.0 MB/s | 2.4 MB 00:00 2024-09-16T00:16:27.597 INFO:teuthology.orchestra.run.smithi001.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-16T00:16:27.622 INFO:teuthology.orchestra.run.smithi001.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:16:27.635 INFO:teuthology.orchestra.run.smithi153.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 1.8 MB/s | 1.4 MB 00:00 2024-09-16T00:16:27.642 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:16:27.642 INFO:teuthology.orchestra.run.smithi153.stdout:Total 15 MB/s | 226 MB 00:14 2024-09-16T00:16:27.647 INFO:teuthology.orchestra.run.smithi001.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T00:16:27.672 INFO:teuthology.orchestra.run.smithi001.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-16T00:16:27.714 INFO:teuthology.orchestra.run.smithi001.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.2 MB/s | 839 kB 00:00 2024-09-16T00:16:27.740 INFO:teuthology.orchestra.run.smithi001.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-16T00:16:27.773 INFO:teuthology.orchestra.run.smithi001.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-16T00:16:27.807 INFO:teuthology.orchestra.run.smithi001.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-16T00:16:27.874 INFO:teuthology.orchestra.run.smithi001.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-09-16T00:16:27.899 INFO:teuthology.orchestra.run.smithi001.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.7 MB/s | 220 kB 00:00 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi001.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi001.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-16T00:16:27.975 INFO:teuthology.orchestra.run.smithi001.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-16T00:16:28.000 INFO:teuthology.orchestra.run.smithi001.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-16T00:16:28.026 INFO:teuthology.orchestra.run.smithi001.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-16T00:16:28.060 INFO:teuthology.orchestra.run.smithi001.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 334 kB/s | 20 kB 00:00 2024-09-16T00:16:28.085 INFO:teuthology.orchestra.run.smithi001.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:16:28.113 INFO:teuthology.orchestra.run.smithi001.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T00:16:28.135 INFO:teuthology.orchestra.run.smithi001.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-16T00:16:28.178 INFO:teuthology.orchestra.run.smithi001.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-16T00:16:28.211 INFO:teuthology.orchestra.run.smithi001.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-16T00:16:28.236 INFO:teuthology.orchestra.run.smithi001.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 282 kB/s | 16 kB 00:00 2024-09-16T00:16:28.262 INFO:teuthology.orchestra.run.smithi001.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-16T00:16:28.287 INFO:teuthology.orchestra.run.smithi001.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-16T00:16:28.321 INFO:teuthology.orchestra.run.smithi001.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-09-16T00:16:28.346 INFO:teuthology.orchestra.run.smithi001.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:16:28.371 INFO:teuthology.orchestra.run.smithi001.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-16T00:16:28.396 INFO:teuthology.orchestra.run.smithi001.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-16T00:16:28.422 INFO:teuthology.orchestra.run.smithi001.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-16T00:16:28.447 INFO:teuthology.orchestra.run.smithi001.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T00:16:28.481 INFO:teuthology.orchestra.run.smithi001.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-16T00:16:28.531 INFO:teuthology.orchestra.run.smithi001.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.8 MB/s | 2.0 MB 00:00 2024-09-16T00:16:28.556 INFO:teuthology.orchestra.run.smithi001.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-09-16T00:16:28.590 INFO:teuthology.orchestra.run.smithi001.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-16T00:16:28.624 INFO:teuthology.orchestra.run.smithi001.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-09-16T00:16:28.674 INFO:teuthology.orchestra.run.smithi001.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-16T00:16:28.894 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-09-16T00:16:28.925 INFO:teuthology.orchestra.run.smithi001.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-16T00:16:29.048 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-09-16T00:16:29.048 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-09-16T00:16:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 5.2 MB/s | 2.3 MB 00:00 2024-09-16T00:16:29.200 INFO:teuthology.orchestra.run.smithi001.stdout:(151/156): librados2-19.1.1-326.g8921c8ef.el9.x 5.6 MB/s | 3.4 MB 00:00 2024-09-16T00:16:29.225 INFO:teuthology.orchestra.run.smithi001.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 375 kB/s | 41 kB 00:00 2024-09-16T00:16:29.409 INFO:teuthology.orchestra.run.smithi001.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 3.7 MB/s | 1.8 MB 00:00 2024-09-16T00:16:29.443 INFO:teuthology.orchestra.run.smithi001.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 1.0 MB/s | 249 kB 00:00 2024-09-16T00:16:29.485 INFO:teuthology.orchestra.run.smithi001.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.2 MB/s | 1.4 MB 00:00 2024-09-16T00:16:29.660 INFO:teuthology.orchestra.run.smithi001.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 2.4 MB 00:00 2024-09-16T00:16:29.666 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:16:29.666 INFO:teuthology.orchestra.run.smithi001.stdout:Total 12 MB/s | 226 MB 00:18 2024-09-16T00:16:30.863 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-09-16T00:16:30.863 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-09-16T00:16:30.928 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-09-16T00:16:31.096 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-09-16T00:16:31.096 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-09-16T00:16:33.434 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-09-16T00:16:33.435 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-09-16T00:16:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-16T00:16:34.214 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:16:34.253 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:16:34.315 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:16:34.349 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:16:34.374 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:16:34.417 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:16:34.764 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:16:34.883 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:16:35.008 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:16:35.034 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:16:35.126 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:16:35.155 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:16:35.303 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:16:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:16:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:16:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:16:35.522 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:16:35.568 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:16:35.590 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:16:35.614 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:16:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:16:35.936 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:16:36.083 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:16:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:16:36.268 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:16:36.493 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:16:36.602 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:16:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:16:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:16:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:16:36.849 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:16:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:16:36.985 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:16:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:16:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:16:37.115 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:16:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:16:37.197 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:16:37.233 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:16:37.311 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:16:37.343 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:16:37.387 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:16:37.415 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:16:37.439 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-09-16T00:16:37.488 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:16:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:16:37.548 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:16:37.594 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:16:37.606 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:16:37.647 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:16:37.681 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:16:37.707 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:16:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:16:37.749 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:16:37.792 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:16:38.074 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:16:38.174 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:16:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:16:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:16:38.350 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:16:38.399 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:16:38.429 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:16:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:16:38.465 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:16:38.500 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:16:38.537 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:16:38.578 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:16:38.582 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:16:38.634 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:16:38.662 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:16:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:16:38.749 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:16:38.772 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:16:38.801 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:16:38.835 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:16:38.857 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:16:38.865 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:16:38.891 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:16:38.931 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:16:38.969 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:16:39.013 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:16:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:16:39.082 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:16:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:16:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:16:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:16:39.212 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:16:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:16:39.337 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:16:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:16:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:16:39.442 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:16:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:16:39.514 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:16:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:16:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:16:39.748 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:16:39.859 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:16:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:16:39.960 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:16:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:16:39.998 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:16:40.033 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:16:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:16:40.106 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:16:40.159 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:16:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:16:40.207 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:16:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:16:40.260 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:16:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:16:40.291 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:16:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:16:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-09-16T00:16:40.319 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:16:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:16:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:16:40.389 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:16:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:16:40.414 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:16:40.442 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:16:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:16:40.458 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:16:40.472 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:16:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:16:40.508 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:16:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:16:40.532 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:16:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:16:40.573 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:16:40.595 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:16:40.598 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:16:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:16:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:16:40.651 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:16:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:16:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:16:40.715 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:16:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:16:40.785 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:16:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:16:40.830 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:16:40.875 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:16:40.905 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:16:41.007 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:16:41.038 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:16:41.091 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:16:41.124 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:16:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:16:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:16:41.359 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:16:41.396 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:16:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:16:41.625 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:16:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:16:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:16:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:16:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:16:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:16:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:16:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:16:41.783 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:16:41.784 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:16:41.813 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:16:41.816 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:16:41.837 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:16:41.870 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:16:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:16:41.888 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:16:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:16:41.914 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:16:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:16:42.004 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:16:42.050 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:16:42.072 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:16:42.118 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:16:42.132 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:16:42.188 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:16:42.241 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:16:42.259 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:16:42.272 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:16:42.291 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:16:42.317 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:16:42.327 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:16:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:16:42.374 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:16:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:16:42.464 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:16:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:16:42.503 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:16:42.584 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:16:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:16:42.664 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:16:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:16:42.721 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:16:42.826 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:16:42.857 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:16:42.894 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:16:42.992 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:16:43.010 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:16:43.026 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:16:43.111 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:16:43.148 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:16:43.153 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:16:43.192 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:16:43.196 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:16:43.223 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:16:43.273 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:16:43.305 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:16:43.323 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:16:43.363 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:16:43.406 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:16:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:16:43.442 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:16:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:16:43.479 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:16:43.507 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:16:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:16:43.541 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:16:43.581 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:16:43.590 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:16:43.628 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:16:43.629 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:16:43.659 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:16:43.672 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:16:43.694 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:16:43.726 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:16:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:16:43.768 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:16:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:16:43.802 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:16:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:16:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:16:43.889 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:16:43.933 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:16:43.936 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:16:43.964 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:16:43.978 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:16:44.055 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:16:44.076 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:16:44.133 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:16:44.200 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:16:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:16:44.433 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:16:44.518 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:16:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:16:44.578 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:16:44.811 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:16:44.863 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:16:44.895 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:16:44.904 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:16:44.905 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:16:44.937 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:16:44.938 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:16:44.941 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:16:44.981 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:16:45.019 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:16:45.033 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:16:45.037 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:16:45.061 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:16:45.063 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:16:45.099 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:16:45.109 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:16:45.187 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:16:45.235 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:16:45.276 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:16:45.289 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:16:45.315 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:16:45.345 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:16:45.419 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:16:45.465 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:16:45.501 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:16:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:16:45.529 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:16:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:16:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:16:45.606 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:16:45.609 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:16:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:16:45.636 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:16:45.681 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:16:45.707 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:16:45.754 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:16:45.908 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:16:45.949 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:16:45.991 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:16:46.099 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:16:46.132 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:16:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:16:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:16:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:16:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:16:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:16:46.370 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:16:46.479 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:16:46.521 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:16:46.550 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:16:46.589 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:16:46.640 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:16:46.688 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:16:46.728 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:16:46.753 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:16:46.786 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:16:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:16:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:16:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:16:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:16:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:16:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:16:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:16:47.595 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:16:47.683 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:16:47.711 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:16:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:16:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:16:48.055 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:16:48.080 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:16:48.098 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:16:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:16:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:16:48.173 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:16:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:16:48.203 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:16:48.236 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:16:48.239 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:16:48.281 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:16:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:16:48.408 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:16:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:16:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:16:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:16:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:16:48.726 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:16:48.760 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:16:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:16:48.831 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:16:48.855 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:16:48.855 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:16:48.855 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:16:48.855 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:16:48.870 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:16:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:16:49.412 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:16:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:16:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:16:50.951 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:16:50.979 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:16:51.012 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:16:51.179 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:16:51.216 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:16:51.338 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:16:51.383 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:16:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:16:51.496 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:16:51.533 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:16:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:16:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:16:51.931 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:16:51.931 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:16:51.931 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:16:51.931 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:16:51.962 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:16:52.057 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:16:52.094 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:16:52.174 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:16:52.204 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:16:54.580 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:16:54.617 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:16:55.201 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:16:55.239 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:16:55.319 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:16:55.320 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:16:55.320 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:16:55.320 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:16:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:17:00.821 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:17:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-09-16T00:17:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-09-16T00:17:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-09-16T00:17:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-09-16T00:17:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-09-16T00:17:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-09-16T00:17:00.822 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-09-16T00:17:00.822 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:01.000 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:17:01.017 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:17:01.017 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:01.017 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:17:01.017 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:17:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:17:02.580 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:02.580 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:17:02.580 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:17:02.581 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:17:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:02.813 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:17:02.836 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:17:02.836 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:02.836 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:17:02.836 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:17:02.836 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:17:02.836 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:03.296 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:17:03.320 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:17:03.320 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:03.320 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:17:03.320 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:17:03.320 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:17:03.320 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:03.427 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:17:03.485 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:17:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:17:03.742 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:17:03.790 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:17:03.845 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:17:03.845 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2024-09-16T00:17:03.845 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2024-09-16T00:17:03.845 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2024-09-16T00:17:03.846 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2024-09-16T00:17:03.846 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2024-09-16T00:17:03.846 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /root 2024-09-16T00:17:03.846 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2024-09-16T00:17:03.846 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:03.870 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:17:03.967 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:17:04.022 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:17:04.040 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:17:04.040 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:04.040 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:17:04.040 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:04.109 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:17:04.156 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:17:04.188 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:17:04.215 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:17:04.680 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:17:04.714 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:17:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:17:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:17:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:17:05.525 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:17:05.553 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:17:05.553 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:05.553 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:17:05.553 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:17:05.554 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:17:05.554 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:05.784 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:17:05.808 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:17:05.808 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:05.808 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:17:05.808 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:17:05.808 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:17:05.808 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:17:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:17:06.251 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:17:06.276 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:17:06.276 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:06.276 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:17:06.276 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:17:06.276 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:17:06.276 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:06.382 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:17:06.441 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:17:06.494 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:17:06.684 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:17:06.744 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:17:06.823 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:17:06.911 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:17:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:17:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:17:07.051 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:17:07.103 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:17:07.129 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:17:07.134 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:17:07.162 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:17:07.300 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:17:07.324 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:17:07.348 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:17:07.348 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:07.348 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:17:07.348 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:17:07.348 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:17:07.349 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:07.400 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:17:07.412 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:17:07.513 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:17:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:17:07.574 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:17:07.602 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:17:07.633 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:17:07.705 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:17:07.725 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:17:07.725 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2024-09-16T00:17:07.725 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2024-09-16T00:17:07.725 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2024-09-16T00:17:07.725 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2024-09-16T00:17:07.725 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2024-09-16T00:17:07.725 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /root 2024-09-16T00:17:07.725 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2024-09-16T00:17:07.725 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:07.733 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:17:07.910 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:17:07.926 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:17:07.926 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:07.926 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:17:07.926 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:08.541 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:17:08.569 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:17:08.593 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:17:08.593 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:08.593 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:17:08.593 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:17:08.593 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:17:08.593 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:08.637 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:17:08.660 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:17:08.660 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:08.660 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:17:08.661 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:08.712 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:17:08.735 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:17:08.736 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:08.736 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:17:08.736 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:17:08.736 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:17:08.736 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:08.784 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:17:08.827 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:17:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:17:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:17:09.077 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:09.077 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:17:09.078 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:17:09.078 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:17:09.078 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:09.446 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:17:09.474 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:17:09.475 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:09.475 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:17:09.475 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:17:09.475 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:17:09.475 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:09.714 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:17:09.739 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:17:09.739 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:09.739 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:17:09.739 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:17:09.739 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:17:09.739 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:09.793 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:17:09.819 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:17:09.923 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:17:10.082 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:17:10.121 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:17:10.145 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:17:10.145 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:10.146 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:17:10.146 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:17:10.146 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:17:10.146 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:10.197 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:17:10.206 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:17:10.209 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:17:10.230 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:17:10.231 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:10.231 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:17:10.231 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:17:10.231 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:17:10.231 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:10.307 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:17:10.353 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:17:10.362 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:17:10.419 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:17:10.483 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:17:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:17:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:17:10.812 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:17:10.904 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:17:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:17:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:17:11.147 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:17:11.174 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:17:11.289 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:17:11.324 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:17:11.348 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:17:11.348 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:11.348 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:17:11.348 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:17:11.348 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:17:11.348 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:11.391 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:17:11.415 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:17:11.415 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:11.415 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:17:11.415 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:11.467 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:17:11.491 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:17:11.491 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:11.491 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:17:11.491 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:17:11.491 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:17:11.492 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:11.630 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:17:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:17:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:17:11.754 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:17:11.779 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:17:11.793 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:17:11.816 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:17:11.817 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:11.817 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:17:11.817 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:17:11.817 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:17:11.817 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:12.880 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:17:12.915 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:17:13.875 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:17:13.925 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:17:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:17:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:17:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:17:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:17:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:17:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:17:14.238 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:17:14.267 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:17:14.292 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:17:14.292 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:14.292 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:17:14.293 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:17:14.293 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:17:14.293 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:17:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:17:14.343 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:17:14.354 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:17:14.369 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:17:14.456 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:17:14.514 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:17:14.977 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:17:15.493 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:17:15.521 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:17:15.546 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:17:15.546 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:15.547 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:17:15.547 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:17:15.547 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:17:15.547 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:15.588 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:17:15.613 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:17:15.613 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:15.613 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:17:15.613 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:17:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:17:15.688 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:15.688 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:17:15.688 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:17:15.688 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:17:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:17:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:17:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:17:16.039 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:17:16.039 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:17:16.039 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:17:16.039 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:17:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:17:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:17:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:17:16.418 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:17:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:17:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:17:16.437 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:17:16.481 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:17:16.511 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:17:16.538 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:17:16.604 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:17:16.604 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:17:16.696 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:17:16.719 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:17:16.767 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:17:16.779 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:17:16.798 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:17:16.827 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:17:16.874 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:17:17.472 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:17:17.521 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:17:17.541 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:17:18.492 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:17:18.533 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:17:18.615 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:17:18.645 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:17:18.670 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:17:18.705 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:17:18.705 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:17:18.780 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:17:18.781 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:17:18.865 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:17:18.889 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:17:18.895 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:17:18.896 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:17:19.075 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:17:19.075 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:17:19.167 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:17:19.240 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:17:20.876 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:17:20.918 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:17:20.958 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:17:20.992 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:17:21.018 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:17:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:17:21.297 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:17:21.328 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:17:21.358 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:17:21.395 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:17:21.954 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:17:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:17:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:17:22.255 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:17:22.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:17:22.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:17:22.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:17:22.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:17:22.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:17:22.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:17:22.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:17:22.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:17:22.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:17:22.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:17:22.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:17:22.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:17:22.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:17:22.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:17:22.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:17:22.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:17:22.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:17:22.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:17:22.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:17:22.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:17:22.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:17:22.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:17:22.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:17:22.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:17:22.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:17:22.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:17:22.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:17:22.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:17:22.282 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:17:22.282 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:17:22.282 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:17:22.282 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:17:22.282 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:17:22.282 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:17:22.282 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:17:22.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:17:22.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:17:22.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:17:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:17:22.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:17:22.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:17:22.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:17:22.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:17:22.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:17:22.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:17:22.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:17:22.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:17:22.288 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:17:22.289 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:17:22.290 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:17:22.291 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:17:22.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:17:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:17:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:17:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:17:23.166 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:17:23.201 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:17:23.226 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:17:23.269 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:17:23.269 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:17:23.353 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:17:23.353 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:17:23.428 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:17:23.453 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:17:23.459 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:17:23.459 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:17:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:17:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:17:23.715 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:17:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:17:24.926 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:17:24.926 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:24.926 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-09-16T00:17:24.927 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:24.927 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:24.927 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:24.927 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:24.927 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.928 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.928 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:17:24.928 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:17:24.928 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-16T00:17:24.928 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:17:24.928 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:17:24.929 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:17:24.929 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:17:24.929 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.929 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.929 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.929 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.929 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.929 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.930 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.930 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.930 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.930 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.930 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.930 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.931 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.931 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.931 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.931 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.931 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.931 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.932 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.932 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.932 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.932 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.932 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.933 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:24.933 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:17:24.933 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:17:24.933 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:17:24.933 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:17:24.933 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:17:24.933 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:17:24.933 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:17:24.934 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:17:24.934 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:17:24.934 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:17:24.934 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:17:24.935 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.935 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:17:24.937 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:17:24.937 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:17:24.937 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:17:24.937 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:17:24.937 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:17:24.937 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:17:24.937 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:17:24.937 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:17:24.938 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:17:24.938 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:17:24.938 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:17:24.938 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:17:24.938 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:17:24.938 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:17:24.938 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:17:24.938 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:17:24.939 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:17:24.939 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:17:24.939 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:17:24.939 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:17:24.939 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:17:24.939 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:17:24.939 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.939 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.939 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:17:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:17:24.941 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:17:24.941 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:17:24.941 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:17:24.941 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:17:24.941 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:17:24.941 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:17:24.941 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:17:24.941 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:17:24.941 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:17:24.942 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.943 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:17:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:17:24.945 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:17:24.946 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:17:24.946 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:17:24.946 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:17:24.946 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:17:24.946 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-16T00:17:24.957 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:17:24.957 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:17:24.957 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:17:24.957 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:17:24.957 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:17:24.957 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:17:24.957 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:17:24.957 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:17:24.958 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:17:24.958 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:17:24.958 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:17:24.958 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:17:24.958 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:17:24.958 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:17:24.958 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:17:24.959 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:17:24.959 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:17:24.959 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:17:24.959 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:17:24.959 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:17:24.959 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:17:24.959 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:17:24.960 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:17:24.960 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:17:24.960 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:17:24.960 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:17:24.960 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:17:24.960 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:17:24.960 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:17:24.961 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:17:24.961 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:17:24.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:17:24.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:17:24.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:17:24.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:17:24.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:17:24.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:17:24.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:17:24.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:17:24.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:17:24.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:17:24.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:17:24.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:17:24.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:17:24.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:17:24.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:17:24.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:17:24.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:17:24.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:17:24.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:17:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:17:24.989 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:17:24.990 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:17:24.990 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:17:24.990 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:17:24.990 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:17:24.990 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:17:24.990 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:17:24.990 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:17:24.990 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:17:24.990 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:17:25.342 DEBUG:teuthology.parallel:result is None 2024-09-16T00:17:27.802 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:17:27.802 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:27.803 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-09-16T00:17:27.803 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:27.803 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:27.803 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:27.803 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:27.804 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.804 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.804 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:17:27.804 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:17:27.804 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-09-16T00:17:27.804 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:17:27.804 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:17:27.805 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:17:27.805 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:17:27.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.807 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.807 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.807 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.807 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.807 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.808 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.808 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.808 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.808 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.808 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.808 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:27.808 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:17:27.809 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:17:27.809 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:17:27.809 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:17:27.809 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:17:27.809 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:17:27.809 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:17:27.809 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:17:27.809 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:17:27.810 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:17:27.810 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:17:27.810 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.810 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.810 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.810 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:17:27.810 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:17:27.810 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:17:27.811 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:17:27.811 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:17:27.811 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.811 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.811 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:17:27.811 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.812 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:17:27.812 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:17:27.812 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:17:27.812 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:17:27.812 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:17:27.812 INFO:teuthology.orchestra.run.smithi153.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:17:27.813 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:17:27.813 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:17:27.813 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:17:27.813 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:17:27.813 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:17:27.813 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:17:27.813 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:17:27.814 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:17:27.814 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:17:27.814 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:17:27.814 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:17:27.814 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:17:27.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:17:27.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:17:27.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:17:27.816 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:17:27.816 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:17:27.816 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:17:27.816 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:17:27.816 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:17:27.816 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:17:27.816 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:17:27.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:17:27.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:17:27.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.820 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.821 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:27.821 INFO:teuthology.orchestra.run.smithi153.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:17:27.821 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:17:27.821 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:17:27.821 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:17:27.821 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:17:27.821 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:17:27.822 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:17:27.822 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:17:27.822 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-09-16T00:17:28.254 DEBUG:teuthology.parallel:result is None 2024-09-16T00:17:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:17:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:17:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:17:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:17:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:17:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:17:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:17:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:17:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:17:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:17:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:17:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:17:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:17:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:17:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:17:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:17:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:17:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:17:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:17:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:17:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:17:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:17:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:17:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:17:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:17:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:17:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:17:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:17:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:17:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:17:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:17:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:17:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:17:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:17:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:17:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:17:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:32.198 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2024-09-16T00:17:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:17:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:17:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:17:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:17:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:17:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:17:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:17:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:17:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:17:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:17:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:17:32.211 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-09-16T00:17:32.630 DEBUG:teuthology.parallel:result is None 2024-09-16T00:17:32.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:17:32.780 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:17:32.804 INFO:teuthology.orchestra.run.smithi001.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:17:32.804 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:17:32.805 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:17:32.806 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:17:32.938 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:17:32.965 INFO:teuthology.orchestra.run.smithi019.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:17:32.965 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:17:32.966 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:17:32.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:17:33.102 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:17:33.129 INFO:teuthology.orchestra.run.smithi153.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:17:33.129 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:17:33.129 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:17:33.131 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T00:17:33.131 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:17:33.131 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:17:33.162 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:17:33.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:17:33.194 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:17:33.195 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:17:33.226 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T00:17:33.227 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:17:33.227 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:17:33.255 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:17:33.323 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:17:33.323 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:17:33.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:17:33.422 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:17:33.422 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:17:33.452 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:17:33.521 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T00:17:33.521 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:17:33.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:17:33.551 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:17:33.621 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:17:33.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:17:33.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:17:33.719 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:17:33.719 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:17:33.749 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:17:33.817 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T00:17:33.817 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:17:33.818 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:17:33.848 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:17:33.919 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:17:33.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:17:33.949 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:17:34.017 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:17:34.017 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:17:34.046 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:17:34.117 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T00:17:34.219 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2}} 2024-09-16T00:17:34.219 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:17:34.219 INFO:tasks.cephadm:Cluster fsid is 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:17:34.220 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T00:17:34.220 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.1', 'mon.b': '172.21.15.19', 'mon.c': '172.21.15.153'} 2024-09-16T00:17:34.220 INFO:tasks.cephadm:First mon is mon.a on smithi001 2024-09-16T00:17:34.220 INFO:tasks.cephadm:First mgr is x 2024-09-16T00:17:34.220 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T00:17:34.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname $(hostname -s) 2024-09-16T00:17:34.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-09-16T00:17:34.281 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname $(hostname -s) 2024-09-16T00:17:34.309 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T00:17:34.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:17:34.447 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T00:17:34.587 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:17:34.588 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T00:17:34.588 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T00:17:34.589 DEBUG:teuthology.orchestra.run.smithi001:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:17:34.956 INFO:teuthology.orchestra.run.smithi001.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:17 /home/ubuntu/cephtest/cephadm 2024-09-16T00:17:34.957 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:17:35.355 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:17 /home/ubuntu/cephtest/cephadm 2024-09-16T00:17:35.356 DEBUG:teuthology.orchestra.run.smithi153:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:17:35.751 INFO:teuthology.orchestra.run.smithi153.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:17 /home/ubuntu/cephtest/cephadm 2024-09-16T00:17:35.752 DEBUG:teuthology.orchestra.run.smithi001:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:17:35.772 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:17:35.792 DEBUG:teuthology.orchestra.run.smithi153:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:17:35.827 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-16T00:17:35.827 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:17:35.829 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:17:35.834 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:17:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:17:37.389 INFO:teuthology.orchestra.run.smithi153.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:17:37.508 INFO:teuthology.orchestra.run.smithi001.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:18:17.522 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:18:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:18:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:18:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "repo_digests": [ 2024-09-16T00:18:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:18:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:18:17.523 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:18:19.304 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-16T00:18:19.304 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:18:19.304 INFO:teuthology.orchestra.run.smithi153.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:18:19.305 INFO:teuthology.orchestra.run.smithi153.stdout: "repo_digests": [ 2024-09-16T00:18:19.305 INFO:teuthology.orchestra.run.smithi153.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:18:19.305 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-09-16T00:18:19.305 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-16T00:18:19.447 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-16T00:18:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:18:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:18:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-09-16T00:18:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:18:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-16T00:18:19.447 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-16T00:18:19.473 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph 2024-09-16T00:18:19.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-09-16T00:18:19.535 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph 2024-09-16T00:18:19.564 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /etc/ceph 2024-09-16T00:18:19.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-09-16T00:18:19.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /etc/ceph 2024-09-16T00:18:19.645 INFO:tasks.cephadm:Writing seed config... 2024-09-16T00:18:19.646 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-16T00:18:19.646 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-16T00:18:19.646 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T00:18:19.646 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-16T00:18:19.647 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-16T00:18:19.648 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-16T00:18:19.649 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T00:18:19.650 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-16T00:18:19.650 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:18:19.650 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T00:18:19.669 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1226ed86-73c1-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-16T00:18:19.670 DEBUG:teuthology.orchestra.run.smithi001:mon.a> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T00:18:19.712 DEBUG:teuthology.orchestra.run.smithi001:mgr.x> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T00:18:19.755 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T00:18:19.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.1 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:18:19.972 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:18:19.972 INFO:teuthology.orchestra.run.smithi001.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', '1226ed86-73c1-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.1', '--skip-admin-label'] 2024-09-16T00:18:19.972 INFO:teuthology.orchestra.run.smithi001.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T00:18:19.973 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying podman|docker is present... 2024-09-16T00:18:19.997 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:18:19.997 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying lvm2 is present... 2024-09-16T00:18:19.998 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying time synchronization is in place... 2024-09-16T00:18:20.005 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:18:20.006 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:18:20.012 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:18:20.012 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-16T00:18:20.019 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-09-16T00:18:20.025 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-09-16T00:18:20.025 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:18:20.025 INFO:teuthology.orchestra.run.smithi001.stdout:Repeating the final host check... 2024-09-16T00:18:20.049 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:18:20.049 INFO:teuthology.orchestra.run.smithi001.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T00:18:20.049 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl is present 2024-09-16T00:18:20.049 INFO:teuthology.orchestra.run.smithi001.stdout:lvcreate is present 2024-09-16T00:18:20.056 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:18:20.056 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:18:20.062 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:18:20.062 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-16T00:18:20.069 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-09-16T00:18:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-09-16T00:18:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:18:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:Host looks OK 2024-09-16T00:18:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:Cluster fsid: 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:18:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:Acquiring lock 139775480992768 on /run/cephadm/1226ed86-73c1-11ef-bceb-c7b262605968.lock 2024-09-16T00:18:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:Lock 139775480992768 acquired on /run/cephadm/1226ed86-73c1-11ef-bceb-c7b262605968.lock 2024-09-16T00:18:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 3300 ... 2024-09-16T00:18:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 6789 ... 2024-09-16T00:18:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:Base mon IP(s) is [172.21.15.1:3300, 172.21.15.1:6789], mon addrv is [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-09-16T00:18:20.080 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.1 metric 100 2024-09-16T00:18:20.080 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.1 metric 100 2024-09-16T00:18:20.082 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T00:18:20.082 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-16T00:18:20.082 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-09-16T00:18:20.085 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T00:18:20.085 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T00:18:20.085 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:18:20.085 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-16T00:18:20.085 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-09-16T00:18:20.085 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:18:20.086 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-09-16T00:18:20.086 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-09-16T00:18:20.086 INFO:teuthology.orchestra.run.smithi001.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T00:18:20.086 INFO:teuthology.orchestra.run.smithi001.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T00:18:20.087 INFO:teuthology.orchestra.run.smithi001.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:18:20.550 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T00:18:20.550 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:18:20.550 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T00:18:20.550 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-16T00:18:20.550 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T00:18:20.550 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T00:18:20.550 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T00:18:21.496 INFO:teuthology.orchestra.run.smithi001.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T00:18:21.496 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T00:18:21.496 INFO:teuthology.orchestra.run.smithi001.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T00:18:22.318 INFO:teuthology.orchestra.run.smithi001.stdout:stat: stdout 167 167 2024-09-16T00:18:22.319 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial keys... 2024-09-16T00:18:23.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBOeedmna1NJRAAhOQTmqvI3qAlRIL79qPlSQ== 2024-09-16T00:18:24.063 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBPeedmF3I/GhAAdm9Tlmj3UblzVpM+nIgaOg== 2024-09-16T00:18:24.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBQeedmhDu6FhAAOnapk0kamQqfTguHFJ6Ypg== 2024-09-16T00:18:24.927 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial monmap... 2024-09-16T00:18:25.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:18:25.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-16T00:18:25.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:18:25.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:18:25.791 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool for a [v2:172.21.15.1:3300,v1:172.21.15.1:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:18:25.792 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = quincy 2024-09-16T00:18:25.792 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: set fsid to 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:18:25.792 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:18:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:18:25.792 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mon... 2024-09-16T00:18:26.763 INFO:teuthology.orchestra.run.smithi001.stdout:create mon.a on 2024-09-16T00:18:26.971 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T00:18:27.129 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T00:18:27.331 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1226ed86-73c1-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1226ed86-73c1-11ef-bceb-c7b262605968.target. 2024-09-16T00:18:27.331 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1226ed86-73c1-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1226ed86-73c1-11ef-bceb-c7b262605968.target. 2024-09-16T00:18:27.578 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a 2024-09-16T00:18:27.578 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T00:18:27.750 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1226ed86-73c1-11ef-bceb-c7b262605968.target.wants/ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1226ed86-73c1-11ef-bceb-c7b262605968@.service. 2024-09-16T00:18:28.314 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:18:28.314 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-16T00:18:28.322 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:18:28.322 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-16T00:18:28.322 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-16T00:18:28.322 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:18:28.322 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon to start... 2024-09-16T00:18:28.322 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon... 2024-09-16T00:18:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:29 smithi001 ceph-mon[30567]: from='client.? 172.21.15.1:0/1004132179' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T00:18:31.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout id: 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout services: 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.562623s) 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout data: 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T00:18:31.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.739+0000 7f162b874640 1 Processor -- start 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.739+0000 7f162b874640 1 -- start start 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.739+0000 7f162b874640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f1624107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.739+0000 7f162b874640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1624107f60 con 0x7f1624107620 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.740+0000 7f1629610640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f1624107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.740+0000 7f1629610640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f1624107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56814/0 (socket says 172.21.15.1:56814) 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.740+0000 7f1629610640 1 -- 172.21.15.1:0/4188182715 learned_addr learned my addr 172.21.15.1:0/4188182715 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:31.376 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.740+0000 7f1629610640 1 -- 172.21.15.1:0/4188182715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16241080e0 con 0x7f1624107620 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.742+0000 7f1629610640 1 --2- 172.21.15.1:0/4188182715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f1624107a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f161c009820 tx=0x7f161c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cb928010d29c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.742+0000 7f1623fff640 1 -- 172.21.15.1:0/4188182715 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c037070 con 0x7f1624107620 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.742+0000 7f1623fff640 1 -- 172.21.15.1:0/4188182715 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c032440 con 0x7f1624107620 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.742+0000 7f1623fff640 1 -- 172.21.15.1:0/4188182715 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c032740 con 0x7f1624107620 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.743+0000 7f162b874640 1 -- 172.21.15.1:0/4188182715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 msgr2=0x7f1624107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.743+0000 7f162b874640 1 --2- 172.21.15.1:0/4188182715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f1624107a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f161c009820 tx=0x7f161c02a270 comp rx=0 tx=0).stop 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.743+0000 7f162b874640 1 -- 172.21.15.1:0/4188182715 shutdown_connections 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.743+0000 7f162b874640 1 --2- 172.21.15.1:0/4188182715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f1624107a20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.743+0000 7f162b874640 1 -- 172.21.15.1:0/4188182715 >> 172.21.15.1:0/4188182715 conn(0x7f1624102e50 msgr2=0x7f1624105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.743+0000 7f162b874640 1 -- 172.21.15.1:0/4188182715 shutdown_connections 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.743+0000 7f162b874640 1 -- 172.21.15.1:0/4188182715 wait complete. 2024-09-16T00:18:31.377 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.744+0000 7f162b874640 1 Processor -- start 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.744+0000 7f162b874640 1 -- start start 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.744+0000 7f162b874640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f162407db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.744+0000 7f162b874640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1624108b40 con 0x7f1624107620 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.744+0000 7f1629610640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f162407db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.744+0000 7f1629610640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f162407db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56830/0 (socket says 172.21.15.1:56830) 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.744+0000 7f1629610640 1 -- 172.21.15.1:0/1004132179 learned_addr learned my addr 172.21.15.1:0/1004132179 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.744+0000 7f1629610640 1 -- 172.21.15.1:0/1004132179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162407e050 con 0x7f1624107620 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.744+0000 7f1629610640 1 --2- 172.21.15.1:0/1004132179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f162407db10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f161c0090b0 tx=0x7f161c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.745+0000 7f16227fc640 1 -- 172.21.15.1:0/1004132179 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c041070 con 0x7f1624107620 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.745+0000 7f16227fc640 1 -- 172.21.15.1:0/1004132179 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c03c7a0 con 0x7f1624107620 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.745+0000 7f162b874640 1 -- 172.21.15.1:0/1004132179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162407c240 con 0x7f1624107620 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.745+0000 7f16227fc640 1 -- 172.21.15.1:0/1004132179 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c037040 con 0x7f1624107620 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.745+0000 7f162b874640 1 -- 172.21.15.1:0/1004132179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162407c780 con 0x7f1624107620 2024-09-16T00:18:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.745+0000 7f16227fc640 1 -- 172.21.15.1:0/1004132179 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f161c044070 con 0x7f1624107620 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.745+0000 7f16227fc640 1 -- 172.21.15.1:0/1004132179 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f161c03d3f0 con 0x7f1624107620 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.746+0000 7f162b874640 1 -- 172.21.15.1:0/1004132179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1624107aa0 con 0x7f1624107620 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.748+0000 7f16227fc640 1 -- 172.21.15.1:0/1004132179 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f161c046450 con 0x7f1624107620 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.809+0000 7f162b874640 1 -- 172.21.15.1:0/1004132179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f16241943e0 con 0x7f1624107620 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.810+0000 7f16227fc640 1 -- 172.21.15.1:0/1004132179 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f161c046630 con 0x7f1624107620 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.811+0000 7f162b874640 1 -- 172.21.15.1:0/1004132179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 msgr2=0x7f162407db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.811+0000 7f162b874640 1 --2- 172.21.15.1:0/1004132179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f162407db10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f161c0090b0 tx=0x7f161c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.811+0000 7f162b874640 1 -- 172.21.15.1:0/1004132179 shutdown_connections 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.811+0000 7f162b874640 1 --2- 172.21.15.1:0/1004132179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624107620 0x7f162407db10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.811+0000 7f162b874640 1 -- 172.21.15.1:0/1004132179 >> 172.21.15.1:0/1004132179 conn(0x7f1624102e50 msgr2=0x7f1624191ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.811+0000 7f162b874640 1 -- 172.21.15.1:0/1004132179 shutdown_connections 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:28.811+0000 7f162b874640 1 -- 172.21.15.1:0/1004132179 wait complete. 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:mon is available 2024-09-16T00:18:31.379 INFO:teuthology.orchestra.run.smithi001.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T00:18:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:34 smithi001 ceph-mon[30567]: from='client.? 172.21.15.1:0/3170271061' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:18:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:34 smithi001 ceph-mon[30567]: from='client.? 172.21.15.1:0/3170271061' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T00:18:39.702 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:39.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:39.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.221+0000 7f65c46c8640 1 Processor -- start 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.221+0000 7f65c46c8640 1 -- start start 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.221+0000 7f65c46c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.221+0000 7f65c46c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc107f60 con 0x7f65bc107620 2024-09-16T00:18:39.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.221+0000 7f65c2464640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.222+0000 7f65c2464640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46852/0 (socket says 172.21.15.1:46852) 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.222+0000 7f65c2464640 1 -- 172.21.15.1:0/1051996750 learned_addr learned my addr 172.21.15.1:0/1051996750 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.222+0000 7f65c2464640 1 -- 172.21.15.1:0/1051996750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc1080e0 con 0x7f65bc107620 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.222+0000 7f65c2464640 1 --2- 172.21.15.1:0/1051996750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc107a20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009820 tx=0x7f65b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6508184e58c8a06c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c1462640 1 -- 172.21.15.1:0/1051996750 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b8037070 con 0x7f65bc107620 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c1462640 1 -- 172.21.15.1:0/1051996750 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b8032440 con 0x7f65bc107620 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c1462640 1 -- 172.21.15.1:0/1051996750 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b8032740 con 0x7f65bc107620 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c46c8640 1 -- 172.21.15.1:0/1051996750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 msgr2=0x7f65bc107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c46c8640 1 --2- 172.21.15.1:0/1051996750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc107a20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009820 tx=0x7f65b802a270 comp rx=0 tx=0).stop 2024-09-16T00:18:39.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c46c8640 1 -- 172.21.15.1:0/1051996750 shutdown_connections 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c46c8640 1 --2- 172.21.15.1:0/1051996750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc107a20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c46c8640 1 -- 172.21.15.1:0/1051996750 >> 172.21.15.1:0/1051996750 conn(0x7f65bc102e50 msgr2=0x7f65bc105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c46c8640 1 -- 172.21.15.1:0/1051996750 shutdown_connections 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.223+0000 7f65c46c8640 1 -- 172.21.15.1:0/1051996750 wait complete. 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.224+0000 7f65c46c8640 1 Processor -- start 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.224+0000 7f65c46c8640 1 -- start start 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.224+0000 7f65c46c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.224+0000 7f65c46c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc108b40 con 0x7f65bc107620 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.224+0000 7f65c2464640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.224+0000 7f65c2464640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc19b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46868/0 (socket says 172.21.15.1:46868) 2024-09-16T00:18:39.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.224+0000 7f65c2464640 1 -- 172.21.15.1:0/3170271061 learned_addr learned my addr 172.21.15.1:0/3170271061 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:39.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.225+0000 7f65c2464640 1 -- 172.21.15.1:0/3170271061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc19b5e0 con 0x7f65bc107620 2024-09-16T00:18:39.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.225+0000 7f65c2464640 1 --2- 172.21.15.1:0/3170271061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc19b0a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f65b80090b0 tx=0x7f65b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:39.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.225+0000 7f65b37fe640 1 -- 172.21.15.1:0/3170271061 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b8041070 con 0x7f65bc107620 2024-09-16T00:18:39.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.225+0000 7f65b37fe640 1 -- 172.21.15.1:0/3170271061 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b803c7a0 con 0x7f65bc107620 2024-09-16T00:18:39.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.225+0000 7f65b37fe640 1 -- 172.21.15.1:0/3170271061 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b8037040 con 0x7f65bc107620 2024-09-16T00:18:39.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.225+0000 7f65c46c8640 1 -- 172.21.15.1:0/3170271061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc19b870 con 0x7f65bc107620 2024-09-16T00:18:39.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.225+0000 7f65c46c8640 1 -- 172.21.15.1:0/3170271061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc19e560 con 0x7f65bc107620 2024-09-16T00:18:39.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.225+0000 7f65b37fe640 1 -- 172.21.15.1:0/3170271061 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f65b8044070 con 0x7f65bc107620 2024-09-16T00:18:39.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.225+0000 7f65b37fe640 1 -- 172.21.15.1:0/3170271061 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65b803d3f0 con 0x7f65bc107620 2024-09-16T00:18:39.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.226+0000 7f65c46c8640 1 -- 172.21.15.1:0/3170271061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65bc107aa0 con 0x7f65bc107620 2024-09-16T00:18:39.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.228+0000 7f65b37fe640 1 -- 172.21.15.1:0/3170271061 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f65b8046450 con 0x7f65bc107620 2024-09-16T00:18:39.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.288+0000 7f65c46c8640 1 -- 172.21.15.1:0/3170271061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f65bc067bb0 con 0x7f65bc107620 2024-09-16T00:18:39.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.291+0000 7f65b37fe640 1 -- 172.21.15.1:0/3170271061 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7f65b8046630 con 0x7f65bc107620 2024-09-16T00:18:39.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.292+0000 7f65b37fe640 1 -- 172.21.15.1:0/3170271061 <== mon.0 v2:172.21.15.1:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65b8056070 con 0x7f65bc107620 2024-09-16T00:18:39.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.293+0000 7f65c46c8640 1 -- 172.21.15.1:0/3170271061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 msgr2=0x7f65bc19b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:39.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.293+0000 7f65c46c8640 1 --2- 172.21.15.1:0/3170271061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc19b0a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f65b80090b0 tx=0x7f65b80046c0 comp rx=0 tx=0).stop 2024-09-16T00:18:39.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.293+0000 7f65c46c8640 1 -- 172.21.15.1:0/3170271061 shutdown_connections 2024-09-16T00:18:39.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.293+0000 7f65c46c8640 1 --2- 172.21.15.1:0/3170271061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc107620 0x7f65bc19b0a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:39.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.293+0000 7f65c46c8640 1 -- 172.21.15.1:0/3170271061 >> 172.21.15.1:0/3170271061 conn(0x7f65bc102e50 msgr2=0x7f65bc191ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:39.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.293+0000 7f65c46c8640 1 -- 172.21.15.1:0/3170271061 shutdown_connections 2024-09-16T00:18:39.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:34.293+0000 7f65c46c8640 1 -- 172.21.15.1:0/3170271061 wait complete. 2024-09-16T00:18:39.717 INFO:teuthology.orchestra.run.smithi001.stdout:Generating new minimal ceph.conf... 2024-09-16T00:18:40.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:40 smithi001 ceph-mon[30567]: from='client.? 172.21.15.1:0/3983077623' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:18:40.750 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.129+0000 7fe628e7d640 1 Processor -- start 2024-09-16T00:18:40.750 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.129+0000 7fe628e7d640 1 -- start start 2024-09-16T00:18:40.750 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.129+0000 7fe628e7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe624107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:40.751 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.129+0000 7fe628e7d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe624107f40 con 0x7fe624107600 2024-09-16T00:18:40.751 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.129+0000 7fe62259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe624107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:40.751 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.129+0000 7fe62259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe624107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46876/0 (socket says 172.21.15.1:46876) 2024-09-16T00:18:40.751 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.129+0000 7fe62259c640 1 -- 172.21.15.1:0/2473774290 learned_addr learned my addr 172.21.15.1:0/2473774290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:40.751 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.130+0000 7fe62259c640 1 -- 172.21.15.1:0/2473774290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6241080c0 con 0x7fe624107600 2024-09-16T00:18:40.751 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.130+0000 7fe62259c640 1 --2- 172.21.15.1:0/2473774290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe624107a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe614009820 tx=0x7fe61402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f416a4fc2506f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:40.752 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe621d9b640 1 -- 172.21.15.1:0/2473774290 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe614037070 con 0x7fe624107600 2024-09-16T00:18:40.752 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe621d9b640 1 -- 172.21.15.1:0/2473774290 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe614032900 con 0x7fe624107600 2024-09-16T00:18:40.752 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe621d9b640 1 -- 172.21.15.1:0/2473774290 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe614030850 con 0x7fe624107600 2024-09-16T00:18:40.752 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe628e7d640 1 -- 172.21.15.1:0/2473774290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 msgr2=0x7fe624107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:40.752 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe628e7d640 1 --2- 172.21.15.1:0/2473774290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe624107a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe614009820 tx=0x7fe61402a270 comp rx=0 tx=0).stop 2024-09-16T00:18:40.753 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe628e7d640 1 -- 172.21.15.1:0/2473774290 shutdown_connections 2024-09-16T00:18:40.753 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe628e7d640 1 --2- 172.21.15.1:0/2473774290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe624107a00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:40.753 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe628e7d640 1 -- 172.21.15.1:0/2473774290 >> 172.21.15.1:0/2473774290 conn(0x7fe624102e50 msgr2=0x7fe624105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:40.753 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe628e7d640 1 -- 172.21.15.1:0/2473774290 shutdown_connections 2024-09-16T00:18:40.754 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.131+0000 7fe628e7d640 1 -- 172.21.15.1:0/2473774290 wait complete. 2024-09-16T00:18:40.754 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.132+0000 7fe628e7d640 1 Processor -- start 2024-09-16T00:18:40.754 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.132+0000 7fe628e7d640 1 -- start start 2024-09-16T00:18:40.754 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.132+0000 7fe628e7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe62419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:40.754 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.132+0000 7fe628e7d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe62410ba70 con 0x7fe624107600 2024-09-16T00:18:40.754 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe62259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe62419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:40.755 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe62259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe62419d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46880/0 (socket says 172.21.15.1:46880) 2024-09-16T00:18:40.755 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe62259c640 1 -- 172.21.15.1:0/3983077623 learned_addr learned my addr 172.21.15.1:0/3983077623 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:40.755 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe62259c640 1 -- 172.21.15.1:0/3983077623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62419dcf0 con 0x7fe624107600 2024-09-16T00:18:40.755 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe62259c640 1 --2- 172.21.15.1:0/3983077623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe62419d7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe614032ce0 tx=0x7fe614032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:40.755 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe61b7fe640 1 -- 172.21.15.1:0/3983077623 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe614042070 con 0x7fe624107600 2024-09-16T00:18:40.755 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe61b7fe640 1 -- 172.21.15.1:0/3983077623 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe6140308c0 con 0x7fe624107600 2024-09-16T00:18:40.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe628e7d640 1 -- 172.21.15.1:0/3983077623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62419df80 con 0x7fe624107600 2024-09-16T00:18:40.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe61b7fe640 1 -- 172.21.15.1:0/3983077623 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe614037040 con 0x7fe624107600 2024-09-16T00:18:40.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.133+0000 7fe628e7d640 1 -- 172.21.15.1:0/3983077623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6241a0c70 con 0x7fe624107600 2024-09-16T00:18:40.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.134+0000 7fe61b7fe640 1 -- 172.21.15.1:0/3983077623 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe614045070 con 0x7fe624107600 2024-09-16T00:18:40.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.134+0000 7fe61b7fe640 1 -- 172.21.15.1:0/3983077623 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe614047bd0 con 0x7fe624107600 2024-09-16T00:18:40.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.134+0000 7fe628e7d640 1 -- 172.21.15.1:0/3983077623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe624107a80 con 0x7fe624107600 2024-09-16T00:18:40.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.137+0000 7fe61b7fe640 1 -- 172.21.15.1:0/3983077623 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe61403c830 con 0x7fe624107600 2024-09-16T00:18:40.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.197+0000 7fe628e7d640 1 -- 172.21.15.1:0/3983077623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe6241a0e40 con 0x7fe624107600 2024-09-16T00:18:40.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.197+0000 7fe61b7fe640 1 -- 172.21.15.1:0/3983077623 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7fe61403ca10 con 0x7fe624107600 2024-09-16T00:18:40.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.198+0000 7fe628e7d640 1 -- 172.21.15.1:0/3983077623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 msgr2=0x7fe62419d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:40.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.198+0000 7fe628e7d640 1 --2- 172.21.15.1:0/3983077623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe62419d7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe614032ce0 tx=0x7fe614032d10 comp rx=0 tx=0).stop 2024-09-16T00:18:40.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.198+0000 7fe628e7d640 1 -- 172.21.15.1:0/3983077623 shutdown_connections 2024-09-16T00:18:40.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.198+0000 7fe628e7d640 1 --2- 172.21.15.1:0/3983077623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe624107600 0x7fe62419d7b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:40.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.198+0000 7fe628e7d640 1 -- 172.21.15.1:0/3983077623 >> 172.21.15.1:0/3983077623 conn(0x7fe624102e50 msgr2=0x7fe6241038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:40.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.198+0000 7fe628e7d640 1 -- 172.21.15.1:0/3983077623 shutdown_connections 2024-09-16T00:18:40.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:40.198+0000 7fe628e7d640 1 -- 172.21.15.1:0/3983077623 wait complete. 2024-09-16T00:18:40.758 INFO:teuthology.orchestra.run.smithi001.stdout:Restarting the monitor... 2024-09-16T00:18:41.043 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:40 smithi001 systemd[1]: Stopping Ceph mon.a for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:18:41.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:41 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a[30563]: 2024-09-16T00:18:41.042+0000 7fc372a5e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:18:41.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:41 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a[30563]: 2024-09-16T00:18:41.042+0000 7fc372a5e640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T00:18:41.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:41 smithi001 podman[30811]: 2024-09-16 00:18:41.247643682 +0000 UTC m=+0.318737112 container died 17dec7989c52ca7c10904722cb33b391c7c7b45b6780e75f7b69144f18db862b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:18:41.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:41 smithi001 podman[30811]: 2024-09-16 00:18:41.608376185 +0000 UTC m=+0.679469610 container cleanup 17dec7989c52ca7c10904722cb33b391c7c7b45b6780e75f7b69144f18db862b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T00:18:41.851 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:41 smithi001 bash[30811]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a 2024-09-16T00:18:42.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:41 smithi001 podman[30823]: 2024-09-16 00:18:41.850910499 +0000 UTC m=+0.601369747 container remove 17dec7989c52ca7c10904722cb33b391c7c7b45b6780e75f7b69144f18db862b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:18:42.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:41 smithi001 systemd[1]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T00:18:42.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:41 smithi001 systemd[1]: Stopped Ceph mon.a for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:18:42.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:41 smithi001 systemd[1]: Starting Ceph mon.a for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:18:42.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 podman[30916]: 2024-09-16 00:18:42.184944648 +0000 UTC m=+0.096951324 container create 196e28c9c2a564bd3fb5c19349da2cadd22c4591b09986bd73004b1284b8e0c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T00:18:42.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 podman[30916]: 2024-09-16 00:18:42.109674805 +0000 UTC m=+0.021681454 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:18:42.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 podman[30916]: 2024-09-16 00:18:42.370920039 +0000 UTC m=+0.282926674 container init 196e28c9c2a564bd3fb5c19349da2cadd22c4591b09986bd73004b1284b8e0c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:18:42.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 podman[30916]: 2024-09-16 00:18:42.374021583 +0000 UTC m=+0.286028218 container start 196e28c9c2a564bd3fb5c19349da2cadd22c4591b09986bd73004b1284b8e0c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T00:18:42.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: pidfile_write: ignore empty --pid-file 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: load: jerasure load: lrc 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: RocksDB version: 7.9.2 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Git sha 0 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: DB SUMMARY 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: DB Session ID: 66YEXQ1MTDHDHTZWO8S3 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:18:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104764 ; 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.env: 0x55b745bc0940 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.info_log: 0x55b74648f6e0 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:18:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.statistics: (nil) 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.use_fsync: 0 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:18:42.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.db_log_dir: 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.wal_dir: 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.write_buffer_manager: 0x55b74649edc0 2024-09-16T00:18:42.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:18:42.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:18:42.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:18:42.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:18:42.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:18:42.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:18:42.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:18:42.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:18:42.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.unordered_write: 0 2024-09-16T00:18:42.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:18:42.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:18:42.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:18:42.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:18:42.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.row_cache: None 2024-09-16T00:18:42.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.wal_filter: None 2024-09-16T00:18:42.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:18:42.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:18:42.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:18:42.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:18:42.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.wal_compression: 0 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T00:18:42.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_open_files: -1 2024-09-16T00:18:42.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Compression algorithms supported: 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: kZSTD supported: 0 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:18:42.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:18:42.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:18:42.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:18:42.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:18:42.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:18:42.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T00:18:42.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T00:18:42.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T00:18:42.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:18:42.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.merge_operator: 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_filter: None 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b74648f320) 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_type: 0 2024-09-16T00:18:42.464 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_index_type: 0 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_shortening: 1 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: checksum: 4 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: no_block_cache: 0 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache: 0x55b7464831f0 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_options: 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: capacity : 536870912 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_shard_bits : 4 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: strict_capacity_limit : 0 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_compressed: (nil) 2024-09-16T00:18:42.465 INFO:journalctl@ceph.mon.a.smithi001.stdout: persistent_cache: (nil) 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size: 4096 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size_deviation: 10 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_restart_interval: 16 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_block_restart_interval: 1 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: metadata_block_size: 4096 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: partition_filters: 0 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: use_delta_encoding: 1 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: filter_policy: bloomfilter 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: whole_key_filtering: 1 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: verify_compression: 0 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:18:42.466 INFO:journalctl@ceph.mon.a.smithi001.stdout: format_version: 5 2024-09-16T00:18:42.467 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_index_compression: 1 2024-09-16T00:18:42.467 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_align: 0 2024-09-16T00:18:42.467 INFO:journalctl@ceph.mon.a.smithi001.stdout: max_auto_readahead_size: 262144 2024-09-16T00:18:42.467 INFO:journalctl@ceph.mon.a.smithi001.stdout: prepopulate_block_cache: 0 2024-09-16T00:18:42.467 INFO:journalctl@ceph.mon.a.smithi001.stdout: initial_auto_readahead_size: 8192 2024-09-16T00:18:42.467 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T00:18:42.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:18:42.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:18:42.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression: NoCompression 2024-09-16T00:18:42.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:18:42.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:18:42.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:18:42.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.num_levels: 7 2024-09-16T00:18:42.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:18:42.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:18:42.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:18:42.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:18:42.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:18:42.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:18:42.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:18:42.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:18:42.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:18:42.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:18:42.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:18:42.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:18:42.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:18:42.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:18:42.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:18:42.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:18:42.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:18:42.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:18:42.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:18:42.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:18:42.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:18:42.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:18:42.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:18:42.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:18:42.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:18:42.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:18:42.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:18:42.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:18:42.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:18:42.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:18:42.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:18:42.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:18:42.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:18:42.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:18:42.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:18:42.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:18:42.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:18:42.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:18:42.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T00:18:42.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T00:18:42.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:18:42.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:18:42.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:18:42.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:18:42.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:18:42.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:18:42.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:18:42.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:18:42.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:18:42.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:18:42.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:18:42.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:18:42.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:18:42.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:18:42.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:18:42.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:18:42.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:18:42.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:18:42.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:18:42.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:18:42.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:18:42.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:18:42.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:18:42.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:18:42.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:18:42.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.ttl: 2592000 2024-09-16T00:18:42.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:18:42.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T00:18:42.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T00:18:42.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:18:42.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:18:42.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:18:42.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:18:42.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:18:42.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:18:42.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T00:18:42.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T00:18:42.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T00:18:42.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T00:18:42.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T00:18:42.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T00:18:42.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 218f7e31-26b6-431a-82ce-e7172d8a85ba 2024-09-16T00:18:42.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445922402226, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T00:18:42.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T00:18:42.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445922403363, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99877, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97763, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14181, "raw_average_key_size": 49, "raw_value_size": 90088, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726445922, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "218f7e31-26b6-431a-82ce-e7172d8a85ba", "db_session_id": "66YEXQ1MTDHDHTZWO8S3", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T00:18:42.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445922403463, "job": 1, "event": "recovery_finished"} 2024-09-16T00:18:42.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T00:18:42.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:18:42.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b7464b0e00 2024-09-16T00:18:42.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: DB pointer 0x55b7465ac000 2024-09-16T00:18:42.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T00:18:42.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T00:18:42.484 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** DB Stats ** 2024-09-16T00:18:42.484 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:18:42.484 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:18:42.484 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:18:42.484 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:18:42.484 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:18:42.484 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:18:42.485 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:18:42.485 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-16T00:18:42.485 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-09-16T00:18:42.485 INFO:journalctl@ceph.mon.a.smithi001.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:18:42.485 INFO:journalctl@ceph.mon.a.smithi001.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T00:18:42.485 INFO:journalctl@ceph.mon.a.smithi001.stdout: L0 2/0 99.40 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:18:42.485 INFO:journalctl@ceph.mon.a.smithi001.stdout: Sum 2/0 99.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 130.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:18:42.486 INFO:journalctl@ceph.mon.a.smithi001.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative compaction: 0.00 GB write, 28.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval compaction: 0.00 GB write, 28.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache BinnedLRUCache@0x55b7464831f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: starting mon.a rank 0 at public addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] at bind addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mon.a@-1(???) e1 preinit fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:18:42.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mon.a@-1(???).mds e1 new map 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mon.a@-1(???).mds e1 print_map 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout: e1 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout: btime 2024-09-16T00:18:28:248994+0000 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout: legacy client fscid: -1 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout: No filesystems configured 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:18:42.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:18:42.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:18:42.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T00:18:42.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 bash[30916]: 196e28c9c2a564bd3fb5c19349da2cadd22c4591b09986bd73004b1284b8e0c9 2024-09-16T00:18:42.489 INFO:teuthology.orchestra.run.smithi001.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T00:18:42.713 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 systemd[1]: Started Ceph mon.a for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: monmap epoch 1 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: last_changed 2024-09-16T00:18:25.266239+0000 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: created 2024-09-16T00:18:25.266239+0000 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: min_mon_release 19 (squid) 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: election_strategy: 1 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: fsmap 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T00:18:42.714 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:42 smithi001 ceph-mon[30934]: mgrmap e1: no daemons active 2024-09-16T00:18:43.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.898+0000 7f8184e97640 1 Processor -- start 2024-09-16T00:18:43.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.898+0000 7f8184e97640 1 -- start start 2024-09-16T00:18:43.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.899+0000 7f8184e97640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f81801079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:43.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.899+0000 7f8184e97640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8180107f80 con 0x7f81801075b0 2024-09-16T00:18:43.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.899+0000 7f817e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f81801079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:43.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.899+0000 7f817e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f81801079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42002/0 (socket says 172.21.15.1:42002) 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.899+0000 7f817e59c640 1 -- 172.21.15.1:0/1754638269 learned_addr learned my addr 172.21.15.1:0/1754638269 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.899+0000 7f817e59c640 1 -- 172.21.15.1:0/1754638269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8180108760 con 0x7f81801075b0 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.901+0000 7f817e59c640 1 --2- 172.21.15.1:0/1754638269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f81801079b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8168009e80 tx=0x7f816802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e07e9c31b22e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.901+0000 7f817d59a640 1 -- 172.21.15.1:0/1754638269 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8168036070 con 0x7f81801075b0 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.901+0000 7f817d59a640 1 -- 172.21.15.1:0/1754638269 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f81680318c0 con 0x7f81801075b0 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.901+0000 7f817d59a640 1 -- 172.21.15.1:0/1754638269 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8168007ab0 con 0x7f81801075b0 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.902+0000 7f8184e97640 1 -- 172.21.15.1:0/1754638269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 msgr2=0x7f81801079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.902+0000 7f8184e97640 1 --2- 172.21.15.1:0/1754638269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f81801079b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8168009e80 tx=0x7f816802a7a0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.902+0000 7f8184e97640 1 -- 172.21.15.1:0/1754638269 shutdown_connections 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.902+0000 7f8184e97640 1 --2- 172.21.15.1:0/1754638269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f81801079b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.902+0000 7f8184e97640 1 -- 172.21.15.1:0/1754638269 >> 172.21.15.1:0/1754638269 conn(0x7f8180102d20 msgr2=0x7f8180105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.902+0000 7f8184e97640 1 -- 172.21.15.1:0/1754638269 shutdown_connections 2024-09-16T00:18:43.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.902+0000 7f8184e97640 1 -- 172.21.15.1:0/1754638269 wait complete. 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.903+0000 7f8184e97640 1 Processor -- start 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.903+0000 7f8184e97640 1 -- start start 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.903+0000 7f8184e97640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f818007ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.903+0000 7f8184e97640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f818010bb10 con 0x7f81801075b0 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.903+0000 7f817e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f818007ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.903+0000 7f817e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f818007ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42008/0 (socket says 172.21.15.1:42008) 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.903+0000 7f817e59c640 1 -- 172.21.15.1:0/77732935 learned_addr learned my addr 172.21.15.1:0/77732935 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.903+0000 7f817e59c640 1 -- 172.21.15.1:0/77732935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818007cf40 con 0x7f81801075b0 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.904+0000 7f817e59c640 1 --2- 172.21.15.1:0/77732935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f818007ca00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f816802acc0 tx=0x7f8168031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.904+0000 7f81777fe640 1 -- 172.21.15.1:0/77732935 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8168036050 con 0x7f81801075b0 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.904+0000 7f8184e97640 1 -- 172.21.15.1:0/77732935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818007b130 con 0x7f81801075b0 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.904+0000 7f81777fe640 1 -- 172.21.15.1:0/77732935 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8168037040 con 0x7f81801075b0 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.904+0000 7f81777fe640 1 -- 172.21.15.1:0/77732935 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f816803d8c0 con 0x7f81801075b0 2024-09-16T00:18:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.904+0000 7f8184e97640 1 -- 172.21.15.1:0/77732935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818007b670 con 0x7f81801075b0 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.904+0000 7f81777fe640 1 -- 172.21.15.1:0/77732935 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8168044020 con 0x7f81801075b0 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.904+0000 7f81777fe640 1 -- 172.21.15.1:0/77732935 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8168042020 con 0x7f81801075b0 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.905+0000 7f8184e97640 1 -- 172.21.15.1:0/77732935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818010bc90 con 0x7f81801075b0 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.907+0000 7f81777fe640 1 -- 172.21.15.1:0/77732935 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f816804b020 con 0x7f81801075b0 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.967+0000 7f8184e97640 1 -- 172.21.15.1:0/77732935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f8180107a30 con 0x7f81801075b0 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.968+0000 7f81777fe640 1 -- 172.21.15.1:0/77732935 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f81680073b0 con 0x7f81801075b0 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.969+0000 7f8184e97640 1 -- 172.21.15.1:0/77732935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 msgr2=0x7f818007ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.969+0000 7f8184e97640 1 --2- 172.21.15.1:0/77732935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f818007ca00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f816802acc0 tx=0x7f8168031ca0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.969+0000 7f8184e97640 1 -- 172.21.15.1:0/77732935 shutdown_connections 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.969+0000 7f8184e97640 1 --2- 172.21.15.1:0/77732935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81801075b0 0x7f818007ca00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.970+0000 7f8184e97640 1 -- 172.21.15.1:0/77732935 >> 172.21.15.1:0/77732935 conn(0x7f8180102d20 msgr2=0x7f8180103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.970+0000 7f8184e97640 1 -- 172.21.15.1:0/77732935 shutdown_connections 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:42.970+0000 7f8184e97640 1 -- 172.21.15.1:0/77732935 wait complete. 2024-09-16T00:18:43.474 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T00:18:43.493 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:18:43.493 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mgr... 2024-09-16T00:18:43.494 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T00:18:43.494 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T00:18:43.749 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.x 2024-09-16T00:18:43.749 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-16T00:18:43.902 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1226ed86-73c1-11ef-bceb-c7b262605968.target.wants/ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1226ed86-73c1-11ef-bceb-c7b262605968@.service. 2024-09-16T00:18:43.934 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:43 smithi001 systemd[1]: Starting Ceph mgr.x for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:18:44.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:43 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/77732935' entity='client.admin' 2024-09-16T00:18:44.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 podman[31148]: 2024-09-16 00:18:44.051985862 +0000 UTC m=+0.016219617 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:18:44.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 podman[31148]: 2024-09-16 00:18:44.170869665 +0000 UTC m=+0.135103411 container create 7c193014c9329b61f157c23e4b6e789a2db5013745fb73646953adbee40dc192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T00:18:44.481 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:18:44.482 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-16T00:18:44.491 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:18:44.491 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-16T00:18:44.491 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-16T00:18:44.491 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:18:44.500 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:18:44.500 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-16T00:18:44.508 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:18:44.508 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-16T00:18:44.508 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-16T00:18:44.508 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T00:18:44.508 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr to start... 2024-09-16T00:18:44.508 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr... 2024-09-16T00:18:44.579 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 podman[31148]: 2024-09-16 00:18:44.38181809 +0000 UTC m=+0.346051831 container init 7c193014c9329b61f157c23e4b6e789a2db5013745fb73646953adbee40dc192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:18:44.579 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 podman[31148]: 2024-09-16 00:18:44.384930525 +0000 UTC m=+0.349164268 container start 7c193014c9329b61f157c23e4b6e789a2db5013745fb73646953adbee40dc192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T00:18:44.579 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 bash[31148]: 7c193014c9329b61f157c23e4b6e789a2db5013745fb73646953adbee40dc192 2024-09-16T00:18:44.579 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 systemd[1]: Started Ceph mgr.x for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:18:44.579 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:44.516+0000 7f3ab74ec040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:18:44.579 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:44.578+0000 7f3ab74ec040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:18:44.899 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:44.739+0000 7f3ab74ec040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:18:44.900 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:44.809+0000 7f3ab74ec040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:18:45.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:45 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3011521680' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:18:45.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:44 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:44.969+0000 7f3ab74ec040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:18:45.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:45 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:45.121+0000 7f3ab74ec040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:18:45.714 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:45 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:45.453+0000 7f3ab74ec040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:18:45.714 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:45 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:45.517+0000 7f3ab74ec040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:18:45.714 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:45 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:45.713+0000 7f3ab74ec040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:18:46.888 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:46 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:46.645+0000 7f3ab74ec040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:18:46.888 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:46 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:46.717+0000 7f3ab74ec040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:18:46.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:46.899 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:18:46.899 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:18:46.899 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:18:46.899 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:18:46.899 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:18:46.899 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:18:46.899 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:46.899 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:18:46.899 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:46.900 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:18:46.901 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:18:28:248994+0000", 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:18:46.902 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:18:28.249645+0000", 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.898+0000 7eff4812c640 1 Processor -- start 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.899+0000 7eff4812c640 1 -- start start 2024-09-16T00:18:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.899+0000 7eff4812c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff4010a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.899+0000 7eff4812c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff40075840 con 0x7eff40108040 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.899+0000 7eff45ec8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff4010a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.899+0000 7eff45ec8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff4010a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42042/0 (socket says 172.21.15.1:42042) 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.899+0000 7eff45ec8640 1 -- 172.21.15.1:0/3593152273 learned_addr learned my addr 172.21.15.1:0/3593152273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.900+0000 7eff45ec8640 1 -- 172.21.15.1:0/3593152273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4010a970 con 0x7eff40108040 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.900+0000 7eff45ec8640 1 --2- 172.21.15.1:0/3593152273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff4010a430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009820 tx=0x7eff3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0b13b3bc4a947d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.900+0000 7eff44ec6640 1 -- 172.21.15.1:0/3593152273 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff3c037070 con 0x7eff40108040 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.900+0000 7eff44ec6640 1 -- 172.21.15.1:0/3593152273 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7eff3c032900 con 0x7eff40108040 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.900+0000 7eff44ec6640 1 -- 172.21.15.1:0/3593152273 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff3c030850 con 0x7eff40108040 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.901+0000 7eff4812c640 1 -- 172.21.15.1:0/3593152273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 msgr2=0x7eff4010a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.901+0000 7eff4812c640 1 --2- 172.21.15.1:0/3593152273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff4010a430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009820 tx=0x7eff3c02a270 comp rx=0 tx=0).stop 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.901+0000 7eff4812c640 1 -- 172.21.15.1:0/3593152273 shutdown_connections 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.901+0000 7eff4812c640 1 --2- 172.21.15.1:0/3593152273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff4010a430 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:46.904 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.901+0000 7eff4812c640 1 -- 172.21.15.1:0/3593152273 >> 172.21.15.1:0/3593152273 conn(0x7eff40101ca0 msgr2=0x7eff401040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.901+0000 7eff4812c640 1 -- 172.21.15.1:0/3593152273 shutdown_connections 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.901+0000 7eff4812c640 1 -- 172.21.15.1:0/3593152273 wait complete. 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.902+0000 7eff4812c640 1 Processor -- start 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.902+0000 7eff4812c640 1 -- start start 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.902+0000 7eff4812c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff401078c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.902+0000 7eff4812c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff4010b9b0 con 0x7eff40108040 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.902+0000 7eff45ec8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff401078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.902+0000 7eff45ec8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff401078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42056/0 (socket says 172.21.15.1:42056) 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.902+0000 7eff45ec8640 1 -- 172.21.15.1:0/3011521680 learned_addr learned my addr 172.21.15.1:0/3011521680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.903+0000 7eff45ec8640 1 -- 172.21.15.1:0/3011521680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff40107e00 con 0x7eff40108040 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.903+0000 7eff45ec8640 1 --2- 172.21.15.1:0/3011521680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff401078c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7eff3c032ce0 tx=0x7eff3c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.903+0000 7eff36ffd640 1 -- 172.21.15.1:0/3011521680 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff3c042070 con 0x7eff40108040 2024-09-16T00:18:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.903+0000 7eff36ffd640 1 -- 172.21.15.1:0/3011521680 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7eff3c0308c0 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.903+0000 7eff4812c640 1 -- 172.21.15.1:0/3011521680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff40105ff0 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.903+0000 7eff36ffd640 1 -- 172.21.15.1:0/3011521680 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff3c037040 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.903+0000 7eff4812c640 1 -- 172.21.15.1:0/3011521680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff40106530 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.903+0000 7eff36ffd640 1 -- 172.21.15.1:0/3011521680 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff3c045070 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.903+0000 7eff36ffd640 1 -- 172.21.15.1:0/3011521680 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7eff3c047b80 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.904+0000 7eff4812c640 1 -- 172.21.15.1:0/3011521680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff401084a0 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.906+0000 7eff36ffd640 1 -- 172.21.15.1:0/3011521680 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7eff3c047e20 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.969+0000 7eff4812c640 1 -- 172.21.15.1:0/3011521680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7eff40106a60 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.970+0000 7eff36ffd640 1 -- 172.21.15.1:0/3011521680 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7eff3c03c760 con 0x7eff40108040 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.970+0000 7eff4812c640 1 -- 172.21.15.1:0/3011521680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 msgr2=0x7eff401078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.970+0000 7eff4812c640 1 --2- 172.21.15.1:0/3011521680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff401078c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7eff3c032ce0 tx=0x7eff3c032d10 comp rx=0 tx=0).stop 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.970+0000 7eff4812c640 1 -- 172.21.15.1:0/3011521680 shutdown_connections 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.970+0000 7eff4812c640 1 --2- 172.21.15.1:0/3011521680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff40108040 0x7eff401078c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:46.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.970+0000 7eff4812c640 1 -- 172.21.15.1:0/3011521680 >> 172.21.15.1:0/3011521680 conn(0x7eff40101ca0 msgr2=0x7eff40102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:46.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.971+0000 7eff4812c640 1 -- 172.21.15.1:0/3011521680 shutdown_connections 2024-09-16T00:18:46.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:44.971+0000 7eff4812c640 1 -- 172.21.15.1:0/3011521680 wait complete. 2024-09-16T00:18:46.907 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (1/15)... 2024-09-16T00:18:47.749 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:47 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:47.478+0000 7f3ab74ec040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:18:47.749 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:47 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:47.599+0000 7f3ab74ec040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:18:48.044 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:47 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:47.749+0000 7f3ab74ec040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:18:48.044 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:47 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:47.812+0000 7f3ab74ec040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:18:48.044 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:47 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:47.971+0000 7f3ab74ec040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:18:48.044 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:48.043+0000 7f3ab74ec040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:18:48.348 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:48.107+0000 7f3ab74ec040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:18:48.349 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:48.277+0000 7f3ab74ec040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:18:48.349 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:48.347+0000 7f3ab74ec040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:18:48.636 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:48.419+0000 7f3ab74ec040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:18:48.636 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:48.508+0000 7f3ab74ec040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:18:48.636 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:48.571+0000 7f3ab74ec040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:18:48.636 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:48.635+0000 7f3ab74ec040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:18:49.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:49 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2646792248' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:18:49.703 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:49 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:49.490+0000 7f3ab74ec040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:18:50.013 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:18:50.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:18:50.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:18:50.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:18:28:248994+0000", 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:18:50.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:18:28.249645+0000", 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.353+0000 7f156aee8640 1 Processor -- start 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.353+0000 7f156aee8640 1 -- start start 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.353+0000 7f156aee8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156410a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:50.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.353+0000 7f156aee8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f156407bd40 con 0x7f1564108110 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.354+0000 7f1568c84640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156410a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.354+0000 7f1568c84640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156410a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42072/0 (socket says 172.21.15.1:42072) 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.354+0000 7f1568c84640 1 -- 172.21.15.1:0/469232886 learned_addr learned my addr 172.21.15.1:0/469232886 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.354+0000 7f1568c84640 1 -- 172.21.15.1:0/469232886 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156410aa40 con 0x7f1564108110 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.355+0000 7f1568c84640 1 --2- 172.21.15.1:0/469232886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156410a500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1558009820 tx=0x7f155802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ff4062b9bc004e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.355+0000 7f15637fe640 1 -- 172.21.15.1:0/469232886 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1558037070 con 0x7f1564108110 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.355+0000 7f15637fe640 1 -- 172.21.15.1:0/469232886 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1558032900 con 0x7f1564108110 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.355+0000 7f15637fe640 1 -- 172.21.15.1:0/469232886 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1558030850 con 0x7f1564108110 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.356+0000 7f156aee8640 1 -- 172.21.15.1:0/469232886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 msgr2=0x7f156410a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.356+0000 7f156aee8640 1 --2- 172.21.15.1:0/469232886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156410a500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1558009820 tx=0x7f155802a270 comp rx=0 tx=0).stop 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.356+0000 7f156aee8640 1 -- 172.21.15.1:0/469232886 shutdown_connections 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.356+0000 7f156aee8640 1 --2- 172.21.15.1:0/469232886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156410a500 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:50.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.356+0000 7f156aee8640 1 -- 172.21.15.1:0/469232886 >> 172.21.15.1:0/469232886 conn(0x7f1564101c70 msgr2=0x7f15641040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.356+0000 7f156aee8640 1 -- 172.21.15.1:0/469232886 shutdown_connections 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.356+0000 7f156aee8640 1 -- 172.21.15.1:0/469232886 wait complete. 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.356+0000 7f156aee8640 1 Processor -- start 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.356+0000 7f156aee8640 1 -- start start 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f156aee8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f156aee8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f156410b9c0 con 0x7f1564108110 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f1568c84640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f1568c84640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156419b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42082/0 (socket says 172.21.15.1:42082) 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f1568c84640 1 -- 172.21.15.1:0/2646792248 learned_addr learned my addr 172.21.15.1:0/2646792248 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f1568c84640 1 -- 172.21.15.1:0/2646792248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156419be10 con 0x7f1564108110 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f1568c84640 1 --2- 172.21.15.1:0/2646792248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156419b8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15580097f0 tx=0x7f1558030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f1561ffb640 1 -- 172.21.15.1:0/2646792248 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1558042070 con 0x7f1564108110 2024-09-16T00:18:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f1561ffb640 1 -- 172.21.15.1:0/2646792248 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1558032ca0 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f1561ffb640 1 -- 172.21.15.1:0/2646792248 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1558037040 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.357+0000 7f156aee8640 1 -- 172.21.15.1:0/2646792248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156419c0a0 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.358+0000 7f156aee8640 1 -- 172.21.15.1:0/2646792248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f156419ed90 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.358+0000 7f1561ffb640 1 -- 172.21.15.1:0/2646792248 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1558045070 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.358+0000 7f1561ffb640 1 -- 172.21.15.1:0/2646792248 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1558047a90 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.359+0000 7f156aee8640 1 -- 172.21.15.1:0/2646792248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1564108570 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.361+0000 7f1561ffb640 1 -- 172.21.15.1:0/2646792248 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f155803db10 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.424+0000 7f156aee8640 1 -- 172.21.15.1:0/2646792248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f156402d7c0 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.424+0000 7f1561ffb640 1 -- 172.21.15.1:0/2646792248 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f155803d430 con 0x7f1564108110 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.426+0000 7f156aee8640 1 -- 172.21.15.1:0/2646792248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 msgr2=0x7f156419b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.426+0000 7f156aee8640 1 --2- 172.21.15.1:0/2646792248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156419b8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15580097f0 tx=0x7f1558030ec0 comp rx=0 tx=0).stop 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.426+0000 7f156aee8640 1 -- 172.21.15.1:0/2646792248 shutdown_connections 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.426+0000 7f156aee8640 1 --2- 172.21.15.1:0/2646792248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1564108110 0x7f156419b8d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.426+0000 7f156aee8640 1 -- 172.21.15.1:0/2646792248 >> 172.21.15.1:0/2646792248 conn(0x7f1564101c70 msgr2=0x7f1564102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:50.031 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.426+0000 7f156aee8640 1 -- 172.21.15.1:0/2646792248 shutdown_connections 2024-09-16T00:18:50.031 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:49.426+0000 7f156aee8640 1 -- 172.21.15.1:0/2646792248 wait complete. 2024-09-16T00:18:50.031 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (2/15)... 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: Activating manager daemon x 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: mgrmap e2: x(active, starting, since 0.00293789s) 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: Manager daemon x is now available 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:18:50.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' 2024-09-16T00:18:50.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' 2024-09-16T00:18:50.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:50 smithi001 ceph-mon[30934]: from='mgr.14100 172.21.15.1:0/2994713371' entity='mgr.x' 2024-09-16T00:18:51.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:51 smithi001 ceph-mon[30934]: mgrmap e3: x(active, since 1.0067s) 2024-09-16T00:18:52.837 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:52 smithi001 ceph-mon[30934]: mgrmap e4: x(active, since 2s) 2024-09-16T00:18:53.467 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:53.467 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:18:53.467 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:18:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:18:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:18:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:18:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:18:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:18:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:18:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:18:53.469 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:18:53.469 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:18:53.469 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:18:53.469 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:18:53.469 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-16T00:18:53.469 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:18:53.469 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:53.469 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:18:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:18:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:18:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:18:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:18:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:18:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:18:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:18:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:18:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:18:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:18:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:18:53.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:18:53.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:18:53.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:18:53.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:18:53.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:18:53.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:18:53.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:53.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:18:53.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:53.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:18:28:248994+0000", 2024-09-16T00:18:53.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:18:53.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:18:53.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:53.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:18:53.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:18:53.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:18:28.249645+0000", 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:18:53.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.583+0000 7fc30db03640 1 Processor -- start 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.583+0000 7fc30db03640 1 -- start start 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.584+0000 7fc30db03640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30810a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.584+0000 7fc30db03640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3080757c0 con 0x7fc3081080c0 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.584+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30810a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.584+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30810a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37250/0 (socket says 172.21.15.1:37250) 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.584+0000 7fc3077fe640 1 -- 172.21.15.1:0/3834203054 learned_addr learned my addr 172.21.15.1:0/3834203054 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.584+0000 7fc3077fe640 1 -- 172.21.15.1:0/3834203054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30810a9f0 con 0x7fc3081080c0 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.585+0000 7fc3077fe640 1 --2- 172.21.15.1:0/3834203054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30810a4b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009820 tx=0x7fc2f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553b568e80734eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.585+0000 7fc3067fc640 1 -- 172.21.15.1:0/3834203054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f8037070 con 0x7fc3081080c0 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.585+0000 7fc3067fc640 1 -- 172.21.15.1:0/3834203054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc2f8032900 con 0x7fc3081080c0 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.585+0000 7fc3067fc640 1 -- 172.21.15.1:0/3834203054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f8030850 con 0x7fc3081080c0 2024-09-16T00:18:53.475 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.586+0000 7fc30db03640 1 -- 172.21.15.1:0/3834203054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 msgr2=0x7fc30810a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:53.476 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.586+0000 7fc30db03640 1 --2- 172.21.15.1:0/3834203054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30810a4b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009820 tx=0x7fc2f802a270 comp rx=0 tx=0).stop 2024-09-16T00:18:53.476 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.586+0000 7fc30db03640 1 -- 172.21.15.1:0/3834203054 shutdown_connections 2024-09-16T00:18:53.476 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.586+0000 7fc30db03640 1 --2- 172.21.15.1:0/3834203054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30810a4b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.476 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.586+0000 7fc30db03640 1 -- 172.21.15.1:0/3834203054 >> 172.21.15.1:0/3834203054 conn(0x7fc308101ca0 msgr2=0x7fc3081040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:53.476 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.586+0000 7fc30db03640 1 -- 172.21.15.1:0/3834203054 shutdown_connections 2024-09-16T00:18:53.476 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.586+0000 7fc30db03640 1 -- 172.21.15.1:0/3834203054 wait complete. 2024-09-16T00:18:53.476 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.586+0000 7fc30db03640 1 Processor -- start 2024-09-16T00:18:53.476 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.587+0000 7fc30db03640 1 -- start start 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.587+0000 7fc30db03640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.587+0000 7fc30db03640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810b9d0 con 0x7fc3081080c0 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.587+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.587+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30819b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37260/0 (socket says 172.21.15.1:37260) 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.587+0000 7fc3077fe640 1 -- 172.21.15.1:0/3580267222 learned_addr learned my addr 172.21.15.1:0/3580267222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.587+0000 7fc3077fe640 1 -- 172.21.15.1:0/3580267222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30819bad0 con 0x7fc3081080c0 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.587+0000 7fc3077fe640 1 --2- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30819b590 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8032ce0 tx=0x7fc2f8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.587+0000 7fc304ff9640 1 -- 172.21.15.1:0/3580267222 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f8042070 con 0x7fc3081080c0 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.588+0000 7fc304ff9640 1 -- 172.21.15.1:0/3580267222 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc2f80308c0 con 0x7fc3081080c0 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.588+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30819bd60 con 0x7fc3081080c0 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.588+0000 7fc304ff9640 1 -- 172.21.15.1:0/3580267222 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f8037040 con 0x7fc3081080c0 2024-09-16T00:18:53.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.588+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30819ea50 con 0x7fc3081080c0 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.588+0000 7fc304ff9640 1 -- 172.21.15.1:0/3580267222 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fc2f8045070 con 0x7fc3081080c0 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.589+0000 7fc304ff9640 1 --2- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fc2f0039160 0x7fc2f003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.589+0000 7fc304ff9640 1 -- 172.21.15.1:0/3580267222 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc2f806c230 con 0x7fc3081080c0 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.589+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc308108520 con 0x7fc3081080c0 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.589+0000 7fc306ffd640 1 --2- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fc2f0039160 0x7fc2f003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.589+0000 7fc306ffd640 1 --2- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fc2f0039160 0x7fc2f003b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0098c0 tx=0x7fc2fc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.594+0000 7fc304ff9640 1 -- 172.21.15.1:0/3580267222 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2f804b170 con 0x7fc3081080c0 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.836+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc30819ef60 con 0x7fc3081080c0 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.837+0000 7fc304ff9640 1 -- 172.21.15.1:0/3580267222 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fc2f803cd50 con 0x7fc3081080c0 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fc2f0039160 msgr2=0x7fc2f003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 --2- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fc2f0039160 0x7fc2f003b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0098c0 tx=0x7fc2fc006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 msgr2=0x7fc30819b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 --2- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30819b590 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8032ce0 tx=0x7fc2f8032d10 comp rx=0 tx=0).stop 2024-09-16T00:18:53.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 shutdown_connections 2024-09-16T00:18:53.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 --2- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fc2f0039160 0x7fc2f003b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 --2- 172.21.15.1:0/3580267222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3081080c0 0x7fc30819b590 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 >> 172.21.15.1:0/3580267222 conn(0x7fc308101ca0 msgr2=0x7fc3081028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:53.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 shutdown_connections 2024-09-16T00:18:53.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:52.839+0000 7fc30db03640 1 -- 172.21.15.1:0/3580267222 wait complete. 2024-09-16T00:18:53.479 INFO:teuthology.orchestra.run.smithi001.stdout:mgr is available 2024-09-16T00:18:53.575 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:53 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3580267222' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:18:54.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:54 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/440907375' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:54.682 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.865+0000 7f60a8bb7640 1 Processor -- start 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.865+0000 7f60a8bb7640 1 -- start start 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.865+0000 7f60a8bb7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a410a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.865+0000 7f60a8bb7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60a407bdb0 con 0x7f60a4108100 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.865+0000 7f60a259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a410a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:54.683 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.865+0000 7f60a259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a410a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37272/0 (socket says 172.21.15.1:37272) 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.865+0000 7f60a259c640 1 -- 172.21.15.1:0/4192135912 learned_addr learned my addr 172.21.15.1:0/4192135912 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.866+0000 7f60a259c640 1 -- 172.21.15.1:0/4192135912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a410aa30 con 0x7f60a4108100 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.866+0000 7f60a259c640 1 --2- 172.21.15.1:0/4192135912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a410a4f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6094009820 tx=0x7f609402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386600c27fe07cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.866+0000 7f60a1d9b640 1 -- 172.21.15.1:0/4192135912 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6094037070 con 0x7f60a4108100 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.867+0000 7f60a1d9b640 1 -- 172.21.15.1:0/4192135912 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6094032900 con 0x7f60a4108100 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.867+0000 7f60a1d9b640 1 -- 172.21.15.1:0/4192135912 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6094030850 con 0x7f60a4108100 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.867+0000 7f60a8bb7640 1 -- 172.21.15.1:0/4192135912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 msgr2=0x7f60a410a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.867+0000 7f60a8bb7640 1 --2- 172.21.15.1:0/4192135912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a410a4f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6094009820 tx=0x7f609402a270 comp rx=0 tx=0).stop 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.867+0000 7f60a8bb7640 1 -- 172.21.15.1:0/4192135912 shutdown_connections 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.867+0000 7f60a8bb7640 1 --2- 172.21.15.1:0/4192135912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a410a4f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.867+0000 7f60a8bb7640 1 -- 172.21.15.1:0/4192135912 >> 172.21.15.1:0/4192135912 conn(0x7f60a4101d20 msgr2=0x7f60a4104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:54.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.867+0000 7f60a8bb7640 1 -- 172.21.15.1:0/4192135912 shutdown_connections 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.867+0000 7f60a8bb7640 1 -- 172.21.15.1:0/4192135912 wait complete. 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.868+0000 7f60a8bb7640 1 Processor -- start 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.868+0000 7f60a8bb7640 1 -- start start 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.868+0000 7f60a8bb7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.868+0000 7f60a8bb7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60a410ba10 con 0x7f60a4108100 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.868+0000 7f60a259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.868+0000 7f60a259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37274/0 (socket says 172.21.15.1:37274) 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.868+0000 7f60a259c640 1 -- 172.21.15.1:0/440907375 learned_addr learned my addr 172.21.15.1:0/440907375 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.868+0000 7f60a259c640 1 -- 172.21.15.1:0/440907375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a419be20 con 0x7f60a4108100 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.869+0000 7f60a259c640 1 --2- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a419b8e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6094031000 tx=0x7f60940326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.869+0000 7f609b7fe640 1 -- 172.21.15.1:0/440907375 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6094042070 con 0x7f60a4108100 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.869+0000 7f609b7fe640 1 -- 172.21.15.1:0/440907375 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6094032be0 con 0x7f60a4108100 2024-09-16T00:18:54.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.869+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a419c0b0 con 0x7f60a4108100 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.869+0000 7f609b7fe640 1 -- 172.21.15.1:0/440907375 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6094037040 con 0x7f60a4108100 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.869+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a419eda0 con 0x7f60a4108100 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.870+0000 7f609b7fe640 1 -- 172.21.15.1:0/440907375 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f6094045070 con 0x7f60a4108100 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.870+0000 7f609b7fe640 1 --2- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f6090039160 0x7f609003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.870+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a4108560 con 0x7f60a4108100 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.870+0000 7f609b7fe640 1 -- 172.21.15.1:0/440907375 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f609406c1b0 con 0x7f60a4108100 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.870+0000 7f609bfff640 1 --2- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f6090039160 0x7f609003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.871+0000 7f609bfff640 1 --2- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f6090039160 0x7f609003b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f608c0098c0 tx=0x7f608c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:53.875+0000 7f609b7fe640 1 -- 172.21.15.1:0/440907375 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6094042210 con 0x7f60a4108100 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.049+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f60a4194540 con 0x7f60a4108100 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.050+0000 7f609b7fe640 1 -- 172.21.15.1:0/440907375 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7f6094032d90 con 0x7f60a4108100 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f6090039160 msgr2=0x7f609003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 --2- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f6090039160 0x7f609003b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f608c0098c0 tx=0x7f608c006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:54.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 msgr2=0x7f60a419b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:54.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 --2- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a419b8e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6094031000 tx=0x7f60940326e0 comp rx=0 tx=0).stop 2024-09-16T00:18:54.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 shutdown_connections 2024-09-16T00:18:54.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 --2- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f6090039160 0x7f609003b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:54.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 --2- 172.21.15.1:0/440907375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a4108100 0x7f60a419b8e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:54.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 >> 172.21.15.1:0/440907375 conn(0x7f60a4101d20 msgr2=0x7f60a4102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:54.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 shutdown_connections 2024-09-16T00:18:54.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:54.053+0000 7f60a8bb7640 1 -- 172.21.15.1:0/440907375 wait complete. 2024-09-16T00:18:54.687 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling cephadm module... 2024-09-16T00:18:55.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:55 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1528291704' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T00:18:55.812 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: ignoring --setuser ceph since I am not root 2024-09-16T00:18:55.812 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: ignoring --setgroup ceph since I am not root 2024-09-16T00:18:55.812 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:55.650+0000 7f0a1131d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:18:55.812 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:55.709+0000 7f0a1131d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:18:56.092 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:55.865+0000 7f0a1131d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:18:56.092 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:55.935+0000 7f0a1131d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:18:56.092 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:56.091+0000 7f0a1131d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:18:56.113 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.023+0000 7fcc7c04f640 1 Processor -- start 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.023+0000 7fcc7c04f640 1 -- start start 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.023+0000 7fcc7c04f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc741068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.023+0000 7fcc7c04f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74106ec0 con 0x7fcc741064f0 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.024+0000 7fcc79deb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc741068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.024+0000 7fcc79deb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc741068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37286/0 (socket says 172.21.15.1:37286) 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.024+0000 7fcc79deb640 1 -- 172.21.15.1:0/4187693764 learned_addr learned my addr 172.21.15.1:0/4187693764 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.024+0000 7fcc79deb640 1 -- 172.21.15.1:0/4187693764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc741076f0 con 0x7fcc741064f0 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.024+0000 7fcc79deb640 1 --2- 172.21.15.1:0/4187693764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc741068f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009820 tx=0x7fcc7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a254d51a55949b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.025+0000 7fcc78de9640 1 -- 172.21.15.1:0/4187693764 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc70037070 con 0x7fcc741064f0 2024-09-16T00:18:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.025+0000 7fcc78de9640 1 -- 172.21.15.1:0/4187693764 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc70032900 con 0x7fcc741064f0 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.025+0000 7fcc78de9640 1 -- 172.21.15.1:0/4187693764 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc70030850 con 0x7fcc741064f0 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.025+0000 7fcc7c04f640 1 -- 172.21.15.1:0/4187693764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 msgr2=0x7fcc741068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.025+0000 7fcc7c04f640 1 --2- 172.21.15.1:0/4187693764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc741068f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009820 tx=0x7fcc7002a270 comp rx=0 tx=0).stop 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.025+0000 7fcc7c04f640 1 -- 172.21.15.1:0/4187693764 shutdown_connections 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.025+0000 7fcc7c04f640 1 --2- 172.21.15.1:0/4187693764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc741068f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.025+0000 7fcc7c04f640 1 -- 172.21.15.1:0/4187693764 >> 172.21.15.1:0/4187693764 conn(0x7fcc74101d20 msgr2=0x7fcc74104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.026+0000 7fcc7c04f640 1 -- 172.21.15.1:0/4187693764 shutdown_connections 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.026+0000 7fcc7c04f640 1 -- 172.21.15.1:0/4187693764 wait complete. 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.026+0000 7fcc7c04f640 1 Processor -- start 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.026+0000 7fcc7c04f640 1 -- start start 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.026+0000 7fcc7c04f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc7419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.026+0000 7fcc7c04f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7410aaa0 con 0x7fcc741064f0 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.026+0000 7fcc79deb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc7419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.026+0000 7fcc79deb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc7419b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37294/0 (socket says 172.21.15.1:37294) 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.026+0000 7fcc79deb640 1 -- 172.21.15.1:0/1528291704 learned_addr learned my addr 172.21.15.1:0/1528291704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.027+0000 7fcc79deb640 1 -- 172.21.15.1:0/1528291704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7419be90 con 0x7fcc741064f0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.027+0000 7fcc79deb640 1 --2- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc7419b950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc700097f0 tx=0x7fcc70030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.027+0000 7fcc6affd640 1 -- 172.21.15.1:0/1528291704 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc70042070 con 0x7fcc741064f0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.027+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7419c120 con 0x7fcc741064f0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.027+0000 7fcc6affd640 1 -- 172.21.15.1:0/1528291704 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc70032ca0 con 0x7fcc741064f0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.027+0000 7fcc6affd640 1 -- 172.21.15.1:0/1528291704 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc70037040 con 0x7fcc741064f0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.027+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7419ee10 con 0x7fcc741064f0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.028+0000 7fcc6affd640 1 -- 172.21.15.1:0/1528291704 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fcc70045070 con 0x7fcc741064f0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.028+0000 7fcc6affd640 1 --2- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fcc60039160 0x7fcc6003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.028+0000 7fcc6affd640 1 -- 172.21.15.1:0/1528291704 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc7006c190 con 0x7fcc741064f0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.028+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7410ac20 con 0x7fcc741064f0 2024-09-16T00:18:56.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.028+0000 7fcc795ea640 1 --2- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fcc60039160 0x7fcc6003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.029+0000 7fcc795ea640 1 --2- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fcc60039160 0x7fcc6003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc640098c0 tx=0x7fcc64006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.033+0000 7fcc6affd640 1 -- 172.21.15.1:0/1528291704 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc7006bbb0 con 0x7fcc741064f0 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.254+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fcc74106970 con 0x7fcc741064f0 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.502+0000 7fcc6affd640 1 -- 172.21.15.1:0/1528291704 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fcc7003ccd0 con 0x7fcc741064f0 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.503+0000 7fcc6affd640 1 -- 172.21.15.1:0/1528291704 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fcc7003c7b0 con 0x7fcc741064f0 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fcc60039160 msgr2=0x7fcc6003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 --2- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fcc60039160 0x7fcc6003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc640098c0 tx=0x7fcc64006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 msgr2=0x7fcc7419b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 --2- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc7419b950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc700097f0 tx=0x7fcc70030ec0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 shutdown_connections 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 --2- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7fcc60039160 0x7fcc6003b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 --2- 172.21.15.1:0/1528291704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc741064f0 0x7fcc7419b950 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 >> 172.21.15.1:0/1528291704 conn(0x7fcc74101d20 msgr2=0x7fcc74102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:56.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 shutdown_connections 2024-09-16T00:18:56.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:55.505+0000 7fcc7c04f640 1 -- 172.21.15.1:0/1528291704 wait complete. 2024-09-16T00:18:56.371 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:56.247+0000 7f0a1131d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:18:56.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:56 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1528291704' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T00:18:56.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:56 smithi001 ceph-mon[30934]: mgrmap e5: x(active, since 6s) 2024-09-16T00:18:56.668 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:56.599+0000 7f0a1131d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:18:56.669 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:56.667+0000 7f0a1131d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:18:56.951 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:56.865+0000 7f0a1131d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:18:57.453 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:18:57.453 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T00:18:57.453 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.553+0000 7f332a64e640 1 Processor -- start 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.553+0000 7f332a64e640 1 -- start start 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.553+0000 7f332a64e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f332410a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.553+0000 7f332a64e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3324075880 con 0x7f3324108000 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.553+0000 7f3323fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f332410a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.553+0000 7f3323fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f332410a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37306/0 (socket says 172.21.15.1:37306) 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.553+0000 7f3323fff640 1 -- 172.21.15.1:0/2932026363 learned_addr learned my addr 172.21.15.1:0/2932026363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.554+0000 7f3323fff640 1 -- 172.21.15.1:0/2932026363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332410a930 con 0x7f3324108000 2024-09-16T00:18:57.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.554+0000 7f3323fff640 1 --2- 172.21.15.1:0/2932026363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f332410a3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3314009820 tx=0x7f331402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93eff9e29077f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.555+0000 7f3322ffd640 1 -- 172.21.15.1:0/2932026363 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3314037070 con 0x7f3324108000 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.555+0000 7f3322ffd640 1 -- 172.21.15.1:0/2932026363 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3314032900 con 0x7f3324108000 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.555+0000 7f3322ffd640 1 -- 172.21.15.1:0/2932026363 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3314030850 con 0x7f3324108000 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.555+0000 7f332a64e640 1 -- 172.21.15.1:0/2932026363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 msgr2=0x7f332410a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.555+0000 7f332a64e640 1 --2- 172.21.15.1:0/2932026363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f332410a3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3314009820 tx=0x7f331402a270 comp rx=0 tx=0).stop 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.555+0000 7f332a64e640 1 -- 172.21.15.1:0/2932026363 shutdown_connections 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.555+0000 7f332a64e640 1 --2- 172.21.15.1:0/2932026363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f332410a3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.555+0000 7f332a64e640 1 -- 172.21.15.1:0/2932026363 >> 172.21.15.1:0/2932026363 conn(0x7f3324101ca0 msgr2=0x7f33241040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.556+0000 7f332a64e640 1 -- 172.21.15.1:0/2932026363 shutdown_connections 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.556+0000 7f332a64e640 1 -- 172.21.15.1:0/2932026363 wait complete. 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.556+0000 7f332a64e640 1 Processor -- start 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.556+0000 7f332a64e640 1 -- start start 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f332a64e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f3324107880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f332a64e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f332410b9d0 con 0x7f3324108000 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f3323fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f3324107880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f3323fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f3324107880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37312/0 (socket says 172.21.15.1:37312) 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f3323fff640 1 -- 172.21.15.1:0/2550643222 learned_addr learned my addr 172.21.15.1:0/2550643222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f3323fff640 1 -- 172.21.15.1:0/2550643222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3324107dc0 con 0x7f3324108000 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f3323fff640 1 --2- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f3324107880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33140097f0 tx=0x7f3314030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f33217fa640 1 -- 172.21.15.1:0/2550643222 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3314037040 con 0x7f3324108000 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f33217fa640 1 -- 172.21.15.1:0/2550643222 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3314040070 con 0x7f3324108000 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3324105fb0 con 0x7f3324108000 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.557+0000 7f33217fa640 1 -- 172.21.15.1:0/2550643222 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f331403a390 con 0x7f3324108000 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.558+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3324106490 con 0x7f3324108000 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.558+0000 7f33217fa640 1 -- 172.21.15.1:0/2550643222 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f331403a630 con 0x7f3324108000 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.559+0000 7f33217fa640 1 --2- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f330c0391b0 0x7f330c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.559+0000 7f33217fa640 1 -- 172.21.15.1:0/2550643222 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f331406c540 con 0x7f3324108000 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.559+0000 7f33237fe640 1 -- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f330c0391b0 msgr2=0x7f330c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/786826717 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.559+0000 7f33237fe640 1 --2- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f330c0391b0 0x7f330c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.559+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3324108460 con 0x7f3324108000 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.564+0000 7f33217fa640 1 -- 172.21.15.1:0/2550643222 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f331404a200 con 0x7f3324108000 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.760+0000 7f33237fe640 1 -- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f330c0391b0 msgr2=0x7f330c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/786826717 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.760+0000 7f33237fe640 1 --2- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f330c0391b0 0x7f330c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.813+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3324106b90 con 0x7f3324108000 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.813+0000 7f33217fa640 1 -- 172.21.15.1:0/2550643222 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f331405e070 con 0x7f3324108000 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f330c0391b0 msgr2=0x7f330c03b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 --2- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f330c0391b0 0x7f330c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 msgr2=0x7f3324107880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:57.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 --2- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f3324107880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33140097f0 tx=0x7f3314030d80 comp rx=0 tx=0).stop 2024-09-16T00:18:57.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 shutdown_connections 2024-09-16T00:18:57.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 --2- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7f330c0391b0 0x7f330c03b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:57.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 --2- 172.21.15.1:0/2550643222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324108000 0x7f3324107880 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:57.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 >> 172.21.15.1:0/2550643222 conn(0x7f3324101ca0 msgr2=0x7f33241028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:57.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 shutdown_connections 2024-09-16T00:18:57.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:56.816+0000 7f332a64e640 1 -- 172.21.15.1:0/2550643222 wait complete. 2024-09-16T00:18:57.458 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-09-16T00:18:57.458 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 5... 2024-09-16T00:18:57.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:18:57 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2550643222' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:18:58.108 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:57 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:57.800+0000 7f0a1131d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:18:58.108 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:57 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:57.872+0000 7f0a1131d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:18:58.898 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:58 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:58.624+0000 7f0a1131d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:18:58.898 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:58 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:58.748+0000 7f0a1131d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:18:59.189 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:58 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:58.897+0000 7f0a1131d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:18:59.189 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:58 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:58.961+0000 7f0a1131d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:18:59.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:59.117+0000 7f0a1131d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:18:59.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:59.187+0000 7f0a1131d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:18:59.451 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:59.254+0000 7f0a1131d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:18:59.452 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:59.420+0000 7f0a1131d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:18:59.768 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:59.488+0000 7f0a1131d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:18:59.768 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:59.557+0000 7f0a1131d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:18:59.768 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:59.644+0000 7f0a1131d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:18:59.768 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:59.706+0000 7f0a1131d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:18:59.768 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:18:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:18:59.767+0000 7f0a1131d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:19:00.951 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:00.596+0000 7f0a1131d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:19:00.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: Active manager daemon x restarted 2024-09-16T00:19:00.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: Activating manager daemon x 2024-09-16T00:19:00.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T00:19:00.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: mgrmap e6: x(active, starting, since 0.00373806s) 2024-09-16T00:19:00.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:19:00.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:19:00.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:19:00.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:19:00.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:19:00.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: Manager daemon x is now available 2024-09-16T00:19:00.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:00.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:00.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:00.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:00.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:19:00.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:00 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:19:01.950 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:01 smithi001 ceph-mon[30934]: Found migration_current of "None". Setting to last migration. 2024-09-16T00:19:01.950 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:01 smithi001 ceph-mon[30934]: mgrmap e7: x(active, since 1.006s) 2024-09-16T00:19:02.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.901+0000 7effcacf0640 1 Processor -- start 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.901+0000 7effcacf0640 1 -- start start 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.901+0000 7effcacf0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc41068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.901+0000 7effcacf0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7effc4106e90 con 0x7effc41064c0 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.901+0000 7effc8a8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc41068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.901+0000 7effc8a8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc41068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37324/0 (socket says 172.21.15.1:37324) 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.901+0000 7effc8a8c640 1 -- 172.21.15.1:0/2234453142 learned_addr learned my addr 172.21.15.1:0/2234453142 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.902+0000 7effc8a8c640 1 -- 172.21.15.1:0/2234453142 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc41076c0 con 0x7effc41064c0 2024-09-16T00:19:02.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.903+0000 7effc8a8c640 1 --2- 172.21.15.1:0/2234453142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc41068c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7effb8009820 tx=0x7effb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea06f77136bd71e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:02.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.903+0000 7effc37fe640 1 -- 172.21.15.1:0/2234453142 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb8037070 con 0x7effc41064c0 2024-09-16T00:19:02.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.903+0000 7effc37fe640 1 -- 172.21.15.1:0/2234453142 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7effb8032900 con 0x7effc41064c0 2024-09-16T00:19:02.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.903+0000 7effc37fe640 1 -- 172.21.15.1:0/2234453142 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb8030850 con 0x7effc41064c0 2024-09-16T00:19:02.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.904+0000 7effcacf0640 1 -- 172.21.15.1:0/2234453142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 msgr2=0x7effc41068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:02.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.904+0000 7effcacf0640 1 --2- 172.21.15.1:0/2234453142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc41068c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7effb8009820 tx=0x7effb802a270 comp rx=0 tx=0).stop 2024-09-16T00:19:02.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.904+0000 7effcacf0640 1 -- 172.21.15.1:0/2234453142 shutdown_connections 2024-09-16T00:19:02.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.904+0000 7effcacf0640 1 --2- 172.21.15.1:0/2234453142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc41068c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.904+0000 7effcacf0640 1 -- 172.21.15.1:0/2234453142 >> 172.21.15.1:0/2234453142 conn(0x7effc4101c70 msgr2=0x7effc4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.904+0000 7effcacf0640 1 -- 172.21.15.1:0/2234453142 shutdown_connections 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.904+0000 7effcacf0640 1 -- 172.21.15.1:0/2234453142 wait complete. 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.904+0000 7effcacf0640 1 Processor -- start 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.905+0000 7effcacf0640 1 -- start start 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.905+0000 7effcacf0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.905+0000 7effcacf0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7effc410aa70 con 0x7effc41064c0 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.905+0000 7effc8a8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.905+0000 7effc8a8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc419b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37330/0 (socket says 172.21.15.1:37330) 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.905+0000 7effc8a8c640 1 -- 172.21.15.1:0/3773090971 learned_addr learned my addr 172.21.15.1:0/3773090971 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.905+0000 7effc8a8c640 1 -- 172.21.15.1:0/3773090971 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc419bec0 con 0x7effc41064c0 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.905+0000 7effc8a8c640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc419b980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7effb8009940 tx=0x7effb80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.906+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb8042070 con 0x7effc41064c0 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.906+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7effb8032ca0 con 0x7effc41064c0 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.906+0000 7effcacf0640 1 -- 172.21.15.1:0/3773090971 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc419c150 con 0x7effc41064c0 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.906+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb8037040 con 0x7effc41064c0 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.906+0000 7effcacf0640 1 -- 172.21.15.1:0/3773090971 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc419ee40 con 0x7effc41064c0 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.907+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7effb8045070 con 0x7effc41064c0 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.907+0000 7effc1ffb640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 0x7effb403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.907+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 --> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7effb403bd80 con 0x7effb40391b0 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.907+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7effb806c0e0 con 0x7effc41064c0 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.907+0000 7effc3fff640 1 -- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 msgr2=0x7effb403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/786826717 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:57.907+0000 7effc3fff640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 0x7effb403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:58.108+0000 7effc3fff640 1 -- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 msgr2=0x7effb403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/786826717 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:58.108+0000 7effc3fff640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 0x7effb403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:58.509+0000 7effc3fff640 1 -- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 msgr2=0x7effb403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/786826717 2024-09-16T00:19:02.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:58.509+0000 7effc3fff640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 0x7effb403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:19:02.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:59.310+0000 7effc3fff640 1 -- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 msgr2=0x7effb403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/786826717 2024-09-16T00:19:02.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:18:59.310+0000 7effc3fff640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 0x7effb403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:19:02.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:00.600+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7effb806ac60 con 0x7effc41064c0 2024-09-16T00:19:02.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:00.600+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 msgr2=0x7effb403b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:19:02.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:00.600+0000 7effc1ffb640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 0x7effb403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.601+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7effb803b330 con 0x7effc41064c0 2024-09-16T00:19:02.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.601+0000 7effc1ffb640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7effb403cae0 0x7effb403eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:02.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.601+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7effb803ee50 con 0x7effb403cae0 2024-09-16T00:19:02.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.602+0000 7effc3fff640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7effb403cae0 0x7effb403eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.603+0000 7effc3fff640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7effb403cae0 0x7effb403eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7effb0003980 tx=0x7effb0006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.603+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 <== mgr.14118 v2:172.21.15.1:6800/178572347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7effb803ee50 con 0x7effb403cae0 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.610+0000 7effcacf0640 1 -- 172.21.15.1:0/3773090971 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7effc4106940 con 0x7effb403cae0 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.610+0000 7effc1ffb640 1 -- 172.21.15.1:0/3773090971 <== mgr.14118 v2:172.21.15.1:6800/178572347 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7effc4106940 con 0x7effb403cae0 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 -- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7effb403cae0 msgr2=0x7effb403eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7effb403cae0 0x7effb403eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7effb0003980 tx=0x7effb0006e10 comp rx=0 tx=0).stop 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 -- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 msgr2=0x7effc419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc419b980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7effb8009940 tx=0x7effb80326e0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 -- 172.21.15.1:0/3773090971 shutdown_connections 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7effb403cae0 0x7effb403eed0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:6800/786826717,v1:172.21.15.1:6801/786826717] conn(0x7effb40391b0 0x7effb403b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 --2- 172.21.15.1:0/3773090971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effc41064c0 0x7effc419b980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 -- 172.21.15.1:0/3773090971 >> 172.21.15.1:0/3773090971 conn(0x7effc4101c70 msgr2=0x7effc4102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:02.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 -- 172.21.15.1:0/3773090971 shutdown_connections 2024-09-16T00:19:02.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:01.611+0000 7effcacf0640 1 -- 172.21.15.1:0/3773090971 wait complete. 2024-09-16T00:19:02.238 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 5 is available 2024-09-16T00:19:02.238 INFO:teuthology.orchestra.run.smithi001.stdout:Setting orchestrator backend to cephadm... 2024-09-16T00:19:02.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:02 smithi001 ceph-mon[30934]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T00:19:02.835 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:02 smithi001 ceph-mon[30934]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T00:19:02.835 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:02 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:02.835 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:02 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:03.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.633+0000 7ff5983ad640 1 Processor -- start 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.633+0000 7ff5983ad640 1 -- start start 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.633+0000 7ff5983ad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff5901079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.633+0000 7ff5983ad640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff590107fb0 con 0x7ff5901075e0 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.633+0000 7ff596149640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff5901079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.634+0000 7ff596149640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff5901079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40800/0 (socket says 172.21.15.1:40800) 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.634+0000 7ff596149640 1 -- 172.21.15.1:0/1114619297 learned_addr learned my addr 172.21.15.1:0/1114619297 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.634+0000 7ff596149640 1 -- 172.21.15.1:0/1114619297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff590108790 con 0x7ff5901075e0 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.634+0000 7ff596149640 1 --2- 172.21.15.1:0/1114619297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff5901079e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009820 tx=0x7ff58c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2b991471f177b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.635+0000 7ff595147640 1 -- 172.21.15.1:0/1114619297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff58c037070 con 0x7ff5901075e0 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.635+0000 7ff595147640 1 -- 172.21.15.1:0/1114619297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff58c032900 con 0x7ff5901075e0 2024-09-16T00:19:03.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.635+0000 7ff595147640 1 -- 172.21.15.1:0/1114619297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff58c030850 con 0x7ff5901075e0 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.635+0000 7ff5983ad640 1 -- 172.21.15.1:0/1114619297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 msgr2=0x7ff5901079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.635+0000 7ff5983ad640 1 --2- 172.21.15.1:0/1114619297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff5901079e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009820 tx=0x7ff58c02a270 comp rx=0 tx=0).stop 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.636+0000 7ff5983ad640 1 -- 172.21.15.1:0/1114619297 shutdown_connections 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.636+0000 7ff5983ad640 1 --2- 172.21.15.1:0/1114619297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff5901079e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.636+0000 7ff5983ad640 1 -- 172.21.15.1:0/1114619297 >> 172.21.15.1:0/1114619297 conn(0x7ff590102dd0 msgr2=0x7ff5901051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.636+0000 7ff5983ad640 1 -- 172.21.15.1:0/1114619297 shutdown_connections 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.636+0000 7ff5983ad640 1 -- 172.21.15.1:0/1114619297 wait complete. 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.636+0000 7ff5983ad640 1 Processor -- start 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.636+0000 7ff5983ad640 1 -- start start 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff5983ad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff59019b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff5983ad640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff59010bb40 con 0x7ff5901075e0 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff596149640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff59019b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff596149640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff59019b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40810/0 (socket says 172.21.15.1:40810) 2024-09-16T00:19:03.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff596149640 1 -- 172.21.15.1:0/1552261083 learned_addr learned my addr 172.21.15.1:0/1552261083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff596149640 1 -- 172.21.15.1:0/1552261083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59019bad0 con 0x7ff5901075e0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff596149640 1 --2- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff59019b590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009940 tx=0x7ff58c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff5877fe640 1 -- 172.21.15.1:0/1552261083 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff58c042070 con 0x7ff5901075e0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59019bd60 con 0x7ff5901075e0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff5877fe640 1 -- 172.21.15.1:0/1552261083 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff58c0308c0 con 0x7ff5901075e0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff5877fe640 1 -- 172.21.15.1:0/1552261083 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff58c037040 con 0x7ff5901075e0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.637+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59019ea50 con 0x7ff5901075e0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.638+0000 7ff5877fe640 1 -- 172.21.15.1:0/1552261083 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7ff58c045070 con 0x7ff5901075e0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.638+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff59010bcc0 con 0x7ff5901075e0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.639+0000 7ff5877fe640 1 --2- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff57c0390a0 0x7ff57c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.639+0000 7ff5877fe640 1 -- 172.21.15.1:0/1552261083 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff58c06c180 con 0x7ff5901075e0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.639+0000 7ff595948640 1 --2- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff57c0390a0 0x7ff57c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.639+0000 7ff595948640 1 --2- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff57c0390a0 0x7ff57c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5800098c0 tx=0x7ff580006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:03.609 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.644+0000 7ff5877fe640 1 -- 172.21.15.1:0/1552261083 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff58c045520 con 0x7ff5901075e0 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.834+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7ff590104090 con 0x7ff57c0390a0 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.839+0000 7ff5877fe640 1 -- 172.21.15.1:0/1552261083 <== mgr.14118 v2:172.21.15.1:6800/178572347 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff590104090 con 0x7ff57c0390a0 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff57c0390a0 msgr2=0x7ff57c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 --2- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff57c0390a0 0x7ff57c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5800098c0 tx=0x7ff580006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 msgr2=0x7ff59019b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 --2- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff59019b590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009940 tx=0x7ff58c032ce0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 shutdown_connections 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 --2- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff57c0390a0 0x7ff57c03b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 --2- 172.21.15.1:0/1552261083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5901075e0 0x7ff59019b590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.610 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 >> 172.21.15.1:0/1552261083 conn(0x7ff590102dd0 msgr2=0x7ff590103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:03.611 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 shutdown_connections 2024-09-16T00:19:03.611 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:02.842+0000 7ff5983ad640 1 -- 172.21.15.1:0/1552261083 wait complete. 2024-09-16T00:19:04.006 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-mon[30934]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:04.006 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:04.006 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:04.006 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-mon[30934]: mgrmap e8: x(active, since 2s) 2024-09-16T00:19:04.006 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:04.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:03.733+0000 7f097de42640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:00:19:03] ENGINE Error in HTTPServer.serve 2024-09-16T00:19:04.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: Traceback (most recent call last): 2024-09-16T00:19:04.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:19:04.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: self._connections.run(self.expiration_interval) 2024-09-16T00:19:04.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:19:04.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: self._run(expiration_interval) 2024-09-16T00:19:04.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:19:04.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: s = self.context.wrap_socket( 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: return self.sslsocket_class._create( 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: self.do_handshake() 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: self._sslobj.do_handshake() 2024-09-16T00:19:04.008 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:19:04.009 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:04.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.005+0000 7f4317a15640 1 Processor -- start 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.005+0000 7f4317a15640 1 -- start start 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.005+0000 7f4317a15640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f4310107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.005+0000 7f4317a15640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4310107f40 con 0x7f4310107600 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.005+0000 7f43157b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f4310107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.005+0000 7f43157b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f4310107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40820/0 (socket says 172.21.15.1:40820) 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.005+0000 7f43157b1640 1 -- 172.21.15.1:0/2128347670 learned_addr learned my addr 172.21.15.1:0/2128347670 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.006+0000 7f43157b1640 1 -- 172.21.15.1:0/2128347670 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43101080c0 con 0x7f4310107600 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.006+0000 7f43157b1640 1 --2- 172.21.15.1:0/2128347670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f4310107a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f430c009820 tx=0x7f430c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97460efc93edf14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4303fff640 1 -- 172.21.15.1:0/2128347670 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f430c037070 con 0x7f4310107600 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4303fff640 1 -- 172.21.15.1:0/2128347670 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f430c032900 con 0x7f4310107600 2024-09-16T00:19:04.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4303fff640 1 -- 172.21.15.1:0/2128347670 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f430c030850 con 0x7f4310107600 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4317a15640 1 -- 172.21.15.1:0/2128347670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 msgr2=0x7f4310107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4317a15640 1 --2- 172.21.15.1:0/2128347670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f4310107a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f430c009820 tx=0x7f430c02a270 comp rx=0 tx=0).stop 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4317a15640 1 -- 172.21.15.1:0/2128347670 shutdown_connections 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4317a15640 1 --2- 172.21.15.1:0/2128347670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f4310107a00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4317a15640 1 -- 172.21.15.1:0/2128347670 >> 172.21.15.1:0/2128347670 conn(0x7f4310102e50 msgr2=0x7f4310105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4317a15640 1 -- 172.21.15.1:0/2128347670 shutdown_connections 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.007+0000 7f4317a15640 1 -- 172.21.15.1:0/2128347670 wait complete. 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.008+0000 7f4317a15640 1 Processor -- start 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.008+0000 7f4317a15640 1 -- start start 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.008+0000 7f4317a15640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f431019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.008+0000 7f4317a15640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f431010ba70 con 0x7f4310107600 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.008+0000 7f43157b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f431019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.008+0000 7f43157b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f431019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40830/0 (socket says 172.21.15.1:40830) 2024-09-16T00:19:04.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.008+0000 7f43157b1640 1 -- 172.21.15.1:0/1383325226 learned_addr learned my addr 172.21.15.1:0/1383325226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.008+0000 7f43157b1640 1 -- 172.21.15.1:0/1383325226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431019baa0 con 0x7f4310107600 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.009+0000 7f43157b1640 1 --2- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f431019b560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f430c0097f0 tx=0x7f430c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.009+0000 7f43027fc640 1 -- 172.21.15.1:0/1383325226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f430c037040 con 0x7f4310107600 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.009+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431019bd30 con 0x7f4310107600 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.009+0000 7f43027fc640 1 -- 172.21.15.1:0/1383325226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f430c040070 con 0x7f4310107600 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.009+0000 7f43027fc640 1 -- 172.21.15.1:0/1383325226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f430c03a390 con 0x7f4310107600 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.009+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431019ea20 con 0x7f4310107600 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.010+0000 7f43027fc640 1 -- 172.21.15.1:0/1383325226 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f430c03a630 con 0x7f4310107600 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.010+0000 7f43027fc640 1 --2- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42fc0391b0 0x7f42fc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.010+0000 7f43027fc640 1 -- 172.21.15.1:0/1383325226 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f430c06c7a0 con 0x7f4310107600 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.010+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4310107a80 con 0x7f4310107600 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.010+0000 7f4314fb0640 1 --2- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42fc0391b0 0x7f42fc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:04.775 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.010+0000 7f4314fb0640 1 --2- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42fc0391b0 0x7f42fc03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f43040098c0 tx=0x7f4304006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.015+0000 7f43027fc640 1 -- 172.21.15.1:0/1383325226 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f430c04a200 con 0x7f4310107600 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.191+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f4310104090 con 0x7f42fc0391b0 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.192+0000 7f43027fc640 1 -- 172.21.15.1:0/1383325226 <== mgr.14118 v2:172.21.15.1:6800/178572347 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f4310104090 con 0x7f42fc0391b0 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42fc0391b0 msgr2=0x7f42fc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 --2- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42fc0391b0 0x7f42fc03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f43040098c0 tx=0x7f4304006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 msgr2=0x7f431019b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 --2- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f431019b560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f430c0097f0 tx=0x7f430c030d80 comp rx=0 tx=0).stop 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 shutdown_connections 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 --2- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42fc0391b0 0x7f42fc03b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 --2- 172.21.15.1:0/1383325226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4310107600 0x7f431019b560 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 >> 172.21.15.1:0/1383325226 conn(0x7f4310102e50 msgr2=0x7f4310103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 shutdown_connections 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:04.194+0000 7f4317a15640 1 -- 172.21.15.1:0/1383325226 wait complete. 2024-09-16T00:19:04.776 INFO:teuthology.orchestra.run.smithi001.stdout:Generating ssh key... 2024-09-16T00:19:05.049 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:04 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:03] ENGINE Bus STARTING 2024-09-16T00:19:05.049 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:04 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:03] ENGINE Serving on https://172.21.15.1:7150 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:04 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:03] ENGINE Error in HTTPServer.serve 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: Traceback (most recent call last): 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._run(expiration_interval) 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: s = self.context.wrap_socket( 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:19:05.050 INFO:journalctl@ceph.mon.a.smithi001.stdout: return self.sslsocket_class._create( 2024-09-16T00:19:05.051 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:19:05.051 INFO:journalctl@ceph.mon.a.smithi001.stdout: self.do_handshake() 2024-09-16T00:19:05.051 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:19:05.051 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._sslobj.do_handshake() 2024-09-16T00:19:05.051 INFO:journalctl@ceph.mon.a.smithi001.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:19:05.051 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:04 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:03] ENGINE Serving on http://172.21.15.1:8765 2024-09-16T00:19:05.051 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:04 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:03] ENGINE Bus STARTED 2024-09-16T00:19:05.051 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:04 smithi001 ceph-mon[30934]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: Generating public/private rsa key pair. 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: Your identification has been saved in /tmp/tmp32nbsei5/key 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: Your public key has been saved in /tmp/tmp32nbsei5/key.pub 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: The key fingerprint is: 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: SHA256:1TANpp7ByoOLsyvcIU5B5PsAsoGpGb5cErzaLDy3D6Y ceph-1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: The key's randomart image is: 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: +---[RSA 3072]----+ 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: |.. =o | 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: |+o . o +. | 2024-09-16T00:19:05.840 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: |O+ + . . | 2024-09-16T00:19:05.841 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: |=*+ o o + | 2024-09-16T00:19:05.841 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: |+*... + S | 2024-09-16T00:19:05.841 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: |+=*o . . | 2024-09-16T00:19:05.841 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: |*=B=o | 2024-09-16T00:19:05.841 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: |.+=+o | 2024-09-16T00:19:05.841 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: | Eoo.. | 2024-09-16T00:19:05.841 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: +----[SHA256]-----+ 2024-09-16T00:19:06.111 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-mon[30934]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:06.111 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-mon[30934]: Generating ssh key... 2024-09-16T00:19:06.111 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:06.111 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:05 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.135+0000 7f2d7ff09640 1 Processor -- start 2024-09-16T00:19:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.136+0000 7f2d7ff09640 1 -- start start 2024-09-16T00:19:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.136+0000 7f2d7ff09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d78107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.136+0000 7f2d7ff09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78107f60 con 0x7f2d78107620 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.136+0000 7f2d7dca5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d78107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.136+0000 7f2d7dca5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d78107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40842/0 (socket says 172.21.15.1:40842) 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.136+0000 7f2d7dca5640 1 -- 172.21.15.1:0/3812399416 learned_addr learned my addr 172.21.15.1:0/3812399416 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.136+0000 7f2d7dca5640 1 -- 172.21.15.1:0/3812399416 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d781080e0 con 0x7f2d78107620 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.137+0000 7f2d7dca5640 1 --2- 172.21.15.1:0/3812399416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d78107a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009e80 tx=0x7f2d7402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e7e34f6a5ccd55 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.137+0000 7f2d7cca3640 1 -- 172.21.15.1:0/3812399416 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d74036070 con 0x7f2d78107620 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.137+0000 7f2d7cca3640 1 -- 172.21.15.1:0/3812399416 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d740318c0 con 0x7f2d78107620 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.137+0000 7f2d7cca3640 1 -- 172.21.15.1:0/3812399416 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d74007ab0 con 0x7f2d78107620 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.138+0000 7f2d7ff09640 1 -- 172.21.15.1:0/3812399416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 msgr2=0x7f2d78107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.138+0000 7f2d7ff09640 1 --2- 172.21.15.1:0/3812399416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d78107a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009e80 tx=0x7f2d7402a7a0 comp rx=0 tx=0).stop 2024-09-16T00:19:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.138+0000 7f2d7ff09640 1 -- 172.21.15.1:0/3812399416 shutdown_connections 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.138+0000 7f2d7ff09640 1 --2- 172.21.15.1:0/3812399416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d78107a20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.138+0000 7f2d7ff09640 1 -- 172.21.15.1:0/3812399416 >> 172.21.15.1:0/3812399416 conn(0x7f2d78102e50 msgr2=0x7f2d78105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.138+0000 7f2d7ff09640 1 -- 172.21.15.1:0/3812399416 shutdown_connections 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.138+0000 7f2d7ff09640 1 -- 172.21.15.1:0/3812399416 wait complete. 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d7ff09640 1 Processor -- start 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d7ff09640 1 -- start start 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d7ff09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d7819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d7ff09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7810ba90 con 0x7f2d78107620 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d7dca5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d7819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d7dca5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d7819b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40848/0 (socket says 172.21.15.1:40848) 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d7dca5640 1 -- 172.21.15.1:0/38641696 learned_addr learned my addr 172.21.15.1:0/38641696 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d7dca5640 1 -- 172.21.15.1:0/38641696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7819bdd0 con 0x7f2d78107620 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d7dca5640 1 --2- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d7819b890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d7402acc0 tx=0x7f2d74031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:06.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.139+0000 7f2d6effd640 1 -- 172.21.15.1:0/38641696 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d74036050 con 0x7f2d78107620 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.140+0000 7f2d6effd640 1 -- 172.21.15.1:0/38641696 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d74037040 con 0x7f2d78107620 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.140+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7819c060 con 0x7f2d78107620 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.140+0000 7f2d6effd640 1 -- 172.21.15.1:0/38641696 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d7403d850 con 0x7f2d78107620 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.140+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7819ed50 con 0x7f2d78107620 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.141+0000 7f2d6effd640 1 -- 172.21.15.1:0/38641696 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f2d74044020 con 0x7f2d78107620 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.141+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d78107aa0 con 0x7f2d78107620 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.141+0000 7f2d6effd640 1 --2- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f2d64039200 0x7f2d6403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.141+0000 7f2d6effd640 1 -- 172.21.15.1:0/38641696 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d74042020 con 0x7f2d78107620 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.141+0000 7f2d7d4a4640 1 --2- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f2d64039200 0x7f2d6403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.141+0000 7f2d7d4a4640 1 --2- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f2d64039200 0x7f2d6403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2d680098c0 tx=0x7f2d68006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.146+0000 7f2d6effd640 1 -- 172.21.15.1:0/38641696 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d7404b020 con 0x7f2d78107620 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.327+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f2d781041c0 con 0x7f2d64039200 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.592+0000 7f2d6effd640 1 -- 172.21.15.1:0/38641696 <== mgr.14118 v2:172.21.15.1:6800/178572347 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2d781041c0 con 0x7f2d64039200 2024-09-16T00:19:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f2d64039200 msgr2=0x7f2d6403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 --2- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f2d64039200 0x7f2d6403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2d680098c0 tx=0x7f2d68006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 msgr2=0x7f2d7819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 --2- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d7819b890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d7402acc0 tx=0x7f2d74031ca0 comp rx=0 tx=0).stop 2024-09-16T00:19:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 shutdown_connections 2024-09-16T00:19:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 --2- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f2d64039200 0x7f2d6403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 --2- 172.21.15.1:0/38641696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d78107620 0x7f2d7819b890 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 >> 172.21.15.1:0/38641696 conn(0x7f2d78102e50 msgr2=0x7f2d78103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 shutdown_connections 2024-09-16T00:19:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:05.594+0000 7f2d7ff09640 1 -- 172.21.15.1:0/38641696 wait complete. 2024-09-16T00:19:07.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXKnCiNbBhTz86bZT9/jVY9mHQ8rixfQ9eeVkpuK1XexzgexIEPfLypnEQo7XXjCtDEjpWv+iEBMIf43kdyS1cdA297BANnNSh6XCB2/B6YWFz+fA/wr7GtrzXFOq1jrZ3uH2XhqOnUnCLklvFrHQAmzoPx5fhoBLf3snB8Zd+fZNuxvtEnFaD430PqqFq3He9MjdgRy8H/uYFZVrk3J9AbdLYBIgTSyqwkwOpb8+GEcso9lBaJPFupH8den0NmZnGAGzjKjavAvdwd5sCbXVUb6UbDzdNOIbyEhdGO84gN1DHc1uKfx2nT90TtPCyj9u+HRj2D917qpFm7PLV6agqGGEmMXgbcNxm4CjrZiyH+Hjlr6iEYxc/+V38NFIEX/H9fGsYnkUF1XlJnmSAi7yEC0gStovmTNphvLdhuHXjSo/ByIDhuToIRHqIg40SE5X8s1jN+WYiMVhUE+eQokO081+HuVGNI9Kc0fiAVa5/EM18SLMhmH0DoHIeoadCAQk= ceph-1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:19:07.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.503+0000 7ff37d0f6640 1 Processor -- start 2024-09-16T00:19:07.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.504+0000 7ff37d0f6640 1 -- start start 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.504+0000 7ff37d0f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37810a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.504+0000 7ff37d0f6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff37807bdb0 con 0x7ff378108100 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.504+0000 7ff376d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37810a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.504+0000 7ff376d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37810a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40852/0 (socket says 172.21.15.1:40852) 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.504+0000 7ff376d9d640 1 -- 172.21.15.1:0/1133378960 learned_addr learned my addr 172.21.15.1:0/1133378960 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.504+0000 7ff376d9d640 1 -- 172.21.15.1:0/1133378960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37810aa30 con 0x7ff378108100 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.505+0000 7ff376d9d640 1 --2- 172.21.15.1:0/1133378960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37810a4f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff368009820 tx=0x7ff36802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05d1402fc27c1e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.505+0000 7ff375d9b640 1 -- 172.21.15.1:0/1133378960 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff368037070 con 0x7ff378108100 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.505+0000 7ff375d9b640 1 -- 172.21.15.1:0/1133378960 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff368032900 con 0x7ff378108100 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.505+0000 7ff375d9b640 1 -- 172.21.15.1:0/1133378960 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff368030850 con 0x7ff378108100 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.506+0000 7ff37d0f6640 1 -- 172.21.15.1:0/1133378960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 msgr2=0x7ff37810a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.506+0000 7ff37d0f6640 1 --2- 172.21.15.1:0/1133378960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37810a4f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff368009820 tx=0x7ff36802a270 comp rx=0 tx=0).stop 2024-09-16T00:19:07.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.506+0000 7ff37d0f6640 1 -- 172.21.15.1:0/1133378960 shutdown_connections 2024-09-16T00:19:07.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.506+0000 7ff37d0f6640 1 --2- 172.21.15.1:0/1133378960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37810a4f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.506+0000 7ff37d0f6640 1 -- 172.21.15.1:0/1133378960 >> 172.21.15.1:0/1133378960 conn(0x7ff378101d20 msgr2=0x7ff378104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:07.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.506+0000 7ff37d0f6640 1 -- 172.21.15.1:0/1133378960 shutdown_connections 2024-09-16T00:19:07.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.506+0000 7ff37d0f6640 1 -- 172.21.15.1:0/1133378960 wait complete. 2024-09-16T00:19:07.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.507+0000 7ff37d0f6640 1 Processor -- start 2024-09-16T00:19:07.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.507+0000 7ff37d0f6640 1 -- start start 2024-09-16T00:19:07.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.507+0000 7ff37d0f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.507+0000 7ff37d0f6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff37810ba10 con 0x7ff378108100 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.507+0000 7ff376d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.507+0000 7ff376d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37819b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40860/0 (socket says 172.21.15.1:40860) 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.507+0000 7ff376d9d640 1 -- 172.21.15.1:0/3998562443 learned_addr learned my addr 172.21.15.1:0/3998562443 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.507+0000 7ff376d9d640 1 -- 172.21.15.1:0/3998562443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37819be20 con 0x7ff378108100 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.508+0000 7ff376d9d640 1 --2- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37819b8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff368031000 tx=0x7ff3680326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.508+0000 7ff35ffff640 1 -- 172.21.15.1:0/3998562443 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff368042070 con 0x7ff378108100 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.508+0000 7ff35ffff640 1 -- 172.21.15.1:0/3998562443 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff368032be0 con 0x7ff378108100 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.508+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37819c0b0 con 0x7ff378108100 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.508+0000 7ff35ffff640 1 -- 172.21.15.1:0/3998562443 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff368037040 con 0x7ff378108100 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.508+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37819eda0 con 0x7ff378108100 2024-09-16T00:19:07.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.509+0000 7ff35ffff640 1 -- 172.21.15.1:0/3998562443 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ff368045070 con 0x7ff378108100 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.509+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff378108560 con 0x7ff378108100 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.509+0000 7ff35ffff640 1 --2- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff3600391b0 0x7ff36003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.509+0000 7ff35ffff640 1 -- 172.21.15.1:0/3998562443 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff36806c2c0 con 0x7ff378108100 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.509+0000 7ff37659c640 1 --2- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff3600391b0 0x7ff36003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.509+0000 7ff37659c640 1 --2- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff3600391b0 0x7ff36003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0098c0 tx=0x7ff36c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.514+0000 7ff35ffff640 1 -- 172.21.15.1:0/3998562443 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff368042210 con 0x7ff378108100 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.690+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7ff378062630 con 0x7ff3600391b0 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.691+0000 7ff35ffff640 1 -- 172.21.15.1:0/3998562443 <== mgr.14118 v2:172.21.15.1:6800/178572347 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7ff378062630 con 0x7ff3600391b0 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.693+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff3600391b0 msgr2=0x7ff36003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.693+0000 7ff37d0f6640 1 --2- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff3600391b0 0x7ff36003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0098c0 tx=0x7ff36c006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.693+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 msgr2=0x7ff37819b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.693+0000 7ff37d0f6640 1 --2- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37819b8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff368031000 tx=0x7ff3680326e0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.693+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 shutdown_connections 2024-09-16T00:19:07.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.693+0000 7ff37d0f6640 1 --2- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7ff3600391b0 0x7ff36003b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.693+0000 7ff37d0f6640 1 --2- 172.21.15.1:0/3998562443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff378108100 0x7ff37819b8e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.693+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 >> 172.21.15.1:0/3998562443 conn(0x7ff378101d20 msgr2=0x7ff378102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:07.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.693+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 shutdown_connections 2024-09-16T00:19:07.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:06.694+0000 7ff37d0f6640 1 -- 172.21.15.1:0/3998562443 wait complete. 2024-09-16T00:19:07.256 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:19:07.256 INFO:teuthology.orchestra.run.smithi001.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T00:19:07.256 INFO:teuthology.orchestra.run.smithi001.stdout:Adding host smithi001... 2024-09-16T00:19:07.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:07 smithi001 ceph-mon[30934]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:08.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:08 smithi001 ceph-mon[30934]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:09.790 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:09 smithi001 ceph-mon[30934]: Deploying cephadm binary to smithi001 2024-09-16T00:19:12.307 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Added host 'smithi001' with addr '172.21.15.1' 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.672+0000 7f0426c21640 1 Processor -- start 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.672+0000 7f0426c21640 1 -- start start 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.672+0000 7f0426c21640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042010a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.672+0000 7f0426c21640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f042007be10 con 0x7f04201080a0 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.672+0000 7f04249bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042010a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.672+0000 7f04249bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042010a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40866/0 (socket says 172.21.15.1:40866) 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.672+0000 7f04249bd640 1 -- 172.21.15.1:0/3081223766 learned_addr learned my addr 172.21.15.1:0/3081223766 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.672+0000 7f04249bd640 1 -- 172.21.15.1:0/3081223766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042010a9d0 con 0x7f04201080a0 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.673+0000 7f04249bd640 1 --2- 172.21.15.1:0/3081223766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042010a490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0408009820 tx=0x7f040802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e91c681eedd360 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:12.308 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.673+0000 7f041f7fe640 1 -- 172.21.15.1:0/3081223766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408037070 con 0x7f04201080a0 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.673+0000 7f041f7fe640 1 -- 172.21.15.1:0/3081223766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0408032900 con 0x7f04201080a0 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.673+0000 7f041f7fe640 1 -- 172.21.15.1:0/3081223766 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408030850 con 0x7f04201080a0 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.674+0000 7f0426c21640 1 -- 172.21.15.1:0/3081223766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 msgr2=0x7f042010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.674+0000 7f0426c21640 1 --2- 172.21.15.1:0/3081223766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042010a490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0408009820 tx=0x7f040802a270 comp rx=0 tx=0).stop 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.674+0000 7f0426c21640 1 -- 172.21.15.1:0/3081223766 shutdown_connections 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.674+0000 7f0426c21640 1 --2- 172.21.15.1:0/3081223766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042010a490 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.674+0000 7f0426c21640 1 -- 172.21.15.1:0/3081223766 >> 172.21.15.1:0/3081223766 conn(0x7f0420101d20 msgr2=0x7f0420104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.674+0000 7f0426c21640 1 -- 172.21.15.1:0/3081223766 shutdown_connections 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.674+0000 7f0426c21640 1 -- 172.21.15.1:0/3081223766 wait complete. 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f0426c21640 1 Processor -- start 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f0426c21640 1 -- start start 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f0426c21640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:12.309 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f0426c21640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f042010ba10 con 0x7f04201080a0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f04249bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f04249bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042019b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40874/0 (socket says 172.21.15.1:40874) 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f04249bd640 1 -- 172.21.15.1:0/2281139656 learned_addr learned my addr 172.21.15.1:0/2281139656 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f04249bd640 1 -- 172.21.15.1:0/2281139656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042019be20 con 0x7f04201080a0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f04249bd640 1 --2- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042019b8e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0408031000 tx=0x7f04080326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.675+0000 7f041dffb640 1 -- 172.21.15.1:0/2281139656 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408042070 con 0x7f04201080a0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.676+0000 7f041dffb640 1 -- 172.21.15.1:0/2281139656 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0408032be0 con 0x7f04201080a0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.676+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042019c0b0 con 0x7f04201080a0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.676+0000 7f041dffb640 1 -- 172.21.15.1:0/2281139656 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408037040 con 0x7f04201080a0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.676+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042019eda0 con 0x7f04201080a0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.677+0000 7f041dffb640 1 -- 172.21.15.1:0/2281139656 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0408045070 con 0x7f04201080a0 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.677+0000 7f041dffb640 1 --2- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f040c0391b0 0x7f040c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:12.310 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.677+0000 7f041dffb640 1 -- 172.21.15.1:0/2281139656 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f040806c2f0 con 0x7f04201080a0 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.677+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0420108500 con 0x7f04201080a0 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.677+0000 7f041ffff640 1 --2- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f040c0391b0 0x7f040c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.677+0000 7f041ffff640 1 --2- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f040c0391b0 0x7f040c03b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f041000ad30 tx=0x7f04100093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.682+0000 7f041dffb640 1 -- 172.21.15.1:0/2281139656 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0408042210 con 0x7f04201080a0 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:07.860+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}) -- 0x7f0420062630 con 0x7f040c0391b0 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.711+0000 7f041dffb640 1 -- 172.21.15.1:0/2281139656 <== mgr.14118 v2:172.21.15.1:6800/178572347 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f0420062630 con 0x7f040c0391b0 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f040c0391b0 msgr2=0x7f040c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 --2- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f040c0391b0 0x7f040c03b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f041000ad30 tx=0x7f04100093f0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 msgr2=0x7f042019b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 --2- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042019b8e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0408031000 tx=0x7f04080326e0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 shutdown_connections 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 --2- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f040c0391b0 0x7f040c03b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.311 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 --2- 172.21.15.1:0/2281139656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201080a0 0x7f042019b8e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.312 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 >> 172.21.15.1:0/2281139656 conn(0x7f0420101d20 msgr2=0x7f0420102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:12.312 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 shutdown_connections 2024-09-16T00:19:12.312 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:11.714+0000 7f0426c21640 1 -- 172.21.15.1:0/2281139656 wait complete. 2024-09-16T00:19:12.312 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mon service... 2024-09-16T00:19:12.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:12 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:12.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:12 smithi001 ceph-mon[30934]: Added host smithi001 2024-09-16T00:19:12.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:12 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:13.704 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T00:19:13.704 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.740+0000 7f16b7c73640 1 Processor -- start 2024-09-16T00:19:13.704 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.740+0000 7f16b7c73640 1 -- start start 2024-09-16T00:19:13.704 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.740+0000 7f16b7c73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b0107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.704 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.740+0000 7f16b7c73640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0107f60 con 0x7f16b0107620 2024-09-16T00:19:13.704 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.740+0000 7f16b5a0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b0107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.740+0000 7f16b5a0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b0107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53896/0 (socket says 172.21.15.1:53896) 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.740+0000 7f16b5a0f640 1 -- 172.21.15.1:0/3271651049 learned_addr learned my addr 172.21.15.1:0/3271651049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.741+0000 7f16b5a0f640 1 -- 172.21.15.1:0/3271651049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b01080e0 con 0x7f16b0107620 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.741+0000 7f16b5a0f640 1 --2- 172.21.15.1:0/3271651049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b0107a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009820 tx=0x7f16ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fde97db1e2bdcd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.741+0000 7f16b4a0d640 1 -- 172.21.15.1:0/3271651049 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac037070 con 0x7f16b0107620 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.741+0000 7f16b4a0d640 1 -- 172.21.15.1:0/3271651049 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f16ac032900 con 0x7f16b0107620 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.741+0000 7f16b4a0d640 1 -- 172.21.15.1:0/3271651049 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac030850 con 0x7f16b0107620 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.742+0000 7f16b7c73640 1 -- 172.21.15.1:0/3271651049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 msgr2=0x7f16b0107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.742+0000 7f16b7c73640 1 --2- 172.21.15.1:0/3271651049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b0107a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009820 tx=0x7f16ac02a270 comp rx=0 tx=0).stop 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.742+0000 7f16b7c73640 1 -- 172.21.15.1:0/3271651049 shutdown_connections 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.742+0000 7f16b7c73640 1 --2- 172.21.15.1:0/3271651049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b0107a20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.705 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.742+0000 7f16b7c73640 1 -- 172.21.15.1:0/3271651049 >> 172.21.15.1:0/3271651049 conn(0x7f16b0102e50 msgr2=0x7f16b0105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.742+0000 7f16b7c73640 1 -- 172.21.15.1:0/3271651049 shutdown_connections 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.742+0000 7f16b7c73640 1 -- 172.21.15.1:0/3271651049 wait complete. 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.743+0000 7f16b7c73640 1 Processor -- start 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.743+0000 7f16b7c73640 1 -- start start 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.743+0000 7f16b7c73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.743+0000 7f16b7c73640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16b010ba90 con 0x7f16b0107620 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.743+0000 7f16b5a0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.743+0000 7f16b5a0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b019b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53912/0 (socket says 172.21.15.1:53912) 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.743+0000 7f16b5a0f640 1 -- 172.21.15.1:0/1026825317 learned_addr learned my addr 172.21.15.1:0/1026825317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.743+0000 7f16b5a0f640 1 -- 172.21.15.1:0/1026825317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b019be10 con 0x7f16b0107620 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.744+0000 7f16b5a0f640 1 --2- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b019b8d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f16ac032ce0 tx=0x7f16ac032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.744+0000 7f16a6ffd640 1 -- 172.21.15.1:0/1026825317 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac042070 con 0x7f16b0107620 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.744+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b019c0a0 con 0x7f16b0107620 2024-09-16T00:19:13.706 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.744+0000 7f16a6ffd640 1 -- 172.21.15.1:0/1026825317 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f16ac0308c0 con 0x7f16b0107620 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.744+0000 7f16a6ffd640 1 -- 172.21.15.1:0/1026825317 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac037040 con 0x7f16b0107620 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.744+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b019ed90 con 0x7f16b0107620 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.745+0000 7f16a6ffd640 1 -- 172.21.15.1:0/1026825317 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f16ac045070 con 0x7f16b0107620 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.745+0000 7f16a6ffd640 1 --2- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f169c0391b0 0x7f169c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.745+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b0107aa0 con 0x7f16b0107620 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.745+0000 7f16b520e640 1 --2- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f169c0391b0 0x7f169c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.745+0000 7f16a6ffd640 1 -- 172.21.15.1:0/1026825317 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f16ac06c4c0 con 0x7f16b0107620 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.746+0000 7f16b520e640 1 --2- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f169c0391b0 0x7f169c03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f16a00098c0 tx=0x7f16a0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.750+0000 7f16a6ffd640 1 -- 172.21.15.1:0/1026825317 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16ac007040 con 0x7f16b0107620 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.964+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f16b0062630 con 0x7f169c0391b0 2024-09-16T00:19:13.707 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.969+0000 7f16a6ffd640 1 -- 172.21.15.1:0/1026825317 <== mgr.14118 v2:172.21.15.1:6800/178572347 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f16b0062630 con 0x7f169c0391b0 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f169c0391b0 msgr2=0x7f169c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 --2- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f169c0391b0 0x7f169c03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f16a00098c0 tx=0x7f16a0006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 msgr2=0x7f16b019b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 --2- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b019b8d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f16ac032ce0 tx=0x7f16ac032d10 comp rx=0 tx=0).stop 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 shutdown_connections 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 --2- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f169c0391b0 0x7f169c03b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 --2- 172.21.15.1:0/1026825317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0107620 0x7f16b019b8d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 >> 172.21.15.1:0/1026825317 conn(0x7f16b0102e50 msgr2=0x7f16b0103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 shutdown_connections 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:12.971+0000 7f16b7c73640 1 -- 172.21.15.1:0/1026825317 wait complete. 2024-09-16T00:19:13.708 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mgr service... 2024-09-16T00:19:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:13 smithi001 ceph-mon[30934]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:13 smithi001 ceph-mon[30934]: Saving service mon spec with placement count:5 2024-09-16T00:19:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:13 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:15.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.145+0000 7f4d0962b640 1 Processor -- start 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.146+0000 7f4d0962b640 1 -- start start 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.146+0000 7f4d0962b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d04107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.146+0000 7f4d0962b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4d04107f40 con 0x7f4d04107600 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.146+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d04107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.146+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d04107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53924/0 (socket says 172.21.15.1:53924) 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.146+0000 7f4d02ffd640 1 -- 172.21.15.1:0/1430988822 learned_addr learned my addr 172.21.15.1:0/1430988822 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.146+0000 7f4d02ffd640 1 -- 172.21.15.1:0/1430988822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d041080c0 con 0x7f4d04107600 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.147+0000 7f4d02ffd640 1 --2- 172.21.15.1:0/1430988822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d04107a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009820 tx=0x7f4cf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885effe9086c8bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:15.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.147+0000 7f4d01ffb640 1 -- 172.21.15.1:0/1430988822 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4037070 con 0x7f4d04107600 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.147+0000 7f4d01ffb640 1 -- 172.21.15.1:0/1430988822 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4cf4032900 con 0x7f4d04107600 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.147+0000 7f4d01ffb640 1 -- 172.21.15.1:0/1430988822 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4030850 con 0x7f4d04107600 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.148+0000 7f4d0962b640 1 -- 172.21.15.1:0/1430988822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 msgr2=0x7f4d04107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.148+0000 7f4d0962b640 1 --2- 172.21.15.1:0/1430988822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d04107a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009820 tx=0x7f4cf402a270 comp rx=0 tx=0).stop 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.148+0000 7f4d0962b640 1 -- 172.21.15.1:0/1430988822 shutdown_connections 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.148+0000 7f4d0962b640 1 --2- 172.21.15.1:0/1430988822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d04107a00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.148+0000 7f4d0962b640 1 -- 172.21.15.1:0/1430988822 >> 172.21.15.1:0/1430988822 conn(0x7f4d04102e50 msgr2=0x7f4d04105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.148+0000 7f4d0962b640 1 -- 172.21.15.1:0/1430988822 shutdown_connections 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.148+0000 7f4d0962b640 1 -- 172.21.15.1:0/1430988822 wait complete. 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.149+0000 7f4d0962b640 1 Processor -- start 2024-09-16T00:19:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.149+0000 7f4d0962b640 1 -- start start 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.149+0000 7f4d0962b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d0419b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.149+0000 7f4d0962b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0410ba70 con 0x7f4d04107600 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.149+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d0419b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.149+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d0419b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53934/0 (socket says 172.21.15.1:53934) 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.149+0000 7f4d02ffd640 1 -- 172.21.15.1:0/3469396845 learned_addr learned my addr 172.21.15.1:0/3469396845 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.149+0000 7f4d02ffd640 1 -- 172.21.15.1:0/3469396845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0419bad0 con 0x7f4d04107600 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.150+0000 7f4d02ffd640 1 --2- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d0419b590 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4032ce0 tx=0x7f4cf4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.150+0000 7f4cfbfff640 1 -- 172.21.15.1:0/3469396845 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4042070 con 0x7f4d04107600 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.150+0000 7f4cfbfff640 1 -- 172.21.15.1:0/3469396845 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4cf40308c0 con 0x7f4d04107600 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.150+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0419bd60 con 0x7f4d04107600 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.150+0000 7f4cfbfff640 1 -- 172.21.15.1:0/3469396845 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4037040 con 0x7f4d04107600 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.150+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0419ea50 con 0x7f4d04107600 2024-09-16T00:19:15.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.151+0000 7f4cfbfff640 1 -- 172.21.15.1:0/3469396845 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4cf4045070 con 0x7f4d04107600 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.151+0000 7f4cfbfff640 1 --2- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f4cf0039200 0x7f4cf003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.151+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d04107a80 con 0x7f4d04107600 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.151+0000 7f4cfbfff640 1 -- 172.21.15.1:0/3469396845 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4cf406c2c0 con 0x7f4d04107600 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.151+0000 7f4d027fc640 1 --2- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f4cf0039200 0x7f4cf003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.152+0000 7f4d027fc640 1 --2- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f4cf0039200 0x7f4cf003b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0098c0 tx=0x7f4cec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.156+0000 7f4cfbfff640 1 -- 172.21.15.1:0/3469396845 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4cf404b170 con 0x7f4d04107600 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.348+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4d04104070 con 0x7f4cf0039200 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.352+0000 7f4cfbfff640 1 -- 172.21.15.1:0/3469396845 <== mgr.14118 v2:172.21.15.1:6800/178572347 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4d04104070 con 0x7f4cf0039200 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f4cf0039200 msgr2=0x7f4cf003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 --2- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f4cf0039200 0x7f4cf003b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0098c0 tx=0x7f4cec006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 msgr2=0x7f4d0419b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 --2- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d0419b590 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4032ce0 tx=0x7f4cf4032d10 comp rx=0 tx=0).stop 2024-09-16T00:19:15.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 shutdown_connections 2024-09-16T00:19:15.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 --2- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f4cf0039200 0x7f4cf003b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:15.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 --2- 172.21.15.1:0/3469396845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d04107600 0x7f4d0419b590 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:15.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 >> 172.21.15.1:0/3469396845 conn(0x7f4d04102e50 msgr2=0x7f4d04103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:15.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 shutdown_connections 2024-09-16T00:19:15.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:14.354+0000 7f4d0962b640 1 -- 172.21.15.1:0/3469396845 wait complete. 2024-09-16T00:19:15.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:15 smithi001 ceph-mon[30934]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:15.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:15 smithi001 ceph-mon[30934]: Saving service mgr spec with placement count:2 2024-09-16T00:19:15.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:15 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:16.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:16 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/272760206' entity='client.admin' 2024-09-16T00:19:16.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:16 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:16.879 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.422+0000 7f9e053ed640 1 Processor -- start 2024-09-16T00:19:16.879 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.422+0000 7f9e053ed640 1 -- start start 2024-09-16T00:19:16.879 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.423+0000 7f9e053ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e00107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.423+0000 7f9e053ed640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e00107f60 con 0x7f9e00107620 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.423+0000 7f9dfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e00107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.423+0000 7f9dfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e00107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53942/0 (socket says 172.21.15.1:53942) 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.423+0000 7f9dfeffd640 1 -- 172.21.15.1:0/4182969800 learned_addr learned my addr 172.21.15.1:0/4182969800 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.423+0000 7f9dfeffd640 1 -- 172.21.15.1:0/4182969800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e001080e0 con 0x7f9e00107620 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.424+0000 7f9dfeffd640 1 --2- 172.21.15.1:0/4182969800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e00107a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009e80 tx=0x7f9df002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621c20c2c0e6ac9a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.424+0000 7f9dfdffb640 1 -- 172.21.15.1:0/4182969800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df0036070 con 0x7f9e00107620 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.424+0000 7f9dfdffb640 1 -- 172.21.15.1:0/4182969800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9df00318c0 con 0x7f9e00107620 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.424+0000 7f9dfdffb640 1 -- 172.21.15.1:0/4182969800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df0007ab0 con 0x7f9e00107620 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.425+0000 7f9e053ed640 1 -- 172.21.15.1:0/4182969800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 msgr2=0x7f9e00107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.425+0000 7f9e053ed640 1 --2- 172.21.15.1:0/4182969800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e00107a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009e80 tx=0x7f9df002a7a0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.425+0000 7f9e053ed640 1 -- 172.21.15.1:0/4182969800 shutdown_connections 2024-09-16T00:19:16.880 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.425+0000 7f9e053ed640 1 --2- 172.21.15.1:0/4182969800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e00107a20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.425+0000 7f9e053ed640 1 -- 172.21.15.1:0/4182969800 >> 172.21.15.1:0/4182969800 conn(0x7f9e00102e50 msgr2=0x7f9e00105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.425+0000 7f9e053ed640 1 -- 172.21.15.1:0/4182969800 shutdown_connections 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.425+0000 7f9e053ed640 1 -- 172.21.15.1:0/4182969800 wait complete. 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.425+0000 7f9e053ed640 1 Processor -- start 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.425+0000 7f9e053ed640 1 -- start start 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9e053ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e0019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9e053ed640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0010ba90 con 0x7f9e00107620 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9dfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e0019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9dfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e0019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53958/0 (socket says 172.21.15.1:53958) 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9dfeffd640 1 -- 172.21.15.1:0/272760206 learned_addr learned my addr 172.21.15.1:0/272760206 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9dfeffd640 1 -- 172.21.15.1:0/272760206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0019bdd0 con 0x7f9e00107620 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9dfeffd640 1 --2- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e0019b890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9df0031ca0 tx=0x7f9df0031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9df7fff640 1 -- 172.21.15.1:0/272760206 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df0036050 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9df7fff640 1 -- 172.21.15.1:0/272760206 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9df0042070 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0019c060 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.426+0000 7f9df7fff640 1 -- 172.21.15.1:0/272760206 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df003d820 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.427+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0019ed50 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.427+0000 7f9df7fff640 1 -- 172.21.15.1:0/272760206 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9df003b610 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.428+0000 7f9df7fff640 1 --2- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f9dec039200 0x7f9dec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.428+0000 7f9df7fff640 1 -- 172.21.15.1:0/272760206 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9df0007e00 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.428+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e00107aa0 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.428+0000 7f9dfe7fc640 1 --2- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f9dec039200 0x7f9dec03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.428+0000 7f9dfe7fc640 1 --2- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f9dec039200 0x7f9dec03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9de80098c0 tx=0x7f9de8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.432+0000 7f9df7fff640 1 -- 172.21.15.1:0/272760206 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9df004c200 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.609+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f9e0019eff0 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.612+0000 7f9df7fff640 1 -- 172.21.15.1:0/272760206 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f9df006cf40 con 0x7f9e00107620 2024-09-16T00:19:16.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f9dec039200 msgr2=0x7f9dec03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 --2- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f9dec039200 0x7f9dec03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9de80098c0 tx=0x7f9de8006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 msgr2=0x7f9e0019b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 --2- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e0019b890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9df0031ca0 tx=0x7f9df0031cd0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 shutdown_connections 2024-09-16T00:19:16.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 --2- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f9dec039200 0x7f9dec03b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 --2- 172.21.15.1:0/272760206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e00107620 0x7f9e0019b890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 >> 172.21.15.1:0/272760206 conn(0x7f9e00102e50 msgr2=0x7f9e00103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:16.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 shutdown_connections 2024-09-16T00:19:16.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:15.615+0000 7f9e053ed640 1 -- 172.21.15.1:0/272760206 wait complete. 2024-09-16T00:19:18.156 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:17 smithi001 ceph-mon[30934]: from='mgr.14118 172.21.15.1:0/37599183' entity='mgr.x' 2024-09-16T00:19:18.156 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:17 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1759743463' entity='client.admin' 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.301+0000 7f42dc202640 1 Processor -- start 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.301+0000 7f42dc202640 1 -- start start 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.301+0000 7f42dc202640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d4106890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.302+0000 7f42dc202640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4106e60 con 0x7f42d4106490 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.302+0000 7f42d9f9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d4106890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.302+0000 7f42d9f9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d4106890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53972/0 (socket says 172.21.15.1:53972) 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.302+0000 7f42d9f9e640 1 -- 172.21.15.1:0/4027621259 learned_addr learned my addr 172.21.15.1:0/4027621259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.302+0000 7f42d9f9e640 1 -- 172.21.15.1:0/4027621259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d4107690 con 0x7f42d4106490 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.303+0000 7f42d9f9e640 1 --2- 172.21.15.1:0/4027621259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d4106890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009820 tx=0x7f42d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd1886a6363b250 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.303+0000 7f42d8f9c640 1 -- 172.21.15.1:0/4027621259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0037070 con 0x7f42d4106490 2024-09-16T00:19:18.463 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.304+0000 7f42d8f9c640 1 -- 172.21.15.1:0/4027621259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d0032920 con 0x7f42d4106490 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.304+0000 7f42d8f9c640 1 -- 172.21.15.1:0/4027621259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0030810 con 0x7f42d4106490 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.304+0000 7f42dc202640 1 -- 172.21.15.1:0/4027621259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 msgr2=0x7f42d4106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.304+0000 7f42dc202640 1 --2- 172.21.15.1:0/4027621259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d4106890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009820 tx=0x7f42d002a270 comp rx=0 tx=0).stop 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.304+0000 7f42dc202640 1 -- 172.21.15.1:0/4027621259 shutdown_connections 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.304+0000 7f42dc202640 1 --2- 172.21.15.1:0/4027621259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d4106890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.305+0000 7f42dc202640 1 -- 172.21.15.1:0/4027621259 >> 172.21.15.1:0/4027621259 conn(0x7f42d4101ca0 msgr2=0x7f42d41040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.305+0000 7f42dc202640 1 -- 172.21.15.1:0/4027621259 shutdown_connections 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.305+0000 7f42dc202640 1 -- 172.21.15.1:0/4027621259 wait complete. 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.306+0000 7f42dc202640 1 Processor -- start 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.306+0000 7f42dc202640 1 -- start start 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.306+0000 7f42dc202640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.306+0000 7f42dc202640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4074b20 con 0x7f42d4106490 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.306+0000 7f42d9f9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.306+0000 7f42d9f9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d419b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53988/0 (socket says 172.21.15.1:53988) 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.306+0000 7f42d9f9e640 1 -- 172.21.15.1:0/1759743463 learned_addr learned my addr 172.21.15.1:0/1759743463 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.306+0000 7f42d9f9e640 1 -- 172.21.15.1:0/1759743463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d419bb60 con 0x7f42d4106490 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.306+0000 7f42d9f9e640 1 --2- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d419b620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42d00097f0 tx=0x7f42d00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.307+0000 7f42caffd640 1 -- 172.21.15.1:0/1759743463 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0037040 con 0x7f42d4106490 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.307+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d419bdf0 con 0x7f42d4106490 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.307+0000 7f42caffd640 1 -- 172.21.15.1:0/1759743463 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d0030cc0 con 0x7f42d4106490 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.307+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d419ea90 con 0x7f42d4106490 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.307+0000 7f42caffd640 1 -- 172.21.15.1:0/1759743463 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d003a640 con 0x7f42d4106490 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.308+0000 7f42caffd640 1 -- 172.21.15.1:0/1759743463 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f42d003a8f0 con 0x7f42d4106490 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.308+0000 7f42caffd640 1 --2- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42c0039200 0x7f42c003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.308+0000 7f42caffd640 1 -- 172.21.15.1:0/1759743463 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f42d006c790 con 0x7f42d4106490 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.308+0000 7f42d979d640 1 --2- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42c0039200 0x7f42c003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.308+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b80052c0 con 0x7f42d4106490 2024-09-16T00:19:18.465 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.309+0000 7f42d979d640 1 --2- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42c0039200 0x7f42c003b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42c40098c0 tx=0x7f42c4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.313+0000 7f42caffd640 1 -- 172.21.15.1:0/1759743463 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42d003ba40 con 0x7f42d4106490 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.508+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f42b80055b0 con 0x7f42d4106490 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.510+0000 7f42caffd640 1 -- 172.21.15.1:0/1759743463 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f42d002fd40 con 0x7f42d4106490 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42c0039200 msgr2=0x7f42c003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 --2- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42c0039200 0x7f42c003b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42c40098c0 tx=0x7f42c4006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 msgr2=0x7f42d419b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 --2- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d419b620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42d00097f0 tx=0x7f42d00327b0 comp rx=0 tx=0).stop 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 shutdown_connections 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 --2- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f42c0039200 0x7f42c003b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 --2- 172.21.15.1:0/1759743463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42d4106490 0x7f42d419b620 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 >> 172.21.15.1:0/1759743463 conn(0x7f42d4101ca0 msgr2=0x7f42d4102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 shutdown_connections 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:17.513+0000 7f42dc202640 1 -- 172.21.15.1:0/1759743463 wait complete. 2024-09-16T00:19:18.466 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling the dashboard module... 2024-09-16T00:19:19.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:19 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2829385721' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T00:19:19.361 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: ignoring --setuser ceph since I am not root 2024-09-16T00:19:19.362 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: ignoring --setgroup ceph since I am not root 2024-09-16T00:19:19.362 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:19.301+0000 7f6e1b91e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:19:19.362 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:19.360+0000 7f6e1b91e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:19:19.614 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:19.524+0000 7f6e1b91e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:19:19.614 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:19.593+0000 7f6e1b91e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:19:19.894 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:19.747+0000 7f6e1b91e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:19:19.894 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:19.893+0000 7f6e1b91e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:19:19.922 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.848+0000 7fa44004f640 1 Processor -- start 2024-09-16T00:19:19.922 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.848+0000 7fa44004f640 1 -- start start 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.848+0000 7fa44004f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.848+0000 7fa44004f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa43807c7e0 con 0x7fa43807dd30 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.849+0000 7fa43ddeb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.849+0000 7fa43ddeb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54000/0 (socket says 172.21.15.1:54000) 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.849+0000 7fa43ddeb640 1 -- 172.21.15.1:0/1256748186 learned_addr learned my addr 172.21.15.1:0/1256748186 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.849+0000 7fa43ddeb640 1 -- 172.21.15.1:0/1256748186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43807c960 con 0x7fa43807dd30 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.850+0000 7fa43ddeb640 1 --2- 172.21.15.1:0/1256748186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43807c2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa434009820 tx=0x7fa43402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fcc2794095bd4b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.850+0000 7fa43cde9640 1 -- 172.21.15.1:0/1256748186 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa434037070 con 0x7fa43807dd30 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.850+0000 7fa43cde9640 1 -- 172.21.15.1:0/1256748186 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa434032920 con 0x7fa43807dd30 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.850+0000 7fa43cde9640 1 -- 172.21.15.1:0/1256748186 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa434030810 con 0x7fa43807dd30 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.851+0000 7fa44004f640 1 -- 172.21.15.1:0/1256748186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 msgr2=0x7fa43807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:19.923 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.851+0000 7fa44004f640 1 --2- 172.21.15.1:0/1256748186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43807c2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa434009820 tx=0x7fa43402a270 comp rx=0 tx=0).stop 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.851+0000 7fa44004f640 1 -- 172.21.15.1:0/1256748186 shutdown_connections 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.851+0000 7fa44004f640 1 --2- 172.21.15.1:0/1256748186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43807c2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.851+0000 7fa44004f640 1 -- 172.21.15.1:0/1256748186 >> 172.21.15.1:0/1256748186 conn(0x7fa438102e50 msgr2=0x7fa438105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.851+0000 7fa44004f640 1 -- 172.21.15.1:0/1256748186 shutdown_connections 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.851+0000 7fa44004f640 1 -- 172.21.15.1:0/1256748186 wait complete. 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.851+0000 7fa44004f640 1 Processor -- start 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.851+0000 7fa44004f640 1 -- start start 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa44004f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa44004f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa43810dba0 con 0x7fa43807dd30 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa43ddeb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa43ddeb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54016/0 (socket says 172.21.15.1:54016) 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa43ddeb640 1 -- 172.21.15.1:0/2829385721 learned_addr learned my addr 172.21.15.1:0/2829385721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:19.924 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa43ddeb640 1 -- 172.21.15.1:0/2829385721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43819dff0 con 0x7fa43807dd30 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa43ddeb640 1 --2- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43819dab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4340097f0 tx=0x7fa4340327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa42effd640 1 -- 172.21.15.1:0/2829385721 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa434037040 con 0x7fa43807dd30 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa42effd640 1 -- 172.21.15.1:0/2829385721 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa434030cc0 con 0x7fa43807dd30 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43819e280 con 0x7fa43807dd30 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa42effd640 1 -- 172.21.15.1:0/2829385721 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa43403a6b0 con 0x7fa43807dd30 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.852+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4381a0f70 con 0x7fa43807dd30 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.853+0000 7fa42effd640 1 -- 172.21.15.1:0/2829385721 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa43403a850 con 0x7fa43807dd30 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.853+0000 7fa42effd640 1 --2- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fa424039200 0x7fa42403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.853+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43807c320 con 0x7fa43807dd30 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.853+0000 7fa42effd640 1 -- 172.21.15.1:0/2829385721 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa43406c8d0 con 0x7fa43807dd30 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.854+0000 7fa43d5ea640 1 --2- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fa424039200 0x7fa42403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.854+0000 7fa43d5ea640 1 --2- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fa424039200 0x7fa42403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa4280098c0 tx=0x7fa428006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:18.858+0000 7fa42effd640 1 -- 172.21.15.1:0/2829385721 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa434042c00 con 0x7fa43807dd30 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.081+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa43802d7c0 con 0x7fa43807dd30 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.132+0000 7fa42effd640 1 -- 172.21.15.1:0/2829385721 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa43403ab40 con 0x7fa43807dd30 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.134+0000 7fa42effd640 1 -- 172.21.15.1:0/2829385721 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fa43403b3f0 con 0x7fa43807dd30 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fa424039200 msgr2=0x7fa42403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 --2- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fa424039200 0x7fa42403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa4280098c0 tx=0x7fa428006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 msgr2=0x7fa43819dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 --2- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43819dab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4340097f0 tx=0x7fa4340327b0 comp rx=0 tx=0).stop 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 shutdown_connections 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 --2- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fa424039200 0x7fa42403b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 --2- 172.21.15.1:0/2829385721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa43807dd30 0x7fa43819dab0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 >> 172.21.15.1:0/2829385721 conn(0x7fa438102e50 msgr2=0x7fa438103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 shutdown_connections 2024-09-16T00:19:19.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:19.135+0000 7fa44004f640 1 -- 172.21.15.1:0/2829385721 wait complete. 2024-09-16T00:19:20.145 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:20 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2829385721' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T00:19:20.145 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:20 smithi001 ceph-mon[30934]: mgrmap e9: x(active, since 18s) 2024-09-16T00:19:20.504 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:20.229+0000 7f6e1b91e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:19:20.504 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:20.294+0000 7f6e1b91e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:19:20.504 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:20.503+0000 7f6e1b91e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:19:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:19:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T00:19:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:19:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T00:19:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:19:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:19:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.332+0000 7f6bce333640 1 Processor -- start 2024-09-16T00:19:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.332+0000 7f6bce333640 1 -- start start 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.332+0000 7f6bce333640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc8107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.332+0000 7f6bce333640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc8107f60 con 0x7f6bc8107620 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.332+0000 7f6bc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc8107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.332+0000 7f6bc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc8107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55806/0 (socket says 172.21.15.1:55806) 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.332+0000 7f6bc7fff640 1 -- 172.21.15.1:0/861159070 learned_addr learned my addr 172.21.15.1:0/861159070 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.333+0000 7f6bc7fff640 1 -- 172.21.15.1:0/861159070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc81080e0 con 0x7f6bc8107620 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.333+0000 7f6bc7fff640 1 --2- 172.21.15.1:0/861159070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc8107a20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009820 tx=0x7f6bb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f4849ee48d0fad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bc6ffd640 1 -- 172.21.15.1:0/861159070 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb8037070 con 0x7f6bc8107620 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bc6ffd640 1 -- 172.21.15.1:0/861159070 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb8032920 con 0x7f6bc8107620 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bc6ffd640 1 -- 172.21.15.1:0/861159070 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb8030810 con 0x7f6bc8107620 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bce333640 1 -- 172.21.15.1:0/861159070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 msgr2=0x7f6bc8107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bce333640 1 --2- 172.21.15.1:0/861159070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc8107a20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009820 tx=0x7f6bb802a270 comp rx=0 tx=0).stop 2024-09-16T00:19:21.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bce333640 1 -- 172.21.15.1:0/861159070 shutdown_connections 2024-09-16T00:19:21.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bce333640 1 --2- 172.21.15.1:0/861159070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc8107a20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:21.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bce333640 1 -- 172.21.15.1:0/861159070 >> 172.21.15.1:0/861159070 conn(0x7f6bc8102e50 msgr2=0x7f6bc8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:21.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bce333640 1 -- 172.21.15.1:0/861159070 shutdown_connections 2024-09-16T00:19:21.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.334+0000 7f6bce333640 1 -- 172.21.15.1:0/861159070 wait complete. 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.335+0000 7f6bce333640 1 Processor -- start 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.335+0000 7f6bce333640 1 -- start start 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.336+0000 7f6bce333640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.336+0000 7f6bce333640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc810ba90 con 0x7f6bc8107620 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.336+0000 7f6bc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.336+0000 7f6bc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55808/0 (socket says 172.21.15.1:55808) 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.336+0000 7f6bc7fff640 1 -- 172.21.15.1:0/210520284 learned_addr learned my addr 172.21.15.1:0/210520284 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.336+0000 7f6bc7fff640 1 -- 172.21.15.1:0/210520284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc819be00 con 0x7f6bc8107620 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.336+0000 7f6bc7fff640 1 --2- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc819b8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8030380 tx=0x7f6bb8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.336+0000 7f6bc57fa640 1 -- 172.21.15.1:0/210520284 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb8037040 con 0x7f6bc8107620 2024-09-16T00:19:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.336+0000 7f6bc57fa640 1 -- 172.21.15.1:0/210520284 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb8040070 con 0x7f6bc8107620 2024-09-16T00:19:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.337+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc819c090 con 0x7f6bc8107620 2024-09-16T00:19:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.337+0000 7f6bc57fa640 1 -- 172.21.15.1:0/210520284 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb803a7b0 con 0x7f6bc8107620 2024-09-16T00:19:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.337+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc819ed80 con 0x7f6bc8107620 2024-09-16T00:19:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.337+0000 7f6bc57fa640 1 -- 172.21.15.1:0/210520284 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f6bb803aa30 con 0x7f6bc8107620 2024-09-16T00:19:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.338+0000 7f6bc57fa640 1 --2- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f6bb0039200 0x7f6bb003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.338+0000 7f6bc57fa640 1 -- 172.21.15.1:0/210520284 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6bb806c920 con 0x7f6bc8107620 2024-09-16T00:19:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.338+0000 7f6bc77fe640 1 -- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f6bb0039200 msgr2=0x7f6bb003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/178572347 2024-09-16T00:19:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.338+0000 7f6bc77fe640 1 --2- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f6bb0039200 0x7f6bb003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:19:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.338+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc8107aa0 con 0x7f6bc8107620 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.343+0000 7f6bc57fa640 1 -- 172.21.15.1:0/210520284 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bb803ba00 con 0x7f6bc8107620 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.539+0000 7f6bc77fe640 1 -- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f6bb0039200 msgr2=0x7f6bb003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/178572347 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.539+0000 7f6bc77fe640 1 --2- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f6bb0039200 0x7f6bb003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.580+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6bc8067b60 con 0x7f6bc8107620 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.581+0000 7f6bc57fa640 1 -- 172.21.15.1:0/210520284 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f6bb803bc00 con 0x7f6bc8107620 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f6bb0039200 msgr2=0x7f6bb003b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 --2- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f6bb0039200 0x7f6bb003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 msgr2=0x7f6bc819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 --2- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc819b8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8030380 tx=0x7f6bb8032670 comp rx=0 tx=0).stop 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 shutdown_connections 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 --2- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7f6bb0039200 0x7f6bb003b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 --2- 172.21.15.1:0/210520284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc8107620 0x7f6bc819b8c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 >> 172.21.15.1:0/210520284 conn(0x7f6bc8102e50 msgr2=0x7f6bc8103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:21.156 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 shutdown_connections 2024-09-16T00:19:21.156 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:20.584+0000 7f6bce333640 1 -- 172.21.15.1:0/210520284 wait complete. 2024-09-16T00:19:21.156 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-09-16T00:19:21.156 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 9... 2024-09-16T00:19:21.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:21 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/210520284' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:19:21.704 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:21 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:21.448+0000 7f6e1b91e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:19:21.705 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:21 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:21.520+0000 7f6e1b91e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:19:22.451 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:22.267+0000 7f6e1b91e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:19:22.451 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:22.387+0000 7f6e1b91e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:19:22.831 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:22.535+0000 7f6e1b91e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:19:22.831 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:22.599+0000 7f6e1b91e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:19:22.831 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:22.754+0000 7f6e1b91e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:19:22.831 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:22.825+0000 7f6e1b91e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:19:23.125 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:22.887+0000 7f6e1b91e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:19:23.125 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:23.052+0000 7f6e1b91e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:19:23.125 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:23.120+0000 7f6e1b91e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:19:23.401 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:23.189+0000 7f6e1b91e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:19:23.401 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:23.275+0000 7f6e1b91e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:19:23.401 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:23.337+0000 7f6e1b91e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:19:23.402 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:23.399+0000 7f6e1b91e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:19:24.481 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:24.223+0000 7f6e1b91e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: Active manager daemon x restarted 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: Activating manager daemon x 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: mgrmap e10: x(active, starting, since 0.00349856s) 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:19:24.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: Manager daemon x is now available 2024-09-16T00:19:24.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:24.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:19:24.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:19:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:19:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T00:19:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:19:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:19:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.562+0000 7fcefa73e640 1 Processor -- start 2024-09-16T00:19:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.562+0000 7fcefa73e640 1 -- start start 2024-09-16T00:19:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.562+0000 7fcefa73e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef410a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.563+0000 7fcefa73e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4075840 con 0x7fcef4108070 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.563+0000 7fcef3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef410a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.563+0000 7fcef3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef410a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55816/0 (socket says 172.21.15.1:55816) 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.563+0000 7fcef3fff640 1 -- 172.21.15.1:0/842482888 learned_addr learned my addr 172.21.15.1:0/842482888 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.563+0000 7fcef3fff640 1 -- 172.21.15.1:0/842482888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef410a9a0 con 0x7fcef4108070 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.564+0000 7fcef3fff640 1 --2- 172.21.15.1:0/842482888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef410a460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009820 tx=0x7fcee402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8445a4805f516038 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.564+0000 7fcef37fe640 1 -- 172.21.15.1:0/842482888 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee4037070 con 0x7fcef4108070 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.564+0000 7fcef37fe640 1 -- 172.21.15.1:0/842482888 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee4032920 con 0x7fcef4108070 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.564+0000 7fcef37fe640 1 -- 172.21.15.1:0/842482888 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee4030810 con 0x7fcef4108070 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.565+0000 7fcefa73e640 1 -- 172.21.15.1:0/842482888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 msgr2=0x7fcef410a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.565+0000 7fcefa73e640 1 --2- 172.21.15.1:0/842482888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef410a460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009820 tx=0x7fcee402a270 comp rx=0 tx=0).stop 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.565+0000 7fcefa73e640 1 -- 172.21.15.1:0/842482888 shutdown_connections 2024-09-16T00:19:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.565+0000 7fcefa73e640 1 --2- 172.21.15.1:0/842482888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef410a460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.565+0000 7fcefa73e640 1 -- 172.21.15.1:0/842482888 >> 172.21.15.1:0/842482888 conn(0x7fcef4101c70 msgr2=0x7fcef4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.565+0000 7fcefa73e640 1 -- 172.21.15.1:0/842482888 shutdown_connections 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.565+0000 7fcefa73e640 1 -- 172.21.15.1:0/842482888 wait complete. 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.566+0000 7fcefa73e640 1 Processor -- start 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.566+0000 7fcefa73e640 1 -- start start 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.566+0000 7fcefa73e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.566+0000 7fcefa73e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcef410ba20 con 0x7fcef4108070 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.566+0000 7fcef3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.566+0000 7fcef3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef419daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55818/0 (socket says 172.21.15.1:55818) 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.566+0000 7fcef3fff640 1 -- 172.21.15.1:0/591495965 learned_addr learned my addr 172.21.15.1:0/591495965 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.566+0000 7fcef3fff640 1 -- 172.21.15.1:0/591495965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef419e030 con 0x7fcef4108070 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.566+0000 7fcef3fff640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef419daf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcee4030420 tx=0x7fcee4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.567+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee4037040 con 0x7fcef4108070 2024-09-16T00:19:26.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.567+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef419e2c0 con 0x7fcef4108070 2024-09-16T00:19:26.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.567+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee4040070 con 0x7fcef4108070 2024-09-16T00:19:26.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.567+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee403a6e0 con 0x7fcef4108070 2024-09-16T00:19:26.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.567+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef41a0fb0 con 0x7fcef4108070 2024-09-16T00:19:26.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.568+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fcee403a9e0 con 0x7fcef4108070 2024-09-16T00:19:26.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.568+0000 7fcef1ffb640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 0x7fcee003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:26.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.568+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 --> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcef41084d0 con 0x7fcee0039200 2024-09-16T00:19:26.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.568+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcee406c940 con 0x7fcef4108070 2024-09-16T00:19:26.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.568+0000 7fceebfff640 1 -- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 msgr2=0x7fcee003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/178572347 2024-09-16T00:19:26.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.568+0000 7fceebfff640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 0x7fcee003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.769+0000 7fceebfff640 1 -- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 msgr2=0x7fcee003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/178572347 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:21.769+0000 7fceebfff640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 0x7fcee003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:22.170+0000 7fceebfff640 1 -- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 msgr2=0x7fcee003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/178572347 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:22.170+0000 7fceebfff640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 0x7fcee003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:22.971+0000 7fceebfff640 1 -- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 msgr2=0x7fcee003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/178572347 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:22.971+0000 7fceebfff640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 0x7fcee003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:24.226+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7fcee406b3f0 con 0x7fcef4108070 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:24.226+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 msgr2=0x7fcee003b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:24.226+0000 7fcef1ffb640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 0x7fcee003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.227+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fcee4039d70 con 0x7fcef4108070 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.228+0000 7fcef1ffb640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcee003cad0 0x7fcee003eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.228+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcee4031ec0 con 0x7fcee003cad0 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.228+0000 7fceebfff640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcee003cad0 0x7fcee003eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:26.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.229+0000 7fceebfff640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcee003cad0 0x7fcee003eec0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcedc003980 tx=0x7fcedc006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.230+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fcee4031ec0 con 0x7fcee003cad0 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.242+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fcef4067a60 con 0x7fcee003cad0 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.242+0000 7fcef1ffb640 1 -- 172.21.15.1:0/591495965 <== mgr.14150 v2:172.21.15.1:6800/3146869121 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fcef4067a60 con 0x7fcee003cad0 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcee003cad0 msgr2=0x7fcee003eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcee003cad0 0x7fcee003eec0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcedc003980 tx=0x7fcedc006e10 comp rx=0 tx=0).stop 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 msgr2=0x7fcef419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef419daf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcee4030420 tx=0x7fcee4032670 comp rx=0 tx=0).stop 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 shutdown_connections 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcee003cad0 0x7fcee003eec0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:6800/178572347,v1:172.21.15.1:6801/178572347] conn(0x7fcee0039200 0x7fcee003b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 --2- 172.21.15.1:0/591495965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef4108070 0x7fcef419daf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 >> 172.21.15.1:0/591495965 conn(0x7fcef4101c70 msgr2=0x7fcef41029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 shutdown_connections 2024-09-16T00:19:26.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:25.243+0000 7fcefa73e640 1 -- 172.21.15.1:0/591495965 wait complete. 2024-09-16T00:19:26.195 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 9 is available 2024-09-16T00:19:26.195 INFO:teuthology.orchestra.run.smithi001.stdout:Generating a dashboard self-signed certificate... 2024-09-16T00:19:26.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:26 smithi001 ceph-mon[30934]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T00:19:26.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:26 smithi001 ceph-mon[30934]: mgrmap e11: x(active, since 1.00564s) 2024-09-16T00:19:26.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:26 smithi001 ceph-mon[30934]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T00:19:26.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:27.482 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T00:19:27.482 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.604+0000 7f586e12c640 1 Processor -- start 2024-09-16T00:19:27.482 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.604+0000 7f586e12c640 1 -- start start 2024-09-16T00:19:27.482 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.605+0000 7f586e12c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f5868106890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:27.482 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.605+0000 7f586e12c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5868106e60 con 0x7f5868106490 2024-09-16T00:19:27.482 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.605+0000 7f58677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f5868106890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.605+0000 7f58677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f5868106890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55902/0 (socket says 172.21.15.1:55902) 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.605+0000 7f58677fe640 1 -- 172.21.15.1:0/3517271150 learned_addr learned my addr 172.21.15.1:0/3517271150 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.605+0000 7f58677fe640 1 -- 172.21.15.1:0/3517271150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5868107690 con 0x7f5868106490 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.606+0000 7f58677fe640 1 --2- 172.21.15.1:0/3517271150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f5868106890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5858009820 tx=0x7f585802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c28a496cffdc48 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.606+0000 7f58667fc640 1 -- 172.21.15.1:0/3517271150 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5858037070 con 0x7f5868106490 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.606+0000 7f58667fc640 1 -- 172.21.15.1:0/3517271150 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5858032920 con 0x7f5868106490 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.606+0000 7f58667fc640 1 -- 172.21.15.1:0/3517271150 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5858030810 con 0x7f5868106490 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.607+0000 7f586e12c640 1 -- 172.21.15.1:0/3517271150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 msgr2=0x7f5868106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.607+0000 7f586e12c640 1 --2- 172.21.15.1:0/3517271150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f5868106890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5858009820 tx=0x7f585802a270 comp rx=0 tx=0).stop 2024-09-16T00:19:27.483 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.607+0000 7f586e12c640 1 -- 172.21.15.1:0/3517271150 shutdown_connections 2024-09-16T00:19:27.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.607+0000 7f586e12c640 1 --2- 172.21.15.1:0/3517271150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f5868106890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:27.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.607+0000 7f586e12c640 1 -- 172.21.15.1:0/3517271150 >> 172.21.15.1:0/3517271150 conn(0x7f5868101ca0 msgr2=0x7f58681040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:27.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.607+0000 7f586e12c640 1 -- 172.21.15.1:0/3517271150 shutdown_connections 2024-09-16T00:19:27.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.607+0000 7f586e12c640 1 -- 172.21.15.1:0/3517271150 wait complete. 2024-09-16T00:19:27.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.608+0000 7f586e12c640 1 Processor -- start 2024-09-16T00:19:27.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.608+0000 7f586e12c640 1 -- start start 2024-09-16T00:19:27.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.608+0000 7f586e12c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f586819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:27.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.608+0000 7f586e12c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5868074b20 con 0x7f5868106490 2024-09-16T00:19:27.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.608+0000 7f58677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f586819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:27.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.608+0000 7f58677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f586819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55904/0 (socket says 172.21.15.1:55904) 2024-09-16T00:19:27.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.608+0000 7f58677fe640 1 -- 172.21.15.1:0/3217365306 learned_addr learned my addr 172.21.15.1:0/3217365306 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:27.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.608+0000 7f58677fe640 1 -- 172.21.15.1:0/3217365306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586819dd00 con 0x7f5868106490 2024-09-16T00:19:27.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.609+0000 7f58677fe640 1 --2- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f586819d7c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58580097f0 tx=0x7f58580327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:27.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.609+0000 7f5864ff9640 1 -- 172.21.15.1:0/3217365306 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5858037040 con 0x7f5868106490 2024-09-16T00:19:27.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.609+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586819df90 con 0x7f5868106490 2024-09-16T00:19:27.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.609+0000 7f5864ff9640 1 -- 172.21.15.1:0/3217365306 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5858030cc0 con 0x7f5868106490 2024-09-16T00:19:27.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.610+0000 7f5864ff9640 1 -- 172.21.15.1:0/3217365306 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f585803a640 con 0x7f5868106490 2024-09-16T00:19:27.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.610+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681a0c80 con 0x7f5868106490 2024-09-16T00:19:27.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.610+0000 7f5864ff9640 1 -- 172.21.15.1:0/3217365306 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f585803a940 con 0x7f5868106490 2024-09-16T00:19:27.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.610+0000 7f5864ff9640 1 --2- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5850039200 0x7f585003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:27.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.610+0000 7f5864ff9640 1 -- 172.21.15.1:0/3217365306 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5858073060 con 0x7f5868106490 2024-09-16T00:19:27.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.610+0000 7f5866ffd640 1 --2- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5850039200 0x7f585003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:27.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.611+0000 7f5866ffd640 1 --2- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5850039200 0x7f585003b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f585c0098c0 tx=0x7f585c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:27.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.611+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5868106890 con 0x7f5868106490 2024-09-16T00:19:27.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.615+0000 7f5864ff9640 1 -- 172.21.15.1:0/3217365306 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5858042300 con 0x7f5868106490 2024-09-16T00:19:27.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.802+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f586802d940 con 0x7f5850039200 2024-09-16T00:19:27.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.887+0000 7f5864ff9640 1 -- 172.21.15.1:0/3217365306 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f586802d940 con 0x7f5850039200 2024-09-16T00:19:27.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.889+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5850039200 msgr2=0x7f585003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:27.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.889+0000 7f586e12c640 1 --2- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5850039200 0x7f585003b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f585c0098c0 tx=0x7f585c006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:27.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.889+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 msgr2=0x7f586819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:27.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.889+0000 7f586e12c640 1 --2- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f586819d7c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58580097f0 tx=0x7f58580327b0 comp rx=0 tx=0).stop 2024-09-16T00:19:27.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.889+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 shutdown_connections 2024-09-16T00:19:27.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.889+0000 7f586e12c640 1 --2- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5850039200 0x7f585003b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:27.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.889+0000 7f586e12c640 1 --2- 172.21.15.1:0/3217365306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5868106490 0x7f586819d7c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:27.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.889+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 >> 172.21.15.1:0/3217365306 conn(0x7f5868101ca0 msgr2=0x7f586810acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:27.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.890+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 shutdown_connections 2024-09-16T00:19:27.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:26.890+0000 7f586e12c640 1 -- 172.21.15.1:0/3217365306 wait complete. 2024-09-16T00:19:27.489 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial admin user... 2024-09-16T00:19:27.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-mon[30934]: mgrmap e12: x(active, since 2s) 2024-09-16T00:19:27.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-mon[30934]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:27.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:27.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:27.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:28.201 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:27.895+0000 7f6d86a72640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:00:19:27] ENGINE Error in HTTPServer.serve 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: Traceback (most recent call last): 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: self._connections.run(self.expiration_interval) 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: self._run(expiration_interval) 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:19:28.202 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: s = self.context.wrap_socket( 2024-09-16T00:19:28.203 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:19:28.203 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: return self.sslsocket_class._create( 2024-09-16T00:19:28.203 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:19:28.203 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: self.do_handshake() 2024-09-16T00:19:28.203 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:19:28.203 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: self._sslobj.do_handshake() 2024-09-16T00:19:28.203 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:19:28.203 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:19:27 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:19:28.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:28 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:27] ENGINE Bus STARTING 2024-09-16T00:19:28.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:28 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:27] ENGINE Serving on https://172.21.15.1:7150 2024-09-16T00:19:28.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:28 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:27] ENGINE Error in HTTPServer.serve 2024-09-16T00:19:28.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: Traceback (most recent call last): 2024-09-16T00:19:28.953 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:19:28.953 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:19:28.953 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:19:28.953 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._run(expiration_interval) 2024-09-16T00:19:28.953 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:19:28.953 INFO:journalctl@ceph.mon.a.smithi001.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:19:28.954 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:19:28.954 INFO:journalctl@ceph.mon.a.smithi001.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:19:28.954 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:19:28.954 INFO:journalctl@ceph.mon.a.smithi001.stdout: s = self.context.wrap_socket( 2024-09-16T00:19:28.954 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:19:28.955 INFO:journalctl@ceph.mon.a.smithi001.stdout: return self.sslsocket_class._create( 2024-09-16T00:19:28.955 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:19:28.955 INFO:journalctl@ceph.mon.a.smithi001.stdout: self.do_handshake() 2024-09-16T00:19:28.955 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:19:28.955 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._sslobj.do_handshake() 2024-09-16T00:19:28.955 INFO:journalctl@ceph.mon.a.smithi001.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:19:28.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:28 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:27] ENGINE Serving on http://172.21.15.1:8765 2024-09-16T00:19:28.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:28 smithi001 ceph-mon[30934]: [16/Sep/2024:00:19:27] ENGINE Bus STARTED 2024-09-16T00:19:28.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:28 smithi001 ceph-mon[30934]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:28.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:29.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$MWqcKo00xfV0LhKud/IFvO5G8ZQ/mxEz6RLFvqxFy4LzdQhd30Ixy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726445968, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T00:19:29.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.908+0000 7fb0574dc640 1 Processor -- start 2024-09-16T00:19:29.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.908+0000 7fb0574dc640 1 -- start start 2024-09-16T00:19:29.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.908+0000 7fb0574dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb0501061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:29.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.908+0000 7fb0574dc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb050075880 con 0x7fb050105df0 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.908+0000 7fb055278640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb0501061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.908+0000 7fb055278640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb0501061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55920/0 (socket says 172.21.15.1:55920) 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.908+0000 7fb055278640 1 -- 172.21.15.1:0/1318631530 learned_addr learned my addr 172.21.15.1:0/1318631530 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.909+0000 7fb055278640 1 -- 172.21.15.1:0/1318631530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb050106730 con 0x7fb050105df0 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.909+0000 7fb055278640 1 --2- 172.21.15.1:0/1318631530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb0501061f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009820 tx=0x7fb04c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a83212903e0502b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.909+0000 7fb047fff640 1 -- 172.21.15.1:0/1318631530 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb04c037070 con 0x7fb050105df0 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.910+0000 7fb047fff640 1 -- 172.21.15.1:0/1318631530 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c032920 con 0x7fb050105df0 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.910+0000 7fb047fff640 1 -- 172.21.15.1:0/1318631530 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb04c030810 con 0x7fb050105df0 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.910+0000 7fb0574dc640 1 -- 172.21.15.1:0/1318631530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 msgr2=0x7fb0501061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:29.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.910+0000 7fb0574dc640 1 --2- 172.21.15.1:0/1318631530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb0501061f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009820 tx=0x7fb04c02a270 comp rx=0 tx=0).stop 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.910+0000 7fb0574dc640 1 -- 172.21.15.1:0/1318631530 shutdown_connections 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.910+0000 7fb0574dc640 1 --2- 172.21.15.1:0/1318631530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb0501061f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.910+0000 7fb0574dc640 1 -- 172.21.15.1:0/1318631530 >> 172.21.15.1:0/1318631530 conn(0x7fb050101ca0 msgr2=0x7fb0501040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.910+0000 7fb0574dc640 1 -- 172.21.15.1:0/1318631530 shutdown_connections 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.910+0000 7fb0574dc640 1 -- 172.21.15.1:0/1318631530 wait complete. 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.911+0000 7fb0574dc640 1 Processor -- start 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.911+0000 7fb0574dc640 1 -- start start 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.911+0000 7fb0574dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb05019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.911+0000 7fb0574dc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb050107870 con 0x7fb050105df0 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.911+0000 7fb055278640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb05019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.911+0000 7fb055278640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb05019d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55928/0 (socket says 172.21.15.1:55928) 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.911+0000 7fb055278640 1 -- 172.21.15.1:0/1654681431 learned_addr learned my addr 172.21.15.1:0/1654681431 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:29.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.911+0000 7fb055278640 1 -- 172.21.15.1:0/1654681431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05019dca0 con 0x7fb050105df0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.912+0000 7fb055278640 1 --2- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb05019d760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0097f0 tx=0x7fb04c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.912+0000 7fb0467fc640 1 -- 172.21.15.1:0/1654681431 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb04c037040 con 0x7fb050105df0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.912+0000 7fb0467fc640 1 -- 172.21.15.1:0/1654681431 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c030cc0 con 0x7fb050105df0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.912+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05019df30 con 0x7fb050105df0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.912+0000 7fb0467fc640 1 -- 172.21.15.1:0/1654681431 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb04c03a6b0 con 0x7fb050105df0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.912+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0501a0c20 con 0x7fb050105df0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.913+0000 7fb0467fc640 1 -- 172.21.15.1:0/1654681431 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fb04c03a8a0 con 0x7fb050105df0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.913+0000 7fb0467fc640 1 --2- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb03c039200 0x7fb03c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.913+0000 7fb0467fc640 1 -- 172.21.15.1:0/1654681431 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb04c06c900 con 0x7fb050105df0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.913+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0501061f0 con 0x7fb050105df0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.913+0000 7fb054a77640 1 --2- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb03c039200 0x7fb03c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.913+0000 7fb054a77640 1 --2- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb03c039200 0x7fb03c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0400098c0 tx=0x7fb040006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:29.653 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:27.918+0000 7fb0467fc640 1 -- 172.21.15.1:0/1654681431 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb04c02fd40 con 0x7fb050105df0 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.118+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fb05002d8c0 con 0x7fb03c039200 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.367+0000 7fb0467fc640 1 -- 172.21.15.1:0/1654681431 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fb05002d8c0 con 0x7fb03c039200 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb03c039200 msgr2=0x7fb03c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 --2- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb03c039200 0x7fb03c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0400098c0 tx=0x7fb040006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 msgr2=0x7fb05019d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 --2- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb05019d760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0097f0 tx=0x7fb04c0327b0 comp rx=0 tx=0).stop 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 shutdown_connections 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 --2- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb03c039200 0x7fb03c03b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 --2- 172.21.15.1:0/1654681431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb050105df0 0x7fb05019d760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 >> 172.21.15.1:0/1654681431 conn(0x7fb050101ca0 msgr2=0x7fb050191a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 shutdown_connections 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:28.369+0000 7fb0574dc640 1 -- 172.21.15.1:0/1654681431 wait complete. 2024-09-16T00:19:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:Fetching dashboard port number... 2024-09-16T00:19:30.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:30 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/816962341' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T00:19:30.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:30 smithi001 ceph-mon[30934]: mgrmap e13: x(active, since 6s) 2024-09-16T00:19:30.831 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T00:19:30.831 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.058+0000 7faffd39b640 1 Processor -- start 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.058+0000 7faffd39b640 1 -- start start 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.058+0000 7faffd39b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.058+0000 7faffd39b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faff807c7e0 con 0x7faff807dd30 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.058+0000 7faff6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.059+0000 7faff6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55940/0 (socket says 172.21.15.1:55940) 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.059+0000 7faff6ffd640 1 -- 172.21.15.1:0/439525204 learned_addr learned my addr 172.21.15.1:0/439525204 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.059+0000 7faff6ffd640 1 -- 172.21.15.1:0/439525204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff807c960 con 0x7faff807dd30 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.059+0000 7faff6ffd640 1 --2- 172.21.15.1:0/439525204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff807c2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009820 tx=0x7fafe802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78059b2b10079ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faff67fc640 1 -- 172.21.15.1:0/439525204 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafe8037070 con 0x7faff807dd30 2024-09-16T00:19:30.832 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faff67fc640 1 -- 172.21.15.1:0/439525204 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe8032920 con 0x7faff807dd30 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faff67fc640 1 -- 172.21.15.1:0/439525204 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafe8030810 con 0x7faff807dd30 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faffd39b640 1 -- 172.21.15.1:0/439525204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 msgr2=0x7faff807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faffd39b640 1 --2- 172.21.15.1:0/439525204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff807c2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009820 tx=0x7fafe802a270 comp rx=0 tx=0).stop 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faffd39b640 1 -- 172.21.15.1:0/439525204 shutdown_connections 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faffd39b640 1 --2- 172.21.15.1:0/439525204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff807c2a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faffd39b640 1 -- 172.21.15.1:0/439525204 >> 172.21.15.1:0/439525204 conn(0x7faff8102e50 msgr2=0x7faff8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faffd39b640 1 -- 172.21.15.1:0/439525204 shutdown_connections 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.060+0000 7faffd39b640 1 -- 172.21.15.1:0/439525204 wait complete. 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.061+0000 7faffd39b640 1 Processor -- start 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.061+0000 7faffd39b640 1 -- start start 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.061+0000 7faffd39b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.061+0000 7faffd39b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faff810dba0 con 0x7faff807dd30 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.061+0000 7faff6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.061+0000 7faff6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55942/0 (socket says 172.21.15.1:55942) 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.061+0000 7faff6ffd640 1 -- 172.21.15.1:0/816962341 learned_addr learned my addr 172.21.15.1:0/816962341 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.061+0000 7faff6ffd640 1 -- 172.21.15.1:0/816962341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff819dff0 con 0x7faff807dd30 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.062+0000 7faff6ffd640 1 --2- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff819dab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fafe80097f0 tx=0x7fafe80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.062+0000 7faff4ff9640 1 -- 172.21.15.1:0/816962341 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafe8037040 con 0x7faff807dd30 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.062+0000 7faff4ff9640 1 -- 172.21.15.1:0/816962341 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe8030cc0 con 0x7faff807dd30 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.062+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff819e280 con 0x7faff807dd30 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.062+0000 7faff4ff9640 1 -- 172.21.15.1:0/816962341 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafe803a6b0 con 0x7faff807dd30 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.062+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81a0f70 con 0x7faff807dd30 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.063+0000 7faff4ff9640 1 -- 172.21.15.1:0/816962341 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fafe803a8a0 con 0x7faff807dd30 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.063+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff807c320 con 0x7faff807dd30 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.063+0000 7faff4ff9640 1 --2- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafe4039200 0x7fafe403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.063+0000 7faff4ff9640 1 -- 172.21.15.1:0/816962341 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fafe806c940 con 0x7faff807dd30 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.063+0000 7fafeffff640 1 --2- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafe4039200 0x7fafe403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:30.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.063+0000 7fafeffff640 1 --2- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafe4039200 0x7fafe403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fafe00098c0 tx=0x7fafe0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.068+0000 7faff4ff9640 1 -- 172.21.15.1:0/816962341 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafe80371e0 con 0x7faff807dd30 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.243+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7faff802d7c0 con 0x7faff807dd30 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.244+0000 7faff4ff9640 1 -- 172.21.15.1:0/816962341 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fafe803bba0 con 0x7faff807dd30 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.246+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafe4039200 msgr2=0x7fafe403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.246+0000 7faffd39b640 1 --2- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafe4039200 0x7fafe403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fafe00098c0 tx=0x7fafe0006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.246+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 msgr2=0x7faff819dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.246+0000 7faffd39b640 1 --2- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff819dab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fafe80097f0 tx=0x7fafe80327b0 comp rx=0 tx=0).stop 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.246+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 shutdown_connections 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.247+0000 7faffd39b640 1 --2- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafe4039200 0x7fafe403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.247+0000 7faffd39b640 1 --2- 172.21.15.1:0/816962341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faff807dd30 0x7faff819dab0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.247+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 >> 172.21.15.1:0/816962341 conn(0x7faff8102e50 msgr2=0x7faff8103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.247+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 shutdown_connections 2024-09-16T00:19:30.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:30.247+0000 7faffd39b640 1 -- 172.21.15.1:0/816962341 wait complete. 2024-09-16T00:19:30.841 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:19:30.841 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-16T00:19:30.846 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:19:30.846 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-16T00:19:30.847 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-16T00:19:30.847 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T00:19:30.848 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph Dashboard is now available at: 2024-09-16T00:19:30.848 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:30.848 INFO:teuthology.orchestra.run.smithi001.stdout: URL: https://smithi001.front.sepia.ceph.com:8443/ 2024-09-16T00:19:30.849 INFO:teuthology.orchestra.run.smithi001.stdout: User: admin 2024-09-16T00:19:30.849 INFO:teuthology.orchestra.run.smithi001.stdout: Password: rrr8touo3s 2024-09-16T00:19:30.849 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:30.849 INFO:teuthology.orchestra.run.smithi001.stdout:Saving cluster configuration to /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config directory 2024-09-16T00:19:32.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.266+0000 7f00f5d1d640 1 Processor -- start 2024-09-16T00:19:32.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.266+0000 7f00f5d1d640 1 -- start start 2024-09-16T00:19:32.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.266+0000 7f00f5d1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f0107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:32.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.267+0000 7f00f5d1d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0107f40 con 0x7f00f0107600 2024-09-16T00:19:32.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.267+0000 7f00ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f0107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:32.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.267+0000 7f00ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f0107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53278/0 (socket says 172.21.15.1:53278) 2024-09-16T00:19:32.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.267+0000 7f00ef7fe640 1 -- 172.21.15.1:0/4039167105 learned_addr learned my addr 172.21.15.1:0/4039167105 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:32.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.267+0000 7f00ef7fe640 1 -- 172.21.15.1:0/4039167105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f01080c0 con 0x7f00f0107600 2024-09-16T00:19:32.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.268+0000 7f00ef7fe640 1 --2- 172.21.15.1:0/4039167105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f0107a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009820 tx=0x7f00e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ddd5b8227f41fe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:32.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.268+0000 7f00ee7fc640 1 -- 172.21.15.1:0/4039167105 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e0037070 con 0x7f00f0107600 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.268+0000 7f00ee7fc640 1 -- 172.21.15.1:0/4039167105 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e0032920 con 0x7f00f0107600 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.268+0000 7f00ee7fc640 1 -- 172.21.15.1:0/4039167105 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e0030810 con 0x7f00f0107600 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.268+0000 7f00f5d1d640 1 -- 172.21.15.1:0/4039167105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 msgr2=0x7f00f0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.268+0000 7f00f5d1d640 1 --2- 172.21.15.1:0/4039167105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f0107a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009820 tx=0x7f00e002a270 comp rx=0 tx=0).stop 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.269+0000 7f00f5d1d640 1 -- 172.21.15.1:0/4039167105 shutdown_connections 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.269+0000 7f00f5d1d640 1 --2- 172.21.15.1:0/4039167105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f0107a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.269+0000 7f00f5d1d640 1 -- 172.21.15.1:0/4039167105 >> 172.21.15.1:0/4039167105 conn(0x7f00f0102e50 msgr2=0x7f00f0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.269+0000 7f00f5d1d640 1 -- 172.21.15.1:0/4039167105 shutdown_connections 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.269+0000 7f00f5d1d640 1 -- 172.21.15.1:0/4039167105 wait complete. 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.269+0000 7f00f5d1d640 1 Processor -- start 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.269+0000 7f00f5d1d640 1 -- start start 2024-09-16T00:19:32.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00f5d1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:32.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00f5d1d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00f010ba70 con 0x7f00f0107600 2024-09-16T00:19:32.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:32.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f019b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53288/0 (socket says 172.21.15.1:53288) 2024-09-16T00:19:32.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00ef7fe640 1 -- 172.21.15.1:0/3186580877 learned_addr learned my addr 172.21.15.1:0/3186580877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:32.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00ef7fe640 1 -- 172.21.15.1:0/3186580877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f019bae0 con 0x7f00f0107600 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00ef7fe640 1 --2- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f019b5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00e00097f0 tx=0x7f00e00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00ecff9640 1 -- 172.21.15.1:0/3186580877 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e0037040 con 0x7f00f0107600 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00ecff9640 1 -- 172.21.15.1:0/3186580877 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e0030cc0 con 0x7f00f0107600 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f019bd70 con 0x7f00f0107600 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00ecff9640 1 -- 172.21.15.1:0/3186580877 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e003a6b0 con 0x7f00f0107600 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.270+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f019ea60 con 0x7f00f0107600 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.271+0000 7f00ecff9640 1 -- 172.21.15.1:0/3186580877 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f00e003a8a0 con 0x7f00f0107600 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.271+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f0107a80 con 0x7f00f0107600 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.272+0000 7f00ecff9640 1 --2- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d80392a0 0x7f00d803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:32.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.272+0000 7f00ecff9640 1 -- 172.21.15.1:0/3186580877 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f00e006ca20 con 0x7f00f0107600 2024-09-16T00:19:32.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.272+0000 7f00eeffd640 1 --2- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d80392a0 0x7f00d803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:32.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.272+0000 7f00eeffd640 1 --2- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d80392a0 0x7f00d803b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f00e40098c0 tx=0x7f00e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:32.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.276+0000 7f00ecff9640 1 -- 172.21.15.1:0/3186580877 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00e003ace0 con 0x7f00f0107600 2024-09-16T00:19:32.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.547+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f00f019f050 con 0x7f00f0107600 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.550+0000 7f00ecff9640 1 -- 172.21.15.1:0/3186580877 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f00e002fd40 con 0x7f00f0107600 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d80392a0 msgr2=0x7f00d803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 --2- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d80392a0 0x7f00d803b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f00e40098c0 tx=0x7f00e4006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 msgr2=0x7f00f019b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 --2- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f019b5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00e00097f0 tx=0x7f00e00327b0 comp rx=0 tx=0).stop 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 shutdown_connections 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 --2- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d80392a0 0x7f00d803b760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 --2- 172.21.15.1:0/3186580877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f0107600 0x7f00f019b5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:32.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 >> 172.21.15.1:0/3186580877 conn(0x7f00f0102e50 msgr2=0x7f00f0103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:32.138 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 shutdown_connections 2024-09-16T00:19:32.138 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-16T00:19:31.552+0000 7f00f5d1d640 1 -- 172.21.15.1:0/3186580877 wait complete. 2024-09-16T00:19:32.138 INFO:teuthology.orchestra.run.smithi001.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T00:19:32.138 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:32.138 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:32.138 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:32.138 INFO:teuthology.orchestra.run.smithi001.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T00:19:32.138 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: ceph telemetry on 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout:For more information see: 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:32.139 INFO:teuthology.orchestra.run.smithi001.stdout:Bootstrap complete. 2024-09-16T00:19:32.174 INFO:tasks.cephadm:Fetching config... 2024-09-16T00:19:32.174 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:19:32.174 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T00:19:32.192 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T00:19:32.192 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:19:32.192 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T00:19:32.249 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T00:19:32.249 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:19:32.249 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T00:19:32.318 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T00:19:32.318 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:19:32.318 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T00:19:32.374 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T00:19:32.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXKnCiNbBhTz86bZT9/jVY9mHQ8rixfQ9eeVkpuK1XexzgexIEPfLypnEQo7XXjCtDEjpWv+iEBMIf43kdyS1cdA297BANnNSh6XCB2/B6YWFz+fA/wr7GtrzXFOq1jrZ3uH2XhqOnUnCLklvFrHQAmzoPx5fhoBLf3snB8Zd+fZNuxvtEnFaD430PqqFq3He9MjdgRy8H/uYFZVrk3J9AbdLYBIgTSyqwkwOpb8+GEcso9lBaJPFupH8den0NmZnGAGzjKjavAvdwd5sCbXVUb6UbDzdNOIbyEhdGO84gN1DHc1uKfx2nT90TtPCyj9u+HRj2D917qpFm7PLV6agqGGEmMXgbcNxm4CjrZiyH+Hjlr6iEYxc/+V38NFIEX/H9fGsYnkUF1XlJnmSAi7yEC0gStovmTNphvLdhuHXjSo/ByIDhuToIRHqIg40SE5X8s1jN+WYiMVhUE+eQokO081+HuVGNI9Kc0fiAVa5/EM18SLMhmH0DoHIeoadCAQk= ceph-1226ed86-73c1-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:19:32.474 INFO:teuthology.orchestra.run.smithi001.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXKnCiNbBhTz86bZT9/jVY9mHQ8rixfQ9eeVkpuK1XexzgexIEPfLypnEQo7XXjCtDEjpWv+iEBMIf43kdyS1cdA297BANnNSh6XCB2/B6YWFz+fA/wr7GtrzXFOq1jrZ3uH2XhqOnUnCLklvFrHQAmzoPx5fhoBLf3snB8Zd+fZNuxvtEnFaD430PqqFq3He9MjdgRy8H/uYFZVrk3J9AbdLYBIgTSyqwkwOpb8+GEcso9lBaJPFupH8den0NmZnGAGzjKjavAvdwd5sCbXVUb6UbDzdNOIbyEhdGO84gN1DHc1uKfx2nT90TtPCyj9u+HRj2D917qpFm7PLV6agqGGEmMXgbcNxm4CjrZiyH+Hjlr6iEYxc/+V38NFIEX/H9fGsYnkUF1XlJnmSAi7yEC0gStovmTNphvLdhuHXjSo/ByIDhuToIRHqIg40SE5X8s1jN+WYiMVhUE+eQokO081+HuVGNI9Kc0fiAVa5/EM18SLMhmH0DoHIeoadCAQk= ceph-1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:19:32.490 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXKnCiNbBhTz86bZT9/jVY9mHQ8rixfQ9eeVkpuK1XexzgexIEPfLypnEQo7XXjCtDEjpWv+iEBMIf43kdyS1cdA297BANnNSh6XCB2/B6YWFz+fA/wr7GtrzXFOq1jrZ3uH2XhqOnUnCLklvFrHQAmzoPx5fhoBLf3snB8Zd+fZNuxvtEnFaD430PqqFq3He9MjdgRy8H/uYFZVrk3J9AbdLYBIgTSyqwkwOpb8+GEcso9lBaJPFupH8den0NmZnGAGzjKjavAvdwd5sCbXVUb6UbDzdNOIbyEhdGO84gN1DHc1uKfx2nT90TtPCyj9u+HRj2D917qpFm7PLV6agqGGEmMXgbcNxm4CjrZiyH+Hjlr6iEYxc/+V38NFIEX/H9fGsYnkUF1XlJnmSAi7yEC0gStovmTNphvLdhuHXjSo/ByIDhuToIRHqIg40SE5X8s1jN+WYiMVhUE+eQokO081+HuVGNI9Kc0fiAVa5/EM18SLMhmH0DoHIeoadCAQk= ceph-1226ed86-73c1-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:19:32.530 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXKnCiNbBhTz86bZT9/jVY9mHQ8rixfQ9eeVkpuK1XexzgexIEPfLypnEQo7XXjCtDEjpWv+iEBMIf43kdyS1cdA297BANnNSh6XCB2/B6YWFz+fA/wr7GtrzXFOq1jrZ3uH2XhqOnUnCLklvFrHQAmzoPx5fhoBLf3snB8Zd+fZNuxvtEnFaD430PqqFq3He9MjdgRy8H/uYFZVrk3J9AbdLYBIgTSyqwkwOpb8+GEcso9lBaJPFupH8den0NmZnGAGzjKjavAvdwd5sCbXVUb6UbDzdNOIbyEhdGO84gN1DHc1uKfx2nT90TtPCyj9u+HRj2D917qpFm7PLV6agqGGEmMXgbcNxm4CjrZiyH+Hjlr6iEYxc/+V38NFIEX/H9fGsYnkUF1XlJnmSAi7yEC0gStovmTNphvLdhuHXjSo/ByIDhuToIRHqIg40SE5X8s1jN+WYiMVhUE+eQokO081+HuVGNI9Kc0fiAVa5/EM18SLMhmH0DoHIeoadCAQk= ceph-1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:19:32.543 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXKnCiNbBhTz86bZT9/jVY9mHQ8rixfQ9eeVkpuK1XexzgexIEPfLypnEQo7XXjCtDEjpWv+iEBMIf43kdyS1cdA297BANnNSh6XCB2/B6YWFz+fA/wr7GtrzXFOq1jrZ3uH2XhqOnUnCLklvFrHQAmzoPx5fhoBLf3snB8Zd+fZNuxvtEnFaD430PqqFq3He9MjdgRy8H/uYFZVrk3J9AbdLYBIgTSyqwkwOpb8+GEcso9lBaJPFupH8den0NmZnGAGzjKjavAvdwd5sCbXVUb6UbDzdNOIbyEhdGO84gN1DHc1uKfx2nT90TtPCyj9u+HRj2D917qpFm7PLV6agqGGEmMXgbcNxm4CjrZiyH+Hjlr6iEYxc/+V38NFIEX/H9fGsYnkUF1XlJnmSAi7yEC0gStovmTNphvLdhuHXjSo/ByIDhuToIRHqIg40SE5X8s1jN+WYiMVhUE+eQokO081+HuVGNI9Kc0fiAVa5/EM18SLMhmH0DoHIeoadCAQk= ceph-1226ed86-73c1-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:19:32.586 INFO:teuthology.orchestra.run.smithi153.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXKnCiNbBhTz86bZT9/jVY9mHQ8rixfQ9eeVkpuK1XexzgexIEPfLypnEQo7XXjCtDEjpWv+iEBMIf43kdyS1cdA297BANnNSh6XCB2/B6YWFz+fA/wr7GtrzXFOq1jrZ3uH2XhqOnUnCLklvFrHQAmzoPx5fhoBLf3snB8Zd+fZNuxvtEnFaD430PqqFq3He9MjdgRy8H/uYFZVrk3J9AbdLYBIgTSyqwkwOpb8+GEcso9lBaJPFupH8den0NmZnGAGzjKjavAvdwd5sCbXVUb6UbDzdNOIbyEhdGO84gN1DHc1uKfx2nT90TtPCyj9u+HRj2D917qpFm7PLV6agqGGEmMXgbcNxm4CjrZiyH+Hjlr6iEYxc/+V38NFIEX/H9fGsYnkUF1XlJnmSAi7yEC0gStovmTNphvLdhuHXjSo/ByIDhuToIRHqIg40SE5X8s1jN+WYiMVhUE+eQokO081+HuVGNI9Kc0fiAVa5/EM18SLMhmH0DoHIeoadCAQk= ceph-1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:19:32.599 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T00:19:32.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:32 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3186580877' entity='client.admin' 2024-09-16T00:19:32.854 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:33.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.323+0000 7f93949c2640 1 -- 172.21.15.1:0/2317872719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 msgr2=0x7f93901019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:33.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.323+0000 7f93949c2640 1 --2- 172.21.15.1:0/2317872719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 0x7f93901019e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9380009880 tx=0x7f938002a5d0 comp rx=0 tx=0).stop 2024-09-16T00:19:33.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.323+0000 7f93949c2640 1 -- 172.21.15.1:0/2317872719 shutdown_connections 2024-09-16T00:19:33.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.323+0000 7f93949c2640 1 --2- 172.21.15.1:0/2317872719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 0x7f93901019e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:33.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.323+0000 7f93949c2640 1 -- 172.21.15.1:0/2317872719 >> 172.21.15.1:0/2317872719 conn(0x7f93900fcd70 msgr2=0x7f93900ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.323+0000 7f93949c2640 1 -- 172.21.15.1:0/2317872719 shutdown_connections 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.323+0000 7f93949c2640 1 -- 172.21.15.1:0/2317872719 wait complete. 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.324+0000 7f93949c2640 1 Processor -- start 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.324+0000 7f93949c2640 1 -- start start 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.324+0000 7f93949c2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 0x7f9390196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.324+0000 7f93949c2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f93901076f0 con 0x7f93901015e0 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.324+0000 7f938e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 0x7f9390196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.324+0000 7f938e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 0x7f9390196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53300/0 (socket says 172.21.15.1:53300) 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.324+0000 7f938e59c640 1 -- 172.21.15.1:0/2964078810 learned_addr learned my addr 172.21.15.1:0/2964078810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.324+0000 7f938e59c640 1 -- 172.21.15.1:0/2964078810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93901972a0 con 0x7f93901015e0 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.324+0000 7f938e59c640 1 --2- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 0x7f9390196d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9380004f60 tx=0x7f93800048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.325+0000 7f937f7fe640 1 -- 172.21.15.1:0/2964078810 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f938003f070 con 0x7f93901015e0 2024-09-16T00:19:33.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.325+0000 7f937f7fe640 1 -- 172.21.15.1:0/2964078810 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9380004cc0 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.325+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9390197530 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.325+0000 7f937f7fe640 1 -- 172.21.15.1:0/2964078810 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9380035030 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.325+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939019a220 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.325+0000 7f937f7fe640 1 -- 172.21.15.1:0/2964078810 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f9380042070 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.326+0000 7f937f7fe640 1 --2- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93780392a0 0x7f937803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.326+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9390107e70 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.326+0000 7f937f7fe640 1 -- 172.21.15.1:0/2964078810 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f938006e0c0 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.326+0000 7f938dd9b640 1 --2- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93780392a0 0x7f937803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.326+0000 7f938dd9b640 1 --2- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93780392a0 0x7f937803b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f93840098c0 tx=0x7f9384006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.331+0000 7f937f7fe640 1 -- 172.21.15.1:0/2964078810 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93800433b0 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.510+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9390101a60 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.513+0000 7f937f7fe640 1 -- 172.21.15.1:0/2964078810 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9390101a60 con 0x7f93901015e0 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93780392a0 msgr2=0x7f937803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:33.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 --2- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93780392a0 0x7f937803b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f93840098c0 tx=0x7f9384006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:33.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 msgr2=0x7f9390196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:33.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 --2- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 0x7f9390196d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9380004f60 tx=0x7f93800048f0 comp rx=0 tx=0).stop 2024-09-16T00:19:33.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 shutdown_connections 2024-09-16T00:19:33.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 --2- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93780392a0 0x7f937803b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:33.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 --2- 172.21.15.1:0/2964078810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93901015e0 0x7f9390196d60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:33.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 >> 172.21.15.1:0/2964078810 conn(0x7f93900fcd70 msgr2=0x7f93900fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:33.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 shutdown_connections 2024-09-16T00:19:33.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:33.515+0000 7f93949c2640 1 -- 172.21.15.1:0/2964078810 wait complete. 2024-09-16T00:19:34.295 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T00:19:34.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T00:19:34.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:34 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2964078810' entity='client.admin' 2024-09-16T00:19:34.786 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:35.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.225+0000 7f4909ea6640 1 -- 172.21.15.1:0/4059426163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 msgr2=0x7f4904101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.225+0000 7f4909ea6640 1 --2- 172.21.15.1:0/4059426163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 0x7f4904101a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009880 tx=0x7f48f402a5d0 comp rx=0 tx=0).stop 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.225+0000 7f4909ea6640 1 -- 172.21.15.1:0/4059426163 shutdown_connections 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.225+0000 7f4909ea6640 1 --2- 172.21.15.1:0/4059426163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 0x7f4904101a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.225+0000 7f4909ea6640 1 -- 172.21.15.1:0/4059426163 >> 172.21.15.1:0/4059426163 conn(0x7f49040fcdb0 msgr2=0x7f49040ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.225+0000 7f4909ea6640 1 -- 172.21.15.1:0/4059426163 shutdown_connections 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.225+0000 7f4909ea6640 1 -- 172.21.15.1:0/4059426163 wait complete. 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.226+0000 7f4909ea6640 1 Processor -- start 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.226+0000 7f4909ea6640 1 -- start start 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.226+0000 7f4909ea6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 0x7f4904075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:35.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.226+0000 7f4909ea6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4904107710 con 0x7f4904101600 2024-09-16T00:19:35.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f49037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 0x7f4904075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:35.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f49037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 0x7f4904075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53324/0 (socket says 172.21.15.1:53324) 2024-09-16T00:19:35.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f49037fe640 1 -- 172.21.15.1:0/1141012225 learned_addr learned my addr 172.21.15.1:0/1141012225 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:35.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f49037fe640 1 -- 172.21.15.1:0/1141012225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49040763f0 con 0x7f4904101600 2024-09-16T00:19:35.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f49037fe640 1 --2- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 0x7f4904075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48f4004f60 tx=0x7f48f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:35.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f4900ff9640 1 -- 172.21.15.1:0/1141012225 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48f403f070 con 0x7f4904101600 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f4900ff9640 1 -- 172.21.15.1:0/1141012225 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f4004cc0 con 0x7f4904101600 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49040745e0 con 0x7f4904101600 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4904074b20 con 0x7f4904101600 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.227+0000 7f4900ff9640 1 -- 172.21.15.1:0/1141012225 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48f4035030 con 0x7f4904101600 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.228+0000 7f4900ff9640 1 -- 172.21.15.1:0/1141012225 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f48f4042070 con 0x7f4904101600 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.228+0000 7f4900ff9640 1 --2- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48ec039300 0x7f48ec03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.228+0000 7f4900ff9640 1 -- 172.21.15.1:0/1141012225 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48f406cff0 con 0x7f4904101600 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.228+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4904107e90 con 0x7f4904101600 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.229+0000 7f4902ffd640 1 --2- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48ec039300 0x7f48ec03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.229+0000 7f4902ffd640 1 --2- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48ec039300 0x7f48ec03b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48f80098c0 tx=0x7f48f8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.233+0000 7f4900ff9640 1 -- 172.21.15.1:0/1141012225 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48f4003dd0 con 0x7f4904101600 2024-09-16T00:19:35.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.415+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f49041900c0 con 0x7f48ec039300 2024-09-16T00:19:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.419+0000 7f4900ff9640 1 -- 172.21.15.1:0/1141012225 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f49041900c0 con 0x7f48ec039300 2024-09-16T00:19:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48ec039300 msgr2=0x7f48ec03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 --2- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48ec039300 0x7f48ec03b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48f80098c0 tx=0x7f48f8006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 msgr2=0x7f4904075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 --2- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 0x7f4904075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48f4004f60 tx=0x7f48f40048f0 comp rx=0 tx=0).stop 2024-09-16T00:19:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 shutdown_connections 2024-09-16T00:19:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 --2- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48ec039300 0x7f48ec03b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 --2- 172.21.15.1:0/1141012225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4904101600 0x7f4904075eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 >> 172.21.15.1:0/1141012225 conn(0x7f49040fcdb0 msgr2=0x7f49040fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 shutdown_connections 2024-09-16T00:19:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:35.421+0000 7f4909ea6640 1 -- 172.21.15.1:0/1141012225 wait complete. 2024-09-16T00:19:36.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:36.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:36.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi001", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:19:36.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:36.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:36.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:36.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:36.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:36.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:36.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:36.092 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi019 2024-09-16T00:19:36.092 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:19:36.092 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/etc/ceph/ceph.conf 2024-09-16T00:19:36.112 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:19:36.112 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:36.170 INFO:tasks.cephadm:Adding host smithi019 to orchestrator... 2024-09-16T00:19:36.170 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch host add smithi019 2024-09-16T00:19:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:36.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:36 smithi001 ceph-mon[30934]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:36.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:36 smithi001 ceph-mon[30934]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-16T00:19:36.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:36 smithi001 ceph-mon[30934]: Updating smithi001:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:36.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:36 smithi001 ceph-mon[30934]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:36.809 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:36 smithi001 ceph-mon[30934]: Updating smithi001:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:19:36.809 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:36.809 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:36.809 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.892+0000 7f12281c6640 1 -- 172.21.15.1:0/3925111065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 msgr2=0x7f1220100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.892+0000 7f12281c6640 1 --2- 172.21.15.1:0/3925111065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 0x7f1220100920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f121c009880 tx=0x7f121c02a670 comp rx=0 tx=0).stop 2024-09-16T00:19:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.892+0000 7f12281c6640 1 -- 172.21.15.1:0/3925111065 shutdown_connections 2024-09-16T00:19:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.892+0000 7f12281c6640 1 --2- 172.21.15.1:0/3925111065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 0x7f1220100920 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.892+0000 7f12281c6640 1 -- 172.21.15.1:0/3925111065 >> 172.21.15.1:0/3925111065 conn(0x7f12200fc1b0 msgr2=0x7f12200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.892+0000 7f12281c6640 1 -- 172.21.15.1:0/3925111065 shutdown_connections 2024-09-16T00:19:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.892+0000 7f12281c6640 1 -- 172.21.15.1:0/3925111065 wait complete. 2024-09-16T00:19:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.892+0000 7f12281c6640 1 Processor -- start 2024-09-16T00:19:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.892+0000 7f12281c6640 1 -- start start 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f12281c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 0x7f12201a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f12281c6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12201122d0 con 0x7f1220100540 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f1225f62640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 0x7f12201a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f1225f62640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 0x7f12201a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53346/0 (socket says 172.21.15.1:53346) 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f1225f62640 1 -- 172.21.15.1:0/1404022760 learned_addr learned my addr 172.21.15.1:0/1404022760 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f1225f62640 1 -- 172.21.15.1:0/1404022760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12201a3980 con 0x7f1220100540 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f1225f62640 1 --2- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 0x7f12201a3440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f121c009230 tx=0x7f121c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f1216ffd640 1 -- 172.21.15.1:0/1404022760 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f121c03f070 con 0x7f1220100540 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12201a3c10 con 0x7f1220100540 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f1216ffd640 1 -- 172.21.15.1:0/1404022760 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f121c03bc80 con 0x7f1220100540 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f1216ffd640 1 -- 172.21.15.1:0/1404022760 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f121c035030 con 0x7f1220100540 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.893+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122019d600 con 0x7f1220100540 2024-09-16T00:19:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.894+0000 7f1216ffd640 1 -- 172.21.15.1:0/1404022760 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f121c042070 con 0x7f1220100540 2024-09-16T00:19:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.894+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122006a7b0 con 0x7f1220100540 2024-09-16T00:19:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.894+0000 7f1216ffd640 1 --2- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f120c0392a0 0x7f120c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.895+0000 7f1216ffd640 1 -- 172.21.15.1:0/1404022760 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f121c06d0a0 con 0x7f1220100540 2024-09-16T00:19:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.895+0000 7f1225761640 1 --2- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f120c0392a0 0x7f120c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.895+0000 7f1225761640 1 --2- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f120c0392a0 0x7f120c03b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f12100098c0 tx=0x7f1210006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:36.899+0000 7f1216ffd640 1 -- 172.21.15.1:0/1404022760 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f121c031e20 con 0x7f1220100540 2024-09-16T00:19:37.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:37.080+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "target": ["mon-mgr", ""]}) -- 0x7f122019eb30 con 0x7f120c0392a0 2024-09-16T00:19:38.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:37 smithi001 ceph-mon[30934]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:39.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:39 smithi001 ceph-mon[30934]: Deploying cephadm binary to smithi019 2024-09-16T00:19:40.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.968+0000 7f1216ffd640 1 -- 172.21.15.1:0/1404022760 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f122019eb30 con 0x7f120c0392a0 2024-09-16T00:19:40.969 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi019' with addr '172.21.15.19' 2024-09-16T00:19:40.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.971+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f120c0392a0 msgr2=0x7f120c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:40.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.971+0000 7f12281c6640 1 --2- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f120c0392a0 0x7f120c03b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f12100098c0 tx=0x7f1210006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:40.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.972+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 msgr2=0x7f12201a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:40.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.972+0000 7f12281c6640 1 --2- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 0x7f12201a3440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f121c009230 tx=0x7f121c003d40 comp rx=0 tx=0).stop 2024-09-16T00:19:40.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.972+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 shutdown_connections 2024-09-16T00:19:40.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.972+0000 7f12281c6640 1 --2- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f120c0392a0 0x7f120c03b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:40.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.972+0000 7f12281c6640 1 --2- 172.21.15.1:0/1404022760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1220100540 0x7f12201a3440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:40.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.972+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 >> 172.21.15.1:0/1404022760 conn(0x7f12200fc1b0 msgr2=0x7f12200fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:40.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.972+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 shutdown_connections 2024-09-16T00:19:40.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:40.972+0000 7f12281c6640 1 -- 172.21.15.1:0/1404022760 wait complete. 2024-09-16T00:19:41.582 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T00:19:41.834 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:42.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:42.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:41 smithi001 ceph-mon[30934]: Added host smithi019 2024-09-16T00:19:42.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:42.198 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:42.198 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:42.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.280+0000 7f1df327e640 1 -- 172.21.15.1:0/3456650365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 msgr2=0x7f1dec107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:42.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.280+0000 7f1df327e640 1 --2- 172.21.15.1:0/3456650365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 0x7f1dec107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009880 tx=0x7f1de802a5d0 comp rx=0 tx=0).stop 2024-09-16T00:19:42.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.280+0000 7f1df327e640 1 -- 172.21.15.1:0/3456650365 shutdown_connections 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.280+0000 7f1df327e640 1 --2- 172.21.15.1:0/3456650365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 0x7f1dec107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.280+0000 7f1df327e640 1 -- 172.21.15.1:0/3456650365 >> 172.21.15.1:0/3456650365 conn(0x7f1dec0fd290 msgr2=0x7f1dec0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.280+0000 7f1df327e640 1 -- 172.21.15.1:0/3456650365 shutdown_connections 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.280+0000 7f1df327e640 1 -- 172.21.15.1:0/3456650365 wait complete. 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.281+0000 7f1df327e640 1 Processor -- start 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.281+0000 7f1df327e640 1 -- start start 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.281+0000 7f1df327e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 0x7f1dec1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.281+0000 7f1df327e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec10bc90 con 0x7f1dec107660 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.281+0000 7f1df101a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 0x7f1dec1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.281+0000 7f1df101a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 0x7f1dec1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52788/0 (socket says 172.21.15.1:52788) 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.281+0000 7f1df101a640 1 -- 172.21.15.1:0/1538804386 learned_addr learned my addr 172.21.15.1:0/1538804386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:42.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.282+0000 7f1df101a640 1 -- 172.21.15.1:0/1538804386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec197630 con 0x7f1dec107660 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.282+0000 7f1df101a640 1 --2- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 0x7f1dec1970f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009230 tx=0x7f1de8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.282+0000 7f1de27fc640 1 -- 172.21.15.1:0/1538804386 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1de803f070 con 0x7f1dec107660 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.282+0000 7f1de27fc640 1 -- 172.21.15.1:0/1538804386 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de803bc80 con 0x7f1dec107660 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.282+0000 7f1de27fc640 1 -- 172.21.15.1:0/1538804386 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1de8035030 con 0x7f1dec107660 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.282+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dec1978c0 con 0x7f1dec107660 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.282+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dec19b9c0 con 0x7f1dec107660 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.283+0000 7f1de27fc640 1 -- 172.21.15.1:0/1538804386 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f1de8042070 con 0x7f1dec107660 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.283+0000 7f1de27fc640 1 --2- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dd8039300 0x7f1dd803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.283+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dec102cd0 con 0x7f1dec107660 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.283+0000 7f1de27fc640 1 -- 172.21.15.1:0/1538804386 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1de806cfb0 con 0x7f1dec107660 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.283+0000 7f1df0819640 1 --2- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dd8039300 0x7f1dd803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.283+0000 7f1df0819640 1 --2- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dd8039300 0x7f1dd803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0098c0 tx=0x7f1ddc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.288+0000 7f1de27fc640 1 -- 172.21.15.1:0/1538804386 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1de80372d0 con 0x7f1dec107660 2024-09-16T00:19:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.469+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1dec198450 con 0x7f1dd8039300 2024-09-16T00:19:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.470+0000 7f1de27fc640 1 -- 172.21.15.1:0/1538804386 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f1dec198450 con 0x7f1dd8039300 2024-09-16T00:19:42.471 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:42.471 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}] 2024-09-16T00:19:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.472+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dd8039300 msgr2=0x7f1dd803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.472+0000 7f1df327e640 1 --2- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dd8039300 0x7f1dd803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0098c0 tx=0x7f1ddc006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.472+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 msgr2=0x7f1dec1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.472+0000 7f1df327e640 1 --2- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 0x7f1dec1970f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009230 tx=0x7f1de8003d40 comp rx=0 tx=0).stop 2024-09-16T00:19:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.472+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 shutdown_connections 2024-09-16T00:19:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.473+0000 7f1df327e640 1 --2- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dd8039300 0x7f1dd803b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.473+0000 7f1df327e640 1 --2- 172.21.15.1:0/1538804386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107660 0x7f1dec1970f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.473+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 >> 172.21.15.1:0/1538804386 conn(0x7f1dec0fd290 msgr2=0x7f1dec0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.473+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 shutdown_connections 2024-09-16T00:19:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:42.473+0000 7f1df327e640 1 -- 172.21.15.1:0/1538804386 wait complete. 2024-09-16T00:19:43.067 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi153 2024-09-16T00:19:43.067 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:19:43.068 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.conf 2024-09-16T00:19:43.086 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:19:43.087 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:43.145 INFO:tasks.cephadm:Adding host smithi153 to orchestrator... 2024-09-16T00:19:43.145 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch host add smithi153 2024-09-16T00:19:43.401 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:43.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:43.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:43 smithi001 ceph-mon[30934]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:19:43.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.828+0000 7f9412f4f640 1 -- 172.21.15.1:0/4017192862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 msgr2=0x7f940c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.828+0000 7f9412f4f640 1 --2- 172.21.15.1:0/4017192862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 0x7f940c107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009880 tx=0x7f93fc02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.828+0000 7f9412f4f640 1 -- 172.21.15.1:0/4017192862 shutdown_connections 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.828+0000 7f9412f4f640 1 --2- 172.21.15.1:0/4017192862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 0x7f940c107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.828+0000 7f9412f4f640 1 -- 172.21.15.1:0/4017192862 >> 172.21.15.1:0/4017192862 conn(0x7f940c0fd290 msgr2=0x7f940c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.828+0000 7f9412f4f640 1 -- 172.21.15.1:0/4017192862 shutdown_connections 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.828+0000 7f9412f4f640 1 -- 172.21.15.1:0/4017192862 wait complete. 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.829+0000 7f9412f4f640 1 Processor -- start 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.829+0000 7f9412f4f640 1 -- start start 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.829+0000 7f9412f4f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 0x7f940c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.829+0000 7f9412f4f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f940c10bc90 con 0x7f940c107660 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.829+0000 7f9410ceb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 0x7f940c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:43.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.829+0000 7f9410ceb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 0x7f940c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52804/0 (socket says 172.21.15.1:52804) 2024-09-16T00:19:43.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.829+0000 7f9410ceb640 1 -- 172.21.15.1:0/499554405 learned_addr learned my addr 172.21.15.1:0/499554405 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:43.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.830+0000 7f9410ceb640 1 -- 172.21.15.1:0/499554405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940c0763e0 con 0x7f940c107660 2024-09-16T00:19:43.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.830+0000 7f9410ceb640 1 --2- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 0x7f940c0744f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009230 tx=0x7f93fc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:43.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.830+0000 7f9409ffb640 1 -- 172.21.15.1:0/499554405 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93fc03f070 con 0x7f940c107660 2024-09-16T00:19:43.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.830+0000 7f9409ffb640 1 -- 172.21.15.1:0/499554405 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93fc03bc80 con 0x7f940c107660 2024-09-16T00:19:43.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.830+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940c074a30 con 0x7f940c107660 2024-09-16T00:19:43.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.830+0000 7f9409ffb640 1 -- 172.21.15.1:0/499554405 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93fc035030 con 0x7f940c107660 2024-09-16T00:19:43.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.830+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f940c074ef0 con 0x7f940c107660 2024-09-16T00:19:43.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.831+0000 7f9409ffb640 1 -- 172.21.15.1:0/499554405 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f93fc042070 con 0x7f940c107660 2024-09-16T00:19:43.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.831+0000 7f9409ffb640 1 --2- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f8039300 0x7f93f803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:43.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.831+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f940c102cd0 con 0x7f940c107660 2024-09-16T00:19:43.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.831+0000 7f9409ffb640 1 -- 172.21.15.1:0/499554405 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f93fc06cfe0 con 0x7f940c107660 2024-09-16T00:19:43.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.831+0000 7f940bfff640 1 --2- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f8039300 0x7f93f803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:43.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.832+0000 7f940bfff640 1 --2- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f8039300 0x7f93f803b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098c0 tx=0x7f93f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:43.836+0000 7f9409ffb640 1 -- 172.21.15.1:0/499554405 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93fc0372d0 con 0x7f940c107660 2024-09-16T00:19:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:44.014+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}) -- 0x7f940c075a20 con 0x7f93f8039300 2024-09-16T00:19:44.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:44 smithi001 ceph-mon[30934]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:46.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:47.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:47 smithi001 ceph-mon[30934]: Deploying cephadm binary to smithi153 2024-09-16T00:19:47.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:47 smithi001 ceph-mon[30934]: Updating smithi019:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:47.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:47 smithi001 ceph-mon[30934]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:47.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:47 smithi001 ceph-mon[30934]: Updating smithi019:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:19:48.044 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi153' with addr '172.21.15.153' 2024-09-16T00:19:48.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.043+0000 7f9409ffb640 1 -- 172.21.15.1:0/499554405 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f940c075a20 con 0x7f93f8039300 2024-09-16T00:19:48.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f8039300 msgr2=0x7f93f803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 --2- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f8039300 0x7f93f803b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098c0 tx=0x7f93f4006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 msgr2=0x7f940c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 --2- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 0x7f940c0744f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009230 tx=0x7f93fc003d40 comp rx=0 tx=0).stop 2024-09-16T00:19:48.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 shutdown_connections 2024-09-16T00:19:48.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 --2- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f8039300 0x7f93f803b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 --2- 172.21.15.1:0/499554405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f940c107660 0x7f940c0744f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 >> 172.21.15.1:0/499554405 conn(0x7f940c0fd290 msgr2=0x7f940c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:48.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 shutdown_connections 2024-09-16T00:19:48.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:48.046+0000 7f9412f4f640 1 -- 172.21.15.1:0/499554405 wait complete. 2024-09-16T00:19:48.342 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:48 smithi001 ceph-mon[30934]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:48.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:48 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:48.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:48 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:48.655 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T00:19:48.909 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:49.319 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:49 smithi001 ceph-mon[30934]: Added host smithi153 2024-09-16T00:19:49.319 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:49.319 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:49.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.402+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2376356554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 msgr2=0x7f1a14107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:49.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.402+0000 7f1a1c7eb640 1 --2- 172.21.15.1:0/2376356554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 0x7f1a14107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009880 tx=0x7f1a1002a5d0 comp rx=0 tx=0).stop 2024-09-16T00:19:49.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.402+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2376356554 shutdown_connections 2024-09-16T00:19:49.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.402+0000 7f1a1c7eb640 1 --2- 172.21.15.1:0/2376356554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 0x7f1a14107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.402+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2376356554 >> 172.21.15.1:0/2376356554 conn(0x7f1a140fd290 msgr2=0x7f1a140ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.402+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2376356554 shutdown_connections 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.402+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2376356554 wait complete. 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1c7eb640 1 Processor -- start 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1c7eb640 1 -- start start 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1c7eb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 0x7f1a141970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1c7eb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1410bc90 con 0x7f1a14107660 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1a587640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 0x7f1a141970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1a587640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 0x7f1a141970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52822/0 (socket says 172.21.15.1:52822) 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1a587640 1 -- 172.21.15.1:0/2414283228 learned_addr learned my addr 172.21.15.1:0/2414283228 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1a587640 1 -- 172.21.15.1:0/2414283228 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a14197630 con 0x7f1a14107660 2024-09-16T00:19:49.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1a587640 1 --2- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 0x7f1a141970f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009230 tx=0x7f1a10003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a0b7fe640 1 -- 172.21.15.1:0/2414283228 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a1003f070 con 0x7f1a14107660 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a0b7fe640 1 -- 172.21.15.1:0/2414283228 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1003bc80 con 0x7f1a14107660 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.403+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a141978c0 con 0x7f1a14107660 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.404+0000 7f1a0b7fe640 1 -- 172.21.15.1:0/2414283228 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a10035030 con 0x7f1a14107660 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.404+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1419b9c0 con 0x7f1a14107660 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.404+0000 7f1a0b7fe640 1 -- 172.21.15.1:0/2414283228 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f1a10042070 con 0x7f1a14107660 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.405+0000 7f1a0b7fe640 1 --2- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1a00039300 0x7f1a0003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.405+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a14102cd0 con 0x7f1a14107660 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.405+0000 7f1a0b7fe640 1 -- 172.21.15.1:0/2414283228 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1a1006cff0 con 0x7f1a14107660 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.405+0000 7f1a19d86640 1 --2- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1a00039300 0x7f1a0003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.405+0000 7f1a19d86640 1 --2- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1a00039300 0x7f1a0003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a040098c0 tx=0x7f1a04006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:49.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.410+0000 7f1a0b7fe640 1 -- 172.21.15.1:0/2414283228 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a100372d0 con 0x7f1a14107660 2024-09-16T00:19:49.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.591+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a14198450 con 0x7f1a00039300 2024-09-16T00:19:49.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.592+0000 7f1a0b7fe640 1 -- 172.21.15.1:0/2414283228 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f1a14198450 con 0x7f1a00039300 2024-09-16T00:19:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:19:49.592 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}] 2024-09-16T00:19:49.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1a00039300 msgr2=0x7f1a0003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:49.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 --2- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1a00039300 0x7f1a0003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a040098c0 tx=0x7f1a04006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:49.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 msgr2=0x7f1a141970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:49.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 --2- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 0x7f1a141970f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009230 tx=0x7f1a10003d40 comp rx=0 tx=0).stop 2024-09-16T00:19:49.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 shutdown_connections 2024-09-16T00:19:49.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 --2- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1a00039300 0x7f1a0003b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:49.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 --2- 172.21.15.1:0/2414283228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a14107660 0x7f1a141970f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:49.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 >> 172.21.15.1:0/2414283228 conn(0x7f1a140fd290 msgr2=0x7f1a140febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:49.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 shutdown_connections 2024-09-16T00:19:49.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:49.594+0000 7f1a1c7eb640 1 -- 172.21.15.1:0/2414283228 wait complete. 2024-09-16T00:19:50.173 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:50 smithi001 ceph-mon[30934]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:50.173 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:50 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:50.204 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T00:19:50.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T00:19:50.457 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:50.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.884+0000 7f40f5dd5640 1 -- 172.21.15.1:0/1789535513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 msgr2=0x7f40f0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:50.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.884+0000 7f40f5dd5640 1 --2- 172.21.15.1:0/1789535513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 0x7f40f0107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009880 tx=0x7f40e002a5d0 comp rx=0 tx=0).stop 2024-09-16T00:19:50.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.884+0000 7f40f5dd5640 1 -- 172.21.15.1:0/1789535513 shutdown_connections 2024-09-16T00:19:50.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.884+0000 7f40f5dd5640 1 --2- 172.21.15.1:0/1789535513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 0x7f40f0107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:50.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.884+0000 7f40f5dd5640 1 -- 172.21.15.1:0/1789535513 >> 172.21.15.1:0/1789535513 conn(0x7f40f00fd290 msgr2=0x7f40f00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:50.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.884+0000 7f40f5dd5640 1 -- 172.21.15.1:0/1789535513 shutdown_connections 2024-09-16T00:19:50.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.884+0000 7f40f5dd5640 1 -- 172.21.15.1:0/1789535513 wait complete. 2024-09-16T00:19:50.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.885+0000 7f40f5dd5640 1 Processor -- start 2024-09-16T00:19:50.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.885+0000 7f40f5dd5640 1 -- start start 2024-09-16T00:19:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.885+0000 7f40f5dd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 0x7f40f0197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.885+0000 7f40f5dd5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f40f010bc90 con 0x7f40f0107660 2024-09-16T00:19:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.885+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 0x7f40f0197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.886+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 0x7f40f0197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40826/0 (socket says 172.21.15.1:40826) 2024-09-16T00:19:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.886+0000 7f40ef7fe640 1 -- 172.21.15.1:0/791580577 learned_addr learned my addr 172.21.15.1:0/791580577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.886+0000 7f40ef7fe640 1 -- 172.21.15.1:0/791580577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f0197660 con 0x7f40f0107660 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.886+0000 7f40ef7fe640 1 --2- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 0x7f40f0197120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009230 tx=0x7f40e0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.886+0000 7f40ed7fa640 1 -- 172.21.15.1:0/791580577 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40e003f070 con 0x7f40f0107660 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.886+0000 7f40ed7fa640 1 -- 172.21.15.1:0/791580577 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e003bc80 con 0x7f40f0107660 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.886+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f01978f0 con 0x7f40f0107660 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.886+0000 7f40ed7fa640 1 -- 172.21.15.1:0/791580577 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40e0035030 con 0x7f40f0107660 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.886+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f019b9f0 con 0x7f40f0107660 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.887+0000 7f40ed7fa640 1 -- 172.21.15.1:0/791580577 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f40e0042070 con 0x7f40f0107660 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.887+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f0102cd0 con 0x7f40f0107660 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.887+0000 7f40ed7fa640 1 --2- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40dc0392a0 0x7f40dc03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.887+0000 7f40ed7fa640 1 -- 172.21.15.1:0/791580577 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f40e006d1b0 con 0x7f40f0107660 2024-09-16T00:19:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.887+0000 7f40e6ffd640 1 --2- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40dc0392a0 0x7f40dc03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:50.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.888+0000 7f40e6ffd640 1 --2- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40dc0392a0 0x7f40dc03b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f40d80098c0 tx=0x7f40d8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:50.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:50.892+0000 7f40ed7fa640 1 -- 172.21.15.1:0/791580577 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40e004c350 con 0x7f40f0107660 2024-09-16T00:19:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.074+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f40f0107ac0 con 0x7f40f0107660 2024-09-16T00:19:51.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.145+0000 7f40ed7fa640 1 -- 172.21.15.1:0/791580577 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f40e00314c0 con 0x7f40f0107660 2024-09-16T00:19:51.145 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-09-16T00:19:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.147+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40dc0392a0 msgr2=0x7f40dc03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.147+0000 7f40f5dd5640 1 --2- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40dc0392a0 0x7f40dc03b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f40d80098c0 tx=0x7f40d8006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.147+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 msgr2=0x7f40f0197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.147+0000 7f40f5dd5640 1 --2- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 0x7f40f0197120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009230 tx=0x7f40e0003d40 comp rx=0 tx=0).stop 2024-09-16T00:19:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.147+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 shutdown_connections 2024-09-16T00:19:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.147+0000 7f40f5dd5640 1 --2- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40dc0392a0 0x7f40dc03b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.147+0000 7f40f5dd5640 1 --2- 172.21.15.1:0/791580577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f0107660 0x7f40f0197120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.147+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 >> 172.21.15.1:0/791580577 conn(0x7f40f00fd290 msgr2=0x7f40f00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:51.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.148+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 shutdown_connections 2024-09-16T00:19:51.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:19:51.148+0000 7f40f5dd5640 1 -- 172.21.15.1:0/791580577 wait complete. 2024-09-16T00:19:51.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:51 smithi001 ceph-mon[30934]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:19:51.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:51 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/791580577' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T00:19:51.767 INFO:tasks.cephadm:Adding mon.a on smithi001 2024-09-16T00:19:51.767 INFO:tasks.cephadm:Adding mon.b on smithi019 2024-09-16T00:19:51.767 INFO:tasks.cephadm:Adding mon.c on smithi153 2024-09-16T00:19:51.768 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi001:172.21.15.1=a;smithi019:172.21.15.19=b;smithi153:172.21.15.153=c' 2024-09-16T00:19:52.346 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:19:52.385 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:19:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:52 smithi001 ceph-mon[30934]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:52 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/791580577' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T00:19:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:52 smithi001 ceph-mon[30934]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:19:52.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.856+0000 7f685aa89640 1 -- 172.21.15.153:0/1785793959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 msgr2=0x7f68541008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:52.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.856+0000 7f685aa89640 1 --2- 172.21.15.153:0/1785793959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 0x7f68541008e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f683c009880 tx=0x7f683c02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:19:52.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.856+0000 7f685aa89640 1 -- 172.21.15.153:0/1785793959 shutdown_connections 2024-09-16T00:19:52.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.856+0000 7f685aa89640 1 --2- 172.21.15.153:0/1785793959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 0x7f68541008e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:52.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.856+0000 7f685aa89640 1 -- 172.21.15.153:0/1785793959 >> 172.21.15.153:0/1785793959 conn(0x7f68540fbc70 msgr2=0x7f68540fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:52.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.856+0000 7f685aa89640 1 -- 172.21.15.153:0/1785793959 shutdown_connections 2024-09-16T00:19:52.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.856+0000 7f685aa89640 1 -- 172.21.15.153:0/1785793959 wait complete. 2024-09-16T00:19:52.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.857+0000 7f685aa89640 1 Processor -- start 2024-09-16T00:19:52.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.857+0000 7f685aa89640 1 -- start start 2024-09-16T00:19:52.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.857+0000 7f685aa89640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 0x7f6854196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:52.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.857+0000 7f685aa89640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68541065f0 con 0x7f68541004e0 2024-09-16T00:19:52.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.857+0000 7f6858825640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 0x7f6854196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:52.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.857+0000 7f6858825640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 0x7f6854196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.153:36284/0 (socket says 172.21.15.153:36284) 2024-09-16T00:19:52.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.857+0000 7f6858825640 1 -- 172.21.15.153:0/398711377 learned_addr learned my addr 172.21.15.153:0/398711377 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:19:52.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.858+0000 7f6858825640 1 -- 172.21.15.153:0/398711377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68541972f0 con 0x7f68541004e0 2024-09-16T00:19:52.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.858+0000 7f6858825640 1 --2- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 0x7f6854196db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f683c004f60 tx=0x7f683c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:52.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.858+0000 7f6851ffb640 1 -- 172.21.15.153:0/398711377 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f683c03f070 con 0x7f68541004e0 2024-09-16T00:19:52.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.858+0000 7f6851ffb640 1 -- 172.21.15.153:0/398711377 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f683c004d40 con 0x7f68541004e0 2024-09-16T00:19:52.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.858+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6854197580 con 0x7f68541004e0 2024-09-16T00:19:52.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.858+0000 7f6851ffb640 1 -- 172.21.15.153:0/398711377 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f683c035030 con 0x7f68541004e0 2024-09-16T00:19:52.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.858+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685419a230 con 0x7f68541004e0 2024-09-16T00:19:52.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.859+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6854106d70 con 0x7f68541004e0 2024-09-16T00:19:52.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.859+0000 7f6851ffb640 1 -- 172.21.15.153:0/398711377 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f683c042070 con 0x7f68541004e0 2024-09-16T00:19:52.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.860+0000 7f6851ffb640 1 --2- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68400392a0 0x7f684003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:52.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.860+0000 7f6851ffb640 1 -- 172.21.15.153:0/398711377 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f683c06d100 con 0x7f68541004e0 2024-09-16T00:19:52.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.860+0000 7f6853fff640 1 --2- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68400392a0 0x7f684003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:52.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.860+0000 7f6853fff640 1 --2- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68400392a0 0x7f684003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f684400ad30 tx=0x7f68440093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:52.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:52.865+0000 7f6851ffb640 1 -- 172.21.15.153:0/398711377 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f683c04a260 con 0x7f68541004e0 2024-09-16T00:19:53.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.055+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi019:172.21.15.19=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}) -- 0x7f685402d950 con 0x7f68400392a0 2024-09-16T00:19:53.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.061+0000 7f6851ffb640 1 -- 172.21.15.153:0/398711377 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f685402d950 con 0x7f68400392a0 2024-09-16T00:19:53.062 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mon update... 2024-09-16T00:19:53.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.063+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68400392a0 msgr2=0x7f684003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.063+0000 7f685aa89640 1 --2- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68400392a0 0x7f684003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f684400ad30 tx=0x7f68440093f0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.063+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 msgr2=0x7f6854196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.063+0000 7f685aa89640 1 --2- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 0x7f6854196db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f683c004f60 tx=0x7f683c0048f0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.064+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 shutdown_connections 2024-09-16T00:19:53.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.064+0000 7f685aa89640 1 --2- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68400392a0 0x7f684003b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.064+0000 7f685aa89640 1 --2- 172.21.15.153:0/398711377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68541004e0 0x7f6854196db0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.064+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 >> 172.21.15.153:0/398711377 conn(0x7f68540fbc70 msgr2=0x7f685406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:53.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.064+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 shutdown_connections 2024-09-16T00:19:53.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:53.064+0000 7f685aa89640 1 -- 172.21.15.153:0/398711377 wait complete. 2024-09-16T00:19:53.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:53.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:53.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:53.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:53.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:53.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:19:53.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: Updating smithi153:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: Updating smithi153:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi019:172.21.15.19=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi019:172.21.15.19=b;smithi153:172.21.15.153=c;count:3 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:19:53.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:53.712 DEBUG:teuthology.orchestra.run.smithi019:mon.b> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T00:19:53.716 DEBUG:teuthology.orchestra.run.smithi153:mon.c> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T00:19:53.719 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:19:53.719 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:19:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:54 smithi001 ceph-mon[30934]: Deploying daemon mon.c on smithi153 2024-09-16T00:19:54.789 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:55.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.488+0000 7f9553286640 1 -- 172.21.15.153:0/771320123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 msgr2=0x7f954c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:55.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.488+0000 7f9553286640 1 --2- 172.21.15.153:0/771320123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 0x7f954c106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f953c009810 tx=0x7f953c02a540 comp rx=0 tx=0).stop 2024-09-16T00:19:55.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.488+0000 7f9553286640 1 -- 172.21.15.153:0/771320123 shutdown_connections 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.488+0000 7f9553286640 1 --2- 172.21.15.153:0/771320123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 0x7f954c106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.488+0000 7f9553286640 1 -- 172.21.15.153:0/771320123 >> 172.21.15.153:0/771320123 conn(0x7f954c077310 msgr2=0x7f954c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.488+0000 7f9553286640 1 -- 172.21.15.153:0/771320123 shutdown_connections 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.488+0000 7f9553286640 1 -- 172.21.15.153:0/771320123 wait complete. 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.488+0000 7f9553286640 1 Processor -- start 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.489+0000 7f9553286640 1 -- start start 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.489+0000 7f9553286640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 0x7f954c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.489+0000 7f9553286640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10ab70 con 0x7f954c106540 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.489+0000 7f9551022640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 0x7f954c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.489+0000 7f9551022640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 0x7f954c197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.153:36304/0 (socket says 172.21.15.153:36304) 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.489+0000 7f9551022640 1 -- 172.21.15.153:0/1023175346 learned_addr learned my addr 172.21.15.153:0/1023175346 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.490+0000 7f9551022640 1 -- 172.21.15.153:0/1023175346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c198510 con 0x7f954c106540 2024-09-16T00:19:55.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.490+0000 7f9551022640 1 --2- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 0x7f954c197fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f953c009930 tx=0x7f953c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.490+0000 7f95427fc640 1 -- 172.21.15.153:0/1023175346 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f953c031400 con 0x7f954c106540 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.490+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c1987a0 con 0x7f954c106540 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.490+0000 7f95427fc640 1 -- 172.21.15.153:0/1023175346 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f953c0035f0 con 0x7f954c106540 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.490+0000 7f95427fc640 1 -- 172.21.15.153:0/1023175346 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f953c03c440 con 0x7f954c106540 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.490+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c198c00 con 0x7f954c106540 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.491+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954c101bb0 con 0x7f954c106540 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.491+0000 7f95427fc640 1 -- 172.21.15.153:0/1023175346 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f953c039040 con 0x7f954c106540 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.491+0000 7f95427fc640 1 --2- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95380392a0 0x7f953803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.491+0000 7f95427fc640 1 -- 172.21.15.153:0/1023175346 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f953c06d540 con 0x7f954c106540 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.491+0000 7f9550821640 1 --2- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95380392a0 0x7f953803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:55.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.492+0000 7f9550821640 1 --2- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95380392a0 0x7f953803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95480098c0 tx=0x7f9548006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:55.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.496+0000 7f95427fc640 1 -- 172.21.15.153:0/1023175346 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f953c040030 con 0x7f954c106540 2024-09-16T00:19:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:55 smithi001 ceph-mon[30934]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:55.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.733+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f954c1069a0 con 0x7f954c106540 2024-09-16T00:19:55.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.733+0000 7f95427fc640 1 -- 172.21.15.153:0/1023175346 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f953c042b60 con 0x7f954c106540 2024-09-16T00:19:55.734 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:19:55.734 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":1,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:18:25.266239Z","created":"2024-09-16T00:18:25.266239Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:19:55.734 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 1 2024-09-16T00:19:55.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.735+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95380392a0 msgr2=0x7f953803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:55.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.735+0000 7f9553286640 1 --2- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95380392a0 0x7f953803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95480098c0 tx=0x7f9548006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:55.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.735+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 msgr2=0x7f954c197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:55.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.735+0000 7f9553286640 1 --2- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 0x7f954c197fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f953c009930 tx=0x7f953c004570 comp rx=0 tx=0).stop 2024-09-16T00:19:55.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.736+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 shutdown_connections 2024-09-16T00:19:55.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.736+0000 7f9553286640 1 --2- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95380392a0 0x7f953803b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:55.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.736+0000 7f9553286640 1 --2- 172.21.15.153:0/1023175346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f954c106540 0x7f954c197fd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:55.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.736+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 >> 172.21.15.153:0/1023175346 conn(0x7f954c077310 msgr2=0x7f954c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:55.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.736+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 shutdown_connections 2024-09-16T00:19:55.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:55.736+0000 7f9553286640 1 -- 172.21.15.153:0/1023175346 wait complete. 2024-09-16T00:19:56.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:56 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/1023175346' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:57.462 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:19:57.462 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:19:57.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:19:57 smithi001 ceph-mon[30934]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:57.724 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:19:58.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.166+0000 7fd52d281640 1 -- 172.21.15.153:0/494800317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 msgr2=0x7fd528107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.166+0000 7fd52d281640 1 --2- 172.21.15.153:0/494800317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 0x7fd528107a40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd518009880 tx=0x7fd51802a5d0 comp rx=0 tx=0).stop 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.166+0000 7fd52d281640 1 -- 172.21.15.153:0/494800317 shutdown_connections 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.166+0000 7fd52d281640 1 --2- 172.21.15.153:0/494800317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 0x7fd528107a40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.166+0000 7fd52d281640 1 -- 172.21.15.153:0/494800317 >> 172.21.15.153:0/494800317 conn(0x7fd5280fb9a0 msgr2=0x7fd5280fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.167+0000 7fd52d281640 1 -- 172.21.15.153:0/494800317 shutdown_connections 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.167+0000 7fd52d281640 1 -- 172.21.15.153:0/494800317 wait complete. 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.167+0000 7fd52d281640 1 Processor -- start 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.167+0000 7fd52d281640 1 -- start start 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.167+0000 7fd52d281640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 0x7fd5281971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.167+0000 7fd52d281640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd52810bd20 con 0x7fd528107660 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd526ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 0x7fd5281971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:58.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd526ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 0x7fd5281971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.153:60078/0 (socket says 172.21.15.153:60078) 2024-09-16T00:19:58.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd526ffd640 1 -- 172.21.15.153:0/2943158700 learned_addr learned my addr 172.21.15.153:0/2943158700 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:19:58.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd526ffd640 1 -- 172.21.15.153:0/2943158700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd528197720 con 0x7fd528107660 2024-09-16T00:19:58.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd526ffd640 1 --2- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 0x7fd5281971e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd518009230 tx=0x7fd518003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:58.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd524ff9640 1 -- 172.21.15.153:0/2943158700 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd51803f070 con 0x7fd528107660 2024-09-16T00:19:58.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd528197a10 con 0x7fd528107660 2024-09-16T00:19:58.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd524ff9640 1 -- 172.21.15.153:0/2943158700 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51803bd40 con 0x7fd528107660 2024-09-16T00:19:58.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd524ff9640 1 -- 172.21.15.153:0/2943158700 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd518035030 con 0x7fd528107660 2024-09-16T00:19:58.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.168+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52819bab0 con 0x7fd528107660 2024-09-16T00:19:58.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.169+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd528102ed0 con 0x7fd528107660 2024-09-16T00:19:58.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.169+0000 7fd524ff9640 1 -- 172.21.15.153:0/2943158700 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fd518042070 con 0x7fd528107660 2024-09-16T00:19:58.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.170+0000 7fd524ff9640 1 --2- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5140392a0 0x7fd51403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:58.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.170+0000 7fd524ff9640 1 -- 172.21.15.153:0/2943158700 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd51806cfb0 con 0x7fd528107660 2024-09-16T00:19:58.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.170+0000 7fd51ffff640 1 --2- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5140392a0 0x7fd51403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:58.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.170+0000 7fd51ffff640 1 --2- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5140392a0 0x7fd51403b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd5100098c0 tx=0x7fd510006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:58.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.175+0000 7fd524ff9640 1 -- 172.21.15.153:0/2943158700 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd518030d60 con 0x7fd528107660 2024-09-16T00:19:58.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.417+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd528198540 con 0x7fd528107660 2024-09-16T00:19:58.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.418+0000 7fd524ff9640 1 -- 172.21.15.153:0/2943158700 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fd5180412c0 con 0x7fd528107660 2024-09-16T00:19:58.419 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:19:58.419 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":1,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:18:25.266239Z","created":"2024-09-16T00:18:25.266239Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:19:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 1 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5140392a0 msgr2=0x7fd51403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 --2- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5140392a0 0x7fd51403b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd5100098c0 tx=0x7fd510006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 msgr2=0x7fd5281971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 --2- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 0x7fd5281971e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd518009230 tx=0x7fd518003d80 comp rx=0 tx=0).stop 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 shutdown_connections 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 --2- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5140392a0 0x7fd51403b760 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 --2- 172.21.15.153:0/2943158700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd528107660 0x7fd5281971e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 >> 172.21.15.153:0/2943158700 conn(0x7fd5280fb9a0 msgr2=0x7fd5280fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 shutdown_connections 2024-09-16T00:19:58.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:19:58.420+0000 7fd52d281640 1 -- 172.21.15.153:0/2943158700 wait complete. 2024-09-16T00:19:59.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:19:59 smithi153 systemd[1]: Started Ceph mon.c for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:20:00.035 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:20:00.036 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:20:00.316 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:20:04.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.327+0000 7fd81ce4b640 1 -- 172.21.15.153:0/2395287617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd804003530 msgr2=0x7fd8040059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:04.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.327+0000 7fd81ce4b640 1 --2- 172.21.15.153:0/2395287617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd804003530 0x7fd8040059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd8181016a0 tx=0x7fd80802c3f0 comp rx=0 tx=0).stop 2024-09-16T00:20:04.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.327+0000 7fd81ce4b640 1 -- 172.21.15.153:0/2395287617 shutdown_connections 2024-09-16T00:20:04.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.327+0000 7fd81ce4b640 1 --2- 172.21.15.153:0/2395287617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd804003530 0x7fd8040059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:04.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.327+0000 7fd81ce4b640 1 -- 172.21.15.153:0/2395287617 >> 172.21.15.153:0/2395287617 conn(0x7fd8180fc220 msgr2=0x7fd8180fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:04.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.327+0000 7fd81ce4b640 1 -- 172.21.15.153:0/2395287617 shutdown_connections 2024-09-16T00:20:04.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.327+0000 7fd81ce4b640 1 -- 172.21.15.153:0/2395287617 wait complete. 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.328+0000 7fd81ce4b640 1 Processor -- start 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.328+0000 7fd81ce4b640 1 -- start start 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81ce4b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 0x7fd818196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81ce4b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd818196650 0x7fd81819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81ce4b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd818108670 con 0x7fd818196650 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81ce4b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181084f0 con 0x7fd804003530 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 0x7fd818196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 0x7fd818196110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50114/0 (socket says 172.21.15.153:50114) 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81659c640 1 -- 172.21.15.153:0/604301490 learned_addr learned my addr 172.21.15.153:0/604301490 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81659c640 1 -- 172.21.15.153:0/604301490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 msgr2=0x7fd818196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81659c640 1 -- 172.21.15.153:0/604301490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 msgr2=0x7fd818196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:20:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81659c640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 0x7fd818196110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd80ffff640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd818196650 0x7fd81819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.329+0000 7fd81659c640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 0x7fd818196110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.330+0000 7fd80ffff640 1 -- 172.21.15.153:0/604301490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 msgr2=0x7fd818196110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.330+0000 7fd80ffff640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 0x7fd818196110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.330+0000 7fd80ffff640 1 -- 172.21.15.153:0/604301490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd818196c70 con 0x7fd818196650 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.330+0000 7fd80ffff640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd818196650 0x7fd81819a9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd80000b3d0 tx=0x7fd80000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.330+0000 7fd80f7fe640 1 -- 172.21.15.153:0/604301490 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd800011630 con 0x7fd818196650 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.330+0000 7fd80f7fe640 1 -- 172.21.15.153:0/604301490 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd800011cb0 con 0x7fd818196650 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.330+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81819af20 con 0x7fd818196650 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.330+0000 7fd80f7fe640 1 -- 172.21.15.153:0/604301490 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd800010df0 con 0x7fd818196650 2024-09-16T00:20:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.330+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81819b4e0 con 0x7fd818196650 2024-09-16T00:20:04.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.331+0000 7fd80f7fe640 1 -- 172.21.15.153:0/604301490 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fd800018020 con 0x7fd818196650 2024-09-16T00:20:04.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.331+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81819b7f0 con 0x7fd818196650 2024-09-16T00:20:04.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.332+0000 7fd80f7fe640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd7f8039090 0x7fd7f803b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:04.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.332+0000 7fd80f7fe640 1 -- 172.21.15.153:0/604301490 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd80004bf60 con 0x7fd818196650 2024-09-16T00:20:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.332+0000 7fd81659c640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd7f8039090 0x7fd7f803b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.332+0000 7fd81659c640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd7f8039090 0x7fd7f803b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd808014a90 tx=0x7fd808005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:04.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.337+0000 7fd80f7fe640 1 -- 172.21.15.153:0/604301490 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd80001f020 con 0x7fd818196650 2024-09-16T00:20:04.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.568+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd8181974d0 con 0x7fd818196650 2024-09-16T00:20:04.569 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:04.569 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:20:04.569 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.569 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: mon.a calling monitor election 2024-09-16T00:20:04.569 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.569 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: mon.c calling monitor election 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: monmap epoch 2 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: last_changed 2024-09-16T00:19:59.198607+0000 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: created 2024-09-16T00:18:25.266239+0000 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: min_mon_release 19 (squid) 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: election_strategy: 1 2024-09-16T00:20:04.570 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: fsmap 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: mgrmap e13: x(active, since 39s) 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: overall HEALTH_OK 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:04.571 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:04.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.569+0000 7fd80f7fe640 1 -- 172.21.15.153:0/604301490 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fd800010510 con 0x7fd818196650 2024-09-16T00:20:04.572 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:20:04.572 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":2,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:19:59.198607Z","created":"2024-09-16T00:18:25.266239Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:20:04.573 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 2 2024-09-16T00:20:04.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.571+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd7f8039090 msgr2=0x7fd7f803b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:04.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.571+0000 7fd81ce4b640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd7f8039090 0x7fd7f803b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd808014a90 tx=0x7fd808005f60 comp rx=0 tx=0).stop 2024-09-16T00:20:04.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.571+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd818196650 msgr2=0x7fd81819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:04.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.571+0000 7fd81ce4b640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd818196650 0x7fd81819a9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd80000b3d0 tx=0x7fd80000b890 comp rx=0 tx=0).stop 2024-09-16T00:20:04.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.572+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 shutdown_connections 2024-09-16T00:20:04.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.572+0000 7fd81ce4b640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd818196650 0x7fd81819a9e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:04.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.572+0000 7fd81ce4b640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd7f8039090 0x7fd7f803b550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:04.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.572+0000 7fd81ce4b640 1 --2- 172.21.15.153:0/604301490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd804003530 0x7fd818196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:04.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.572+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 >> 172.21.15.153:0/604301490 conn(0x7fd8180fc220 msgr2=0x7fd8180fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:04.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.572+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 shutdown_connections 2024-09-16T00:20:04.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:04.572+0000 7fd81ce4b640 1 -- 172.21.15.153:0/604301490 wait complete. 2024-09-16T00:20:04.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:04.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:20:04.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: mon.a calling monitor election 2024-09-16T00:20:04.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: mon.c calling monitor election 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: monmap epoch 2 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: last_changed 2024-09-16T00:19:59.198607+0000 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: created 2024-09-16T00:18:25.266239+0000 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: min_mon_release 19 (squid) 2024-09-16T00:20:04.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: election_strategy: 1 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: fsmap 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: mgrmap e13: x(active, since 39s) 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: overall HEALTH_OK 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:04.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:05.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:05 smithi153 ceph-mon[35538]: Deploying daemon mon.b on smithi019 2024-09-16T00:20:05.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:05 smithi153 ceph-mon[35538]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:05.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:05 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/604301490' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:20:05.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:05.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:05 smithi001 ceph-mon[30934]: Deploying daemon mon.b on smithi019 2024-09-16T00:20:05.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:05 smithi001 ceph-mon[30934]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:05.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:05 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/604301490' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:20:05.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:06.220 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:20:06.220 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:20:06.474 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:20:06.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.854+0000 7fe4947f3640 1 -- 172.21.15.153:0/3666110491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c102840 msgr2=0x7fe48c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:06.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.854+0000 7fe4947f3640 1 --2- 172.21.15.153:0/3666110491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c102840 0x7fe48c102c20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe488009c20 tx=0x7fe48802ab20 comp rx=0 tx=0).stop 2024-09-16T00:20:06.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.854+0000 7fe4947f3640 1 -- 172.21.15.153:0/3666110491 shutdown_connections 2024-09-16T00:20:06.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.854+0000 7fe4947f3640 1 --2- 172.21.15.153:0/3666110491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c102840 0x7fe48c102c20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:06.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.854+0000 7fe4947f3640 1 -- 172.21.15.153:0/3666110491 >> 172.21.15.153:0/3666110491 conn(0x7fe48c0fc280 msgr2=0x7fe48c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:06.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.854+0000 7fe4947f3640 1 -- 172.21.15.153:0/3666110491 shutdown_connections 2024-09-16T00:20:06.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.854+0000 7fe4947f3640 1 -- 172.21.15.153:0/3666110491 wait complete. 2024-09-16T00:20:06.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe4947f3640 1 Processor -- start 2024-09-16T00:20:06.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe4947f3640 1 -- start start 2024-09-16T00:20:06.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe4947f3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102840 0x7fe48c1004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:06.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe4947f3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c1009e0 0x7fe48c100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:06.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe4947f3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c10bdb0 con 0x7fe48c1009e0 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe4947f3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c10bc30 con 0x7fe48c102840 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe491d8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c1009e0 0x7fe48c100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe491d8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c1009e0 0x7fe48c100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.153:54390/0 (socket says 172.21.15.153:54390) 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe491d8e640 1 -- 172.21.15.153:0/1572441593 learned_addr learned my addr 172.21.15.153:0/1572441593 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.855+0000 7fe49258f640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102840 0x7fe48c1004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe49258f640 1 -- 172.21.15.153:0/1572441593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102840 msgr2=0x7fe48c1004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe49258f640 1 -- 172.21.15.153:0/1572441593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102840 msgr2=0x7fe48c1004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe49258f640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102840 0x7fe48c1004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe49258f640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102840 0x7fe48c1004a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe491d8e640 1 -- 172.21.15.153:0/1572441593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102840 msgr2=0x7fe48c1004a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:20:06.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe491d8e640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102840 0x7fe48c1004a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:06.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe491d8e640 1 -- 172.21.15.153:0/1572441593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48c102390 con 0x7fe48c1009e0 2024-09-16T00:20:06.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe491d8e640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c1009e0 0x7fe48c100e40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00b4d0 tx=0x7fe47c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:06.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe4837fe640 1 -- 172.21.15.153:0/1572441593 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe47c010670 con 0x7fe48c1009e0 2024-09-16T00:20:06.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48c1a3650 con 0x7fe48c1009e0 2024-09-16T00:20:06.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe4837fe640 1 -- 172.21.15.153:0/1572441593 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47c010cf0 con 0x7fe48c1009e0 2024-09-16T00:20:06.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe4837fe640 1 -- 172.21.15.153:0/1572441593 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe47c00fe60 con 0x7fe48c1009e0 2024-09-16T00:20:06.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.856+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48c1a3be0 con 0x7fe48c1009e0 2024-09-16T00:20:06.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.857+0000 7fe4837fe640 1 -- 172.21.15.153:0/1572441593 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fe47c017070 con 0x7fe48c1009e0 2024-09-16T00:20:06.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.857+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe48c06a4f0 con 0x7fe48c1009e0 2024-09-16T00:20:06.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.857+0000 7fe4837fe640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe478039040 0x7fe47803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:06.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.857+0000 7fe4837fe640 1 -- 172.21.15.153:0/1572441593 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe47c04bf30 con 0x7fe48c1009e0 2024-09-16T00:20:06.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.858+0000 7fe49258f640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe478039040 0x7fe47803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:06.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.858+0000 7fe49258f640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe478039040 0x7fe47803b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe488000f80 tx=0x7fe48802c570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:06.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:06.863+0000 7fe4837fe640 1 -- 172.21.15.153:0/1572441593 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe47c01e030 con 0x7fe48c1009e0 2024-09-16T00:20:07.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.095+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe48c1a3ef0 con 0x7fe48c1009e0 2024-09-16T00:20:07.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.095+0000 7fe4837fe640 1 -- 172.21.15.153:0/1572441593 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fe47c04b720 con 0x7fe48c1009e0 2024-09-16T00:20:07.096 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:20:07.096 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":2,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:19:59.198607Z","created":"2024-09-16T00:18:25.266239Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:20:07.096 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 2 2024-09-16T00:20:07.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe478039040 msgr2=0x7fe47803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:07.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe478039040 0x7fe47803b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe488000f80 tx=0x7fe48802c570 comp rx=0 tx=0).stop 2024-09-16T00:20:07.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c1009e0 msgr2=0x7fe48c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:07.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c1009e0 0x7fe48c100e40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00b4d0 tx=0x7fe47c00b990 comp rx=0 tx=0).stop 2024-09-16T00:20:07.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 shutdown_connections 2024-09-16T00:20:07.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe48c1009e0 0x7fe48c100e40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:07.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe478039040 0x7fe47803b500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:07.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 --2- 172.21.15.153:0/1572441593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102840 0x7fe48c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:07.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 >> 172.21.15.153:0/1572441593 conn(0x7fe48c0fc280 msgr2=0x7fe48c105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:07.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 shutdown_connections 2024-09-16T00:20:07.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:07.097+0000 7fe4947f3640 1 -- 172.21.15.153:0/1572441593 wait complete. 2024-09-16T00:20:07.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:07 smithi153 ceph-mon[35538]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:07.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:07 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/1572441593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:20:07.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:07 smithi001 ceph-mon[30934]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:07.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:07 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/1572441593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:20:08.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:07 smithi019 systemd[1]: Started Ceph mon.b for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:20:08.663 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:20:08.664 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:20:08.913 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:20:12.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.957+0000 7f15a5497640 1 -- 172.21.15.153:0/1644086730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588003530 msgr2=0x7f15880059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:12.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.957+0000 7f15a5497640 1 --2- 172.21.15.153:0/1644086730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588003530 0x7f15880059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f15a01016a0 tx=0x7f159002c3f0 comp rx=0 tx=0).stop 2024-09-16T00:20:12.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.957+0000 7f15a5497640 1 -- 172.21.15.153:0/1644086730 shutdown_connections 2024-09-16T00:20:12.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.957+0000 7f15a5497640 1 --2- 172.21.15.153:0/1644086730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588003530 0x7f15880059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:12.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.957+0000 7f15a5497640 1 -- 172.21.15.153:0/1644086730 >> 172.21.15.153:0/1644086730 conn(0x7f15a00fc220 msgr2=0x7f15a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:12.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.957+0000 7f15a5497640 1 -- 172.21.15.153:0/1644086730 shutdown_connections 2024-09-16T00:20:12.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.957+0000 7f15a5497640 1 -- 172.21.15.153:0/1644086730 wait complete. 2024-09-16T00:20:12.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.958+0000 7f15a5497640 1 Processor -- start 2024-09-16T00:20:12.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.958+0000 7f15a5497640 1 -- start start 2024-09-16T00:20:12.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.958+0000 7f15a5497640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588003530 0x7f15a01972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:12.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.958+0000 7f15a5497640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 0x7f15a019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:12.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.958+0000 7f15a5497640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a019c0e0 0x7f15a019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:12.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.958+0000 7f15a5497640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0108670 con 0x7f1588003530 2024-09-16T00:20:12.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.958+0000 7f15a5497640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01084f0 con 0x7f15a019c0e0 2024-09-16T00:20:12.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.958+0000 7f15a5497640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01087f0 con 0x7f15a0197810 2024-09-16T00:20:12.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.958+0000 7f159e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 0x7f15a019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:12.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 0x7f15a019bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60884/0 (socket says 172.21.15.153:60884) 2024-09-16T00:20:12.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159e7fc640 1 -- 172.21.15.153:0/1631559486 learned_addr learned my addr 172.21.15.153:0/1631559486 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:20:12.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159effd640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588003530 0x7f15a01972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:12.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159f7fe640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a019c0e0 0x7f15a019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:12.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159e7fc640 1 -- 172.21.15.153:0/1631559486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 msgr2=0x7f15a019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T00:20:12.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159e7fc640 1 -- 172.21.15.153:0/1631559486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 msgr2=0x7f15a019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:20:12.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159e7fc640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 0x7f15a019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:20:12.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159e7fc640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 0x7f15a019bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:20:12.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159f7fe640 1 -- 172.21.15.153:0/1631559486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a019c0e0 msgr2=0x7f15a019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-16T00:20:12.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159f7fe640 1 -- 172.21.15.153:0/1631559486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a019c0e0 msgr2=0x7f15a019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:20:12.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159f7fe640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a019c0e0 0x7f15a019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159f7fe640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a019c0e0 0x7f15a019c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159effd640 1 -- 172.21.15.153:0/1631559486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 msgr2=0x7f15a019bba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159effd640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 0x7f15a019bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159effd640 1 -- 172.21.15.153:0/1631559486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a019c0e0 msgr2=0x7f15a019c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159effd640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a019c0e0 0x7f15a019c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159effd640 1 -- 172.21.15.153:0/1631559486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a01a3e50 con 0x7f1588003530 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f159effd640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588003530 0x7f15a01972d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1590005f10 tx=0x7f15900048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.959+0000 7f1587fff640 1 -- 172.21.15.153:0/1631559486 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590012450 con 0x7f1588003530 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.960+0000 7f1587fff640 1 -- 172.21.15.153:0/1631559486 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1590042070 con 0x7f1588003530 2024-09-16T00:20:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.960+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a01a4020 con 0x7f1588003530 2024-09-16T00:20:12.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.960+0000 7f1587fff640 1 -- 172.21.15.153:0/1631559486 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159003d8f0 con 0x7f1588003530 2024-09-16T00:20:12.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.960+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a01a4600 con 0x7f1588003530 2024-09-16T00:20:12.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.961+0000 7f1587fff640 1 -- 172.21.15.153:0/1631559486 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f159003da90 con 0x7f1588003530 2024-09-16T00:20:12.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.961+0000 7f1587fff640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f158c039730 0x7f158c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:12.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.961+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a01097a0 con 0x7f1588003530 2024-09-16T00:20:12.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.961+0000 7f1587fff640 1 -- 172.21.15.153:0/1631559486 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f159006e900 con 0x7f1588003530 2024-09-16T00:20:12.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.961+0000 7f159e7fc640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f158c039730 0x7f158c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:12.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.962+0000 7f159e7fc640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f158c039730 0x7f158c03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1594009100 tx=0x7f1594008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:12.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:12.966+0000 7f1587fff640 1 -- 172.21.15.153:0/1631559486 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1590045c20 con 0x7f1588003530 2024-09-16T00:20:13.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.210+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f15a01a4830 con 0x7f1588003530 2024-09-16T00:20:13.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.210+0000 7f1587fff640 1 -- 172.21.15.153:0/1631559486 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f1590062020 con 0x7f1588003530 2024-09-16T00:20:13.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:20:13.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: mon.a calling monitor election 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: mon.c calling monitor election 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:20:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: monmap epoch 3 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: last_changed 2024-09-16T00:20:07.893039+0000 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: created 2024-09-16T00:18:25.266239+0000 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: min_mon_release 19 (squid) 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: election_strategy: 1 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: 2: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.b 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: fsmap 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: mgrmap e13: x(active, since 48s) 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: overall HEALTH_OK 2024-09-16T00:20:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:13.214 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:13.214 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:13.214 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:13.214 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:20:13.214 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":3,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:20:07.893039Z","created":"2024-09-16T00:18:25.266239Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:20:13.215 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 3 2024-09-16T00:20:13.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f158c039730 msgr2=0x7f158c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:13.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f158c039730 0x7f158c03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1594009100 tx=0x7f1594008040 comp rx=0 tx=0).stop 2024-09-16T00:20:13.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588003530 msgr2=0x7f15a01972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:13.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588003530 0x7f15a01972d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1590005f10 tx=0x7f15900048d0 comp rx=0 tx=0).stop 2024-09-16T00:20:13.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 shutdown_connections 2024-09-16T00:20:13.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a019c0e0 0x7f15a019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:13.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f158c039730 0x7f158c03bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:13.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a0197810 0x7f15a019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:13.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 --2- 172.21.15.153:0/1631559486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588003530 0x7f15a01972d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:13.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 >> 172.21.15.153:0/1631559486 conn(0x7f15a00fc220 msgr2=0x7f15a0105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:13.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 shutdown_connections 2024-09-16T00:20:13.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:13.213+0000 7f15a5497640 1 -- 172.21.15.153:0/1631559486 wait complete. 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: mon.a calling monitor election 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: mon.c calling monitor election 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: monmap epoch 3 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: last_changed 2024-09-16T00:20:07.893039+0000 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: created 2024-09-16T00:18:25.266239+0000 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: min_mon_release 19 (squid) 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: election_strategy: 1 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: 2: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.b 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: fsmap 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: mgrmap e13: x(active, since 48s) 2024-09-16T00:20:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: overall HEALTH_OK 2024-09-16T00:20:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:13.763 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T00:20:13.763 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T00:20:14.027 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:14.136 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:13 smithi153 ceph-mon[35538]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:14.136 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:13 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:14.136 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:13 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/1631559486' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:20:14.136 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:13 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:14.149 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:13 smithi001 ceph-mon[30934]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:14.149 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:13 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:14.149 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:13 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/1631559486' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:20:14.149 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:13 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:14.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.487+0000 7f12fb2b9640 1 -- 172.21.15.1:0/3160291294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f4102810 msgr2=0x7f12f4102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:14.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.487+0000 7f12fb2b9640 1 --2- 172.21.15.1:0/3160291294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f4102810 0x7f12f4102bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009880 tx=0x7f12f002a670 comp rx=0 tx=0).stop 2024-09-16T00:20:14.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.487+0000 7f12fb2b9640 1 -- 172.21.15.1:0/3160291294 shutdown_connections 2024-09-16T00:20:14.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.487+0000 7f12fb2b9640 1 --2- 172.21.15.1:0/3160291294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f4102810 0x7f12f4102bf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:14.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.487+0000 7f12fb2b9640 1 -- 172.21.15.1:0/3160291294 >> 172.21.15.1:0/3160291294 conn(0x7f12f40fc1b0 msgr2=0x7f12f40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:14.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.487+0000 7f12fb2b9640 1 -- 172.21.15.1:0/3160291294 shutdown_connections 2024-09-16T00:20:14.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.488+0000 7f12fb2b9640 1 -- 172.21.15.1:0/3160291294 wait complete. 2024-09-16T00:20:14.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.488+0000 7f12fb2b9640 1 Processor -- start 2024-09-16T00:20:14.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.488+0000 7f12fb2b9640 1 -- start start 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.488+0000 7f12fb2b9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f4102810 0x7f12f4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.488+0000 7f12fb2b9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f4197950 0x7f12f419bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.488+0000 7f12fb2b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f419c220 0x7f12f419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.488+0000 7f12fb2b9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12f410bc60 con 0x7f12f419c220 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.488+0000 7f12fb2b9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12f410bae0 con 0x7f12f4102810 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.488+0000 7f12fb2b9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12f410bde0 con 0x7f12f4197950 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9856640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f419c220 0x7f12f419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9856640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f419c220 0x7f12f419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56930/0 (socket says 172.21.15.1:56930) 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9856640 1 -- 172.21.15.1:0/2408123610 learned_addr learned my addr 172.21.15.1:0/2408123610 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:20:14.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f8854640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f4197950 0x7f12f419bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:14.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9055640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f4102810 0x7f12f4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:14.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9856640 1 -- 172.21.15.1:0/2408123610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f4197950 msgr2=0x7f12f419bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:14.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9856640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f4197950 0x7f12f419bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:14.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9856640 1 -- 172.21.15.1:0/2408123610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f4102810 msgr2=0x7f12f4197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:14.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9856640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f4102810 0x7f12f4197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:14.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9856640 1 -- 172.21.15.1:0/2408123610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f41a3f90 con 0x7f12f419c220 2024-09-16T00:20:14.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9055640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f4102810 0x7f12f4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:14.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12f9856640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f419c220 0x7f12f419c6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00ba50 tx=0x7f12ec00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12ea7fc640 1 -- 172.21.15.1:0/2408123610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec019070 con 0x7f12f419c220 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12ea7fc640 1 -- 172.21.15.1:0/2408123610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12ec00cbe0 con 0x7f12f419c220 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f41a41c0 con 0x7f12f419c220 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12ea7fc640 1 -- 172.21.15.1:0/2408123610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0076a0 con 0x7f12f419c220 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.489+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f41a4680 con 0x7f12f419c220 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.490+0000 7f12ea7fc640 1 -- 172.21.15.1:0/2408123610 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f12ec01c070 con 0x7f12f419c220 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.490+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f4102c70 con 0x7f12f419c220 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.490+0000 7f12ea7fc640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12e0039730 0x7f12e003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.491+0000 7f12ea7fc640 1 -- 172.21.15.1:0/2408123610 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f12ec04bf80 con 0x7f12f419c220 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.491+0000 7f12f9055640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12e0039730 0x7f12e003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:14.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.491+0000 7f12f9055640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12e0039730 0x7f12e003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f12f0031550 tx=0x7f12f0005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:14.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.495+0000 7f12ea7fc640 1 -- 172.21.15.1:0/2408123610 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12ec0112a0 con 0x7f12f419c220 2024-09-16T00:20:14.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.677+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f12f41a4a20 con 0x7f12f419c220 2024-09-16T00:20:14.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.678+0000 7f12ea7fc640 1 -- 172.21.15.1:0/2408123610 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f12ec00ce00 con 0x7f12f419c220 2024-09-16T00:20:14.679 INFO:teuthology.orchestra.run.smithi001.stdout:# minimal ceph.conf for 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:14.679 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-09-16T00:20:14.679 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:14.679 INFO:teuthology.orchestra.run.smithi001.stdout: mon_host = [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] 2024-09-16T00:20:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12e0039730 msgr2=0x7f12e003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12e0039730 0x7f12e003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f12f0031550 tx=0x7f12f0005a40 comp rx=0 tx=0).stop 2024-09-16T00:20:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f419c220 msgr2=0x7f12f419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f419c220 0x7f12f419c6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00ba50 tx=0x7f12ec00bf10 comp rx=0 tx=0).stop 2024-09-16T00:20:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 shutdown_connections 2024-09-16T00:20:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12f419c220 0x7f12f419c6d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12e0039730 0x7f12e003bbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f4197950 0x7f12f419bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 --2- 172.21.15.1:0/2408123610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f4102810 0x7f12f4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:14.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.680+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 >> 172.21.15.1:0/2408123610 conn(0x7f12f40fc1b0 msgr2=0x7f12f41055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:14.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.681+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 shutdown_connections 2024-09-16T00:20:14.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:14.681+0000 7f12fb2b9640 1 -- 172.21.15.1:0/2408123610 wait complete. 2024-09-16T00:20:15.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:20:15.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: mon.a calling monitor election 2024-09-16T00:20:15.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: mon.c calling monitor election 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: monmap epoch 3 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:15.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: last_changed 2024-09-16T00:20:07.893039+0000 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: created 2024-09-16T00:18:25.266239+0000 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: min_mon_release 19 (squid) 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: election_strategy: 1 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: 2: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.b 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: fsmap 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: mgrmap e13: x(active, since 48s) 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: overall HEALTH_OK 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:15.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:15.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='client.? 172.21.15.153:0/1631559486' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:20:15.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:15.406 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T00:20:15.407 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:20:15.407 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:20:15.435 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:20:15.435 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:20:15.505 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:20:15.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:20:15.534 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:20:15.534 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:20:15.600 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:20:15.600 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:20:15.627 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:20:15.628 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:20:15.693 INFO:tasks.cephadm:Adding mgr.x on smithi001 2024-09-16T00:20:15.693 INFO:tasks.cephadm:Adding mgr.y on smithi019 2024-09-16T00:20:15.693 INFO:tasks.cephadm:Adding mgr.z on smithi153 2024-09-16T00:20:15.693 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi001=x;smithi019=y;smithi153=z' 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: mon.b calling monitor election 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: Updating smithi001:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: Updating smithi153:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: Updating smithi019:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: mon.b calling monitor election 2024-09-16T00:20:15.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: mon.c calling monitor election 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: mon.a calling monitor election 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: monmap epoch 3 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: last_changed 2024-09-16T00:20:07.893039+0000 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: created 2024-09-16T00:18:25.266239+0000 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: min_mon_release 19 (squid) 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: election_strategy: 1 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: 2: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.b 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: fsmap 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:20:15.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: mgrmap e13: x(active, since 50s) 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: overall HEALTH_OK 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:15.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:20:15.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:15.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: mon.b calling monitor election 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: Updating smithi001:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: Updating smithi153:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: Updating smithi019:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: mon.b calling monitor election 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: mon.c calling monitor election 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: mon.a calling monitor election 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: monmap epoch 3 2024-09-16T00:20:15.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: last_changed 2024-09-16T00:20:07.893039+0000 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: created 2024-09-16T00:18:25.266239+0000 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: min_mon_release 19 (squid) 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: election_strategy: 1 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: 2: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.b 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: fsmap 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: mgrmap e13: x(active, since 50s) 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: overall HEALTH_OK 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.859 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:20:15.860 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:15.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: mon.b calling monitor election 2024-09-16T00:20:15.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:15.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-16T00:20:15.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-16T00:20:15.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: Updating smithi001:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: Updating smithi153:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: Updating smithi019:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: mon.b calling monitor election 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: mon.c calling monitor election 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: mon.a calling monitor election 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: monmap epoch 3 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: last_changed 2024-09-16T00:20:07.893039+0000 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: created 2024-09-16T00:18:25.266239+0000 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: min_mon_release 19 (squid) 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: election_strategy: 1 2024-09-16T00:20:15.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: 2: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.b 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: fsmap 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: mgrmap e13: x(active, since 50s) 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: overall HEALTH_OK 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:15.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:15.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:20:15.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:15.973 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:20:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.373+0000 7fd6ee12b640 1 -- 172.21.15.153:0/1338665064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6e81005e0 msgr2=0x7fd6e81009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.373+0000 7fd6ee12b640 1 --2- 172.21.15.153:0/1338665064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6e81005e0 0x7fd6e81009e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009880 tx=0x7fd6d802a5d0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.373+0000 7fd6ee12b640 1 -- 172.21.15.153:0/1338665064 shutdown_connections 2024-09-16T00:20:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.373+0000 7fd6ee12b640 1 --2- 172.21.15.153:0/1338665064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6e81005e0 0x7fd6e81009e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.373+0000 7fd6ee12b640 1 -- 172.21.15.153:0/1338665064 >> 172.21.15.153:0/1338665064 conn(0x7fd6e80fbdb0 msgr2=0x7fd6e80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.373+0000 7fd6ee12b640 1 -- 172.21.15.153:0/1338665064 shutdown_connections 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.373+0000 7fd6ee12b640 1 -- 172.21.15.153:0/1338665064 wait complete. 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.374+0000 7fd6ee12b640 1 Processor -- start 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.374+0000 7fd6ee12b640 1 -- start start 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.374+0000 7fd6ee12b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6e81005e0 0x7fd6e8197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.374+0000 7fd6ee12b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6e8197580 0x7fd6e819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.374+0000 7fd6ee12b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6e8197a00 0x7fd6e819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.374+0000 7fd6ee12b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81066d0 con 0x7fd6e81005e0 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.374+0000 7fd6ee12b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8106550 con 0x7fd6e8197a00 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.374+0000 7fd6ee12b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8106850 con 0x7fd6e8197580 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6deffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6e8197580 0x7fd6e819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6deffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6e8197580 0x7fd6e819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34562/0 (socket says 172.21.15.153:34562) 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6deffd640 1 -- 172.21.15.153:0/3186316261 learned_addr learned my addr 172.21.15.153:0/3186316261 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:20:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6e77fe640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6e81005e0 0x7fd6e8197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6e7fff640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6e8197a00 0x7fd6e819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6deffd640 1 -- 172.21.15.153:0/3186316261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6e8197a00 msgr2=0x7fd6e819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6deffd640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6e8197a00 0x7fd6e819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6deffd640 1 -- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6e81005e0 msgr2=0x7fd6e8197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6deffd640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6e81005e0 0x7fd6e8197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6deffd640 1 -- 172.21.15.153:0/3186316261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e819f440 con 0x7fd6e8197580 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6e7fff640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6e8197a00 0x7fd6e819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6deffd640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6e8197580 0x7fd6e819c5f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000b520 tx=0x7fd6d000b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6e57fa640 1 -- 172.21.15.153:0/3186316261 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0004230 con 0x7fd6e8197580 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6e57fa640 1 -- 172.21.15.153:0/3186316261 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6d00043d0 con 0x7fd6e8197580 2024-09-16T00:20:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6e57fa640 1 -- 172.21.15.153:0/3186316261 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0003d60 con 0x7fd6e8197580 2024-09-16T00:20:16.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e819f730 con 0x7fd6e8197580 2024-09-16T00:20:16.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.375+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e819fd40 con 0x7fd6e8197580 2024-09-16T00:20:16.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.376+0000 7fd6e57fa640 1 -- 172.21.15.153:0/3186316261 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fd6d000e070 con 0x7fd6e8197580 2024-09-16T00:20:16.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.377+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e8190860 con 0x7fd6e8197580 2024-09-16T00:20:16.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.377+0000 7fd6e57fa640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6d4039730 0x7fd6d403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:16.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.377+0000 7fd6e57fa640 1 -- 172.21.15.153:0/3186316261 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd6d004bc20 con 0x7fd6e8197580 2024-09-16T00:20:16.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.377+0000 7fd6e77fe640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6d4039730 0x7fd6d403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:16.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.378+0000 7fd6e77fe640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6d4039730 0x7fd6d403bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009230 tx=0x7fd6d80315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:16.382 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.382+0000 7fd6e57fa640 1 -- 172.21.15.153:0/3186316261 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6d0014020 con 0x7fd6e8197580 2024-09-16T00:20:16.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.558+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi019=y;smithi153=z", "target": ["mon-mgr", ""]}) -- 0x7fd6e80626c0 con 0x7fd6d4039730 2024-09-16T00:20:16.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.563+0000 7fd6e57fa640 1 -- 172.21.15.153:0/3186316261 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd6e80626c0 con 0x7fd6d4039730 2024-09-16T00:20:16.564 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mgr update... 2024-09-16T00:20:16.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6d4039730 msgr2=0x7fd6d403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:16.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6d4039730 0x7fd6d403bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009230 tx=0x7fd6d80315d0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6e8197580 msgr2=0x7fd6e819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:16.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6e8197580 0x7fd6e819c5f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000b520 tx=0x7fd6d000b9e0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 shutdown_connections 2024-09-16T00:20:16.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6e8197a00 0x7fd6e819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6d4039730 0x7fd6d403bbf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6e8197580 0x7fd6e819c5f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 --2- 172.21.15.153:0/3186316261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6e81005e0 0x7fd6e8197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:16.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.565+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 >> 172.21.15.153:0/3186316261 conn(0x7fd6e80fbdb0 msgr2=0x7fd6e806a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:16.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.566+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 shutdown_connections 2024-09-16T00:20:16.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:20:16.566+0000 7fd6ee12b640 1 -- 172.21.15.153:0/3186316261 wait complete. 2024-09-16T00:20:16.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:16 smithi153 ceph-mon[35538]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:20:16.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:16 smithi153 ceph-mon[35538]: Reconfiguring daemon mon.a on smithi001 2024-09-16T00:20:16.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:16.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:16.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:16.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:16.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:20:16.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:16.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:16.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:16 smithi019 ceph-mon[35186]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:20:16.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:16 smithi019 ceph-mon[35186]: Reconfiguring daemon mon.a on smithi001 2024-09-16T00:20:16.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:16.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:16.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:16.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:16.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:20:16.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:16.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:16.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:16 smithi001 ceph-mon[30934]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:20:16.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:16 smithi001 ceph-mon[30934]: Reconfiguring daemon mon.a on smithi001 2024-09-16T00:20:16.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:20:16.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:16.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:16.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:16.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:20:16.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:16.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:17.180 DEBUG:teuthology.orchestra.run.smithi019:mgr.y> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T00:20:17.182 DEBUG:teuthology.orchestra.run.smithi153:mgr.z> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.z.service 2024-09-16T00:20:17.185 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T00:20:17.185 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:20:17.185 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:20:17.207 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:20:17.207 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-09-16T00:20:17.266 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:20:17.266 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:17.267 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T00:20:17.267 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:17.267 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:17.267 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-16 00:19:33.043844718 +0000 2024-09-16T00:20:17.267 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-16 00:15:17.443638643 +0000 2024-09-16T00:20:17.267 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-16 00:15:17.443638643 +0000 2024-09-16T00:20:17.267 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-16 00:15:17.443638643 +0000 2024-09-16T00:20:17.267 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:20:17.338 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-16T00:20:17.338 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-16T00:20:17.338 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000124269 s, 4.1 MB/s 2024-09-16T00:20:17.339 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:20:17.399 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-09-16T00:20:17.458 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:20:17.458 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:17.458 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T00:20:17.458 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:17.459 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:17.459 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-16 00:19:33.070844377 +0000 2024-09-16T00:20:17.459 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-16 00:15:17.765634720 +0000 2024-09-16T00:20:17.459 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-16 00:15:17.765634720 +0000 2024-09-16T00:20:17.459 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-16 00:15:17.765634720 +0000 2024-09-16T00:20:17.459 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:20:17.526 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-16T00:20:17.526 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-16T00:20:17.526 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000123199 s, 4.2 MB/s 2024-09-16T00:20:17.528 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:20:17.588 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-09-16T00:20:17.646 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:20:17.647 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:17.647 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T00:20:17.647 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:17.647 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:17.647 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-16 00:19:33.097844037 +0000 2024-09-16T00:20:17.647 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-16 00:15:18.082630858 +0000 2024-09-16T00:20:17.647 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-16 00:15:18.082630858 +0000 2024-09-16T00:20:17.647 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-16 00:15:18.082630858 +0000 2024-09-16T00:20:17.648 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:20:17.714 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-16T00:20:17.714 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-16T00:20:17.715 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000107121 s, 4.8 MB/s 2024-09-16T00:20:17.716 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:20:17.775 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-09-16T00:20:17.833 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:20:17.834 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:17.834 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T00:20:17.834 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:17.834 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:17.834 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-16 00:19:33.128843646 +0000 2024-09-16T00:20:17.834 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-16 00:15:18.404626935 +0000 2024-09-16T00:20:17.834 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-16 00:15:18.404626935 +0000 2024-09-16T00:20:17.834 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-16 00:15:18.404626935 +0000 2024-09-16T00:20:17.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:20:17.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:17 smithi019 ceph-mon[35186]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:17.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:17 smithi019 ceph-mon[35186]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:20:17.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:17 smithi019 ceph-mon[35186]: Reconfiguring daemon mon.b on smithi019 2024-09-16T00:20:17.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:17 smithi019 ceph-mon[35186]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi019=y;smithi153=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:17.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:17 smithi019 ceph-mon[35186]: Saving service mgr spec with placement smithi001=x;smithi019=y;smithi153=z;count:3 2024-09-16T00:20:17.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:17 smithi001 ceph-mon[30934]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:17.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:17 smithi001 ceph-mon[30934]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:20:17.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:17 smithi001 ceph-mon[30934]: Reconfiguring daemon mon.b on smithi019 2024-09-16T00:20:17.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:17 smithi001 ceph-mon[30934]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi019=y;smithi153=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:17.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:17 smithi001 ceph-mon[30934]: Saving service mgr spec with placement smithi001=x;smithi019=y;smithi153=z;count:3 2024-09-16T00:20:17.900 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-16T00:20:17.900 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-16T00:20:17.901 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000106446 s, 4.8 MB/s 2024-09-16T00:20:17.902 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:20:17.960 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-16T00:20:17.960 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:20:17.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:17 smithi153 ceph-mon[35538]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:17.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:17 smithi153 ceph-mon[35538]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:20:17.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:17 smithi153 ceph-mon[35538]: Reconfiguring daemon mon.b on smithi019 2024-09-16T00:20:17.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:17 smithi153 ceph-mon[35538]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi019=y;smithi153=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:17.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:17 smithi153 ceph-mon[35538]: Saving service mgr spec with placement smithi001=x;smithi019=y;smithi153=z;count:3 2024-09-16T00:20:17.978 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:20:17.978 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-09-16T00:20:18.035 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:20:18.035 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:18.035 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T00:20:18.035 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:18.035 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:18.035 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-16 00:20:07.355683885 +0000 2024-09-16T00:20:18.035 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-16 00:15:30.635544041 +0000 2024-09-16T00:20:18.035 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-16 00:15:30.635544041 +0000 2024-09-16T00:20:18.035 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-16 00:15:30.635544041 +0000 2024-09-16T00:20:18.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:20:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-16T00:20:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-16T00:20:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000273302 s, 1.9 MB/s 2024-09-16T00:20:18.102 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:20:18.159 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-09-16T00:20:18.215 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:20:18.215 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:18.215 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-16T00:20:18.215 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:18.215 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:18.215 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-16 00:20:07.355683885 +0000 2024-09-16T00:20:18.215 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-16 00:15:30.984540515 +0000 2024-09-16T00:20:18.215 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-16 00:15:30.984540515 +0000 2024-09-16T00:20:18.216 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-16 00:15:30.984540515 +0000 2024-09-16T00:20:18.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:20:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-16T00:20:18.280 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-16T00:20:18.280 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000192802 s, 2.7 MB/s 2024-09-16T00:20:18.281 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:20:18.338 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-09-16T00:20:18.393 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:20:18.394 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:18.394 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-16T00:20:18.394 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:18.394 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:18.394 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-16 00:20:07.355683885 +0000 2024-09-16T00:20:18.394 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-16 00:15:31.312537160 +0000 2024-09-16T00:20:18.394 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-16 00:15:31.312537160 +0000 2024-09-16T00:20:18.394 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-16 00:15:31.312537160 +0000 2024-09-16T00:20:18.394 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:20:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-16T00:20:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-16T00:20:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000192806 s, 2.7 MB/s 2024-09-16T00:20:18.460 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:20:18.516 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-09-16T00:20:18.571 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:20:18.572 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:18.572 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T00:20:18.572 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:18.572 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:18.572 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-16 00:20:07.356683874 +0000 2024-09-16T00:20:18.572 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-16 00:15:31.643533678 +0000 2024-09-16T00:20:18.572 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-16 00:15:31.643533678 +0000 2024-09-16T00:20:18.572 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-16 00:15:31.643533678 +0000 2024-09-16T00:20:18.572 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:20:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-16T00:20:18.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-16T00:20:18.636 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.00020137 s, 2.5 MB/s 2024-09-16T00:20:18.637 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:20:18.694 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-16T00:20:18.694 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:20:18.711 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:20:18.711 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-09-16T00:20:18.768 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:20:18.768 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:18.768 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-16T00:20:18.768 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:18.768 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:18.768 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-16 00:19:57.867795343 +0000 2024-09-16T00:20:18.768 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-16 00:15:26.186129985 +0000 2024-09-16T00:20:18.768 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-16 00:15:26.186129985 +0000 2024-09-16T00:20:18.768 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-16 00:15:26.186129985 +0000 2024-09-16T00:20:18.768 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:20:18.833 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-16T00:20:18.833 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-16T00:20:18.834 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000234033 s, 2.2 MB/s 2024-09-16T00:20:18.836 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:20:18.894 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-09-16T00:20:18.950 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:20:18.950 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:18.950 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T00:20:18.950 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:18.950 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:18.950 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-16 00:19:57.868795325 +0000 2024-09-16T00:20:18.950 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-16 00:15:26.529124226 +0000 2024-09-16T00:20:18.950 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-16 00:15:26.529124226 +0000 2024-09-16T00:20:18.950 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-16 00:15:26.529124226 +0000 2024-09-16T00:20:18.951 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:20:19.014 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-16T00:20:19.014 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-16T00:20:19.014 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000181056 s, 2.8 MB/s 2024-09-16T00:20:19.015 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:20:19.072 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-09-16T00:20:19.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:19.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:19.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:18 smithi019 ceph-mon[35186]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:20:19.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:19.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:20:19.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:19.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:18 smithi019 ceph-mon[35186]: Reconfiguring daemon mon.c on smithi153 2024-09-16T00:20:19.127 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:20:19.128 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:19.128 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-16T00:20:19.128 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:19.128 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:19.128 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-16 00:19:57.868795325 +0000 2024-09-16T00:20:19.128 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-16 00:15:26.849118852 +0000 2024-09-16T00:20:19.128 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-16 00:15:26.849118852 +0000 2024-09-16T00:20:19.128 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-16 00:15:26.849118852 +0000 2024-09-16T00:20:19.128 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:20:19.173 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:19.173 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:19.173 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:18 smithi153 ceph-mon[35538]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:20:19.174 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:19.174 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:20:19.174 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:19.174 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:18 smithi153 ceph-mon[35538]: Reconfiguring daemon mon.c on smithi153 2024-09-16T00:20:19.192 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-16T00:20:19.192 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-16T00:20:19.192 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000219077 s, 2.3 MB/s 2024-09-16T00:20:19.193 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:20:19.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:19.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:19.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:18 smithi001 ceph-mon[30934]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:20:19.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:20:19.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:20:19.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:19.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:18 smithi001 ceph-mon[30934]: Reconfiguring daemon mon.c on smithi153 2024-09-16T00:20:19.250 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-09-16T00:20:19.305 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:20:19.306 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:20:19.306 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-16T00:20:19.306 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:20:19.306 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:20:19.306 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-16 00:19:57.869795308 +0000 2024-09-16T00:20:19.306 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-16 00:15:27.173113412 +0000 2024-09-16T00:20:19.307 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-16 00:15:27.173113412 +0000 2024-09-16T00:20:19.307 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-16 00:15:27.173113412 +0000 2024-09-16T00:20:19.307 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:20:19.369 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-16T00:20:19.369 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-16T00:20:19.369 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000181345 s, 2.8 MB/s 2024-09-16T00:20:19.371 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:20:19.427 INFO:tasks.cephadm:Deploying osd.0 on smithi001 with /dev/vg_nvme/lv_4... 2024-09-16T00:20:19.428 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:20:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:19.955 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:19.955 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:19.955 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:19.955 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:19.955 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:19.955 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:19.955 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:19.956 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:20:19.956 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:20:19.956 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:20:19.956 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:20.088 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:20.089 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:20.089 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:20:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:20:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:20:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:20.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:20.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:20:20.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:20:20.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:20:20.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:20.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:20 smithi153 ceph-mon[35538]: Deploying daemon mgr.z on smithi153 2024-09-16T00:20:21.033 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:20 smithi001 ceph-mon[30934]: Deploying daemon mgr.z on smithi153 2024-09-16T00:20:21.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:20 smithi019 ceph-mon[35186]: Deploying daemon mgr.z on smithi153 2024-09-16T00:20:21.711 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:21.605+0000 7f0c7ce48040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:20:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:20:22.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:21 smithi001 ceph-mon[30934]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:22.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:22.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:22.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:22.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:20:22.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:20:22.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:20:22.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:22.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_4 2024-09-16T00:20:22.104 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:21.753+0000 7f0c7ce48040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:21.818+0000 7f0c7ce48040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:21.966+0000 7f0c7ce48040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:22 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:22.104+0000 7f0c7ce48040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-mon[35538]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:20:22.105 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:20:22.106 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:21 smithi019 ceph-mon[35186]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:20:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:20:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:20:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:22.338 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:22.711 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:22 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:22.418+0000 7f0c7ce48040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:20:22.712 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:22 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:22.479+0000 7f0c7ce48040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:20:22.712 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:22 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:22.661+0000 7f0c7ce48040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:20:22.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.778+0000 7fa5e4f02640 1 -- 172.21.15.1:0/1036553771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e0101c00 msgr2=0x7fa5e0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.778+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/1036553771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e0101c00 0x7fa5e0102000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009900 tx=0x7fa5d002a410 comp rx=0 tx=0).stop 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.778+0000 7fa5e4f02640 1 -- 172.21.15.1:0/1036553771 shutdown_connections 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.778+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/1036553771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e01037c0 0x7fa5e010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.778+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/1036553771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5e0102e00 0x7fa5e0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.778+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/1036553771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e0101c00 0x7fa5e0102000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.778+0000 7fa5e4f02640 1 -- 172.21.15.1:0/1036553771 >> 172.21.15.1:0/1036553771 conn(0x7fa5e00fd390 msgr2=0x7fa5e00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.778+0000 7fa5e4f02640 1 -- 172.21.15.1:0/1036553771 shutdown_connections 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.778+0000 7fa5e4f02640 1 -- 172.21.15.1:0/1036553771 wait complete. 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5e4f02640 1 Processor -- start 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5e4f02640 1 -- start start 2024-09-16T00:20:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5e4f02640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0101c00 0x7fa5e019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5e4f02640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5e0102e00 0x7fa5e019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5e4f02640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e01037c0 0x7fa5e01a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5e4f02640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e010d8b0 con 0x7fa5e01037c0 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5e4f02640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e010d730 con 0x7fa5e0102e00 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5e4f02640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e010da30 con 0x7fa5e0101c00 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5ded9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e01037c0 0x7fa5e01a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5ded9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e01037c0 0x7fa5e01a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50028/0 (socket says 172.21.15.1:50028) 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5ded9d640 1 -- 172.21.15.1:0/3334781126 learned_addr learned my addr 172.21.15.1:0/3334781126 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.779+0000 7fa5de59c640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0101c00 0x7fa5e019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5ddd9b640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5e0102e00 0x7fa5e019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5ded9d640 1 -- 172.21.15.1:0/3334781126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0101c00 msgr2=0x7fa5e019a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5ded9d640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0101c00 0x7fa5e019a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5ded9d640 1 -- 172.21.15.1:0/3334781126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5e0102e00 msgr2=0x7fa5e019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5ded9d640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5e0102e00 0x7fa5e019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5ded9d640 1 -- 172.21.15.1:0/3334781126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e01a1d80 con 0x7fa5e01037c0 2024-09-16T00:20:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5ded9d640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e01037c0 0x7fa5e01a1710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009fd0 tx=0x7fa5d800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5cf7fe640 1 -- 172.21.15.1:0/3334781126 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8017070 con 0x7fa5e01037c0 2024-09-16T00:20:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e01a2070 con 0x7fa5e01037c0 2024-09-16T00:20:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5cf7fe640 1 -- 172.21.15.1:0/3334781126 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d8003e50 con 0x7fa5e01037c0 2024-09-16T00:20:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5cf7fe640 1 -- 172.21.15.1:0/3334781126 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80106c0 con 0x7fa5e01037c0 2024-09-16T00:20:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.780+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e01a25b0 con 0x7fa5e01037c0 2024-09-16T00:20:22.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.781+0000 7fa5cf7fe640 1 -- 172.21.15.1:0/3334781126 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fa5d8010860 con 0x7fa5e01037c0 2024-09-16T00:20:22.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.781+0000 7fa5cf7fe640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5c8039500 0x7fa5c803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:22.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.781+0000 7fa5cf7fe640 1 -- 172.21.15.1:0/3334781126 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa5d804c600 con 0x7fa5e01037c0 2024-09-16T00:20:22.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.781+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e01a2810 con 0x7fa5e01037c0 2024-09-16T00:20:22.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.781+0000 7fa5de59c640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5c8039500 0x7fa5c803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:22.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.782+0000 7fa5de59c640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5c8039500 0x7fa5c803b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5d002aa00 tx=0x7fa5d00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:22.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.786+0000 7fa5cf7fe640 1 -- 172.21.15.1:0/3334781126 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5d800e9a0 con 0x7fa5e01037c0 2024-09-16T00:20:22.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:22.970+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa5e01a2aa0 con 0x7fa5c8039500 2024-09-16T00:20:23.062 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:22 smithi019 ceph-mon[35186]: Deploying daemon mgr.y on smithi019 2024-09-16T00:20:23.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:22 smithi001 ceph-mon[30934]: Deploying daemon mgr.y on smithi019 2024-09-16T00:20:23.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:22 smithi153 ceph-mon[35538]: Deploying daemon mgr.y on smithi019 2024-09-16T00:20:23.357 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 systemd[1]: Starting Ceph mgr.y for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:20:23.699 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 podman[36527]: 2024-09-16 00:20:23.344296692 +0000 UTC m=+0.015896969 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:20:23.699 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 podman[36527]: 2024-09-16 00:20:23.446717754 +0000 UTC m=+0.118318022 container create cf88e9079cd1fd9fe310ef192d2d9d9d1863d42cb57b8349729335d56b399a6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T00:20:23.699 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 podman[36527]: 2024-09-16 00:20:23.640532969 +0000 UTC m=+0.312133235 container init cf88e9079cd1fd9fe310ef192d2d9d9d1863d42cb57b8349729335d56b399a6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y, ceph=True, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T00:20:23.699 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 podman[36527]: 2024-09-16 00:20:23.643210315 +0000 UTC m=+0.314810581 container start cf88e9079cd1fd9fe310ef192d2d9d9d1863d42cb57b8349729335d56b399a6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:20:23.699 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mgr[36546]: -- 172.21.15.19:0/4071013577 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe054c50e0 con 0x55fe054a3000 2024-09-16T00:20:23.700 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 bash[36527]: cf88e9079cd1fd9fe310ef192d2d9d9d1863d42cb57b8349729335d56b399a6d 2024-09-16T00:20:23.833 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:23.542+0000 7f0c7ce48040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:20:23.834 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:23.608+0000 7f0c7ce48040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:20:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:23 smithi001 ceph-mon[30934]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:23 smithi001 ceph-mon[30934]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:23.994 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 systemd[1]: Started Ceph mgr.y for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:20:23.994 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:23.776+0000 7f05bcff1040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:20:23.994 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:23.836+0000 7f05bcff1040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:20:23.994 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:23.993+0000 7f05bcff1040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:20:23.994 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mon[35186]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:23.994 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mon[35186]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:23.994 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:23.994 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:23.995 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:23.995 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:23.995 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:23.995 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:23.995 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:24.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-mon[35538]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:24.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-mon[35538]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:24.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:24.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:24.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:24.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:24.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:24.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:24.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:24.357 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:24.063+0000 7f05bcff1040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:20:24.357 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:24.229+0000 7f05bcff1040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:20:24.648 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:24.380+0000 7f0c7ce48040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:20:24.648 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:24.498+0000 7f0c7ce48040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:20:24.648 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:24.647+0000 7f0c7ce48040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:20:24.676 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:24.380+0000 7f05bcff1040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:20:24.930 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:24.930 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:24.930 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:24.708+0000 7f0c7ce48040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:20:24.930 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:24.861+0000 7f0c7ce48040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:20:24.930 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:24.929+0000 7f0c7ce48040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:20:24.979 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:24.979 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:24.979 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:24.716+0000 7f05bcff1040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:20:24.979 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:24.779+0000 7f05bcff1040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:20:24.979 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:24.978+0000 7f05bcff1040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:20:25.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:25.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:25.211 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:25.002+0000 7f0c7ce48040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:20:25.212 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:25.164+0000 7f0c7ce48040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:20:25.494 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:25.229+0000 7f0c7ce48040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:20:25.494 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:25.295+0000 7f0c7ce48040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:20:25.494 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:25.376+0000 7f0c7ce48040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:20:25.494 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:25.435+0000 7f0c7ce48040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:20:25.494 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:25.493+0000 7f0c7ce48040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:20:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: Reconfiguring daemon mgr.x on smithi001 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2826091362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1bfb29-43e6-4e65-8d96-715b74e01491"}]: dispatch 2024-09-16T00:20:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2826091362' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1bfb29-43e6-4e65-8d96-715b74e01491"}]': finished 2024-09-16T00:20:26.203 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:20:26.203 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:26.203 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:25 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/996079391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:20:26.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:20:26.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:20:26.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:26.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: Reconfiguring daemon mgr.x on smithi001 2024-09-16T00:20:26.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2826091362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1bfb29-43e6-4e65-8d96-715b74e01491"}]: dispatch 2024-09-16T00:20:26.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2826091362' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1bfb29-43e6-4e65-8d96-715b74e01491"}]': finished 2024-09-16T00:20:26.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:20:26.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:26.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:25 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/996079391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:26.357 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:25.899+0000 7f05bcff1040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:20:26.357 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:25.969+0000 7f05bcff1040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:20:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: Reconfiguring daemon mgr.x on smithi001 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2826091362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1bfb29-43e6-4e65-8d96-715b74e01491"}]: dispatch 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2826091362' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1bfb29-43e6-4e65-8d96-715b74e01491"}]': finished 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:26.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:25 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/996079391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:26.711 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:20:26 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z[37346]: 2024-09-16T00:20:26.280+0000 7f0c7ce48040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:20:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:26.946+0000 7fa5cf7fe640 1 -- 172.21.15.1:0/3334781126 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 14) ==== 92407+0+0 (secure 0 0 0) 0x7fa5d800f8e0 con 0x7fa5e01037c0 2024-09-16T00:20:26.983 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:26.715+0000 7f05bcff1040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:20:26.983 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:26.834+0000 7f05bcff1040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:20:26.983 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:26 smithi019 ceph-mon[35186]: Standby manager daemon z started 2024-09-16T00:20:26.983 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:26 smithi019 ceph-mon[35186]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:20:26.983 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:26 smithi019 ceph-mon[35186]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:20:26.983 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:26 smithi019 ceph-mon[35186]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:20:26.983 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:26 smithi019 ceph-mon[35186]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:20:27.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:26 smithi001 ceph-mon[30934]: Standby manager daemon z started 2024-09-16T00:20:27.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:26 smithi001 ceph-mon[30934]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:20:27.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:26 smithi001 ceph-mon[30934]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:20:27.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:26 smithi001 ceph-mon[30934]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:20:27.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:26 smithi001 ceph-mon[30934]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:20:27.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:26 smithi153 ceph-mon[35538]: Standby manager daemon z started 2024-09-16T00:20:27.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:26 smithi153 ceph-mon[35538]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:20:27.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:26 smithi153 ceph-mon[35538]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:20:27.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:26 smithi153 ceph-mon[35538]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:20:27.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:26 smithi153 ceph-mon[35538]: from='mgr.? 172.21.15.153:0/2387355479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:20:27.271 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:26.982+0000 7f05bcff1040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:20:27.271 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.046+0000 7f05bcff1040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:20:27.272 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.201+0000 7f05bcff1040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:20:27.272 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.271+0000 7f05bcff1040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:20:27.565 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.333+0000 7f05bcff1040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:20:27.565 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.497+0000 7f05bcff1040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:20:27.565 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.564+0000 7f05bcff1040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:20:27.842 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.633+0000 7f05bcff1040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:20:27.842 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.719+0000 7f05bcff1040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:20:27.842 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.780+0000 7f05bcff1040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:20:27.842 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:27.841+0000 7f05bcff1040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:20:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-mon[35186]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-mon[35186]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-16T00:20:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:27 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:20:28.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:27 smithi001 ceph-mon[30934]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:28.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:27 smithi001 ceph-mon[30934]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-16T00:20:28.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:27 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:20:28.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:27 smithi153 ceph-mon[35538]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:28.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:27 smithi153 ceph-mon[35538]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-16T00:20:28.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:27 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:20:28.947 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:20:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y[36542]: 2024-09-16T00:20:28.665+0000 7f05bcff1040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:20:28.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:28.956+0000 7fa5cf7fe640 1 -- 172.21.15.1:0/3334781126 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7fa5d800f3c0 con 0x7fa5e01037c0 2024-09-16T00:20:29.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:28 smithi001 ceph-mon[30934]: Standby manager daemon y started 2024-09-16T00:20:29.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:28 smithi001 ceph-mon[30934]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:20:29.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:28 smithi001 ceph-mon[30934]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:20:29.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:28 smithi001 ceph-mon[30934]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:20:29.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:28 smithi001 ceph-mon[30934]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:20:29.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:28 smithi153 ceph-mon[35538]: Standby manager daemon y started 2024-09-16T00:20:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:28 smithi153 ceph-mon[35538]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:20:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:28 smithi153 ceph-mon[35538]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:20:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:28 smithi153 ceph-mon[35538]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:20:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:28 smithi153 ceph-mon[35538]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:20:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:28 smithi019 ceph-mon[35186]: Standby manager daemon y started 2024-09-16T00:20:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:28 smithi019 ceph-mon[35186]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:20:29.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:28 smithi019 ceph-mon[35186]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:20:29.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:28 smithi019 ceph-mon[35186]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:20:29.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:28 smithi019 ceph-mon[35186]: from='mgr.? 172.21.15.19:0/2620364831' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:20:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:29 smithi153 ceph-mon[35538]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:29 smithi153 ceph-mon[35538]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-16T00:20:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:20:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:29 smithi019 ceph-mon[35186]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:29 smithi019 ceph-mon[35186]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-16T00:20:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:20:30.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:29 smithi001 ceph-mon[30934]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:30.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:29 smithi001 ceph-mon[30934]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-16T00:20:30.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:20:32.056 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:31 smithi001 ceph-mon[30934]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:31 smithi153 ceph-mon[35538]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:31 smithi019 ceph-mon[35186]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:33.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:33.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:33.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:33.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:33.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:33.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:34.382 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:34 smithi153 ceph-mon[35538]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:34.410 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:34 smithi001 ceph-mon[30934]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:34.508 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:34 smithi019 ceph-mon[35186]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:35.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:20:35.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:35.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:20:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:35.523 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:20:35.523 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:36.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:36 smithi001 ceph-mon[30934]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:36.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:36 smithi001 ceph-mon[30934]: Deploying daemon osd.0 on smithi001 2024-09-16T00:20:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:36 smithi153 ceph-mon[35538]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:36 smithi153 ceph-mon[35538]: Deploying daemon osd.0 on smithi001 2024-09-16T00:20:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:36 smithi019 ceph-mon[35186]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:36 smithi019 ceph-mon[35186]: Deploying daemon osd.0 on smithi001 2024-09-16T00:20:38.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:38 smithi001 ceph-mon[30934]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:38 smithi153 ceph-mon[35538]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:38 smithi019 ceph-mon[35186]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:40.256 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:40 smithi001 ceph-mon[30934]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:40 smithi153 ceph-mon[35538]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:40.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:40 smithi019 ceph-mon[35186]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:41.405 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:41.405 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:41.405 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:41.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:41 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:41.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:41 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:41.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:41 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:41.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:41 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:41.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:41 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:41.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:41 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:42.407 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:42 smithi001 ceph-mon[30934]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:42.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:42 smithi153 ceph-mon[35538]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:42 smithi019 ceph-mon[35186]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:44.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:44 smithi001 ceph-mon[30934]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:44.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:44.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:44.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:44.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:44.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:44.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:44 smithi153 ceph-mon[35538]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:44.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:44.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:44.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:44.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:44.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:44 smithi019 ceph-mon[35186]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:45.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.030+0000 7fa5cf7fe640 1 -- 172.21.15.1:0/3334781126 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa5e01a2aa0 con 0x7fa5c8039500 2024-09-16T00:20:45.032 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 0 on host 'smithi001' 2024-09-16T00:20:45.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5c8039500 msgr2=0x7fa5c803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:45.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5c8039500 0x7fa5c803b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5d002aa00 tx=0x7fa5d00315b0 comp rx=0 tx=0).stop 2024-09-16T00:20:45.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e01037c0 msgr2=0x7fa5e01a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:45.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e01037c0 0x7fa5e01a1710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009fd0 tx=0x7fa5d800b260 comp rx=0 tx=0).stop 2024-09-16T00:20:45.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 shutdown_connections 2024-09-16T00:20:45.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e01037c0 0x7fa5e01a1710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:45.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5e0102e00 0x7fa5e019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:45.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5c8039500 0x7fa5c803b9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:45.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 --2- 172.21.15.1:0/3334781126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0101c00 0x7fa5e019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:45.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 >> 172.21.15.1:0/3334781126 conn(0x7fa5e00fd390 msgr2=0x7fa5e0074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:45.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.032+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 shutdown_connections 2024-09-16T00:20:45.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:20:45.033+0000 7fa5e4f02640 1 -- 172.21.15.1:0/3334781126 wait complete. 2024-09-16T00:20:45.397 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:45.398 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:45.398 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:45.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:45.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:45.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:45.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:45.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:45.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:45.734 DEBUG:teuthology.orchestra.run.smithi001:osd.0> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T00:20:45.737 INFO:tasks.cephadm:Deploying osd.1 on smithi019 with /dev/vg_nvme/lv_4... 2024-09-16T00:20:45.737 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:20:45.993 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:20:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:46 smithi019 ceph-mon[35186]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:46 smithi001 ceph-mon[30934]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:46 smithi153 ceph-mon[35538]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:47.014 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:20:46 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0[41979]: 2024-09-16T00:20:46.702+0000 7fa015971640 -1 osd.0 0 log_to_monitors true 2024-09-16T00:20:47.300 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:47 smithi001 ceph-mon[30934]: from='osd.0 [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:20:47.300 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:47 smithi001 ceph-mon[30934]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:20:47.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:47 smithi019 ceph-mon[35186]: from='osd.0 [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:20:47.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:47 smithi019 ceph-mon[35186]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:20:47.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:47 smithi153 ceph-mon[35538]: from='osd.0 [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:20:47.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:47 smithi153 ceph-mon[35538]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:20:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:20:48.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-09-16T00:20:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:48 smithi019 ceph-mon[35186]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:48 smithi019 ceph-mon[35186]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:20:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:48 smithi019 ceph-mon[35186]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:20:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:48 smithi019 ceph-mon[35186]: from='osd.0 [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:20:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:48 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:48 smithi019 ceph-mon[35186]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:20:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:48 smithi001 ceph-mon[30934]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:48 smithi001 ceph-mon[30934]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:20:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:48 smithi001 ceph-mon[30934]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:20:48.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:48 smithi001 ceph-mon[30934]: from='osd.0 [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:20:48.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:48 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:48.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:48 smithi001 ceph-mon[30934]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:20:48.452 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:20:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0[41979]: 2024-09-16T00:20:48.174+0000 7fa0118f4640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T00:20:48.452 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:20:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0[41979]: 2024-09-16T00:20:48.177+0000 7fa00d71c640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:20:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:48 smithi153 ceph-mon[35538]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:48 smithi153 ceph-mon[35538]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:20:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:48 smithi153 ceph-mon[35538]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:20:48.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:48 smithi153 ceph-mon[35538]: from='osd.0 [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:20:48.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:48 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:48.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:48 smithi153 ceph-mon[35538]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:20:48.591 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:20:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3095757638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0101cd0 msgr2=0x7f7ba0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3095757638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0101cd0 0x7f7ba0102150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009900 tx=0x7f7b8c02a410 comp rx=0 tx=0).stop 2024-09-16T00:20:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3095757638 shutdown_connections 2024-09-16T00:20:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3095757638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba0102690 0x7f7ba0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3095757638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0101cd0 0x7f7ba0102150 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3095757638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba0100ad0 0x7f7ba0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3095757638 >> 172.21.15.19:0/3095757638 conn(0x7f7ba00fc280 msgr2=0x7f7ba00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3095757638 shutdown_connections 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3095757638 wait complete. 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.996+0000 7f7ba70cd640 1 Processor -- start 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba70cd640 1 -- start start 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba70cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0100ad0 0x7f7ba006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba70cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba0101cd0 0x7f7ba019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba70cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba0102690 0x7f7ba019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba70cd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010c730 con 0x7f7ba0102690 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba70cd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010c5b0 con 0x7f7ba0100ad0 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba70cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010c8b0 con 0x7f7ba0101cd0 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba4e69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0100ad0 0x7f7ba006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba4e69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0100ad0 0x7f7ba006aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53944/0 (socket says 172.21.15.19:53944) 2024-09-16T00:20:49.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba4e69640 1 -- 172.21.15.19:0/3025890116 learned_addr learned my addr 172.21.15.19:0/3025890116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba566a640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba0102690 0x7f7ba019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7b9ffff640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba0101cd0 0x7f7ba019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba4e69640 1 -- 172.21.15.19:0/3025890116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba0101cd0 msgr2=0x7f7ba019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba4e69640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba0101cd0 0x7f7ba019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba4e69640 1 -- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba0102690 msgr2=0x7f7ba019f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba4e69640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba0102690 0x7f7ba019f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba4e69640 1 -- 172.21.15.19:0/3025890116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba019fb00 con 0x7f7ba0100ad0 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7ba4e69640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0100ad0 0x7f7ba006aad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c970 tx=0x7f7b9400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.997+0000 7f7b9dffb640 1 -- 172.21.15.19:0/3025890116 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94007c10 con 0x7f7ba0100ad0 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.998+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba019fd90 con 0x7f7ba0100ad0 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.998+0000 7f7b9dffb640 1 -- 172.21.15.19:0/3025890116 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b94007db0 con 0x7f7ba0100ad0 2024-09-16T00:20:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.998+0000 7f7b9dffb640 1 -- 172.21.15.19:0/3025890116 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400f8a0 con 0x7f7ba0100ad0 2024-09-16T00:20:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.998+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba01a01f0 con 0x7f7ba0100ad0 2024-09-16T00:20:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:48.999+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba0107e70 con 0x7f7ba0100ad0 2024-09-16T00:20:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:49.000+0000 7f7b9dffb640 1 -- 172.21.15.19:0/3025890116 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7f7b9400fa60 con 0x7f7ba0100ad0 2024-09-16T00:20:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:49.000+0000 7f7b9dffb640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7b900a31b0 0x7f7b900a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:49.000+0000 7f7b9dffb640 1 -- 172.21.15.19:0/3025890116 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f7b940ce090 con 0x7f7ba0100ad0 2024-09-16T00:20:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:49.001+0000 7f7b9ffff640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7b900a31b0 0x7f7b900a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:49.001+0000 7f7b9ffff640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7b900a31b0 0x7f7b900a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c000c00 tx=0x7f7b8c0315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:49.005+0000 7f7b9dffb640 1 -- 172.21.15.19:0/3025890116 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b9409c9d0 con 0x7f7ba0100ad0 2024-09-16T00:20:49.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:20:49.192+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7ba0062630 con 0x7f7b900a31b0 2024-09-16T00:20:49.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:20:49.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:20:49.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:49.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:49.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:49.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:49.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:49.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:20:49.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:20:49.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:49.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:49.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:49.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:49.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:49.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.605 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:20:49.605 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:20:49.605 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:49.605 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:49.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:49.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:49.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:49.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:49.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:20:50.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi001 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi001 to 17092M 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: osd.0 [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] boot 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:50 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi001 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi001 to 17092M 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: osd.0 [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] boot 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:50.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:50 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:50.500 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:20:50.500 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:20:50.500 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:20:50.500 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi001 2024-09-16T00:20:50.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi001 to 17092M 2024-09-16T00:20:50.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: osd.0 [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] boot 2024-09-16T00:20:50.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:20:50.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:20:50.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:50.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:50.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:50.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:50 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:51.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:51 smithi001 ceph-mon[30934]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:20:51.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:51 smithi153 ceph-mon[35538]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:20:51.557 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:51 smithi019 ceph-mon[35186]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:20:52.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:52 smithi001 ceph-mon[30934]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:52 smithi001 ceph-mon[30934]: from='client.? 172.21.15.19:0/147423318' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6449fa21-488a-46cb-9e6c-1391e84c0f35"}]: dispatch 2024-09-16T00:20:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:52 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6449fa21-488a-46cb-9e6c-1391e84c0f35"}]: dispatch 2024-09-16T00:20:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:52 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6449fa21-488a-46cb-9e6c-1391e84c0f35"}]': finished 2024-09-16T00:20:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:52 smithi001 ceph-mon[30934]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:20:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:52 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:52 smithi001 ceph-mon[30934]: from='client.? 172.21.15.19:0/1715456656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:52 smithi153 ceph-mon[35538]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:52 smithi153 ceph-mon[35538]: from='client.? 172.21.15.19:0/147423318' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6449fa21-488a-46cb-9e6c-1391e84c0f35"}]: dispatch 2024-09-16T00:20:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:52 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6449fa21-488a-46cb-9e6c-1391e84c0f35"}]: dispatch 2024-09-16T00:20:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:52 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6449fa21-488a-46cb-9e6c-1391e84c0f35"}]': finished 2024-09-16T00:20:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:52 smithi153 ceph-mon[35538]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:20:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:52 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:52 smithi153 ceph-mon[35538]: from='client.? 172.21.15.19:0/1715456656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:52.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:52 smithi019 ceph-mon[35186]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:52.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:52 smithi019 ceph-mon[35186]: from='client.? 172.21.15.19:0/147423318' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6449fa21-488a-46cb-9e6c-1391e84c0f35"}]: dispatch 2024-09-16T00:20:52.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:52 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6449fa21-488a-46cb-9e6c-1391e84c0f35"}]: dispatch 2024-09-16T00:20:52.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:52 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6449fa21-488a-46cb-9e6c-1391e84c0f35"}]': finished 2024-09-16T00:20:52.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:52 smithi019 ceph-mon[35186]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:20:52.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:52 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:52.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:52 smithi019 ceph-mon[35186]: from='client.? 172.21.15.19:0/1715456656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:54 smithi001 ceph-mon[30934]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:54 smithi153 ceph-mon[35538]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:54 smithi019 ceph-mon[35186]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:56 smithi001 ceph-mon[30934]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:56 smithi153 ceph-mon[35538]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:56.548 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:56 smithi019 ceph-mon[35186]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:58.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:20:58 smithi001 ceph-mon[30934]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:20:58 smithi153 ceph-mon[35538]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:58.491 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:20:58 smithi019 ceph-mon[35186]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:00 smithi001 ceph-mon[30934]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:00 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:21:00.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:00 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:00 smithi153 ceph-mon[35538]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:00 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:21:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:00 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:00.469 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:00 smithi019 ceph-mon[35186]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:00.469 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:00 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:21:00.469 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:00 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:01.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:01 smithi019 ceph-mon[35186]: Deploying daemon osd.1 on smithi019 2024-09-16T00:21:01.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:01 smithi153 ceph-mon[35538]: Deploying daemon osd.1 on smithi019 2024-09-16T00:21:01.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:01 smithi001 ceph-mon[30934]: Deploying daemon osd.1 on smithi019 2024-09-16T00:21:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:02 smithi153 ceph-mon[35538]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:02.488 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:02 smithi019 ceph-mon[35186]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:02.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:02 smithi001 ceph-mon[30934]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:04.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:04 smithi019 ceph-mon[35186]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:04.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:04 smithi153 ceph-mon[35538]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:04.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:04 smithi001 ceph-mon[30934]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:05.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:05 smithi019 ceph-mon[35186]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:05.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:05 smithi001 ceph-mon[30934]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:05.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:05 smithi153 ceph-mon[35538]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:06.465 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:06 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:06.466 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:06 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:06.466 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:06 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:06.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:06 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:06.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:06 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:06.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:06 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:06 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:06 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:06.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:06 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:07.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:07 smithi019 ceph-mon[35186]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:07.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:07 smithi001 ceph-mon[30934]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:07.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:07 smithi153 ceph-mon[35538]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:08.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.798+0000 7f7b9dffb640 1 -- 172.21.15.19:0/3025890116 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7ba0062630 con 0x7f7b900a31b0 2024-09-16T00:21:08.799 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 1 on host 'smithi019' 2024-09-16T00:21:08.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7b900a31b0 msgr2=0x7f7b900a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7b900a31b0 0x7f7b900a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c000c00 tx=0x7f7b8c0315d0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0100ad0 msgr2=0x7f7ba006aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0100ad0 0x7f7ba006aad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c970 tx=0x7f7b9400ce30 comp rx=0 tx=0).stop 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 shutdown_connections 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba0102690 0x7f7ba019f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7b900a31b0 0x7f7b900a5670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba0101cd0 0x7f7ba019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 --2- 172.21.15.19:0/3025890116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba0100ad0 0x7f7ba006aad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 >> 172.21.15.19:0/3025890116 conn(0x7f7ba00fc280 msgr2=0x7f7ba0105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 shutdown_connections 2024-09-16T00:21:08.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:21:08.800+0000 7f7ba70cd640 1 -- 172.21.15.19:0/3025890116 wait complete. 2024-09-16T00:21:09.095 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.095 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.095 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:09.095 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:09.095 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.095 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:09.095 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.095 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:09.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:09.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:09.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:09.391 DEBUG:teuthology.orchestra.run.smithi019:osd.1> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T00:21:09.432 INFO:tasks.cephadm:Deploying osd.2 on smithi153 with /dev/vg_nvme/lv_4... 2024-09-16T00:21:09.433 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:21:09.686 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:21:10.116 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:09 smithi153 ceph-mon[35538]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:09 smithi019 ceph-mon[35186]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:09 smithi001 ceph-mon[30934]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:11.357 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:11 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1[39671]: 2024-09-16T00:21:11.076+0000 7f22efeff640 -1 osd.1 0 log_to_monitors true 2024-09-16T00:21:12.080 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:21:12.106 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_4 2024-09-16T00:21:12.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:11 smithi153 ceph-mon[35538]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:12.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:11 smithi153 ceph-mon[35538]: from='osd.1 [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:21:12.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:11 smithi153 ceph-mon[35538]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:21:12.352 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:21:12.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:11 smithi019 ceph-mon[35186]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:12.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:11 smithi019 ceph-mon[35186]: from='osd.1 [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:21:12.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:11 smithi019 ceph-mon[35186]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:21:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:11 smithi001 ceph-mon[30934]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:12.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:11 smithi001 ceph-mon[30934]: from='osd.1 [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:21:12.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:11 smithi001 ceph-mon[30934]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:21:12.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.762+0000 7fa40b681640 1 -- 172.21.15.153:0/3233253870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404101cd0 msgr2=0x7fa404102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:12.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 --2- 172.21.15.153:0/3233253870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404101cd0 0x7fa404102150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa400009900 tx=0x7fa40002a410 comp rx=0 tx=0).stop 2024-09-16T00:21:12.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 -- 172.21.15.153:0/3233253870 shutdown_connections 2024-09-16T00:21:12.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 --2- 172.21.15.153:0/3233253870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa404102690 0x7fa404108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:12.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 --2- 172.21.15.153:0/3233253870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404101cd0 0x7fa404102150 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:12.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 --2- 172.21.15.153:0/3233253870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa404100ad0 0x7fa404100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:12.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 -- 172.21.15.153:0/3233253870 >> 172.21.15.153:0/3233253870 conn(0x7fa4040fc280 msgr2=0x7fa4040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 -- 172.21.15.153:0/3233253870 shutdown_connections 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 -- 172.21.15.153:0/3233253870 wait complete. 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 Processor -- start 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.763+0000 7fa40b681640 1 -- start start 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40b681640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404100ad0 0x7fa40406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40b681640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa404101cd0 0x7fa40419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40b681640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa404102690 0x7fa40419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40b681640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410c7b0 con 0x7fa404102690 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40b681640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410c630 con 0x7fa404101cd0 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40b681640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410c930 con 0x7fa404100ad0 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40941d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404100ad0 0x7fa40406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:12.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40941d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404100ad0 0x7fa40406aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59788/0 (socket says 172.21.15.153:59788) 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40941d640 1 -- 172.21.15.153:0/1291096312 learned_addr learned my addr 172.21.15.153:0/1291096312 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa408c1c640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa404101cd0 0x7fa40419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa409c1e640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa404102690 0x7fa40419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40941d640 1 -- 172.21.15.153:0/1291096312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa404101cd0 msgr2=0x7fa40419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40941d640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa404101cd0 0x7fa40419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40941d640 1 -- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa404102690 msgr2=0x7fa40419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40941d640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa404102690 0x7fa40419f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40941d640 1 -- 172.21.15.153:0/1291096312 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40419fbc0 con 0x7fa404100ad0 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40941d640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404100ad0 0x7fa40406aad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c970 tx=0x7fa3f800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:12.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa3f67fc640 1 -- 172.21.15.153:0/1291096312 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8004220 con 0x7fa404100ad0 2024-09-16T00:21:12.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa3f67fc640 1 -- 172.21.15.153:0/1291096312 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f80043c0 con 0x7fa404100ad0 2024-09-16T00:21:12.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40419fe50 con 0x7fa404100ad0 2024-09-16T00:21:12.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa3f67fc640 1 -- 172.21.15.153:0/1291096312 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800f450 con 0x7fa404100ad0 2024-09-16T00:21:12.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.764+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4041a0320 con 0x7fa404100ad0 2024-09-16T00:21:12.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.765+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa404107e70 con 0x7fa404100ad0 2024-09-16T00:21:12.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.766+0000 7fa3f67fc640 1 -- 172.21.15.153:0/1291096312 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7fa3f80079b0 con 0x7fa404100ad0 2024-09-16T00:21:12.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.767+0000 7fa3f67fc640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3f00a31b0 0x7fa3f00a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:12.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.767+0000 7fa3f67fc640 1 -- 172.21.15.153:0/1291096312 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7fa3f80cdcd0 con 0x7fa404100ad0 2024-09-16T00:21:12.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.767+0000 7fa408c1c640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3f00a31b0 0x7fa3f00a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:12.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.768+0000 7fa408c1c640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3f00a31b0 0x7fa3f00a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa400000c00 tx=0x7fa400031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:12.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.771+0000 7fa3f67fc640 1 -- 172.21.15.153:0/1291096312 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3f8014030 con 0x7fa404100ad0 2024-09-16T00:21:12.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:12.946+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa404062630 con 0x7fa3f00a31b0 2024-09-16T00:21:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:21:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='osd.1 [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:21:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:21:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:21:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi019 2024-09-16T00:21:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi019 to 17092M 2024-09-16T00:21:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:13.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:13.357 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1[39671]: 2024-09-16T00:21:12.977+0000 7f22ebe82640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T00:21:13.357 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1[39671]: 2024-09-16T00:21:12.981+0000 7f22e7caa640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:21:13.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:21:13.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='osd.1 [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:21:13.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:21:13.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:13.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:21:13.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi019 2024-09-16T00:21:13.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:13.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi019 to 17092M 2024-09-16T00:21:13.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:13.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:13.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:13.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:13.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:21:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='osd.1 [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:21:13.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:21:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:21:13.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi019 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi019 to 17092M 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:13.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:14.263 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:13 smithi153 ceph-mon[35538]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:14.263 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:13 smithi153 ceph-mon[35538]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:14.263 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:13 smithi153 ceph-mon[35538]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:21:14.264 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:13 smithi153 ceph-mon[35538]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:21:14.264 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:13 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:14.264 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:13 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:13 smithi019 ceph-mon[35186]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:13 smithi019 ceph-mon[35186]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:13 smithi019 ceph-mon[35186]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:21:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:13 smithi019 ceph-mon[35186]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:21:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:13 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:13 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:14.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:13 smithi001 ceph-mon[30934]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:14.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:13 smithi001 ceph-mon[30934]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:14.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:13 smithi001 ceph-mon[30934]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:21:14.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:13 smithi001 ceph-mon[30934]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:21:14.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:13 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:14.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:13 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:15.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:14 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:21:15.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:14 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:21:15.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:14 smithi153 ceph-mon[35538]: osd.1 [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] boot 2024-09-16T00:21:15.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:14 smithi153 ceph-mon[35538]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:21:15.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:14 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:15.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:14 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:21:15.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:14 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:21:15.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:14 smithi019 ceph-mon[35186]: osd.1 [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] boot 2024-09-16T00:21:15.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:14 smithi019 ceph-mon[35186]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:21:15.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:15.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:14 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:21:15.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:14 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:21:15.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:14 smithi001 ceph-mon[30934]: osd.1 [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] boot 2024-09-16T00:21:15.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:14 smithi001 ceph-mon[30934]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:21:15.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:14 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:21:16.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:15 smithi001 ceph-mon[30934]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:16.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:15 smithi001 ceph-mon[30934]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:21:16.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:15 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/2489512162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d"}]: dispatch 2024-09-16T00:21:16.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:15 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d"}]: dispatch 2024-09-16T00:21:16.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:15 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d"}]': finished 2024-09-16T00:21:16.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:15 smithi001 ceph-mon[30934]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:21:16.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:16.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:15 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/3888849416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:15 smithi019 ceph-mon[35186]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:15 smithi019 ceph-mon[35186]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:21:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:15 smithi019 ceph-mon[35186]: from='client.? 172.21.15.153:0/2489512162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d"}]: dispatch 2024-09-16T00:21:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:15 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d"}]: dispatch 2024-09-16T00:21:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:15 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d"}]': finished 2024-09-16T00:21:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:15 smithi019 ceph-mon[35186]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:21:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:15 smithi019 ceph-mon[35186]: from='client.? 172.21.15.153:0/3888849416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:15 smithi153 ceph-mon[35538]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:21:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:15 smithi153 ceph-mon[35538]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:21:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:15 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/2489512162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d"}]: dispatch 2024-09-16T00:21:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:15 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d"}]: dispatch 2024-09-16T00:21:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:15 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d"}]': finished 2024-09-16T00:21:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:15 smithi153 ceph-mon[35538]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:21:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:15 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/3888849416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:17 smithi019 ceph-mon[35186]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:17 smithi001 ceph-mon[30934]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:17 smithi153 ceph-mon[35538]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:20.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:19 smithi153 ceph-mon[35538]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:20.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:19 smithi019 ceph-mon[35186]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:20.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:19 smithi001 ceph-mon[30934]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:22.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:21 smithi153 ceph-mon[35538]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:22.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:21 smithi019 ceph-mon[35186]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:22.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:21 smithi001 ceph-mon[30934]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:24.032 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:23 smithi153 ceph-mon[35538]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:24.032 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:21:24.033 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:24.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:23 smithi019 ceph-mon[35186]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:24.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:21:24.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:24.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:23 smithi001 ceph-mon[30934]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:24.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:21:24.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:25 smithi153 ceph-mon[35538]: Deploying daemon osd.2 on smithi153 2024-09-16T00:21:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:25 smithi019 ceph-mon[35186]: Deploying daemon osd.2 on smithi153 2024-09-16T00:21:25.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:25 smithi001 ceph-mon[30934]: Deploying daemon osd.2 on smithi153 2024-09-16T00:21:26.105 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:26 smithi153 ceph-mon[35538]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:26 smithi019 ceph-mon[35186]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:26.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:26 smithi001 ceph-mon[30934]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:28.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:28 smithi019 ceph-mon[35186]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:28.380 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:28 smithi153 ceph-mon[35538]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:28.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:28 smithi001 ceph-mon[30934]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:30.296 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:30 smithi153 ceph-mon[35538]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:30.296 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:30.297 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:30.297 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:30 smithi019 ceph-mon[35186]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:30 smithi001 ceph-mon[30934]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:32 smithi019 ceph-mon[35186]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:32 smithi001 ceph-mon[30934]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:32 smithi153 ceph-mon[35538]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:33.450 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:33.451 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:33.451 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:33.451 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:33.451 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:33.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:33.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:33.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:33.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:33.777 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 2 on host 'smithi153' 2024-09-16T00:21:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.776+0000 7fa3f67fc640 1 -- 172.21.15.153:0/1291096312 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa404062630 con 0x7fa3f00a31b0 2024-09-16T00:21:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3f00a31b0 msgr2=0x7fa3f00a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3f00a31b0 0x7fa3f00a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa400000c00 tx=0x7fa400031400 comp rx=0 tx=0).stop 2024-09-16T00:21:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404100ad0 msgr2=0x7fa40406aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404100ad0 0x7fa40406aad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c970 tx=0x7fa3f800ce30 comp rx=0 tx=0).stop 2024-09-16T00:21:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 shutdown_connections 2024-09-16T00:21:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa404102690 0x7fa40419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa404101cd0 0x7fa40419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3f00a31b0 0x7fa3f00a5670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 --2- 172.21.15.153:0/1291096312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa404100ad0 0x7fa40406aad0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 >> 172.21.15.153:0/1291096312 conn(0x7fa4040fc280 msgr2=0x7fa404105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.778+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 shutdown_connections 2024-09-16T00:21:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:21:33.779+0000 7fa40b681640 1 -- 172.21.15.153:0/1291096312 wait complete. 2024-09-16T00:21:34.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:34 smithi153 ceph-mon[35538]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:34.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:34 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:34.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:34 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:34.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:34 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:34.431 DEBUG:teuthology.orchestra.run.smithi153:osd.2> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T00:21:34.434 INFO:tasks.cephadm:Deploying osd.3 on smithi001 with /dev/vg_nvme/lv_3... 2024-09-16T00:21:34.434 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:21:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:34 smithi001 ceph-mon[30934]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:34 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:34 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:34 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:34.508 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:34 smithi019 ceph-mon[35186]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:34.508 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:34 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:34.508 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:34 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:34.509 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:34 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:35.617 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2[40494]: 2024-09-16T00:21:35.328+0000 7f3a7451b640 -1 osd.2 0 log_to_monitors true 2024-09-16T00:21:36.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:36 smithi001 ceph-mon[30934]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:36.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:36 smithi001 ceph-mon[30934]: from='osd.2 [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:21:36.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:36 smithi001 ceph-mon[30934]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:21:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:36 smithi153 ceph-mon[35538]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:36 smithi153 ceph-mon[35538]: from='osd.2 [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:21:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:36 smithi153 ceph-mon[35538]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:21:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:36 smithi019 ceph-mon[35186]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:36 smithi019 ceph-mon[35186]: from='osd.2 [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:21:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:36 smithi019 ceph-mon[35186]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:21:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:21:37.275 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_3 2024-09-16T00:21:37.576 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:37 smithi001 ceph-mon[30934]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:21:37.576 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:37 smithi001 ceph-mon[30934]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:21:37.576 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:37 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:37.577 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:37 smithi001 ceph-mon[30934]: from='osd.2 [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:21:37.577 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:37 smithi001 ceph-mon[30934]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:21:37.577 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:37.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:37 smithi019 ceph-mon[35186]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:21:37.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:37 smithi019 ceph-mon[35186]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:21:37.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:37 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:37.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:37 smithi019 ceph-mon[35186]: from='osd.2 [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:21:37.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:37 smithi019 ceph-mon[35186]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:21:37.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:37 smithi153 ceph-mon[35538]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:21:37.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:37 smithi153 ceph-mon[35538]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:21:37.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:37 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:37.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:37 smithi153 ceph-mon[35538]: from='osd.2 [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:21:37.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:37 smithi153 ceph-mon[35538]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:21:38.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 -- 172.21.15.1:0/1962540902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070101cd0 msgr2=0x7ff070102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:38.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 --2- 172.21.15.1:0/1962540902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070101cd0 0x7ff070102150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff064009900 tx=0x7ff06402a410 comp rx=0 tx=0).stop 2024-09-16T00:21:38.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 -- 172.21.15.1:0/1962540902 shutdown_connections 2024-09-16T00:21:38.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 --2- 172.21.15.1:0/1962540902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff070102690 0x7ff070108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 --2- 172.21.15.1:0/1962540902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070101cd0 0x7ff070102150 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 --2- 172.21.15.1:0/1962540902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff070100ad0 0x7ff070100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 -- 172.21.15.1:0/1962540902 >> 172.21.15.1:0/1962540902 conn(0x7ff0700fc280 msgr2=0x7ff0700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 -- 172.21.15.1:0/1962540902 shutdown_connections 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 -- 172.21.15.1:0/1962540902 wait complete. 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.016+0000 7ff0766d4640 1 Processor -- start 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff0766d4640 1 -- start start 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff0766d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070100ad0 0x7ff070104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff0766d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff070101cd0 0x7ff0701030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff0766d4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff070102690 0x7ff0701035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:38.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff0766d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff07010c730 con 0x7ff070100ad0 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff0766d4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff07010c5b0 con 0x7ff070101cd0 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff0766d4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff07010c8b0 con 0x7ff070102690 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070100ad0 0x7ff070104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070100ad0 0x7ff070104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33034/0 (socket says 172.21.15.1:33034) 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06ffff640 1 -- 172.21.15.1:0/2668380076 learned_addr learned my addr 172.21.15.1:0/2668380076 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff074c71640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff070102690 0x7ff0701035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06f7fe640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff070101cd0 0x7ff0701030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06ffff640 1 -- 172.21.15.1:0/2668380076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff070102690 msgr2=0x7ff0701035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06ffff640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff070102690 0x7ff0701035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:38.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06ffff640 1 -- 172.21.15.1:0/2668380076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff070101cd0 msgr2=0x7ff0701030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06ffff640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff070101cd0 0x7ff0701030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06ffff640 1 -- 172.21.15.1:0/2668380076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff070103eb0 con 0x7ff070100ad0 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.017+0000 7ff06f7fe640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff070101cd0 0x7ff0701030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.018+0000 7ff06ffff640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070100ad0 0x7ff070104a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff06000c970 tx=0x7ff06000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.018+0000 7ff06d7fa640 1 -- 172.21.15.1:0/2668380076 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060007c10 con 0x7ff070100ad0 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.018+0000 7ff06d7fa640 1 -- 172.21.15.1:0/2668380076 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff060007db0 con 0x7ff070100ad0 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.018+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0701a1fb0 con 0x7ff070100ad0 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.018+0000 7ff06d7fa640 1 -- 172.21.15.1:0/2668380076 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06000f940 con 0x7ff070100ad0 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.018+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0701a2400 con 0x7ff070100ad0 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.019+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0701a2690 con 0x7ff070100ad0 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.020+0000 7ff06d7fa640 1 -- 172.21.15.1:0/2668380076 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7ff06000fae0 con 0x7ff070100ad0 2024-09-16T00:21:38.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.021+0000 7ff06d7fa640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0580a31b0 0x7ff0580a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:38.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.021+0000 7ff06d7fa640 1 -- 172.21.15.1:0/2668380076 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7ff0600ce630 con 0x7ff070100ad0 2024-09-16T00:21:38.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.022+0000 7ff06f7fe640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0580a31b0 0x7ff0580a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:38.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.022+0000 7ff06f7fe640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0580a31b0 0x7ff0580a5670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff064000c00 tx=0x7ff0640095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:38.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.025+0000 7ff06d7fa640 1 -- 172.21.15.1:0/2668380076 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff06009cb30 con 0x7ff070100ad0 2024-09-16T00:21:38.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:38.210+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff0701a2860 con 0x7ff0580a31b0 2024-09-16T00:21:38.211 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2[40494]: 2024-09-16T00:21:37.895+0000 7f3a7049e640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T00:21:38.211 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2[40494]: 2024-09-16T00:21:37.900+0000 7f3a682be640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:21:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:21:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:21:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:38.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:38.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:38.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:38.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:38.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:38 smithi019 ceph-mon[35186]: from='osd.2 ' entity='osd.2' 2024-09-16T00:21:38.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:38.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:21:38.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:21:38.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:38.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:38.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:38.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:38.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:38.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:38 smithi001 ceph-mon[30934]: from='osd.2 ' entity='osd.2' 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:38.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:38.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:38 smithi153 ceph-mon[35538]: from='osd.2 ' entity='osd.2' 2024-09-16T00:21:39.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:21:39.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:21:39.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi153 2024-09-16T00:21:39.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-16T00:21:39.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:39.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:39.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:39.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:39.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:39.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: osd.2 [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] boot 2024-09-16T00:21:39.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:21:39.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:39.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:21:39.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:21:39.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi153 2024-09-16T00:21:39.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-16T00:21:39.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:39.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:39.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:39.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:39.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:39.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: osd.2 [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] boot 2024-09-16T00:21:39.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:21:39.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi153 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:39.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: osd.2 [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] boot 2024-09-16T00:21:39.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:21:39.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:21:40.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:40 smithi019 ceph-mon[35186]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:40.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:40 smithi001 ceph-mon[30934]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:40.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:40 smithi153 ceph-mon[35538]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:21:41.612 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:21:41.612 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:41.612 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:21:41.612 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3518239283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "926a23b3-e3d6-41bf-856e-226f69d8915b"}]: dispatch 2024-09-16T00:21:41.612 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "926a23b3-e3d6-41bf-856e-226f69d8915b"}]: dispatch 2024-09-16T00:21:41.612 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:21:41.612 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "926a23b3-e3d6-41bf-856e-226f69d8915b"}]': finished 2024-09-16T00:21:41.613 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T00:21:41.613 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:41.613 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:41 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3518239283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "926a23b3-e3d6-41bf-856e-226f69d8915b"}]: dispatch 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "926a23b3-e3d6-41bf-856e-226f69d8915b"}]: dispatch 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "926a23b3-e3d6-41bf-856e-226f69d8915b"}]': finished 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:41.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:41 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:21:41.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:21:41.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:41.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:21:41.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3518239283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "926a23b3-e3d6-41bf-856e-226f69d8915b"}]: dispatch 2024-09-16T00:21:41.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "926a23b3-e3d6-41bf-856e-226f69d8915b"}]: dispatch 2024-09-16T00:21:41.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:21:41.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "926a23b3-e3d6-41bf-856e-226f69d8915b"}]': finished 2024-09-16T00:21:41.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T00:21:41.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:41.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:41 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:21:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:42 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1005254475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:42 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:21:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:42 smithi019 ceph-mon[35186]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:21:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:42 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:42.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:42 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1005254475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:42.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:42 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:21:42.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:42 smithi001 ceph-mon[30934]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:21:42.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:42 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:42.702 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:21:42 smithi001 sudo[46648]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:21:42.702 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:21:42 smithi001 sudo[46648]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:42.702 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:21:42 smithi001 sudo[46648]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:42.702 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:21:42 smithi001 sudo[46648]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:42.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:42 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1005254475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:42.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:42 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:21:42.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:42 smithi153 ceph-mon[35538]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:21:42.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:42 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:43.201 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:21:42 smithi001 sudo[46835]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:21:43.201 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:21:42 smithi001 sudo[46835]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:43.201 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:21:42 smithi001 sudo[46835]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:43.201 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:21:42 smithi001 sudo[46835]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:43.243+0000 7ff06d7fa640 1 -- 172.21.15.1:0/2668380076 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7ff060098aa0 con 0x7ff070100ad0 2024-09-16T00:21:43.277 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:43 smithi019 sudo[43498]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:21:43.277 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:43 smithi019 sudo[43498]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:43.277 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:43 smithi019 sudo[43498]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:43.277 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:43 smithi019 sudo[43498]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:43.607 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:43 smithi019 sudo[43501]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:21:43.607 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:43 smithi019 sudo[43501]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:43.607 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:43 smithi019 sudo[43501]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:43.608 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:21:43 smithi019 sudo[43501]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:43.804 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:43 smithi153 ceph-mon[35538]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:43.805 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:43 smithi153 ceph-mon[35538]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:21:43.805 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:43.805 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:43.805 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:43 smithi153 ceph-mon[35538]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:21:43.805 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:43.805 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:43 smithi153 sudo[44411]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:21:43.805 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:43 smithi153 sudo[44411]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:43.805 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:43 smithi153 sudo[44411]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:43.805 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:43 smithi153 sudo[44411]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:43 smithi019 ceph-mon[35186]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:43 smithi019 ceph-mon[35186]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:21:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:43 smithi019 ceph-mon[35186]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:21:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:43 smithi001 ceph-mon[30934]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:43 smithi001 ceph-mon[30934]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:21:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:44.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:43 smithi001 ceph-mon[30934]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:21:44.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:44.211 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:43 smithi153 sudo[44414]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:21:44.211 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:43 smithi153 sudo[44414]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:44.212 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:43 smithi153 sudo[44414]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:44.212 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:21:43 smithi153 sudo[44414]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:44.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:44 smithi001 sudo[47588]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:21:44.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:44 smithi001 sudo[47588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:44.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:44 smithi001 sudo[47588]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:44.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:44 smithi001 sudo[47588]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:45.126 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:44 smithi001 sudo[47592]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:21:45.126 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:44 smithi001 sudo[47592]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:45.126 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:44 smithi001 sudo[47592]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:45.126 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:44 smithi001 sudo[47592]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:45.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:44 smithi153 sudo[44417]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:21:45.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:44 smithi153 sudo[44417]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:45.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:44 smithi153 sudo[44417]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:45.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:44 smithi153 sudo[44417]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:45.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:45.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:45 smithi001 ceph-mon[30934]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:21:45.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:45 smithi001 ceph-mon[30934]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:21:45.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:21:45.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:21:45.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:21:45.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:21:45.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:21:45.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:21:45.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 sudo[43504]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:21:45.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 sudo[43504]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:45.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 sudo[43504]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:45.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 sudo[43504]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:45.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 sudo[44420]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 sudo[44420]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 sudo[44420]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 sudo[44420]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 ceph-mon[35538]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 ceph-mon[35538]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:21:45.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:21:45.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:21:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 sudo[43507]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:21:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 sudo[43507]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:21:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 sudo[43507]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:21:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 sudo[43507]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:21:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 ceph-mon[35186]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:21:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 ceph-mon[35186]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:21:46.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:21:46.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:21:46.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:21:46.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:21:46.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:21:46.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:21:46.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:46 smithi001 ceph-mon[30934]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:46.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:46 smithi001 ceph-mon[30934]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:21:46.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:46 smithi001 ceph-mon[30934]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:21:46.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:46 smithi001 ceph-mon[30934]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:21:46.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:21:46.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:21:46.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:21:46.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:46 smithi001 ceph-mon[30934]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:21:46.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:46 smithi153 ceph-mon[35538]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:46.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:46 smithi153 ceph-mon[35538]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:21:46.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:46 smithi153 ceph-mon[35538]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:21:46.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:46 smithi153 ceph-mon[35538]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:21:46.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:21:46.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:21:46.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:21:46.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:46 smithi153 ceph-mon[35538]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:21:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:46 smithi019 ceph-mon[35186]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:46 smithi019 ceph-mon[35186]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:21:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:46 smithi019 ceph-mon[35186]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:21:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:46 smithi019 ceph-mon[35186]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:21:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:21:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:21:46.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:21:46.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:46 smithi019 ceph-mon[35186]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:21:47.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:47 smithi001 ceph-mon[30934]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:47.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:47 smithi153 ceph-mon[35538]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:47.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:47 smithi019 ceph-mon[35186]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:49.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:49 smithi019 ceph-mon[35186]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:49.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:21:49.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:49.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:49 smithi019 ceph-mon[35186]: Deploying daemon osd.3 on smithi001 2024-09-16T00:21:49.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:49 smithi001 ceph-mon[30934]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:49.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:21:49.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:49.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:49 smithi001 ceph-mon[30934]: Deploying daemon osd.3 on smithi001 2024-09-16T00:21:49.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:49 smithi153 ceph-mon[35538]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:21:49.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:21:49.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:49.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:49 smithi153 ceph-mon[35538]: Deploying daemon osd.3 on smithi001 2024-09-16T00:21:51.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:51 smithi019 ceph-mon[35186]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-16T00:21:51.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:51 smithi001 ceph-mon[30934]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-16T00:21:51.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:51 smithi153 ceph-mon[35538]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-16T00:21:53.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:53 smithi019 ceph-mon[35186]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T00:21:53.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:53 smithi001 ceph-mon[30934]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T00:21:53.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:53 smithi153 ceph-mon[35538]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T00:21:55.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:55 smithi019 ceph-mon[35186]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T00:21:55.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:55 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:55.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:55 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:55.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:55 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:55.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:55 smithi001 ceph-mon[30934]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T00:21:55.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:55 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:55.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:55 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:55.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:55 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:55.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:55 smithi153 ceph-mon[35538]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T00:21:55.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:55 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:55.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:55 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:55.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:55 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:57.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:57 smithi019 ceph-mon[35186]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T00:21:57.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:57 smithi001 ceph-mon[30934]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T00:21:57.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:57 smithi153 ceph-mon[35538]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T00:21:59.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:58 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:59.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:58 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:59.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:58 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:59.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:58 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:59.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:58 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:59.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:58 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:59.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:58 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:59.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:58 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:59.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:58 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:59.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:58 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:59.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.169+0000 7ff06d7fa640 1 -- 172.21.15.1:0/2668380076 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff0701a2860 con 0x7ff0580a31b0 2024-09-16T00:21:59.170 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 3 on host 'smithi001' 2024-09-16T00:21:59.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0580a31b0 msgr2=0x7ff0580a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:59.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0580a31b0 0x7ff0580a5670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff064000c00 tx=0x7ff0640095f0 comp rx=0 tx=0).stop 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070100ad0 msgr2=0x7ff070104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070100ad0 0x7ff070104a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff06000c970 tx=0x7ff06000ce30 comp rx=0 tx=0).stop 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 shutdown_connections 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0580a31b0 0x7ff0580a5670 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff070102690 0x7ff0701035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff070101cd0 0x7ff0701030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 --2- 172.21.15.1:0/2668380076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff070100ad0 0x7ff070104a60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 >> 172.21.15.1:0/2668380076 conn(0x7ff0700fc280 msgr2=0x7ff0700fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 shutdown_connections 2024-09-16T00:21:59.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:21:59.171+0000 7ff0766d4640 1 -- 172.21.15.1:0/2668380076 wait complete. 2024-09-16T00:21:59.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:58 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:59.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:58 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:59.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:58 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:59.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:58 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:59.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:58 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:21:59.862 DEBUG:teuthology.orchestra.run.smithi001:osd.3> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T00:21:59.864 INFO:tasks.cephadm:Deploying osd.4 on smithi019 with /dev/vg_nvme/lv_3... 2024-09-16T00:21:59.864 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:22:00.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:59 smithi019 ceph-mon[35186]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:22:00.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:00.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:00.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:21:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:22:00.156 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:59 smithi001 ceph-mon[30934]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:22:00.157 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:00.157 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:00.157 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:21:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:00.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:59 smithi153 ceph-mon[35538]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:22:00.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:00.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:00.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:21:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:01.018 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:00 smithi001 ceph-mon[30934]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:22:01.018 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:22:00 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3[48428]: 2024-09-16T00:22:00.799+0000 7fac2e396640 -1 osd.3 0 log_to_monitors true 2024-09-16T00:22:01.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:00 smithi019 ceph-mon[35186]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:22:01.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:00 smithi153 ceph-mon[35538]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:22:02.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:01 smithi019 ceph-mon[35186]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:22:02.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:01 smithi019 ceph-mon[35186]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:22:02.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:01 smithi019 ceph-mon[35186]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:22:02.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:01 smithi019 ceph-mon[35186]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:22:02.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:02.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:01 smithi001 ceph-mon[30934]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:22:02.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:01 smithi001 ceph-mon[30934]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:22:02.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:01 smithi001 ceph-mon[30934]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:22:02.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:01 smithi001 ceph-mon[30934]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:22:02.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:02.203 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:22:01 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3[48428]: 2024-09-16T00:22:01.843+0000 7fac2a319640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T00:22:02.203 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:22:01 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3[48428]: 2024-09-16T00:22:01.847+0000 7fac26141640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:22:02.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:01 smithi153 ceph-mon[35538]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:22:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:01 smithi153 ceph-mon[35538]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:22:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:01 smithi153 ceph-mon[35538]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:22:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:01 smithi153 ceph-mon[35538]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:22:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:22:02.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-09-16T00:22:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:22:02.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:22:02.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T00:22:02.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] boot 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:02.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:03.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:22:03.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T00:22:03.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:03.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:03.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] boot 2024-09-16T00:22:03.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:22:03.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:03.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:03.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:03.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:03.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:03.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:03.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:03.111 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:03.111 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: osd.3 [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] boot 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:03.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:03.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:03.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:03.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:03.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 -- 172.21.15.19:0/1278258671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c101cd0 msgr2=0x7f792c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 --2- 172.21.15.19:0/1278258671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c101cd0 0x7f792c102150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7918009900 tx=0x7f791802a410 comp rx=0 tx=0).stop 2024-09-16T00:22:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 -- 172.21.15.19:0/1278258671 shutdown_connections 2024-09-16T00:22:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 --2- 172.21.15.19:0/1278258671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c102690 0x7f792c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 --2- 172.21.15.19:0/1278258671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c101cd0 0x7f792c102150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 --2- 172.21.15.19:0/1278258671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f792c100ad0 0x7f792c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 -- 172.21.15.19:0/1278258671 >> 172.21.15.19:0/1278258671 conn(0x7f792c0fc280 msgr2=0x7f792c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 -- 172.21.15.19:0/1278258671 shutdown_connections 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 -- 172.21.15.19:0/1278258671 wait complete. 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 Processor -- start 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.249+0000 7f79330a7640 1 -- start start 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f79330a7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f792c100ad0 0x7f792c196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f79330a7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c101cd0 0x7f792c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f79330a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c102690 0x7f792c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f79330a7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f792c10c750 con 0x7f792c100ad0 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f79330a7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f792c10c5d0 con 0x7f792c102690 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f79330a7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f792c10c8d0 con 0x7f792c101cd0 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7931644640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c102690 0x7f792c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7930e43640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f792c100ad0 0x7f792c196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7931644640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c102690 0x7f792c19e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57310/0 (socket says 172.21.15.19:57310) 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7931644640 1 -- 172.21.15.19:0/416492981 learned_addr learned my addr 172.21.15.19:0/416492981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f792bfff640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c101cd0 0x7f792c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7931644640 1 -- 172.21.15.19:0/416492981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c101cd0 msgr2=0x7f792c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7931644640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c101cd0 0x7f792c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7931644640 1 -- 172.21.15.19:0/416492981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f792c100ad0 msgr2=0x7f792c196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7931644640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f792c100ad0 0x7f792c196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7931644640 1 -- 172.21.15.19:0/416492981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c19ea40 con 0x7f792c102690 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7931644640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c102690 0x7f792c19e3d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f792400cad0 tx=0x7f792400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7929ffb640 1 -- 172.21.15.19:0/416492981 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792400ae20 con 0x7f792c102690 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c19ed30 con 0x7f792c102690 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7929ffb640 1 -- 172.21.15.19:0/416492981 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7924005020 con 0x7f792c102690 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.250+0000 7f7929ffb640 1 -- 172.21.15.19:0/416492981 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792400f6b0 con 0x7f792c102690 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.251+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c19f230 con 0x7f792c102690 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.252+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c107f70 con 0x7f792c102690 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.252+0000 7f7929ffb640 1 -- 172.21.15.19:0/416492981 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f79240051d0 con 0x7f792c102690 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.253+0000 7f7929ffb640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f791c0a32f0 0x7f791c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.253+0000 7f7929ffb640 1 -- 172.21.15.19:0/416492981 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f79240cea60 con 0x7f792c102690 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.253+0000 7f7930e43640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f791c0a32f0 0x7f791c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.254+0000 7f7930e43640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f791c0a32f0 0x7f791c0a57b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f792c101b30 tx=0x7f792000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:03.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.258+0000 7f7929ffb640 1 -- 172.21.15.19:0/416492981 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7924014030 con 0x7f792c102690 2024-09-16T00:22:03.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:03.446+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f792c062630 con 0x7f791c0a32f0 2024-09-16T00:22:04.086 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:03 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:22:04.086 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:03 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:22:04.086 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:03 smithi019 ceph-mon[35186]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:22:04.086 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:03 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi001 2024-09-16T00:22:04.086 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:03 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi001 to 8546M 2024-09-16T00:22:04.086 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:03 smithi019 ceph-mon[35186]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:22:04.087 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:22:04.087 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:22:04.087 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:04.113 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:03 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:22:04.113 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:03 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:22:04.113 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:03 smithi153 ceph-mon[35538]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:22:04.113 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:03 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi001 2024-09-16T00:22:04.113 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:03 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi001 to 8546M 2024-09-16T00:22:04.114 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:03 smithi153 ceph-mon[35538]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:22:04.114 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:22:04.114 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:22:04.114 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:04.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:03 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:22:04.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:03 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:22:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:03 smithi001 ceph-mon[30934]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:22:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:03 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi001 2024-09-16T00:22:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:03 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi001 to 8546M 2024-09-16T00:22:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:03 smithi001 ceph-mon[30934]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:22:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:22:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:22:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:05.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:04 smithi019 ceph-mon[35186]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:22:05.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:04 smithi019 ceph-mon[35186]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T00:22:05.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:04 smithi001 ceph-mon[30934]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:22:05.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:04 smithi001 ceph-mon[30934]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T00:22:05.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:04 smithi153 ceph-mon[35538]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:22:05.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:04 smithi153 ceph-mon[35538]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T00:22:06.128 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:05 smithi019 ceph-mon[35186]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:06.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:05 smithi019 ceph-mon[35186]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T00:22:06.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:05 smithi019 ceph-mon[35186]: from='client.? 172.21.15.19:0/2586176775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8536388-50a8-4dd3-acd2-bd8a71f8cfad"}]: dispatch 2024-09-16T00:22:06.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:05 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8536388-50a8-4dd3-acd2-bd8a71f8cfad"}]: dispatch 2024-09-16T00:22:06.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:05 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8536388-50a8-4dd3-acd2-bd8a71f8cfad"}]': finished 2024-09-16T00:22:06.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:05 smithi019 ceph-mon[35186]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:22:06.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:06.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:05 smithi001 ceph-mon[30934]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:06.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:05 smithi001 ceph-mon[30934]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T00:22:06.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:05 smithi001 ceph-mon[30934]: from='client.? 172.21.15.19:0/2586176775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8536388-50a8-4dd3-acd2-bd8a71f8cfad"}]: dispatch 2024-09-16T00:22:06.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:05 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8536388-50a8-4dd3-acd2-bd8a71f8cfad"}]: dispatch 2024-09-16T00:22:06.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:05 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8536388-50a8-4dd3-acd2-bd8a71f8cfad"}]': finished 2024-09-16T00:22:06.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:05 smithi001 ceph-mon[30934]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:22:06.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:06.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:05 smithi153 ceph-mon[35538]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:05 smithi153 ceph-mon[35538]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T00:22:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:05 smithi153 ceph-mon[35538]: from='client.? 172.21.15.19:0/2586176775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8536388-50a8-4dd3-acd2-bd8a71f8cfad"}]: dispatch 2024-09-16T00:22:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:05 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8536388-50a8-4dd3-acd2-bd8a71f8cfad"}]: dispatch 2024-09-16T00:22:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:05 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8536388-50a8-4dd3-acd2-bd8a71f8cfad"}]': finished 2024-09-16T00:22:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:05 smithi153 ceph-mon[35538]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:22:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:07.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:06 smithi001 ceph-mon[30934]: from='client.? 172.21.15.19:0/2013728247' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:22:07.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:06 smithi153 ceph-mon[35538]: from='client.? 172.21.15.19:0/2013728247' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:22:07.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:06 smithi019 ceph-mon[35186]: from='client.? 172.21.15.19:0/2013728247' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:22:08.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:07 smithi001 ceph-mon[30934]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-16T00:22:08.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:07 smithi153 ceph-mon[35538]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-16T00:22:08.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:07 smithi019 ceph-mon[35186]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-16T00:22:10.133 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:09 smithi019 ceph-mon[35186]: pgmap v101: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-16T00:22:10.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:09 smithi001 ceph-mon[30934]: pgmap v101: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-16T00:22:10.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:09 smithi153 ceph-mon[35538]: pgmap v101: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-16T00:22:11.998 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:11 smithi019 ceph-mon[35186]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:22:12.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:11 smithi001 ceph-mon[30934]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:22:12.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:11 smithi153 ceph-mon[35538]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:22:14.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:13 smithi019 ceph-mon[35186]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T00:22:14.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:13 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:22:14.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:13 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:13 smithi001 ceph-mon[30934]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T00:22:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:13 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:22:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:13 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:14.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:13 smithi153 ceph-mon[35538]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T00:22:14.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:13 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:22:14.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:13 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:15.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:14 smithi001 ceph-mon[30934]: Deploying daemon osd.4 on smithi019 2024-09-16T00:22:15.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:14 smithi153 ceph-mon[35538]: Deploying daemon osd.4 on smithi019 2024-09-16T00:22:15.297 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:14 smithi019 ceph-mon[35186]: Deploying daemon osd.4 on smithi019 2024-09-16T00:22:16.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:15 smithi019 ceph-mon[35186]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:16.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:15 smithi001 ceph-mon[30934]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:16.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:15 smithi153 ceph-mon[35538]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:18.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:17 smithi001 ceph-mon[30934]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:18.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:17 smithi153 ceph-mon[35538]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:17 smithi019 ceph-mon[35186]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:20.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:19 smithi001 ceph-mon[30934]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:20.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:19 smithi153 ceph-mon[35538]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:20.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:19 smithi019 ceph-mon[35186]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:21.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:21.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:21.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:21.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:21.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:21.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:21.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:21.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:21.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:22.330 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:22 smithi019 ceph-mon[35186]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:22.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:22 smithi001 ceph-mon[30934]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:22 smithi153 ceph-mon[35538]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:23 smithi019 ceph-mon[35186]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:23.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:23.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:23.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:23.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:23.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:23 smithi001 ceph-mon[30934]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:23 smithi153 ceph-mon[35538]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:23.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:23.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:23.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:23.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:23.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:23.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.987+0000 7f7929ffb640 1 -- 172.21.15.19:0/416492981 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f792c062630 con 0x7f791c0a32f0 2024-09-16T00:22:23.988 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 4 on host 'smithi019' 2024-09-16T00:22:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f791c0a32f0 msgr2=0x7f791c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f791c0a32f0 0x7f791c0a57b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f792c101b30 tx=0x7f792000b390 comp rx=0 tx=0).stop 2024-09-16T00:22:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c102690 msgr2=0x7f792c19e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c102690 0x7f792c19e3d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f792400cad0 tx=0x7f792400cf90 comp rx=0 tx=0).stop 2024-09-16T00:22:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 shutdown_connections 2024-09-16T00:22:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792c102690 0x7f792c19e3d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f791c0a32f0 0x7f791c0a57b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f792c101cd0 0x7f792c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 --2- 172.21.15.19:0/416492981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f792c100ad0 0x7f792c196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 >> 172.21.15.19:0/416492981 conn(0x7f792c0fc280 msgr2=0x7f792c105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:23.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 shutdown_connections 2024-09-16T00:22:23.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:22:23.990+0000 7f79330a7640 1 -- 172.21.15.19:0/416492981 wait complete. 2024-09-16T00:22:24.552 DEBUG:teuthology.orchestra.run.smithi019:osd.4> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T00:22:24.554 INFO:tasks.cephadm:Deploying osd.5 on smithi153 with /dev/vg_nvme/lv_3... 2024-09-16T00:22:24.554 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:22:24.811 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:22:24.845 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:24.845 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:24.845 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:24.923 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:24.923 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:24.923 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:24.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:24.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:24.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:26.143 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:25 smithi153 ceph-mon[35538]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:26.215 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:25 smithi019 ceph-mon[35186]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:26.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:25 smithi001 ceph-mon[30934]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:26.607 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:22:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4[46243]: 2024-09-16T00:22:26.214+0000 7ff895127640 -1 osd.4 0 log_to_monitors true 2024-09-16T00:22:27.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:26 smithi001 ceph-mon[30934]: from='osd.4 [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:22:27.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:26 smithi001 ceph-mon[30934]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:22:27.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:26 smithi153 ceph-mon[35538]: from='osd.4 [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:22:27.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:26 smithi153 ceph-mon[35538]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:22:27.256 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:22:27.284 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_3 2024-09-16T00:22:27.288 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:26 smithi019 ceph-mon[35186]: from='osd.4 [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:22:27.288 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:26 smithi019 ceph-mon[35186]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:22:27.538 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:22:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.966+0000 7f32b8de7640 1 -- 172.21.15.153:0/1962741884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 msgr2=0x7f32b4102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.966+0000 7f32b8de7640 1 --2- 172.21.15.153:0/1962741884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 0x7f32b4102530 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009930 tx=0x7f32a402a420 comp rx=0 tx=0).stop 2024-09-16T00:22:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.966+0000 7f32b8de7640 1 -- 172.21.15.153:0/1962741884 shutdown_connections 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.966+0000 7f32b8de7640 1 --2- 172.21.15.153:0/1962741884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b41017a0 0x7f32b4109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.966+0000 7f32b8de7640 1 --2- 172.21.15.153:0/1962741884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b4100de0 0x7f32b4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.966+0000 7f32b8de7640 1 --2- 172.21.15.153:0/1962741884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 0x7f32b4102530 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.966+0000 7f32b8de7640 1 -- 172.21.15.153:0/1962741884 >> 172.21.15.153:0/1962741884 conn(0x7f32b40fba80 msgr2=0x7f32b40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.966+0000 7f32b8de7640 1 -- 172.21.15.153:0/1962741884 shutdown_connections 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.966+0000 7f32b8de7640 1 -- 172.21.15.153:0/1962741884 wait complete. 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b8de7640 1 Processor -- start 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b8de7640 1 -- start start 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b8de7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b4100de0 0x7f32b4100870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b8de7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b41017a0 0x7f32b419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b8de7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 0x7f32b419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b8de7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410c760 con 0x7f32b4100de0 2024-09-16T00:22:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b8de7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410c5e0 con 0x7f32b41017a0 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b8de7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410c8e0 con 0x7f32b4102130 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b2d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 0x7f32b419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b2d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 0x7f32b419f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58908/0 (socket says 172.21.15.153:58908) 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b4100de0 0x7f32b4100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b2d9d640 1 -- 172.21.15.153:0/192099522 learned_addr learned my addr 172.21.15.153:0/192099522 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.967+0000 7f32b1d9b640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b41017a0 0x7f32b419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32b2d9d640 1 -- 172.21.15.153:0/192099522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b41017a0 msgr2=0x7f32b419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32b2d9d640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b41017a0 0x7f32b419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32b2d9d640 1 -- 172.21.15.153:0/192099522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b4100de0 msgr2=0x7f32b4100870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32b2d9d640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b4100de0 0x7f32b4100870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32b2d9d640 1 -- 172.21.15.153:0/192099522 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b419fb30 con 0x7f32b4102130 2024-09-16T00:22:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32b1d9b640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b41017a0 0x7f32b419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32b2d9d640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 0x7f32b419f270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00cce0 tx=0x7f32ac00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32ab7fe640 1 -- 172.21.15.153:0/192099522 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00f450 con 0x7f32b4102130 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b419fe20 con 0x7f32b4102130 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32ab7fe640 1 -- 172.21.15.153:0/192099522 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ac00fad0 con 0x7f32b4102130 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32ab7fe640 1 -- 172.21.15.153:0/192099522 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac018c10 con 0x7f32b4102130 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.968+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b41a0280 con 0x7f32b4102130 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.969+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b4107ff0 con 0x7f32b4102130 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.970+0000 7f32ab7fe640 1 -- 172.21.15.153:0/192099522 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f32ac00f5f0 con 0x7f32b4102130 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.971+0000 7f32ab7fe640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32a00a3240 0x7f32a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.971+0000 7f32ab7fe640 1 -- 172.21.15.153:0/192099522 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3805+0+0 (secure 0 0 0) 0x7f32ac0ceea0 con 0x7f32b4102130 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.971+0000 7f32b259c640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32a00a3240 0x7f32a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.972+0000 7f32b259c640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32a00a3240 0x7f32a00a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32a4031440 tx=0x7f32a4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:27.975+0000 7f32ab7fe640 1 -- 172.21.15.153:0/192099522 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32ac014030 con 0x7f32b4102130 2024-09-16T00:22:28.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:28.156+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f32b4062630 con 0x7f32a00a3240 2024-09-16T00:22:28.157 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:28.157 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='osd.4 [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:28.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.357 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4[46243]: 2024-09-16T00:22:28.003+0000 7ff8910aa640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T00:22:28.357 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4[46243]: 2024-09-16T00:22:28.011+0000 7ff88ced2640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:22:28.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='osd.4 [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:27 smithi001 ceph-mon[30934]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:28.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:27 smithi001 ceph-mon[30934]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:27 smithi001 ceph-mon[30934]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:27 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:27 smithi001 ceph-mon[30934]: from='osd.4 [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:27 smithi001 ceph-mon[30934]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:28.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:28.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi019 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi019 to 8546M 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:22:29.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi019 2024-09-16T00:22:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi019 to 8546M 2024-09-16T00:22:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:22:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:22:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:22:29.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:22:29.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:22:29.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi019 2024-09-16T00:22:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi019 to 8546M 2024-09-16T00:22:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:22:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:22:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:22:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:22:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:22:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:30 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:22:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:30 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:22:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:30 smithi153 ceph-mon[35538]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:30 smithi153 ceph-mon[35538]: osd.4 [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] boot 2024-09-16T00:22:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:30 smithi153 ceph-mon[35538]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:22:30.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:30 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:22:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:30 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:22:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:30 smithi019 ceph-mon[35186]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:30 smithi019 ceph-mon[35186]: osd.4 [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] boot 2024-09-16T00:22:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:30 smithi019 ceph-mon[35186]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:22:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:30 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:22:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:30 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:22:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:30 smithi001 ceph-mon[30934]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:22:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:30 smithi001 ceph-mon[30934]: osd.4 [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] boot 2024-09-16T00:22:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:30 smithi001 ceph-mon[30934]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:22:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:22:31.150 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:31 smithi153 ceph-mon[35538]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:22:31.150 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:31 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/2825375619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "617ccae2-59c9-45bd-83f0-59b1f3555020"}]: dispatch 2024-09-16T00:22:31.150 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:31 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "617ccae2-59c9-45bd-83f0-59b1f3555020"}]: dispatch 2024-09-16T00:22:31.150 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:31 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "617ccae2-59c9-45bd-83f0-59b1f3555020"}]': finished 2024-09-16T00:22:31.150 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:31 smithi153 ceph-mon[35538]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:22:31.150 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:31.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:31 smithi019 ceph-mon[35186]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:22:31.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:31 smithi019 ceph-mon[35186]: from='client.? 172.21.15.153:0/2825375619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "617ccae2-59c9-45bd-83f0-59b1f3555020"}]: dispatch 2024-09-16T00:22:31.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:31 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "617ccae2-59c9-45bd-83f0-59b1f3555020"}]: dispatch 2024-09-16T00:22:31.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:31 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "617ccae2-59c9-45bd-83f0-59b1f3555020"}]': finished 2024-09-16T00:22:31.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:31 smithi019 ceph-mon[35186]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:22:31.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:31 smithi001 ceph-mon[30934]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:22:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:31 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/2825375619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "617ccae2-59c9-45bd-83f0-59b1f3555020"}]: dispatch 2024-09-16T00:22:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:31 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "617ccae2-59c9-45bd-83f0-59b1f3555020"}]: dispatch 2024-09-16T00:22:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:31 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "617ccae2-59c9-45bd-83f0-59b1f3555020"}]': finished 2024-09-16T00:22:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:31 smithi001 ceph-mon[30934]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:22:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:32 smithi019 ceph-mon[35186]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:32 smithi019 ceph-mon[35186]: from='client.? 172.21.15.153:0/2369896226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:22:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:32 smithi001 ceph-mon[30934]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:32 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/2369896226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:22:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:32 smithi153 ceph-mon[35538]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:32.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:32 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/2369896226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:22:34.273 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:34 smithi153 ceph-mon[35538]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:34 smithi019 ceph-mon[35186]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:34 smithi001 ceph-mon[30934]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:36 smithi019 ceph-mon[35186]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:36.421 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:36 smithi153 ceph-mon[35538]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:36 smithi001 ceph-mon[30934]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:38.078 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:38 smithi153 ceph-mon[35538]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:38.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:38 smithi019 ceph-mon[35186]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:38.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:38 smithi001 ceph-mon[30934]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:40 smithi153 ceph-mon[35538]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:40 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:22:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:40 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:40 smithi153 ceph-mon[35538]: Deploying daemon osd.5 on smithi153 2024-09-16T00:22:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:40 smithi019 ceph-mon[35186]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:40 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:22:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:40 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:40 smithi019 ceph-mon[35186]: Deploying daemon osd.5 on smithi153 2024-09-16T00:22:40.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:40 smithi001 ceph-mon[30934]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:40.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:40 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:22:40.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:40 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:40.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:40 smithi001 ceph-mon[30934]: Deploying daemon osd.5 on smithi153 2024-09-16T00:22:42.180 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:42 smithi153 ceph-mon[35538]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:42.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:42 smithi019 ceph-mon[35186]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:42.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:42 smithi001 ceph-mon[30934]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:44.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:44 smithi153 ceph-mon[35538]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:44.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:44 smithi019 ceph-mon[35186]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:44.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:44 smithi001 ceph-mon[30934]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:46.112 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:46 smithi153 ceph-mon[35538]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:46.112 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:46.112 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:46.112 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:46 smithi019 ceph-mon[35186]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:46 smithi001 ceph-mon[30934]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:48.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:48 smithi153 ceph-mon[35538]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:48 smithi019 ceph-mon[35186]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:48 smithi001 ceph-mon[30934]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:49.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:49.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:49.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:49.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:49.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:49.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.292+0000 7f32ab7fe640 1 -- 172.21.15.153:0/192099522 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f32b4062630 con 0x7f32a00a3240 2024-09-16T00:22:49.294 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 5 on host 'smithi153' 2024-09-16T00:22:49.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32a00a3240 msgr2=0x7f32a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32a00a3240 0x7f32a00a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32a4031440 tx=0x7f32a4000bc0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 msgr2=0x7f32b419f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 0x7f32b419f270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00cce0 tx=0x7f32ac00a620 comp rx=0 tx=0).stop 2024-09-16T00:22:49.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 shutdown_connections 2024-09-16T00:22:49.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32a00a3240 0x7f32a00a5700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102130 0x7f32b419f270 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b41017a0 0x7f32b419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 --2- 172.21.15.153:0/192099522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b4100de0 0x7f32b4100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 >> 172.21.15.153:0/192099522 conn(0x7f32b40fba80 msgr2=0x7f32b41051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:49.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.295+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 shutdown_connections 2024-09-16T00:22:49.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:22:49.296+0000 7f32b8de7640 1 -- 172.21.15.153:0/192099522 wait complete. 2024-09-16T00:22:49.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:49.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:49.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:49.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:49.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:49.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:49.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:49.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:49.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:49.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:49.876 DEBUG:teuthology.orchestra.run.smithi153:osd.5> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T00:22:49.879 INFO:tasks.cephadm:Deploying osd.6 on smithi001 with /dev/vg_nvme/lv_2... 2024-09-16T00:22:49.879 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:22:50.148 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:50.174 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:50 smithi153 ceph-mon[35538]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:50.175 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:50 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:50.175 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:50 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:50.175 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:50 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:50.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:50 smithi001 ceph-mon[30934]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:50.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:50 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:50.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:50 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:50.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:50 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:50.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:50 smithi019 ceph-mon[35186]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:50.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:50 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:50.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:50 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:50.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:50 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:51.962 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:22:51 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:22:51.670+0000 7f0c2be57640 -1 osd.5 0 log_to_monitors true 2024-09-16T00:22:52.093 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:52 smithi001 ceph-mon[30934]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:52.093 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:52 smithi001 ceph-mon[30934]: from='osd.5 [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:52.094 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:52 smithi001 ceph-mon[30934]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:52 smithi019 ceph-mon[35186]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:52 smithi019 ceph-mon[35186]: from='osd.5 [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:52 smithi019 ceph-mon[35186]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:52.391 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:52 smithi153 ceph-mon[35538]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:52.392 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:52 smithi153 ceph-mon[35538]: from='osd.5 [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:52.392 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:52 smithi153 ceph-mon[35538]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:52.869 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:22:52.896 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_2 2024-09-16T00:22:53.151 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:53.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='osd.5 [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:53.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:22:53.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:22:53.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:53.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='osd.5 [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:22:53.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:22:53.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:53.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:53.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:53.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:53.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='osd.5 [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:53.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:22:53.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.605+0000 7f09e43ea640 1 -- 172.21.15.1:0/647348798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 msgr2=0x7f09dc109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:53.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.605+0000 7f09e43ea640 1 --2- 172.21.15.1:0/647348798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 0x7f09dc109d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009ca0 tx=0x7f09d402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 -- 172.21.15.1:0/647348798 shutdown_connections 2024-09-16T00:22:53.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 --2- 172.21.15.1:0/647348798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 0x7f09dc109d50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 --2- 172.21.15.1:0/647348798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09dc104fd0 0x7f09dc1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 --2- 172.21.15.1:0/647348798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09dc1026a0 0x7f09dc104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 -- 172.21.15.1:0/647348798 >> 172.21.15.1:0/647348798 conn(0x7f09dc0fc2a0 msgr2=0x7f09dc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 -- 172.21.15.1:0/647348798 shutdown_connections 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 -- 172.21.15.1:0/647348798 wait complete. 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 Processor -- start 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 -- start start 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09dc1026a0 0x7f09dc102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09dc104fd0 0x7f09dc100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 0x7f09dc100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc10d8a0 con 0x7f09dc107900 2024-09-16T00:22:53.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc10d720 con 0x7f09dc104fd0 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.606+0000 7f09e43ea640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc10da20 con 0x7f09dc1026a0 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2987640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 0x7f09dc100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2987640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 0x7f09dc100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52602/0 (socket says 172.21.15.1:52602) 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2987640 1 -- 172.21.15.1:0/563689322 learned_addr learned my addr 172.21.15.1:0/563689322 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2186640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09dc1026a0 0x7f09dc102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e1985640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09dc104fd0 0x7f09dc100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2987640 1 -- 172.21.15.1:0/563689322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09dc1026a0 msgr2=0x7f09dc102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2987640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09dc1026a0 0x7f09dc102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2987640 1 -- 172.21.15.1:0/563689322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09dc104fd0 msgr2=0x7f09dc100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2987640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09dc104fd0 0x7f09dc100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2987640 1 -- 172.21.15.1:0/563689322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc101450 con 0x7f09dc107900 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e2987640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 0x7f09dc100b90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f09d4031ae0 tx=0x7f09d4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09cf7fe640 1 -- 172.21.15.1:0/563689322 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4003c30 con 0x7f09dc107900 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc1a2000 con 0x7f09dc107900 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09cf7fe640 1 -- 172.21.15.1:0/563689322 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4003dd0 con 0x7f09dc107900 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09cf7fe640 1 -- 172.21.15.1:0/563689322 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d402fcb0 con 0x7f09dc107900 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.607+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc1a2510 con 0x7f09dc107900 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.608+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc0fdd90 con 0x7f09dc107900 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.610+0000 7f09cf7fe640 1 -- 172.21.15.1:0/563689322 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f09d4004050 con 0x7f09dc107900 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.610+0000 7f09cf7fe640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09c80a32c0 0x7f09c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.611+0000 7f09e2186640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09c80a32c0 0x7f09c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.611+0000 7f09cf7fe640 1 -- 172.21.15.1:0/563689322 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7f09d40efdf0 con 0x7f09dc107900 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.611+0000 7f09e2186640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09c80a32c0 0x7f09c80a5780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009bd0 tx=0x7f09d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:53.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.614+0000 7f09cf7fe640 1 -- 172.21.15.1:0/563689322 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09d40bda50 con 0x7f09dc107900 2024-09-16T00:22:53.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:22:53.798+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f09dc062630 con 0x7f09c80a32c0 2024-09-16T00:22:54.087 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:22:53.757+0000 7f0c285ee640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T00:22:54.087 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:22:53 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:22:53.763+0000 7f0c1fbfa640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:22:54.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi153 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: from='osd.5 ' entity='osd.5' 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:22:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi153 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: from='osd.5 ' entity='osd.5' 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:22:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi153 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: from='osd.5 ' entity='osd.5' 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:22:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:55.250 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:55 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:22:55.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:55 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:22:55.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:55 smithi001 ceph-mon[30934]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:22:55.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:55 smithi001 ceph-mon[30934]: osd.5 [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] boot 2024-09-16T00:22:55.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:55 smithi001 ceph-mon[30934]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:22:55.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:55 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:55.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:55 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:22:55.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:55 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:22:55.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:55 smithi019 ceph-mon[35186]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:22:55.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:55 smithi019 ceph-mon[35186]: osd.5 [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] boot 2024-09-16T00:22:55.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:55 smithi019 ceph-mon[35186]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:22:55.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:55 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:55.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:55 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:22:55.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:55 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:22:55.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:55 smithi153 ceph-mon[35538]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:22:55.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:55 smithi153 ceph-mon[35538]: osd.5 [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] boot 2024-09-16T00:22:55.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:55 smithi153 ceph-mon[35538]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:22:55.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:55 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:56.203 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:56 smithi001 ceph-mon[30934]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:56.204 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:56 smithi001 ceph-mon[30934]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:22:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:56 smithi019 ceph-mon[35186]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:56 smithi019 ceph-mon[35186]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:22:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:56 smithi153 ceph-mon[35538]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:22:56.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:56 smithi153 ceph-mon[35538]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:22:57.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:57 smithi019 ceph-mon[35186]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:22:57.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:57 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2709498191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f"}]: dispatch 2024-09-16T00:22:57.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:57 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2709498191' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f"}]': finished 2024-09-16T00:22:57.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:57 smithi019 ceph-mon[35186]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:22:57.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:57 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:22:57.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:57 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/335083293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:22:57.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:57 smithi001 ceph-mon[30934]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:22:57.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:57 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2709498191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f"}]: dispatch 2024-09-16T00:22:57.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:57 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2709498191' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f"}]': finished 2024-09-16T00:22:57.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:57 smithi001 ceph-mon[30934]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:22:57.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:57 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:22:57.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:57 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/335083293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:22:57.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:57 smithi153 ceph-mon[35538]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:22:57.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:57 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2709498191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f"}]: dispatch 2024-09-16T00:22:57.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:57 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2709498191' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f"}]': finished 2024-09-16T00:22:57.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:57 smithi153 ceph-mon[35538]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:22:57.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:57 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:22:57.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:57 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/335083293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:22:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:58 smithi019 ceph-mon[35186]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:22:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:58 smithi019 ceph-mon[35186]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:22:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:58 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:22:58.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:58 smithi001 ceph-mon[30934]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:22:58.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:58 smithi001 ceph-mon[30934]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:22:58.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:58 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:22:58.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:58 smithi153 ceph-mon[35538]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:22:58.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:58 smithi153 ceph-mon[35538]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:22:58.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:58 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:22:59.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:59 smithi019 ceph-mon[35186]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:22:59.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:22:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:22:59.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:59 smithi001 ceph-mon[30934]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:22:59.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:22:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:22:59.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:59 smithi153 ceph-mon[35538]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:22:59.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:22:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:00.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:00 smithi019 ceph-mon[35186]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:00.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:00 smithi001 ceph-mon[30934]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:00.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:00 smithi153 ceph-mon[35538]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:02.546 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:02 smithi001 ceph-mon[30934]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-16T00:23:02.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:02 smithi019 ceph-mon[35186]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-16T00:23:02.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:02 smithi153 ceph-mon[35538]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-16T00:23:04.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:04 smithi153 ceph-mon[35538]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T00:23:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:04 smithi019 ceph-mon[35186]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T00:23:04.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:04 smithi001 ceph-mon[30934]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T00:23:05.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:23:05.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:05.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:23:05.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:05.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:23:05.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:06.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:06 smithi001 ceph-mon[30934]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:23:06.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:06 smithi001 ceph-mon[30934]: Deploying daemon osd.6 on smithi001 2024-09-16T00:23:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:06 smithi019 ceph-mon[35186]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:23:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:06 smithi019 ceph-mon[35186]: Deploying daemon osd.6 on smithi001 2024-09-16T00:23:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:06 smithi153 ceph-mon[35538]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:23:06.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:06 smithi153 ceph-mon[35538]: Deploying daemon osd.6 on smithi001 2024-09-16T00:23:07.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:07 smithi001 ceph-mon[30934]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T00:23:07.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:07 smithi019 ceph-mon[35186]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T00:23:07.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:07 smithi153 ceph-mon[35538]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T00:23:09.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:09 smithi001 ceph-mon[30934]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-16T00:23:09.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:09 smithi019 ceph-mon[35186]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-16T00:23:09.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:09 smithi153 ceph-mon[35538]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-16T00:23:11.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:11 smithi001 ceph-mon[30934]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:23:11.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:11 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:11.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:11 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:11.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:11 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:11.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:11 smithi019 ceph-mon[35186]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:23:11.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:11 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:11.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:11 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:11.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:11 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:11.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:11 smithi153 ceph-mon[35538]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:23:11.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:11 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:11.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:11 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:11.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:11 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:13.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:13 smithi001 ceph-mon[30934]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:13.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:13 smithi019 ceph-mon[35186]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:13.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:13 smithi153 ceph-mon[35538]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:14.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:14 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:14.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:14 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:14.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:14 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:14.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:14 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:14.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:14 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:14.762 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 6 on host 'smithi001' 2024-09-16T00:23:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.762+0000 7f09cf7fe640 1 -- 172.21.15.1:0/563689322 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f09dc062630 con 0x7f09c80a32c0 2024-09-16T00:23:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09c80a32c0 msgr2=0x7f09c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09c80a32c0 0x7f09c80a5780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009bd0 tx=0x7f09d0005f90 comp rx=0 tx=0).stop 2024-09-16T00:23:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 msgr2=0x7f09dc100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 0x7f09dc100b90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f09d4031ae0 tx=0x7f09d4031ee0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 shutdown_connections 2024-09-16T00:23:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc107900 0x7f09dc100b90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09dc104fd0 0x7f09dc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09c80a32c0 0x7f09c80a5780 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 --2- 172.21.15.1:0/563689322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09dc1026a0 0x7f09dc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 >> 172.21.15.1:0/563689322 conn(0x7f09dc0fc2a0 msgr2=0x7f09dc102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 shutdown_connections 2024-09-16T00:23:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:23:14.765+0000 7f09e43ea640 1 -- 172.21.15.1:0/563689322 wait complete. 2024-09-16T00:23:14.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:14 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:14.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:14 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:14.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:14 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:14.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:14 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:14.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:14 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:15.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:15.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:15.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:15.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:15.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:14 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:15.467 DEBUG:teuthology.orchestra.run.smithi001:osd.6> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T00:23:15.470 INFO:tasks.cephadm:Deploying osd.7 on smithi019 with /dev/vg_nvme/lv_2... 2024-09-16T00:23:15.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:23:15.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:15 smithi001 ceph-mon[30934]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:15.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:15.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:15.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:15.732 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:23:15.853 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:15 smithi019 ceph-mon[35186]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:15.853 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:15.853 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:15.853 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:15.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:15 smithi153 ceph-mon[35538]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:15.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:15.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:15.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:17.274 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:23:17 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:23:17.137+0000 7fd768f43640 -1 osd.6 0 log_to_monitors true 2024-09-16T00:23:17.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:17 smithi019 ceph-mon[35186]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:17.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:17 smithi019 ceph-mon[35186]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:23:17.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:17 smithi001 ceph-mon[30934]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:17.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:17 smithi001 ceph-mon[30934]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:23:17.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:17 smithi153 ceph-mon[35538]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:17.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:17 smithi153 ceph-mon[35538]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:23:18.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:23:18.674 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-09-16T00:23:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:23:18.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:23:18.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T00:23:18.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:23:18.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:18.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:18.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:18.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:18.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:18.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:18.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.956 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:23:18.956 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T00:23:18.956 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:23:18.956 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:18.956 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.956 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.956 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:18.956 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:18.957 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:18.957 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.957 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:18.957 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:18.957 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:18.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.371+0000 7efc71b9b640 1 -- 172.21.15.19:0/3532846814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c102690 msgr2=0x7efc6c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.371+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3532846814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c102690 0x7efc6c108f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efc64009ca0 tx=0x7efc6402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.371+0000 7efc71b9b640 1 -- 172.21.15.19:0/3532846814 shutdown_connections 2024-09-16T00:23:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.371+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3532846814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c102690 0x7efc6c108f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.371+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3532846814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc6c101cd0 0x7efc6c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.371+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3532846814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc6c100ad0 0x7efc6c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.371+0000 7efc71b9b640 1 -- 172.21.15.19:0/3532846814 >> 172.21.15.19:0/3532846814 conn(0x7efc6c0fc280 msgr2=0x7efc6c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.371+0000 7efc71b9b640 1 -- 172.21.15.19:0/3532846814 shutdown_connections 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.371+0000 7efc71b9b640 1 -- 172.21.15.19:0/3532846814 wait complete. 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.372+0000 7efc71b9b640 1 Processor -- start 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.372+0000 7efc71b9b640 1 -- start start 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.372+0000 7efc71b9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c100ad0 0x7efc6c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.372+0000 7efc71b9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc6c101cd0 0x7efc6c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.372+0000 7efc71b9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc6c102690 0x7efc6c19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.372+0000 7efc71b9b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c10c750 con 0x7efc6c102690 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.372+0000 7efc71b9b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c10c5d0 con 0x7efc6c100ad0 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.372+0000 7efc71b9b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c10c8d0 con 0x7efc6c101cd0 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.372+0000 7efc6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c100ad0 0x7efc6c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c100ad0 0x7efc6c06aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34002/0 (socket says 172.21.15.19:34002) 2024-09-16T00:23:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6b7fe640 1 -- 172.21.15.19:0/3392362981 learned_addr learned my addr 172.21.15.19:0/3392362981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6bfff640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc6c102690 0x7efc6c19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6affd640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc6c101cd0 0x7efc6c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6b7fe640 1 -- 172.21.15.19:0/3392362981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc6c101cd0 msgr2=0x7efc6c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6b7fe640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc6c101cd0 0x7efc6c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6b7fe640 1 -- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc6c102690 msgr2=0x7efc6c19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6b7fe640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc6c102690 0x7efc6c19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6b7fe640 1 -- 172.21.15.19:0/3392362981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c19fb30 con 0x7efc6c100ad0 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc6b7fe640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c100ad0 0x7efc6c06aad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00d950 tx=0x7efc5c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc68ff9640 1 -- 172.21.15.19:0/3392362981 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c017070 con 0x7efc6c100ad0 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c19fdc0 con 0x7efc6c100ad0 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc68ff9640 1 -- 172.21.15.19:0/3392362981 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5c009c30 con 0x7efc6c100ad0 2024-09-16T00:23:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc68ff9640 1 -- 172.21.15.19:0/3392362981 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c00f600 con 0x7efc6c100ad0 2024-09-16T00:23:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.373+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c1a0270 con 0x7efc6c100ad0 2024-09-16T00:23:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.374+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6c107ea0 con 0x7efc6c100ad0 2024-09-16T00:23:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.376+0000 7efc68ff9640 1 -- 172.21.15.19:0/3392362981 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7efc5c01a070 con 0x7efc6c100ad0 2024-09-16T00:23:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.376+0000 7efc68ff9640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc540a3240 0x7efc540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.376+0000 7efc68ff9640 1 -- 172.21.15.19:0/3392362981 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4685+0+0 (secure 0 0 0) 0x7efc5c0ce540 con 0x7efc6c100ad0 2024-09-16T00:23:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.376+0000 7efc6affd640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc540a3240 0x7efc540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.377+0000 7efc6affd640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc540a3240 0x7efc540a5700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc60005fd0 tx=0x7efc60005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.384+0000 7efc68ff9640 1 -- 172.21.15.19:0/3392362981 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc5c009de0 con 0x7efc6c100ad0 2024-09-16T00:23:19.451 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:23:19.173+0000 7fd7656da640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T00:23:19.451 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:23:19.182+0000 7fd75cce6640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:23:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:19.569+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efc6c062630 con 0x7efc540a3240 2024-09-16T00:23:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi001 2024-09-16T00:23:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi001 to 5697M 2024-09-16T00:23:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:23:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T00:23:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' 2024-09-16T00:23:19.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:23:19.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:23:19.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:19.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi001 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi001 to 5697M 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:23:19.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:19.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:19.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi001 2024-09-16T00:23:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi001 to 5697M 2024-09-16T00:23:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:23:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T00:23:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: from='osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121]' entity='osd.6' 2024-09-16T00:23:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:23:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:23:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:20.943 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:20 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:23:20.943 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:20 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:23:20.943 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:20 smithi019 ceph-mon[35186]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:20.943 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:20.943 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:20 smithi019 ceph-mon[35186]: osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] boot 2024-09-16T00:23:20.943 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:20 smithi019 ceph-mon[35186]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T00:23:20.943 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:20.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:20 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:23:20.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:20 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:23:20.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:20 smithi001 ceph-mon[30934]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:20.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:20.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:20 smithi001 ceph-mon[30934]: osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] boot 2024-09-16T00:23:20.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:20 smithi001 ceph-mon[30934]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T00:23:20.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:20.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:20 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:23:20.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:20 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:23:20.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:20 smithi153 ceph-mon[35538]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:20.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:20.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:20 smithi153 ceph-mon[35538]: osd.6 [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] boot 2024-09-16T00:23:20.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:20 smithi153 ceph-mon[35538]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T00:23:20.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:23:22.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:22 smithi001 ceph-mon[30934]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:22.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:22 smithi001 ceph-mon[30934]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T00:23:22.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:22 smithi001 ceph-mon[30934]: from='client.? 172.21.15.19:0/1921063438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9860f5c4-575b-402c-98a2-3436d37b4cfa"}]: dispatch 2024-09-16T00:23:22.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:22 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9860f5c4-575b-402c-98a2-3436d37b4cfa"}]: dispatch 2024-09-16T00:23:22.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:22 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9860f5c4-575b-402c-98a2-3436d37b4cfa"}]': finished 2024-09-16T00:23:22.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:22 smithi001 ceph-mon[30934]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:23:22.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:22 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:22 smithi153 ceph-mon[35538]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:22 smithi153 ceph-mon[35538]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T00:23:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:22 smithi153 ceph-mon[35538]: from='client.? 172.21.15.19:0/1921063438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9860f5c4-575b-402c-98a2-3436d37b4cfa"}]: dispatch 2024-09-16T00:23:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:22 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9860f5c4-575b-402c-98a2-3436d37b4cfa"}]: dispatch 2024-09-16T00:23:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:22 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9860f5c4-575b-402c-98a2-3436d37b4cfa"}]': finished 2024-09-16T00:23:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:22 smithi153 ceph-mon[35538]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:23:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:22 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:22.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:22 smithi019 ceph-mon[35186]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:23:22.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:22 smithi019 ceph-mon[35186]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T00:23:22.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:22 smithi019 ceph-mon[35186]: from='client.? 172.21.15.19:0/1921063438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9860f5c4-575b-402c-98a2-3436d37b4cfa"}]: dispatch 2024-09-16T00:23:22.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:22 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9860f5c4-575b-402c-98a2-3436d37b4cfa"}]: dispatch 2024-09-16T00:23:22.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:22 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9860f5c4-575b-402c-98a2-3436d37b4cfa"}]': finished 2024-09-16T00:23:22.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:22 smithi019 ceph-mon[35186]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:23:22.501 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:22 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:23.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:23 smithi001 ceph-mon[30934]: from='client.? 172.21.15.19:0/2148538666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:23:23.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:23 smithi153 ceph-mon[35538]: from='client.? 172.21.15.19:0/2148538666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:23:23.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:23 smithi019 ceph-mon[35186]: from='client.? 172.21.15.19:0/2148538666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:23:24.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:24 smithi001 ceph-mon[30934]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:24.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:24 smithi153 ceph-mon[35538]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:24 smithi019 ceph-mon[35186]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:26.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:26 smithi001 ceph-mon[30934]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:26 smithi153 ceph-mon[35538]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:26.477 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:26 smithi019 ceph-mon[35186]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:28.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:28 smithi001 ceph-mon[30934]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:28.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:28 smithi153 ceph-mon[35538]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:28.557 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:28 smithi019 ceph-mon[35186]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:30 smithi019 ceph-mon[35186]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:23:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:30 smithi001 ceph-mon[30934]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:23:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:30.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:30 smithi153 ceph-mon[35538]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:30.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:23:30.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:31.429 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:31 smithi019 ceph-mon[35186]: Deploying daemon osd.7 on smithi019 2024-09-16T00:23:31.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:31 smithi153 ceph-mon[35538]: Deploying daemon osd.7 on smithi019 2024-09-16T00:23:31.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:31 smithi001 ceph-mon[30934]: Deploying daemon osd.7 on smithi019 2024-09-16T00:23:32.326 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:32 smithi019 ceph-mon[35186]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:32 smithi153 ceph-mon[35538]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:32.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:32 smithi001 ceph-mon[30934]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:34 smithi153 ceph-mon[35538]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:34.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:34 smithi019 ceph-mon[35186]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:34.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:34 smithi001 ceph-mon[30934]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:36 smithi019 ceph-mon[35186]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:36 smithi153 ceph-mon[35538]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:36 smithi001 ceph-mon[30934]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:38.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:38 smithi019 ceph-mon[35186]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:38.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:38 smithi001 ceph-mon[30934]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:38.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:38 smithi153 ceph-mon[35538]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:39.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.773+0000 7efc68ff9640 1 -- 172.21.15.19:0/3392362981 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efc6c062630 con 0x7efc540a3240 2024-09-16T00:23:39.777 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 7 on host 'smithi019' 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc540a3240 msgr2=0x7efc540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc540a3240 0x7efc540a5700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc60005fd0 tx=0x7efc60005850 comp rx=0 tx=0).stop 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c100ad0 msgr2=0x7efc6c06aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c100ad0 0x7efc6c06aad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00d950 tx=0x7efc5c00de10 comp rx=0 tx=0).stop 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 shutdown_connections 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc6c102690 0x7efc6c19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc540a3240 0x7efc540a5700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc6c101cd0 0x7efc6c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 --2- 172.21.15.19:0/3392362981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c100ad0 0x7efc6c06aad0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 >> 172.21.15.19:0/3392362981 conn(0x7efc6c0fc280 msgr2=0x7efc6c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 shutdown_connections 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:23:39.776+0000 7efc71b9b640 1 -- 172.21.15.19:0/3392362981 wait complete. 2024-09-16T00:23:40.028 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:39 smithi019 ceph-mon[35186]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:40.029 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.029 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.029 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:40.029 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:40.029 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.029 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:40.029 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.029 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:39 smithi001 ceph-mon[30934]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:40.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:40.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:40.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:40.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:39 smithi153 ceph-mon[35538]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:40.340 DEBUG:teuthology.orchestra.run.smithi019:osd.7> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T00:23:40.343 INFO:tasks.cephadm:Deploying osd.8 on smithi153 with /dev/vg_nvme/lv_2... 2024-09-16T00:23:40.343 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:23:40.599 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:23:42.019 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:41 smithi153 ceph-mon[35538]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:42.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:41 smithi019 ceph-mon[35186]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:42.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:41 smithi001 ceph-mon[30934]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:42.843 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:23:42 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:23:42.380+0000 7f925339b640 -1 osd.7 0 log_to_monitors true 2024-09-16T00:23:43.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:42 smithi019 ceph-mon[35186]: from='osd.7 [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:23:43.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:42 smithi019 ceph-mon[35186]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:23:43.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:42 smithi001 ceph-mon[30934]: from='osd.7 [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:23:43.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:42 smithi001 ceph-mon[30934]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:23:43.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:42 smithi153 ceph-mon[35538]: from='osd.7 [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:23:43.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:42 smithi153 ceph-mon[35538]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:23:43.240 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:23:43.264 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_2 2024-09-16T00:23:43.526 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:23:43.899 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:43.899 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:23:43.899 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T00:23:43.899 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='osd.7 [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi019 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi019 to 5697M 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:43.900 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:43.901 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:44.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.983+0000 7fcc62d63640 1 -- 172.21.15.153:0/569815472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c101c00 msgr2=0x7fcc5c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.983+0000 7fcc62d63640 1 --2- 172.21.15.153:0/569815472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c101c00 0x7fcc5c102000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009900 tx=0x7fcc5002a410 comp rx=0 tx=0).stop 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.983+0000 7fcc62d63640 1 -- 172.21.15.153:0/569815472 shutdown_connections 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.983+0000 7fcc62d63640 1 --2- 172.21.15.153:0/569815472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc5c1037c0 0x7fcc5c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.983+0000 7fcc62d63640 1 --2- 172.21.15.153:0/569815472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc5c102e00 0x7fcc5c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.983+0000 7fcc62d63640 1 --2- 172.21.15.153:0/569815472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c101c00 0x7fcc5c102000 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.983+0000 7fcc62d63640 1 -- 172.21.15.153:0/569815472 >> 172.21.15.153:0/569815472 conn(0x7fcc5c0fd390 msgr2=0x7fcc5c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.983+0000 7fcc62d63640 1 -- 172.21.15.153:0/569815472 shutdown_connections 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.983+0000 7fcc62d63640 1 -- 172.21.15.153:0/569815472 wait complete. 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc62d63640 1 Processor -- start 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc62d63640 1 -- start start 2024-09-16T00:23:44.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc62d63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc5c101c00 0x7fcc5c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc62d63640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c102e00 0x7fcc5c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc62d63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc5c1037c0 0x7fcc5c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc62d63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c10d860 con 0x7fcc5c1037c0 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc62d63640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c10d6e0 con 0x7fcc5c101c00 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc62d63640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c10d9e0 con 0x7fcc5c102e00 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc5bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c102e00 0x7fcc5c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc5bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c102e00 0x7fcc5c0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60352/0 (socket says 172.21.15.153:60352) 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc5bfff640 1 -- 172.21.15.153:0/1720272478 learned_addr learned my addr 172.21.15.153:0/1720272478 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.984+0000 7fcc60aff640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc5c101c00 0x7fcc5c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:44.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc61300640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc5c1037c0 0x7fcc5c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc5bfff640 1 -- 172.21.15.153:0/1720272478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc5c101c00 msgr2=0x7fcc5c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc5bfff640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc5c101c00 0x7fcc5c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc5bfff640 1 -- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc5c1037c0 msgr2=0x7fcc5c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc5bfff640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc5c1037c0 0x7fcc5c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc5bfff640 1 -- 172.21.15.153:0/1720272478 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c0753e0 con 0x7fcc5c102e00 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc61300640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc5c1037c0 0x7fcc5c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc5bfff640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c102e00 0x7fcc5c0745e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800d950 tx=0x7fcc4800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc59ffb640 1 -- 172.21.15.153:0/1720272478 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48017070 con 0x7fcc5c102e00 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c1a1fe0 con 0x7fcc5c102e00 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc59ffb640 1 -- 172.21.15.153:0/1720272478 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc48009c30 con 0x7fcc5c102e00 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc59ffb640 1 -- 172.21.15.153:0/1720272478 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4800f6b0 con 0x7fcc5c102e00 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.985+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c1a24e0 con 0x7fcc5c102e00 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.986+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5c109020 con 0x7fcc5c102e00 2024-09-16T00:23:44.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.987+0000 7fcc59ffb640 1 -- 172.21.15.153:0/1720272478 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fcc480109d0 con 0x7fcc5c102e00 2024-09-16T00:23:44.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.988+0000 7fcc59ffb640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc4c0a3240 0x7fcc4c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:44.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.988+0000 7fcc59ffb640 1 -- 172.21.15.153:0/1720272478 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5117+0+0 (secure 0 0 0) 0x7fcc480ce8d0 con 0x7fcc5c102e00 2024-09-16T00:23:44.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.988+0000 7fcc60aff640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc4c0a3240 0x7fcc4c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:44.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.989+0000 7fcc60aff640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc4c0a3240 0x7fcc4c0a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005ab0 tx=0x7fcc50000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:44.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:43.992+0000 7fcc59ffb640 1 -- 172.21.15.153:0/1720272478 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc48009de0 con 0x7fcc5c102e00 2024-09-16T00:23:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:23:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T00:23:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='osd.7 [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi019 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi019 to 5697M 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:44.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:44.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:23:44.168+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fcc5c062630 con 0x7fcc4c0a3240 2024-09-16T00:23:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:44.222 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='osd.7 [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi019 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi019 to 5697M 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:44.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:44.224 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:23:44.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:23:44.567+0000 7f924f31e640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T00:23:44.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:23:44.577+0000 7f924713e640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:23:45.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:44 smithi001 ceph-mon[30934]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:23:45.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:44 smithi001 ceph-mon[30934]: osdmap e49: 8 total, 7 up, 8 in 2024-09-16T00:23:45.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:45.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:45.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:44 smithi001 ceph-mon[30934]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:45.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:23:45.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:23:45.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:44 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:45.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:44 smithi001 ceph-mon[30934]: from='osd.7 ' entity='osd.7' 2024-09-16T00:23:45.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:44 smithi153 ceph-mon[35538]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:23:45.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:44 smithi153 ceph-mon[35538]: osdmap e49: 8 total, 7 up, 8 in 2024-09-16T00:23:45.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:45.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:45.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:44 smithi153 ceph-mon[35538]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:45.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:23:45.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:23:45.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:44 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:45.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:44 smithi153 ceph-mon[35538]: from='osd.7 ' entity='osd.7' 2024-09-16T00:23:45.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-mon[35186]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:23:45.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-mon[35186]: osdmap e49: 8 total, 7 up, 8 in 2024-09-16T00:23:45.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:45.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:45.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-mon[35186]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:45.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:23:45.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:23:45.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:45.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:44 smithi019 ceph-mon[35186]: from='osd.7 ' entity='osd.7' 2024-09-16T00:23:46.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:45 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:23:46.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:45 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:23:46.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:45 smithi001 ceph-mon[30934]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:46.118 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:45 smithi001 ceph-mon[30934]: osd.7 [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] boot 2024-09-16T00:23:46.118 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:45 smithi001 ceph-mon[30934]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:23:46.118 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:45 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:46.118 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:45 smithi001 ceph-mon[30934]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T00:23:46.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:45 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:23:46.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:45 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:23:46.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:45 smithi153 ceph-mon[35538]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:46.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:45 smithi153 ceph-mon[35538]: osd.7 [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] boot 2024-09-16T00:23:46.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:45 smithi153 ceph-mon[35538]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:23:46.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:45 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:46.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:45 smithi153 ceph-mon[35538]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T00:23:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:45 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:23:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:45 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:23:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:45 smithi019 ceph-mon[35186]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:23:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:45 smithi019 ceph-mon[35186]: osd.7 [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] boot 2024-09-16T00:23:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:45 smithi019 ceph-mon[35186]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:23:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:45 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:23:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:45 smithi019 ceph-mon[35186]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T00:23:47.094 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:46 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/3679641277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2371d5a7-e6c3-415f-b6d3-35df1a4c3a27"}]: dispatch 2024-09-16T00:23:47.094 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:46 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2371d5a7-e6c3-415f-b6d3-35df1a4c3a27"}]: dispatch 2024-09-16T00:23:47.094 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:46 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2371d5a7-e6c3-415f-b6d3-35df1a4c3a27"}]': finished 2024-09-16T00:23:47.094 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:46 smithi153 ceph-mon[35538]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T00:23:47.095 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:23:47.178 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:46 smithi019 ceph-mon[35186]: from='client.? 172.21.15.153:0/3679641277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2371d5a7-e6c3-415f-b6d3-35df1a4c3a27"}]: dispatch 2024-09-16T00:23:47.178 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:46 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2371d5a7-e6c3-415f-b6d3-35df1a4c3a27"}]: dispatch 2024-09-16T00:23:47.178 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:46 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2371d5a7-e6c3-415f-b6d3-35df1a4c3a27"}]': finished 2024-09-16T00:23:47.178 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:46 smithi019 ceph-mon[35186]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T00:23:47.178 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:23:47.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:46 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/3679641277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2371d5a7-e6c3-415f-b6d3-35df1a4c3a27"}]: dispatch 2024-09-16T00:23:47.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:46 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2371d5a7-e6c3-415f-b6d3-35df1a4c3a27"}]: dispatch 2024-09-16T00:23:47.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:46 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2371d5a7-e6c3-415f-b6d3-35df1a4c3a27"}]': finished 2024-09-16T00:23:47.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:46 smithi001 ceph-mon[30934]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T00:23:47.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:23:48.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:47 smithi001 ceph-mon[30934]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:48.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:47 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/2010609269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:23:48.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:47 smithi001 ceph-mon[30934]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:23:48.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:47 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:23:48.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:47 smithi153 ceph-mon[35538]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:48.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:47 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/2010609269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:23:48.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:47 smithi153 ceph-mon[35538]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:23:48.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:47 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:23:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:47 smithi019 ceph-mon[35186]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:47 smithi019 ceph-mon[35186]: from='client.? 172.21.15.153:0/2010609269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:23:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:47 smithi019 ceph-mon[35186]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:23:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:47 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:23:49.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:49 smithi019 ceph-mon[35186]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:49.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:49 smithi019 ceph-mon[35186]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:23:49.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:23:49.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:49 smithi001 ceph-mon[30934]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:49.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:49 smithi001 ceph-mon[30934]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:23:49.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:23:49.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:49 smithi153 ceph-mon[35538]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:49.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:49 smithi153 ceph-mon[35538]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:23:49.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:23:51.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:51 smithi153 ceph-mon[35538]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:51 smithi019 ceph-mon[35186]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:51.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:51 smithi001 ceph-mon[30934]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:53.626 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:53 smithi153 ceph-mon[35538]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:53.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:53 smithi019 ceph-mon[35186]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:53.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:53 smithi001 ceph-mon[30934]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:55.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:55 smithi153 ceph-mon[35538]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:55.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:55 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:23:55.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:55 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:55.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:55 smithi153 ceph-mon[35538]: Deploying daemon osd.8 on smithi153 2024-09-16T00:23:55.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:55 smithi019 ceph-mon[35186]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:55.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:55 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:23:55.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:55 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:55.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:55 smithi019 ceph-mon[35186]: Deploying daemon osd.8 on smithi153 2024-09-16T00:23:55.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:55 smithi001 ceph-mon[30934]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:55.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:55 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:23:55.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:55 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:55.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:55 smithi001 ceph-mon[30934]: Deploying daemon osd.8 on smithi153 2024-09-16T00:23:57.689 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:57 smithi153 ceph-mon[35538]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:57.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:57 smithi019 ceph-mon[35186]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:57.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:57 smithi001 ceph-mon[30934]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:59.568 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:23:59 smithi153 ceph-mon[35538]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:59.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:23:59 smithi019 ceph-mon[35186]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:23:59.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:23:59 smithi001 ceph-mon[30934]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:00.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:00 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:00.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:00 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:00.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:00 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:00.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:00 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:00.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:00 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:00.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:00 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:00.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:00 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:00.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:00 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:00.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:00 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:01 smithi019 ceph-mon[35186]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:01.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:01 smithi001 ceph-mon[30934]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:01.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:01 smithi153 ceph-mon[35538]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:03.759 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:03 smithi153 ceph-mon[35538]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:03.759 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:03.759 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:03.759 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:03.759 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:03.759 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:03.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:03 smithi019 ceph-mon[35186]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:03.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:03.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:03.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:03.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:03.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:03.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:03 smithi001 ceph-mon[30934]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:03.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:03.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:03.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:03.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:03.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:04.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.446+0000 7fcc59ffb640 1 -- 172.21.15.153:0/1720272478 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcc5c062630 con 0x7fcc4c0a3240 2024-09-16T00:24:04.447 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 8 on host 'smithi153' 2024-09-16T00:24:04.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc4c0a3240 msgr2=0x7fcc4c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:04.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc4c0a3240 0x7fcc4c0a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005ab0 tx=0x7fcc50000bc0 comp rx=0 tx=0).stop 2024-09-16T00:24:04.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c102e00 msgr2=0x7fcc5c0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:04.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c102e00 0x7fcc5c0745e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800d950 tx=0x7fcc4800de10 comp rx=0 tx=0).stop 2024-09-16T00:24:04.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 shutdown_connections 2024-09-16T00:24:04.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc5c1037c0 0x7fcc5c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:04.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc4c0a3240 0x7fcc4c0a5700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:04.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc5c102e00 0x7fcc5c0745e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 --2- 172.21.15.153:0/1720272478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc5c101c00 0x7fcc5c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 >> 172.21.15.153:0/1720272478 conn(0x7fcc5c0fd390 msgr2=0x7fcc5c1061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 shutdown_connections 2024-09-16T00:24:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:24:04.448+0000 7fcc62d63640 1 -- 172.21.15.153:0/1720272478 wait complete. 2024-09-16T00:24:04.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:04.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:04.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:04.758 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:04 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:04.758 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:04 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:04.758 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:04 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:04.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:04.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:04.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:05.053 DEBUG:teuthology.orchestra.run.smithi153:osd.8> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.8.service 2024-09-16T00:24:05.056 INFO:tasks.cephadm:Deploying osd.9 on smithi001 with /dev/vg_nvme/lv_1... 2024-09-16T00:24:05.056 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:24:05.325 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:05.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:05 smithi153 ceph-mon[35538]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:05.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:05 smithi001 ceph-mon[30934]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:05.775 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:05 smithi019 ceph-mon[35186]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:06.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:24:06 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:24:06.750+0000 7fe3cb102640 -1 osd.8 0 log_to_monitors true 2024-09-16T00:24:07.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:07 smithi153 ceph-mon[35538]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:07.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:07 smithi153 ceph-mon[35538]: from='osd.8 [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:24:07.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:07 smithi153 ceph-mon[35538]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:24:07.730 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:07 smithi001 ceph-mon[30934]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:07.731 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:07 smithi001 ceph-mon[30934]: from='osd.8 [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:24:07.731 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:07 smithi001 ceph-mon[30934]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:24:07.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:07 smithi019 ceph-mon[35186]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:07.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:07 smithi019 ceph-mon[35186]: from='osd.8 [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:24:07.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:07 smithi019 ceph-mon[35186]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:24:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:24:08.280 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_1 2024-09-16T00:24:08.541 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:08.833 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:24:08.833 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: osdmap e55: 9 total, 8 up, 9 in 2024-09-16T00:24:08.833 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:08.833 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='osd.8 [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:24:08.833 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:24:08.833 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi153 2024-09-16T00:24:08.833 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:08.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:08.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:08.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-16T00:24:08.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:08.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:08.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:08 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:24:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: osdmap e55: 9 total, 8 up, 9 in 2024-09-16T00:24:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='osd.8 [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:24:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:24:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi153 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:08.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:08 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: osdmap e55: 9 total, 8 up, 9 in 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='osd.8 [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi153 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:08.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:08.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-16T00:24:08.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:08.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:08.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:08.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:08 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:09.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.169+0000 7fb912b6c640 1 -- 172.21.15.1:0/1940100006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 msgr2=0x7fb90c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:09.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.169+0000 7fb912b6c640 1 --2- 172.21.15.1:0/1940100006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 0x7fb90c10a050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb904009ca0 tx=0x7fb90402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.169+0000 7fb912b6c640 1 -- 172.21.15.1:0/1940100006 shutdown_connections 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.169+0000 7fb912b6c640 1 --2- 172.21.15.1:0/1940100006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 0x7fb90c10a050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.169+0000 7fb912b6c640 1 --2- 172.21.15.1:0/1940100006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c102e00 0x7fb90c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.169+0000 7fb912b6c640 1 --2- 172.21.15.1:0/1940100006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c101c00 0x7fb90c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.169+0000 7fb912b6c640 1 -- 172.21.15.1:0/1940100006 >> 172.21.15.1:0/1940100006 conn(0x7fb90c0fd390 msgr2=0x7fb90c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.169+0000 7fb912b6c640 1 -- 172.21.15.1:0/1940100006 shutdown_connections 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.169+0000 7fb912b6c640 1 -- 172.21.15.1:0/1940100006 wait complete. 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb912b6c640 1 Processor -- start 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb912b6c640 1 -- start start 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb912b6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c101c00 0x7fb90c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb912b6c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c102e00 0x7fb90c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb912b6c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 0x7fb90c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb912b6c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c10d880 con 0x7fb90c1037c0 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb912b6c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c10d700 con 0x7fb90c101c00 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb912b6c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c10da00 con 0x7fb90c102e00 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb911109640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 0x7fb90c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb911109640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 0x7fb90c19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39194/0 (socket says 172.21.15.1:39194) 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb911109640 1 -- 172.21.15.1:0/3415753056 learned_addr learned my addr 172.21.15.1:0/3415753056 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.170+0000 7fb90bfff640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c102e00 0x7fb90c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb910908640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c101c00 0x7fb90c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb911109640 1 -- 172.21.15.1:0/3415753056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c102e00 msgr2=0x7fb90c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb911109640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c102e00 0x7fb90c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb911109640 1 -- 172.21.15.1:0/3415753056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c101c00 msgr2=0x7fb90c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb911109640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c101c00 0x7fb90c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb911109640 1 -- 172.21.15.1:0/3415753056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c19fa20 con 0x7fb90c1037c0 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb910908640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c101c00 0x7fb90c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb911109640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 0x7fb90c19f3b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb904005e30 tx=0x7fb904031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb909ffb640 1 -- 172.21.15.1:0/3415753056 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904003b70 con 0x7fb90c1037c0 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90c19fcb0 con 0x7fb90c1037c0 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb909ffb640 1 -- 172.21.15.1:0/3415753056 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb904003d10 con 0x7fb90c1037c0 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb909ffb640 1 -- 172.21.15.1:0/3415753056 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90402fd30 con 0x7fb90c1037c0 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.171+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90c1a01e0 con 0x7fb90c1037c0 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.172+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90c108f30 con 0x7fb90c1037c0 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.174+0000 7fb909ffb640 1 -- 172.21.15.1:0/3415753056 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fb904004050 con 0x7fb90c1037c0 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.175+0000 7fb909ffb640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8fc0a3270 0x7fb8fc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.175+0000 7fb910908640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8fc0a3270 0x7fb8fc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.175+0000 7fb909ffb640 1 -- 172.21.15.1:0/3415753056 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5549+0+0 (secure 0 0 0) 0x7fb9040f02b0 con 0x7fb90c1037c0 2024-09-16T00:24:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.175+0000 7fb910908640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8fc0a3270 0x7fb8fc0a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb90c102c60 tx=0x7fb900005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.178+0000 7fb909ffb640 1 -- 172.21.15.1:0/3415753056 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9040bdaa0 con 0x7fb90c1037c0 2024-09-16T00:24:09.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:09.366+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb90c062630 con 0x7fb8fc0a3270 2024-09-16T00:24:09.462 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:24:09.186+0000 7fe3c7085640 -1 osd.8 0 waiting for initial osdmap 2024-09-16T00:24:09.462 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:24:09.196+0000 7fe3beea5640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:24:09.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:09 smithi001 ceph-mon[30934]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:09.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:09 smithi001 ceph-mon[30934]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:24:09.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:09 smithi001 ceph-mon[30934]: osdmap e56: 9 total, 8 up, 9 in 2024-09-16T00:24:09.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:09.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:09.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:09 smithi001 ceph-mon[30934]: from='osd.8 ' entity='osd.8' 2024-09-16T00:24:09.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:09.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:09.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:09 smithi019 ceph-mon[35186]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:09 smithi019 ceph-mon[35186]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:24:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:09 smithi019 ceph-mon[35186]: osdmap e56: 9 total, 8 up, 9 in 2024-09-16T00:24:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:09 smithi019 ceph-mon[35186]: from='osd.8 ' entity='osd.8' 2024-09-16T00:24:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:09.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:09.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:09.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-mon[35538]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:09.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-mon[35538]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:24:09.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-mon[35538]: osdmap e56: 9 total, 8 up, 9 in 2024-09-16T00:24:09.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:09.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:09.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-mon[35538]: from='osd.8 ' entity='osd.8' 2024-09-16T00:24:09.964 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:09.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:09.965 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:10.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:10 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:24:10.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:10 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:24:10.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:10 smithi019 ceph-mon[35186]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:10.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:10 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:10.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:10 smithi019 ceph-mon[35186]: osd.8 [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] boot 2024-09-16T00:24:10.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:10 smithi019 ceph-mon[35186]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:24:10.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:10 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:10.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:10 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:24:10.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:10 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:24:10.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:10 smithi001 ceph-mon[30934]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:10.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:10 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:10.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:10 smithi001 ceph-mon[30934]: osd.8 [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] boot 2024-09-16T00:24:10.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:10 smithi001 ceph-mon[30934]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:24:10.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:10 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:10.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:10 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:24:10.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:10 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:24:10.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:10 smithi153 ceph-mon[35538]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:10.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:10 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:10.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:10 smithi153 ceph-mon[35538]: osd.8 [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] boot 2024-09-16T00:24:10.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:10 smithi153 ceph-mon[35538]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:24:10.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:10 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:24:11.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:11 smithi001 ceph-mon[30934]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:11.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:11 smithi001 ceph-mon[30934]: osdmap e58: 9 total, 9 up, 9 in 2024-09-16T00:24:11.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:11 smithi019 ceph-mon[35186]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:11.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:11 smithi019 ceph-mon[35186]: osdmap e58: 9 total, 9 up, 9 in 2024-09-16T00:24:11.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:11 smithi153 ceph-mon[35538]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:24:11.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:11 smithi153 ceph-mon[35538]: osdmap e58: 9 total, 9 up, 9 in 2024-09-16T00:24:12.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:12 smithi001 ceph-mon[30934]: osdmap e59: 9 total, 9 up, 9 in 2024-09-16T00:24:12.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:12 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3375212348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92377e46-86e8-49b1-91b3-8180013968bb"}]: dispatch 2024-09-16T00:24:12.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:12 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3375212348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92377e46-86e8-49b1-91b3-8180013968bb"}]': finished 2024-09-16T00:24:12.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:12 smithi001 ceph-mon[30934]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:24:12.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:12.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:12 smithi019 ceph-mon[35186]: osdmap e59: 9 total, 9 up, 9 in 2024-09-16T00:24:12.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:12 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3375212348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92377e46-86e8-49b1-91b3-8180013968bb"}]: dispatch 2024-09-16T00:24:12.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:12 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3375212348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92377e46-86e8-49b1-91b3-8180013968bb"}]': finished 2024-09-16T00:24:12.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:12 smithi019 ceph-mon[35186]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:24:12.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:12.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:12 smithi153 ceph-mon[35538]: osdmap e59: 9 total, 9 up, 9 in 2024-09-16T00:24:12.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:12 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3375212348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92377e46-86e8-49b1-91b3-8180013968bb"}]: dispatch 2024-09-16T00:24:12.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:12 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3375212348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92377e46-86e8-49b1-91b3-8180013968bb"}]': finished 2024-09-16T00:24:12.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:12 smithi153 ceph-mon[35538]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:24:12.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:13.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:13 smithi019 ceph-mon[35186]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:13.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:13 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2809964733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:13.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:13 smithi019 ceph-mon[35186]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:24:13.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:13 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:13.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:13 smithi001 ceph-mon[30934]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:13.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:13 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2809964733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:13.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:13 smithi001 ceph-mon[30934]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:24:13.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:13 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:13.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:13 smithi153 ceph-mon[35538]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:13.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:13 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2809964733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:13.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:13 smithi153 ceph-mon[35538]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:24:13.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:13 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:15.829 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:15 smithi153 ceph-mon[35538]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:15.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:15 smithi019 ceph-mon[35186]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:15.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:15 smithi001 ceph-mon[30934]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:17.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:17 smithi001 ceph-mon[30934]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-16T00:24:17.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:17 smithi019 ceph-mon[35186]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-16T00:24:17.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:17 smithi153 ceph-mon[35538]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-16T00:24:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:19 smithi001 ceph-mon[30934]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-16T00:24:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:19 smithi019 ceph-mon[35186]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-16T00:24:19.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:19 smithi153 ceph-mon[35538]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-16T00:24:21.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:21 smithi001 ceph-mon[30934]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T00:24:21.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:24:21.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:21.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:21 smithi001 ceph-mon[30934]: Deploying daemon osd.9 on smithi001 2024-09-16T00:24:21.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:21 smithi019 ceph-mon[35186]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T00:24:21.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:24:21.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:21.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:21 smithi019 ceph-mon[35186]: Deploying daemon osd.9 on smithi001 2024-09-16T00:24:21.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:21 smithi153 ceph-mon[35538]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T00:24:21.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:24:21.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:21.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:21 smithi153 ceph-mon[35538]: Deploying daemon osd.9 on smithi001 2024-09-16T00:24:23.852 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:23 smithi001 ceph-mon[30934]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:24:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:23 smithi019 ceph-mon[35186]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:24:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:23 smithi153 ceph-mon[35538]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:24:25.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:25 smithi001 ceph-mon[30934]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:24:25.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:25 smithi019 ceph-mon[35186]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:24:25.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:25 smithi153 ceph-mon[35538]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:24:27.821 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:27 smithi001 ceph-mon[30934]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:24:27.821 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:27 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:27.821 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:27 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:27.821 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:27 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:27.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:27 smithi019 ceph-mon[35186]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:24:27.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:27 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:27.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:27 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:27.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:27 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:27.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:27 smithi153 ceph-mon[35538]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:24:27.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:27 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:27.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:27 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:27.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:27 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:29.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:29 smithi001 ceph-mon[30934]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:29.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:29 smithi019 ceph-mon[35186]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:29 smithi153 ceph-mon[35538]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.958+0000 7fb909ffb640 1 -- 172.21.15.1:0/3415753056 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb90c062630 con 0x7fb8fc0a3270 2024-09-16T00:24:30.959 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 9 on host 'smithi001' 2024-09-16T00:24:30.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8fc0a3270 msgr2=0x7fb8fc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:30.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8fc0a3270 0x7fb8fc0a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb90c102c60 tx=0x7fb900005e70 comp rx=0 tx=0).stop 2024-09-16T00:24:30.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 msgr2=0x7fb90c19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:30.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 0x7fb90c19f3b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb904005e30 tx=0x7fb904031ee0 comp rx=0 tx=0).stop 2024-09-16T00:24:30.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 shutdown_connections 2024-09-16T00:24:30.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb90c1037c0 0x7fb90c19f3b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:30.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8fc0a3270 0x7fb8fc0a5730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:30.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c102e00 0x7fb90c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:30.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 --2- 172.21.15.1:0/3415753056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c101c00 0x7fb90c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:30.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.961+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 >> 172.21.15.1:0/3415753056 conn(0x7fb90c0fd390 msgr2=0x7fb90c106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:30.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.962+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 shutdown_connections 2024-09-16T00:24:30.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:24:30.962+0000 7fb912b6c640 1 -- 172.21.15.1:0/3415753056 wait complete. 2024-09-16T00:24:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:31.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:31.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:31.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:31.762 DEBUG:teuthology.orchestra.run.smithi001:osd.9> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.9.service 2024-09-16T00:24:31.765 INFO:tasks.cephadm:Deploying osd.10 on smithi019 with /dev/vg_nvme/lv_1... 2024-09-16T00:24:31.766 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:24:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:24:32.400 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:32 smithi019 ceph-mon[35186]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:32 smithi001 ceph-mon[30934]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:32 smithi153 ceph-mon[35538]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:33.198 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:24:33 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:24:33.109+0000 7fe6f29d6640 -1 osd.9 0 log_to_monitors true 2024-09-16T00:24:33.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:33 smithi001 ceph-mon[30934]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:24:33.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:33 smithi153 ceph-mon[35538]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:24:33.531 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:33 smithi019 ceph-mon[35186]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:24:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:34 smithi019 ceph-mon[35186]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:34 smithi019 ceph-mon[35186]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:24:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:34 smithi019 ceph-mon[35186]: osdmap e62: 10 total, 9 up, 10 in 2024-09-16T00:24:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:34 smithi019 ceph-mon[35186]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:24:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:34 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:34 smithi153 ceph-mon[35538]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:34.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:34 smithi153 ceph-mon[35538]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:24:34.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:34 smithi153 ceph-mon[35538]: osdmap e62: 10 total, 9 up, 10 in 2024-09-16T00:24:34.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:34 smithi153 ceph-mon[35538]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:24:34.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:34 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:34.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:34 smithi001 ceph-mon[30934]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:34.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:34 smithi001 ceph-mon[30934]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:24:34.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:34 smithi001 ceph-mon[30934]: osdmap e62: 10 total, 9 up, 10 in 2024-09-16T00:24:34.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:34 smithi001 ceph-mon[30934]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-16T00:24:34.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:34 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:34.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:24:34 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:24:34.210+0000 7fe6ee959640 -1 osd.9 0 waiting for initial osdmap 2024-09-16T00:24:34.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:24:34 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:24:34.225+0000 7fe6ea781640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:24:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-16T00:24:34.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-09-16T00:24:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:24:35.459 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: osdmap e63: 10 total, 9 up, 10 in 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.460 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:35.461 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:35.461 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:24:35.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: osdmap e63: 10 total, 9 up, 10 in 2024-09-16T00:24:35.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:35.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.544+0000 7f7141b28640 1 -- 172.21.15.19:0/4101179519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c102e00 msgr2=0x7f713c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.544+0000 7f7141b28640 1 --2- 172.21.15.19:0/4101179519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c102e00 0x7f713c103280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7124009900 tx=0x7f712402a410 comp rx=0 tx=0).stop 2024-09-16T00:24:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.545+0000 7f7141b28640 1 -- 172.21.15.19:0/4101179519 shutdown_connections 2024-09-16T00:24:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.545+0000 7f7141b28640 1 --2- 172.21.15.19:0/4101179519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f713c1037c0 0x7f713c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.545+0000 7f7141b28640 1 --2- 172.21.15.19:0/4101179519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c102e00 0x7f713c103280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.545+0000 7f7141b28640 1 --2- 172.21.15.19:0/4101179519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c101c00 0x7f713c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.545+0000 7f7141b28640 1 -- 172.21.15.19:0/4101179519 >> 172.21.15.19:0/4101179519 conn(0x7f713c0fd390 msgr2=0x7f713c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.545+0000 7f7141b28640 1 -- 172.21.15.19:0/4101179519 shutdown_connections 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.545+0000 7f7141b28640 1 -- 172.21.15.19:0/4101179519 wait complete. 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.545+0000 7f7141b28640 1 Processor -- start 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.545+0000 7f7141b28640 1 -- start start 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f7141b28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c101c00 0x7f713c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f7141b28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c102e00 0x7f713c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f7141b28640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f713c1037c0 0x7f713c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f7141b28640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f713c10d8d0 con 0x7f713c102e00 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f7141b28640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f713c10d750 con 0x7f713c101c00 2024-09-16T00:24:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f7141b28640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f713c10da50 con 0x7f713c1037c0 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c101c00 0x7f713c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c101c00 0x7f713c075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39696/0 (socket says 172.21.15.19:39696) 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713b7fe640 1 -- 172.21.15.19:0/3889619256 learned_addr learned my addr 172.21.15.19:0/3889619256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713bfff640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f713c1037c0 0x7f713c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f7132ffd640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c102e00 0x7f713c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713b7fe640 1 -- 172.21.15.19:0/3889619256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f713c1037c0 msgr2=0x7f713c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713b7fe640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f713c1037c0 0x7f713c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713b7fe640 1 -- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c102e00 msgr2=0x7f713c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713b7fe640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c102e00 0x7f713c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713b7fe640 1 -- 172.21.15.19:0/3889619256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713c0753e0 con 0x7f713c101c00 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f713b7fe640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c101c00 0x7f713c075f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c970 tx=0x7f712c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f71397fa640 1 -- 172.21.15.19:0/3889619256 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c007c10 con 0x7f713c101c00 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f71397fa640 1 -- 172.21.15.19:0/3889619256 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f712c007db0 con 0x7f713c101c00 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713c1a1fc0 con 0x7f713c101c00 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.546+0000 7f71397fa640 1 -- 172.21.15.19:0/3889619256 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c00f940 con 0x7f713c101c00 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.547+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713c1a2470 con 0x7f713c101c00 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.548+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713c108f30 con 0x7f713c101c00 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.549+0000 7f71397fa640 1 -- 172.21.15.19:0/3889619256 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f712c00fae0 con 0x7f713c101c00 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.549+0000 7f71397fa640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71280a32c0 0x7f71280a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.550+0000 7f71397fa640 1 -- 172.21.15.19:0/3889619256 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6261+0+0 (secure 0 0 0) 0x7f712c0cf660 con 0x7f713c101c00 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.550+0000 7f7132ffd640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71280a32c0 0x7f71280a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.550+0000 7f7132ffd640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71280a32c0 0x7f71280a5780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7124031440 tx=0x7f71240315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.553+0000 7f71397fa640 1 -- 172.21.15.19:0/3889619256 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f712c09cca0 con 0x7f713c101c00 2024-09-16T00:24:35.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: osdmap e63: 10 total, 9 up, 10 in 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:35.737+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f713c062630 con 0x7f71280a32c0 2024-09-16T00:24:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi001 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi001 to 4273M 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] boot 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: osdmap e64: 10 total, 10 up, 10 in 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:36.488 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:24:36.488 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:24:36.488 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:36.488 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi001 2024-09-16T00:24:36.488 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi001 to 4273M 2024-09-16T00:24:36.488 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:36.488 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] boot 2024-09-16T00:24:36.488 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: osdmap e64: 10 total, 10 up, 10 in 2024-09-16T00:24:36.489 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:36.489 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:36.489 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:36.489 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:24:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:24:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:24:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi001 2024-09-16T00:24:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi001 to 4273M 2024-09-16T00:24:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: osd.9 [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] boot 2024-09-16T00:24:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: osdmap e64: 10 total, 10 up, 10 in 2024-09-16T00:24:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:24:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:37.445 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:37 smithi019 ceph-mon[35186]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:37.445 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:37 smithi019 ceph-mon[35186]: osdmap e65: 10 total, 10 up, 10 in 2024-09-16T00:24:37.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:37 smithi001 ceph-mon[30934]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:37.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:37 smithi001 ceph-mon[30934]: osdmap e65: 10 total, 10 up, 10 in 2024-09-16T00:24:37.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:37 smithi153 ceph-mon[35538]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:37.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:37 smithi153 ceph-mon[35538]: osdmap e65: 10 total, 10 up, 10 in 2024-09-16T00:24:38.493 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:38 smithi019 ceph-mon[35186]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:38.493 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:38 smithi019 ceph-mon[35186]: from='client.? 172.21.15.19:0/2329316003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74fcd28a-7ae8-4dca-a314-b22a8c871db8"}]: dispatch 2024-09-16T00:24:38.493 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:38 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74fcd28a-7ae8-4dca-a314-b22a8c871db8"}]: dispatch 2024-09-16T00:24:38.493 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:38 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74fcd28a-7ae8-4dca-a314-b22a8c871db8"}]': finished 2024-09-16T00:24:38.494 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:38 smithi019 ceph-mon[35186]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:24:38.494 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:38 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:24:38.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:38 smithi001 ceph-mon[30934]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:38.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:38 smithi001 ceph-mon[30934]: from='client.? 172.21.15.19:0/2329316003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74fcd28a-7ae8-4dca-a314-b22a8c871db8"}]: dispatch 2024-09-16T00:24:38.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:38 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74fcd28a-7ae8-4dca-a314-b22a8c871db8"}]: dispatch 2024-09-16T00:24:38.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:38 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74fcd28a-7ae8-4dca-a314-b22a8c871db8"}]': finished 2024-09-16T00:24:38.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:38 smithi001 ceph-mon[30934]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:24:38.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:38 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:24:38.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:38 smithi153 ceph-mon[35538]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:38.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:38 smithi153 ceph-mon[35538]: from='client.? 172.21.15.19:0/2329316003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74fcd28a-7ae8-4dca-a314-b22a8c871db8"}]: dispatch 2024-09-16T00:24:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:38 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74fcd28a-7ae8-4dca-a314-b22a8c871db8"}]: dispatch 2024-09-16T00:24:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:38 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74fcd28a-7ae8-4dca-a314-b22a8c871db8"}]': finished 2024-09-16T00:24:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:38 smithi153 ceph-mon[35538]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:24:38.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:38 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:24:39.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:39 smithi019 ceph-mon[35186]: from='client.? 172.21.15.19:0/3029312703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:39.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:39 smithi001 ceph-mon[30934]: from='client.? 172.21.15.19:0/3029312703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:39.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:39 smithi153 ceph-mon[35538]: from='client.? 172.21.15.19:0/3029312703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:40.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:40 smithi019 ceph-mon[35186]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:40.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:40 smithi001 ceph-mon[30934]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:40.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:40 smithi153 ceph-mon[35538]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:42 smithi019 ceph-mon[35186]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:42.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:42 smithi001 ceph-mon[30934]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:42.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:42 smithi153 ceph-mon[35538]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:44 smithi019 ceph-mon[35186]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:44.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:44 smithi001 ceph-mon[30934]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:44.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:44 smithi153 ceph-mon[35538]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:46.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:46 smithi019 ceph-mon[35186]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:46 smithi153 ceph-mon[35538]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:46.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:46 smithi001 ceph-mon[30934]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:47.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:47 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:24:47.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:47 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:47.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:47 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:24:47.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:47 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:47.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:47 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:24:47.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:47 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:48 smithi019 ceph-mon[35186]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:48 smithi019 ceph-mon[35186]: Deploying daemon osd.10 on smithi019 2024-09-16T00:24:48.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:48 smithi001 ceph-mon[30934]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:48.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:48 smithi001 ceph-mon[30934]: Deploying daemon osd.10 on smithi019 2024-09-16T00:24:48.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:48 smithi153 ceph-mon[35538]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:48.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:48 smithi153 ceph-mon[35538]: Deploying daemon osd.10 on smithi019 2024-09-16T00:24:50.561 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:50 smithi019 ceph-mon[35186]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:50.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:50 smithi001 ceph-mon[30934]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:50.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:50 smithi153 ceph-mon[35538]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:51.577 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:51 smithi019 ceph-mon[35186]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:51.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:51 smithi001 ceph-mon[30934]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:51.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:51 smithi153 ceph-mon[35538]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:53.487 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:53 smithi019 ceph-mon[35186]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:53.487 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:53.487 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:53.487 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:53 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:53.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:53 smithi001 ceph-mon[30934]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:53.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:53.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:53.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:53 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:53.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:53 smithi153 ceph-mon[35538]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:53.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:53.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:53.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:53 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:55.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:55 smithi019 ceph-mon[35186]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:55 smithi001 ceph-mon[30934]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:55.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:55 smithi153 ceph-mon[35538]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:55.989 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 10 on host 'smithi019' 2024-09-16T00:24:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.989+0000 7f71397fa640 1 -- 172.21.15.19:0/3889619256 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f713c062630 con 0x7f71280a32c0 2024-09-16T00:24:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.991+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71280a32c0 msgr2=0x7f71280a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.991+0000 7f7141b28640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71280a32c0 0x7f71280a5780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7124031440 tx=0x7f71240315b0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.991+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c101c00 msgr2=0x7f713c075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.991+0000 7f7141b28640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c101c00 0x7f713c075f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c970 tx=0x7f712c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:24:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.992+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 shutdown_connections 2024-09-16T00:24:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.992+0000 7f7141b28640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71280a32c0 0x7f71280a5780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.992+0000 7f7141b28640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f713c1037c0 0x7f713c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.992+0000 7f7141b28640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c102e00 0x7f713c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.992+0000 7f7141b28640 1 --2- 172.21.15.19:0/3889619256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f713c101c00 0x7f713c075f90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.992+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 >> 172.21.15.19:0/3889619256 conn(0x7f713c0fd390 msgr2=0x7f713c106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.992+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 shutdown_connections 2024-09-16T00:24:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-16T00:24:55.992+0000 7f7141b28640 1 -- 172.21.15.19:0/3889619256 wait complete. 2024-09-16T00:24:56.648 DEBUG:teuthology.orchestra.run.smithi019:osd.10> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.10.service 2024-09-16T00:24:56.651 INFO:tasks.cephadm:Deploying osd.11 on smithi153 with /dev/vg_nvme/lv_1... 2024-09-16T00:24:56.651 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:24:56.911 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:24:57.275 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:56 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.275 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:56 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.275 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:56 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.275 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:56 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.280 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:56 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.280 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:56 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.280 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:56 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.280 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:56 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:56 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:56 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:56 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:57.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:56 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:24:58.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:57 smithi001 ceph-mon[30934]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:58.277 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:57 smithi153 ceph-mon[35538]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:58.284 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:57 smithi019 ceph-mon[35186]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:24:58.550 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:24:58 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:24:58.504+0000 7f5ca24c5640 -1 osd.10 0 log_to_monitors true 2024-09-16T00:24:59.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:58 smithi153 ceph-mon[35538]: from='osd.10 [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:24:59.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:24:58 smithi153 ceph-mon[35538]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:24:59.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:58 smithi019 ceph-mon[35186]: from='osd.10 [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:24:59.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:24:58 smithi019 ceph-mon[35186]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:24:59.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:58 smithi001 ceph-mon[30934]: from='osd.10 [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:24:59.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:24:58 smithi001 ceph-mon[30934]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:24:59.625 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-16T00:24:59.652 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_1 2024-09-16T00:24:59.911 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:25:00.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:00 smithi153 ceph-mon[35538]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:25:00.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:00 smithi153 ceph-mon[35538]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:25:00.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:00 smithi153 ceph-mon[35538]: from='osd.10 [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:25:00.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:00 smithi153 ceph-mon[35538]: osdmap e67: 11 total, 10 up, 11 in 2024-09-16T00:25:00.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:00 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:00.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:00 smithi153 ceph-mon[35538]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:25:00.357 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:25:00 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:25:00.003+0000 7f5c9e448640 -1 osd.10 0 waiting for initial osdmap 2024-09-16T00:25:00.357 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:25:00 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:25:00.020+0000 7f5c9a270640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:25:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:00 smithi019 ceph-mon[35186]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:25:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:00 smithi019 ceph-mon[35186]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:25:00.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:00 smithi019 ceph-mon[35186]: from='osd.10 [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:25:00.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:00 smithi019 ceph-mon[35186]: osdmap e67: 11 total, 10 up, 11 in 2024-09-16T00:25:00.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:00 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:00.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:00 smithi019 ceph-mon[35186]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:25:00.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 -- 172.21.15.153:0/1043902773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09541037b0 msgr2=0x7f095410a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:00.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 --2- 172.21.15.153:0/1043902773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09541037b0 0x7f095410a040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f094c009ca0 tx=0x7f094c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:25:00.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 -- 172.21.15.153:0/1043902773 shutdown_connections 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 --2- 172.21.15.153:0/1043902773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09541037b0 0x7f095410a040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 --2- 172.21.15.153:0/1043902773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0954102df0 0x7f0954103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 --2- 172.21.15.153:0/1043902773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0954101bf0 0x7f0954101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 -- 172.21.15.153:0/1043902773 >> 172.21.15.153:0/1043902773 conn(0x7f09540fd3c0 msgr2=0x7f09540ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 -- 172.21.15.153:0/1043902773 shutdown_connections 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 -- 172.21.15.153:0/1043902773 wait complete. 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 Processor -- start 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.370+0000 7f095a86d640 1 -- start start 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f095a86d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954101bf0 0x7f0954197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f095a86d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0954102df0 0x7f0954198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f095a86d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09541037b0 0x7f095419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f095a86d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f095410d8a0 con 0x7f09541037b0 2024-09-16T00:25:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f095a86d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f095410d720 con 0x7f0954102df0 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f095a86d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f095410da20 con 0x7f0954101bf0 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f0953fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954101bf0 0x7f0954197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f0953fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954101bf0 0x7f0954197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51712/0 (socket says 172.21.15.153:51712) 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f0953fff640 1 -- 172.21.15.153:0/1287033551 learned_addr learned my addr 172.21.15.153:0/1287033551 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f0958e0a640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09541037b0 0x7f095419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f09537fe640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0954102df0 0x7f0954198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f0953fff640 1 -- 172.21.15.153:0/1287033551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0954102df0 msgr2=0x7f0954198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f0953fff640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0954102df0 0x7f0954198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f0953fff640 1 -- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09541037b0 msgr2=0x7f095419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f0953fff640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09541037b0 0x7f095419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.371+0000 7f0953fff640 1 -- 172.21.15.153:0/1287033551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095419fa60 con 0x7f0954101bf0 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.372+0000 7f0953fff640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954101bf0 0x7f0954197e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f094400d770 tx=0x7f094400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:00.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.372+0000 7f09517fa640 1 -- 172.21.15.153:0/1287033551 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944017070 con 0x7f0954101bf0 2024-09-16T00:25:00.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.372+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095419fd50 con 0x7f0954101bf0 2024-09-16T00:25:00.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.372+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09541a0200 con 0x7f0954101bf0 2024-09-16T00:25:00.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.372+0000 7f09517fa640 1 -- 172.21.15.153:0/1287033551 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0944009c30 con 0x7f0954101bf0 2024-09-16T00:25:00.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.372+0000 7f09517fa640 1 -- 172.21.15.153:0/1287033551 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944003dd0 con 0x7f0954101bf0 2024-09-16T00:25:00.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.373+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09340052c0 con 0x7f0954101bf0 2024-09-16T00:25:00.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.374+0000 7f09517fa640 1 -- 172.21.15.153:0/1287033551 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f0944004000 con 0x7f0954101bf0 2024-09-16T00:25:00.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.375+0000 7f09517fa640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09400a32f0 0x7f09400a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:00.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.375+0000 7f09517fa640 1 -- 172.21.15.153:0/1287033551 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6414+0+0 (secure 0 0 0) 0x7f09440cee70 con 0x7f0954101bf0 2024-09-16T00:25:00.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.375+0000 7f09537fe640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09400a32f0 0x7f09400a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:00.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.376+0000 7f09537fe640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09400a32f0 0x7f09400a57b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f093c005fd0 tx=0x7f093c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:00.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.378+0000 7f09517fa640 1 -- 172.21.15.153:0/1287033551 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f094409c380 con 0x7f0954101bf0 2024-09-16T00:25:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:00 smithi001 ceph-mon[30934]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:25:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:00 smithi001 ceph-mon[30934]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:25:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:00 smithi001 ceph-mon[30934]: from='osd.10 [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:25:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:00 smithi001 ceph-mon[30934]: osdmap e67: 11 total, 10 up, 11 in 2024-09-16T00:25:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:00 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:00.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:00 smithi001 ceph-mon[30934]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-16T00:25:00.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:00.559+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0934000bf0 con 0x7f09400a32f0 2024-09-16T00:25:01.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:25:01.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi019 2024-09-16T00:25:01.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: osdmap e68: 11 total, 10 up, 11 in 2024-09-16T00:25:01.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:01.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:01.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi019 to 4273M 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:01.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:01.214 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.214 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:25:01.214 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:25:01.214 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:01 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:01.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:25:01.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi019 2024-09-16T00:25:01.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: osdmap e68: 11 total, 10 up, 11 in 2024-09-16T00:25:01.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi019 to 4273M 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:01.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:01.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:01.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:25:01.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:25:01.359 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:01 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:01.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi019 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: osdmap e68: 11 total, 10 up, 11 in 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi019 to 4273M 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:01.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:01.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:01.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:01.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:01.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:01.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:25:01.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:25:01.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:01 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:02 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:25:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:02 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:25:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:02 smithi153 ceph-mon[35538]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:25:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:02 smithi153 ceph-mon[35538]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:02 smithi153 ceph-mon[35538]: osd.10 [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] boot 2024-09-16T00:25:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:02 smithi153 ceph-mon[35538]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:25:02.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:02 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:02 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:25:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:02 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:25:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:02 smithi019 ceph-mon[35186]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:25:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:02 smithi019 ceph-mon[35186]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:02 smithi019 ceph-mon[35186]: osd.10 [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] boot 2024-09-16T00:25:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:02 smithi019 ceph-mon[35186]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:25:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:02 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:02 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:25:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:02 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:25:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:02 smithi001 ceph-mon[30934]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:25:02.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:02 smithi001 ceph-mon[30934]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:02.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:02.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:02 smithi001 ceph-mon[30934]: osd.10 [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] boot 2024-09-16T00:25:02.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:02 smithi001 ceph-mon[30934]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:25:02.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:02 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:25:03.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:03 smithi019 ceph-mon[35186]: osdmap e70: 11 total, 11 up, 11 in 2024-09-16T00:25:03.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:03 smithi019 ceph-mon[35186]: from='client.? 172.21.15.153:0/4178632711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf7afb8e-9c11-460a-960d-57f1eba3ead6"}]: dispatch 2024-09-16T00:25:03.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:03 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf7afb8e-9c11-460a-960d-57f1eba3ead6"}]: dispatch 2024-09-16T00:25:03.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:03 smithi019 ceph-mon[35186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf7afb8e-9c11-460a-960d-57f1eba3ead6"}]': finished 2024-09-16T00:25:03.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:03 smithi019 ceph-mon[35186]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T00:25:03.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:03.372 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:03 smithi153 ceph-mon[35538]: osdmap e70: 11 total, 11 up, 11 in 2024-09-16T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:03 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/4178632711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf7afb8e-9c11-460a-960d-57f1eba3ead6"}]: dispatch 2024-09-16T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:03 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf7afb8e-9c11-460a-960d-57f1eba3ead6"}]: dispatch 2024-09-16T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:03 smithi153 ceph-mon[35538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf7afb8e-9c11-460a-960d-57f1eba3ead6"}]': finished 2024-09-16T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:03 smithi153 ceph-mon[35538]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:03.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:03 smithi001 ceph-mon[30934]: osdmap e70: 11 total, 11 up, 11 in 2024-09-16T00:25:03.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:03 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/4178632711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf7afb8e-9c11-460a-960d-57f1eba3ead6"}]: dispatch 2024-09-16T00:25:03.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:03 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf7afb8e-9c11-460a-960d-57f1eba3ead6"}]: dispatch 2024-09-16T00:25:03.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:03 smithi001 ceph-mon[30934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf7afb8e-9c11-460a-960d-57f1eba3ead6"}]': finished 2024-09-16T00:25:03.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:03 smithi001 ceph-mon[30934]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T00:25:03.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:04.279 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:04 smithi019 ceph-mon[35186]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:04.279 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:04 smithi019 ceph-mon[35186]: from='client.? 172.21.15.153:0/2223714054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:25:04.279 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:04 smithi019 ceph-mon[35186]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:25:04.279 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:04 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:04.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:04 smithi001 ceph-mon[30934]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:04.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:04 smithi001 ceph-mon[30934]: from='client.? 172.21.15.153:0/2223714054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:25:04.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:04 smithi001 ceph-mon[30934]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:25:04.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:04 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:04.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:04 smithi153 ceph-mon[35538]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:04.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:04 smithi153 ceph-mon[35538]: from='client.? 172.21.15.153:0/2223714054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:25:04.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:04 smithi153 ceph-mon[35538]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:25:04.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:04 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:06.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:05 smithi019 ceph-mon[35186]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:06.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:05 smithi019 ceph-mon[35186]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:25:06.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:06.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:05 smithi001 ceph-mon[30934]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:06.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:05 smithi001 ceph-mon[30934]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:25:06.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:06.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:05 smithi153 ceph-mon[35538]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:06.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:05 smithi153 ceph-mon[35538]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:25:06.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:08.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:08 smithi019 ceph-mon[35186]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-16T00:25:08.451 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:08 smithi153 ceph-mon[35538]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-16T00:25:08.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:08 smithi001 ceph-mon[30934]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-16T00:25:10.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:10 smithi153 ceph-mon[35538]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:25:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:10 smithi019 ceph-mon[35186]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:25:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:10 smithi001 ceph-mon[30934]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:25:12.295 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:12 smithi153 ceph-mon[35538]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-16T00:25:12.295 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:25:12.295 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:12.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:12 smithi019 ceph-mon[35186]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-16T00:25:12.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:25:12.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:12 smithi001 ceph-mon[30934]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-16T00:25:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:25:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:13.083 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:13 smithi153 ceph-mon[35538]: Deploying daemon osd.11 on smithi153 2024-09-16T00:25:13.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:13 smithi019 ceph-mon[35186]: Deploying daemon osd.11 on smithi153 2024-09-16T00:25:13.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:13 smithi001 ceph-mon[30934]: Deploying daemon osd.11 on smithi153 2024-09-16T00:25:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:14 smithi019 ceph-mon[35186]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-16T00:25:14.416 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:14 smithi153 ceph-mon[35538]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-16T00:25:14.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:14 smithi001 ceph-mon[30934]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-16T00:25:16.314 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:16 smithi153 ceph-mon[35538]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:25:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:16 smithi019 ceph-mon[35186]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:25:16.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:16 smithi001 ceph-mon[30934]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:25:18.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:18 smithi153 ceph-mon[35538]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:18.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:18.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:18.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:18 smithi019 ceph-mon[35186]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:18 smithi001 ceph-mon[30934]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:20.293 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:20 smithi153 ceph-mon[35538]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:20.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:20 smithi019 ceph-mon[35186]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:20.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:20 smithi001 ceph-mon[30934]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:21.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.481+0000 7f09517fa640 1 -- 172.21.15.153:0/1287033551 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0934000bf0 con 0x7f09400a32f0 2024-09-16T00:25:21.482 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 11 on host 'smithi153' 2024-09-16T00:25:21.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09400a32f0 msgr2=0x7f09400a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:21.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09400a32f0 0x7f09400a57b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f093c005fd0 tx=0x7f093c005850 comp rx=0 tx=0).stop 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954101bf0 msgr2=0x7f0954197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954101bf0 0x7f0954197e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f094400d770 tx=0x7f094400dc30 comp rx=0 tx=0).stop 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 shutdown_connections 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09541037b0 0x7f095419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0954102df0 0x7f0954198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09400a32f0 0x7f09400a57b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 --2- 172.21.15.153:0/1287033551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954101bf0 0x7f0954197e30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 >> 172.21.15.153:0/1287033551 conn(0x7f09540fd3c0 msgr2=0x7f0954106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 shutdown_connections 2024-09-16T00:25:21.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:21.484+0000 7f095a86d640 1 -- 172.21.15.153:0/1287033551 wait complete. 2024-09-16T00:25:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:21 smithi153 ceph-mon[35538]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:21.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:21 smithi001 ceph-mon[30934]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:21.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:21.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:21.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:21.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:21.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:21.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:21.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:21.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:21 smithi019 ceph-mon[35186]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:22.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:22.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:22.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:22.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:22.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:22.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:23.332 DEBUG:teuthology.orchestra.run.smithi153:osd.11> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.11.service 2024-09-16T00:25:23.334 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-16T00:25:23.334 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:25:23.565 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:25:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:25:23.438+0000 7f7666dab640 -1 osd.11 0 log_to_monitors true 2024-09-16T00:25:23.604 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:23.819 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:23 smithi153 ceph-mon[35538]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:23.819 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:23 smithi153 ceph-mon[35538]: from='osd.11 [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:25:23.819 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:23 smithi153 ceph-mon[35538]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:25:23.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:23 smithi001 ceph-mon[30934]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:23.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:23 smithi001 ceph-mon[30934]: from='osd.11 [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:25:23.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:23 smithi001 ceph-mon[30934]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:25:24.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:23 smithi019 ceph-mon[35186]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:24.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:23 smithi019 ceph-mon[35186]: from='osd.11 [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:25:24.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:23 smithi019 ceph-mon[35186]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:25:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.091+0000 7fc44e7e1640 1 -- 172.21.15.1:0/1367462435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc448105090 msgr2=0x7fc448107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.091+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/1367462435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc448105090 0x7fc448107480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009900 tx=0x7fc43c02a410 comp rx=0 tx=0).stop 2024-09-16T00:25:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.091+0000 7fc44e7e1640 1 -- 172.21.15.1:0/1367462435 shutdown_connections 2024-09-16T00:25:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.091+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/1367462435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4481079c0 0x7fc448109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.091+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/1367462435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc448105090 0x7fc448107480 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.091+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/1367462435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc448102760 0x7fc448104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.091+0000 7fc44e7e1640 1 -- 172.21.15.1:0/1367462435 >> 172.21.15.1:0/1367462435 conn(0x7fc4480fc2a0 msgr2=0x7fc4480fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.091+0000 7fc44e7e1640 1 -- 172.21.15.1:0/1367462435 shutdown_connections 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.091+0000 7fc44e7e1640 1 -- 172.21.15.1:0/1367462435 wait complete. 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc44e7e1640 1 Processor -- start 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc44e7e1640 1 -- start start 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc44e7e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc448102760 0x7fc4481020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc44e7e1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc448105090 0x7fc448100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc44e7e1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4481079c0 0x7fc448100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc44e7e1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc44810d8a0 con 0x7fc448105090 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc44e7e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc44810d720 con 0x7fc448102760 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc44e7e1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc44810da20 con 0x7fc4481079c0 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc4477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc448105090 0x7fc448100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc447fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc448102760 0x7fc4481020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc447fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc448102760 0x7fc4481020c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:38170/0 (socket says 172.21.15.1:38170) 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc447fff640 1 -- 172.21.15.1:0/2741411136 learned_addr learned my addr 172.21.15.1:0/2741411136 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.092+0000 7fc44cd7e640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4481079c0 0x7fc448100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc4477fe640 1 -- 172.21.15.1:0/2741411136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4481079c0 msgr2=0x7fc448100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc4477fe640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4481079c0 0x7fc448100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc4477fe640 1 -- 172.21.15.1:0/2741411136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc448102760 msgr2=0x7fc4481020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc4477fe640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc448102760 0x7fc4481020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc4477fe640 1 -- 172.21.15.1:0/2741411136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc448101510 con 0x7fc448105090 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc447fff640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc448102760 0x7fc4481020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc4477fe640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc448105090 0x7fc448100710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009230 tx=0x7fc43c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc4457fa640 1 -- 172.21.15.1:0/2741411136 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c03f070 con 0x7fc448105090 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc4457fa640 1 -- 172.21.15.1:0/2741411136 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43c003cd0 con 0x7fc448105090 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4481a20b0 con 0x7fc448105090 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc4457fa640 1 -- 172.21.15.1:0/2741411136 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c035070 con 0x7fc448105090 2024-09-16T00:25:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.093+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4481a2560 con 0x7fc448105090 2024-09-16T00:25:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.094+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4480fdd90 con 0x7fc448105090 2024-09-16T00:25:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.095+0000 7fc4457fa640 1 -- 172.21.15.1:0/2741411136 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fc43c03b4f0 con 0x7fc448105090 2024-09-16T00:25:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.096+0000 7fc4457fa640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4300a3240 0x7fc4300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.096+0000 7fc4457fa640 1 -- 172.21.15.1:0/2741411136 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6831+0+0 (secure 0 0 0) 0x7fc43c0ef790 con 0x7fc448105090 2024-09-16T00:25:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.096+0000 7fc447fff640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4300a3240 0x7fc4300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.096+0000 7fc447fff640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4300a3240 0x7fc4300a5700 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc438011500 tx=0x7fc438005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.100+0000 7fc4457fa640 1 -- 172.21.15.1:0/2741411136 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc43c0bc430 con 0x7fc448105090 2024-09-16T00:25:24.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.281+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc44802da10 con 0x7fc448105090 2024-09-16T00:25:24.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.281+0000 7fc4457fa640 1 -- 172.21.15.1:0/2741411136 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fc43c0c0730 con 0x7fc448105090 2024-09-16T00:25:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4300a3240 msgr2=0x7fc4300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4300a3240 0x7fc4300a5700 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc438011500 tx=0x7fc438005f90 comp rx=0 tx=0).stop 2024-09-16T00:25:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc448105090 msgr2=0x7fc448100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc448105090 0x7fc448100710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009230 tx=0x7fc43c031b60 comp rx=0 tx=0).stop 2024-09-16T00:25:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 shutdown_connections 2024-09-16T00:25:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4300a3240 0x7fc4300a5700 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4481079c0 0x7fc448100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc448105090 0x7fc448100710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 --2- 172.21.15.1:0/2741411136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc448102760 0x7fc4481020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 >> 172.21.15.1:0/2741411136 conn(0x7fc4480fc2a0 msgr2=0x7fc4481030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 shutdown_connections 2024-09-16T00:25:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:24.284+0000 7fc44e7e1640 1 -- 172.21.15.1:0/2741411136 wait complete. 2024-09-16T00:25:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:24 smithi153 ceph-mon[35538]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:25:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:24 smithi153 ceph-mon[35538]: osdmap e74: 12 total, 11 up, 12 in 2024-09-16T00:25:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:24 smithi153 ceph-mon[35538]: from='osd.11 [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:25:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:24 smithi153 ceph-mon[35538]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:25:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:24 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2741411136' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:25:24.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:24 smithi001 ceph-mon[30934]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:25:24.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:24 smithi001 ceph-mon[30934]: osdmap e74: 12 total, 11 up, 12 in 2024-09-16T00:25:24.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:24 smithi001 ceph-mon[30934]: from='osd.11 [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:25:24.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:24.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:24 smithi001 ceph-mon[30934]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:25:24.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:24 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2741411136' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:25:25.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:24 smithi019 ceph-mon[35186]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:25:25.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:24 smithi019 ceph-mon[35186]: osdmap e74: 12 total, 11 up, 12 in 2024-09-16T00:25:25.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:24 smithi019 ceph-mon[35186]: from='osd.11 [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:25:25.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:25.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:24 smithi019 ceph-mon[35186]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-16T00:25:25.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:24 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2741411136' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:25:25.253 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726446301,"num_in_osds":12,"osd_in_since":1726446302,"num_remapped_pgs":0} 2024-09-16T00:25:25.647 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:25:25.379+0000 7f7662d2e640 -1 osd.11 0 waiting for initial osdmap 2024-09-16T00:25:25.647 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:25:25.394+0000 7f765ab4e640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:25:25.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:25.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:25:25.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: osdmap e75: 12 total, 11 up, 12 in 2024-09-16T00:25:25.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:25.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:25.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: Detected new or changed devices on smithi153 2024-09-16T00:25:25.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:25.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:25.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:25.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:25.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:25.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:25.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-16T00:25:25.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:25.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:25.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:25.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:25.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:25 smithi001 ceph-mon[30934]: from='osd.11 ' entity='osd.11' 2024-09-16T00:25:25.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:25.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: osdmap e75: 12 total, 11 up, 12 in 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: Detected new or changed devices on smithi153 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:25.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:25.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:25.963 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:25 smithi153 ceph-mon[35538]: from='osd.11 ' entity='osd.11' 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: osdmap e75: 12 total, 11 up, 12 in 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: Detected new or changed devices on smithi153 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:26.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:26.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:26.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:26.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-16T00:25:26.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:26.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:26.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:26.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:26.110 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:25 smithi019 ceph-mon[35186]: from='osd.11 ' entity='osd.11' 2024-09-16T00:25:26.255 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:25:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:26.905 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:26 smithi001 ceph-mon[30934]: purged_snaps scrub starts 2024-09-16T00:25:26.905 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:26 smithi001 ceph-mon[30934]: purged_snaps scrub ok 2024-09-16T00:25:26.905 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:26.905 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:26 smithi001 ceph-mon[30934]: osd.11 [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] boot 2024-09-16T00:25:26.906 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:26 smithi001 ceph-mon[30934]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:25:26.906 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:26.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:26 smithi153 ceph-mon[35538]: purged_snaps scrub starts 2024-09-16T00:25:26.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:26 smithi153 ceph-mon[35538]: purged_snaps scrub ok 2024-09-16T00:25:26.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:26.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:26 smithi153 ceph-mon[35538]: osd.11 [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] boot 2024-09-16T00:25:26.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:26 smithi153 ceph-mon[35538]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:25:26.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:27.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:26 smithi019 ceph-mon[35186]: purged_snaps scrub starts 2024-09-16T00:25:27.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:26 smithi019 ceph-mon[35186]: purged_snaps scrub ok 2024-09-16T00:25:27.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:27.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:26 smithi019 ceph-mon[35186]: osd.11 [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] boot 2024-09-16T00:25:27.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:26 smithi019 ceph-mon[35186]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:25:27.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:25:27.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.117+0000 7fbe56689640 1 -- 172.21.15.1:0/1863718671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 msgr2=0x7fbe5010a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.117+0000 7fbe56689640 1 --2- 172.21.15.1:0/1863718671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 0x7fbe5010a100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009cd0 tx=0x7fbe4802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.117+0000 7fbe56689640 1 -- 172.21.15.1:0/1863718671 shutdown_connections 2024-09-16T00:25:27.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.117+0000 7fbe56689640 1 --2- 172.21.15.1:0/1863718671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 0x7fbe5010a100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.117+0000 7fbe56689640 1 --2- 172.21.15.1:0/1863718671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe50102eb0 0x7fbe50103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.117+0000 7fbe56689640 1 --2- 172.21.15.1:0/1863718671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe500ffed0 0x7fbe501002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.117+0000 7fbe56689640 1 -- 172.21.15.1:0/1863718671 >> 172.21.15.1:0/1863718671 conn(0x7fbe500fba40 msgr2=0x7fbe500fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.117+0000 7fbe56689640 1 -- 172.21.15.1:0/1863718671 shutdown_connections 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.117+0000 7fbe56689640 1 -- 172.21.15.1:0/1863718671 wait complete. 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe56689640 1 Processor -- start 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe56689640 1 -- start start 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe56689640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe500ffed0 0x7fbe50102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe56689640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe50102eb0 0x7fbe50100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe56689640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 0x7fbe501013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe56689640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5010d910 con 0x7fbe50103870 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe56689640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5010d790 con 0x7fbe500ffed0 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe56689640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5010da90 con 0x7fbe50102eb0 2024-09-16T00:25:27.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe54c26640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 0x7fbe501013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe54c26640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 0x7fbe501013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49440/0 (socket says 172.21.15.1:49440) 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe54c26640 1 -- 172.21.15.1:0/436815736 learned_addr learned my addr 172.21.15.1:0/436815736 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe4ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe500ffed0 0x7fbe50102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe4f7fe640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe50102eb0 0x7fbe50100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe54c26640 1 -- 172.21.15.1:0/436815736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe50102eb0 msgr2=0x7fbe50100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe54c26640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe50102eb0 0x7fbe50100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe54c26640 1 -- 172.21.15.1:0/436815736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe500ffed0 msgr2=0x7fbe50102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe54c26640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe500ffed0 0x7fbe50102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.118+0000 7fbe54c26640 1 -- 172.21.15.1:0/436815736 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe50101c60 con 0x7fbe50103870 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.119+0000 7fbe54c26640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 0x7fbe501013a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009df0 tx=0x7fbe48003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.119+0000 7fbe4d7fa640 1 -- 172.21.15.1:0/436815736 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48004450 con 0x7fbe50103870 2024-09-16T00:25:27.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.119+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe501a20d0 con 0x7fbe50103870 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.119+0000 7fbe4d7fa640 1 -- 172.21.15.1:0/436815736 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe48003e40 con 0x7fbe50103870 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.119+0000 7fbe4d7fa640 1 -- 172.21.15.1:0/436815736 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe480466e0 con 0x7fbe50103870 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.119+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe501a26b0 con 0x7fbe50103870 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.120+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe50109100 con 0x7fbe50103870 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.121+0000 7fbe4d7fa640 1 -- 172.21.15.1:0/436815736 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fbe480305a0 con 0x7fbe50103870 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.122+0000 7fbe4d7fa640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbe380a3240 0x7fbe380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.122+0000 7fbe4ffff640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbe380a3240 0x7fbe380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.122+0000 7fbe4d7fa640 1 -- 172.21.15.1:0/436815736 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fbe4802e030 con 0x7fbe50103870 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.122+0000 7fbe4ffff640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbe380a3240 0x7fbe380a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe5006a590 tx=0x7fbe4000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:27.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.125+0000 7fbe4d7fa640 1 -- 172.21.15.1:0/436815736 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe4804b060 con 0x7fbe50103870 2024-09-16T00:25:27.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.298+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbe50062df0 con 0x7fbe50103870 2024-09-16T00:25:27.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.299+0000 7fbe4d7fa640 1 -- 172.21.15.1:0/436815736 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7fbe480bdcd0 con 0x7fbe50103870 2024-09-16T00:25:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:27.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbe380a3240 msgr2=0x7fbe380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbe380a3240 0x7fbe380a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe5006a590 tx=0x7fbe4000b390 comp rx=0 tx=0).stop 2024-09-16T00:25:27.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 msgr2=0x7fbe501013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 0x7fbe501013a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009df0 tx=0x7fbe48003c20 comp rx=0 tx=0).stop 2024-09-16T00:25:27.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 shutdown_connections 2024-09-16T00:25:27.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe50103870 0x7fbe501013a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbe380a3240 0x7fbe380a5700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe50102eb0 0x7fbe50100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 --2- 172.21.15.1:0/436815736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe500ffed0 0x7fbe50102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 >> 172.21.15.1:0/436815736 conn(0x7fbe500fba40 msgr2=0x7fbe501062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:27.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 shutdown_connections 2024-09-16T00:25:27.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:27.301+0000 7fbe56689640 1 -- 172.21.15.1:0/436815736 wait complete. 2024-09-16T00:25:27.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:27 smithi001 ceph-mon[30934]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:27.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:27 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/436815736' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:25:27.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:27 smithi001 ceph-mon[30934]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:25:27.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:27 smithi153 ceph-mon[35538]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:27.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:27 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/436815736' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:25:27.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:27 smithi153 ceph-mon[35538]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:25:28.024 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726446326,"num_in_osds":12,"osd_in_since":1726446302,"num_remapped_pgs":0} 2024-09-16T00:25:28.025 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:25:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:27 smithi019 ceph-mon[35186]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:25:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:27 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/436815736' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:25:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:27 smithi019 ceph-mon[35186]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:25:28.283 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.783+0000 7f5543b88640 1 -- 172.21.15.1:0/991848081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074590 msgr2=0x7f553c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.783+0000 7f5543b88640 1 --2- 172.21.15.1:0/991848081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074590 0x7f553c074970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5538009900 tx=0x7f553802a480 comp rx=0 tx=0).stop 2024-09-16T00:25:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.783+0000 7f5543b88640 1 -- 172.21.15.1:0/991848081 shutdown_connections 2024-09-16T00:25:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.783+0000 7f5543b88640 1 --2- 172.21.15.1:0/991848081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c112b90 0x7f553c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.783+0000 7f5543b88640 1 --2- 172.21.15.1:0/991848081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c074eb0 0x7f553c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.783+0000 7f5543b88640 1 --2- 172.21.15.1:0/991848081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074590 0x7f553c074970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.783+0000 7f5543b88640 1 -- 172.21.15.1:0/991848081 >> 172.21.15.1:0/991848081 conn(0x7f553c0fd8c0 msgr2=0x7f553c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 -- 172.21.15.1:0/991848081 shutdown_connections 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 -- 172.21.15.1:0/991848081 wait complete. 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 Processor -- start 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 -- start start 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c074590 0x7f553c1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074eb0 0x7f553c1a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c112b90 0x7f553c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f553c118550 con 0x7f553c074eb0 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1183d0 con 0x7f553c074590 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.784+0000 7f5543b88640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1186d0 con 0x7f553c112b90 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541123640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074eb0 0x7f553c1a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541123640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074eb0 0x7f553c1a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49456/0 (socket says 172.21.15.1:49456) 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541123640 1 -- 172.21.15.1:0/67542350 learned_addr learned my addr 172.21.15.1:0/67542350 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541924640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c074590 0x7f553c1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5542125640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c112b90 0x7f553c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541123640 1 -- 172.21.15.1:0/67542350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c112b90 msgr2=0x7f553c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541123640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c112b90 0x7f553c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541123640 1 -- 172.21.15.1:0/67542350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c074590 msgr2=0x7f553c1a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541123640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c074590 0x7f553c1a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541123640 1 -- 172.21.15.1:0/67542350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c19f010 con 0x7f553c074eb0 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5542125640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c112b90 0x7f553c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5541123640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074eb0 0x7f553c1a4c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f552c00d950 tx=0x7f552c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5532ffd640 1 -- 172.21.15.1:0/67542350 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c017070 con 0x7f553c074eb0 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c19f300 con 0x7f553c074eb0 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5532ffd640 1 -- 172.21.15.1:0/67542350 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f552c009c30 con 0x7f553c074eb0 2024-09-16T00:25:28.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5532ffd640 1 -- 172.21.15.1:0/67542350 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c00f6b0 con 0x7f553c074eb0 2024-09-16T00:25:28.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.785+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c1ab5f0 con 0x7f553c074eb0 2024-09-16T00:25:28.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.786+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553c074970 con 0x7f553c074eb0 2024-09-16T00:25:28.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.787+0000 7f5532ffd640 1 -- 172.21.15.1:0/67542350 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f552c0109d0 con 0x7f553c074eb0 2024-09-16T00:25:28.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.788+0000 7f5532ffd640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55280a3240 0x7f55280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:28.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.788+0000 7f5541924640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55280a3240 0x7f55280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:28.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.788+0000 7f5532ffd640 1 -- 172.21.15.1:0/67542350 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f552c0cf190 con 0x7f553c074eb0 2024-09-16T00:25:28.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.789+0000 7f5541924640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55280a3240 0x7f55280a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5538000c00 tx=0x7f5538005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:28.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.792+0000 7f5532ffd640 1 -- 172.21.15.1:0/67542350 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f552c009de0 con 0x7f553c074eb0 2024-09-16T00:25:28.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.970+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f553c19fe10 con 0x7f553c074eb0 2024-09-16T00:25:28.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.972+0000 7f5532ffd640 1 -- 172.21.15.1:0/67542350 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18358 (secure 0 0 0) 0x7f553c19fe10 con 0x7f553c074eb0 2024-09-16T00:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":77,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:25:27.384443+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:25:28.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55280a3240 msgr2=0x7f55280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:28.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55280a3240 0x7f55280a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5538000c00 tx=0x7f5538005a20 comp rx=0 tx=0).stop 2024-09-16T00:25:28.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074eb0 msgr2=0x7f553c1a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074eb0 0x7f553c1a4c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f552c00d950 tx=0x7f552c00de10 comp rx=0 tx=0).stop 2024-09-16T00:25:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 shutdown_connections 2024-09-16T00:25:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55280a3240 0x7f55280a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c112b90 0x7f553c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:28.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553c074eb0 0x7f553c1a4c60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:28.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 --2- 172.21.15.1:0/67542350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c074590 0x7f553c1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:28.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 >> 172.21.15.1:0/67542350 conn(0x7f553c0fd8c0 msgr2=0x7f553c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:28.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 shutdown_connections 2024-09-16T00:25:28.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:28.974+0000 7f5543b88640 1 -- 172.21.15.1:0/67542350 wait complete. 2024-09-16T00:25:29.696 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:21:40.721155+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T00:25:29.696 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T00:25:29.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:29 smithi001 ceph-mon[30934]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:29.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:29 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/67542350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:25:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:29 smithi153 ceph-mon[35538]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:29 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/67542350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:25:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:29 smithi019 ceph-mon[35186]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:29 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/67542350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.446+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/1085290491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0102d90 msgr2=0x7ffbe0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.446+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/1085290491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0102d90 0x7ffbe0103170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009930 tx=0x7ffbd802a420 comp rx=0 tx=0).stop 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.446+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/1085290491 shutdown_connections 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.446+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/1085290491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe0108290 0x7ffbe010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.446+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/1085290491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbe0103740 0x7ffbe0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.446+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/1085290491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0102d90 0x7ffbe0103170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.446+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/1085290491 >> 172.21.15.1:0/1085290491 conn(0x7ffbe00fc7d0 msgr2=0x7ffbe00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.446+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/1085290491 shutdown_connections 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.446+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/1085290491 wait complete. 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe6fd7640 1 Processor -- start 2024-09-16T00:25:30.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe6fd7640 1 -- start start 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe6fd7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbe0102d90 0x7ffbe01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe6fd7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0103740 0x7ffbe0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe6fd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe0108290 0x7ffbe0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe6fd7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0112090 con 0x7ffbe0103740 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe6fd7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0111f10 con 0x7ffbe0108290 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe6fd7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0112210 con 0x7ffbe0102d90 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbdffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0103740 0x7ffbe0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbdffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0103740 0x7ffbe0100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33650/0 (socket says 172.21.15.1:33650) 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbdffff640 1 -- 172.21.15.1:0/2945607738 learned_addr learned my addr 172.21.15.1:0/2945607738 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe5574640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe0108290 0x7ffbe0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe4d73640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbe0102d90 0x7ffbe01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbdffff640 1 -- 172.21.15.1:0/2945607738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbe0102d90 msgr2=0x7ffbe01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbdffff640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbe0102d90 0x7ffbe01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:30.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbdffff640 1 -- 172.21.15.1:0/2945607738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe0108290 msgr2=0x7ffbe0101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbdffff640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe0108290 0x7ffbe0101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbdffff640 1 -- 172.21.15.1:0/2945607738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe01a69e0 con 0x7ffbe0103740 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbe4d73640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbe0102d90 0x7ffbe01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.447+0000 7ffbdffff640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0103740 0x7ffbe0100f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800ccb0 tx=0x7ffbc800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.448+0000 7ffbddffb640 1 -- 172.21.15.1:0/2945607738 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc800ce60 con 0x7ffbe0103740 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.448+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe01a6cd0 con 0x7ffbe0103740 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.448+0000 7ffbddffb640 1 -- 172.21.15.1:0/2945607738 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc800fb30 con 0x7ffbe0103740 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.448+0000 7ffbddffb640 1 -- 172.21.15.1:0/2945607738 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8018c30 con 0x7ffbe0103740 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.448+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe01a71b0 con 0x7ffbe0103740 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.449+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe006a7f0 con 0x7ffbe0103740 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.450+0000 7ffbddffb640 1 -- 172.21.15.1:0/2945607738 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7ffbc800f650 con 0x7ffbe0103740 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.451+0000 7ffbddffb640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbcc0a3240 0x7ffbcc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:30.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.451+0000 7ffbe4d73640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbcc0a3240 0x7ffbcc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:30.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.451+0000 7ffbddffb640 1 -- 172.21.15.1:0/2945607738 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ffbc80cf8e0 con 0x7ffbe0103740 2024-09-16T00:25:30.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.451+0000 7ffbe4d73640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbcc0a3240 0x7ffbcc0a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:30.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.455+0000 7ffbddffb640 1 -- 172.21.15.1:0/2945607738 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbc8014030 con 0x7ffbe0103740 2024-09-16T00:25:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.630+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffbe010a2a0 con 0x7ffbe0103740 2024-09-16T00:25:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.630+0000 7ffbddffb640 1 -- 172.21.15.1:0/2945607738 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7ffbc809caa0 con 0x7ffbe0103740 2024-09-16T00:25:30.631 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-09-16T00:25:30.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbcc0a3240 msgr2=0x7ffbcc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:30.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbcc0a3240 0x7ffbcc0a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd8005a20 comp rx=0 tx=0).stop 2024-09-16T00:25:30.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0103740 msgr2=0x7ffbe0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:30.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0103740 0x7ffbe0100f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800ccb0 tx=0x7ffbc800a620 comp rx=0 tx=0).stop 2024-09-16T00:25:30.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 shutdown_connections 2024-09-16T00:25:30.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe0108290 0x7ffbe0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:30.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbe0103740 0x7ffbe0100f30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:30.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbcc0a3240 0x7ffbcc0a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:30.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 --2- 172.21.15.1:0/2945607738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbe0102d90 0x7ffbe01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:30.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 >> 172.21.15.1:0/2945607738 conn(0x7ffbe00fc7d0 msgr2=0x7ffbe0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:30.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 shutdown_connections 2024-09-16T00:25:30.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:30.632+0000 7ffbe6fd7640 1 -- 172.21.15.1:0/2945607738 wait complete. 2024-09-16T00:25:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:30 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2945607738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:25:30.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:30 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2945607738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:25:31.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:30 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2945607738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:25:31.385 INFO:tasks.cephadm:Adding mds.a on smithi001 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.d on smithi001 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.g on smithi001 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.j on smithi001 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.b on smithi019 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.e on smithi019 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.h on smithi019 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.k on smithi019 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.c on smithi153 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.f on smithi153 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.i on smithi153 2024-09-16T00:25:31.386 INFO:tasks.cephadm:Adding mds.l on smithi153 2024-09-16T00:25:31.387 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l' 2024-09-16T00:25:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:25:31.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:31 smithi153 ceph-mon[35538]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:31.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:31 smithi001 ceph-mon[30934]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:32.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:31 smithi019 ceph-mon[35186]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:32.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.133+0000 7fbabf086640 1 -- 172.21.15.153:0/1883571092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 msgr2=0x7fbab8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:32.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.133+0000 7fbabf086640 1 --2- 172.21.15.153:0/1883571092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 0x7fbab8108f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009ca0 tx=0x7fbab002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.133+0000 7fbabf086640 1 -- 172.21.15.153:0/1883571092 shutdown_connections 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.133+0000 7fbabf086640 1 --2- 172.21.15.153:0/1883571092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 0x7fbab8108f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.133+0000 7fbabf086640 1 --2- 172.21.15.153:0/1883571092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8101cd0 0x7fbab8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.133+0000 7fbabf086640 1 --2- 172.21.15.153:0/1883571092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8100ad0 0x7fbab8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.133+0000 7fbabf086640 1 -- 172.21.15.153:0/1883571092 >> 172.21.15.153:0/1883571092 conn(0x7fbab80fc280 msgr2=0x7fbab80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.133+0000 7fbabf086640 1 -- 172.21.15.153:0/1883571092 shutdown_connections 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.133+0000 7fbabf086640 1 -- 172.21.15.153:0/1883571092 wait complete. 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabf086640 1 Processor -- start 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabf086640 1 -- start start 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabf086640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8100ad0 0x7fbab8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabf086640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8101cd0 0x7fbab81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabf086640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 0x7fbab819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabf086640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbab810c730 con 0x7fbab8100ad0 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabf086640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbab810c5b0 con 0x7fbab8101cd0 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabf086640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbab810c8b0 con 0x7fbab8102690 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabd623640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 0x7fbab819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabd623640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 0x7fbab819f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:56298/0 (socket says 172.21.15.153:56298) 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabd623640 1 -- 172.21.15.153:0/2297558318 learned_addr learned my addr 172.21.15.153:0/2297558318 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbab7fff640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8101cd0 0x7fbab81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.134+0000 7fbabce22640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8100ad0 0x7fbab8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbabd623640 1 -- 172.21.15.153:0/2297558318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8101cd0 msgr2=0x7fbab81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbabd623640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8101cd0 0x7fbab81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbabd623640 1 -- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8100ad0 msgr2=0x7fbab8197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:32.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbabd623640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8100ad0 0x7fbab8197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbabd623640 1 -- 172.21.15.153:0/2297558318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab819fae0 con 0x7fbab8102690 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbabd623640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 0x7fbab819f470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005e30 tx=0x7fbab0031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbab5ffb640 1 -- 172.21.15.153:0/2297558318 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0003c30 con 0x7fbab8102690 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab819fd70 con 0x7fbab8102690 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbab5ffb640 1 -- 172.21.15.153:0/2297558318 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab0003dd0 con 0x7fbab8102690 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbab5ffb640 1 -- 172.21.15.153:0/2297558318 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab002fde0 con 0x7fbab8102690 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.135+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab81a0250 con 0x7fbab8102690 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.136+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab81a0480 con 0x7fbab8102690 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.138+0000 7fbab5ffb640 1 -- 172.21.15.153:0/2297558318 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fbab0004050 con 0x7fbab8102690 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.139+0000 7fbab5ffb640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa80a3270 0x7fbaa80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.139+0000 7fbab5ffb640 1 -- 172.21.15.153:0/2297558318 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fbab00f0910 con 0x7fbab8102690 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.139+0000 7fbabce22640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa80a3270 0x7fbaa80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.140+0000 7fbabce22640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa80a3270 0x7fbaa80a5730 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005fd0 tx=0x7fbaac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:32.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.141+0000 7fbab5ffb640 1 -- 172.21.15.153:0/2297558318 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbab00bdad0 con 0x7fbab8102690 2024-09-16T00:25:32.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.323+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}) -- 0x7fbab806a730 con 0x7fbaa80a3270 2024-09-16T00:25:32.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.331+0000 7fbab5ffb640 1 -- 172.21.15.153:0/2297558318 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fbab806a730 con 0x7fbaa80a3270 2024-09-16T00:25:32.333 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mds.all update... 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa80a3270 msgr2=0x7fbaa80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa80a3270 0x7fbaa80a5730 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005fd0 tx=0x7fbaac005850 comp rx=0 tx=0).stop 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 msgr2=0x7fbab819f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 0x7fbab819f470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005e30 tx=0x7fbab0031ee0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 shutdown_connections 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa80a3270 0x7fbaa80a5730 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8102690 0x7fbab819f470 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8101cd0 0x7fbab81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 --2- 172.21.15.153:0/2297558318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8100ad0 0x7fbab8197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 >> 172.21.15.153:0/2297558318 conn(0x7fbab80fc280 msgr2=0x7fbab806a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 shutdown_connections 2024-09-16T00:25:32.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-16T00:25:32.333+0000 7fbabf086640 1 -- 172.21.15.153:0/2297558318 wait complete. 2024-09-16T00:25:33.030 DEBUG:teuthology.orchestra.run.smithi001:mds.a> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.a.service 2024-09-16T00:25:33.033 DEBUG:teuthology.orchestra.run.smithi001:mds.d> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.d.service 2024-09-16T00:25:33.078 DEBUG:teuthology.orchestra.run.smithi001:mds.g> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.g.service 2024-09-16T00:25:33.121 DEBUG:teuthology.orchestra.run.smithi001:mds.j> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.j.service 2024-09-16T00:25:33.164 DEBUG:teuthology.orchestra.run.smithi019:mds.b> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.b.service 2024-09-16T00:25:33.167 DEBUG:teuthology.orchestra.run.smithi019:mds.e> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.e.service 2024-09-16T00:25:33.213 DEBUG:teuthology.orchestra.run.smithi019:mds.h> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.h.service 2024-09-16T00:25:33.256 DEBUG:teuthology.orchestra.run.smithi019:mds.k> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.k.service 2024-09-16T00:25:33.299 DEBUG:teuthology.orchestra.run.smithi153:mds.c> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.c.service 2024-09-16T00:25:33.301 DEBUG:teuthology.orchestra.run.smithi153:mds.f> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.f.service 2024-09-16T00:25:33.349 DEBUG:teuthology.orchestra.run.smithi153:mds.i> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.i.service 2024-09-16T00:25:33.393 DEBUG:teuthology.orchestra.run.smithi153:mds.l> sudo journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.l.service 2024-09-16T00:25:33.436 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-16T00:25:33.436 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:25:33.488 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:33.489 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:33.489 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-16T00:25:33.489 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:33.489 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:33.489 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:33.489 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:33.489 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:33.489 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:33.489 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:33.490 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:33.490 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:33 smithi153 ceph-mon[35538]: Deploying daemon mds.l on smithi153 2024-09-16T00:25:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-16T00:25:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:33.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:33 smithi019 ceph-mon[35186]: Deploying daemon mds.l on smithi153 2024-09-16T00:25:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:33.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:33.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:33.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-16T00:25:33.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:33.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:33.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:33.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:33.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:33.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:33.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:33.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:33.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:33 smithi001 ceph-mon[30934]: Deploying daemon mds.l on smithi153 2024-09-16T00:25:34.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.188+0000 7f27e7f1f640 1 -- 172.21.15.1:0/1566787957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 msgr2=0x7f27e0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.188+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/1566787957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 0x7f27e0100ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009930 tx=0x7f27d002a420 comp rx=0 tx=0).stop 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.189+0000 7f27e7f1f640 1 -- 172.21.15.1:0/1566787957 shutdown_connections 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.189+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/1566787957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e0101420 0x7f27e010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.189+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/1566787957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 0x7f27e0100ee0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.189+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/1566787957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0106a80 0x7f27e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.189+0000 7f27e7f1f640 1 -- 172.21.15.1:0/1566787957 >> 172.21.15.1:0/1566787957 conn(0x7f27e00fc7a0 msgr2=0x7f27e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.189+0000 7f27e7f1f640 1 -- 172.21.15.1:0/1566787957 shutdown_connections 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.189+0000 7f27e7f1f640 1 -- 172.21.15.1:0/1566787957 wait complete. 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.189+0000 7f27e7f1f640 1 Processor -- start 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.189+0000 7f27e7f1f640 1 -- start start 2024-09-16T00:25:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e7f1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 0x7f27e0197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e7f1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0101420 0x7f27e0197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e7f1f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e0106a80 0x7f27e019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e7f1f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0110e80 con 0x7f27e0100a80 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e7f1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0110d00 con 0x7f27e0101420 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e7f1f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0111000 con 0x7f27e0106a80 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e5cbb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 0x7f27e0197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e5cbb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 0x7f27e0197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33662/0 (socket says 172.21.15.1:33662) 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e5cbb640 1 -- 172.21.15.1:0/3851440848 learned_addr learned my addr 172.21.15.1:0/3851440848 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e64bc640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e0106a80 0x7f27e019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e54ba640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0101420 0x7f27e0197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e5cbb640 1 -- 172.21.15.1:0/3851440848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e0106a80 msgr2=0x7f27e019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e5cbb640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e0106a80 0x7f27e019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e5cbb640 1 -- 172.21.15.1:0/3851440848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0101420 msgr2=0x7f27e0197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e5cbb640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0101420 0x7f27e0197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e5cbb640 1 -- 172.21.15.1:0/3851440848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e019c2f0 con 0x7f27e0100a80 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e54ba640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0101420 0x7f27e0197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.190+0000 7f27e5cbb640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 0x7f27e0197340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c890 tx=0x7f27dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.191+0000 7f27d6ffd640 1 -- 172.21.15.1:0/3851440848 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc004220 con 0x7f27e0100a80 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.191+0000 7f27d6ffd640 1 -- 172.21.15.1:0/3851440848 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc0043c0 con 0x7f27e0100a80 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.191+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e019c5e0 con 0x7f27e0100a80 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.191+0000 7f27d6ffd640 1 -- 172.21.15.1:0/3851440848 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0049e0 con 0x7f27e0100a80 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.191+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e01a3ec0 con 0x7f27e0100a80 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.192+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e0100ee0 con 0x7f27e0100a80 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.193+0000 7f27d6ffd640 1 -- 172.21.15.1:0/3851440848 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f27dc0079b0 con 0x7f27e0100a80 2024-09-16T00:25:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.194+0000 7f27d6ffd640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27cc0a3240 0x7f27cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.194+0000 7f27e54ba640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27cc0a3240 0x7f27cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.194+0000 7f27d6ffd640 1 -- 172.21.15.1:0/3851440848 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f27dc0cf4b0 con 0x7f27e0100a80 2024-09-16T00:25:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.194+0000 7f27e54ba640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27cc0a3240 0x7f27cc0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f27d0000c00 tx=0x7f27d0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.197+0000 7f27d6ffd640 1 -- 172.21.15.1:0/3851440848 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27dc014030 con 0x7f27e0100a80 2024-09-16T00:25:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.408+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27e01986a0 con 0x7f27e0100a80 2024-09-16T00:25:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.408+0000 7f27d6ffd640 1 -- 172.21.15.1:0/3851440848 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f27dc09c650 con 0x7f27e0100a80 2024-09-16T00:25:34.409 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:34.409 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":1,"btime":"2024-09-16T00:18:28:248994+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-16T00:25:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 1 2024-09-16T00:25:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27cc0a3240 msgr2=0x7f27cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27cc0a3240 0x7f27cc0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f27d0000c00 tx=0x7f27d0005a20 comp rx=0 tx=0).stop 2024-09-16T00:25:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 msgr2=0x7f27e0197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 0x7f27e0197340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c890 tx=0x7f27dc00cd50 comp rx=0 tx=0).stop 2024-09-16T00:25:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 shutdown_connections 2024-09-16T00:25:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27cc0a3240 0x7f27cc0a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e0106a80 0x7f27e019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0101420 0x7f27e0197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 --2- 172.21.15.1:0/3851440848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e0100a80 0x7f27e0197340 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 >> 172.21.15.1:0/3851440848 conn(0x7f27e00fc7a0 msgr2=0x7f27e00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 shutdown_connections 2024-09-16T00:25:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:34.411+0000 7f27e7f1f640 1 -- 172.21.15.1:0/3851440848 wait complete. 2024-09-16T00:25:34.941 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:25:34 smithi153 systemd[1]: Starting Ceph mds.l for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:25:34.941 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:25:34 smithi153 podman[64952]: 2024-09-16 00:25:34.842877567 +0000 UTC m=+0.016291498 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:25:35.198 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:25:34 smithi153 podman[64952]: 2024-09-16 00:25:34.94570321 +0000 UTC m=+0.119117142 container create b0883f0ab6220ac596755f4de666a39b6fe389ef6f8051ac06e67df98f083a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T00:25:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:35 smithi153 ceph-mon[35538]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:35 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3851440848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:35 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.153:6832/1581415887,v1:172.21.15.153:6833/1581415887] up:boot 2024-09-16T00:25:35.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:35 smithi153 ceph-mon[35538]: fsmap 1 up:standby 2024-09-16T00:25:35.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:35 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:25:35.463 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:25:35 smithi153 podman[64952]: 2024-09-16 00:25:35.198466488 +0000 UTC m=+0.371880422 container init b0883f0ab6220ac596755f4de666a39b6fe389ef6f8051ac06e67df98f083a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:25:35.463 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:25:35 smithi153 podman[64952]: 2024-09-16 00:25:35.201430194 +0000 UTC m=+0.374844128 container start b0883f0ab6220ac596755f4de666a39b6fe389ef6f8051ac06e67df98f083a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-16T00:25:35.463 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:25:35 smithi153 ceph-mds[64971]: -- 172.21.15.153:0/3162751078 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5602b22e7c20 con 0x5602b22c5000 2024-09-16T00:25:35.463 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:25:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l[64967]: starting mds.l at 2024-09-16T00:25:35.464 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:25:35 smithi153 bash[64952]: b0883f0ab6220ac596755f4de666a39b6fe389ef6f8051ac06e67df98f083a86 2024-09-16T00:25:35.464 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:25:35 smithi153 systemd[1]: Started Ceph mds.l for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:25:35.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:35 smithi019 ceph-mon[35186]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:35.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:35 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3851440848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:35.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:35 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.153:6832/1581415887,v1:172.21.15.153:6833/1581415887] up:boot 2024-09-16T00:25:35.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:35 smithi019 ceph-mon[35186]: fsmap 1 up:standby 2024-09-16T00:25:35.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:35 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:25:35.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:35 smithi001 ceph-mon[30934]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:35.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:35 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3851440848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:35.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:35 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.153:6832/1581415887,v1:172.21.15.153:6833/1581415887] up:boot 2024-09-16T00:25:35.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:35 smithi001 ceph-mon[30934]: fsmap 1 up:standby 2024-09-16T00:25:35.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:35 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:25:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:36 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:36.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:36 smithi001 ceph-mon[30934]: Deploying daemon mds.k on smithi019 2024-09-16T00:25:36.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:36.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:36.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:36.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:36.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:36.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:36 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:36.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:36 smithi153 ceph-mon[35538]: Deploying daemon mds.k on smithi019 2024-09-16T00:25:36.816 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:36.816 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:36.816 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:36.816 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:36.816 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:36.816 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:36 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:36.816 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:36 smithi019 ceph-mon[35186]: Deploying daemon mds.k on smithi019 2024-09-16T00:25:37.239 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:25:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:37.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:37 smithi001 ceph-mon[30934]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:37.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:37 smithi153 ceph-mon[35538]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:37.732 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:37 smithi019 ceph-mon[35186]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:37.733 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:25:37 smithi019 systemd[1]: Starting Ceph mds.k for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:25:37.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.978+0000 7fc07bfc4640 1 -- 172.21.15.1:0/284427658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074106f10 msgr2=0x7fc07410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:37.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.978+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/284427658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074106f10 0x7fc07410ea90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009ca0 tx=0x7fc06c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:25:37.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.978+0000 7fc07bfc4640 1 -- 172.21.15.1:0/284427658 shutdown_connections 2024-09-16T00:25:37.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.978+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/284427658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074106f10 0x7fc07410ea90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:37.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.978+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/284427658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074102790 0x7fc0741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:37.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.978+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/284427658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074101de0 0x7fc0741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:37.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.978+0000 7fc07bfc4640 1 -- 172.21.15.1:0/284427658 >> 172.21.15.1:0/284427658 conn(0x7fc0740fd8c0 msgr2=0x7fc0740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:37.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.978+0000 7fc07bfc4640 1 -- 172.21.15.1:0/284427658 shutdown_connections 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.978+0000 7fc07bfc4640 1 -- 172.21.15.1:0/284427658 wait complete. 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc07bfc4640 1 Processor -- start 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc07bfc4640 1 -- start start 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc07bfc4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074101de0 0x7fc0741983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc07bfc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074102790 0x7fc074198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc07bfc4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074106f10 0x7fc07419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc07bfc4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc074111f70 con 0x7fc074101de0 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc07bfc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc074111df0 con 0x7fc074102790 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc07bfc4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741120f0 con 0x7fc074106f10 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc079d60640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074101de0 0x7fc0741983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc079d60640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074101de0 0x7fc0741983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33676/0 (socket says 172.21.15.1:33676) 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc079d60640 1 -- 172.21.15.1:0/1782817065 learned_addr learned my addr 172.21.15.1:0/1782817065 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:37.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.979+0000 7fc07955f640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074102790 0x7fc074198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc07a561640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074106f10 0x7fc07419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc079d60640 1 -- 172.21.15.1:0/1782817065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074106f10 msgr2=0x7fc07419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc079d60640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074106f10 0x7fc07419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc079d60640 1 -- 172.21.15.1:0/1782817065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074102790 msgr2=0x7fc074198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc079d60640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074102790 0x7fc074198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc079d60640 1 -- 172.21.15.1:0/1782817065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07419d3a0 con 0x7fc074101de0 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc07955f640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074102790 0x7fc074198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc079d60640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074101de0 0x7fc0741983f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc07000e9f0 tx=0x7fc07000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc06affd640 1 -- 172.21.15.1:0/1782817065 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07000cd80 con 0x7fc074101de0 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc06affd640 1 -- 172.21.15.1:0/1782817065 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc070010910 con 0x7fc074101de0 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07419d690 con 0x7fc074101de0 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc06affd640 1 -- 172.21.15.1:0/1782817065 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070018990 con 0x7fc074101de0 2024-09-16T00:25:37.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.980+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741a4f70 con 0x7fc074101de0 2024-09-16T00:25:37.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.981+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0741021c0 con 0x7fc074101de0 2024-09-16T00:25:37.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.982+0000 7fc06affd640 1 -- 172.21.15.1:0/1782817065 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fc070010ac0 con 0x7fc074101de0 2024-09-16T00:25:37.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.983+0000 7fc06affd640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc0600a3240 0x7fc0600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:37.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.983+0000 7fc07955f640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc0600a3240 0x7fc0600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:37.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.983+0000 7fc06affd640 1 -- 172.21.15.1:0/1782817065 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fc0700cfc70 con 0x7fc074101de0 2024-09-16T00:25:37.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.984+0000 7fc07955f640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc0600a3240 0x7fc0600a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc0740fdce0 tx=0x7fc06400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:37.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:37.986+0000 7fc06affd640 1 -- 172.21.15.1:0/1782817065 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc07009ce30 con 0x7fc074101de0 2024-09-16T00:25:37.993 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:25:37 smithi019 podman[64864]: 2024-09-16 00:25:37.634757029 +0000 UTC m=+0.015865803 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:25:37.993 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:25:37 smithi019 podman[64864]: 2024-09-16 00:25:37.747940638 +0000 UTC m=+0.129049468 container create c29345750f6f0a53bc4205db9ec63ef907eb4bfbce59742d88289e8e33e565e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:25:37.993 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:25:37 smithi019 podman[64864]: 2024-09-16 00:25:37.967113709 +0000 UTC m=+0.348222474 container init c29345750f6f0a53bc4205db9ec63ef907eb4bfbce59742d88289e8e33e565e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:25:37.993 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:25:37 smithi019 podman[64864]: 2024-09-16 00:25:37.969868688 +0000 UTC m=+0.350977452 container start c29345750f6f0a53bc4205db9ec63ef907eb4bfbce59742d88289e8e33e565e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:25:37.993 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:25:37 smithi019 ceph-mds[64883]: -- 172.21.15.19:0/2210221195 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56133d3c5860 con 0x56133d3a3000 2024-09-16T00:25:38.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.211+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc074199750 con 0x7fc074101de0 2024-09-16T00:25:38.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.212+0000 7fc06affd640 1 -- 172.21.15.1:0/1782817065 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fc07009c730 con 0x7fc074101de0 2024-09-16T00:25:38.212 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:38.212 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":2,"btime":"2024-09-16T00:25:35:229586+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T00:25:38.213 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 2 2024-09-16T00:25:38.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc0600a3240 msgr2=0x7fc0600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc0600a3240 0x7fc0600a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc0740fdce0 tx=0x7fc06400b410 comp rx=0 tx=0).stop 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074101de0 msgr2=0x7fc0741983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074101de0 0x7fc0741983f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc07000e9f0 tx=0x7fc07000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 shutdown_connections 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc0600a3240 0x7fc0600a5700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074106f10 0x7fc07419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074102790 0x7fc074198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 --2- 172.21.15.1:0/1782817065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc074101de0 0x7fc0741983f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 >> 172.21.15.1:0/1782817065 conn(0x7fc0740fd8c0 msgr2=0x7fc0740feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 shutdown_connections 2024-09-16T00:25:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:38.215+0000 7fc07bfc4640 1 -- 172.21.15.1:0/1782817065 wait complete. 2024-09-16T00:25:38.357 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:25:37 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k[64879]: starting mds.k at 2024-09-16T00:25:38.357 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:25:38 smithi019 bash[64864]: c29345750f6f0a53bc4205db9ec63ef907eb4bfbce59742d88289e8e33e565e0 2024-09-16T00:25:38.357 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:25:38 smithi019 systemd[1]: Started Ceph mds.k for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:25:39.296 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:39.296 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:39.296 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:39.296 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: Deploying daemon mds.j on smithi001 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1782817065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.19:6832/1257631294,v1:172.21.15.19:6833/1257631294] up:boot 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: fsmap 2 up:standby 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:39 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:25:39.390 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:39.390 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:39.390 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:39.390 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:39.390 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:39.390 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:39.391 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: Deploying daemon mds.j on smithi001 2024-09-16T00:25:39.391 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1782817065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:39.391 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.19:6832/1257631294,v1:172.21.15.19:6833/1257631294] up:boot 2024-09-16T00:25:39.391 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: fsmap 2 up:standby 2024-09-16T00:25:39.391 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:39 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: Deploying daemon mds.j on smithi001 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1782817065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.19:6832/1257631294,v1:172.21.15.19:6833/1257631294] up:boot 2024-09-16T00:25:39.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: fsmap 2 up:standby 2024-09-16T00:25:39.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:39 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:25:40.377 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:40 smithi019 ceph-mon[35186]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:40.405 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:40 smithi001 ceph-mon[30934]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:40 smithi153 ceph-mon[35538]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:41.068 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:25:41.093 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:25:40 smithi001 systemd[1]: Starting Ceph mds.j for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:25:41.361 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:25:41 smithi001 podman[66889]: 2024-09-16 00:25:40.999670078 +0000 UTC m=+0.016142959 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:25:41.361 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:25:41 smithi001 podman[66889]: 2024-09-16 00:25:41.11568239 +0000 UTC m=+0.132155248 container create 5c3cdfa00a055977fae6e05b15119355a2f90829658651c1d7ef59b6f1504487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T00:25:41.361 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:25:41 smithi001 podman[66889]: 2024-09-16 00:25:41.360663584 +0000 UTC m=+0.377136443 container init 5c3cdfa00a055977fae6e05b15119355a2f90829658651c1d7ef59b6f1504487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T00:25:41.568 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:41.692 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:25:41 smithi001 podman[66889]: 2024-09-16 00:25:41.364037769 +0000 UTC m=+0.380510630 container start 5c3cdfa00a055977fae6e05b15119355a2f90829658651c1d7ef59b6f1504487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:25:41.692 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:25:41 smithi001 ceph-mds[66939]: -- 172.21.15.1:0/1310759071 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560079311860 con 0x5600792ef400 2024-09-16T00:25:41.692 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:25:41 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-j[66923]: starting mds.j at 2024-09-16T00:25:41.692 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:25:41 smithi001 bash[66889]: 5c3cdfa00a055977fae6e05b15119355a2f90829658651c1d7ef59b6f1504487 2024-09-16T00:25:41.693 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:25:41 smithi001 systemd[1]: Started Ceph mds.j for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 -- 172.21.15.1:0/3292010022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c0101380 msgr2=0x7ff2c010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 --2- 172.21.15.1:0/3292010022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c0101380 0x7ff2c010dfb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009900 tx=0x7ff2b402a410 comp rx=0 tx=0).stop 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 -- 172.21.15.1:0/3292010022 shutdown_connections 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 --2- 172.21.15.1:0/3292010022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c010e4f0 0x7ff2c0110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 --2- 172.21.15.1:0/3292010022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c0101380 0x7ff2c010dfb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 --2- 172.21.15.1:0/3292010022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c01009d0 0x7ff2c0100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 -- 172.21.15.1:0/3292010022 >> 172.21.15.1:0/3292010022 conn(0x7ff2c00fc7d0 msgr2=0x7ff2c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 -- 172.21.15.1:0/3292010022 shutdown_connections 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 -- 172.21.15.1:0/3292010022 wait complete. 2024-09-16T00:25:42.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 Processor -- start 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.074+0000 7ff2c815d640 1 -- start start 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c815d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c01009d0 0x7ff2c019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c815d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0101380 0x7ff2c019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c815d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c010e4f0 0x7ff2c019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c815d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0114240 con 0x7ff2c010e4f0 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c815d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01140c0 con 0x7ff2c0101380 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c815d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01143c0 con 0x7ff2c01009d0 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c66fa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c010e4f0 0x7ff2c019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c66fa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c010e4f0 0x7ff2c019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56140/0 (socket says 172.21.15.1:56140) 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c66fa640 1 -- 172.21.15.1:0/1640922137 learned_addr learned my addr 172.21.15.1:0/1640922137 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c5ef9640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c01009d0 0x7ff2c019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c56f8640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0101380 0x7ff2c019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c66fa640 1 -- 172.21.15.1:0/1640922137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c01009d0 msgr2=0x7ff2c019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c66fa640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c01009d0 0x7ff2c019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c66fa640 1 -- 172.21.15.1:0/1640922137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0101380 msgr2=0x7ff2c019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c66fa640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0101380 0x7ff2c019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c66fa640 1 -- 172.21.15.1:0/1640922137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c019f6f0 con 0x7ff2c010e4f0 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c5ef9640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c01009d0 0x7ff2c019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.075+0000 7ff2c66fa640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c010e4f0 0x7ff2c019f010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800c7a0 tx=0x7ff2b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.076+0000 7ff2beffd640 1 -- 172.21.15.1:0/1640922137 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8007bb0 con 0x7ff2c010e4f0 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.076+0000 7ff2beffd640 1 -- 172.21.15.1:0/1640922137 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b8007d50 con 0x7ff2c010e4f0 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.076+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c019f9e0 con 0x7ff2c010e4f0 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.076+0000 7ff2beffd640 1 -- 172.21.15.1:0/1640922137 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800f450 con 0x7ff2c010e4f0 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.076+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c01a72c0 con 0x7ff2c010e4f0 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.077+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c0100db0 con 0x7ff2c010e4f0 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.078+0000 7ff2beffd640 1 -- 172.21.15.1:0/1640922137 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7ff2b8004f40 con 0x7ff2c010e4f0 2024-09-16T00:25:42.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.078+0000 7ff2beffd640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2a80a3240 0x7ff2a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.079+0000 7ff2c5ef9640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2a80a3240 0x7ff2a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.079+0000 7ff2beffd640 1 -- 172.21.15.1:0/1640922137 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ff2b80cf1f0 con 0x7ff2c010e4f0 2024-09-16T00:25:42.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.079+0000 7ff2c5ef9640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2a80a3240 0x7ff2a80a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005c40 tx=0x7ff2b000a430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:42.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.082+0000 7ff2beffd640 1 -- 172.21.15.1:0/1640922137 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2b8014030 con 0x7ff2c010e4f0 2024-09-16T00:25:42.245 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:42 smithi001 ceph-mon[30934]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:42.245 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:42 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:42.245 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:42 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:42.245 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:42 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:42.245 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:42 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:42.245 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:42 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:42.245 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:42 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:42.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.301+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2c0109680 con 0x7ff2c010e4f0 2024-09-16T00:25:42.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.302+0000 7ff2beffd640 1 -- 172.21.15.1:0/1640922137 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7ff2b809c310 con 0x7ff2c010e4f0 2024-09-16T00:25:42.302 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:42.302 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":4,"btime":"2024-09-16T00:25:42:127432+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T00:25:42.303 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 4 2024-09-16T00:25:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2a80a3240 msgr2=0x7ff2a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2a80a3240 0x7ff2a80a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005c40 tx=0x7ff2b000a430 comp rx=0 tx=0).stop 2024-09-16T00:25:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c010e4f0 msgr2=0x7ff2c019f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c010e4f0 0x7ff2c019f010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800c7a0 tx=0x7ff2b800cc60 comp rx=0 tx=0).stop 2024-09-16T00:25:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 shutdown_connections 2024-09-16T00:25:42.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c010e4f0 0x7ff2c019f010 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0101380 0x7ff2c019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2a80a3240 0x7ff2a80a5700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 --2- 172.21.15.1:0/1640922137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c01009d0 0x7ff2c019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 >> 172.21.15.1:0/1640922137 conn(0x7ff2c00fc7d0 msgr2=0x7ff2c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:42.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 shutdown_connections 2024-09-16T00:25:42.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:42.304+0000 7ff2c815d640 1 -- 172.21.15.1:0/1640922137 wait complete. 2024-09-16T00:25:42.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:42 smithi153 ceph-mon[35538]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:42.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:42 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:42.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:42 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:42.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:42 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:42.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:42 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:42.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:42 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:42.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:42 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:42 smithi019 ceph-mon[35186]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:42 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:42 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:42 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:42 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:42.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:42 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:42.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:42 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:43.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:43 smithi001 ceph-mon[30934]: Deploying daemon mds.d on smithi001 2024-09-16T00:25:43.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:43 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.1:6834/2811424948,v1:172.21.15.1:6835/2811424948] up:boot 2024-09-16T00:25:43.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:43 smithi001 ceph-mon[30934]: fsmap 3 up:standby 2024-09-16T00:25:43.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:43 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:25:43.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:43 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1640922137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:43.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:43 smithi153 ceph-mon[35538]: Deploying daemon mds.d on smithi001 2024-09-16T00:25:43.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:43 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.1:6834/2811424948,v1:172.21.15.1:6835/2811424948] up:boot 2024-09-16T00:25:43.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:43 smithi153 ceph-mon[35538]: fsmap 3 up:standby 2024-09-16T00:25:43.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:43 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:25:43.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:43 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1640922137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:43.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:43 smithi019 ceph-mon[35186]: Deploying daemon mds.d on smithi001 2024-09-16T00:25:43.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:43 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.1:6834/2811424948,v1:172.21.15.1:6835/2811424948] up:boot 2024-09-16T00:25:43.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:43 smithi019 ceph-mon[35186]: fsmap 3 up:standby 2024-09-16T00:25:43.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:43 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:25:43.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:43 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1640922137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:44.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:44 smithi001 ceph-mon[30934]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:44 smithi153 ceph-mon[35538]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:44 smithi019 ceph-mon[35186]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:44.701 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:25:44 smithi001 systemd[1]: Starting Ceph mds.d for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:25:45.128 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:25:44 smithi001 podman[67307]: 2024-09-16 00:25:44.744529926 +0000 UTC m=+0.016995881 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:25:45.128 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:25:44 smithi001 podman[67307]: 2024-09-16 00:25:44.866912297 +0000 UTC m=+0.139378298 container create d05f3acf2a8b54aba002251e25b41e72a8372e9475d57043503e811ff519cf57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T00:25:45.128 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:25:45 smithi001 podman[67307]: 2024-09-16 00:25:45.127855335 +0000 UTC m=+0.400321290 container init d05f3acf2a8b54aba002251e25b41e72a8372e9475d57043503e811ff519cf57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T00:25:45.451 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:25:45 smithi001 podman[67307]: 2024-09-16 00:25:45.13135508 +0000 UTC m=+0.403821039 container start d05f3acf2a8b54aba002251e25b41e72a8372e9475d57043503e811ff519cf57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T00:25:45.451 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:25:45 smithi001 ceph-mds[67326]: -- 172.21.15.1:0/381546759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a62f9b860 con 0x559a62f79400 2024-09-16T00:25:45.451 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:25:45 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-d[67322]: starting mds.d at 2024-09-16T00:25:45.451 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:25:45 smithi001 bash[67307]: d05f3acf2a8b54aba002251e25b41e72a8372e9475d57043503e811ff519cf57 2024-09-16T00:25:45.452 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:25:45 smithi001 systemd[1]: Started Ceph mds.d for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:25:45.681 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:25:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:46 smithi019 ceph-mon[35186]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:46 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:46.358 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:46 smithi001 ceph-mon[30934]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:46.358 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:46.358 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:46.358 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:46.358 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:46.358 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:46.358 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:46 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:46.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:46 smithi153 ceph-mon[35538]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:46.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:46.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:46.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:46.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:46.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:46.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:46 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:46.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.443+0000 7f72cc322640 1 -- 172.21.15.1:0/1494878306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 msgr2=0x7f72c410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:46.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.443+0000 7f72cc322640 1 --2- 172.21.15.1:0/1494878306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 0x7f72c410da40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009ca0 tx=0x7f72bc02a950 comp rx=0 tx=0).stop 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.443+0000 7f72cc322640 1 -- 172.21.15.1:0/1494878306 shutdown_connections 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.443+0000 7f72cc322640 1 --2- 172.21.15.1:0/1494878306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 0x7f72c410da40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.443+0000 7f72cc322640 1 --2- 172.21.15.1:0/1494878306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72c4102570 0x7f72c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.443+0000 7f72cc322640 1 --2- 172.21.15.1:0/1494878306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c4100e20 0x7f72c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.443+0000 7f72cc322640 1 -- 172.21.15.1:0/1494878306 >> 172.21.15.1:0/1494878306 conn(0x7f72c40fc7d0 msgr2=0x7f72c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.443+0000 7f72cc322640 1 -- 172.21.15.1:0/1494878306 shutdown_connections 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.443+0000 7f72cc322640 1 -- 172.21.15.1:0/1494878306 wait complete. 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72cc322640 1 Processor -- start 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72cc322640 1 -- start start 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72cc322640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72c4100e20 0x7f72c40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72cc322640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c4102570 0x7f72c40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72cc322640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 0x7f72c40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72cc322640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4110f80 con 0x7f72c4102ef0 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72cc322640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4110e00 con 0x7f72c4102570 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72cc322640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4111100 con 0x7f72c4100e20 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72ca8bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 0x7f72c40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72ca8bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 0x7f72c40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56172/0 (socket says 172.21.15.1:56172) 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72ca8bf640 1 -- 172.21.15.1:0/3679532073 learned_addr learned my addr 172.21.15.1:0/3679532073 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72ca0be640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72c4100e20 0x7f72c40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72c98bd640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c4102570 0x7f72c40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72ca8bf640 1 -- 172.21.15.1:0/3679532073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72c4100e20 msgr2=0x7f72c40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72ca8bf640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72c4100e20 0x7f72c40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72ca8bf640 1 -- 172.21.15.1:0/3679532073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c4102570 msgr2=0x7f72c40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72ca8bf640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c4102570 0x7f72c40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.444+0000 7f72ca8bf640 1 -- 172.21.15.1:0/3679532073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c41a6950 con 0x7f72c4102ef0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.445+0000 7f72ca8bf640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 0x7f72c40fd700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f72bc004af0 tx=0x7f72bc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.445+0000 7f72b77fe640 1 -- 172.21.15.1:0/3679532073 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc034040 con 0x7f72c4102ef0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.445+0000 7f72b77fe640 1 -- 172.21.15.1:0/3679532073 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72bc030920 con 0x7f72c4102ef0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.445+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c41a6be0 con 0x7f72c4102ef0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.445+0000 7f72b77fe640 1 -- 172.21.15.1:0/3679532073 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc004740 con 0x7f72c4102ef0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.445+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c41a70c0 con 0x7f72c4102ef0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.446+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c4101200 con 0x7f72c4102ef0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.447+0000 7f72b77fe640 1 -- 172.21.15.1:0/3679532073 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f72bc037020 con 0x7f72c4102ef0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.448+0000 7f72b77fe640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f72b00a3240 0x7f72b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.448+0000 7f72ca0be640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f72b00a3240 0x7f72b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.448+0000 7f72b77fe640 1 -- 172.21.15.1:0/3679532073 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f72bc0f0350 con 0x7f72c4102ef0 2024-09-16T00:25:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.448+0000 7f72ca0be640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f72b00a3240 0x7f72b00a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f72b800abe0 tx=0x7f72b800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.455+0000 7f72b77fe640 1 -- 172.21.15.1:0/3679532073 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72bc0811b0 con 0x7f72c4102ef0 2024-09-16T00:25:46.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.669+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72c410a0a0 con 0x7f72c4102ef0 2024-09-16T00:25:46.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.669+0000 7f72b77fe640 1 -- 172.21.15.1:0/3679532073 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3904 (secure 0 0 0) 0x7f72bc0bd510 con 0x7f72c4102ef0 2024-09-16T00:25:46.670 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:46.670 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"btime":"2024-09-16T00:25:46:140431+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T00:25:46.670 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 5 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f72b00a3240 msgr2=0x7f72b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f72b00a3240 0x7f72b00a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f72b800abe0 tx=0x7f72b800a2d0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 msgr2=0x7f72c40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 0x7f72c40fd700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f72bc004af0 tx=0x7f72bc02f5b0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 shutdown_connections 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4102ef0 0x7f72c40fd700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c4102570 0x7f72c40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f72b00a3240 0x7f72b00a5700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 --2- 172.21.15.1:0/3679532073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72c4100e20 0x7f72c40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 >> 172.21.15.1:0/3679532073 conn(0x7f72c40fc7d0 msgr2=0x7f72c406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.672+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 shutdown_connections 2024-09-16T00:25:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:46.673+0000 7f72cc322640 1 -- 172.21.15.1:0/3679532073 wait complete. 2024-09-16T00:25:47.357 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:25:47 smithi019 systemd[1]: Starting Ceph mds.e for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:25:47.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:47 smithi019 ceph-mon[35186]: Deploying daemon mds.e on smithi019 2024-09-16T00:25:47.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:47 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.1:6836/436431990,v1:172.21.15.1:6837/436431990] up:boot 2024-09-16T00:25:47.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:47 smithi019 ceph-mon[35186]: fsmap 4 up:standby 2024-09-16T00:25:47.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:47 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:25:47.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:47 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3679532073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:47.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:47 smithi001 ceph-mon[30934]: Deploying daemon mds.e on smithi019 2024-09-16T00:25:47.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:47 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.1:6836/436431990,v1:172.21.15.1:6837/436431990] up:boot 2024-09-16T00:25:47.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:47 smithi001 ceph-mon[30934]: fsmap 4 up:standby 2024-09-16T00:25:47.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:47 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:25:47.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:47 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3679532073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:47.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:47 smithi153 ceph-mon[35538]: Deploying daemon mds.e on smithi019 2024-09-16T00:25:47.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:47 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.1:6836/436431990,v1:172.21.15.1:6837/436431990] up:boot 2024-09-16T00:25:47.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:47 smithi153 ceph-mon[35538]: fsmap 4 up:standby 2024-09-16T00:25:47.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:47 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:25:47.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:47 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3679532073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:47.731 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:25:47 smithi019 podman[65179]: 2024-09-16 00:25:47.460866459 +0000 UTC m=+0.102835429 container create c8d7c23eff5391ddd0be3c85cc8a0a8a0aab748fa89418d36ea061c129f63c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:25:47.731 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:25:47 smithi019 podman[65179]: 2024-09-16 00:25:47.374210842 +0000 UTC m=+0.016179762 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:25:47.731 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:25:47 smithi019 podman[65179]: 2024-09-16 00:25:47.705174763 +0000 UTC m=+0.347143680 container init c8d7c23eff5391ddd0be3c85cc8a0a8a0aab748fa89418d36ea061c129f63c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-16T00:25:47.731 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:25:47 smithi019 podman[65179]: 2024-09-16 00:25:47.708009193 +0000 UTC m=+0.349978111 container start c8d7c23eff5391ddd0be3c85cc8a0a8a0aab748fa89418d36ea061c129f63c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:25:48.107 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:25:47 smithi019 ceph-mds[65198]: -- 172.21.15.19:0/3028790046 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e15b97860 con 0x559e15b74800 2024-09-16T00:25:48.107 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:25:47 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e[65194]: starting mds.e at 2024-09-16T00:25:48.107 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:25:47 smithi019 bash[65179]: c8d7c23eff5391ddd0be3c85cc8a0a8a0aab748fa89418d36ea061c129f63c1f 2024-09-16T00:25:48.107 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:25:47 smithi019 systemd[1]: Started Ceph mds.e for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:25:48.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:48 smithi153 ceph-mon[35538]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:48.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:48 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:48.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:48 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:48.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:48 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:48.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:48 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:48.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:48 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:48.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:48 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:48 smithi001 ceph-mon[30934]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:48.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:48 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:48.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:48 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:48.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:48 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:48.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:48 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:48.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:48 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:48.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:48 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:48 smithi019 ceph-mon[35186]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:48 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:48 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:48 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:48 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:48 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:48 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:49.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:49 smithi153 ceph-mon[35538]: Deploying daemon mds.f on smithi153 2024-09-16T00:25:49.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:49 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.19:6834/389176790,v1:172.21.15.19:6835/389176790] up:boot 2024-09-16T00:25:49.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:49 smithi153 ceph-mon[35538]: fsmap 5 up:standby 2024-09-16T00:25:49.424 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:49 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:25:49.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:49 smithi001 ceph-mon[30934]: Deploying daemon mds.f on smithi153 2024-09-16T00:25:49.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:49 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.19:6834/389176790,v1:172.21.15.19:6835/389176790] up:boot 2024-09-16T00:25:49.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:49 smithi001 ceph-mon[30934]: fsmap 5 up:standby 2024-09-16T00:25:49.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:49 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:25:49.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:25:49.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:49 smithi019 ceph-mon[35186]: Deploying daemon mds.f on smithi153 2024-09-16T00:25:49.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:49 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.19:6834/389176790,v1:172.21.15.19:6835/389176790] up:boot 2024-09-16T00:25:49.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:49 smithi019 ceph-mon[35186]: fsmap 5 up:standby 2024-09-16T00:25:49.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:49 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:25:49.759 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:50.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 -- 172.21.15.1:0/3309916935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680101de0 msgr2=0x7f56801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:50.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 --2- 172.21.15.1:0/3309916935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680101de0 0x7f56801021c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f567c009900 tx=0x7f567c02a410 comp rx=0 tx=0).stop 2024-09-16T00:25:50.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 -- 172.21.15.1:0/3309916935 shutdown_connections 2024-09-16T00:25:50.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 --2- 172.21.15.1:0/3309916935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5680106f10 0x7f568010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 --2- 172.21.15.1:0/3309916935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5680102790 0x7f56801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 --2- 172.21.15.1:0/3309916935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680101de0 0x7f56801021c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 -- 172.21.15.1:0/3309916935 >> 172.21.15.1:0/3309916935 conn(0x7f56800fd8c0 msgr2=0x7f56800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:50.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 -- 172.21.15.1:0/3309916935 shutdown_connections 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 -- 172.21.15.1:0/3309916935 wait complete. 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.299+0000 7f56886e1640 1 Processor -- start 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f56886e1640 1 -- start start 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f56886e1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5680101de0 0x7f56800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f56886e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5680102790 0x7f5680074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f56886e1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680106f10 0x7f5680074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f56886e1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5680111f70 con 0x7f5680106f10 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f56886e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5680111df0 con 0x7f5680102790 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f56886e1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56801120f0 con 0x7f5680101de0 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f5686c7e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680106f10 0x7f5680074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f5686c7e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680106f10 0x7f5680074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46850/0 (socket says 172.21.15.1:46850) 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f5686c7e640 1 -- 172.21.15.1:0/1761440388 learned_addr learned my addr 172.21.15.1:0/1761440388 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:50.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f568647d640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5680101de0 0x7f56800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f5685c7c640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5680102790 0x7f5680074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f5686c7e640 1 -- 172.21.15.1:0/1761440388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5680101de0 msgr2=0x7f56800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f5686c7e640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5680101de0 0x7f56800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f5686c7e640 1 -- 172.21.15.1:0/1761440388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5680102790 msgr2=0x7f5680074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f5686c7e640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5680102790 0x7f5680074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.300+0000 7f5686c7e640 1 -- 172.21.15.1:0/1761440388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56801a68f0 con 0x7f5680106f10 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.301+0000 7f5686c7e640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680106f10 0x7f5680074f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f567800eb10 tx=0x7f567800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.301+0000 7f56737fe640 1 -- 172.21.15.1:0/1761440388 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678004b20 con 0x7f5680106f10 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.301+0000 7f56737fe640 1 -- 172.21.15.1:0/1761440388 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5678004cc0 con 0x7f5680106f10 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.301+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56801a6be0 con 0x7f5680106f10 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.301+0000 7f56737fe640 1 -- 172.21.15.1:0/1761440388 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678010760 con 0x7f5680106f10 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.301+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56801a7120 con 0x7f5680106f10 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.302+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56801021c0 con 0x7f5680106f10 2024-09-16T00:25:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.303+0000 7f56737fe640 1 -- 172.21.15.1:0/1761440388 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f5678005020 con 0x7f5680106f10 2024-09-16T00:25:50.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.304+0000 7f56737fe640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f566c0a3240 0x7f566c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:50.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.304+0000 7f568647d640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f566c0a3240 0x7f566c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:50.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.304+0000 7f56737fe640 1 -- 172.21.15.1:0/1761440388 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f56780cfaa0 con 0x7f5680106f10 2024-09-16T00:25:50.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.305+0000 7f568647d640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f566c0a3240 0x7f566c0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f567c009230 tx=0x7f567c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:50.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.308+0000 7f56737fe640 1 -- 172.21.15.1:0/1761440388 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f567809cc60 con 0x7f5680106f10 2024-09-16T00:25:50.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:50 smithi153 ceph-mon[35538]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:50.430 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:25:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f[65267]: starting mds.f at 2024-09-16T00:25:50.445 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:50 smithi019 ceph-mon[35186]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:50.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:50 smithi001 ceph-mon[30934]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.524+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f568010a0f0 con 0x7f5680106f10 2024-09-16T00:25:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.524+0000 7f56737fe640 1 -- 172.21.15.1:0/1761440388 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4726 (secure 0 0 0) 0x7f567809c560 con 0x7f5680106f10 2024-09-16T00:25:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:50.525 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":6,"btime":"2024-09-16T00:25:48:149859+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-16T00:25:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 6 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.526+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f566c0a3240 msgr2=0x7f566c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.526+0000 7f56886e1640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f566c0a3240 0x7f566c0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f567c009230 tx=0x7f567c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.526+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680106f10 msgr2=0x7f5680074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.526+0000 7f56886e1640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680106f10 0x7f5680074f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f567800eb10 tx=0x7f567800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.526+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 shutdown_connections 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.526+0000 7f56886e1640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5680106f10 0x7f5680074f70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.527+0000 7f56886e1640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5680102790 0x7f5680074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.527+0000 7f56886e1640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f566c0a3240 0x7f566c0a5700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.527+0000 7f56886e1640 1 --2- 172.21.15.1:0/1761440388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5680101de0 0x7f56800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.527+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 >> 172.21.15.1:0/1761440388 conn(0x7f56800fd8c0 msgr2=0x7f56800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.527+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 shutdown_connections 2024-09-16T00:25:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:50.527+0000 7f56886e1640 1 -- 172.21.15.1:0/1761440388 wait complete. 2024-09-16T00:25:50.711 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:25:50 smithi153 bash[65252]: 6a5b145e9c1f0c66d6ef359ee6cee1ed7755fa9555725ff3783649b39bdcd086 2024-09-16T00:25:50.711 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:25:50 smithi153 systemd[1]: Started Ceph mds.f for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:25:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:51.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: Deploying daemon mds.c on smithi153 2024-09-16T00:25:51.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1761440388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:51.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.153:6834/3078906320,v1:172.21.15.153:6835/3078906320] up:boot 2024-09-16T00:25:51.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: fsmap 6 up:standby 2024-09-16T00:25:51.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:51 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:25:51.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:51.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:51.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:51.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:51.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:51.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:51.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:51.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: Deploying daemon mds.c on smithi153 2024-09-16T00:25:51.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1761440388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:51.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.153:6834/3078906320,v1:172.21.15.153:6835/3078906320] up:boot 2024-09-16T00:25:51.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: fsmap 6 up:standby 2024-09-16T00:25:51.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:51 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:25:51.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:51.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:51.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:51.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:51.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:51.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:51.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:51.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: Deploying daemon mds.c on smithi153 2024-09-16T00:25:51.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1761440388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:51.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.153:6834/3078906320,v1:172.21.15.153:6835/3078906320] up:boot 2024-09-16T00:25:51.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: fsmap 6 up:standby 2024-09-16T00:25:51.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:51 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:25:52.657 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:25:52 smithi153 systemd[1]: Starting Ceph mds.c for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:25:52.962 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:25:52 smithi153 podman[65557]: 2024-09-16 00:25:52.560259646 +0000 UTC m=+0.016625323 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:25:52.962 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:25:52 smithi153 podman[65557]: 2024-09-16 00:25:52.674666353 +0000 UTC m=+0.131032068 container create 3e37f8eed2c9fd4523be2e150069d7ceb8d772b11c5e285da8a64ebd33be783f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T00:25:53.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:25:53.521 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:53.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:53 smithi001 ceph-mon[30934]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:53.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:53 smithi153 ceph-mon[35538]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:53.713 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:25:53 smithi153 podman[65557]: 2024-09-16 00:25:53.35588421 +0000 UTC m=+0.812249882 container init 3e37f8eed2c9fd4523be2e150069d7ceb8d772b11c5e285da8a64ebd33be783f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:25:53.713 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:25:53 smithi153 podman[65557]: 2024-09-16 00:25:53.359596196 +0000 UTC m=+0.815961868 container start 3e37f8eed2c9fd4523be2e150069d7ceb8d772b11c5e285da8a64ebd33be783f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:25:53.713 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:25:53 smithi153 ceph-mds[65576]: -- 172.21.15.153:0/194934759 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4dc7c5860 con 0x55f4dc7a3400 2024-09-16T00:25:53.713 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:25:53 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c[65572]: starting mds.c at 2024-09-16T00:25:53.713 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:25:53 smithi153 bash[65557]: 3e37f8eed2c9fd4523be2e150069d7ceb8d772b11c5e285da8a64ebd33be783f 2024-09-16T00:25:53.714 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:25:53 smithi153 systemd[1]: Started Ceph mds.c for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:25:53.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:53 smithi019 ceph-mon[35186]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.084+0000 7f35e4e17640 1 -- 172.21.15.1:0/2468260432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0106f10 msgr2=0x7f35e010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.084+0000 7f35e4e17640 1 --2- 172.21.15.1:0/2468260432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0106f10 0x7f35e010ea90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009ca0 tx=0x7f35d802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.084+0000 7f35e4e17640 1 -- 172.21.15.1:0/2468260432 shutdown_connections 2024-09-16T00:25:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.084+0000 7f35e4e17640 1 --2- 172.21.15.1:0/2468260432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0106f10 0x7f35e010ea90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.084+0000 7f35e4e17640 1 --2- 172.21.15.1:0/2468260432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e0102790 0x7f35e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.084+0000 7f35e4e17640 1 --2- 172.21.15.1:0/2468260432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e0101de0 0x7f35e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.084+0000 7f35e4e17640 1 -- 172.21.15.1:0/2468260432 >> 172.21.15.1:0/2468260432 conn(0x7f35e00fd8c0 msgr2=0x7f35e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.084+0000 7f35e4e17640 1 -- 172.21.15.1:0/2468260432 shutdown_connections 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.084+0000 7f35e4e17640 1 -- 172.21.15.1:0/2468260432 wait complete. 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35e4e17640 1 Processor -- start 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35e4e17640 1 -- start start 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35e4e17640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e0101de0 0x7f35e0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35e4e17640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0102790 0x7f35e0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35e4e17640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e0106f10 0x7f35e019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35e4e17640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0111f70 con 0x7f35e0102790 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35e4e17640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0111df0 con 0x7f35e0106f10 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35e4e17640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01120f0 con 0x7f35e0101de0 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35ddd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0102790 0x7f35e0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35ddd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0102790 0x7f35e0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46872/0 (socket says 172.21.15.1:46872) 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35ddd9b640 1 -- 172.21.15.1:0/1440861370 learned_addr learned my addr 172.21.15.1:0/1440861370 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35ded9d640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e0106f10 0x7f35e019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.085+0000 7f35de59c640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e0101de0 0x7f35e0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35ddd9b640 1 -- 172.21.15.1:0/1440861370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e0101de0 msgr2=0x7f35e0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35ddd9b640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e0101de0 0x7f35e0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35ddd9b640 1 -- 172.21.15.1:0/1440861370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e0106f10 msgr2=0x7f35e019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35ddd9b640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e0106f10 0x7f35e019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35ddd9b640 1 -- 172.21.15.1:0/1440861370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e019d2e0 con 0x7f35e0102790 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35de59c640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e0101de0 0x7f35e0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35ddd9b640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0102790 0x7f35e0198870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c970 tx=0x7f35d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35cf7fe640 1 -- 172.21.15.1:0/1440861370 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4007bb0 con 0x7f35e0102790 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35cf7fe640 1 -- 172.21.15.1:0/1440861370 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d4007d50 con 0x7f35e0102790 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e019d5d0 con 0x7f35e0102790 2024-09-16T00:25:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35cf7fe640 1 -- 172.21.15.1:0/1440861370 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d400f450 con 0x7f35e0102790 2024-09-16T00:25:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.086+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e01a4eb0 con 0x7f35e0102790 2024-09-16T00:25:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.087+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e01021c0 con 0x7f35e0102790 2024-09-16T00:25:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.088+0000 7f35cf7fe640 1 -- 172.21.15.1:0/1440861370 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f35d4004fe0 con 0x7f35e0102790 2024-09-16T00:25:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.089+0000 7f35cf7fe640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35c80a3240 0x7f35c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.089+0000 7f35de59c640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35c80a3240 0x7f35c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.089+0000 7f35cf7fe640 1 -- 172.21.15.1:0/1440861370 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f35d40cf6a0 con 0x7f35e0102790 2024-09-16T00:25:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.089+0000 7f35de59c640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35c80a3240 0x7f35c80a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f35d000abe0 tx=0x7f35d000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.092+0000 7f35cf7fe640 1 -- 172.21.15.1:0/1440861370 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35d4014030 con 0x7f35e0102790 2024-09-16T00:25:54.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.305+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35e002d7c0 con 0x7f35e0102790 2024-09-16T00:25:54.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.305+0000 7f35cf7fe640 1 -- 172.21.15.1:0/1440861370 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6382 (secure 0 0 0) 0x7f35d409c7c0 con 0x7f35e0102790 2024-09-16T00:25:54.306 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:54.306 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":8,"btime":"2024-09-16T00:25:53:482404+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-16T00:25:54.306 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 8 2024-09-16T00:25:54.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35c80a3240 msgr2=0x7f35c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:54.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35c80a3240 0x7f35c80a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f35d000abe0 tx=0x7f35d000a5c0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0102790 msgr2=0x7f35e0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:54.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0102790 0x7f35e0198870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c970 tx=0x7f35d400ce30 comp rx=0 tx=0).stop 2024-09-16T00:25:54.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 shutdown_connections 2024-09-16T00:25:54.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e0106f10 0x7f35e019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e0102790 0x7f35e0198870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35c80a3240 0x7f35c80a5700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 --2- 172.21.15.1:0/1440861370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e0101de0 0x7f35e0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:54.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 >> 172.21.15.1:0/1440861370 conn(0x7f35e00fd8c0 msgr2=0x7f35e00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:54.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 shutdown_connections 2024-09-16T00:25:54.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:54.308+0000 7f35e4e17640 1 -- 172.21.15.1:0/1440861370 wait complete. 2024-09-16T00:25:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.153:6836/3490951208,v1:172.21.15.153:6837/3490951208] up:boot 2024-09-16T00:25:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: fsmap 7 up:standby 2024-09-16T00:25:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:25:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:54.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:54.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: Deploying daemon mds.h on smithi019 2024-09-16T00:25:54.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:54 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1440861370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.153:6836/3490951208,v1:172.21.15.153:6837/3490951208] up:boot 2024-09-16T00:25:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: fsmap 7 up:standby 2024-09-16T00:25:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:25:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: Deploying daemon mds.h on smithi019 2024-09-16T00:25:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:54 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1440861370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:54.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.153:6836/3490951208,v1:172.21.15.153:6837/3490951208] up:boot 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: fsmap 7 up:standby 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: Deploying daemon mds.h on smithi019 2024-09-16T00:25:54.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:54 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1440861370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:55.838 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:55 smithi019 ceph-mon[35186]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:55.838 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:25:55 smithi019 systemd[1]: Starting Ceph mds.h for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:25:55.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:55 smithi001 ceph-mon[30934]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:55.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:55 smithi153 ceph-mon[35538]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:56.107 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:25:55 smithi019 podman[65480]: 2024-09-16 00:25:55.740810036 +0000 UTC m=+0.016292360 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:25:56.107 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:25:55 smithi019 podman[65480]: 2024-09-16 00:25:55.865892822 +0000 UTC m=+0.141375170 container create ce175bee792691bc37017b7196f794a352253e9553d771ae90e83f166bbeec05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:25:56.607 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:25:56 smithi019 podman[65480]: 2024-09-16 00:25:56.110242361 +0000 UTC m=+0.385724677 container init ce175bee792691bc37017b7196f794a352253e9553d771ae90e83f166bbeec05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:25:56.607 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:25:56 smithi019 podman[65480]: 2024-09-16 00:25:56.112997672 +0000 UTC m=+0.388479980 container start ce175bee792691bc37017b7196f794a352253e9553d771ae90e83f166bbeec05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T00:25:56.607 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:25:56 smithi019 ceph-mds[65499]: -- 172.21.15.19:0/1961834278 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5623b5d25860 con 0x5623b5d03000 2024-09-16T00:25:56.607 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:25:56 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h[65495]: starting mds.h at 2024-09-16T00:25:56.607 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:25:56 smithi019 bash[65480]: ce175bee792691bc37017b7196f794a352253e9553d771ae90e83f166bbeec05 2024-09-16T00:25:56.607 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:25:56 smithi019 systemd[1]: Started Ceph mds.h for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:25:56.989 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:25:57.251 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: Deploying daemon mds.i on smithi153 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] up:boot 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: fsmap 8 up:standby 2024-09-16T00:25:57.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:57 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:25:57.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:57.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:57.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:57.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:57.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:57.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:57.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: Deploying daemon mds.i on smithi153 2024-09-16T00:25:57.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] up:boot 2024-09-16T00:25:57.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: fsmap 8 up:standby 2024-09-16T00:25:57.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:57 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: Deploying daemon mds.i on smithi153 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] up:boot 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: fsmap 8 up:standby 2024-09-16T00:25:57.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:57 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:25:57.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.774+0000 7fc47b147640 1 -- 172.21.15.1:0/1570833942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 msgr2=0x7fc474100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.774+0000 7fc47b147640 1 --2- 172.21.15.1:0/1570833942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 0x7fc474100ee0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc464009cd0 tx=0x7fc46402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.774+0000 7fc47b147640 1 -- 172.21.15.1:0/1570833942 shutdown_connections 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.774+0000 7fc47b147640 1 --2- 172.21.15.1:0/1570833942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc474101420 0x7fc47410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.774+0000 7fc47b147640 1 --2- 172.21.15.1:0/1570833942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 0x7fc474100ee0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.774+0000 7fc47b147640 1 --2- 172.21.15.1:0/1570833942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474106a80 0x7fc474106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.774+0000 7fc47b147640 1 -- 172.21.15.1:0/1570833942 >> 172.21.15.1:0/1570833942 conn(0x7fc4740fc7a0 msgr2=0x7fc4740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.774+0000 7fc47b147640 1 -- 172.21.15.1:0/1570833942 shutdown_connections 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.774+0000 7fc47b147640 1 -- 172.21.15.1:0/1570833942 wait complete. 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc47b147640 1 Processor -- start 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc47b147640 1 -- start start 2024-09-16T00:25:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc47b147640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 0x7fc474198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc47b147640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474101420 0x7fc474198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc47b147640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc474106a80 0x7fc47419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc47b147640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc474110e80 con 0x7fc474100a80 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc47b147640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc474110d00 con 0x7fc474101420 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc47b147640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc474111000 con 0x7fc474106a80 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc478ee3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 0x7fc474198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc478ee3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 0x7fc474198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46886/0 (socket says 172.21.15.1:46886) 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc478ee3640 1 -- 172.21.15.1:0/2392953794 learned_addr learned my addr 172.21.15.1:0/2392953794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc4796e4640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc474106a80 0x7fc47419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.775+0000 7fc473fff640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474101420 0x7fc474198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc478ee3640 1 -- 172.21.15.1:0/2392953794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc474106a80 msgr2=0x7fc47419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc478ee3640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc474106a80 0x7fc47419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc478ee3640 1 -- 172.21.15.1:0/2392953794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474101420 msgr2=0x7fc474198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc478ee3640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474101420 0x7fc474198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc478ee3640 1 -- 172.21.15.1:0/2392953794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47419d3e0 con 0x7fc474100a80 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc478ee3640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 0x7fc474198430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00d950 tx=0x7fc45c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc471ffb640 1 -- 172.21.15.1:0/2392953794 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c017070 con 0x7fc474100a80 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc471ffb640 1 -- 172.21.15.1:0/2392953794 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c009c30 con 0x7fc474100a80 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47419d6d0 con 0x7fc474100a80 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc471ffb640 1 -- 172.21.15.1:0/2392953794 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00f6b0 con 0x7fc474100a80 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.776+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a4fb0 con 0x7fc474100a80 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.777+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc474100ee0 con 0x7fc474100a80 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.778+0000 7fc471ffb640 1 -- 172.21.15.1:0/2392953794 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fc45c0109d0 con 0x7fc474100a80 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.779+0000 7fc471ffb640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4600a3240 0x7fc4600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:57.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.779+0000 7fc473fff640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4600a3240 0x7fc4600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.779+0000 7fc471ffb640 1 -- 172.21.15.1:0/2392953794 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fc45c0cf000 con 0x7fc474100a80 2024-09-16T00:25:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.780+0000 7fc473fff640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4600a3240 0x7fc4600a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc4741051a0 tx=0x7fc4640093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.784+0000 7fc471ffb640 1 -- 172.21.15.1:0/2392953794 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc45c009de0 con 0x7fc474100a80 2024-09-16T00:25:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.997+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc474199790 con 0x7fc474100a80 2024-09-16T00:25:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.997+0000 7fc471ffb640 1 -- 172.21.15.1:0/2392953794 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7204 (secure 0 0 0) 0x7fc45c09c1c0 con 0x7fc474100a80 2024-09-16T00:25:57.999 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:25:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"btime":"2024-09-16T00:25:56:602911+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T00:25:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-09-16T00:25:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.999+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4600a3240 msgr2=0x7fc4600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.999+0000 7fc47b147640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4600a3240 0x7fc4600a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc4741051a0 tx=0x7fc4640093f0 comp rx=0 tx=0).stop 2024-09-16T00:25:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.999+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 msgr2=0x7fc474198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:57.999+0000 7fc47b147640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 0x7fc474198430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00d950 tx=0x7fc45c00de10 comp rx=0 tx=0).stop 2024-09-16T00:25:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:58.000+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 shutdown_connections 2024-09-16T00:25:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:58.000+0000 7fc47b147640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc4600a3240 0x7fc4600a5700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:58.000+0000 7fc47b147640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc474106a80 0x7fc47419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:58.000+0000 7fc47b147640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474101420 0x7fc474198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:58.000+0000 7fc47b147640 1 --2- 172.21.15.1:0/2392953794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc474100a80 0x7fc474198430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:58.000+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 >> 172.21.15.1:0/2392953794 conn(0x7fc4740fc7a0 msgr2=0x7fc47406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:58.000+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 shutdown_connections 2024-09-16T00:25:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:25:58.000+0000 7fc47b147640 1 -- 172.21.15.1:0/2392953794 wait complete. 2024-09-16T00:25:58.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:58 smithi001 ceph-mon[30934]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:58.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:58 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2392953794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:58.443 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:58 smithi153 ceph-mon[35538]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:58.444 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:58 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2392953794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:58 smithi019 ceph-mon[35186]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:25:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:58 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2392953794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:58.712 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:25:58 smithi153 podman[65860]: 2024-09-16 00:25:58.60588205 +0000 UTC m=+0.276158267 container create f0b544629d6c2f509583e555f4132fe62dd23187ab98138ab5ac34df4a326316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:25:59.212 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:25:58 smithi153 podman[65860]: 2024-09-16 00:25:58.850455415 +0000 UTC m=+0.520731573 container init f0b544629d6c2f509583e555f4132fe62dd23187ab98138ab5ac34df4a326316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T00:25:59.212 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:25:58 smithi153 podman[65860]: 2024-09-16 00:25:58.853193096 +0000 UTC m=+0.523469255 container start f0b544629d6c2f509583e555f4132fe62dd23187ab98138ab5ac34df4a326316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T00:25:59.212 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:25:58 smithi153 ceph-mds[65879]: -- 172.21.15.153:0/1654035439 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e829a53860 con 0x55e829a31400 2024-09-16T00:25:59.212 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:25:58 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i[65875]: starting mds.i at 2024-09-16T00:25:59.212 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:25:58 smithi153 bash[65860]: f0b544629d6c2f509583e555f4132fe62dd23187ab98138ab5ac34df4a326316 2024-09-16T00:25:59.212 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:25:58 smithi153 systemd[1]: Started Ceph mds.i for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:26:00.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:00.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:00.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:00.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:00.361 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:26:00.361 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:26:00.361 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:00.361 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: Deploying daemon mds.g on smithi001 2024-09-16T00:26:00.361 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] up:boot 2024-09-16T00:26:00.361 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: fsmap 9 up:standby 2024-09-16T00:26:00.361 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:25:59 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:26:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:00.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:00.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:00.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:00.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:26:00.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:26:00.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:00.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: Deploying daemon mds.g on smithi001 2024-09-16T00:26:00.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] up:boot 2024-09-16T00:26:00.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: fsmap 9 up:standby 2024-09-16T00:26:00.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:25:59 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:26:00.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: Deploying daemon mds.g on smithi001 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] up:boot 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: fsmap 9 up:standby 2024-09-16T00:26:00.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:25:59 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:26:00.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:26:01.034 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:01.162 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:26:01 smithi001 systemd[1]: Starting Ceph mds.g for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:26:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 -- 172.21.15.1:0/3332445778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 msgr2=0x7fbbac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 --2- 172.21.15.1:0/3332445778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 0x7fbbac101200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c02a480 comp rx=0 tx=0).stop 2024-09-16T00:26:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 -- 172.21.15.1:0/3332445778 shutdown_connections 2024-09-16T00:26:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 --2- 172.21.15.1:0/3332445778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac102ef0 0x7fbbac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 --2- 172.21.15.1:0/3332445778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac102570 0x7fbbac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 --2- 172.21.15.1:0/3332445778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 0x7fbbac101200 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 -- 172.21.15.1:0/3332445778 >> 172.21.15.1:0/3332445778 conn(0x7fbbac0fc7d0 msgr2=0x7fbbac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 -- 172.21.15.1:0/3332445778 shutdown_connections 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 -- 172.21.15.1:0/3332445778 wait complete. 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.520+0000 7fbbb3153640 1 Processor -- start 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb3153640 1 -- start start 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb3153640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 0x7fbbac1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb3153640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac102570 0x7fbbac198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb3153640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac102ef0 0x7fbbac19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb3153640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac110fd0 con 0x7fbbac100e20 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb3153640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac110e50 con 0x7fbbac102ef0 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb3153640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac111150 con 0x7fbbac102570 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb0eef640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 0x7fbbac1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb0eef640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 0x7fbbac1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41366/0 (socket says 172.21.15.1:41366) 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb0eef640 1 -- 172.21.15.1:0/1060708564 learned_addr learned my addr 172.21.15.1:0/1060708564 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbabfff640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac102570 0x7fbbac198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb16f0640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac102ef0 0x7fbbac19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb0eef640 1 -- 172.21.15.1:0/1060708564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac102570 msgr2=0x7fbbac198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb0eef640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac102570 0x7fbbac198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb0eef640 1 -- 172.21.15.1:0/1060708564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac102ef0 msgr2=0x7fbbac19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb0eef640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac102ef0 0x7fbbac19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb0eef640 1 -- 172.21.15.1:0/1060708564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac19d590 con 0x7fbbac100e20 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb16f0640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac102ef0 0x7fbbac19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.521+0000 7fbbb0eef640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 0x7fbbac1985e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c02aa70 tx=0x7fbb9c003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.522+0000 7fbba9ffb640 1 -- 172.21.15.1:0/1060708564 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c03f070 con 0x7fbbac100e20 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.522+0000 7fbba9ffb640 1 -- 172.21.15.1:0/1060708564 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb9c031c20 con 0x7fbbac100e20 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.522+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac19d820 con 0x7fbbac100e20 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.522+0000 7fbba9ffb640 1 -- 172.21.15.1:0/1060708564 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c035030 con 0x7fbbac100e20 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.522+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac1a5160 con 0x7fbbac100e20 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.523+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbac101200 con 0x7fbbac100e20 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.524+0000 7fbba9ffb640 1 -- 172.21.15.1:0/1060708564 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fbb9c042070 con 0x7fbbac100e20 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.525+0000 7fbba9ffb640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb940a32c0 0x7fbb940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.525+0000 7fbba9ffb640 1 -- 172.21.15.1:0/1060708564 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fbb9c0f0130 con 0x7fbbac100e20 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.525+0000 7fbbabfff640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb940a32c0 0x7fbb940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.526+0000 7fbbabfff640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb940a32c0 0x7fbb940a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0fff40 tx=0x7fbba000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:01.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.528+0000 7fbba9ffb640 1 -- 172.21.15.1:0/1060708564 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb9c0bd2f0 con 0x7fbbac100e20 2024-09-16T00:26:01.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.748+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbac1a5330 con 0x7fbbac100e20 2024-09-16T00:26:01.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.748+0000 7fbba9ffb640 1 -- 172.21.15.1:0/1060708564 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8034 (secure 0 0 0) 0x7fbb9c031dd0 con 0x7fbbac100e20 2024-09-16T00:26:01.750 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:01.750 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"btime":"2024-09-16T00:25:59:258697+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T00:26:01.750 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-09-16T00:26:01.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb940a32c0 msgr2=0x7fbb940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:01.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb940a32c0 0x7fbb940a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0fff40 tx=0x7fbba000b390 comp rx=0 tx=0).stop 2024-09-16T00:26:01.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 msgr2=0x7fbbac1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:01.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 0x7fbbac1985e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c02aa70 tx=0x7fbb9c003b60 comp rx=0 tx=0).stop 2024-09-16T00:26:01.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 shutdown_connections 2024-09-16T00:26:01.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac102ef0 0x7fbbac19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb940a32c0 0x7fbb940a5780 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbac102570 0x7fbbac198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 --2- 172.21.15.1:0/1060708564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbac100e20 0x7fbbac1985e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 >> 172.21.15.1:0/1060708564 conn(0x7fbbac0fc7d0 msgr2=0x7fbbac06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:01.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 shutdown_connections 2024-09-16T00:26:01.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:01.751+0000 7fbbb3153640 1 -- 172.21.15.1:0/1060708564 wait complete. 2024-09-16T00:26:01.951 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:26:01 smithi001 podman[68189]: 2024-09-16 00:26:01.600518803 +0000 UTC m=+0.017156277 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:26:01.951 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:26:01 smithi001 podman[68189]: 2024-09-16 00:26:01.728434358 +0000 UTC m=+0.145071825 container create 20f03c28ee4eccc64cceec799dfd9e83a194c1c40a891a2c00673b95de6c1cb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T00:26:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:01 smithi019 ceph-mon[35186]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:01 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1060708564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:01 smithi001 ceph-mon[30934]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:01 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1060708564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:01 smithi153 ceph-mon[35538]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:01 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1060708564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:02.951 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:26:02 smithi001 podman[68189]: 2024-09-16 00:26:02.655840217 +0000 UTC m=+1.072477686 container init 20f03c28ee4eccc64cceec799dfd9e83a194c1c40a891a2c00673b95de6c1cb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:26:02.951 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:26:02 smithi001 podman[68189]: 2024-09-16 00:26:02.65939274 +0000 UTC m=+1.076030198 container start 20f03c28ee4eccc64cceec799dfd9e83a194c1c40a891a2c00673b95de6c1cb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T00:26:02.951 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:26:02 smithi001 ceph-mds[68223]: -- 172.21.15.1:0/1292153892 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1e8dc1860 con 0x55c1e8d9f000 2024-09-16T00:26:02.952 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:26:02 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-g[68217]: starting mds.g at 2024-09-16T00:26:02.952 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:26:02 smithi001 bash[68189]: 20f03c28ee4eccc64cceec799dfd9e83a194c1c40a891a2c00673b95de6c1cb9 2024-09-16T00:26:02.952 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:26:02 smithi001 systemd[1]: Started Ceph mds.g for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:26:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:26:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:26:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: Deploying daemon mds.a on smithi001 2024-09-16T00:26:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.1:6838/4066219801,v1:172.21.15.1:6839/4066219801] up:boot 2024-09-16T00:26:04.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: fsmap 10 up:standby 2024-09-16T00:26:04.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:03 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:26:04.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:04.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:04.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:04.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:04.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:26:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:26:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: Deploying daemon mds.a on smithi001 2024-09-16T00:26:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.1:6838/4066219801,v1:172.21.15.1:6839/4066219801] up:boot 2024-09-16T00:26:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: fsmap 10 up:standby 2024-09-16T00:26:04.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:03 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:26:04.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:04.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:04.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:04.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:04.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:26:04.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:26:04.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:04.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: Deploying daemon mds.a on smithi001 2024-09-16T00:26:04.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.1:6838/4066219801,v1:172.21.15.1:6839/4066219801] up:boot 2024-09-16T00:26:04.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: fsmap 10 up:standby 2024-09-16T00:26:04.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:03 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:26:04.570 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:26:04.951 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:26:04 smithi001 systemd[1]: Starting Ceph mds.a for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:26:05.286 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:26:05 smithi001 podman[68522]: 2024-09-16 00:26:04.909719673 +0000 UTC m=+0.018923703 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:26:05.287 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:26:05 smithi001 podman[68522]: 2024-09-16 00:26:05.050520318 +0000 UTC m=+0.159724329 container create ca4a96a20d49b34b74dc02f5a5295fdaa44cd15c89dd18bc46ddfd370c75a19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:26:05.287 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:26:05 smithi001 podman[68522]: 2024-09-16 00:26:05.287153117 +0000 UTC m=+0.396357141 container init ca4a96a20d49b34b74dc02f5a5295fdaa44cd15c89dd18bc46ddfd370c75a19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T00:26:05.441 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:05.565 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:26:05 smithi001 podman[68522]: 2024-09-16 00:26:05.290278474 +0000 UTC m=+0.399482489 container start ca4a96a20d49b34b74dc02f5a5295fdaa44cd15c89dd18bc46ddfd370c75a19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:26:05.565 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:26:05 smithi001 ceph-mds[68566]: -- 172.21.15.1:0/970980964 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556dfa4eb860 con 0x556dfa4c8800 2024-09-16T00:26:05.565 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:26:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-a[68550]: starting mds.a at 2024-09-16T00:26:05.565 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:26:05 smithi001 bash[68522]: ca4a96a20d49b34b74dc02f5a5295fdaa44cd15c89dd18bc46ddfd370c75a19a 2024-09-16T00:26:05.565 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:26:05 smithi001 systemd[1]: Started Ceph mds.a for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:26:06.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.989+0000 7f1dd8398640 1 -- 172.21.15.1:0/1293057878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0102790 msgr2=0x7f1dd01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:06.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.989+0000 7f1dd8398640 1 --2- 172.21.15.1:0/1293057878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0102790 0x7f1dd01067e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009900 tx=0x7f1dc002a410 comp rx=0 tx=0).stop 2024-09-16T00:26:06.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.989+0000 7f1dd8398640 1 -- 172.21.15.1:0/1293057878 shutdown_connections 2024-09-16T00:26:06.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.989+0000 7f1dd8398640 1 --2- 172.21.15.1:0/1293057878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd0106f10 0x7f1dd010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.989+0000 7f1dd8398640 1 --2- 172.21.15.1:0/1293057878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0102790 0x7f1dd01067e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.989+0000 7f1dd8398640 1 --2- 172.21.15.1:0/1293057878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dd0101de0 0x7f1dd01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 -- 172.21.15.1:0/1293057878 >> 172.21.15.1:0/1293057878 conn(0x7f1dd00fd8c0 msgr2=0x7f1dd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:06.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 -- 172.21.15.1:0/1293057878 shutdown_connections 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 -- 172.21.15.1:0/1293057878 wait complete. 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 Processor -- start 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 -- start start 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0101de0 0x7f1dd0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd0102790 0x7f1dd0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dd0106f10 0x7f1dd019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0111f30 con 0x7f1dd0101de0 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0111db0 con 0x7f1dd0102790 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.990+0000 7f1dd8398640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01120b0 con 0x7f1dd0106f10 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6134640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0101de0 0x7f1dd0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6134640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0101de0 0x7f1dd0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41430/0 (socket says 172.21.15.1:41430) 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6134640 1 -- 172.21.15.1:0/4272169691 learned_addr learned my addr 172.21.15.1:0/4272169691 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd5933640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd0102790 0x7f1dd0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6935640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dd0106f10 0x7f1dd019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6134640 1 -- 172.21.15.1:0/4272169691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dd0106f10 msgr2=0x7f1dd019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6134640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dd0106f10 0x7f1dd019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6134640 1 -- 172.21.15.1:0/4272169691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd0102790 msgr2=0x7f1dd0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6134640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd0102790 0x7f1dd0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6134640 1 -- 172.21.15.1:0/4272169691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd019d3e0 con 0x7f1dd0101de0 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd5933640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd0102790 0x7f1dd0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd6134640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0101de0 0x7f1dd0198430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d950 tx=0x7f1dcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dc77fe640 1 -- 172.21.15.1:0/4272169691 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc017030 con 0x7f1dd0101de0 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dc77fe640 1 -- 172.21.15.1:0/4272169691 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dcc004020 con 0x7f1dd0101de0 2024-09-16T00:26:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd019d6d0 con 0x7f1dd0101de0 2024-09-16T00:26:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dc77fe640 1 -- 172.21.15.1:0/4272169691 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0047a0 con 0x7f1dd0101de0 2024-09-16T00:26:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.991+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd01a4fb0 con 0x7f1dd0101de0 2024-09-16T00:26:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.992+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd01021c0 con 0x7f1dd0101de0 2024-09-16T00:26:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.994+0000 7f1dc77fe640 1 -- 172.21.15.1:0/4272169691 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f1dcc0041d0 con 0x7f1dd0101de0 2024-09-16T00:26:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.994+0000 7f1dc77fe640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dbc0a32c0 0x7f1dbc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.994+0000 7f1dd5933640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dbc0a32c0 0x7f1dbc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.994+0000 7f1dc77fe640 1 -- 172.21.15.1:0/4272169691 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f1dcc0cf400 con 0x7f1dd0101de0 2024-09-16T00:26:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.995+0000 7f1dd5933640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dbc0a32c0 0x7f1dbc0a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009230 tx=0x7f1dc00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:05.997+0000 7f1dc77fe640 1 -- 172.21.15.1:0/4272169691 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1dcc09c640 con 0x7f1dd0101de0 2024-09-16T00:26:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:05 smithi019 ceph-mon[35186]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:26:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:26:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:05 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:06.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:05 smithi001 ceph-mon[30934]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:06.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:06.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:06.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:06.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:26:06.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:26:06.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:05 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:06.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:05 smithi153 ceph-mon[35538]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:06.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:26:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:26:06.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:05 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:06.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.211+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dd002d7c0 con 0x7f1dd0101de0 2024-09-16T00:26:06.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.211+0000 7f1dc77fe640 1 -- 172.21.15.1:0/4272169691 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9677 (secure 0 0 0) 0x7f1dcc09bf40 con 0x7f1dd0101de0 2024-09-16T00:26:06.213 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:06.213 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":12,"btime":"2024-09-16T00:26:05:819017+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T00:26:06.213 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 12 2024-09-16T00:26:06.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dbc0a32c0 msgr2=0x7f1dbc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:06.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dbc0a32c0 0x7f1dbc0a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009230 tx=0x7f1dc00058d0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0101de0 msgr2=0x7f1dd0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:06.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0101de0 0x7f1dd0198430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d950 tx=0x7f1dcc00de10 comp rx=0 tx=0).stop 2024-09-16T00:26:06.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 shutdown_connections 2024-09-16T00:26:06.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1dbc0a32c0 0x7f1dbc0a5780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dd0106f10 0x7f1dd019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd0102790 0x7f1dd0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 --2- 172.21.15.1:0/4272169691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0101de0 0x7f1dd0198430 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.213+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 >> 172.21.15.1:0/4272169691 conn(0x7f1dd00fd8c0 msgr2=0x7f1dd00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:06.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.214+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 shutdown_connections 2024-09-16T00:26:06.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:06.214+0000 7f1dd8398640 1 -- 172.21.15.1:0/4272169691 wait complete. 2024-09-16T00:26:06.938 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:06 smithi001 ceph-mon[30934]: Deploying daemon mds.b on smithi019 2024-09-16T00:26:06.938 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:06 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.1:6840/208912636,v1:172.21.15.1:6841/208912636] up:boot 2024-09-16T00:26:06.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:06 smithi001 ceph-mon[30934]: fsmap 11 up:standby 2024-09-16T00:26:06.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:06 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:26:06.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:06 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4272169691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:07.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:06 smithi019 ceph-mon[35186]: Deploying daemon mds.b on smithi019 2024-09-16T00:26:07.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:06 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.1:6840/208912636,v1:172.21.15.1:6841/208912636] up:boot 2024-09-16T00:26:07.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:06 smithi019 ceph-mon[35186]: fsmap 11 up:standby 2024-09-16T00:26:07.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:06 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:26:07.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:06 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4272169691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:07.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:06 smithi153 ceph-mon[35538]: Deploying daemon mds.b on smithi019 2024-09-16T00:26:07.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:06 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.1:6840/208912636,v1:172.21.15.1:6841/208912636] up:boot 2024-09-16T00:26:07.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:06 smithi153 ceph-mon[35538]: fsmap 11 up:standby 2024-09-16T00:26:07.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:06 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:26:07.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:06 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4272169691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:07.607 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:07 smithi019 systemd[1]: Starting Ceph mds.b for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:26:07.924 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:07 smithi019 ceph-mon[35186]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:07.924 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:07 smithi019 podman[65792]: 2024-09-16 00:26:07.571145499 +0000 UTC m=+0.015873187 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:26:07.924 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:07 smithi019 podman[65792]: 2024-09-16 00:26:07.670871679 +0000 UTC m=+0.115599403 container create 09e23b1983f32fbce3f38ef35b5bb98d7a4359b5226e50468e1bc78f04afe849 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-16T00:26:07.925 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:07 smithi019 podman[65792]: 2024-09-16 00:26:07.898597317 +0000 UTC m=+0.343324993 container init 09e23b1983f32fbce3f38ef35b5bb98d7a4359b5226e50468e1bc78f04afe849 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T00:26:07.925 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:07 smithi019 podman[65792]: 2024-09-16 00:26:07.901685612 +0000 UTC m=+0.346413298 container start 09e23b1983f32fbce3f38ef35b5bb98d7a4359b5226e50468e1bc78f04afe849 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T00:26:07.925 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:07 smithi019 ceph-mds[65811]: -- 172.21.15.19:0/3170073658 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ba39cf860 con 0x562ba39ad400 2024-09-16T00:26:08.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:07 smithi001 ceph-mon[30934]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:08.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:07 smithi153 ceph-mon[35538]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:08.357 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b[65807]: starting mds.b at 2024-09-16T00:26:08.357 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:07 smithi019 bash[65792]: 09e23b1983f32fbce3f38ef35b5bb98d7a4359b5226e50468e1bc78f04afe849 2024-09-16T00:26:08.357 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:08 smithi019 systemd[1]: Started Ceph mds.b for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:26:08.972 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:26:09.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:09 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] up:boot 2024-09-16T00:26:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:09 smithi153 ceph-mon[35538]: fsmap 12 up:standby 2024-09-16T00:26:09.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:09 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:26:09.349 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:09.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:09.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:09 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] up:boot 2024-09-16T00:26:09.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:09 smithi019 ceph-mon[35186]: fsmap 12 up:standby 2024-09-16T00:26:09.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:09 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:26:09.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:09.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:09.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:09 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] up:boot 2024-09-16T00:26:09.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:09 smithi001 ceph-mon[30934]: fsmap 12 up:standby 2024-09-16T00:26:09.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:09 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:26:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.836+0000 7f376b124640 1 -- 172.21.15.1:0/156925280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764106f30 msgr2=0x7f376410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.836+0000 7f376b124640 1 --2- 172.21.15.1:0/156925280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764106f30 0x7f376410eab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f375c009ca0 tx=0x7f375c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:26:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.836+0000 7f376b124640 1 -- 172.21.15.1:0/156925280 shutdown_connections 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.836+0000 7f376b124640 1 --2- 172.21.15.1:0/156925280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764106f30 0x7f376410eab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.836+0000 7f376b124640 1 --2- 172.21.15.1:0/156925280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3764100cc0 0x7f3764106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.836+0000 7f376b124640 1 --2- 172.21.15.1:0/156925280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3764100310 0x7f37641006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.836+0000 7f376b124640 1 -- 172.21.15.1:0/156925280 >> 172.21.15.1:0/156925280 conn(0x7f3764077210 msgr2=0x7f37640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.836+0000 7f376b124640 1 -- 172.21.15.1:0/156925280 shutdown_connections 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.836+0000 7f376b124640 1 -- 172.21.15.1:0/156925280 wait complete. 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f376b124640 1 Processor -- start 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f376b124640 1 -- start start 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f376b124640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764100310 0x7f3764101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f376b124640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3764100cc0 0x7f37641017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f376b124640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3764106f30 0x7f3764101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f376b124640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111f90 con 0x7f3764100310 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f376b124640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111e10 con 0x7f3764106f30 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f376b124640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3764112110 con 0x7f3764100cc0 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f3768ec0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764100310 0x7f3764101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f3768ec0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764100310 0x7f3764101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41444/0 (socket says 172.21.15.1:41444) 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f3768ec0640 1 -- 172.21.15.1:0/2001619299 learned_addr learned my addr 172.21.15.1:0/2001619299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f375bfff640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3764100cc0 0x7f37641017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f37696c1640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3764106f30 0x7f3764101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f3768ec0640 1 -- 172.21.15.1:0/2001619299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3764100cc0 msgr2=0x7f37641017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f3768ec0640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3764100cc0 0x7f37641017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f3768ec0640 1 -- 172.21.15.1:0/2001619299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3764106f30 msgr2=0x7f3764101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f3768ec0640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3764106f30 0x7f3764101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f3768ec0640 1 -- 172.21.15.1:0/2001619299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37641a68d0 con 0x7f3764100310 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.837+0000 7f37696c1640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3764106f30 0x7f3764101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.838+0000 7f3768ec0640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764100310 0x7f3764101290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f376000e9f0 tx=0x7f376000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.838+0000 7f3759ffb640 1 -- 172.21.15.1:0/2001619299 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376000cd80 con 0x7f3764100310 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.838+0000 7f3759ffb640 1 -- 172.21.15.1:0/2001619299 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3760005500 con 0x7f3764100310 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.838+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37641a6bc0 con 0x7f3764100310 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.838+0000 7f3759ffb640 1 -- 172.21.15.1:0/2001619299 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760010690 con 0x7f3764100310 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.838+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641a7100 con 0x7f3764100310 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.839+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37641006f0 con 0x7f3764100310 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.840+0000 7f3759ffb640 1 -- 172.21.15.1:0/2001619299 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f37600046c0 con 0x7f3764100310 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.841+0000 7f3759ffb640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f374c0a32c0 0x7f374c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.841+0000 7f375bfff640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f374c0a32c0 0x7f374c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.841+0000 7f3759ffb640 1 -- 172.21.15.1:0/2001619299 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f37600cf8a0 con 0x7f3764100310 2024-09-16T00:26:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.841+0000 7f375bfff640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f374c0a32c0 0x7f374c0a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3764102d20 tx=0x7f375400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:09.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:09.845+0000 7f3759ffb640 1 -- 172.21.15.1:0/2001619299 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f376009ca60 con 0x7f3764100310 2024-09-16T00:26:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.097+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f376410a1a0 con 0x7f3764100310 2024-09-16T00:26:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.098+0000 7f3759ffb640 1 -- 172.21.15.1:0/2001619299 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10503 (secure 0 0 0) 0x7f376009c360 con 0x7f3764100310 2024-09-16T00:26:10.099 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:10.100 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":13,"btime":"2024-09-16T00:26:08:843184+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-16T00:26:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 13 2024-09-16T00:26:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f374c0a32c0 msgr2=0x7f374c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f374c0a32c0 0x7f374c0a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3764102d20 tx=0x7f375400b390 comp rx=0 tx=0).stop 2024-09-16T00:26:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764100310 msgr2=0x7f3764101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764100310 0x7f3764101290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f376000e9f0 tx=0x7f376000edf0 comp rx=0 tx=0).stop 2024-09-16T00:26:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 shutdown_connections 2024-09-16T00:26:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3764106f30 0x7f3764101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f374c0a32c0 0x7f374c0a5780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3764100cc0 0x7f37641017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 --2- 172.21.15.1:0/2001619299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3764100310 0x7f3764101290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 >> 172.21.15.1:0/2001619299 conn(0x7f3764077210 msgr2=0x7f3764105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 shutdown_connections 2024-09-16T00:26:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:10.100+0000 7f376b124640 1 -- 172.21.15.1:0/2001619299 wait complete. 2024-09-16T00:26:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:10 smithi001 ceph-mon[30934]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:10 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:10 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:10 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:10 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:10 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:10 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2001619299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:10 smithi019 ceph-mon[35186]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:10 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:10 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:10 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:10 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:10 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:10 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2001619299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:10.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:10 smithi153 ceph-mon[35538]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:10.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:10 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:10 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:10 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:10 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:10 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:10.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:10 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2001619299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:10.907 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-16T00:26:11.515 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:11 smithi001 ceph-mon[30934]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:11.515 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:11 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:11.515 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:11 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:11.515 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:11 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:11.515 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:11 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:11.515 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:11 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:11.515 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:11 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:11.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:11 smithi153 ceph-mon[35538]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:11.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:11 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:11.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:11 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:11.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:11 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:11.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:11 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:11.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:11 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:11.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:11 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:11.814 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:11.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:11 smithi019 ceph-mon[35186]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:11.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:11 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:11.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:11 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:11.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:11 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:11.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:11 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:11.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:11 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:11.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:11 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:12.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/1350493296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102570 msgr2=0x7fa4d41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:12.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/1350493296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102570 0x7fa4d41029b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009900 tx=0x7fa4c802a480 comp rx=0 tx=0).stop 2024-09-16T00:26:12.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/1350493296 shutdown_connections 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/1350493296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d4102ef0 0x7fa4d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/1350493296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102570 0x7fa4d41029b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/1350493296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4100e20 0x7fa4d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/1350493296 >> 172.21.15.1:0/1350493296 conn(0x7fa4d40fc7d0 msgr2=0x7fa4d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/1350493296 shutdown_connections 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/1350493296 wait complete. 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.324+0000 7fa4dc7b2640 1 Processor -- start 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dc7b2640 1 -- start start 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dc7b2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d4100e20 0x7fa4d41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dc7b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4102570 0x7fa4d4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dc7b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102ef0 0x7fa4d419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dc7b2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4110fa0 con 0x7fa4d4102ef0 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dc7b2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4110e20 con 0x7fa4d4102570 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dc7b2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4111120 con 0x7fa4d4100e20 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dad4f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102ef0 0x7fa4d419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dad4f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102ef0 0x7fa4d419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59814/0 (socket says 172.21.15.1:59814) 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dad4f640 1 -- 172.21.15.1:0/2980794163 learned_addr learned my addr 172.21.15.1:0/2980794163 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4da54e640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d4100e20 0x7fa4d41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4d9d4d640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4102570 0x7fa4d4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dad4f640 1 -- 172.21.15.1:0/2980794163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d4100e20 msgr2=0x7fa4d41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dad4f640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d4100e20 0x7fa4d41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dad4f640 1 -- 172.21.15.1:0/2980794163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4102570 msgr2=0x7fa4d4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dad4f640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4102570 0x7fa4d4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dad4f640 1 -- 172.21.15.1:0/2980794163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d419d480 con 0x7fa4d4102ef0 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4d9d4d640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4102570 0x7fa4d4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4dad4f640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102ef0 0x7fa4d419cda0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00c970 tx=0x7fa4cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.325+0000 7fa4c77fe640 1 -- 172.21.15.1:0/2980794163 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc007bb0 con 0x7fa4d4102ef0 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.326+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d419d770 con 0x7fa4d4102ef0 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.326+0000 7fa4c77fe640 1 -- 172.21.15.1:0/2980794163 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4cc007d50 con 0x7fa4d4102ef0 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.326+0000 7fa4c77fe640 1 -- 172.21.15.1:0/2980794163 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00f450 con 0x7fa4d4102ef0 2024-09-16T00:26:12.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.326+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d41a5050 con 0x7fa4d4102ef0 2024-09-16T00:26:12.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.327+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d4101200 con 0x7fa4d4102ef0 2024-09-16T00:26:12.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.328+0000 7fa4c77fe640 1 -- 172.21.15.1:0/2980794163 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fa4cc004f00 con 0x7fa4d4102ef0 2024-09-16T00:26:12.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.328+0000 7fa4c77fe640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4c00a3240 0x7fa4c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:12.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.329+0000 7fa4da54e640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4c00a3240 0x7fa4c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:12.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.329+0000 7fa4c77fe640 1 -- 172.21.15.1:0/2980794163 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fa4cc0cf5c0 con 0x7fa4d4102ef0 2024-09-16T00:26:12.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.329+0000 7fa4da54e640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4c00a3240 0x7fa4c00a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0007ec0 tx=0x7fa4d00075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:12.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.332+0000 7fa4c77fe640 1 -- 172.21.15.1:0/2980794163 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4cc014030 con 0x7fa4d4102ef0 2024-09-16T00:26:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.541+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa4d410a0a0 con 0x7fa4d4102ef0 2024-09-16T00:26:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.541+0000 7fa4c77fe640 1 -- 172.21.15.1:0/2980794163 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7fa4cc09c6e0 con 0x7fa4d4102ef0 2024-09-16T00:26:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:12.543 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-09-16T00:26:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:26:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-09-16T00:26:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-09-16T00:26:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:26:12.543 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4c00a3240 msgr2=0x7fa4c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4c00a3240 0x7fa4c00a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0007ec0 tx=0x7fa4d00075b0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102ef0 msgr2=0x7fa4d419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102ef0 0x7fa4d419cda0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00c970 tx=0x7fa4cc00ce30 comp rx=0 tx=0).stop 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 shutdown_connections 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4d4102ef0 0x7fa4d419cda0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4102570 0x7fa4d4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4c00a3240 0x7fa4c00a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 --2- 172.21.15.1:0/2980794163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d4100e20 0x7fa4d41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 >> 172.21.15.1:0/2980794163 conn(0x7fa4d40fc7d0 msgr2=0x7fa4d406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 shutdown_connections 2024-09-16T00:26:12.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:12.544+0000 7fa4dc7b2640 1 -- 172.21.15.1:0/2980794163 wait complete. 2024-09-16T00:26:13.232 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.233 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.233 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.233 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.233 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2980794163' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:26:13.233 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.233 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.234 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:13.234 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:13.234 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:12 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.265 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T00:26:13.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T00:26:13.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2980794163' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:26:13.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:13.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:13.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:12 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2980794163' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:26:13.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:13.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:13.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:12 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:26:13.526 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:14.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 -- 172.21.15.1:0/695005197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c106f40 msgr2=0x7f3d0c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:14.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 --2- 172.21.15.1:0/695005197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c106f40 0x7f3d0c10eac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009900 tx=0x7f3d0402a410 comp rx=0 tx=0).stop 2024-09-16T00:26:14.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 -- 172.21.15.1:0/695005197 shutdown_connections 2024-09-16T00:26:14.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 --2- 172.21.15.1:0/695005197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c106f40 0x7f3d0c10eac0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:14.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 --2- 172.21.15.1:0/695005197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c1027c0 0x7f3d0c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:14.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 --2- 172.21.15.1:0/695005197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d0c101e10 0x7f3d0c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:14.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 -- 172.21.15.1:0/695005197 >> 172.21.15.1:0/695005197 conn(0x7f3d0c0fd8d0 msgr2=0x7f3d0c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 -- 172.21.15.1:0/695005197 shutdown_connections 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 -- 172.21.15.1:0/695005197 wait complete. 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.022+0000 7f3d13e58640 1 Processor -- start 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d13e58640 1 -- start start 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d13e58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d0c101e10 0x7f3d0c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d13e58640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c1027c0 0x7f3d0c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d13e58640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c106f40 0x7f3d0c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d13e58640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c111fc0 con 0x7f3d0c1027c0 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d13e58640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c111e40 con 0x7f3d0c101e10 2024-09-16T00:26:14.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d13e58640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c112140 con 0x7f3d0c106f40 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d113f3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c1027c0 0x7f3d0c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d113f3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c1027c0 0x7f3d0c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59828/0 (socket says 172.21.15.1:59828) 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d113f3640 1 -- 172.21.15.1:0/4008957606 learned_addr learned my addr 172.21.15.1:0/4008957606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d123f5640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c106f40 0x7f3d0c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d11bf4640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d0c101e10 0x7f3d0c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d113f3640 1 -- 172.21.15.1:0/4008957606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c106f40 msgr2=0x7f3d0c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d113f3640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c106f40 0x7f3d0c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d113f3640 1 -- 172.21.15.1:0/4008957606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d0c101e10 msgr2=0x7f3d0c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d113f3640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d0c101e10 0x7f3d0c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d113f3640 1 -- 172.21.15.1:0/4008957606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c19d400 con 0x7f3d0c1027c0 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.023+0000 7f3d11bf4640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d0c101e10 0x7f3d0c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:14.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.024+0000 7f3d113f3640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c1027c0 0x7f3d0c198990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b4a0 tx=0x7f3d0000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.024+0000 7f3d0affd640 1 -- 172.21.15.1:0/4008957606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00014070 con 0x7f3d0c1027c0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.024+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c19d6f0 con 0x7f3d0c1027c0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.024+0000 7f3d0affd640 1 -- 172.21.15.1:0/4008957606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d00010af0 con 0x7f3d0c1027c0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.024+0000 7f3d0affd640 1 -- 172.21.15.1:0/4008957606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00017d10 con 0x7f3d0c1027c0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.024+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1a4fd0 con 0x7f3d0c1027c0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.025+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c1021f0 con 0x7f3d0c1027c0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.026+0000 7f3d0affd640 1 -- 172.21.15.1:0/4008957606 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f3d00017070 con 0x7f3d0c1027c0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.026+0000 7f3d0affd640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3cf40a3240 0x7f3cf40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.027+0000 7f3d11bf4640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3cf40a3240 0x7f3cf40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.027+0000 7f3d0affd640 1 -- 172.21.15.1:0/4008957606 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f3d000ce930 con 0x7f3d0c1027c0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.027+0000 7f3d11bf4640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3cf40a3240 0x7f3cf40a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc005fd0 tx=0x7f3cfc009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:14.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.032+0000 7f3d0affd640 1 -- 172.21.15.1:0/4008957606 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d0009ba50 con 0x7f3d0c1027c0 2024-09-16T00:26:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:14 smithi001 ceph-mon[30934]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:14.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:14.210+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3d0c10a180 con 0x7f3d0c1027c0 2024-09-16T00:26:14.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:14 smithi153 ceph-mon[35538]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:14.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:14 smithi019 ceph-mon[35186]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:15.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.127+0000 7f3d0affd640 1 -- 172.21.15.1:0/4008957606 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f3d0009b350 con 0x7f3d0c1027c0 2024-09-16T00:26:15.129 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-09-16T00:26:15.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3cf40a3240 msgr2=0x7f3cf40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:15.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3cf40a3240 0x7f3cf40a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc005fd0 tx=0x7f3cfc009520 comp rx=0 tx=0).stop 2024-09-16T00:26:15.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c1027c0 msgr2=0x7f3d0c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c1027c0 0x7f3d0c198990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b4a0 tx=0x7f3d0000b960 comp rx=0 tx=0).stop 2024-09-16T00:26:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 shutdown_connections 2024-09-16T00:26:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3cf40a3240 0x7f3cf40a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c106f40 0x7f3d0c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d0c1027c0 0x7f3d0c198990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 --2- 172.21.15.1:0/4008957606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d0c101e10 0x7f3d0c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 >> 172.21.15.1:0/4008957606 conn(0x7f3d0c0fd8d0 msgr2=0x7f3d0c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 shutdown_connections 2024-09-16T00:26:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:15.130+0000 7f3d13e58640 1 -- 172.21.15.1:0/4008957606 wait complete. 2024-09-16T00:26:15.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:15 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4008957606' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:26:15.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:15 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4008957606' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:26:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:15 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4008957606' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:26:15.868 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-16T00:26:16.135 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:16.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:16 smithi153 ceph-mon[35538]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:16.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:16 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4008957606' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:26:16.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:16 smithi153 ceph-mon[35538]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:26:16.398 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:16 smithi001 ceph-mon[30934]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:16.398 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:16 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4008957606' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:26:16.398 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:16 smithi001 ceph-mon[30934]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:26:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:16 smithi019 ceph-mon[35186]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:16 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4008957606' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:26:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:16 smithi019 ceph-mon[35186]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:26:16.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 -- 172.21.15.1:0/2253690099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 msgr2=0x7f2220101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:16.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 --2- 172.21.15.1:0/2253690099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 0x7f2220101200 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2210009900 tx=0x7f221002a480 comp rx=0 tx=0).stop 2024-09-16T00:26:16.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 -- 172.21.15.1:0/2253690099 shutdown_connections 2024-09-16T00:26:16.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 --2- 172.21.15.1:0/2253690099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2220102ef0 0x7f222010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:16.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 --2- 172.21.15.1:0/2253690099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2220102570 0x7f22201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 --2- 172.21.15.1:0/2253690099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 0x7f2220101200 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 -- 172.21.15.1:0/2253690099 >> 172.21.15.1:0/2253690099 conn(0x7f22200fc7d0 msgr2=0x7f22200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 -- 172.21.15.1:0/2253690099 shutdown_connections 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 -- 172.21.15.1:0/2253690099 wait complete. 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.625+0000 7f22255af640 1 Processor -- start 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f22255af640 1 -- start start 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f22255af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 0x7f22201984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f22255af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2220102570 0x7f2220198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f22255af640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2220102ef0 0x7f222019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f22255af640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2220110f50 con 0x7f2220100e20 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f22255af640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2220110dd0 con 0x7f2220102570 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f22255af640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22201110d0 con 0x7f2220102ef0 2024-09-16T00:26:16.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 0x7f22201984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 0x7f22201984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59858/0 (socket says 172.21.15.1:59858) 2024-09-16T00:26:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221effd640 1 -- 172.21.15.1:0/2251790159 learned_addr learned my addr 172.21.15.1:0/2251790159 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221e7fc640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2220102570 0x7f2220198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221f7fe640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2220102ef0 0x7f222019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221effd640 1 -- 172.21.15.1:0/2251790159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2220102ef0 msgr2=0x7f222019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221effd640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2220102ef0 0x7f222019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221effd640 1 -- 172.21.15.1:0/2251790159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2220102570 msgr2=0x7f2220198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221effd640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2220102570 0x7f2220198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221effd640 1 -- 172.21.15.1:0/2251790159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222019d470 con 0x7f2220100e20 2024-09-16T00:26:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.626+0000 7f221effd640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 0x7f22201984c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f2210003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.627+0000 7f2207fff640 1 -- 172.21.15.1:0/2251790159 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221003f070 con 0x7f2220100e20 2024-09-16T00:26:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.627+0000 7f2207fff640 1 -- 172.21.15.1:0/2251790159 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2210031c30 con 0x7f2220100e20 2024-09-16T00:26:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.627+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222019d700 con 0x7f2220100e20 2024-09-16T00:26:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.627+0000 7f2207fff640 1 -- 172.21.15.1:0/2251790159 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210035030 con 0x7f2220100e20 2024-09-16T00:26:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.627+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22201a5040 con 0x7f2220100e20 2024-09-16T00:26:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.628+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22201021b0 con 0x7f2220100e20 2024-09-16T00:26:16.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.629+0000 7f2207fff640 1 -- 172.21.15.1:0/2251790159 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f2210042070 con 0x7f2220100e20 2024-09-16T00:26:16.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.630+0000 7f2207fff640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22080a3240 0x7f22080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:16.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.630+0000 7f221e7fc640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22080a3240 0x7f22080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:16.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.630+0000 7f2207fff640 1 -- 172.21.15.1:0/2251790159 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f22100f01b0 con 0x7f2220100e20 2024-09-16T00:26:16.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.630+0000 7f221e7fc640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22080a3240 0x7f22080a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22200fff30 tx=0x7f221400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:16.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.633+0000 7f2207fff640 1 -- 172.21.15.1:0/2251790159 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2210031de0 con 0x7f2220100e20 2024-09-16T00:26:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:16.810+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f2220101200 con 0x7f2220100e20 2024-09-16T00:26:17.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.244+0000 7f2207fff640 1 -- 172.21.15.1:0/2251790159 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) ==== 171+0+0 (secure 0 0 0) 0x7f2220101200 con 0x7f2220100e20 2024-09-16T00:26:17.244 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.246+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22080a3240 msgr2=0x7f22080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.246+0000 7f22255af640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22080a3240 0x7f22080a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22200fff30 tx=0x7f221400b390 comp rx=0 tx=0).stop 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.246+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 msgr2=0x7f22201984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.246+0000 7f22255af640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 0x7f22201984c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f2210003c10 comp rx=0 tx=0).stop 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.247+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 shutdown_connections 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.247+0000 7f22255af640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22080a3240 0x7f22080a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.247+0000 7f22255af640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2220102ef0 0x7f222019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.247+0000 7f22255af640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2220102570 0x7f2220198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.247+0000 7f22255af640 1 --2- 172.21.15.1:0/2251790159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2220100e20 0x7f22201984c0 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f2210003c10 comp rx=0 tx=0).stop 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.247+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 >> 172.21.15.1:0/2251790159 conn(0x7f22200fc7d0 msgr2=0x7f222006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:17.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.247+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 shutdown_connections 2024-09-16T00:26:17.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:17.247+0000 7f22255af640 1 -- 172.21.15.1:0/2251790159 wait complete. 2024-09-16T00:26:17.544 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:17 smithi001 ceph-mon[30934]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:26:17.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:17 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2251790159' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:26:17.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:17 smithi019 ceph-mon[35186]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:26:17.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:17 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2251790159' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:26:17.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:17 smithi153 ceph-mon[35538]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:26:17.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:17 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2251790159' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:26:17.912 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T00:26:18.177 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:18.579 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:18 smithi001 ceph-mon[30934]: pgmap v275: 33 pgs: 9 creating+peering, 20 unknown, 4 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:18.579 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:18 smithi001 ceph-mon[30934]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:26:18.579 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:18 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2251790159' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:26:18.579 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:18 smithi001 ceph-mon[30934]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:26:18.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:18 smithi001 ceph-mon[30934]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:26:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:18 smithi019 ceph-mon[35186]: pgmap v275: 33 pgs: 9 creating+peering, 20 unknown, 4 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:18 smithi019 ceph-mon[35186]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:26:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:18 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2251790159' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:26:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:18 smithi019 ceph-mon[35186]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:26:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:18 smithi019 ceph-mon[35186]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:26:18.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:18 smithi153 ceph-mon[35538]: pgmap v275: 33 pgs: 9 creating+peering, 20 unknown, 4 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:18.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:18 smithi153 ceph-mon[35538]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:26:18.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:18 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2251790159' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:26:18.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:18 smithi153 ceph-mon[35538]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:26:18.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:18 smithi153 ceph-mon[35538]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.665+0000 7f5a4676f640 1 -- 172.21.15.1:0/1624754872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 msgr2=0x7f5a4010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.665+0000 7f5a4676f640 1 --2- 172.21.15.1:0/1624754872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 0x7f5a4010ea90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ca0 tx=0x7f5a3802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.665+0000 7f5a4676f640 1 -- 172.21.15.1:0/1624754872 shutdown_connections 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.665+0000 7f5a4676f640 1 --2- 172.21.15.1:0/1624754872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 0x7f5a4010ea90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.665+0000 7f5a4676f640 1 --2- 172.21.15.1:0/1624754872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40102790 0x7f5a401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.665+0000 7f5a4676f640 1 --2- 172.21.15.1:0/1624754872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a40101de0 0x7f5a401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.665+0000 7f5a4676f640 1 -- 172.21.15.1:0/1624754872 >> 172.21.15.1:0/1624754872 conn(0x7f5a400fd8c0 msgr2=0x7f5a400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.666+0000 7f5a4676f640 1 -- 172.21.15.1:0/1624754872 shutdown_connections 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.666+0000 7f5a4676f640 1 -- 172.21.15.1:0/1624754872 wait complete. 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.666+0000 7f5a4676f640 1 Processor -- start 2024-09-16T00:26:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.666+0000 7f5a4676f640 1 -- start start 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a4676f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40101de0 0x7f5a400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a4676f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a40102790 0x7f5a40074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a4676f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 0x7f5a40074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a4676f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40111f90 con 0x7f5a40106f10 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a4676f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40111e10 con 0x7f5a40102790 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a4676f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40112110 con 0x7f5a40101de0 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a44d0c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 0x7f5a40074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a44d0c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 0x7f5a40074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59876/0 (socket says 172.21.15.1:59876) 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a40102790 0x7f5a40074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a44d0c640 1 -- 172.21.15.1:0/2649675763 learned_addr learned my addr 172.21.15.1:0/2649675763 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a3ffff640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40101de0 0x7f5a400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a44d0c640 1 -- 172.21.15.1:0/2649675763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40101de0 msgr2=0x7f5a400744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a44d0c640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40101de0 0x7f5a400744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a44d0c640 1 -- 172.21.15.1:0/2649675763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a40102790 msgr2=0x7f5a40074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a44d0c640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a40102790 0x7f5a40074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a44d0c640 1 -- 172.21.15.1:0/2649675763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a401a6930 con 0x7f5a40106f10 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a37fff640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a40102790 0x7f5a40074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a44d0c640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 0x7f5a40074f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a38031a20 tx=0x7f5a38031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a3dffb640 1 -- 172.21.15.1:0/2649675763 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38003b70 con 0x7f5a40106f10 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a3dffb640 1 -- 172.21.15.1:0/2649675763 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a38003d10 con 0x7f5a40106f10 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a401a6bc0 con 0x7f5a40106f10 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.667+0000 7f5a3dffb640 1 -- 172.21.15.1:0/2649675763 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3802fc80 con 0x7f5a40106f10 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.668+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a401a7100 con 0x7f5a40106f10 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.669+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a401021c0 con 0x7f5a40106f10 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.671+0000 7f5a3dffb640 1 -- 172.21.15.1:0/2649675763 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f5a380046a0 con 0x7f5a40106f10 2024-09-16T00:26:18.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.671+0000 7f5a3dffb640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5a2c0a3240 0x7f5a2c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:18.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.671+0000 7f5a3ffff640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5a2c0a3240 0x7f5a2c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:18.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.671+0000 7f5a3dffb640 1 -- 172.21.15.1:0/2649675763 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7f5a380f0b60 con 0x7f5a40106f10 2024-09-16T00:26:18.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.672+0000 7f5a3ffff640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5a2c0a3240 0x7f5a2c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005fd0 tx=0x7f5a30005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:18.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.674+0000 7f5a3dffb640 1 -- 172.21.15.1:0/2649675763 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a380bdac0 con 0x7f5a40106f10 2024-09-16T00:26:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:18.888+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5a40075a20 con 0x7f5a40106f10 2024-09-16T00:26:19.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.267+0000 7f5a3dffb640 1 -- 172.21.15.1:0/2649675763 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5a40075a20 con 0x7f5a40106f10 2024-09-16T00:26:19.268 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T00:26:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5a2c0a3240 msgr2=0x7f5a2c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5a2c0a3240 0x7f5a2c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005fd0 tx=0x7f5a30005850 comp rx=0 tx=0).stop 2024-09-16T00:26:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 msgr2=0x7f5a40074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 0x7f5a40074f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a38031a20 tx=0x7f5a38031a50 comp rx=0 tx=0).stop 2024-09-16T00:26:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 shutdown_connections 2024-09-16T00:26:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a40106f10 0x7f5a40074f70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a40102790 0x7f5a40074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5a2c0a3240 0x7f5a2c0a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 --2- 172.21.15.1:0/2649675763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40101de0 0x7f5a400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 >> 172.21.15.1:0/2649675763 conn(0x7f5a400fd8c0 msgr2=0x7f5a400fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 shutdown_connections 2024-09-16T00:26:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:19.270+0000 7f5a4676f640 1 -- 172.21.15.1:0/2649675763 wait complete. 2024-09-16T00:26:19.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:19 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2649675763' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:26:19.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:19 smithi019 ceph-mon[35186]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:26:19.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:19 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2649675763' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:26:19.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:19 smithi001 ceph-mon[30934]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:26:19.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:19 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2649675763' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:26:19.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:19 smithi153 ceph-mon[35538]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:26:19.961 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-16T00:26:19.961 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-16T00:26:20.225 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:20.523 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:20 smithi001 ceph-mon[30934]: pgmap v278: 97 pgs: 17 creating+peering, 71 unknown, 9 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:20.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:20 smithi001 ceph-mon[30934]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:26:20.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:20 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2649675763' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:26:20.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:20 smithi001 ceph-mon[30934]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:26:20.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:20 smithi001 ceph-mon[30934]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:26:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:20 smithi019 ceph-mon[35186]: pgmap v278: 97 pgs: 17 creating+peering, 71 unknown, 9 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:20 smithi019 ceph-mon[35186]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:26:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:20 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2649675763' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:26:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:20 smithi019 ceph-mon[35186]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:26:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:20 smithi019 ceph-mon[35186]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:26:20.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:20 smithi153 ceph-mon[35538]: pgmap v278: 97 pgs: 17 creating+peering, 71 unknown, 9 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:20.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:20 smithi153 ceph-mon[35538]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:26:20.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:20 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2649675763' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:26:20.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:20 smithi153 ceph-mon[35538]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:26:20.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:20 smithi153 ceph-mon[35538]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:26:20.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.726+0000 7f00b4ee2640 1 -- 172.21.15.1:0/1150338890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 msgr2=0x7f00b010dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.726+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/1150338890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 0x7f00b010dfa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0098009900 tx=0x7f009802a410 comp rx=0 tx=0).stop 2024-09-16T00:26:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.726+0000 7f00b4ee2640 1 -- 172.21.15.1:0/1150338890 shutdown_connections 2024-09-16T00:26:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.726+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/1150338890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b010e4e0 0x7f00b01108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.726+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/1150338890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 0x7f00b010dfa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.726+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/1150338890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00b0100930 0x7f00b0100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.726+0000 7f00b4ee2640 1 -- 172.21.15.1:0/1150338890 >> 172.21.15.1:0/1150338890 conn(0x7f00b00fc7a0 msgr2=0x7f00b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.726+0000 7f00b4ee2640 1 -- 172.21.15.1:0/1150338890 shutdown_connections 2024-09-16T00:26:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.726+0000 7f00b4ee2640 1 -- 172.21.15.1:0/1150338890 wait complete. 2024-09-16T00:26:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00b4ee2640 1 Processor -- start 2024-09-16T00:26:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00b4ee2640 1 -- start start 2024-09-16T00:26:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00b4ee2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00b0100930 0x7f00b019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00b4ee2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 0x7f00b019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00b4ee2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b010e4e0 0x7f00b019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00b4ee2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0114160 con 0x7f00b01012e0 2024-09-16T00:26:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00b4ee2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0113fe0 con 0x7f00b0100930 2024-09-16T00:26:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00b4ee2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00b01142e0 con 0x7f00b010e4e0 2024-09-16T00:26:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00add9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 0x7f00b019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00add9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 0x7f00b019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44646/0 (socket says 172.21.15.1:44646) 2024-09-16T00:26:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00add9b640 1 -- 172.21.15.1:0/3986420217 learned_addr learned my addr 172.21.15.1:0/3986420217 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00ae59c640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00b0100930 0x7f00b019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00aed9d640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b010e4e0 0x7f00b019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00add9b640 1 -- 172.21.15.1:0/3986420217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b010e4e0 msgr2=0x7f00b019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00add9b640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b010e4e0 0x7f00b019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:20.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00add9b640 1 -- 172.21.15.1:0/3986420217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00b0100930 msgr2=0x7f00b019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:20.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00add9b640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00b0100930 0x7f00b019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:20.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.727+0000 7f00add9b640 1 -- 172.21.15.1:0/3986420217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b019f650 con 0x7f00b01012e0 2024-09-16T00:26:20.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.728+0000 7f00add9b640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 0x7f00b019abe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0098006010 tx=0x7f00980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:20.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.728+0000 7f00a77fe640 1 -- 172.21.15.1:0/3986420217 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009803f030 con 0x7f00b01012e0 2024-09-16T00:26:20.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.728+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b019f8e0 con 0x7f00b01012e0 2024-09-16T00:26:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.728+0000 7f00a77fe640 1 -- 172.21.15.1:0/3986420217 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0098003ce0 con 0x7f00b01012e0 2024-09-16T00:26:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.728+0000 7f00a77fe640 1 -- 172.21.15.1:0/3986420217 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098035070 con 0x7f00b01012e0 2024-09-16T00:26:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.728+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b01a7220 con 0x7f00b01012e0 2024-09-16T00:26:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.729+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b0100d10 con 0x7f00b01012e0 2024-09-16T00:26:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.730+0000 7f00a77fe640 1 -- 172.21.15.1:0/3986420217 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f0098042030 con 0x7f00b01012e0 2024-09-16T00:26:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.731+0000 7f00a77fe640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f009c0a3240 0x7f009c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.731+0000 7f00ae59c640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f009c0a3240 0x7f009c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.731+0000 7f00a77fe640 1 -- 172.21.15.1:0/3986420217 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f00980efe90 con 0x7f00b01012e0 2024-09-16T00:26:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.731+0000 7f00ae59c640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f009c0a3240 0x7f009c0a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00a00058c0 tx=0x7f00a000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.735+0000 7f00a77fe640 1 -- 172.21.15.1:0/3986420217 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00980bcd20 con 0x7f00b01012e0 2024-09-16T00:26:20.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:20.919+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f00b01096a0 con 0x7f00b01012e0 2024-09-16T00:26:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.278+0000 7f00a77fe640 1 -- 172.21.15.1:0/3986420217 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) ==== 157+0+0 (secure 0 0 0) 0x7f00980bc620 con 0x7f00b01012e0 2024-09-16T00:26:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f009c0a3240 msgr2=0x7f009c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f009c0a3240 0x7f009c0a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00a00058c0 tx=0x7f00a000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 msgr2=0x7f00b019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 0x7f00b019abe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0098006010 tx=0x7f00980042f0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 shutdown_connections 2024-09-16T00:26:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f009c0a3240 0x7f009c0a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b010e4e0 0x7f00b019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00b01012e0 0x7f00b019abe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 --2- 172.21.15.1:0/3986420217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00b0100930 0x7f00b019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 >> 172.21.15.1:0/3986420217 conn(0x7f00b00fc7a0 msgr2=0x7f00b010f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 shutdown_connections 2024-09-16T00:26:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:21.281+0000 7f00b4ee2640 1 -- 172.21.15.1:0/3986420217 wait complete. 2024-09-16T00:26:21.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:21 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] up:active 2024-09-16T00:26:21.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:21 smithi019 ceph-mon[35186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:21.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:21 smithi019 ceph-mon[35186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:21.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:21 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3986420217' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:26:21.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:21 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] up:active 2024-09-16T00:26:21.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:21 smithi001 ceph-mon[30934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:21.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:21 smithi001 ceph-mon[30934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:21.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:21 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3986420217' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:26:21.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:21 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] up:active 2024-09-16T00:26:21.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:21 smithi153 ceph-mon[35538]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:21 smithi153 ceph-mon[35538]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:21.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:21 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3986420217' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:26:21.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T00:26:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:22.531 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:22 smithi001 ceph-mon[30934]: pgmap v280: 97 pgs: 17 creating+peering, 23 unknown, 57 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 490 B/s wr, 1 op/s 2024-09-16T00:26:22.531 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:22 smithi001 ceph-mon[30934]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:26:22.531 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:22 smithi001 ceph-mon[30934]: Cluster is now healthy 2024-09-16T00:26:22.531 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:22 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3986420217' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:26:22.532 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:22 smithi001 ceph-mon[30934]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:26:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:22 smithi019 ceph-mon[35186]: pgmap v280: 97 pgs: 17 creating+peering, 23 unknown, 57 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 490 B/s wr, 1 op/s 2024-09-16T00:26:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:22 smithi019 ceph-mon[35186]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:26:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:22 smithi019 ceph-mon[35186]: Cluster is now healthy 2024-09-16T00:26:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:22 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3986420217' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:26:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:22 smithi019 ceph-mon[35186]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:26:22.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:22 smithi153 ceph-mon[35538]: pgmap v280: 97 pgs: 17 creating+peering, 23 unknown, 57 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 490 B/s wr, 1 op/s 2024-09-16T00:26:22.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:22 smithi153 ceph-mon[35538]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:26:22.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:22 smithi153 ceph-mon[35538]: Cluster is now healthy 2024-09-16T00:26:22.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:22 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3986420217' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:26:22.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:22 smithi153 ceph-mon[35538]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 -- 172.21.15.1:0/3450281152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2814102d30 msgr2=0x7f2814103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 --2- 172.21.15.1:0/3450281152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2814102d30 0x7f2814103110 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2804009930 tx=0x7f280402a420 comp rx=0 tx=0).stop 2024-09-16T00:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 -- 172.21.15.1:0/3450281152 shutdown_connections 2024-09-16T00:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 --2- 172.21.15.1:0/3450281152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28141082a0 0x7f281410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 --2- 172.21.15.1:0/3450281152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28141036e0 0x7f2814107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 --2- 172.21.15.1:0/3450281152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2814102d30 0x7f2814103110 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 -- 172.21.15.1:0/3450281152 >> 172.21.15.1:0/3450281152 conn(0x7f28140fc7d0 msgr2=0x7f28140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 -- 172.21.15.1:0/3450281152 shutdown_connections 2024-09-16T00:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 -- 172.21.15.1:0/3450281152 wait complete. 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.727+0000 7f281a09f640 1 Processor -- start 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f281a09f640 1 -- start start 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f281a09f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2814102d30 0x7f2814100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f281a09f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28141036e0 0x7f2814100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f281a09f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28141082a0 0x7f2814101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f281a09f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f28141120e0 con 0x7f28141082a0 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f281a09f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2814111f60 con 0x7f28141036e0 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f281a09f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2814112260 con 0x7f2814102d30 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f2813fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28141082a0 0x7f2814101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f2813fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28141082a0 0x7f2814101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44652/0 (socket says 172.21.15.1:44652) 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f2813fff640 1 -- 172.21.15.1:0/2022791947 learned_addr learned my addr 172.21.15.1:0/2022791947 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f28137fe640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2814102d30 0x7f2814100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f2812ffd640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28141036e0 0x7f2814100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f2813fff640 1 -- 172.21.15.1:0/2022791947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2814102d30 msgr2=0x7f2814100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f2813fff640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2814102d30 0x7f2814100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f2813fff640 1 -- 172.21.15.1:0/2022791947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28141036e0 msgr2=0x7f2814100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f2813fff640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28141036e0 0x7f2814100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.728+0000 7f2813fff640 1 -- 172.21.15.1:0/2022791947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28141a6a60 con 0x7f28141082a0 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.729+0000 7f2813fff640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28141082a0 0x7f2814101410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f280c00eaa0 tx=0x7f280c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.729+0000 7f2810ff9640 1 -- 172.21.15.1:0/2022791947 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c00cdb0 con 0x7f28141082a0 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.729+0000 7f2810ff9640 1 -- 172.21.15.1:0/2022791947 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280c005020 con 0x7f28141082a0 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.729+0000 7f2810ff9640 1 -- 172.21.15.1:0/2022791947 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0116b0 con 0x7f28141082a0 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.729+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28141a6d50 con 0x7f28141082a0 2024-09-16T00:26:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.729+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141a7240 con 0x7f28141082a0 2024-09-16T00:26:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.730+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281406a850 con 0x7f28141082a0 2024-09-16T00:26:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.731+0000 7f2810ff9640 1 -- 172.21.15.1:0/2022791947 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f280c005560 con 0x7f28141082a0 2024-09-16T00:26:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.732+0000 7f2810ff9640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27fc0a32c0 0x7f27fc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.732+0000 7f2810ff9640 1 -- 172.21.15.1:0/2022791947 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7f280c0cfe30 con 0x7f28141082a0 2024-09-16T00:26:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.732+0000 7f28137fe640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27fc0a32c0 0x7f27fc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.732+0000 7f28137fe640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27fc0a32c0 0x7f27fc0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2804006010 tx=0x7f2804000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.735+0000 7f2810ff9640 1 -- 172.21.15.1:0/2022791947 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f280c016030 con 0x7f28141082a0 2024-09-16T00:26:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:22.913+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2814103110 con 0x7f28141082a0 2024-09-16T00:26:23.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:23 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2022791947' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:26:23.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:23 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2022791947' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:26:23.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:23 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2022791947' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:26:24.107 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:26:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b[65807]: 2024-09-16T00:26:23.850+0000 7fa6359e1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T00:26:24.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.279+0000 7f2810ff9640 1 -- 172.21.15.1:0/2022791947 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) ==== 228+0+0 (secure 0 0 0) 0x7f2814103110 con 0x7f28141082a0 2024-09-16T00:26:24.279 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' created 2024-09-16T00:26:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27fc0a32c0 msgr2=0x7f27fc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27fc0a32c0 0x7f27fc0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2804006010 tx=0x7f2804000bc0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28141082a0 msgr2=0x7f2814101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28141082a0 0x7f2814101410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f280c00eaa0 tx=0x7f280c00ef60 comp rx=0 tx=0).stop 2024-09-16T00:26:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 shutdown_connections 2024-09-16T00:26:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28141082a0 0x7f2814101410 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28141036e0 0x7f2814100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27fc0a32c0 0x7f27fc0a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 --2- 172.21.15.1:0/2022791947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2814102d30 0x7f2814100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.283+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 >> 172.21.15.1:0/2022791947 conn(0x7f28140fc7d0 msgr2=0x7f2814106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.284+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 shutdown_connections 2024-09-16T00:26:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:24.284+0000 7f281a09f640 1 -- 172.21.15.1:0/2022791947 wait complete. 2024-09-16T00:26:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: pgmap v282: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-09-16T00:26:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:26:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-16T00:26:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-16T00:26:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-16T00:26:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-16T00:26:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-16T00:26:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-09-16T00:26:24.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-16T00:26:24.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-09-16T00:26:24.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]: dispatch 2024-09-16T00:26:24.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: pgmap v282: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-09-16T00:26:24.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:26:24.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-16T00:26:24.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-16T00:26:24.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-16T00:26:24.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-16T00:26:24.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-16T00:26:24.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-09-16T00:26:24.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-16T00:26:24.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-09-16T00:26:24.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]: dispatch 2024-09-16T00:26:24.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: pgmap v282: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-09-16T00:26:24.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:26:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-16T00:26:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-16T00:26:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-16T00:26:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-16T00:26:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-16T00:26:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-09-16T00:26:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-16T00:26:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-09-16T00:26:24.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]: dispatch 2024-09-16T00:26:25.237 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-16T00:26:25.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2022791947' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:26:25.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-16T00:26:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-16T00:26:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-16T00:26:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-16T00:26:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-16T00:26:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-09-16T00:26:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-16T00:26:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-09-16T00:26:25.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]': finished 2024-09-16T00:26:25.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:26:25.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: pgmap v285: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2024-09-16T00:26:25.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:25 smithi001 ceph-mon[30934]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:26:25.517 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2022791947' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:26:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-16T00:26:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-16T00:26:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-16T00:26:25.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-16T00:26:25.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-16T00:26:25.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-09-16T00:26:25.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-16T00:26:25.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-09-16T00:26:25.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]': finished 2024-09-16T00:26:25.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:26:25.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: pgmap v285: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2024-09-16T00:26:25.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:25 smithi019 ceph-mon[35186]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:26:25.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2022791947' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]': finished 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: pgmap v285: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2024-09-16T00:26:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:25 smithi153 ceph-mon[35538]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 -- 172.21.15.1:0/831144771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 msgr2=0x7f40681031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 --2- 172.21.15.1:0/831144771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 0x7f40681031b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4058009930 tx=0x7f405802a420 comp rx=0 tx=0).stop 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 -- 172.21.15.1:0/831144771 shutdown_connections 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 --2- 172.21.15.1:0/831144771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068108300 0x7f406810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 --2- 172.21.15.1:0/831144771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103780 0x7f4068107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 --2- 172.21.15.1:0/831144771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 0x7f40681031b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 -- 172.21.15.1:0/831144771 >> 172.21.15.1:0/831144771 conn(0x7f40680fc7d0 msgr2=0x7f40680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 -- 172.21.15.1:0/831144771 shutdown_connections 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 -- 172.21.15.1:0/831144771 wait complete. 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.107+0000 7f406e270640 1 Processor -- start 2024-09-16T00:26:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f406e270640 1 -- start start 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f406e270640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 0x7f4068198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f406e270640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068103780 0x7f4068198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f406e270640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068108300 0x7f406819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f406e270640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4068112110 con 0x7f4068102dd0 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f406e270640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4068111f90 con 0x7f4068103780 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f406e270640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4068112290 con 0x7f4068108300 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f4067fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 0x7f4068198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f4067fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 0x7f4068198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44676/0 (socket says 172.21.15.1:44676) 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f4067fff640 1 -- 172.21.15.1:0/4136308823 learned_addr learned my addr 172.21.15.1:0/4136308823 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f406c80d640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068108300 0x7f406819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f405ffff640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068103780 0x7f4068198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f4067fff640 1 -- 172.21.15.1:0/4136308823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068108300 msgr2=0x7f406819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f4067fff640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068108300 0x7f406819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f4067fff640 1 -- 172.21.15.1:0/4136308823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068103780 msgr2=0x7f4068198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f4067fff640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068103780 0x7f4068198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f4067fff640 1 -- 172.21.15.1:0/4136308823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406819d5e0 con 0x7f4068102dd0 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.108+0000 7f4067fff640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 0x7f4068198630 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4058009230 tx=0x7f4058004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.109+0000 7f4065ffb640 1 -- 172.21.15.1:0/4136308823 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405803f030 con 0x7f4068102dd0 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.109+0000 7f4065ffb640 1 -- 172.21.15.1:0/4136308823 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4058003ad0 con 0x7f4068102dd0 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.109+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406819d870 con 0x7f4068102dd0 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.109+0000 7f4065ffb640 1 -- 172.21.15.1:0/4136308823 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058035070 con 0x7f4068102dd0 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.109+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40681a51b0 con 0x7f4068102dd0 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.110+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40681031b0 con 0x7f4068102dd0 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.112+0000 7f4065ffb640 1 -- 172.21.15.1:0/4136308823 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f4058042030 con 0x7f4068102dd0 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.112+0000 7f4065ffb640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40540a32c0 0x7f40540a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.113+0000 7f405ffff640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40540a32c0 0x7f40540a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.113+0000 7f4065ffb640 1 -- 172.21.15.1:0/4136308823 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9039+0+0 (secure 0 0 0) 0x7f40580f0610 con 0x7f4068102dd0 2024-09-16T00:26:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.113+0000 7f405ffff640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40540a32c0 0x7f40540a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4050005fd0 tx=0x7f4050005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.116+0000 7f4065ffb640 1 -- 172.21.15.1:0/4136308823 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40580bcfb0 con 0x7f4068102dd0 2024-09-16T00:26:26.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:26.296+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f4068199990 con 0x7f4068102dd0 2024-09-16T00:26:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:26 smithi019 ceph-mon[35186]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:26:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:26 smithi019 ceph-mon[35186]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:26:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:26 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4136308823' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:26:26.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:26 smithi001 ceph-mon[30934]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:26:26.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:26 smithi001 ceph-mon[30934]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:26:26.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:26 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4136308823' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:26:26.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:26 smithi153 ceph-mon[35538]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:26:26.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:26 smithi153 ceph-mon[35538]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:26:26.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:26 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4136308823' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:26:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.288+0000 7f4065ffb640 1 -- 172.21.15.1:0/4136308823 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f40580bc8b0 con 0x7f4068102dd0 2024-09-16T00:26:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 4 allow_ec_overwrites to true 2024-09-16T00:26:27.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.291+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40540a32c0 msgr2=0x7f40540a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:27.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.291+0000 7f406e270640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40540a32c0 0x7f40540a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4050005fd0 tx=0x7f4050005850 comp rx=0 tx=0).stop 2024-09-16T00:26:27.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 msgr2=0x7f4068198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:27.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 0x7f4068198630 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4058009230 tx=0x7f4058004270 comp rx=0 tx=0).stop 2024-09-16T00:26:27.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 shutdown_connections 2024-09-16T00:26:27.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40540a32c0 0x7f40540a5780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:27.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068108300 0x7f406819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:27.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068103780 0x7f4068198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:27.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 --2- 172.21.15.1:0/4136308823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4068102dd0 0x7f4068198630 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 >> 172.21.15.1:0/4136308823 conn(0x7f40680fc7d0 msgr2=0x7f4068105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 shutdown_connections 2024-09-16T00:26:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:27.292+0000 7f406e270640 1 -- 172.21.15.1:0/4136308823 wait complete. 2024-09-16T00:26:27.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-16T00:26:28.230 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:28.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:28 smithi001 ceph-mon[30934]: pgmap v288: 129 pgs: 15 creating+peering, 3 peering, 9 unknown, 102 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-16T00:26:28.551 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:28 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4136308823' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:26:28.551 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:28 smithi001 ceph-mon[30934]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:26:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:28 smithi019 ceph-mon[35186]: pgmap v288: 129 pgs: 15 creating+peering, 3 peering, 9 unknown, 102 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-16T00:26:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:28 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4136308823' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:26:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:28 smithi019 ceph-mon[35186]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:26:28.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:28 smithi153 ceph-mon[35538]: pgmap v288: 129 pgs: 15 creating+peering, 3 peering, 9 unknown, 102 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-16T00:26:28.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:28 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4136308823' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:26:28.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:28 smithi153 ceph-mon[35538]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:26:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.764+0000 7f2e0c827640 1 -- 172.21.15.1:0/3368059824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 msgr2=0x7f2e08106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.764+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3368059824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 0x7f2e08106e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009930 tx=0x7f2df802a420 comp rx=0 tx=0).stop 2024-09-16T00:26:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.765+0000 7f2e0c827640 1 -- 172.21.15.1:0/3368059824 shutdown_connections 2024-09-16T00:26:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.765+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3368059824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e08101420 0x7f2e0810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.765+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3368059824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e08100a80 0x7f2e08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.765+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3368059824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 0x7f2e08106e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.765+0000 7f2e0c827640 1 -- 172.21.15.1:0/3368059824 >> 172.21.15.1:0/3368059824 conn(0x7f2e080fc7a0 msgr2=0x7f2e080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.765+0000 7f2e0c827640 1 -- 172.21.15.1:0/3368059824 shutdown_connections 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.765+0000 7f2e0c827640 1 -- 172.21.15.1:0/3368059824 wait complete. 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.765+0000 7f2e0c827640 1 Processor -- start 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e0c827640 1 -- start start 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e0c827640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e08100a80 0x7f2e081984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e0c827640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e08101420 0x7f2e081989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e0c827640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 0x7f2e0819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e0c827640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08111fa0 con 0x7f2e08106a80 2024-09-16T00:26:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e0c827640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08111e20 con 0x7f2e08100a80 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e0c827640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08112120 con 0x7f2e08101420 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e06d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 0x7f2e0819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e06d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 0x7f2e0819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44698/0 (socket says 172.21.15.1:44698) 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e06d9d640 1 -- 172.21.15.1:0/3142296039 learned_addr learned my addr 172.21.15.1:0/3142296039 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e05d9b640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e08101420 0x7f2e081989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e0659c640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e08100a80 0x7f2e081984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e06d9d640 1 -- 172.21.15.1:0/3142296039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e08101420 msgr2=0x7f2e081989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e06d9d640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e08101420 0x7f2e081989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e06d9d640 1 -- 172.21.15.1:0/3142296039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e08100a80 msgr2=0x7f2e081984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e06d9d640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e08100a80 0x7f2e081984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.766+0000 7f2e06d9d640 1 -- 172.21.15.1:0/3142296039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0819d450 con 0x7f2e08106a80 2024-09-16T00:26:28.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.767+0000 7f2e06d9d640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 0x7f2e0819cd70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009fd0 tx=0x7f2e0000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.767+0000 7f2df77fe640 1 -- 172.21.15.1:0/3142296039 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00017070 con 0x7f2e08106a80 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.767+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0819d740 con 0x7f2e08106a80 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.767+0000 7f2df77fe640 1 -- 172.21.15.1:0/3142296039 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e00003e50 con 0x7f2e08106a80 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.767+0000 7f2df77fe640 1 -- 172.21.15.1:0/3142296039 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000106c0 con 0x7f2e08106a80 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.767+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e081a5020 con 0x7f2e08106a80 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.768+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e08100ee0 con 0x7f2e08106a80 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.769+0000 7f2df77fe640 1 -- 172.21.15.1:0/3142296039 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f2e00003970 con 0x7f2e08106a80 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.769+0000 7f2df77fe640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2df00a3240 0x7f2df00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.770+0000 7f2e0659c640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2df00a3240 0x7f2df00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.770+0000 7f2df77fe640 1 -- 172.21.15.1:0/3142296039 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9039+0+0 (secure 0 0 0) 0x7f2e000cf950 con 0x7f2e08106a80 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.770+0000 7f2e0659c640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2df00a3240 0x7f2df00a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009900 tx=0x7f2df80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.774+0000 7f2df77fe640 1 -- 172.21.15.1:0/3142296039 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e0009bca0 con 0x7f2e08106a80 2024-09-16T00:26:28.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:28.989+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f2e08108ff0 con 0x7f2e08106a80 2024-09-16T00:26:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.293+0000 7f2df77fe640 1 -- 172.21.15.1:0/3142296039 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f2e0009ffa0 con 0x7f2e08106a80 2024-09-16T00:26:29.294 INFO:teuthology.orchestra.run.smithi001.stderr:added data pool 4 to fsmap 2024-09-16T00:26:29.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2df00a3240 msgr2=0x7f2df00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2df00a3240 0x7f2df00a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009900 tx=0x7f2df80058d0 comp rx=0 tx=0).stop 2024-09-16T00:26:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 msgr2=0x7f2e0819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 0x7f2e0819cd70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009fd0 tx=0x7f2e0000b260 comp rx=0 tx=0).stop 2024-09-16T00:26:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 shutdown_connections 2024-09-16T00:26:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106a80 0x7f2e0819cd70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:29.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2df00a3240 0x7f2df00a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:29.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e08101420 0x7f2e081989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:29.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 --2- 172.21.15.1:0/3142296039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e08100a80 0x7f2e081984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:29.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.295+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 >> 172.21.15.1:0/3142296039 conn(0x7f2e080fc7a0 msgr2=0x7f2e080fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:29.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.296+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 shutdown_connections 2024-09-16T00:26:29.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:29.296+0000 7f2e0c827640 1 -- 172.21.15.1:0/3142296039 wait complete. 2024-09-16T00:26:29.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:29 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3142296039' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:26:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:29 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3142296039' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:26:29.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:29 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3142296039' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:26:30.032 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:26:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:30.600 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:30 smithi001 ceph-mon[30934]: pgmap v290: 129 pgs: 15 creating+peering, 3 peering, 4 unknown, 107 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 400 B/s, 1 objects/s recovering 2024-09-16T00:26:30.600 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:30 smithi001 ceph-mon[30934]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:26:30.600 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:30 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3142296039' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:26:30.601 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:30 smithi001 ceph-mon[30934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:30.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:30 smithi019 ceph-mon[35186]: pgmap v290: 129 pgs: 15 creating+peering, 3 peering, 4 unknown, 107 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 400 B/s, 1 objects/s recovering 2024-09-16T00:26:30.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:30 smithi019 ceph-mon[35186]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:26:30.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:30 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3142296039' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:26:30.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:30 smithi019 ceph-mon[35186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:30.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:30 smithi153 ceph-mon[35538]: pgmap v290: 129 pgs: 15 creating+peering, 3 peering, 4 unknown, 107 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 400 B/s, 1 objects/s recovering 2024-09-16T00:26:30.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:30 smithi153 ceph-mon[35538]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:26:30.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:30 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3142296039' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:26:30.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:30 smithi153 ceph-mon[35538]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.864+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3233580853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0102790 msgr2=0x7f8cc01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.864+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3233580853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0102790 0x7f8cc01067e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009900 tx=0x7f8ca802a410 comp rx=0 tx=0).stop 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.864+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3233580853 shutdown_connections 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.864+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3233580853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0106f10 0x7f8cc010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.864+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3233580853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0102790 0x7f8cc01067e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.864+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3233580853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0101de0 0x7f8cc01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.864+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3233580853 >> 172.21.15.1:0/3233580853 conn(0x7f8cc00fd8c0 msgr2=0x7f8cc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.864+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3233580853 shutdown_connections 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.864+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3233580853 wait complete. 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.865+0000 7f8cc4ae5640 1 Processor -- start 2024-09-16T00:26:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.865+0000 7f8cc4ae5640 1 -- start start 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.865+0000 7f8cc4ae5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0101de0 0x7f8cc00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cc4ae5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0102790 0x7f8cc0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cc4ae5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0106f10 0x7f8cc0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cc4ae5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0111f90 con 0x7f8cc0101de0 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cc4ae5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0111e10 con 0x7f8cc0102790 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cc4ae5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0112110 con 0x7f8cc0106f10 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0101de0 0x7f8cc00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0101de0 0x7f8cc00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50218/0 (socket says 172.21.15.1:50218) 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbe59c640 1 -- 172.21.15.1:0/3504741684 learned_addr learned my addr 172.21.15.1:0/3504741684 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cb7fff640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0102790 0x7f8cc0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbed9d640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0106f10 0x7f8cc0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbe59c640 1 -- 172.21.15.1:0/3504741684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0106f10 msgr2=0x7f8cc0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbe59c640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0106f10 0x7f8cc0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbe59c640 1 -- 172.21.15.1:0/3504741684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0102790 msgr2=0x7f8cc0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbe59c640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0102790 0x7f8cc0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbe59c640 1 -- 172.21.15.1:0/3504741684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc01a68d0 con 0x7f8cc0101de0 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cbe59c640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0101de0 0x7f8cc00744f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b4a0 tx=0x7f8cb000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cb77fe640 1 -- 172.21.15.1:0/3504741684 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0014070 con 0x7f8cc0101de0 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc01a6bc0 con 0x7f8cc0101de0 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cb77fe640 1 -- 172.21.15.1:0/3504741684 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb0010af0 con 0x7f8cc0101de0 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cb77fe640 1 -- 172.21.15.1:0/3504741684 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0017d10 con 0x7f8cc0101de0 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.866+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc01a70d0 con 0x7f8cc0101de0 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.867+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc01021c0 con 0x7f8cc0101de0 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.869+0000 7f8cb77fe640 1 -- 172.21.15.1:0/3504741684 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f8cb0017070 con 0x7f8cc0101de0 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.869+0000 7f8cb77fe640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cac0a3240 0x7f8cac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.870+0000 7f8cb7fff640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cac0a3240 0x7f8cac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.870+0000 7f8cb77fe640 1 -- 172.21.15.1:0/3504741684 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f8cb00cefc0 con 0x7f8cc0101de0 2024-09-16T00:26:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.870+0000 7f8cb7fff640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cac0a3240 0x7f8cac0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8000c00 tx=0x7f8ca80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:30.874+0000 7f8cb77fe640 1 -- 172.21.15.1:0/3504741684 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cb009b9a0 con 0x7f8cc0101de0 2024-09-16T00:26:31.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.085+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cc0109b60 con 0x7f8cc0101de0 2024-09-16T00:26:31.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.086+0000 7f8cb77fe640 1 -- 172.21.15.1:0/3504741684 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11821 (secure 0 0 0) 0x7f8cb009b2a0 con 0x7f8cc0101de0 2024-09-16T00:26:31.087 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:31.087 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"btime":"2024-09-16T00:26:29:286635+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:26:28.990397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24506,"qdb_cluster":[24506]},"id":1}]} 2024-09-16T00:26:31.087 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 17 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.088+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cac0a3240 msgr2=0x7f8cac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.088+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cac0a3240 0x7f8cac0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8000c00 tx=0x7f8ca80058d0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.088+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0101de0 msgr2=0x7f8cc00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.088+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0101de0 0x7f8cc00744f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b4a0 tx=0x7f8cb000b960 comp rx=0 tx=0).stop 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.089+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 shutdown_connections 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.089+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cac0a3240 0x7f8cac0a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.089+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0106f10 0x7f8cc0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.089+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0102790 0x7f8cc0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.089+0000 7f8cc4ae5640 1 --2- 172.21.15.1:0/3504741684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc0101de0 0x7f8cc00744f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.089+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 >> 172.21.15.1:0/3504741684 conn(0x7f8cc00fd8c0 msgr2=0x7f8cc00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.089+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 shutdown_connections 2024-09-16T00:26:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:31.089+0000 7f8cc4ae5640 1 -- 172.21.15.1:0/3504741684 wait complete. 2024-09-16T00:26:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:31 smithi019 ceph-mon[35186]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 320 B/s, 0 objects/s recovering 2024-09-16T00:26:31.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:31 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3504741684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:31.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:31 smithi001 ceph-mon[30934]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 320 B/s, 0 objects/s recovering 2024-09-16T00:26:31.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:31 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3504741684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:31.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:31 smithi153 ceph-mon[35538]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 320 B/s, 0 objects/s recovering 2024-09-16T00:26:31.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:31 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3504741684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:31.765 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:26:32.023 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:32 smithi001 ceph-mon[30934]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:26:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:32 smithi001 ceph-mon[30934]: Cluster is now healthy 2024-09-16T00:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 -- 172.21.15.1:0/2647342445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08101420 msgr2=0x7f3c0810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/2647342445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08101420 0x7f3c0810d950 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009cd0 tx=0x7f3c0002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 -- 172.21.15.1:0/2647342445 shutdown_connections 2024-09-16T00:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/2647342445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08101420 0x7f3c0810d950 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/2647342445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c08100a80 0x7f3c08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/2647342445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c08106a80 0x7f3c08106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 -- 172.21.15.1:0/2647342445 >> 172.21.15.1:0/2647342445 conn(0x7f3c080fc7a0 msgr2=0x7f3c080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 -- 172.21.15.1:0/2647342445 shutdown_connections 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 -- 172.21.15.1:0/2647342445 wait complete. 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 Processor -- start 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.558+0000 7f3c0dab8640 1 -- start start 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c0dab8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c08100a80 0x7f3c080fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c0dab8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c08101420 0x7f3c080fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c0dab8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08106a80 0x7f3c080fd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c0dab8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c08110ed0 con 0x7f3c08106a80 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c0dab8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c08110d50 con 0x7f3c08100a80 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c0dab8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c08111050 con 0x7f3c08101420 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08106a80 0x7f3c080fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08106a80 0x7f3c080fd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50242/0 (socket says 172.21.15.1:50242) 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c07fff640 1 -- 172.21.15.1:0/3824093286 learned_addr learned my addr 172.21.15.1:0/3824093286 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c06ffd640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c08101420 0x7f3c080fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c077fe640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c08100a80 0x7f3c080fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c07fff640 1 -- 172.21.15.1:0/3824093286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c08101420 msgr2=0x7f3c080fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c07fff640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c08101420 0x7f3c080fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c07fff640 1 -- 172.21.15.1:0/3824093286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c08100a80 msgr2=0x7f3c080fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c07fff640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c08100a80 0x7f3c080fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c07fff640 1 -- 172.21.15.1:0/3824093286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c081a6940 con 0x7f3c08106a80 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c07fff640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08106a80 0x7f3c080fd790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009ca0 tx=0x7f3c00003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.559+0000 7f3c04ff9640 1 -- 172.21.15.1:0/3824093286 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00004240 con 0x7f3c08106a80 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.560+0000 7f3c04ff9640 1 -- 172.21.15.1:0/3824093286 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c000305a0 con 0x7f3c08106a80 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.560+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c081a6bd0 con 0x7f3c08106a80 2024-09-16T00:26:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.560+0000 7f3c04ff9640 1 -- 172.21.15.1:0/3824093286 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0003a660 con 0x7f3c08106a80 2024-09-16T00:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.560+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c081a7110 con 0x7f3c08106a80 2024-09-16T00:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.561+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c08100ee0 con 0x7f3c08106a80 2024-09-16T00:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.562+0000 7f3c04ff9640 1 -- 172.21.15.1:0/3824093286 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f3c00030750 con 0x7f3c08106a80 2024-09-16T00:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.562+0000 7f3c04ff9640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3bf00a3240 0x7f3bf00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.563+0000 7f3c077fe640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3bf00a3240 0x7f3bf00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.563+0000 7f3c04ff9640 1 -- 172.21.15.1:0/3824093286 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f3c0002e030 con 0x7f3c08106a80 2024-09-16T00:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.563+0000 7f3c077fe640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3bf00a3240 0x7f3bf00a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80058c0 tx=0x7f3bf800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.566+0000 7f3c04ff9640 1 -- 172.21.15.1:0/3824093286 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c000bd930 con 0x7f3c08106a80 2024-09-16T00:26:32.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:32 smithi019 ceph-mon[35186]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:26:32.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:32 smithi019 ceph-mon[35186]: Cluster is now healthy 2024-09-16T00:26:32.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:32 smithi153 ceph-mon[35538]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:26:32.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:32 smithi153 ceph-mon[35538]: Cluster is now healthy 2024-09-16T00:26:32.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.744+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c0810a000 con 0x7f3c08106a80 2024-09-16T00:26:32.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.746+0000 7f3c04ff9640 1 -- 172.21.15.1:0/3824093286 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24601 (secure 0 0 0) 0x7f3c000bd230 con 0x7f3c08106a80 2024-09-16T00:26:32.746 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:32.747 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":89,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:26:28.990229+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:26:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3bf00a3240 msgr2=0x7f3bf00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3bf00a3240 0x7f3bf00a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80058c0 tx=0x7f3bf800a2d0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08106a80 msgr2=0x7f3c080fd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08106a80 0x7f3c080fd790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009ca0 tx=0x7f3c00003ba0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 shutdown_connections 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c08106a80 0x7f3c080fd790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3bf00a3240 0x7f3bf00a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c08101420 0x7f3c080fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 --2- 172.21.15.1:0/3824093286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c08100a80 0x7f3c080fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.748+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 >> 172.21.15.1:0/3824093286 conn(0x7f3c080fc7a0 msgr2=0x7f3c0806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.749+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 shutdown_connections 2024-09-16T00:26:32.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:32.749+0000 7f3c0dab8640 1 -- 172.21.15.1:0/3824093286 wait complete. 2024-09-16T00:26:33.440 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:26:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:33.819 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:33 smithi001 ceph-mon[30934]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 267 B/s, 0 objects/s recovering 2024-09-16T00:26:33.819 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:33 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3824093286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:33.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:33 smithi019 ceph-mon[35186]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 267 B/s, 0 objects/s recovering 2024-09-16T00:26:33.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:33 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3824093286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:33.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:33 smithi153 ceph-mon[35538]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 267 B/s, 0 objects/s recovering 2024-09-16T00:26:33.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:33 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3824093286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:34.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.207+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/561273538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8101de0 msgr2=0x7fa0c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.207+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/561273538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8101de0 0x7fa0c81021c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009900 tx=0x7fa0b802a410 comp rx=0 tx=0).stop 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.207+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/561273538 shutdown_connections 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.207+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/561273538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0c8106f10 0x7fa0c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.207+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/561273538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c8102790 0x7fa0c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.207+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/561273538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8101de0 0x7fa0c81021c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.207+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/561273538 >> 172.21.15.1:0/561273538 conn(0x7fa0c80fd8c0 msgr2=0x7fa0c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.207+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/561273538 shutdown_connections 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.207+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/561273538 wait complete. 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.208+0000 7fa0ce4f4640 1 Processor -- start 2024-09-16T00:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.208+0000 7fa0ce4f4640 1 -- start start 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.208+0000 7fa0ce4f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c8101de0 0x7fa0c80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.208+0000 7fa0ce4f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8102790 0x7fa0c8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.208+0000 7fa0ce4f4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0c8106f10 0x7fa0c8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.208+0000 7fa0ce4f4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8111f90 con 0x7fa0c8102790 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.208+0000 7fa0ce4f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8111e10 con 0x7fa0c8101de0 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.208+0000 7fa0ce4f4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8112110 con 0x7fa0c8106f10 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.208+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8102790 0x7fa0c8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8102790 0x7fa0c8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50260/0 (socket says 172.21.15.1:50260) 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c77fe640 1 -- 172.21.15.1:0/3692787815 learned_addr learned my addr 172.21.15.1:0/3692787815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c7fff640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c8101de0 0x7fa0c80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0cca91640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0c8106f10 0x7fa0c8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c77fe640 1 -- 172.21.15.1:0/3692787815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0c8106f10 msgr2=0x7fa0c8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c77fe640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0c8106f10 0x7fa0c8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c77fe640 1 -- 172.21.15.1:0/3692787815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c8101de0 msgr2=0x7fa0c80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c77fe640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c8101de0 0x7fa0c80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c77fe640 1 -- 172.21.15.1:0/3692787815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c81a6950 con 0x7fa0c8102790 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0cca91640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0c8106f10 0x7fa0c8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c7fff640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c8101de0 0x7fa0c80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c77fe640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8102790 0x7fa0c8074a30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00c890 tx=0x7fa0bc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c57fa640 1 -- 172.21.15.1:0/3692787815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc004220 con 0x7fa0c8102790 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c57fa640 1 -- 172.21.15.1:0/3692787815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0bc0043c0 con 0x7fa0c8102790 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c81a6c40 con 0x7fa0c8102790 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0c57fa640 1 -- 172.21.15.1:0/3692787815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0049e0 con 0x7fa0c8102790 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.209+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c81a7180 con 0x7fa0c8102790 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.210+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c81021c0 con 0x7fa0c8102790 2024-09-16T00:26:34.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.211+0000 7fa0c57fa640 1 -- 172.21.15.1:0/3692787815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fa0bc0079b0 con 0x7fa0c8102790 2024-09-16T00:26:34.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.212+0000 7fa0c57fa640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0b00a3240 0x7fa0b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:34.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.212+0000 7fa0c7fff640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0b00a3240 0x7fa0b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:34.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.212+0000 7fa0c57fa640 1 -- 172.21.15.1:0/3692787815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fa0bc0cfcb0 con 0x7fa0c8102790 2024-09-16T00:26:34.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.213+0000 7fa0c7fff640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0b00a3240 0x7fa0b00a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8000c00 tx=0x7fa0b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:34.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.216+0000 7fa0c57fa640 1 -- 172.21.15.1:0/3692787815 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0bc014030 con 0x7fa0c8102790 2024-09-16T00:26:34.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.398+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0c8109b60 con 0x7fa0c8102790 2024-09-16T00:26:34.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.400+0000 7fa0c57fa640 1 -- 172.21.15.1:0/3692787815 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24601 (secure 0 0 0) 0x7fa0bc09c690 con 0x7fa0c8102790 2024-09-16T00:26:34.401 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:34.401 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":89,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:26:28.990229+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0b00a3240 msgr2=0x7fa0b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0b00a3240 0x7fa0b00a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8000c00 tx=0x7fa0b80058d0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8102790 msgr2=0x7fa0c8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8102790 0x7fa0c8074a30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00c890 tx=0x7fa0bc00cd50 comp rx=0 tx=0).stop 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 shutdown_connections 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0b00a3240 0x7fa0b00a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0c8106f10 0x7fa0c8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0c8102790 0x7fa0c8074a30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 --2- 172.21.15.1:0/3692787815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c8101de0 0x7fa0c80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 >> 172.21.15.1:0/3692787815 conn(0x7fa0c80fd8c0 msgr2=0x7fa0c80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.402+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 shutdown_connections 2024-09-16T00:26:34.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:34.403+0000 7fa0ce4f4640 1 -- 172.21.15.1:0/3692787815 wait complete. 2024-09-16T00:26:34.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:34 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3692787815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:34.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:34 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3692787815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:34.862 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:34 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3692787815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:35.146 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:26:35.404 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:35.758 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:35 smithi019 ceph-mon[35186]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-09-16T00:26:35.834 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:35 smithi001 ceph-mon[30934]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-09-16T00:26:35.878 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:35 smithi153 ceph-mon[35538]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-09-16T00:26:35.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.916+0000 7f6a9a30a640 1 -- 172.21.15.1:0/1292182431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9410c170 msgr2=0x7f6a9410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.916+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/1292182431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9410c170 0x7f6a9410e610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009be0 tx=0x7f6a8c02a750 comp rx=0 tx=0).stop 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.916+0000 7f6a9a30a640 1 -- 172.21.15.1:0/1292182431 shutdown_connections 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.916+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/1292182431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9410c170 0x7f6a9410e610 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.916+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/1292182431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a94103660 0x7f6a9410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.916+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/1292182431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94102d40 0x7f6a94103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.916+0000 7f6a9a30a640 1 -- 172.21.15.1:0/1292182431 >> 172.21.15.1:0/1292182431 conn(0x7f6a940fc7a0 msgr2=0x7f6a940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.916+0000 7f6a9a30a640 1 -- 172.21.15.1:0/1292182431 shutdown_connections 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.916+0000 7f6a9a30a640 1 -- 172.21.15.1:0/1292182431 wait complete. 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a9a30a640 1 Processor -- start 2024-09-16T00:26:35.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a9a30a640 1 -- start start 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a9a30a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a94102d40 0x7f6a941983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a9a30a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a94103660 0x7f6a94198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a9a30a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a9410c170 0x7f6a9419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a9a30a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94111f20 con 0x7f6a94103660 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a9a30a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94111da0 con 0x7f6a94102d40 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a9a30a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941120a0 con 0x7f6a9410c170 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a94103660 0x7f6a94198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a94103660 0x7f6a94198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50270/0 (socket says 172.21.15.1:50270) 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a937fe640 1 -- 172.21.15.1:0/3911166955 learned_addr learned my addr 172.21.15.1:0/3911166955 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a93fff640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a94102d40 0x7f6a941983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.917+0000 7f6a988a7640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a9410c170 0x7f6a9419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a937fe640 1 -- 172.21.15.1:0/3911166955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a9410c170 msgr2=0x7f6a9419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:35.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a937fe640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a9410c170 0x7f6a9419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a937fe640 1 -- 172.21.15.1:0/3911166955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a94102d40 msgr2=0x7f6a941983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a937fe640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a94102d40 0x7f6a941983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a937fe640 1 -- 172.21.15.1:0/3911166955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9419d390 con 0x7f6a94103660 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a937fe640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a94103660 0x7f6a94198920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800e790 tx=0x7f6a8800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a917fa640 1 -- 172.21.15.1:0/3911166955 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800eea0 con 0x7f6a94103660 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a917fa640 1 -- 172.21.15.1:0/3911166955 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a88005500 con 0x7f6a94103660 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9419d680 con 0x7f6a94103660 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a917fa640 1 -- 172.21.15.1:0/3911166955 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88010430 con 0x7f6a94103660 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.918+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941a4f60 con 0x7f6a94103660 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.919+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a94103120 con 0x7f6a94103660 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.920+0000 7f6a917fa640 1 -- 172.21.15.1:0/3911166955 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f6a88005000 con 0x7f6a94103660 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.921+0000 7f6a917fa640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a7c0a3240 0x7f6a7c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:35.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.921+0000 7f6a93fff640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a7c0a3240 0x7f6a7c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:35.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.921+0000 7f6a917fa640 1 -- 172.21.15.1:0/3911166955 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f6a880cfd90 con 0x7f6a94103660 2024-09-16T00:26:35.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.921+0000 7f6a93fff640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a7c0a3240 0x7f6a7c0a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009bd0 tx=0x7f6a84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:35.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:35.925+0000 7f6a917fa640 1 -- 172.21.15.1:0/3911166955 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a8809c770 con 0x7f6a94103660 2024-09-16T00:26:36.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.137+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a9402d7c0 con 0x7f6a94103660 2024-09-16T00:26:36.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.138+0000 7f6a917fa640 1 -- 172.21.15.1:0/3911166955 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11821 (secure 0 0 0) 0x7f6a8809c070 con 0x7f6a94103660 2024-09-16T00:26:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:36.139 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"btime":"2024-09-16T00:26:29:286635+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:26:28.990397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24506,"qdb_cluster":[24506]},"id":1}]} 2024-09-16T00:26:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 17 2024-09-16T00:26:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a7c0a3240 msgr2=0x7f6a7c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a7c0a3240 0x7f6a7c0a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009bd0 tx=0x7f6a84005f90 comp rx=0 tx=0).stop 2024-09-16T00:26:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a94103660 msgr2=0x7f6a94198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a94103660 0x7f6a94198920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800e790 tx=0x7f6a8800ec50 comp rx=0 tx=0).stop 2024-09-16T00:26:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 shutdown_connections 2024-09-16T00:26:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a7c0a3240 0x7f6a7c0a5700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a9410c170 0x7f6a9419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a94103660 0x7f6a94198920 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 --2- 172.21.15.1:0/3911166955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a94102d40 0x7f6a941983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 >> 172.21.15.1:0/3911166955 conn(0x7f6a940fc7a0 msgr2=0x7f6a940fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 shutdown_connections 2024-09-16T00:26:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:36.141+0000 7f6a9a30a640 1 -- 172.21.15.1:0/3911166955 wait complete. 2024-09-16T00:26:36.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:36 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3911166955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:36.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:36 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3911166955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:36.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:36 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3911166955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:37.006 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:26:37.267 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:37 smithi001 ceph-mon[30934]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.775+0000 7fda0f36c640 1 -- 172.21.15.1:0/3802540194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08102790 msgr2=0x7fda081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.775+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3802540194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08102790 0x7fda081067e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009900 tx=0x7fd9f802a410 comp rx=0 tx=0).stop 2024-09-16T00:26:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.775+0000 7fda0f36c640 1 -- 172.21.15.1:0/3802540194 shutdown_connections 2024-09-16T00:26:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.775+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3802540194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08106f10 0x7fda0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.775+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3802540194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08102790 0x7fda081067e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.775+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3802540194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda08101de0 0x7fda081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.775+0000 7fda0f36c640 1 -- 172.21.15.1:0/3802540194 >> 172.21.15.1:0/3802540194 conn(0x7fda080fd8c0 msgr2=0x7fda080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.775+0000 7fda0f36c640 1 -- 172.21.15.1:0/3802540194 shutdown_connections 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.775+0000 7fda0f36c640 1 -- 172.21.15.1:0/3802540194 wait complete. 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.776+0000 7fda0f36c640 1 Processor -- start 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.776+0000 7fda0f36c640 1 -- start start 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.776+0000 7fda0f36c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda08101de0 0x7fda08198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.776+0000 7fda0f36c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08102790 0x7fda081989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.776+0000 7fda0f36c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08106f10 0x7fda0819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.776+0000 7fda0f36c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fda08111f70 con 0x7fda08106f10 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.776+0000 7fda0f36c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda08111df0 con 0x7fda08101de0 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.776+0000 7fda0f36c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda081120f0 con 0x7fda08102790 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.776+0000 7fda0d909640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08106f10 0x7fda0819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0d909640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08106f10 0x7fda0819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50288/0 (socket says 172.21.15.1:50288) 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0d909640 1 -- 172.21.15.1:0/3059118084 learned_addr learned my addr 172.21.15.1:0/3059118084 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0d108640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda08101de0 0x7fda08198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0c907640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08102790 0x7fda081989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0d909640 1 -- 172.21.15.1:0/3059118084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08102790 msgr2=0x7fda081989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0d909640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08102790 0x7fda081989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0d909640 1 -- 172.21.15.1:0/3059118084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda08101de0 msgr2=0x7fda08198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0d909640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda08101de0 0x7fda08198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0d909640 1 -- 172.21.15.1:0/3059118084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0819d440 con 0x7fda08106f10 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0d909640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08106f10 0x7fda0819cd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fda0000eaa0 tx=0x7fda0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fd9fe7fc640 1 -- 172.21.15.1:0/3059118084 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0000cdb0 con 0x7fda08106f10 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fd9fe7fc640 1 -- 172.21.15.1:0/3059118084 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda00005020 con 0x7fda08106f10 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0819d730 con 0x7fda08106f10 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fd9fe7fc640 1 -- 172.21.15.1:0/3059118084 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000116b0 con 0x7fda08106f10 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.777+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda081a5010 con 0x7fda08106f10 2024-09-16T00:26:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.778+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda081021c0 con 0x7fda08106f10 2024-09-16T00:26:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.779+0000 7fd9fe7fc640 1 -- 172.21.15.1:0/3059118084 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fda00005560 con 0x7fda08106f10 2024-09-16T00:26:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.780+0000 7fd9fe7fc640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9f40a3240 0x7fd9f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.780+0000 7fda0d108640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9f40a3240 0x7fd9f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.780+0000 7fd9fe7fc640 1 -- 172.21.15.1:0/3059118084 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fda000d00f0 con 0x7fda08106f10 2024-09-16T00:26:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.781+0000 7fda0d108640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9f40a3240 0x7fd9f40a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fda0400bbe0 tx=0x7fda0400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.783+0000 7fd9fe7fc640 1 -- 172.21.15.1:0/3059118084 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda00016030 con 0x7fda08106f10 2024-09-16T00:26:37.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:37 smithi019 ceph-mon[35186]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:37.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.960+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda0810a150 con 0x7fda08106f10 2024-09-16T00:26:37.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:37 smithi153 ceph-mon[35538]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:37.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.962+0000 7fd9fe7fc640 1 -- 172.21.15.1:0/3059118084 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24601 (secure 0 0 0) 0x7fda0009c480 con 0x7fda08106f10 2024-09-16T00:26:37.962 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:37.962 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":89,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:26:28.990229+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:26:37.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9f40a3240 msgr2=0x7fd9f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9f40a3240 0x7fd9f40a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fda0400bbe0 tx=0x7fda0400b390 comp rx=0 tx=0).stop 2024-09-16T00:26:37.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08106f10 msgr2=0x7fda0819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08106f10 0x7fda0819cd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fda0000eaa0 tx=0x7fda0000ef60 comp rx=0 tx=0).stop 2024-09-16T00:26:37.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 shutdown_connections 2024-09-16T00:26:37.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda08106f10 0x7fda0819cd60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9f40a3240 0x7fd9f40a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08102790 0x7fda081989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 --2- 172.21.15.1:0/3059118084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda08101de0 0x7fda08198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 >> 172.21.15.1:0/3059118084 conn(0x7fda080fd8c0 msgr2=0x7fda080fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:37.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 shutdown_connections 2024-09-16T00:26:37.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:37.964+0000 7fda0f36c640 1 -- 172.21.15.1:0/3059118084 wait complete. 2024-09-16T00:26:38.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:38 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3059118084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:38.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T00:26:18.890037+0000', 'modified': '2024-09-16T00:26:28.990397+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6839/3056769092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 3056769092}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 3056769092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24506, 'qdb_cluster': [24506]} 2024-09-16T00:26:38.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T00:26:38.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:26:38.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:38 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3059118084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:38.945 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:38.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:38 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3059118084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/2482361052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38102790 msgr2=0x7fbd381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/2482361052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38102790 0x7fbd381067e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009900 tx=0x7fbd2c02a410 comp rx=0 tx=0).stop 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/2482361052 shutdown_connections 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/2482361052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd38106f10 0x7fbd3810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/2482361052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38102790 0x7fbd381067e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/2482361052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd38101de0 0x7fbd381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/2482361052 >> 172.21.15.1:0/2482361052 conn(0x7fbd380fd8c0 msgr2=0x7fbd380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/2482361052 shutdown_connections 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/2482361052 wait complete. 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 Processor -- start 2024-09-16T00:26:39.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.469+0000 7fbd3dfd7640 1 -- start start 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd3dfd7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38101de0 0x7fbd381984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd3dfd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd38102790 0x7fbd381989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd3dfd7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd38106f10 0x7fbd3819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd3dfd7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbd38111f70 con 0x7fbd38101de0 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd3dfd7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd38111df0 con 0x7fbd38102790 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd3dfd7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381120f0 con 0x7fbd38106f10 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38101de0 0x7fbd381984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38101de0 0x7fbd381984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50298/0 (socket says 172.21.15.1:50298) 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd377fe640 1 -- 172.21.15.1:0/358161195 learned_addr learned my addr 172.21.15.1:0/358161195 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd38102790 0x7fbd381989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd37fff640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd38106f10 0x7fbd3819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd377fe640 1 -- 172.21.15.1:0/358161195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd38106f10 msgr2=0x7fbd3819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd377fe640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd38106f10 0x7fbd3819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd377fe640 1 -- 172.21.15.1:0/358161195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd38102790 msgr2=0x7fbd381989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd377fe640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd38102790 0x7fbd381989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd377fe640 1 -- 172.21.15.1:0/358161195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3819d460 con 0x7fbd38101de0 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.470+0000 7fbd377fe640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38101de0 0x7fbd381984b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800d770 tx=0x7fbd2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.471+0000 7fbd34ff9640 1 -- 172.21.15.1:0/358161195 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28017070 con 0x7fbd38101de0 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.471+0000 7fbd34ff9640 1 -- 172.21.15.1:0/358161195 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd28005500 con 0x7fbd38101de0 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.471+0000 7fbd34ff9640 1 -- 172.21.15.1:0/358161195 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28004e00 con 0x7fbd38101de0 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.471+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/358161195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3819d750 con 0x7fbd38101de0 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.471+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/358161195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd381a5030 con 0x7fbd38101de0 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd403b0740 1 Processor -- start 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd34ff9640 1 -- 172.21.15.1:0/358161195 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fbd28004fa0 con 0x7fbd38101de0 2024-09-16T00:26:39.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd403b0740 1 -- start start 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd403b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa525090 con 0x55dcfa5c9bb0 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd403b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa4cca70 con 0x55dcfa5be9d0 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd403b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa521050 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd36ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd36ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50310/0 (socket says 172.21.15.1:50310) 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd36ffd640 1 -- 172.21.15.1:0/3032736538 learned_addr learned my addr 172.21.15.1:0/3032736538 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd377fe640 1 --2- 172.21.15.1:0/3032736538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd37fff640 1 --2- 172.21.15.1:0/3032736538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd36ffd640 1 -- 172.21.15.1:0/3032736538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 msgr2=0x55dcfa5c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/3032736538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.474+0000 7fbd36ffd640 1 -- 172.21.15.1:0/3032736538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5be9d0 msgr2=0x55dcfa5bedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.475+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/3032736538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.475+0000 7fbd36ffd640 1 -- 172.21.15.1:0/3032736538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcfa4d9130 con 0x55dcfa5c9bb0 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.475+0000 7fbd34ff9640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd200a3240 0x7fbd200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.475+0000 7fbd37fff640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd200a3240 0x7fbd200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.475+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/3032736538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5cbfa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009900 tx=0x7fbd2c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b81af8d9052e7bd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.475+0000 7fbd34ff9640 1 -- 172.21.15.1:0/358161195 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fbd280cf620 con 0x7fbd38101de0 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.475+0000 7fbd1dffb640 1 -- 172.21.15.1:0/3032736538 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c040070 con 0x55dcfa5c9bb0 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.475+0000 7fbd1dffb640 1 -- 172.21.15.1:0/3032736538 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd2c0319f0 con 0x55dcfa5c9bb0 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.475+0000 7fbd1dffb640 1 -- 172.21.15.1:0/3032736538 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c035040 con 0x55dcfa5c9bb0 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 -- 172.21.15.1:0/3032736538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5c9bb0 msgr2=0x55dcfa5cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd37fff640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd200a3240 0x7fbd200a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd30005fd0 tx=0x7fbd30009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 --2- 172.21.15.1:0/3032736538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5cbfa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009900 tx=0x7fbd2c005e80 comp rx=0 tx=0).stop 2024-09-16T00:26:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 -- 172.21.15.1:0/3032736538 shutdown_connections 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 --2- 172.21.15.1:0/3032736538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5cbfa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 --2- 172.21.15.1:0/3032736538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 --2- 172.21.15.1:0/3032736538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 -- 172.21.15.1:0/3032736538 >> 172.21.15.1:0/3032736538 conn(0x55dcfa5baf60 msgr2=0x55dcfa5bb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 -- 172.21.15.1:0/3032736538 shutdown_connections 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 -- 172.21.15.1:0/3032736538 wait complete. 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 -- start start 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.476+0000 7fbd403b0740 1 Processor -- start 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd403b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa5cc780 con 0x55dcfa5be9d0 2024-09-16T00:26:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd403b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa4dff20 con 0x55dcfa5c9bb0 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd403b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa5cc900 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd403b0740 10 client.0 _ll_register_callbacks cb 0x55dcfa4a9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50324/0 (socket says 172.21.15.1:50324) 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd377fe640 1 -- 172.21.15.1:0/799049488 learned_addr learned my addr 172.21.15.1:0/799049488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd37fff640 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd377fe640 1 -- 172.21.15.1:0/799049488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 msgr2=0x55dcfa5c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd377fe640 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd377fe640 1 -- 172.21.15.1:0/799049488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 msgr2=0x55dcfa5c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd377fe640 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd377fe640 1 -- 172.21.15.1:0/799049488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcfa2962b0 con 0x55dcfa5be9d0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd37fff640 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd377fe640 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c93f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd280ccde0 tx=0x7fbd280057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef3e4b2808a4428 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.477+0000 7fbd16ffd640 1 -- 172.21.15.1:0/799049488 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28020050 con 0x55dcfa5be9d0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd16ffd640 1 -- 172.21.15.1:0/799049488 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fbd280860b0 con 0x55dcfa5be9d0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd403b0740 1 -- 172.21.15.1:0/799049488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd2c0319f0 con 0x55dcfa5be9d0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd16ffd640 1 -- 172.21.15.1:0/799049488 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbd2809b4f0 con 0x55dcfa5be9d0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd403b0740 10 client.14718 fetch_fsmap learned FSMap version 17 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd403b0740 1 -- 172.21.15.1:0/799049488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55dcfa5d80a0 con 0x55dcfa5be9d0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd16ffd640 1 -- 172.21.15.1:0/799049488 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd2c0319f0 con 0x55dcfa5be9d0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd16ffd640 1 -- 172.21.15.1:0/799049488 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fbd280cf030 con 0x55dcfa5be9d0 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd403b0740 10 client.14718 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T00:26:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd403b0740 10 client.14718 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd403b0740 20 client.14718 populate_metadata read hostname 'smithi001' 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd403b0740 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dcfa5da6d0 0x55dcfa5dbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd403b0740 4 client.14718 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd403b0740 1 -- 172.21.15.1:0/799049488 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcfa532600 con 0x55dcfa5da6d0 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.478+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dcfa5da6d0 0x55dcfa5dbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.479+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dcfa5da6d0 0x55dcfa5dbb40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d2f0e8424086fcd server_cookie=a033120e7103138f in_seq=0 out_seq=0 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.479+0000 7fbd16ffd640 10 client.14718 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd16ffd640 1 -- 172.21.15.1:0/799049488 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcfa532600 con 0x55dcfa5da6d0 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd16ffd640 10 client.14718 handle_command_reply: tid=0 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 client.14718 shutdown 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 -- 172.21.15.1:0/799049488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 msgr2=0x55dcfa5c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c93f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd280ccde0 tx=0x7fbd280057b0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 -- 172.21.15.1:0/799049488 shutdown_connections 2024-09-16T00:26:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dcfa5da6d0 0x55dcfa5dbb40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd16ffd640 0 client.14718 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:26:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 --2- 172.21.15.1:0/799049488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c93f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 -- 172.21.15.1:0/799049488 >> 172.21.15.1:0/799049488 conn(0x55dcfa5baf60 msgr2=0x55dcfa5c0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 -- 172.21.15.1:0/799049488 shutdown_connections 2024-09-16T00:26:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 1 -- 172.21.15.1:0/799049488 wait complete. 2024-09-16T00:26:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.480+0000 7fbd403b0740 20 client.14718 trim_cache size 0 max 16384 2024-09-16T00:26:39.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.498+0000 7fbd403b0740 1 Processor -- start 2024-09-16T00:26:39.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.498+0000 7fbd403b0740 1 -- start start 2024-09-16T00:26:39.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.498+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.498+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.498+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.498+0000 7fbd403b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa5d9e80 con 0x55dcfa5be9d0 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.498+0000 7fbd403b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa5da550 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.498+0000 7fbd403b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa5dac20 con 0x55dcfa5c9bb0 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.498+0000 7fbd377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50332/0 (socket says 172.21.15.1:50332) 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd377fe640 1 -- 172.21.15.1:0/895347469 learned_addr learned my addr 172.21.15.1:0/895347469 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd377fe640 1 -- 172.21.15.1:0/895347469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5c9bb0 msgr2=0x55dcfa5c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd37fff640 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd377fe640 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd377fe640 1 -- 172.21.15.1:0/895347469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5bf2f0 msgr2=0x55dcfa5e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd377fe640 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5e4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd377fe640 1 -- 172.21.15.1:0/895347469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcfa5c9760 con 0x55dcfa5be9d0 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd37fff640 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd377fe640 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bb340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd2809b940 tx=0x7fbd280d55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e9647691e37701 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd1d7fa640 1 -- 172.21.15.1:0/895347469 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28014020 con 0x55dcfa5be9d0 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd1d7fa640 1 -- 172.21.15.1:0/895347469 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd280d5240 con 0x55dcfa5be9d0 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.499+0000 7fbd1d7fa640 1 -- 172.21.15.1:0/895347469 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28020050 con 0x55dcfa5be9d0 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 -- 172.21.15.1:0/895347469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 msgr2=0x55dcfa5bb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bb340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd2809b940 tx=0x7fbd280d55c0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 -- 172.21.15.1:0/895347469 shutdown_connections 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa5c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 --2- 172.21.15.1:0/895347469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5bb340 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 -- 172.21.15.1:0/895347469 >> 172.21.15.1:0/895347469 conn(0x55dcfa5baf60 msgr2=0x55dcfa5c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 -- 172.21.15.1:0/895347469 shutdown_connections 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 -- 172.21.15.1:0/895347469 wait complete. 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.500+0000 7fbd403b0740 1 -- start start 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd403b0740 1 Processor -- start 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd403b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa63feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd403b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa5db9c0 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd403b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa5db2f0 con 0x55dcfa5c9bb0 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd403b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa5dc090 con 0x55dcfa5be9d0 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd403b0740 10 client.0 _ll_register_callbacks cb 0x7fbd2800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:39.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd36ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd36ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50336/0 (socket says 172.21.15.1:50336) 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd36ffd640 1 -- 172.21.15.1:0/2077439896 learned_addr learned my addr 172.21.15.1:0/2077439896 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd377fe640 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd37fff640 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa63feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd36ffd640 1 -- 172.21.15.1:0/2077439896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5be9d0 msgr2=0x55dcfa5c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd36ffd640 1 -- 172.21.15.1:0/2077439896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 msgr2=0x55dcfa63feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa63feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd36ffd640 1 -- 172.21.15.1:0/2077439896 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcfa640550 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd377fe640 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.501+0000 7fbd37fff640 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa63feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd36ffd640 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c3a60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009900 tx=0x7fbd2c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23508414692042 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd16ffd640 1 -- 172.21.15.1:0/2077439896 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c055020 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd16ffd640 1 -- 172.21.15.1:0/2077439896 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fbd2c004240 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd403b0740 1 -- 172.21.15.1:0/2077439896 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd280d5240 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd16ffd640 1 -- 172.21.15.1:0/2077439896 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbd2c060050 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd403b0740 10 client.14724 fetch_fsmap learned FSMap version 17 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd403b0740 1 -- 172.21.15.1:0/2077439896 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55dcfa648a90 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd16ffd640 1 -- 172.21.15.1:0/2077439896 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd280d5240 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd16ffd640 1 -- 172.21.15.1:0/2077439896 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fbd2c050420 con 0x55dcfa5bf2f0 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd403b0740 10 client.14724 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd403b0740 10 client.14724 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd403b0740 20 client.14724 populate_metadata read hostname 'smithi001' 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd403b0740 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dcfa64d290 0x55dcfa64bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd403b0740 4 client.14724 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.502+0000 7fbd403b0740 1 -- 172.21.15.1:0/2077439896 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "status"}) -- 0x55dcfa52cb10 con 0x55dcfa64d290 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd377fe640 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dcfa64d290 0x55dcfa64bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd16ffd640 10 client.14724 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd377fe640 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dcfa64d290 0x55dcfa64bfa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10063b79523cafb6 server_cookie=ff589820a578b499 in_seq=0 out_seq=0 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd16ffd640 1 -- 172.21.15.1:0/2077439896 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55dcfa52cb10 con 0x55dcfa64d290 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd16ffd640 10 client.14724 handle_command_reply: tid=0 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd403b0740 1 client.14724 shutdown 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd403b0740 1 -- 172.21.15.1:0/2077439896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5bf2f0 msgr2=0x55dcfa5c3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd403b0740 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c3a60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009900 tx=0x7fbd2c005b50 comp rx=0 tx=0).stop 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd403b0740 1 -- 172.21.15.1:0/2077439896 shutdown_connections 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd403b0740 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dcfa64d290 0x55dcfa64bfa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd403b0740 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcfa5c9bb0 0x55dcfa63feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd403b0740 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcfa5bf2f0 0x55dcfa5c3a60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd403b0740 1 --2- 172.21.15.1:0/2077439896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcfa5be9d0 0x55dcfa5c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd403b0740 1 -- 172.21.15.1:0/2077439896 >> 172.21.15.1:0/2077439896 conn(0x55dcfa5baf60 msgr2=0x55dcfa5c4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.503+0000 7fbd16ffd640 0 client.14724 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd403b0740 1 -- 172.21.15.1:0/2077439896 shutdown_connections 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd403b0740 1 -- 172.21.15.1:0/2077439896 wait complete. 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd403b0740 20 client.14724 trim_cache size 0 max 16384 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-16T00:26:39.516 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24506, 2024-09-16T00:26:39.517 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:26:39.517 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:26:39.517 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:26:39.517 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 20.232298552, 2024-09-16T00:26:39.517 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 17, 2024-09-16T00:26:39.517 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 89, 2024-09-16T00:26:39.517 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 89, 2024-09-16T00:26:39.517 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 31.576652336999999 2024-09-16T00:26:39.517 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd200a3240 msgr2=0x7fbd200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd200a3240 0x7fbd200a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd30005fd0 tx=0x7fbd30009600 comp rx=0 tx=0).stop 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38101de0 msgr2=0x7fbd381984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38101de0 0x7fbd381984b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800d770 tx=0x7fbd2800dc30 comp rx=0 tx=0).stop 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/358161195 shutdown_connections 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd200a3240 0x7fbd200a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd38106f10 0x7fbd3819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd38102790 0x7fbd381989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 --2- 172.21.15.1:0/358161195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd38101de0 0x7fbd381984b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.504+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/358161195 >> 172.21.15.1:0/358161195 conn(0x7fbd380fd8c0 msgr2=0x7fbd380fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.505+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/358161195 shutdown_connections 2024-09-16T00:26:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:39.505+0000 7fbd3dfd7640 1 -- 172.21.15.1:0/358161195 wait complete. 2024-09-16T00:26:39.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:39 smithi001 ceph-mon[30934]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:39.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:39 smithi019 ceph-mon[35186]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:39.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:39 smithi153 ceph-mon[35538]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:40.312 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:26:40.312 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:26:40.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:26:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:41.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.085+0000 7f70866c3640 1 -- 172.21.15.1:0/3428151500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080106f10 msgr2=0x7f708010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:41.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.085+0000 7f70866c3640 1 --2- 172.21.15.1:0/3428151500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080106f10 0x7f708010ea90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7078009ca0 tx=0x7f707802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.085+0000 7f70866c3640 1 -- 172.21.15.1:0/3428151500 shutdown_connections 2024-09-16T00:26:41.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.085+0000 7f70866c3640 1 --2- 172.21.15.1:0/3428151500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080106f10 0x7f708010ea90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.085+0000 7f70866c3640 1 --2- 172.21.15.1:0/3428151500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080102790 0x7f70801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.085+0000 7f70866c3640 1 --2- 172.21.15.1:0/3428151500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7080101de0 0x7f70801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.085+0000 7f70866c3640 1 -- 172.21.15.1:0/3428151500 >> 172.21.15.1:0/3428151500 conn(0x7f70800fd8c0 msgr2=0x7f70800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:41.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.085+0000 7f70866c3640 1 -- 172.21.15.1:0/3428151500 shutdown_connections 2024-09-16T00:26:41.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.085+0000 7f70866c3640 1 -- 172.21.15.1:0/3428151500 wait complete. 2024-09-16T00:26:41.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f70866c3640 1 Processor -- start 2024-09-16T00:26:41.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f70866c3640 1 -- start start 2024-09-16T00:26:41.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f70866c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080101de0 0x7f7080198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:41.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f70866c3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080102790 0x7f7080198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:41.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f70866c3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7080106f10 0x7f708019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:41.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f70866c3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7080111f90 con 0x7f7080101de0 2024-09-16T00:26:41.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f70866c3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7080111e10 con 0x7f7080102790 2024-09-16T00:26:41.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f70866c3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7080112110 con 0x7f7080106f10 2024-09-16T00:26:41.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f707ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080101de0 0x7f7080198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:41.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f707ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080101de0 0x7f7080198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58810/0 (socket says 172.21.15.1:58810) 2024-09-16T00:26:41.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f707ffff640 1 -- 172.21.15.1:0/241402815 learned_addr learned my addr 172.21.15.1:0/241402815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:41.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f707f7fe640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080102790 0x7f7080198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:41.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.086+0000 7f7084c60640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7080106f10 0x7f708019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:41.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f707ffff640 1 -- 172.21.15.1:0/241402815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7080106f10 msgr2=0x7f708019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:41.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f707ffff640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7080106f10 0x7f708019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f707ffff640 1 -- 172.21.15.1:0/241402815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080102790 msgr2=0x7f7080198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f707ffff640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080102790 0x7f7080198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f707ffff640 1 -- 172.21.15.1:0/241402815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708019d3c0 con 0x7f7080101de0 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f7084c60640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7080106f10 0x7f708019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f707ffff640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080101de0 0x7f7080198410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f707000b4a0 tx=0x7f707000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f707d7fa640 1 -- 172.21.15.1:0/241402815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070015020 con 0x7f7080101de0 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f707d7fa640 1 -- 172.21.15.1:0/241402815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7070011d10 con 0x7f7080101de0 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708019d6b0 con 0x7f7080101de0 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f707d7fa640 1 -- 172.21.15.1:0/241402815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707000f2e0 con 0x7f7080101de0 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.087+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70801a4f90 con 0x7f7080101de0 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.088+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70801021c0 con 0x7f7080101de0 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.089+0000 7f707d7fa640 1 -- 172.21.15.1:0/241402815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f7070018020 con 0x7f7080101de0 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.090+0000 7f707d7fa640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70680a32c0 0x7f70680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:41.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.090+0000 7f707f7fe640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70680a32c0 0x7f70680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:41.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.090+0000 7f707d7fa640 1 -- 172.21.15.1:0/241402815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f70700cf8f0 con 0x7f7080101de0 2024-09-16T00:26:41.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.090+0000 7f707f7fe640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70680a32c0 0x7f70680a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70800ff790 tx=0x7f707400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:41.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.093+0000 7f707d7fa640 1 -- 172.21.15.1:0/241402815 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f707001f020 con 0x7f7080101de0 2024-09-16T00:26:41.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.309+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f70801a5220 con 0x7f7080101de0 2024-09-16T00:26:41.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.310+0000 7f707d7fa640 1 -- 172.21.15.1:0/241402815 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f707009c270 con 0x7f7080101de0 2024-09-16T00:26:41.311 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T00:26:41.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.312+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70680a32c0 msgr2=0x7f70680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:41.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70680a32c0 0x7f70680a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70800ff790 tx=0x7f707400a2d0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080101de0 msgr2=0x7f7080198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:41.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080101de0 0x7f7080198410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f707000b4a0 tx=0x7f707000b8a0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 shutdown_connections 2024-09-16T00:26:41.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70680a32c0 0x7f70680a5780 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7080106f10 0x7f708019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080102790 0x7f7080198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 --2- 172.21.15.1:0/241402815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7080101de0 0x7f7080198410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:41.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 >> 172.21.15.1:0/241402815 conn(0x7f70800fd8c0 msgr2=0x7f70800ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:41.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 shutdown_connections 2024-09-16T00:26:41.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:41.313+0000 7f70866c3640 1 -- 172.21.15.1:0/241402815 wait complete. 2024-09-16T00:26:41.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:41 smithi001 ceph-mon[30934]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:41.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:41 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/241402815' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:26:41.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:41 smithi019 ceph-mon[35186]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:41.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:41 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/241402815' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:26:41.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:41 smithi153 ceph-mon[35538]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:41.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:41 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/241402815' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:26:41.974 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T00:26:41.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:26:41.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-09-16T00:26:41.974 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-16T00:26:41.974 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:26:41.974 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T00:26:41.974 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:26:41.974 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:26:41.974 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T00:26:41.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:41.975 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-09-16T00:26:41.993 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:26:41.993 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:26:41.993 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:26:41.993 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:26:41.994 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-09-16T00:26:41.994 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:26:41.994 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:26:41.994 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-09-16T00:26:41.994 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-09-16T00:26:41.994 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:26:41.995 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-09-16T00:26:41.995 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-09-16T00:26:41.995 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:26:41.995 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:26:41.996 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-09-16T00:26:41.996 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T00:26:41.996 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42298sec preferred_lft 42298sec 2024-09-16T00:26:41.996 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-09-16T00:26:41.996 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:26:41.997 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:26:41.997 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:26:41.997 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-09-16T00:26:41.997 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:26:41.997 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:26:41.998 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:26:41.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:26:41.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-09-16T00:26:41.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-09-16T00:26:41.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:26:41.998 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:26:42.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:26:42.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:26:42.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:42.200 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:26:42.227 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-09-16T00:26:42.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:42.229 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-09-16T00:26:42.306 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-09-16T00:26:42.307 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:26:42.307 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T00:26:42.307 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T00:26:42.307 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:26:42.307 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:26:42.307 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:26:42.308 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T00:26:42.308 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T00:26:42.308 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T00:26:42.308 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:26:42.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:26:42.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:26:42.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:42.612 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-16T00:26:42.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:42.629 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-09-16T00:26:42.685 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:26:42.685 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:26:42.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:26:42.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-16T00:26:42.685 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:26:42.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:26:42.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:26:42.809 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-16T00:26:42.809 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:26:42.809 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:26:42.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-16T00:26:42.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:26:42.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-16T00:26:42.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-16T00:26:42.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-16T00:26:42.810 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:26:42.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:26:42.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:26:42.937 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:26:42.937 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:26:42.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-09-16T00:26:42.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:26:42.937 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:26:42.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:26:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:26:43.016 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-16T00:26:43.016 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:26:43.016 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:26:43.032 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T00:26:43.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:26:43.032 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:26:43.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-09-16T00:26:43.155 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:26:43.212 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-09-16T00:26:43.212 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-09-16T00:26:43.212 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-09-16T00:26:43.212 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-09-16T00:26:43.212 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-09-16T00:26:43.212 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-09-16T00:26:43.212 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-09-16T00:26:43.212 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-09-16T00:26:43.212 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:26:43.213 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/03d46f11786d2a6e72038f5eb5485d18fd8a39ccba4e546835717cb14565a1e8/merged 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/d45d9c66bbff51a91030c37663fc5c13b0637357ce10b8c17676fe89d5f6ee0f/merged 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/0 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/318e342dd8fb9f65505b283874a9c6db7a44ccf41e47c3b0bdc17e2dda497320/merged 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/1a194b615b351bda169da8864436ad1274c79933687e4c9e6735f74d5dc1414f/merged 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/790306c812509e9ca1a786f29f5f87ce15ec75bbac62e211117ad9c672eeb789/merged 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/15a083530d809ea2497af5509aba455ccdff04ee6ff0201c6490103b38bc9b63/merged 2024-09-16T00:26:43.214 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/ad9ec78532b2876e0f5d40734cb4970d02a23e80acacccc43869c0704bd3286e/merged 2024-09-16T00:26:43.215 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/c6ae9be7167f62fd90ce217dbe3b8876292fb616fb7e15ad407d5a7d20854756/merged 2024-09-16T00:26:43.215 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/25f2a9cda2e00e8dd5b4c022d1f921434d57f0ce5879ad1ae0195079b2a0a9da/merged 2024-09-16T00:26:43.215 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/b068ef96c250e8fdb1a106c5861662e590fad64c6a1c97f5db1e2c6c39ea88cb/merged 2024-09-16T00:26:43.215 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-09-16T00:26:43.215 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:26:43.215 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:26:43.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:43.215 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-09-16T00:26:43.268 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T00:26:43.268 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-16T00:26:43.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-09-16T00:26:43.336 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:26:43.394 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-09-16T00:26:43.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-09-16T00:26:43.394 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-09-16T00:26:43.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-09-16T00:26:43.394 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-09-16T00:26:43.394 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-09-16T00:26:43.394 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-09-16T00:26:43.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-09-16T00:26:43.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-09-16T00:26:43.395 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/03d46f11786d2a6e72038f5eb5485d18fd8a39ccba4e546835717cb14565a1e8/merged 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/d45d9c66bbff51a91030c37663fc5c13b0637357ce10b8c17676fe89d5f6ee0f/merged 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/0 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/318e342dd8fb9f65505b283874a9c6db7a44ccf41e47c3b0bdc17e2dda497320/merged 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/1a194b615b351bda169da8864436ad1274c79933687e4c9e6735f74d5dc1414f/merged 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/790306c812509e9ca1a786f29f5f87ce15ec75bbac62e211117ad9c672eeb789/merged 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/15a083530d809ea2497af5509aba455ccdff04ee6ff0201c6490103b38bc9b63/merged 2024-09-16T00:26:43.396 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/ad9ec78532b2876e0f5d40734cb4970d02a23e80acacccc43869c0704bd3286e/merged 2024-09-16T00:26:43.397 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/c6ae9be7167f62fd90ce217dbe3b8876292fb616fb7e15ad407d5a7d20854756/merged 2024-09-16T00:26:43.397 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/25f2a9cda2e00e8dd5b4c022d1f921434d57f0ce5879ad1ae0195079b2a0a9da/merged 2024-09-16T00:26:43.397 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/b068ef96c250e8fdb1a106c5861662e590fad64c6a1c97f5db1e2c6c39ea88cb/merged 2024-09-16T00:26:43.397 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-09-16T00:26:43.397 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:26:43.397 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:26:43.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:43.397 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-09-16T00:26:43.428 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:2024-09-16T00:26:43.427+0000 7fd8fcd29480 -1 init, newargv = 0x55ff1f859970 newargc=15 2024-09-16T00:26:43.428 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[71708]: starting ceph client 2024-09-16T00:26:43.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[71708]: starting fuse 2024-09-16T00:26:43.444 INFO:teuthology.orchestra.run.smithi001.stdout:120 2024-09-16T00:26:43.444 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-16T00:26:43.444 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-09-16T00:26:43.444 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-09-16T00:26:43.444 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-09-16T00:26:43.444 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-09-16T00:26:43.444 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T00:26:43.445 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T00:26:43.446 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T00:26:43.446 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-09-16T00:26:43.446 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-09-16T00:26:43.446 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-09-16T00:26:43.446 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T00:26:43.446 DEBUG:teuthology.orchestra.run.smithi001:> 2024-09-16T00:26:43.446 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.admin")) 2024-09-16T00:26:43.446 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-09-16T00:26:43.566 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.admin.71708.asok 2024-09-16T00:26:43.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:26:43.576 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.71708.asok 2024-09-16T00:26:43.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:43.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.71708.asok status 2024-09-16T00:26:43.713 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "admin", 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "71708", 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-09-16T00:26:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 14736, 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 14736 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1085375306 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-16T00:26:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1085375306 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.14736 192.168.0.1:0/1085375306", 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/1085375306", 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 17, 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 89, 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-09-16T00:26:43.716 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:26:43.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:43.722 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-09-16T00:26:43.722 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:26:43.739 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-09-16T00:26:43.739 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:26:43.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:43.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:26:43.801 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:43 smithi001 ceph-mon[30934]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:43.806 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-16T00:26:43.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:43 smithi019 ceph-mon[35186]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:26:43.920 INFO:teuthology.orchestra.run.smithi001.stdout:# file: . 2024-09-16T00:26:43.921 INFO:teuthology.orchestra.run.smithi001.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-16T00:26:43.921 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:43.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:26:43.926 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:26:43.927 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:26:43.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:26:43.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:43 smithi153 ceph-mon[35538]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:44.242 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.724+0000 7f58a9131640 1 -- 172.21.15.1:0/1211426839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 msgr2=0x7f58a4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.724+0000 7f58a9131640 1 --2- 172.21.15.1:0/1211426839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 0x7f58a4103120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5894009900 tx=0x7f589402a480 comp rx=0 tx=0).stop 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.724+0000 7f58a9131640 1 -- 172.21.15.1:0/1211426839 shutdown_connections 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.724+0000 7f58a9131640 1 --2- 172.21.15.1:0/1211426839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a410c170 0x7f58a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.724+0000 7f58a9131640 1 --2- 172.21.15.1:0/1211426839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103660 0x7f58a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.724+0000 7f58a9131640 1 --2- 172.21.15.1:0/1211426839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 0x7f58a4103120 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.724+0000 7f58a9131640 1 -- 172.21.15.1:0/1211426839 >> 172.21.15.1:0/1211426839 conn(0x7f58a40fc7a0 msgr2=0x7f58a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.724+0000 7f58a9131640 1 -- 172.21.15.1:0/1211426839 shutdown_connections 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.724+0000 7f58a9131640 1 -- 172.21.15.1:0/1211426839 wait complete. 2024-09-16T00:26:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.725+0000 7f58a9131640 1 Processor -- start 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.725+0000 7f58a9131640 1 -- start start 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.725+0000 7f58a9131640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 0x7f58a4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.725+0000 7f58a9131640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103660 0x7f58a4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.725+0000 7f58a9131640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a410c170 0x7f58a419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.725+0000 7f58a9131640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4111fb0 con 0x7f58a4102d40 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.725+0000 7f58a9131640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4111e30 con 0x7f58a4103660 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.725+0000 7f58a9131640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4112130 con 0x7f58a410c170 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 0x7f58a4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 0x7f58a4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58828/0 (socket says 172.21.15.1:58828) 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a2d9d640 1 -- 172.21.15.1:0/473045630 learned_addr learned my addr 172.21.15.1:0/473045630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a259c640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103660 0x7f58a4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a359e640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a410c170 0x7f58a419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a2d9d640 1 -- 172.21.15.1:0/473045630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a410c170 msgr2=0x7f58a419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a2d9d640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a410c170 0x7f58a419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a2d9d640 1 -- 172.21.15.1:0/473045630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103660 msgr2=0x7f58a4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a2d9d640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103660 0x7f58a4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a2d9d640 1 -- 172.21.15.1:0/473045630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a419d3c0 con 0x7f58a4102d40 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a359e640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a410c170 0x7f58a419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a259c640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103660 0x7f58a4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a2d9d640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 0x7f58a4198410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5894005ab0 tx=0x7f58940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f588bfff640 1 -- 172.21.15.1:0/473045630 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589403f030 con 0x7f58a4102d40 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a419d650 con 0x7f58a4102d40 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f588bfff640 1 -- 172.21.15.1:0/473045630 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5894003ce0 con 0x7f58a4102d40 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f588bfff640 1 -- 172.21.15.1:0/473045630 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894035070 con 0x7f58a4102d40 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.726+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41a4f90 con 0x7f58a4102d40 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.727+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a4103120 con 0x7f58a4102d40 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.729+0000 7f588bfff640 1 -- 172.21.15.1:0/473045630 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f5894042030 con 0x7f58a4102d40 2024-09-16T00:26:44.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.730+0000 7f588bfff640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f588c0a3240 0x7f588c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.730+0000 7f588bfff640 1 -- 172.21.15.1:0/473045630 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f58940f08a0 con 0x7f58a4102d40 2024-09-16T00:26:44.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.730+0000 7f58a259c640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f588c0a3240 0x7f588c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.730+0000 7f58a259c640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f588c0a3240 0x7f588c0a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58a410e030 tx=0x7f58980083e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:44.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.733+0000 7f588bfff640 1 -- 172.21.15.1:0/473045630 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58940bd280 con 0x7f58a4102d40 2024-09-16T00:26:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.922+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f58a4199770 con 0x7f58a4102d40 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.922+0000 7f588bfff640 1 -- 172.21.15.1:0/473045630 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+576 (secure 0 0 0) 0x7f58a4199770 con 0x7f58a4102d40 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:listed 10 entries 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/178572347 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3932831772 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3251079055 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3191294475 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/178572347 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1613718962 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/786826717 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/786826717 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/639100255 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:44.924 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/365027668 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f588c0a3240 msgr2=0x7f588c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f588c0a3240 0x7f588c0a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58a410e030 tx=0x7f58980083e0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 msgr2=0x7f58a4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 0x7f58a4198410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5894005ab0 tx=0x7f58940042f0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 shutdown_connections 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f588c0a3240 0x7f588c0a5700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a410c170 0x7f58a419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103660 0x7f58a4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 --2- 172.21.15.1:0/473045630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58a4102d40 0x7f58a4198410 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.924+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 >> 172.21.15.1:0/473045630 conn(0x7f58a40fc7a0 msgr2=0x7f58a40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.925+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 shutdown_connections 2024-09-16T00:26:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:44.925+0000 7f58a9131640 1 -- 172.21.15.1:0/473045630 wait complete. 2024-09-16T00:26:45.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:45 smithi019 ceph-mon[35186]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:45.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:45 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/473045630' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:26:45.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:45 smithi001 ceph-mon[30934]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:45.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:45 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/473045630' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:26:45.923 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:26:45.923 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:26:45.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:26:45.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:45 smithi153 ceph-mon[35538]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:45.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:45 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/473045630' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:26:46.251 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.744+0000 7f9e5f945640 1 -- 172.21.15.1:0/1835297517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58101de0 msgr2=0x7f9e581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.744+0000 7f9e5f945640 1 --2- 172.21.15.1:0/1835297517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58101de0 0x7f9e581021c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009900 tx=0x7f9e5402a410 comp rx=0 tx=0).stop 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.744+0000 7f9e5f945640 1 -- 172.21.15.1:0/1835297517 shutdown_connections 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.744+0000 7f9e5f945640 1 --2- 172.21.15.1:0/1835297517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e58106f10 0x7f9e5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.744+0000 7f9e5f945640 1 --2- 172.21.15.1:0/1835297517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e58102790 0x7f9e581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.744+0000 7f9e5f945640 1 --2- 172.21.15.1:0/1835297517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58101de0 0x7f9e581021c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.744+0000 7f9e5f945640 1 -- 172.21.15.1:0/1835297517 >> 172.21.15.1:0/1835297517 conn(0x7f9e580fd8c0 msgr2=0x7f9e580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.744+0000 7f9e5f945640 1 -- 172.21.15.1:0/1835297517 shutdown_connections 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.744+0000 7f9e5f945640 1 -- 172.21.15.1:0/1835297517 wait complete. 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5f945640 1 Processor -- start 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5f945640 1 -- start start 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5f945640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e58101de0 0x7f9e580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5f945640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e58102790 0x7f9e58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5f945640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58106f10 0x7f9e58074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5f945640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58111f90 con 0x7f9e58106f10 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5f945640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58111e10 con 0x7f9e58101de0 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5f945640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58112110 con 0x7f9e58102790 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5dee2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58106f10 0x7f9e58074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5dee2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58106f10 0x7f9e58074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58848/0 (socket says 172.21.15.1:58848) 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5dee2640 1 -- 172.21.15.1:0/215771113 learned_addr learned my addr 172.21.15.1:0/215771113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5cee0640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e58102790 0x7f9e58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5d6e1640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e58101de0 0x7f9e580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.745+0000 7f9e5dee2640 1 -- 172.21.15.1:0/215771113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e58102790 msgr2=0x7f9e58074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e5dee2640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e58102790 0x7f9e58074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e5dee2640 1 -- 172.21.15.1:0/215771113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e58101de0 msgr2=0x7f9e580744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e5dee2640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e58101de0 0x7f9e580744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e5dee2640 1 -- 172.21.15.1:0/215771113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e581a6930 con 0x7f9e58106f10 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e5dee2640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58106f10 0x7f9e58074f70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000b970 tx=0x7f9e5000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e4e7fc640 1 -- 172.21.15.1:0/215771113 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50019070 con 0x7f9e58106f10 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e4e7fc640 1 -- 172.21.15.1:0/215771113 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5000cb40 con 0x7f9e58106f10 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e581a6c20 con 0x7f9e58106f10 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e4e7fc640 1 -- 172.21.15.1:0/215771113 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50007650 con 0x7f9e58106f10 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.746+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e581a7110 con 0x7f9e58106f10 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.747+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e581021c0 con 0x7f9e58106f10 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.748+0000 7f9e4e7fc640 1 -- 172.21.15.1:0/215771113 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f9e5001c070 con 0x7f9e58106f10 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.749+0000 7f9e4e7fc640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9e440a3240 0x7f9e440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.749+0000 7f9e5d6e1640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9e440a3240 0x7f9e440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.749+0000 7f9e4e7fc640 1 -- 172.21.15.1:0/215771113 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f9e500cf340 con 0x7f9e58106f10 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.749+0000 7f9e5d6e1640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9e440a3240 0x7f9e440a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9e540096c0 tx=0x7f9e54031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.753+0000 7f9e4e7fc640 1 -- 172.21.15.1:0/215771113 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e5009bcc0 con 0x7f9e58106f10 2024-09-16T00:26:46.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.957+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9e5810a0f0 con 0x7f9e58106f10 2024-09-16T00:26:46.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.958+0000 7f9e4e7fc640 1 -- 172.21.15.1:0/215771113 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+576 (secure 0 0 0) 0x7f9e5009b5c0 con 0x7f9e58106f10 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/178572347 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3932831772 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3251079055 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3191294475 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/178572347 2024-09-17T00:19:24.226024+0000 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1613718962 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/786826717 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/786826717 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/639100255 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:46.959 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/365027668 2024-09-17T00:19:00.598574+0000 2024-09-16T00:26:46.960 INFO:teuthology.orchestra.run.smithi001.stderr:listed 10 entries 2024-09-16T00:26:46.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9e440a3240 msgr2=0x7f9e440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9e440a3240 0x7f9e440a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9e540096c0 tx=0x7f9e54031490 comp rx=0 tx=0).stop 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58106f10 msgr2=0x7f9e58074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58106f10 0x7f9e58074f70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000b970 tx=0x7f9e5000be30 comp rx=0 tx=0).stop 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 shutdown_connections 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e58106f10 0x7f9e58074f70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9e440a3240 0x7f9e440a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e58102790 0x7f9e58074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 --2- 172.21.15.1:0/215771113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e58101de0 0x7f9e580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 >> 172.21.15.1:0/215771113 conn(0x7f9e580fd8c0 msgr2=0x7f9e580fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 shutdown_connections 2024-09-16T00:26:46.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:46.961+0000 7f9e5f945640 1 -- 172.21.15.1:0/215771113 wait complete. 2024-09-16T00:26:47.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:47 smithi001 ceph-mon[30934]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:47.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:47 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/215771113' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:26:47.787 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-09-16T00:26:47.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:47.788 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:26:47.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[71708]: fuse finished with error 0 and tester_r 0 2024-09-16T00:26:47.817 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:26:47.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:47 smithi019 ceph-mon[35186]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:47.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:47 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/215771113' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:26:47.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:47 smithi153 ceph-mon[35538]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:26:47.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:47 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/215771113' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:26:49.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:49 smithi019 ceph-mon[35186]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:26:49.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:49 smithi001 ceph-mon[30934]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:26:49.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:49 smithi153 ceph-mon[35538]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:26:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:51 smithi019 ceph-mon[35186]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:26:51.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:51 smithi001 ceph-mon[30934]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:26:51.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:51 smithi153 ceph-mon[35538]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:26:53.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:53 smithi019 ceph-mon[35186]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:53.919 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-09-16T00:26:53.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:26:53.919 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-16T00:26:53.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:26:53.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:53 smithi001 ceph-mon[30934]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:53.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:53 smithi153 ceph-mon[35538]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 -- 172.21.15.1:0/4140524076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54102d40 msgr2=0x7fca54103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 --2- 172.21.15.1:0/4140524076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54102d40 0x7fca54103120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca44009900 tx=0x7fca4402a480 comp rx=0 tx=0).stop 2024-09-16T00:26:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 -- 172.21.15.1:0/4140524076 shutdown_connections 2024-09-16T00:26:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 --2- 172.21.15.1:0/4140524076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5410c170 0x7fca5410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 --2- 172.21.15.1:0/4140524076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca54103660 0x7fca5410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 --2- 172.21.15.1:0/4140524076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54102d40 0x7fca54103120 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 -- 172.21.15.1:0/4140524076 >> 172.21.15.1:0/4140524076 conn(0x7fca540fc7a0 msgr2=0x7fca540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 -- 172.21.15.1:0/4140524076 shutdown_connections 2024-09-16T00:26:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 -- 172.21.15.1:0/4140524076 wait complete. 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 Processor -- start 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.876+0000 7fca58aec640 1 -- start start 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca58aec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca54102d40 0x7fca54198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca58aec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54103660 0x7fca54198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca58aec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5410c170 0x7fca5419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca58aec640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fca54112030 con 0x7fca54103660 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca58aec640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca54111eb0 con 0x7fca5410c170 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca58aec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca541121b0 con 0x7fca54102d40 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca51d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54103660 0x7fca54198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca5259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca54102d40 0x7fca54198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca5259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca54102d40 0x7fca54198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:46102/0 (socket says 172.21.15.1:46102) 2024-09-16T00:26:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca5259c640 1 -- 172.21.15.1:0/505638521 learned_addr learned my addr 172.21.15.1:0/505638521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca51d9b640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54103660 0x7fca54198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53584/0 (socket says 172.21.15.1:53584) 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca52d9d640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5410c170 0x7fca5419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca51d9b640 1 -- 172.21.15.1:0/505638521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca54102d40 msgr2=0x7fca54198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca51d9b640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca54102d40 0x7fca54198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca51d9b640 1 -- 172.21.15.1:0/505638521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5410c170 msgr2=0x7fca5419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca51d9b640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5410c170 0x7fca5419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca51d9b640 1 -- 172.21.15.1:0/505638521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5419d640 con 0x7fca54103660 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.877+0000 7fca51d9b640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54103660 0x7fca54198bd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca4800be20 tx=0x7fca4800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.878+0000 7fca437fe640 1 -- 172.21.15.1:0/505638521 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48019070 con 0x7fca54103660 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.878+0000 7fca437fe640 1 -- 172.21.15.1:0/505638521 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca48014920 con 0x7fca54103660 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.878+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5419d930 con 0x7fca54103660 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.878+0000 7fca437fe640 1 -- 172.21.15.1:0/505638521 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480128b0 con 0x7fca54103660 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.878+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca541a5210 con 0x7fca54103660 2024-09-16T00:26:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.879+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca541031a0 con 0x7fca54103660 2024-09-16T00:26:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.880+0000 7fca437fe640 1 -- 172.21.15.1:0/505638521 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fca4801c070 con 0x7fca54103660 2024-09-16T00:26:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.880+0000 7fca437fe640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca3c0a3240 0x7fca3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.881+0000 7fca5259c640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca3c0a3240 0x7fca3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.881+0000 7fca437fe640 1 -- 172.21.15.1:0/505638521 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fca480cf4c0 con 0x7fca54103660 2024-09-16T00:26:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.881+0000 7fca5259c640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca3c0a3240 0x7fca3c0a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca440096c0 tx=0x7fca44005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:54.885+0000 7fca437fe640 1 -- 172.21.15.1:0/505638521 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca4809b850 con 0x7fca54103660 2024-09-16T00:26:55.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.062+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca541071e0 con 0x7fca54103660 2024-09-16T00:26:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.064+0000 7fca437fe640 1 -- 172.21.15.1:0/505638521 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24601 (secure 0 0 0) 0x7fca4809fb50 con 0x7fca54103660 2024-09-16T00:26:55.065 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:55.065 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":89,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:26:28.990229+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:26:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca3c0a3240 msgr2=0x7fca3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca3c0a3240 0x7fca3c0a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca440096c0 tx=0x7fca44005a20 comp rx=0 tx=0).stop 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54103660 msgr2=0x7fca54198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54103660 0x7fca54198bd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca4800be20 tx=0x7fca4800bf10 comp rx=0 tx=0).stop 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 shutdown_connections 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5410c170 0x7fca5419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca54103660 0x7fca54198bd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca3c0a3240 0x7fca3c0a5700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 --2- 172.21.15.1:0/505638521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca54102d40 0x7fca54198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 >> 172.21.15.1:0/505638521 conn(0x7fca540fc7a0 msgr2=0x7fca540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 shutdown_connections 2024-09-16T00:26:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:55.067+0000 7fca58aec640 1 -- 172.21.15.1:0/505638521 wait complete. 2024-09-16T00:26:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:55 smithi001 ceph-mon[30934]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:55 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/505638521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:55.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:26:55.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:55 smithi019 ceph-mon[35186]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:55.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:55 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/505638521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:55.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:55 smithi153 ceph-mon[35538]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:55.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:55 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/505638521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:56.093 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.645+0000 7f5d3b427640 1 -- 172.21.15.1:0/1471272059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34100a80 msgr2=0x7f5d34100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.645+0000 7f5d3b427640 1 --2- 172.21.15.1:0/1471272059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34100a80 0x7f5d34100ee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009930 tx=0x7f5d2402a420 comp rx=0 tx=0).stop 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.645+0000 7f5d3b427640 1 -- 172.21.15.1:0/1471272059 shutdown_connections 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.645+0000 7f5d3b427640 1 --2- 172.21.15.1:0/1471272059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d34101420 0x7f5d3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.645+0000 7f5d3b427640 1 --2- 172.21.15.1:0/1471272059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34100a80 0x7f5d34100ee0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.645+0000 7f5d3b427640 1 --2- 172.21.15.1:0/1471272059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d34106a80 0x7f5d34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.645+0000 7f5d3b427640 1 -- 172.21.15.1:0/1471272059 >> 172.21.15.1:0/1471272059 conn(0x7f5d340fc7a0 msgr2=0x7f5d340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.645+0000 7f5d3b427640 1 -- 172.21.15.1:0/1471272059 shutdown_connections 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.645+0000 7f5d3b427640 1 -- 172.21.15.1:0/1471272059 wait complete. 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d3b427640 1 Processor -- start 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d3b427640 1 -- start start 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d3b427640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d34100a80 0x7f5d341973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d3b427640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34101420 0x7f5d34197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d3b427640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d34106a80 0x7f5d3419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d3b427640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d34110e80 con 0x7f5d34101420 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d3b427640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d34110d00 con 0x7f5d34106a80 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d3b427640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d34111000 con 0x7f5d34100a80 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d389c2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34101420 0x7f5d34197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d389c2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34101420 0x7f5d34197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53600/0 (socket says 172.21.15.1:53600) 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.646+0000 7f5d389c2640 1 -- 172.21.15.1:0/2487947035 learned_addr learned my addr 172.21.15.1:0/2487947035 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d399c4640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d34106a80 0x7f5d3419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d391c3640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d34100a80 0x7f5d341973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d389c2640 1 -- 172.21.15.1:0/2487947035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d34100a80 msgr2=0x7f5d341973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d389c2640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d34100a80 0x7f5d341973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d389c2640 1 -- 172.21.15.1:0/2487947035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d34106a80 msgr2=0x7f5d3419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d389c2640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d34106a80 0x7f5d3419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d389c2640 1 -- 172.21.15.1:0/2487947035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3419c370 con 0x7f5d34101420 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d391c3640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d34100a80 0x7f5d341973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d389c2640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34101420 0x7f5d34197900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009230 tx=0x7f5d24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d2a7fc640 1 -- 172.21.15.1:0/2487947035 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2403f030 con 0x7f5d34101420 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d2a7fc640 1 -- 172.21.15.1:0/2487947035 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d24003ad0 con 0x7f5d34101420 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3419c600 con 0x7f5d34101420 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d2a7fc640 1 -- 172.21.15.1:0/2487947035 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24035070 con 0x7f5d34101420 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.647+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d341a3f40 con 0x7f5d34101420 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.648+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d341021c0 con 0x7f5d34101420 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.649+0000 7f5d2a7fc640 1 -- 172.21.15.1:0/2487947035 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f5d24042030 con 0x7f5d34101420 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.650+0000 7f5d2a7fc640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5d200a3240 0x7f5d200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.650+0000 7f5d2a7fc640 1 -- 172.21.15.1:0/2487947035 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f5d240f0180 con 0x7f5d34101420 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.650+0000 7f5d391c3640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5d200a3240 0x7f5d200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.651+0000 7f5d391c3640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5d200a3240 0x7f5d200a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d30005fd0 tx=0x7f5d3000b5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:56.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.654+0000 7f5d2a7fc640 1 -- 172.21.15.1:0/2487947035 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d240bcb00 con 0x7f5d34101420 2024-09-16T00:26:56.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.832+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d34108ff0 con 0x7f5d34101420 2024-09-16T00:26:56.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.834+0000 7f5d2a7fc640 1 -- 172.21.15.1:0/2487947035 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24601 (secure 0 0 0) 0x7f5d240bc400 con 0x7f5d34101420 2024-09-16T00:26:56.834 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:26:56.835 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":89,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:26:28.990229+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:26:56.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5d200a3240 msgr2=0x7f5d200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5d200a3240 0x7f5d200a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d30005fd0 tx=0x7f5d3000b5d0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34101420 msgr2=0x7f5d34197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34101420 0x7f5d34197900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009230 tx=0x7f5d24004270 comp rx=0 tx=0).stop 2024-09-16T00:26:56.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 shutdown_connections 2024-09-16T00:26:56.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d34106a80 0x7f5d3419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d34101420 0x7f5d34197900 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5d200a3240 0x7f5d200a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 --2- 172.21.15.1:0/2487947035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d34100a80 0x7f5d341973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 >> 172.21.15.1:0/2487947035 conn(0x7f5d340fc7a0 msgr2=0x7f5d340fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:56.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.836+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 shutdown_connections 2024-09-16T00:26:56.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:56.837+0000 7f5d3b427640 1 -- 172.21.15.1:0/2487947035 wait complete. 2024-09-16T00:26:57.543 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-16T00:26:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:57.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:57 smithi019 ceph-mon[35186]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:57.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:57 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2487947035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:57.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:57 smithi001 ceph-mon[30934]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:57.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:57 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2487947035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:57.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:57 smithi153 ceph-mon[35538]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:57.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:57 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2487947035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:26:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.332+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3133985994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 msgr2=0x7ffbdc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.332+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3133985994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 0x7ffbdc10ec40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009cd0 tx=0x7ffbd402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.332+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3133985994 shutdown_connections 2024-09-16T00:26:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.332+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3133985994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 0x7ffbdc10ec40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.332+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3133985994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbdc1036e0 0x7ffbdc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.332+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3133985994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbdc102d30 0x7ffbdc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.332+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3133985994 >> 172.21.15.1:0/3133985994 conn(0x7ffbdc0fc7d0 msgr2=0x7ffbdc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.332+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3133985994 shutdown_connections 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.332+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3133985994 wait complete. 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.333+0000 7ffbe29ad640 1 Processor -- start 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.333+0000 7ffbe29ad640 1 -- start start 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.333+0000 7ffbe29ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbdc102d30 0x7ffbdc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.333+0000 7ffbe29ad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbdc1036e0 0x7ffbdc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.333+0000 7ffbe29ad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 0x7ffbdc19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.333+0000 7ffbe29ad640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc112100 con 0x7ffbdc1082a0 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.333+0000 7ffbe29ad640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc111f80 con 0x7ffbdc102d30 2024-09-16T00:26:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.333+0000 7ffbe29ad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc112280 con 0x7ffbdc1036e0 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe0f4a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 0x7ffbdc19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe0f4a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 0x7ffbdc19cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53608/0 (socket says 172.21.15.1:53608) 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbdc102d30 0x7ffbdc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe0f4a640 1 -- 172.21.15.1:0/3139308033 learned_addr learned my addr 172.21.15.1:0/3139308033 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbdb7fe640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbdc1036e0 0x7ffbdc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe0f4a640 1 -- 172.21.15.1:0/3139308033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbdc1036e0 msgr2=0x7ffbdc198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe0f4a640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbdc1036e0 0x7ffbdc198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe0f4a640 1 -- 172.21.15.1:0/3139308033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbdc102d30 msgr2=0x7ffbdc1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe0f4a640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbdc102d30 0x7ffbdc1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe0f4a640 1 -- 172.21.15.1:0/3139308033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc19d580 con 0x7ffbdc1082a0 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe0f4a640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 0x7ffbdc19cea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005cc0 tx=0x7ffbd4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbd97fa640 1 -- 172.21.15.1:0/3139308033 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4004040 con 0x7ffbdc1082a0 2024-09-16T00:26:58.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc19d810 con 0x7ffbdc1082a0 2024-09-16T00:26:58.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbd97fa640 1 -- 172.21.15.1:0/3139308033 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd40046c0 con 0x7ffbdc1082a0 2024-09-16T00:26:58.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbd97fa640 1 -- 172.21.15.1:0/3139308033 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4030e70 con 0x7ffbdc1082a0 2024-09-16T00:26:58.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.334+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbdc1a5150 con 0x7ffbdc1082a0 2024-09-16T00:26:58.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.335+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbdc103110 con 0x7ffbdc1082a0 2024-09-16T00:26:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.337+0000 7ffbd97fa640 1 -- 172.21.15.1:0/3139308033 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7ffbd40305a0 con 0x7ffbdc1082a0 2024-09-16T00:26:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.337+0000 7ffbd97fa640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbc40a3240 0x7ffbc40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.337+0000 7ffbdbfff640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbc40a3240 0x7ffbc40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.337+0000 7ffbd97fa640 1 -- 172.21.15.1:0/3139308033 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7ffbd402e030 con 0x7ffbdc1082a0 2024-09-16T00:26:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.338+0000 7ffbdbfff640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbc40a3240 0x7ffbc40a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009990 tx=0x7ffbcc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:58.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.341+0000 7ffbd97fa640 1 -- 172.21.15.1:0/3139308033 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbd40bdcb0 con 0x7ffbdc1082a0 2024-09-16T00:26:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.555+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffbdc199930 con 0x7ffbdc1082a0 2024-09-16T00:26:58.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.581+0000 7ffbd97fa640 1 -- 172.21.15.1:0/3139308033 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7ffbdc199930 con 0x7ffbdc1082a0 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbc40a3240 msgr2=0x7ffbc40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbc40a3240 0x7ffbc40a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009990 tx=0x7ffbcc009110 comp rx=0 tx=0).stop 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 msgr2=0x7ffbdc19cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 0x7ffbdc19cea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005cc0 tx=0x7ffbd4003970 comp rx=0 tx=0).stop 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 shutdown_connections 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbdc1082a0 0x7ffbdc19cea0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffbc40a3240 0x7ffbc40a5700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbdc1036e0 0x7ffbdc198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 --2- 172.21.15.1:0/3139308033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbdc102d30 0x7ffbdc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 >> 172.21.15.1:0/3139308033 conn(0x7ffbdc0fc7d0 msgr2=0x7ffbdc105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:58.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.583+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 shutdown_connections 2024-09-16T00:26:58.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:26:58.584+0000 7ffbe29ad640 1 -- 172.21.15.1:0/3139308033 wait complete. 2024-09-16T00:26:58.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:58 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3139308033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:26:58.874 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:58 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3139308033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:26:58.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:58 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3139308033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:26:59.380 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-16T00:26:59.380 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-16T00:26:59.642 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:59.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:59 smithi001 ceph-mon[30934]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:59.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:59 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3139308033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:26:59.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:26:59 smithi001 ceph-mon[30934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:59.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:59 smithi019 ceph-mon[35186]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:59.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:59 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3139308033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:26:59.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:26:59 smithi019 ceph-mon[35186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:26:59.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:59 smithi153 ceph-mon[35538]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:26:59.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:59 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3139308033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:26:59.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:26:59 smithi153 ceph-mon[35538]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:27:00.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3255886630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc106f10 msgr2=0x7f2ccc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:00.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3255886630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc106f10 0x7f2ccc10ea90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009ca0 tx=0x7f2cc402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3255886630 shutdown_connections 2024-09-16T00:27:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3255886630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc106f10 0x7f2ccc10ea90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3255886630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ccc102790 0x7f2ccc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3255886630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc101de0 0x7f2ccc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3255886630 >> 172.21.15.1:0/3255886630 conn(0x7f2ccc0fd8c0 msgr2=0x7f2ccc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3255886630 shutdown_connections 2024-09-16T00:27:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3255886630 wait complete. 2024-09-16T00:27:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 Processor -- start 2024-09-16T00:27:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.157+0000 7f2cd26ec640 1 -- start start 2024-09-16T00:27:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2cd26ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc101de0 0x7f2ccc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2cd26ec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc102790 0x7f2ccc198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2cd26ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ccc106f10 0x7f2ccc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2cd26ec640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc111f70 con 0x7f2ccc102790 2024-09-16T00:27:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2cd26ec640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc111df0 con 0x7f2ccc101de0 2024-09-16T00:27:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2cd26ec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1120f0 con 0x7f2ccc106f10 2024-09-16T00:27:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc102790 0x7f2ccc198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc102790 0x7f2ccc198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53612/0 (socket says 172.21.15.1:53612) 2024-09-16T00:27:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccb7fe640 1 -- 172.21.15.1:0/3870653921 learned_addr learned my addr 172.21.15.1:0/3870653921 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2cd0c89640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ccc106f10 0x7f2ccc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccbfff640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc101de0 0x7f2ccc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccb7fe640 1 -- 172.21.15.1:0/3870653921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ccc106f10 msgr2=0x7f2ccc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccb7fe640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ccc106f10 0x7f2ccc19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccb7fe640 1 -- 172.21.15.1:0/3870653921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc101de0 msgr2=0x7f2ccc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccb7fe640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc101de0 0x7f2ccc1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccb7fe640 1 -- 172.21.15.1:0/3870653921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc19d290 con 0x7f2ccc102790 2024-09-16T00:27:00.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.158+0000 7f2ccbfff640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc101de0 0x7f2ccc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:00.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.159+0000 7f2ccb7fe640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc102790 0x7f2ccc198820 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000c920 tx=0x7f2cc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:00.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.159+0000 7f2cc97fa640 1 -- 172.21.15.1:0/3870653921 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0007c70 con 0x7f2ccc102790 2024-09-16T00:27:00.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.159+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc19d580 con 0x7f2ccc102790 2024-09-16T00:27:00.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.159+0000 7f2cc97fa640 1 -- 172.21.15.1:0/3870653921 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0007e10 con 0x7f2ccc102790 2024-09-16T00:27:00.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.159+0000 7f2cc97fa640 1 -- 172.21.15.1:0/3870653921 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0018890 con 0x7f2ccc102790 2024-09-16T00:27:00.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.159+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc1a4e60 con 0x7f2ccc102790 2024-09-16T00:27:00.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.160+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc1021c0 con 0x7f2ccc102790 2024-09-16T00:27:00.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.161+0000 7f2cc97fa640 1 -- 172.21.15.1:0/3870653921 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f2cc0018a70 con 0x7f2ccc102790 2024-09-16T00:27:00.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.162+0000 7f2cc97fa640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb40a3240 0x7f2cb40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:00.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.162+0000 7f2ccbfff640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb40a3240 0x7f2cb40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:00.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.162+0000 7f2cc97fa640 1 -- 172.21.15.1:0/3870653921 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f2cc00cfd50 con 0x7f2ccc102790 2024-09-16T00:27:00.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.163+0000 7f2ccbfff640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb40a3240 0x7f2cb40a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00abe0 tx=0x7f2cbc00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.166+0000 7f2cc97fa640 1 -- 172.21.15.1:0/3870653921 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2cc00d3030 con 0x7f2ccc102790 2024-09-16T00:27:00.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.377+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f2ccc1a51a0 con 0x7f2ccc102790 2024-09-16T00:27:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.588+0000 7f2cc97fa640 1 -- 172.21.15.1:0/3870653921 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f2cc0014030 con 0x7f2ccc102790 2024-09-16T00:27:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb40a3240 msgr2=0x7f2cb40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb40a3240 0x7f2cb40a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00abe0 tx=0x7f2cbc00a5c0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc102790 msgr2=0x7f2ccc198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc102790 0x7f2ccc198820 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000c920 tx=0x7f2cc000cde0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 shutdown_connections 2024-09-16T00:27:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb40a3240 0x7f2cb40a5700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ccc106f10 0x7f2ccc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ccc102790 0x7f2ccc198820 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 --2- 172.21.15.1:0/3870653921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc101de0 0x7f2ccc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 >> 172.21.15.1:0/3870653921 conn(0x7f2ccc0fd8c0 msgr2=0x7f2ccc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 shutdown_connections 2024-09-16T00:27:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:00.591+0000 7f2cd26ec640 1 -- 172.21.15.1:0/3870653921 wait complete. 2024-09-16T00:27:00.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:00 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3870653921' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T00:27:00.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:00 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3870653921' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T00:27:00.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:00 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3870653921' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T00:27:01.407 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-16T00:27:01.678 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:01.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:01 smithi001 ceph-mon[30934]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-16T00:27:01.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:01 smithi001 ceph-mon[30934]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:27:01.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:01 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3870653921' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-16T00:27:01.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:01 smithi001 ceph-mon[30934]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:27:01.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:01 smithi001 ceph-mon[30934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:27:01.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:01 smithi001 ceph-mon[30934]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-16T00:27:01.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:01 smithi001 ceph-mon[30934]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-16T00:27:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:01 smithi019 ceph-mon[35186]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-16T00:27:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:01 smithi019 ceph-mon[35186]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:27:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:01 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3870653921' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-16T00:27:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:01 smithi019 ceph-mon[35186]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:27:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:01 smithi019 ceph-mon[35186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:27:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:01 smithi019 ceph-mon[35186]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-16T00:27:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:01 smithi019 ceph-mon[35186]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-16T00:27:01.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:01 smithi153 ceph-mon[35538]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-16T00:27:01.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:01 smithi153 ceph-mon[35538]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:27:01.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:01 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3870653921' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-16T00:27:01.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:01 smithi153 ceph-mon[35538]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:27:01.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:01 smithi153 ceph-mon[35538]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:27:01.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:01 smithi153 ceph-mon[35538]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-16T00:27:01.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:01 smithi153 ceph-mon[35538]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-16T00:27:02.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 -- 172.21.15.1:0/4002621020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484100e20 msgr2=0x7f2484101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:02.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 --2- 172.21.15.1:0/4002621020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484100e20 0x7f2484101200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2474009900 tx=0x7f247402a480 comp rx=0 tx=0).stop 2024-09-16T00:27:02.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 -- 172.21.15.1:0/4002621020 shutdown_connections 2024-09-16T00:27:02.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 --2- 172.21.15.1:0/4002621020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2484102ef0 0x7f248410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 --2- 172.21.15.1:0/4002621020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2484102570 0x7f24841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 --2- 172.21.15.1:0/4002621020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484100e20 0x7f2484101200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 -- 172.21.15.1:0/4002621020 >> 172.21.15.1:0/4002621020 conn(0x7f24840fc7d0 msgr2=0x7f24840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:02.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 -- 172.21.15.1:0/4002621020 shutdown_connections 2024-09-16T00:27:02.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 -- 172.21.15.1:0/4002621020 wait complete. 2024-09-16T00:27:02.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.160+0000 7f2488b90640 1 Processor -- start 2024-09-16T00:27:02.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2488b90640 1 -- start start 2024-09-16T00:27:02.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2488b90640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2484100e20 0x7f24841984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:02.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2488b90640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484102570 0x7f2484198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:02.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2488b90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2484102ef0 0x7f248419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:02.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2488b90640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24841120a0 con 0x7f2484102570 2024-09-16T00:27:02.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2488b90640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2484111f20 con 0x7f2484102ef0 2024-09-16T00:27:02.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2488b90640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2484112220 con 0x7f2484100e20 2024-09-16T00:27:02.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2479d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484102570 0x7f2484198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:02.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2479d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484102570 0x7f2484198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48250/0 (socket says 172.21.15.1:48250) 2024-09-16T00:27:02.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2479d9b640 1 -- 172.21.15.1:0/3047904744 learned_addr learned my addr 172.21.15.1:0/3047904744 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:02.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f248259c640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2484100e20 0x7f24841984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:02.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2482d9d640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2484102ef0 0x7f248419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:02.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2479d9b640 1 -- 172.21.15.1:0/3047904744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2484100e20 msgr2=0x7f24841984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:02.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2479d9b640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2484100e20 0x7f24841984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2479d9b640 1 -- 172.21.15.1:0/3047904744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2484102ef0 msgr2=0x7f248419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2479d9b640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2484102ef0 0x7f248419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2479d9b640 1 -- 172.21.15.1:0/3047904744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248419d480 con 0x7f2484102570 2024-09-16T00:27:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.161+0000 7f2479d9b640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484102570 0x7f2484198a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f246c00e9f0 tx=0x7f246c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.162+0000 7f247bfff640 1 -- 172.21.15.1:0/3047904744 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c00cd80 con 0x7f2484102570 2024-09-16T00:27:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.162+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248419d770 con 0x7f2484102570 2024-09-16T00:27:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.162+0000 7f247bfff640 1 -- 172.21.15.1:0/3047904744 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f246c010910 con 0x7f2484102570 2024-09-16T00:27:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.162+0000 7f247bfff640 1 -- 172.21.15.1:0/3047904744 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0188f0 con 0x7f2484102570 2024-09-16T00:27:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.162+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24841a5050 con 0x7f2484102570 2024-09-16T00:27:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.163+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2484101200 con 0x7f2484102570 2024-09-16T00:27:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.164+0000 7f247bfff640 1 -- 172.21.15.1:0/3047904744 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f246c010ac0 con 0x7f2484102570 2024-09-16T00:27:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.165+0000 7f247bfff640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24700a3240 0x7f24700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.165+0000 7f248259c640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24700a3240 0x7f24700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.165+0000 7f247bfff640 1 -- 172.21.15.1:0/3047904744 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f246c0d03e0 con 0x7f2484102570 2024-09-16T00:27:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.165+0000 7f248259c640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24700a3240 0x7f24700a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2474009230 tx=0x7f24740058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.168+0000 7f247bfff640 1 -- 172.21.15.1:0/3047904744 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f246c09c770 con 0x7f2484102570 2024-09-16T00:27:02.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.384+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2484199830 con 0x7f2484102570 2024-09-16T00:27:02.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.505+0000 7f247bfff640 1 -- 172.21.15.1:0/3047904744 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f2484199830 con 0x7f2484102570 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24700a3240 msgr2=0x7f24700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24700a3240 0x7f24700a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2474009230 tx=0x7f24740058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484102570 msgr2=0x7f2484198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484102570 0x7f2484198a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f246c00e9f0 tx=0x7f246c00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 shutdown_connections 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2484102ef0 0x7f248419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2484102570 0x7f2484198a10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24700a3240 0x7f24700a5700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 --2- 172.21.15.1:0/3047904744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2484100e20 0x7f24841984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.507+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 >> 172.21.15.1:0/3047904744 conn(0x7f24840fc7d0 msgr2=0x7f24840fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.508+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 shutdown_connections 2024-09-16T00:27:02.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:02.508+0000 7f2488b90640 1 -- 172.21.15.1:0/3047904744 wait complete. 2024-09-16T00:27:02.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] up:active 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: Cluster is now healthy 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3047904744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3047904744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] up:active 2024-09-16T00:27:02.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:02 smithi001 ceph-mon[30934]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:27:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] up:active 2024-09-16T00:27:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T00:27:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:27:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: Cluster is now healthy 2024-09-16T00:27:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-16T00:27:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-16T00:27:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-16T00:27:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3047904744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:27:02.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3047904744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:27:02.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] up:active 2024-09-16T00:27:02.858 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:02 smithi019 ceph-mon[35186]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] up:active 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: Cluster is now healthy 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3047904744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3047904744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] up:active 2024-09-16T00:27:02.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:02 smithi153 ceph-mon[35538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:27:03.342 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-16T00:27:03.609 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:03.854 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:03 smithi019 ceph-mon[35186]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-16T00:27:03.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:03 smithi001 ceph-mon[30934]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-16T00:27:03.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:03 smithi153 ceph-mon[35538]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-16T00:27:04.107 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:27:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b[65807]: 2024-09-16T00:27:03.853+0000 7fa6359e1640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-16T00:27:04.107 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:27:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b[65807]: 2024-09-16T00:27:03.853+0000 7fa6359e1640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-16T00:27:04.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.110+0000 7ff2aa524640 1 -- 172.21.15.1:0/269948282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4102d40 msgr2=0x7ff2a4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.110+0000 7ff2aa524640 1 --2- 172.21.15.1:0/269948282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4102d40 0x7ff2a4103120 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff294009900 tx=0x7ff29402a480 comp rx=0 tx=0).stop 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.110+0000 7ff2aa524640 1 -- 172.21.15.1:0/269948282 shutdown_connections 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.110+0000 7ff2aa524640 1 --2- 172.21.15.1:0/269948282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a410c170 0x7ff2a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.110+0000 7ff2aa524640 1 --2- 172.21.15.1:0/269948282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a4103660 0x7ff2a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.110+0000 7ff2aa524640 1 --2- 172.21.15.1:0/269948282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4102d40 0x7ff2a4103120 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.110+0000 7ff2aa524640 1 -- 172.21.15.1:0/269948282 >> 172.21.15.1:0/269948282 conn(0x7ff2a40fc7a0 msgr2=0x7ff2a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.110+0000 7ff2aa524640 1 -- 172.21.15.1:0/269948282 shutdown_connections 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.110+0000 7ff2aa524640 1 -- 172.21.15.1:0/269948282 wait complete. 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff2aa524640 1 Processor -- start 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff2aa524640 1 -- start start 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff2aa524640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a4102d40 0x7ff2a41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:04.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff2aa524640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4103660 0x7ff2a4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff2aa524640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a410c170 0x7ff2a419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff2aa524640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4111f70 con 0x7ff2a4103660 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff2aa524640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4111df0 con 0x7ff2a4102d40 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff2aa524640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41120f0 con 0x7ff2a410c170 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4103660 0x7ff2a4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4103660 0x7ff2a4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48262/0 (socket says 172.21.15.1:48262) 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff29bfff640 1 -- 172.21.15.1:0/183384471 learned_addr learned my addr 172.21.15.1:0/183384471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.111+0000 7ff2a3fff640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a4102d40 0x7ff2a41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff2a8ac1640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a410c170 0x7ff2a419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff29bfff640 1 -- 172.21.15.1:0/183384471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a410c170 msgr2=0x7ff2a419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff29bfff640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a410c170 0x7ff2a419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff29bfff640 1 -- 172.21.15.1:0/183384471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a4102d40 msgr2=0x7ff2a41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:04.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff29bfff640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a4102d40 0x7ff2a41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff29bfff640 1 -- 172.21.15.1:0/183384471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a419d390 con 0x7ff2a4103660 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff2a3fff640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a4102d40 0x7ff2a41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff29bfff640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4103660 0x7ff2a4198920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c940 tx=0x7ff28c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff2a1ffb640 1 -- 172.21.15.1:0/183384471 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004220 con 0x7ff2a4103660 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a419d680 con 0x7ff2a4103660 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff2a1ffb640 1 -- 172.21.15.1:0/183384471 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c0043c0 con 0x7ff2a4103660 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff2a1ffb640 1 -- 172.21.15.1:0/183384471 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004a00 con 0x7ff2a4103660 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.112+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a41a4f60 con 0x7ff2a4103660 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.113+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a4103120 con 0x7ff2a4103660 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.114+0000 7ff2a1ffb640 1 -- 172.21.15.1:0/183384471 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7ff28c0079b0 con 0x7ff2a4103660 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.115+0000 7ff2a1ffb640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2900a3240 0x7ff2900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.115+0000 7ff2a1ffb640 1 -- 172.21.15.1:0/183384471 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7ff28c0cfd20 con 0x7ff2a4103660 2024-09-16T00:27:04.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.115+0000 7ff2a3fff640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2900a3240 0x7ff2900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:04.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.116+0000 7ff2a3fff640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2900a3240 0x7ff2900a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff294005ab0 tx=0x7ff294005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:04.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.118+0000 7ff2a1ffb640 1 -- 172.21.15.1:0/183384471 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff28c014030 con 0x7ff2a4103660 2024-09-16T00:27:04.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.333+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff2a402d7c0 con 0x7ff2a4103660 2024-09-16T00:27:04.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.606+0000 7ff2a1ffb640 1 -- 172.21.15.1:0/183384471 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7ff28c09c6a0 con 0x7ff2a4103660 2024-09-16T00:27:04.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2900a3240 msgr2=0x7ff2900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:04.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2900a3240 0x7ff2900a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff294005ab0 tx=0x7ff294005a20 comp rx=0 tx=0).stop 2024-09-16T00:27:04.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4103660 msgr2=0x7ff2a4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:04.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4103660 0x7ff2a4198920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c940 tx=0x7ff28c00ce00 comp rx=0 tx=0).stop 2024-09-16T00:27:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 shutdown_connections 2024-09-16T00:27:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff2900a3240 0x7ff2900a5700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a410c170 0x7ff2a419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a4103660 0x7ff2a4198920 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 --2- 172.21.15.1:0/183384471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a4102d40 0x7ff2a41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 >> 172.21.15.1:0/183384471 conn(0x7ff2a40fc7a0 msgr2=0x7ff2a40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 shutdown_connections 2024-09-16T00:27:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:04.609+0000 7ff2aa524640 1 -- 172.21.15.1:0/183384471 wait complete. 2024-09-16T00:27:04.863 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:04 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/183384471' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:27:04.921 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:04 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/183384471' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:27:05.007 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:04 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/183384471' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:27:05.291 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-16T00:27:05.292 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-16T00:27:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:05.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:05 smithi001 ceph-mon[30934]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:27:05.864 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:05 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/183384471' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:27:05.864 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:05 smithi001 ceph-mon[30934]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:27:05.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:05 smithi153 ceph-mon[35538]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:27:05.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:05 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/183384471' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:27:05.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:05 smithi153 ceph-mon[35538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:27:06.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.094+0000 7faab718f640 1 -- 172.21.15.1:0/1094404075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab01027a0 msgr2=0x7faab01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:06.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.094+0000 7faab718f640 1 --2- 172.21.15.1:0/1094404075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab01027a0 0x7faab01067f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009900 tx=0x7faa9c02a410 comp rx=0 tx=0).stop 2024-09-16T00:27:06.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.094+0000 7faab718f640 1 -- 172.21.15.1:0/1094404075 shutdown_connections 2024-09-16T00:27:06.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.094+0000 7faab718f640 1 --2- 172.21.15.1:0/1094404075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faab0106f20 0x7faab010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.094+0000 7faab718f640 1 --2- 172.21.15.1:0/1094404075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab01027a0 0x7faab01067f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.094+0000 7faab718f640 1 --2- 172.21.15.1:0/1094404075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faab0101df0 0x7faab01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.094+0000 7faab718f640 1 -- 172.21.15.1:0/1094404075 >> 172.21.15.1:0/1094404075 conn(0x7faab00fd8b0 msgr2=0x7faab00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.094+0000 7faab718f640 1 -- 172.21.15.1:0/1094404075 shutdown_connections 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.094+0000 7faab718f640 1 -- 172.21.15.1:0/1094404075 wait complete. 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab718f640 1 Processor -- start 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab718f640 1 -- start start 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab718f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab0101df0 0x7faab00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab718f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faab01027a0 0x7faab0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab718f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faab0106f20 0x7faab0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab718f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faab0111f80 con 0x7faab0101df0 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab718f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faab0111e00 con 0x7faab01027a0 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab718f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faab0112100 con 0x7faab0106f20 2024-09-16T00:27:06.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab4f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab0101df0 0x7faab00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab4f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab0101df0 0x7faab00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48280/0 (socket says 172.21.15.1:48280) 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab4f2b640 1 -- 172.21.15.1:0/313272967 learned_addr learned my addr 172.21.15.1:0/313272967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faaaffff640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faab01027a0 0x7faab0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab572c640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faab0106f20 0x7faab0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab4f2b640 1 -- 172.21.15.1:0/313272967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faab0106f20 msgr2=0x7faab0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab4f2b640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faab0106f20 0x7faab0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab4f2b640 1 -- 172.21.15.1:0/313272967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faab01027a0 msgr2=0x7faab0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab4f2b640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faab01027a0 0x7faab0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab4f2b640 1 -- 172.21.15.1:0/313272967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab01a68c0 con 0x7faab0101df0 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab572c640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faab0106f20 0x7faab0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.095+0000 7faab4f2b640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab0101df0 0x7faab00744f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faaa400d950 tx=0x7faaa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.096+0000 7faaadffb640 1 -- 172.21.15.1:0/313272967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4017070 con 0x7faab0101df0 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.096+0000 7faaadffb640 1 -- 172.21.15.1:0/313272967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa4009c30 con 0x7faab0101df0 2024-09-16T00:27:06.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.096+0000 7faaadffb640 1 -- 172.21.15.1:0/313272967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa400f6b0 con 0x7faab0101df0 2024-09-16T00:27:06.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.096+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab01a6bb0 con 0x7faab0101df0 2024-09-16T00:27:06.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.096+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab01a70c0 con 0x7faab0101df0 2024-09-16T00:27:06.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.097+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab01021d0 con 0x7faab0101df0 2024-09-16T00:27:06.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.098+0000 7faaadffb640 1 -- 172.21.15.1:0/313272967 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7faaa40109d0 con 0x7faab0101df0 2024-09-16T00:27:06.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.098+0000 7faaadffb640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faaa00a3240 0x7faaa00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:06.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.099+0000 7faaaffff640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faaa00a3240 0x7faaa00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:06.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.099+0000 7faaadffb640 1 -- 172.21.15.1:0/313272967 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7faaa40cf8a0 con 0x7faab0101df0 2024-09-16T00:27:06.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.099+0000 7faaaffff640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faaa00a3240 0x7faaa00a5700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009230 tx=0x7faa9c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:06.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.103+0000 7faaadffb640 1 -- 172.21.15.1:0/313272967 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faaa4009de0 con 0x7faab0101df0 2024-09-16T00:27:06.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:05 smithi019 ceph-mon[35186]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:27:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:05 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/183384471' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:27:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:05 smithi019 ceph-mon[35186]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:27:06.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.306+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faab0062630 con 0x7faaa00a3240 2024-09-16T00:27:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.331+0000 7faaadffb640 1 -- 172.21.15.1:0/313272967 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faab0062630 con 0x7faaa00a3240 2024-09-16T00:27:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faaa00a3240 msgr2=0x7faaa00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faaa00a3240 0x7faaa00a5700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009230 tx=0x7faa9c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab0101df0 msgr2=0x7faab00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab0101df0 0x7faab00744f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faaa400d950 tx=0x7faaa400de10 comp rx=0 tx=0).stop 2024-09-16T00:27:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 shutdown_connections 2024-09-16T00:27:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faaa00a3240 0x7faaa00a5700 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faab0106f20 0x7faab0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faab01027a0 0x7faab0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 --2- 172.21.15.1:0/313272967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faab0101df0 0x7faab00744f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 >> 172.21.15.1:0/313272967 conn(0x7faab00fd8b0 msgr2=0x7faab00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 shutdown_connections 2024-09-16T00:27:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:06.333+0000 7faab718f640 1 -- 172.21.15.1:0/313272967 wait complete. 2024-09-16T00:27:06.649 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:27:06 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:27:06.311+0000 7f6d874b3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:27:06.649 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:27:06 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:27:06.311+0000 7f6d874b3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:27:06.649 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:27:06 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:27:06.311+0000 7f6d874b3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:27:06.649 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:27:06 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:27:06.311+0000 7f6d874b3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:27:06.649 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:27:06 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:27:06.311+0000 7f6d874b3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:27:07.021 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-16T00:27:07.278 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:07.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:07 smithi001 ceph-mon[30934]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:07.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:07 smithi001 ceph-mon[30934]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-16T00:27:07.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:07 smithi001 ceph-mon[30934]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:27:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.765+0000 7f44661be640 1 -- 172.21.15.1:0/82330047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 msgr2=0x7f446010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.765+0000 7f44661be640 1 --2- 172.21.15.1:0/82330047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 0x7f446010bb50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4454009900 tx=0x7f445402a480 comp rx=0 tx=0).stop 2024-09-16T00:27:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.765+0000 7f44661be640 1 -- 172.21.15.1:0/82330047 shutdown_connections 2024-09-16T00:27:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.765+0000 7f44661be640 1 --2- 172.21.15.1:0/82330047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f446010c170 0x7f446010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.765+0000 7f44661be640 1 --2- 172.21.15.1:0/82330047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 0x7f446010bb50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.765+0000 7f44661be640 1 --2- 172.21.15.1:0/82330047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4460102d40 0x7f4460103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.765+0000 7f44661be640 1 -- 172.21.15.1:0/82330047 >> 172.21.15.1:0/82330047 conn(0x7f44600fc7a0 msgr2=0x7f44600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.765+0000 7f44661be640 1 -- 172.21.15.1:0/82330047 shutdown_connections 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.765+0000 7f44661be640 1 -- 172.21.15.1:0/82330047 wait complete. 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f44661be640 1 Processor -- start 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f44661be640 1 -- start start 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f44661be640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4460102d40 0x7f44601984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f44661be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 0x7f4460198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f44661be640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f446010c170 0x7f446019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f44661be640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4460111f70 con 0x7f4460103660 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f44661be640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4460111df0 con 0x7f446010c170 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f44661be640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44601120f0 con 0x7f4460102d40 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f445effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 0x7f4460198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f445effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 0x7f4460198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48324/0 (socket says 172.21.15.1:48324) 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.766+0000 7f445effd640 1 -- 172.21.15.1:0/287148652 learned_addr learned my addr 172.21.15.1:0/287148652 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445f7fe640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4460102d40 0x7f44601984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445ffff640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f446010c170 0x7f446019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445effd640 1 -- 172.21.15.1:0/287148652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4460102d40 msgr2=0x7f44601984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445effd640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4460102d40 0x7f44601984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445effd640 1 -- 172.21.15.1:0/287148652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f446010c170 msgr2=0x7f446019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445effd640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f446010c170 0x7f446019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445effd640 1 -- 172.21.15.1:0/287148652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446019d480 con 0x7f4460103660 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445ffff640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f446010c170 0x7f446019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445effd640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 0x7f4460198a10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4454006010 tx=0x7f44540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445cff9640 1 -- 172.21.15.1:0/287148652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445403f030 con 0x7f4460103660 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445cff9640 1 -- 172.21.15.1:0/287148652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4454003ce0 con 0x7f4460103660 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446019d710 con 0x7f4460103660 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f445cff9640 1 -- 172.21.15.1:0/287148652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454035070 con 0x7f4460103660 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.767+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44601a5050 con 0x7f4460103660 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.768+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4460103120 con 0x7f4460103660 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.769+0000 7f445cff9640 1 -- 172.21.15.1:0/287148652 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f4454042030 con 0x7f4460103660 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.770+0000 7f445cff9640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f44480a3350 0x7f44480a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.770+0000 7f445f7fe640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f44480a3350 0x7f44480a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.770+0000 7f445cff9640 1 -- 172.21.15.1:0/287148652 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f44540eff10 con 0x7f4460103660 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.771+0000 7f445f7fe640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f44480a3350 0x7f44480a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4450005fd0 tx=0x7f445000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.775+0000 7f445cff9640 1 -- 172.21.15.1:0/287148652 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44540bc790 con 0x7f4460103660 2024-09-16T00:27:07.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:07 smithi153 ceph-mon[35538]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:07.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:07 smithi153 ceph-mon[35538]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-16T00:27:07.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:07 smithi153 ceph-mon[35538]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:27:07.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:07.976+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4460062630 con 0x7f44480a3350 2024-09-16T00:27:08.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.001+0000 7f445cff9640 1 -- 172.21.15.1:0/287148652 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4460062630 con 0x7f44480a3350 2024-09-16T00:27:08.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f44480a3350 msgr2=0x7f44480a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:08.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f44480a3350 0x7f44480a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4450005fd0 tx=0x7f445000b390 comp rx=0 tx=0).stop 2024-09-16T00:27:08.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 msgr2=0x7f4460198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:08.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 0x7f4460198a10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4454006010 tx=0x7f44540042f0 comp rx=0 tx=0).stop 2024-09-16T00:27:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 shutdown_connections 2024-09-16T00:27:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f446010c170 0x7f446019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4460103660 0x7f4460198a10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f44480a3350 0x7f44480a5810 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 --2- 172.21.15.1:0/287148652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4460102d40 0x7f44601984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 >> 172.21.15.1:0/287148652 conn(0x7f44600fc7a0 msgr2=0x7f44600fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 shutdown_connections 2024-09-16T00:27:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:08.003+0000 7f44661be640 1 -- 172.21.15.1:0/287148652 wait complete. 2024-09-16T00:27:08.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:07 smithi019 ceph-mon[35186]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:08.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:07 smithi019 ceph-mon[35186]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-16T00:27:08.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:07 smithi019 ceph-mon[35186]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:27:08.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:08 smithi001 ceph-mon[30934]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:08.740 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-16T00:27:08.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:08 smithi153 ceph-mon[35538]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:08.999 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:09.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:08 smithi019 ceph-mon[35186]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:09.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.492+0000 7f5c18919640 1 -- 172.21.15.1:0/2594578951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14102570 msgr2=0x7f5c141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.492+0000 7f5c18919640 1 --2- 172.21.15.1:0/2594578951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14102570 0x7f5c141029b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009900 tx=0x7f5bfc02a480 comp rx=0 tx=0).stop 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.492+0000 7f5c18919640 1 -- 172.21.15.1:0/2594578951 shutdown_connections 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.492+0000 7f5c18919640 1 --2- 172.21.15.1:0/2594578951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c14102ef0 0x7f5c1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.492+0000 7f5c18919640 1 --2- 172.21.15.1:0/2594578951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14102570 0x7f5c141029b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.492+0000 7f5c18919640 1 --2- 172.21.15.1:0/2594578951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c14100e20 0x7f5c14101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.492+0000 7f5c18919640 1 -- 172.21.15.1:0/2594578951 >> 172.21.15.1:0/2594578951 conn(0x7f5c140fc7d0 msgr2=0x7f5c140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.492+0000 7f5c18919640 1 -- 172.21.15.1:0/2594578951 shutdown_connections 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.492+0000 7f5c18919640 1 -- 172.21.15.1:0/2594578951 wait complete. 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c18919640 1 Processor -- start 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c18919640 1 -- start start 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c18919640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14100e20 0x7f5c14199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c18919640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c14102570 0x7f5c14199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c18919640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c14102ef0 0x7f5c141a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c18919640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c14110f80 con 0x7f5c14100e20 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c18919640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c14110e00 con 0x7f5c14102ef0 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c18919640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5c14111100 con 0x7f5c14102570 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c1259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14100e20 0x7f5c14199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c1259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14100e20 0x7f5c14199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48338/0 (socket says 172.21.15.1:48338) 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.493+0000 7f5c1259c640 1 -- 172.21.15.1:0/3644654270 learned_addr learned my addr 172.21.15.1:0/3644654270 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c12d9d640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c14102ef0 0x7f5c141a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c0bfff640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c14102570 0x7f5c14199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c1259c640 1 -- 172.21.15.1:0/3644654270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c14102570 msgr2=0x7f5c14199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c1259c640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c14102570 0x7f5c14199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c1259c640 1 -- 172.21.15.1:0/3644654270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c14102ef0 msgr2=0x7f5c141a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c1259c640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c14102ef0 0x7f5c141a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:09.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c1259c640 1 -- 172.21.15.1:0/3644654270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c141a48d0 con 0x7f5c14100e20 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c0bfff640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c14102570 0x7f5c14199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c1259c640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14100e20 0x7f5c14199310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400c890 tx=0x7f5c0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/3644654270 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004220 con 0x7f5c14100e20 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/3644654270 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c040043c0 con 0x7f5c14100e20 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c141a4aa0 con 0x7f5c14100e20 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/3644654270 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c040049e0 con 0x7f5c14100e20 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.494+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c141a4fb0 con 0x7f5c14100e20 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.495+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c14101200 con 0x7f5c14100e20 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.496+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/3644654270 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f5c040079b0 con 0x7f5c14100e20 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.497+0000 7f5c0b7fe640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c000a32d0 0x7f5c000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.497+0000 7f5c0bfff640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c000a32d0 0x7f5c000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.497+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/3644654270 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f5c040cff50 con 0x7f5c14100e20 2024-09-16T00:27:09.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.498+0000 7f5c0bfff640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c000a32d0 0x7f5c000a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc000c00 tx=0x7f5bfc004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:09.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.501+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/3644654270 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c04014030 con 0x7f5c14100e20 2024-09-16T00:27:09.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:09 smithi001 ceph-mon[30934]: pgmap v311: 129 pgs: 129 active+clean; 590 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-16T00:27:09.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.707+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f5c14106f70 con 0x7f5c000a32d0 2024-09-16T00:27:09.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.711+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/3644654270 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5c14106f70 con 0x7f5c000a32d0 2024-09-16T00:27:09.712 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_0/990c69e3-774b-4c69-904c-27d7d72a647c 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c000a32d0 msgr2=0x7f5c000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c000a32d0 0x7f5c000a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc000c00 tx=0x7f5bfc004100 comp rx=0 tx=0).stop 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14100e20 msgr2=0x7f5c14199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14100e20 0x7f5c14199310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400c890 tx=0x7f5c0400cd50 comp rx=0 tx=0).stop 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 shutdown_connections 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c14102ef0 0x7f5c141a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c000a32d0 0x7f5c000a5790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c14102570 0x7f5c14199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 --2- 172.21.15.1:0/3644654270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c14100e20 0x7f5c14199310 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.713+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 >> 172.21.15.1:0/3644654270 conn(0x7f5c140fc7d0 msgr2=0x7f5c140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.714+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 shutdown_connections 2024-09-16T00:27:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:09.714+0000 7f5c18919640 1 -- 172.21.15.1:0/3644654270 wait complete. 2024-09-16T00:27:09.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:09 smithi153 ceph-mon[35538]: pgmap v311: 129 pgs: 129 active+clean; 590 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-16T00:27:10.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:09 smithi019 ceph-mon[35186]: pgmap v311: 129 pgs: 129 active+clean; 590 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-16T00:27:10.430 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-16T00:27:10.687 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:10.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:10 smithi001 ceph-mon[30934]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:10.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:10 smithi153 ceph-mon[35538]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:11.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:10 smithi019 ceph-mon[35186]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 -- 172.21.15.1:0/3663517741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 msgr2=0x7fe7f0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 --2- 172.21.15.1:0/3663517741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 0x7fe7f0100ee0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009930 tx=0x7fe7d802a420 comp rx=0 tx=0).stop 2024-09-16T00:27:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 -- 172.21.15.1:0/3663517741 shutdown_connections 2024-09-16T00:27:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 --2- 172.21.15.1:0/3663517741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0101420 0x7fe7f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 --2- 172.21.15.1:0/3663517741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 0x7fe7f0100ee0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 --2- 172.21.15.1:0/3663517741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f0106a80 0x7fe7f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 -- 172.21.15.1:0/3663517741 >> 172.21.15.1:0/3663517741 conn(0x7fe7f00fc7a0 msgr2=0x7fe7f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 -- 172.21.15.1:0/3663517741 shutdown_connections 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 -- 172.21.15.1:0/3663517741 wait complete. 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 Processor -- start 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.236+0000 7fe7f647f640 1 -- start start 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f647f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 0x7fe7f01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f647f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f0101420 0x7fe7f01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f647f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0106a80 0x7fe7f019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f647f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0111fa0 con 0x7fe7f0100a80 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f647f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0111e20 con 0x7fe7f0101420 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f647f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0112120 con 0x7fe7f0106a80 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7effff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 0x7fe7f01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7effff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 0x7fe7f01984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35562/0 (socket says 172.21.15.1:35562) 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7effff640 1 -- 172.21.15.1:0/2074125901 learned_addr learned my addr 172.21.15.1:0/2074125901 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7e7fff640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f0101420 0x7fe7f01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f4a1c640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0106a80 0x7fe7f019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7effff640 1 -- 172.21.15.1:0/2074125901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0106a80 msgr2=0x7fe7f019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7effff640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0106a80 0x7fe7f019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7effff640 1 -- 172.21.15.1:0/2074125901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f0101420 msgr2=0x7fe7f01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7effff640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f0101420 0x7fe7f01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7effff640 1 -- 172.21.15.1:0/2074125901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f019d450 con 0x7fe7f0100a80 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f4a1c640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0106a80 0x7fe7f019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7effff640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 0x7fe7f01984a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c890 tx=0x7fe7e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7edffb640 1 -- 172.21.15.1:0/2074125901 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0004220 con 0x7fe7f0100a80 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f019d740 con 0x7fe7f0100a80 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7edffb640 1 -- 172.21.15.1:0/2074125901 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e00043c0 con 0x7fe7f0100a80 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7edffb640 1 -- 172.21.15.1:0/2074125901 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0004a00 con 0x7fe7f0100a80 2024-09-16T00:27:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.237+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f01a5020 con 0x7fe7f0100a80 2024-09-16T00:27:11.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.239+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f0100ee0 con 0x7fe7f0100a80 2024-09-16T00:27:11.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.240+0000 7fe7edffb640 1 -- 172.21.15.1:0/2074125901 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe7e00079b0 con 0x7fe7f0100a80 2024-09-16T00:27:11.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.241+0000 7fe7edffb640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7dc0a32d0 0x7fe7dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:11.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.241+0000 7fe7e7fff640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7dc0a32d0 0x7fe7dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:11.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.241+0000 7fe7edffb640 1 -- 172.21.15.1:0/2074125901 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fe7e00cfae0 con 0x7fe7f0100a80 2024-09-16T00:27:11.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.241+0000 7fe7e7fff640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7dc0a32d0 0x7fe7dc0a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009900 tx=0x7fe7d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:11.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.245+0000 7fe7edffb640 1 -- 172.21.15.1:0/2074125901 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7e0014030 con 0x7fe7f0100a80 2024-09-16T00:27:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.451+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe7f0062630 con 0x7fe7dc0a32d0 2024-09-16T00:27:11.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.476+0000 7fe7edffb640 1 -- 172.21.15.1:0/2074125901 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7f0062630 con 0x7fe7dc0a32d0 2024-09-16T00:27:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7dc0a32d0 msgr2=0x7fe7dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7dc0a32d0 0x7fe7dc0a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009900 tx=0x7fe7d80058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 msgr2=0x7fe7f01984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 0x7fe7f01984a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c890 tx=0x7fe7e000cd50 comp rx=0 tx=0).stop 2024-09-16T00:27:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 shutdown_connections 2024-09-16T00:27:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7dc0a32d0 0x7fe7dc0a5790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0106a80 0x7fe7f019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f0101420 0x7fe7f01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 --2- 172.21.15.1:0/2074125901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7f0100a80 0x7fe7f01984a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:11.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 >> 172.21.15.1:0/2074125901 conn(0x7fe7f00fc7a0 msgr2=0x7fe7f00fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:11.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 shutdown_connections 2024-09-16T00:27:11.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:11.479+0000 7fe7f647f640 1 -- 172.21.15.1:0/2074125901 wait complete. 2024-09-16T00:27:11.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:11 smithi001 ceph-mon[30934]: pgmap v312: 129 pgs: 129 active+clean; 591 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-16T00:27:11.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:11 smithi153 ceph-mon[35538]: pgmap v312: 129 pgs: 129 active+clean; 591 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-16T00:27:12.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:11 smithi019 ceph-mon[35186]: pgmap v312: 129 pgs: 129 active+clean; 591 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-16T00:27:12.175 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-16T00:27:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:12.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:12 smithi001 ceph-mon[30934]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:12.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:12 smithi153 ceph-mon[35538]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:12.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 -- 172.21.15.1:0/3048951090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 msgr2=0x7f7be810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:12.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 --2- 172.21.15.1:0/3048951090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 0x7f7be810e610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009ca0 tx=0x7f7be002a950 comp rx=0 tx=0).stop 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 -- 172.21.15.1:0/3048951090 shutdown_connections 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 --2- 172.21.15.1:0/3048951090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 0x7f7be810e610 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 --2- 172.21.15.1:0/3048951090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7be8103660 0x7f7be810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 --2- 172.21.15.1:0/3048951090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7be8102d40 0x7f7be8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 -- 172.21.15.1:0/3048951090 >> 172.21.15.1:0/3048951090 conn(0x7f7be80fc7a0 msgr2=0x7f7be80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 -- 172.21.15.1:0/3048951090 shutdown_connections 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 -- 172.21.15.1:0/3048951090 wait complete. 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 Processor -- start 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.949+0000 7f7bf010c640 1 -- start start 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bf010c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7be8102d40 0x7f7be8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:12.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bf010c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7be8103660 0x7f7be8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bf010c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 0x7f7be819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bf010c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7be8111f50 con 0x7f7be810c170 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bf010c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7be8111dd0 con 0x7f7be8102d40 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bf010c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7be81120d0 con 0x7f7be8103660 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bee6a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 0x7f7be819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bee6a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 0x7f7be819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35592/0 (socket says 172.21.15.1:35592) 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bedea8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7be8102d40 0x7f7be8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bee6a9640 1 -- 172.21.15.1:0/1062470677 learned_addr learned my addr 172.21.15.1:0/1062470677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bed6a7640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7be8103660 0x7f7be8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bee6a9640 1 -- 172.21.15.1:0/1062470677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7be8103660 msgr2=0x7f7be8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bee6a9640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7be8103660 0x7f7be8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bee6a9640 1 -- 172.21.15.1:0/1062470677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7be8102d40 msgr2=0x7f7be8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:12.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bee6a9640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7be8102d40 0x7f7be8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bee6a9640 1 -- 172.21.15.1:0/1062470677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be819d3c0 con 0x7f7be810c170 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.950+0000 7f7bee6a9640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 0x7f7be819cce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004af0 tx=0x7f7be002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.951+0000 7f7bdeffd640 1 -- 172.21.15.1:0/1062470677 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0034040 con 0x7f7be810c170 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.951+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be819d650 con 0x7f7be810c170 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.951+0000 7f7bdeffd640 1 -- 172.21.15.1:0/1062470677 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be0030920 con 0x7f7be810c170 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.951+0000 7f7bdeffd640 1 -- 172.21.15.1:0/1062470677 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0004670 con 0x7f7be810c170 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.951+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be81a4f90 con 0x7f7be810c170 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.952+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be806b070 con 0x7f7be810c170 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.953+0000 7f7bdeffd640 1 -- 172.21.15.1:0/1062470677 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f7be0037020 con 0x7f7be810c170 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.954+0000 7f7bdeffd640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7bd40a32d0 0x7f7bd40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.954+0000 7f7bedea8640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7bd40a32d0 0x7f7bd40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.954+0000 7f7bdeffd640 1 -- 172.21.15.1:0/1062470677 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f7be00f0970 con 0x7f7be810c170 2024-09-16T00:27:12.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.954+0000 7f7bedea8640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7bd40a32d0 0x7f7bd40a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009bd0 tx=0x7f7be4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:12.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:12.958+0000 7f7bdeffd640 1 -- 172.21.15.1:0/1062470677 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7be00bd300 con 0x7f7be810c170 2024-09-16T00:27:13.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:12 smithi019 ceph-mon[35186]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:13.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.174+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f7be8107250 con 0x7f7bd40a32d0 2024-09-16T00:27:13.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.179+0000 7f7bdeffd640 1 -- 172.21.15.1:0/1062470677 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7be401b3f0 con 0x7f7bd40a32d0 2024-09-16T00:27:13.179 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_1/4c1f3cd4-b1c5-443e-970a-b5a707971cba 2024-09-16T00:27:13.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.180+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7bd40a32d0 msgr2=0x7f7bd40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:13.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7bd40a32d0 0x7f7bd40a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009bd0 tx=0x7f7be4005f90 comp rx=0 tx=0).stop 2024-09-16T00:27:13.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 msgr2=0x7f7be819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:13.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 0x7f7be819cce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004af0 tx=0x7f7be002f3a0 comp rx=0 tx=0).stop 2024-09-16T00:27:13.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 shutdown_connections 2024-09-16T00:27:13.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7be810c170 0x7f7be819cce0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:13.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7bd40a32d0 0x7f7bd40a5790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:13.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7be8103660 0x7f7be8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:13.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 --2- 172.21.15.1:0/1062470677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7be8102d40 0x7f7be8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:13.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 >> 172.21.15.1:0/1062470677 conn(0x7f7be80fc7a0 msgr2=0x7f7be80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:13.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 shutdown_connections 2024-09-16T00:27:13.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:13.181+0000 7f7bf010c640 1 -- 172.21.15.1:0/1062470677 wait complete. 2024-09-16T00:27:13.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:13 smithi001 ceph-mon[30934]: pgmap v313: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-16T00:27:13.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:13 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:13.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:13 smithi001 ceph-mon[30934]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:13.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:13 smithi153 ceph-mon[35538]: pgmap v313: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-16T00:27:13.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:13 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:13.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:13 smithi153 ceph-mon[35538]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:14.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:13 smithi019 ceph-mon[35186]: pgmap v313: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-16T00:27:14.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:13 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:14.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:13 smithi019 ceph-mon[35186]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:14.491 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:27:14.767 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:15.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:15 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.247+0000 7fb620aec640 1 -- 172.21.15.1:0/1393844192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c102d40 msgr2=0x7fb61c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.247+0000 7fb620aec640 1 --2- 172.21.15.1:0/1393844192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c102d40 0x7fb61c103120 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb60c009900 tx=0x7fb60c02a480 comp rx=0 tx=0).stop 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.247+0000 7fb620aec640 1 -- 172.21.15.1:0/1393844192 shutdown_connections 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.247+0000 7fb620aec640 1 --2- 172.21.15.1:0/1393844192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61c10c170 0x7fb61c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.247+0000 7fb620aec640 1 --2- 172.21.15.1:0/1393844192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb61c103660 0x7fb61c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.247+0000 7fb620aec640 1 --2- 172.21.15.1:0/1393844192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c102d40 0x7fb61c103120 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.247+0000 7fb620aec640 1 -- 172.21.15.1:0/1393844192 >> 172.21.15.1:0/1393844192 conn(0x7fb61c0fc7a0 msgr2=0x7fb61c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.247+0000 7fb620aec640 1 -- 172.21.15.1:0/1393844192 shutdown_connections 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.247+0000 7fb620aec640 1 -- 172.21.15.1:0/1393844192 wait complete. 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb620aec640 1 Processor -- start 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb620aec640 1 -- start start 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb620aec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61c102d40 0x7fb61c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb620aec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb61c103660 0x7fb61c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb620aec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c10c170 0x7fb61c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb620aec640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c111f70 con 0x7fb61c10c170 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb620aec640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c111df0 con 0x7fb61c103660 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb620aec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1120f0 con 0x7fb61c102d40 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb61ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c10c170 0x7fb61c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb61ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c10c170 0x7fb61c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35612/0 (socket says 172.21.15.1:35612) 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb619d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb61c103660 0x7fb61c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.248+0000 7fb61ad9d640 1 -- 172.21.15.1:0/2212403018 learned_addr learned my addr 172.21.15.1:0/2212403018 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb61a59c640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61c102d40 0x7fb61c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb61ad9d640 1 -- 172.21.15.1:0/2212403018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61c102d40 msgr2=0x7fb61c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb61ad9d640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61c102d40 0x7fb61c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb61ad9d640 1 -- 172.21.15.1:0/2212403018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb61c103660 msgr2=0x7fb61c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:15.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb61ad9d640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb61c103660 0x7fb61c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb61ad9d640 1 -- 172.21.15.1:0/2212403018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c19d400 con 0x7fb61c10c170 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb619d9b640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb61c103660 0x7fb61c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb61a59c640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61c102d40 0x7fb61c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb61ad9d640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c10c170 0x7fb61c19cd20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb61400d750 tx=0x7fb61400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb60b7fe640 1 -- 172.21.15.1:0/2212403018 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb614017070 con 0x7fb61c10c170 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb60b7fe640 1 -- 172.21.15.1:0/2212403018 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb614005500 con 0x7fb61c10c170 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61c19d6f0 con 0x7fb61c10c170 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb60b7fe640 1 -- 172.21.15.1:0/2212403018 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb614004d90 con 0x7fb61c10c170 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.249+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61c1a4fd0 con 0x7fb61c10c170 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.251+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb61c103120 con 0x7fb61c10c170 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.251+0000 7fb60b7fe640 1 -- 172.21.15.1:0/2212403018 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fb614004f30 con 0x7fb61c10c170 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.252+0000 7fb60b7fe640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb6040a32d0 0x7fb6040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.252+0000 7fb61a59c640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb6040a32d0 0x7fb6040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.252+0000 7fb60b7fe640 1 -- 172.21.15.1:0/2212403018 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fb6140cf730 con 0x7fb61c10c170 2024-09-16T00:27:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.253+0000 7fb61a59c640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb6040a32d0 0x7fb6040a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb60c005ab0 tx=0x7fb60c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.256+0000 7fb60b7fe640 1 -- 172.21.15.1:0/2212403018 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb61409bfb0 con 0x7fb61c10c170 2024-09-16T00:27:15.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:15 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.492+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb61c107210 con 0x7fb61c10c170 2024-09-16T00:27:15.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.492+0000 7fb60b7fe640 1 -- 172.21.15.1:0/2212403018 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11894 (secure 0 0 0) 0x7fb61409b8b0 con 0x7fb61c10c170 2024-09-16T00:27:15.493 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:27:15.493 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"btime":"2024-09-16T00:27:04:602627+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:27:04.334709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24470,24469]},"id":1}]} 2024-09-16T00:27:15.493 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 24 2024-09-16T00:27:15.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb6040a32d0 msgr2=0x7fb6040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb6040a32d0 0x7fb6040a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb60c005ab0 tx=0x7fb60c005a20 comp rx=0 tx=0).stop 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c10c170 msgr2=0x7fb61c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c10c170 0x7fb61c19cd20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb61400d750 tx=0x7fb61400dc10 comp rx=0 tx=0).stop 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 shutdown_connections 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb61c10c170 0x7fb61c19cd20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb61c103660 0x7fb61c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb6040a32d0 0x7fb6040a5790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 --2- 172.21.15.1:0/2212403018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61c102d40 0x7fb61c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 >> 172.21.15.1:0/2212403018 conn(0x7fb61c0fc7a0 msgr2=0x7fb61c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 shutdown_connections 2024-09-16T00:27:15.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:15.495+0000 7fb620aec640 1 -- 172.21.15.1:0/2212403018 wait complete. 2024-09-16T00:27:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:15 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.163 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:16 smithi001 ceph-mon[30934]: pgmap v314: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-09-16T00:27:16.163 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.163 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.163 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:16 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2212403018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:16.163 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:16.163 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:16.163 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.191 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:27:16.449 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:16.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:16 smithi153 ceph-mon[35538]: pgmap v314: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-09-16T00:27:16.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:16 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2212403018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:16.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:16 smithi019 ceph-mon[35186]: pgmap v314: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-09-16T00:27:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:16 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2212403018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:27:16.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.973+0000 7ff3ed994640 1 -- 172.21.15.1:0/4165310855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8100a90 msgr2=0x7ff3e8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:16.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.973+0000 7ff3ed994640 1 --2- 172.21.15.1:0/4165310855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8100a90 0x7ff3e8100ef0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009930 tx=0x7ff3d002a420 comp rx=0 tx=0).stop 2024-09-16T00:27:16.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.973+0000 7ff3ed994640 1 -- 172.21.15.1:0/4165310855 shutdown_connections 2024-09-16T00:27:16.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.973+0000 7ff3ed994640 1 --2- 172.21.15.1:0/4165310855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3e8101430 0x7ff3e810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.973+0000 7ff3ed994640 1 --2- 172.21.15.1:0/4165310855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8100a90 0x7ff3e8100ef0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.973+0000 7ff3ed994640 1 --2- 172.21.15.1:0/4165310855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8106a90 0x7ff3e8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.973+0000 7ff3ed994640 1 -- 172.21.15.1:0/4165310855 >> 172.21.15.1:0/4165310855 conn(0x7ff3e80fc7d0 msgr2=0x7ff3e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.973+0000 7ff3ed994640 1 -- 172.21.15.1:0/4165310855 shutdown_connections 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.973+0000 7ff3ed994640 1 -- 172.21.15.1:0/4165310855 wait complete. 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3ed994640 1 Processor -- start 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3ed994640 1 -- start start 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3ed994640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8100a90 0x7ff3e819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3ed994640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3e8101430 0x7ff3e819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3ed994640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8106a90 0x7ff3e819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3ed994640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8110eb0 con 0x7ff3e8106a90 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3ed994640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8110d30 con 0x7ff3e8101430 2024-09-16T00:27:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3ed994640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8111030 con 0x7ff3e8100a90 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8106a90 0x7ff3e819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8106a90 0x7ff3e819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35626/0 (socket says 172.21.15.1:35626) 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e77fe640 1 -- 172.21.15.1:0/2793155745 learned_addr learned my addr 172.21.15.1:0/2793155745 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e6ffd640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8100a90 0x7ff3e819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3dffff640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3e8101430 0x7ff3e819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e77fe640 1 -- 172.21.15.1:0/2793155745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8100a90 msgr2=0x7ff3e819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e77fe640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8100a90 0x7ff3e819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e77fe640 1 -- 172.21.15.1:0/2793155745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3e8101430 msgr2=0x7ff3e819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e77fe640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3e8101430 0x7ff3e819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e77fe640 1 -- 172.21.15.1:0/2793155745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e819f6d0 con 0x7ff3e8106a90 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e6ffd640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8100a90 0x7ff3e819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.974+0000 7ff3e77fe640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8106a90 0x7ff3e819eff0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000cad0 tx=0x7ff3e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.975+0000 7ff3ec992640 1 -- 172.21.15.1:0/2793155745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000ae20 con 0x7ff3e8106a90 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.975+0000 7ff3ec992640 1 -- 172.21.15.1:0/2793155745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e0005020 con 0x7ff3e8106a90 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.975+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e819f9c0 con 0x7ff3e8106a90 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.975+0000 7ff3ec992640 1 -- 172.21.15.1:0/2793155745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000f6f0 con 0x7ff3e8106a90 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.975+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e81a72a0 con 0x7ff3e8106a90 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.976+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e8100ef0 con 0x7ff3e8106a90 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.977+0000 7ff3ec992640 1 -- 172.21.15.1:0/2793155745 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ff3e00051d0 con 0x7ff3e8106a90 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.978+0000 7ff3ec992640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3d40a3350 0x7ff3d40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.978+0000 7ff3e6ffd640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3d40a3350 0x7ff3d40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.978+0000 7ff3ec992640 1 -- 172.21.15.1:0/2793155745 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7ff3e00cfea0 con 0x7ff3e8106a90 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.979+0000 7ff3e6ffd640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3d40a3350 0x7ff3d40a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006fd0 tx=0x7ff3d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:16.982+0000 7ff3ec992640 1 -- 172.21.15.1:0/2793155745 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3e0014030 con 0x7ff3e8106a90 2024-09-16T00:27:17.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.159+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3e81a74d0 con 0x7ff3e8106a90 2024-09-16T00:27:17.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.160+0000 7ff3ec992640 1 -- 172.21.15.1:0/2793155745 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24601 (secure 0 0 0) 0x7ff3e009c720 con 0x7ff3e8106a90 2024-09-16T00:27:17.162 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:27:17.162 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":89,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:26:28.990229+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:27:17.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3d40a3350 msgr2=0x7ff3d40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:17.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3d40a3350 0x7ff3d40a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006fd0 tx=0x7ff3d8008040 comp rx=0 tx=0).stop 2024-09-16T00:27:17.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8106a90 msgr2=0x7ff3e819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:17.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8106a90 0x7ff3e819eff0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000cad0 tx=0x7ff3e000cf90 comp rx=0 tx=0).stop 2024-09-16T00:27:17.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 shutdown_connections 2024-09-16T00:27:17.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3e8106a90 0x7ff3e819eff0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:17.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3e8101430 0x7ff3e819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:17.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3d40a3350 0x7ff3d40a5810 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:17.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 --2- 172.21.15.1:0/2793155745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8100a90 0x7ff3e819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:17.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 >> 172.21.15.1:0/2793155745 conn(0x7ff3e80fc7d0 msgr2=0x7ff3e806a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:17.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 shutdown_connections 2024-09-16T00:27:17.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:17.163+0000 7ff3ed994640 1 -- 172.21.15.1:0/2793155745 wait complete. 2024-09-16T00:27:17.843 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:27:17.843 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:27:18.107 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:18.389 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:18 smithi001 ceph-mon[30934]: pgmap v315: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-16T00:27:18.389 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:18 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2793155745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:27:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:18 smithi153 ceph-mon[35538]: pgmap v315: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-16T00:27:18.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:18 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2793155745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:27:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:18 smithi019 ceph-mon[35186]: pgmap v315: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-16T00:27:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:18 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2793155745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:27:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.635+0000 7f8403c99640 1 -- 172.21.15.1:0/2261744836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 msgr2=0x7f83fc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.635+0000 7f8403c99640 1 --2- 172.21.15.1:0/2261744836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 0x7f83fc106e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009930 tx=0x7f83f802a420 comp rx=0 tx=0).stop 2024-09-16T00:27:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.635+0000 7f8403c99640 1 -- 172.21.15.1:0/2261744836 shutdown_connections 2024-09-16T00:27:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.635+0000 7f8403c99640 1 --2- 172.21.15.1:0/2261744836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83fc101420 0x7f83fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.635+0000 7f8403c99640 1 --2- 172.21.15.1:0/2261744836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83fc100a80 0x7f83fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.635+0000 7f8403c99640 1 --2- 172.21.15.1:0/2261744836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 0x7f83fc106e60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.635+0000 7f8403c99640 1 -- 172.21.15.1:0/2261744836 >> 172.21.15.1:0/2261744836 conn(0x7f83fc0fc7a0 msgr2=0x7f83fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.635+0000 7f8403c99640 1 -- 172.21.15.1:0/2261744836 shutdown_connections 2024-09-16T00:27:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.635+0000 7f8403c99640 1 -- 172.21.15.1:0/2261744836 wait complete. 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8403c99640 1 Processor -- start 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8403c99640 1 -- start start 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8403c99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83fc100a80 0x7f83fc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8403c99640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83fc101420 0x7f83fc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8403c99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 0x7f83fc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8403c99640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc110e40 con 0x7f83fc106a80 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8403c99640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc110cc0 con 0x7f83fc100a80 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8403c99640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc110fc0 con 0x7f83fc101420 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8402236640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 0x7f83fc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8402236640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 0x7f83fc19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35632/0 (socket says 172.21.15.1:35632) 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8402236640 1 -- 172.21.15.1:0/3630593058 learned_addr learned my addr 172.21.15.1:0/3630593058 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8401a35640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83fc100a80 0x7f83fc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8401234640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83fc101420 0x7f83fc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8402236640 1 -- 172.21.15.1:0/3630593058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83fc101420 msgr2=0x7f83fc198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8402236640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83fc101420 0x7f83fc198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8402236640 1 -- 172.21.15.1:0/3630593058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83fc100a80 msgr2=0x7f83fc1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8402236640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83fc100a80 0x7f83fc1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.636+0000 7f8402236640 1 -- 172.21.15.1:0/3630593058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc19d370 con 0x7f83fc106a80 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.637+0000 7f8402236640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 0x7f83fc19cc90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83f400eaa0 tx=0x7f83f400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.637+0000 7f83f2ffd640 1 -- 172.21.15.1:0/3630593058 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4004d20 con 0x7f83fc106a80 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.637+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc19d660 con 0x7f83fc106a80 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.637+0000 7f83f2ffd640 1 -- 172.21.15.1:0/3630593058 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f4005020 con 0x7f83fc106a80 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.637+0000 7f83f2ffd640 1 -- 172.21.15.1:0/3630593058 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f40116d0 con 0x7f83fc106a80 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.637+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc1a4f40 con 0x7f83fc106a80 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.638+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc100ee0 con 0x7f83fc106a80 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.639+0000 7f83f2ffd640 1 -- 172.21.15.1:0/3630593058 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f83f40051d0 con 0x7f83fc106a80 2024-09-16T00:27:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.640+0000 7f83f2ffd640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83e80a32d0 0x7f83e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.640+0000 7f8401a35640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83e80a32d0 0x7f83e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.640+0000 7f83f2ffd640 1 -- 172.21.15.1:0/3630593058 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f83f40cfef0 con 0x7f83fc106a80 2024-09-16T00:27:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.640+0000 7f8401a35640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83e80a32d0 0x7f83e80a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009900 tx=0x7f83f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.644+0000 7f83f2ffd640 1 -- 172.21.15.1:0/3630593058 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83f4016030 con 0x7f83fc106a80 2024-09-16T00:27:18.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.818+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83fc062630 con 0x7f83e80a32d0 2024-09-16T00:27:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.827+0000 7f83f2ffd640 1 -- 172.21.15.1:0/3630593058 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340357 (secure 0 0 0) 0x7f83fc062630 con 0x7f83e80a32d0 2024-09-16T00:27:18.833 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:27:18.834 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83e80a32d0 msgr2=0x7f83e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83e80a32d0 0x7f83e80a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009900 tx=0x7f83f80058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 msgr2=0x7f83fc19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 0x7f83fc19cc90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83f400eaa0 tx=0x7f83f400eea0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 shutdown_connections 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc106a80 0x7f83fc19cc90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83e80a32d0 0x7f83e80a5790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83fc101420 0x7f83fc198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 --2- 172.21.15.1:0/3630593058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83fc100a80 0x7f83fc1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 >> 172.21.15.1:0/3630593058 conn(0x7f83fc0fc7a0 msgr2=0x7f83fc06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 shutdown_connections 2024-09-16T00:27:18.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:18.830+0000 7f8403c99640 1 -- 172.21.15.1:0/3630593058 wait complete. 2024-09-16T00:27:19.553 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-16T00:27:18.322486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643703,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":236,"num_read_kb":223,"num_write":337,"num_write_kb":3039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375536,"kb_used_data":13808,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566288,"statfs":{"total":1151940427776,"available":1151555878912,"internally_reserved":0,"allocated":14139392,"data_stored":7857179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45881,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":92,"num_write":27,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003166"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.610128+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:27:01.610128+0000","last_peered":"2024-09-16T00:27:01.610128+0000","last_clean":"2024-09-16T00:27:01.610128+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:27:01.610128+0000","last_undegraded":"2024-09-16T00:27:01.610128+0000","last_fullsized":"2024-09-16T00:27:01.610128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969951+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:26:29.969951+0000","last_peered":"2024-09-16T00:26:29.969951+0000","last_clean":"2024-09-16T00:26:29.969951+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:26:29.969951+0000","last_undegraded":"2024-09-16T00:26:29.969951+0000","last_fullsized":"2024-09-16T00:26:29.969951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290691+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:26:29.290691+0000","last_peered":"2024-09-16T00:26:29.290691+0000","last_clean":"2024-09-16T00:26:29.290691+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:26:29.290691+0000","last_undegraded":"2024-09-16T00:26:29.290691+0000","last_fullsized":"2024-09-16T00:26:29.290691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970131+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:26:29.970131+0000","last_peered":"2024-09-16T00:26:29.970131+0000","last_clean":"2024-09-16T00:26:29.970131+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:26:29.970131+0000","last_undegraded":"2024-09-16T00:26:29.970131+0000","last_fullsized":"2024-09-16T00:26:29.970131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552533+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:26:29.552533+0000","last_peered":"2024-09-16T00:26:29.552533+0000","last_clean":"2024-09-16T00:26:29.552533+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:26:29.552533+0000","last_undegraded":"2024-09-16T00:26:29.552533+0000","last_fullsized":"2024-09-16T00:26:29.552533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970105+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:26:29.970105+0000","last_peered":"2024-09-16T00:26:29.970105+0000","last_clean":"2024-09-16T00:26:29.970105+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:26:29.970105+0000","last_undegraded":"2024-09-16T00:26:29.970105+0000","last_fullsized":"2024-09-16T00:26:29.970105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.608620+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:01.608620+0000","last_peered":"2024-09-16T00:27:01.608620+0000","last_clean":"2024-09-16T00:27:01.608620+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:01.608620+0000","last_undegraded":"2024-09-16T00:27:01.608620+0000","last_fullsized":"2024-09-16T00:27:01.608620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971638+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:26:29.971638+0000","last_peered":"2024-09-16T00:26:29.971638+0000","last_clean":"2024-09-16T00:26:29.971638+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:26:29.971638+0000","last_undegraded":"2024-09-16T00:26:29.971638+0000","last_fullsized":"2024-09-16T00:26:29.971638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971028+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:26:29.971028+0000","last_peered":"2024-09-16T00:26:29.971028+0000","last_clean":"2024-09-16T00:26:29.971028+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:26:29.971028+0000","last_undegraded":"2024-09-16T00:26:29.971028+0000","last_fullsized":"2024-09-16T00:26:29.971028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970207+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:26:29.970207+0000","last_peered":"2024-09-16T00:26:29.970207+0000","last_clean":"2024-09-16T00:26:29.970207+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:26:29.970207+0000","last_undegraded":"2024-09-16T00:26:29.970207+0000","last_fullsized":"2024-09-16T00:26:29.970207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970678+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:26:29.970678+0000","last_peered":"2024-09-16T00:26:29.970678+0000","last_clean":"2024-09-16T00:26:29.970678+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:26:29.970678+0000","last_undegraded":"2024-09-16T00:26:29.970678+0000","last_fullsized":"2024-09-16T00:26:29.970678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970756+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:26:29.970756+0000","last_peered":"2024-09-16T00:26:29.970756+0000","last_clean":"2024-09-16T00:26:29.970756+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:26:29.970756+0000","last_undegraded":"2024-09-16T00:26:29.970756+0000","last_fullsized":"2024-09-16T00:26:29.970756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970362+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:26:29.970362+0000","last_peered":"2024-09-16T00:26:29.970362+0000","last_clean":"2024-09-16T00:26:29.970362+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:26:29.970362+0000","last_undegraded":"2024-09-16T00:26:29.970362+0000","last_fullsized":"2024-09-16T00:26:29.970362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.604370+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:00.604370+0000","last_peered":"2024-09-16T00:27:00.604370+0000","last_clean":"2024-09-16T00:27:00.604370+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:00.604370+0000","last_undegraded":"2024-09-16T00:27:00.604370+0000","last_fullsized":"2024-09-16T00:27:00.604370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971345+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:26:29.971345+0000","last_peered":"2024-09-16T00:26:29.971345+0000","last_clean":"2024-09-16T00:26:29.971345+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:26:29.971345+0000","last_undegraded":"2024-09-16T00:26:29.971345+0000","last_fullsized":"2024-09-16T00:26:29.971345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969805+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:26:29.969805+0000","last_peered":"2024-09-16T00:26:29.969805+0000","last_clean":"2024-09-16T00:26:29.969805+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:26:29.969805+0000","last_undegraded":"2024-09-16T00:26:29.969805+0000","last_fullsized":"2024-09-16T00:26:29.969805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290960+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:26:29.290960+0000","last_peered":"2024-09-16T00:26:29.290960+0000","last_clean":"2024-09-16T00:26:29.290960+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:26:29.290960+0000","last_undegraded":"2024-09-16T00:26:29.290960+0000","last_fullsized":"2024-09-16T00:26:29.290960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971635+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:26:29.971635+0000","last_peered":"2024-09-16T00:26:29.971635+0000","last_clean":"2024-09-16T00:26:29.971635+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:26:29.971635+0000","last_undegraded":"2024-09-16T00:26:29.971635+0000","last_fullsized":"2024-09-16T00:26:29.971635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970212+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:26:29.970212+0000","last_peered":"2024-09-16T00:26:29.970212+0000","last_clean":"2024-09-16T00:26:29.970212+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:26:29.970212+0000","last_undegraded":"2024-09-16T00:26:29.970212+0000","last_fullsized":"2024-09-16T00:26:29.970212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.605861+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:00.605861+0000","last_peered":"2024-09-16T00:27:00.605861+0000","last_clean":"2024-09-16T00:27:00.605861+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:00.605861+0000","last_undegraded":"2024-09-16T00:27:00.605861+0000","last_fullsized":"2024-09-16T00:27:00.605861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970738+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:26:29.970738+0000","last_peered":"2024-09-16T00:26:29.970738+0000","last_clean":"2024-09-16T00:26:29.970738+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:26:29.970738+0000","last_undegraded":"2024-09-16T00:26:29.970738+0000","last_fullsized":"2024-09-16T00:26:29.970738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970609+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:26:29.970609+0000","last_peered":"2024-09-16T00:26:29.970609+0000","last_clean":"2024-09-16T00:26:29.970609+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:26:29.970609+0000","last_undegraded":"2024-09-16T00:26:29.970609+0000","last_fullsized":"2024-09-16T00:26:29.970609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.604570+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:00.604570+0000","last_peered":"2024-09-16T00:27:00.604570+0000","last_clean":"2024-09-16T00:27:00.604570+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:00.604570+0000","last_undegraded":"2024-09-16T00:27:00.604570+0000","last_fullsized":"2024-09-16T00:27:00.604570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971399+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:26:29.971399+0000","last_peered":"2024-09-16T00:26:29.971399+0000","last_clean":"2024-09-16T00:26:29.971399+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:26:29.971399+0000","last_undegraded":"2024-09-16T00:26:29.971399+0000","last_fullsized":"2024-09-16T00:26:29.971399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552385+0000","last_change":"2024-09-16T00:26:18.267540+0000","last_active":"2024-09-16T00:26:29.552385+0000","last_peered":"2024-09-16T00:26:29.552385+0000","last_clean":"2024-09-16T00:26:29.552385+0000","last_became_active":"2024-09-16T00:26:18.267230+0000","last_became_peered":"2024-09-16T00:26:18.267230+0000","last_unstale":"2024-09-16T00:26:29.552385+0000","last_undegraded":"2024-09-16T00:26:29.552385+0000","last_fullsized":"2024-09-16T00:26:29.552385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:03.127446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.603092+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:00.603092+0000","last_peered":"2024-09-16T00:27:00.603092+0000","last_clean":"2024-09-16T00:27:00.603092+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:00.603092+0000","last_undegraded":"2024-09-16T00:27:00.603092+0000","last_fullsized":"2024-09-16T00:27:00.603092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290772+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:26:29.290772+0000","last_peered":"2024-09-16T00:26:29.290772+0000","last_clean":"2024-09-16T00:26:29.290772+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:26:29.290772+0000","last_undegraded":"2024-09-16T00:26:29.290772+0000","last_fullsized":"2024-09-16T00:26:29.290772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969957+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:26:29.969957+0000","last_peered":"2024-09-16T00:26:29.969957+0000","last_clean":"2024-09-16T00:26:29.969957+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:26:29.969957+0000","last_undegraded":"2024-09-16T00:26:29.969957+0000","last_fullsized":"2024-09-16T00:26:29.969957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:13.284686+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:27:13.284686+0000","last_peered":"2024-09-16T00:27:13.284686+0000","last_clean":"2024-09-16T00:27:13.284686+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:27:13.284686+0000","last_undegraded":"2024-09-16T00:27:13.284686+0000","last_fullsized":"2024-09-16T00:27:13.284686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969341+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:26:29.969341+0000","last_peered":"2024-09-16T00:26:29.969341+0000","last_clean":"2024-09-16T00:26:29.969341+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:26:29.969341+0000","last_undegraded":"2024-09-16T00:26:29.969341+0000","last_fullsized":"2024-09-16T00:26:29.969341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289591+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:26:29.289591+0000","last_peered":"2024-09-16T00:26:29.289591+0000","last_clean":"2024-09-16T00:26:29.289591+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:26:29.289591+0000","last_undegraded":"2024-09-16T00:26:29.289591+0000","last_fullsized":"2024-09-16T00:26:29.289591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.602507+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:00.602507+0000","last_peered":"2024-09-16T00:27:00.602507+0000","last_clean":"2024-09-16T00:27:00.602507+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:00.602507+0000","last_undegraded":"2024-09-16T00:27:00.602507+0000","last_fullsized":"2024-09-16T00:27:00.602507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970529+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:26:29.970529+0000","last_peered":"2024-09-16T00:26:29.970529+0000","last_clean":"2024-09-16T00:26:29.970529+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:26:29.970529+0000","last_undegraded":"2024-09-16T00:26:29.970529+0000","last_fullsized":"2024-09-16T00:26:29.970529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289772+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:26:29.289772+0000","last_peered":"2024-09-16T00:26:29.289772+0000","last_clean":"2024-09-16T00:26:29.289772+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:26:29.289772+0000","last_undegraded":"2024-09-16T00:26:29.289772+0000","last_fullsized":"2024-09-16T00:26:29.289772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970481+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:26:29.970481+0000","last_peered":"2024-09-16T00:26:29.970481+0000","last_clean":"2024-09-16T00:26:29.970481+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:26:29.970481+0000","last_undegraded":"2024-09-16T00:26:29.970481+0000","last_fullsized":"2024-09-16T00:26:29.970481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971076+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:26:29.971076+0000","last_peered":"2024-09-16T00:26:29.971076+0000","last_clean":"2024-09-16T00:26:29.971076+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:26:29.971076+0000","last_undegraded":"2024-09-16T00:26:29.971076+0000","last_fullsized":"2024-09-16T00:26:29.971076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289710+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:26:29.289710+0000","last_peered":"2024-09-16T00:26:29.289710+0000","last_clean":"2024-09-16T00:26:29.289710+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:26:29.289710+0000","last_undegraded":"2024-09-16T00:26:29.289710+0000","last_fullsized":"2024-09-16T00:26:29.289710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970572+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:26:29.970572+0000","last_peered":"2024-09-16T00:26:29.970572+0000","last_clean":"2024-09-16T00:26:29.970572+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:26:29.970572+0000","last_undegraded":"2024-09-16T00:26:29.970572+0000","last_fullsized":"2024-09-16T00:26:29.970572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289482+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:26:29.289482+0000","last_peered":"2024-09-16T00:26:29.289482+0000","last_clean":"2024-09-16T00:26:29.289482+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:26:29.289482+0000","last_undegraded":"2024-09-16T00:26:29.289482+0000","last_fullsized":"2024-09-16T00:26:29.289482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969994+0000","last_change":"2024-09-16T00:26:25.295179+0000","last_active":"2024-09-16T00:26:29.969994+0000","last_peered":"2024-09-16T00:26:29.969994+0000","last_clean":"2024-09-16T00:26:29.969994+0000","last_became_active":"2024-09-16T00:26:25.294751+0000","last_became_peered":"2024-09-16T00:26:25.294751+0000","last_unstale":"2024-09-16T00:26:29.969994+0000","last_undegraded":"2024-09-16T00:26:29.969994+0000","last_fullsized":"2024-09-16T00:26:29.969994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:19.180985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:14.532309+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:27:14.532309+0000","last_peered":"2024-09-16T00:27:14.532309+0000","last_clean":"2024-09-16T00:27:14.532309+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:27:14.532309+0000","last_undegraded":"2024-09-16T00:27:14.532309+0000","last_fullsized":"2024-09-16T00:27:14.532309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289727+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:26:29.289727+0000","last_peered":"2024-09-16T00:26:29.289727+0000","last_clean":"2024-09-16T00:26:29.289727+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:26:29.289727+0000","last_undegraded":"2024-09-16T00:26:29.289727+0000","last_fullsized":"2024-09-16T00:26:29.289727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289050+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:26:29.289050+0000","last_peered":"2024-09-16T00:26:29.289050+0000","last_clean":"2024-09-16T00:26:29.289050+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:26:29.289050+0000","last_undegraded":"2024-09-16T00:26:29.289050+0000","last_fullsized":"2024-09-16T00:26:29.289050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.609137+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:01.609137+0000","last_peered":"2024-09-16T00:27:01.609137+0000","last_clean":"2024-09-16T00:27:01.609137+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:01.609137+0000","last_undegraded":"2024-09-16T00:27:01.609137+0000","last_fullsized":"2024-09-16T00:27:01.609137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970998+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:26:29.970998+0000","last_peered":"2024-09-16T00:26:29.970998+0000","last_clean":"2024-09-16T00:26:29.970998+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:26:29.970998+0000","last_undegraded":"2024-09-16T00:26:29.970998+0000","last_fullsized":"2024-09-16T00:26:29.970998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969686+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:26:29.969686+0000","last_peered":"2024-09-16T00:26:29.969686+0000","last_clean":"2024-09-16T00:26:29.969686+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:26:29.969686+0000","last_undegraded":"2024-09-16T00:26:29.969686+0000","last_fullsized":"2024-09-16T00:26:29.969686+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:02.511885+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:02.511885+0000","last_peered":"2024-09-16T00:27:02.511885+0000","last_clean":"2024-09-16T00:27:02.511885+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:02.511885+0000","last_undegraded":"2024-09-16T00:27:02.511885+0000","last_fullsized":"2024-09-16T00:27:02.511885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970170+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:26:29.970170+0000","last_peered":"2024-09-16T00:26:29.970170+0000","last_clean":"2024-09-16T00:26:29.970170+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:26:29.970170+0000","last_undegraded":"2024-09-16T00:26:29.970170+0000","last_fullsized":"2024-09-16T00:26:29.970170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971001+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:26:29.971001+0000","last_peered":"2024-09-16T00:26:29.971001+0000","last_clean":"2024-09-16T00:26:29.971001+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:26:29.971001+0000","last_undegraded":"2024-09-16T00:26:29.971001+0000","last_fullsized":"2024-09-16T00:26:29.971001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971253+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:26:29.971253+0000","last_peered":"2024-09-16T00:26:29.971253+0000","last_clean":"2024-09-16T00:26:29.971253+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:26:29.971253+0000","last_undegraded":"2024-09-16T00:26:29.971253+0000","last_fullsized":"2024-09-16T00:26:29.971253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.609685+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:01.609685+0000","last_peered":"2024-09-16T00:27:01.609685+0000","last_clean":"2024-09-16T00:27:01.609685+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:01.609685+0000","last_undegraded":"2024-09-16T00:27:01.609685+0000","last_fullsized":"2024-09-16T00:27:01.609685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970137+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:26:29.970137+0000","last_peered":"2024-09-16T00:26:29.970137+0000","last_clean":"2024-09-16T00:26:29.970137+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:26:29.970137+0000","last_undegraded":"2024-09-16T00:26:29.970137+0000","last_fullsized":"2024-09-16T00:26:29.970137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970495+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:26:29.970495+0000","last_peered":"2024-09-16T00:26:29.970495+0000","last_clean":"2024-09-16T00:26:29.970495+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:26:29.970495+0000","last_undegraded":"2024-09-16T00:26:29.970495+0000","last_fullsized":"2024-09-16T00:26:29.970495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971066+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:26:29.971066+0000","last_peered":"2024-09-16T00:26:29.971066+0000","last_clean":"2024-09-16T00:26:29.971066+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:26:29.971066+0000","last_undegraded":"2024-09-16T00:26:29.971066+0000","last_fullsized":"2024-09-16T00:26:29.971066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552289+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:26:29.552289+0000","last_peered":"2024-09-16T00:26:29.552289+0000","last_clean":"2024-09-16T00:26:29.552289+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:26:29.552289+0000","last_undegraded":"2024-09-16T00:26:29.552289+0000","last_fullsized":"2024-09-16T00:26:29.552289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970514+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:26:29.970514+0000","last_peered":"2024-09-16T00:26:29.970514+0000","last_clean":"2024-09-16T00:26:29.970514+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:26:29.970514+0000","last_undegraded":"2024-09-16T00:26:29.970514+0000","last_fullsized":"2024-09-16T00:26:29.970514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289740+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:26:29.289740+0000","last_peered":"2024-09-16T00:26:29.289740+0000","last_clean":"2024-09-16T00:26:29.289740+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:26:29.289740+0000","last_undegraded":"2024-09-16T00:26:29.289740+0000","last_fullsized":"2024-09-16T00:26:29.289740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969973+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:26:29.969973+0000","last_peered":"2024-09-16T00:26:29.969973+0000","last_clean":"2024-09-16T00:26:29.969973+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:26:29.969973+0000","last_undegraded":"2024-09-16T00:26:29.969973+0000","last_fullsized":"2024-09-16T00:26:29.969973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971037+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:26:29.971037+0000","last_peered":"2024-09-16T00:26:29.971037+0000","last_clean":"2024-09-16T00:26:29.971037+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:26:29.971037+0000","last_undegraded":"2024-09-16T00:26:29.971037+0000","last_fullsized":"2024-09-16T00:26:29.971037+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:02.512404+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:02.512404+0000","last_peered":"2024-09-16T00:27:02.512404+0000","last_clean":"2024-09-16T00:27:02.512404+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:02.512404+0000","last_undegraded":"2024-09-16T00:27:02.512404+0000","last_fullsized":"2024-09-16T00:27:02.512404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970610+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:26:29.970610+0000","last_peered":"2024-09-16T00:26:29.970610+0000","last_clean":"2024-09-16T00:26:29.970610+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:26:29.970610+0000","last_undegraded":"2024-09-16T00:26:29.970610+0000","last_fullsized":"2024-09-16T00:26:29.970610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970807+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:26:29.970807+0000","last_peered":"2024-09-16T00:26:29.970807+0000","last_clean":"2024-09-16T00:26:29.970807+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:26:29.970807+0000","last_undegraded":"2024-09-16T00:26:29.970807+0000","last_fullsized":"2024-09-16T00:26:29.970807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.610455+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:01.610455+0000","last_peered":"2024-09-16T00:27:01.610455+0000","last_clean":"2024-09-16T00:27:01.610455+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:01.610455+0000","last_undegraded":"2024-09-16T00:27:01.610455+0000","last_fullsized":"2024-09-16T00:27:01.610455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289638+0000","last_change":"2024-09-16T00:26:25.314911+0000","last_active":"2024-09-16T00:26:29.289638+0000","last_peered":"2024-09-16T00:26:29.289638+0000","last_clean":"2024-09-16T00:26:29.289638+0000","last_became_active":"2024-09-16T00:26:25.314429+0000","last_became_peered":"2024-09-16T00:26:25.314429+0000","last_unstale":"2024-09-16T00:26:29.289638+0000","last_undegraded":"2024-09-16T00:26:29.289638+0000","last_fullsized":"2024-09-16T00:26:29.289638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:57.436518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969825+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:26:29.969825+0000","last_peered":"2024-09-16T00:26:29.969825+0000","last_clean":"2024-09-16T00:26:29.969825+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:26:29.969825+0000","last_undegraded":"2024-09-16T00:26:29.969825+0000","last_fullsized":"2024-09-16T00:26:29.969825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:13.531547+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:27:13.531547+0000","last_peered":"2024-09-16T00:27:13.531547+0000","last_clean":"2024-09-16T00:27:13.531547+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:27:13.531547+0000","last_undegraded":"2024-09-16T00:27:13.531547+0000","last_fullsized":"2024-09-16T00:27:13.531547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970760+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:26:29.970760+0000","last_peered":"2024-09-16T00:26:29.970760+0000","last_clean":"2024-09-16T00:26:29.970760+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:26:29.970760+0000","last_undegraded":"2024-09-16T00:26:29.970760+0000","last_fullsized":"2024-09-16T00:26:29.970760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970211+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:26:29.970211+0000","last_peered":"2024-09-16T00:26:29.970211+0000","last_clean":"2024-09-16T00:26:29.970211+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:26:29.970211+0000","last_undegraded":"2024-09-16T00:26:29.970211+0000","last_fullsized":"2024-09-16T00:26:29.970211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.604616+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:00.604616+0000","last_peered":"2024-09-16T00:27:00.604616+0000","last_clean":"2024-09-16T00:27:00.604616+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:00.604616+0000","last_undegraded":"2024-09-16T00:27:00.604616+0000","last_fullsized":"2024-09-16T00:27:00.604616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971204+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:26:29.971204+0000","last_peered":"2024-09-16T00:26:29.971204+0000","last_clean":"2024-09-16T00:26:29.971204+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:26:29.971204+0000","last_undegraded":"2024-09-16T00:26:29.971204+0000","last_fullsized":"2024-09-16T00:26:29.971204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289254+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:26:29.289254+0000","last_peered":"2024-09-16T00:26:29.289254+0000","last_clean":"2024-09-16T00:26:29.289254+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:26:29.289254+0000","last_undegraded":"2024-09-16T00:26:29.289254+0000","last_fullsized":"2024-09-16T00:26:29.289254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.609640+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:01.609640+0000","last_peered":"2024-09-16T00:27:01.609640+0000","last_clean":"2024-09-16T00:27:01.609640+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:01.609640+0000","last_undegraded":"2024-09-16T00:27:01.609640+0000","last_fullsized":"2024-09-16T00:27:01.609640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970917+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:26:29.970917+0000","last_peered":"2024-09-16T00:26:29.970917+0000","last_clean":"2024-09-16T00:26:29.970917+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:26:29.970917+0000","last_undegraded":"2024-09-16T00:26:29.970917+0000","last_fullsized":"2024-09-16T00:26:29.970917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970908+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:26:29.970908+0000","last_peered":"2024-09-16T00:26:29.970908+0000","last_clean":"2024-09-16T00:26:29.970908+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:26:29.970908+0000","last_undegraded":"2024-09-16T00:26:29.970908+0000","last_fullsized":"2024-09-16T00:26:29.970908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'26","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:16.536670+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:27:16.536670+0000","last_peered":"2024-09-16T00:27:16.536670+0000","last_clean":"2024-09-16T00:27:16.536670+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:27:16.536670+0000","last_undegraded":"2024-09-16T00:27:16.536670+0000","last_fullsized":"2024-09-16T00:27:16.536670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":66,"num_write":27,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969660+0000","last_change":"2024-09-16T00:26:25.298202+0000","last_active":"2024-09-16T00:26:29.969660+0000","last_peered":"2024-09-16T00:26:29.969660+0000","last_clean":"2024-09-16T00:26:29.969660+0000","last_became_active":"2024-09-16T00:26:25.297336+0000","last_became_peered":"2024-09-16T00:26:25.297336+0000","last_unstale":"2024-09-16T00:26:29.969660+0000","last_undegraded":"2024-09-16T00:26:29.969660+0000","last_fullsized":"2024-09-16T00:26:29.969660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:44.405293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971761+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:26:29.971761+0000","last_peered":"2024-09-16T00:26:29.971761+0000","last_clean":"2024-09-16T00:26:29.971761+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:26:29.971761+0000","last_undegraded":"2024-09-16T00:26:29.971761+0000","last_fullsized":"2024-09-16T00:26:29.971761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.605642+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:00.605642+0000","last_peered":"2024-09-16T00:27:00.605642+0000","last_clean":"2024-09-16T00:27:00.605642+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:00.605642+0000","last_undegraded":"2024-09-16T00:27:00.605642+0000","last_fullsized":"2024-09-16T00:27:00.605642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971220+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:26:29.971220+0000","last_peered":"2024-09-16T00:26:29.971220+0000","last_clean":"2024-09-16T00:26:29.971220+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:26:29.971220+0000","last_undegraded":"2024-09-16T00:26:29.971220+0000","last_fullsized":"2024-09-16T00:26:29.971220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552587+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:26:29.552587+0000","last_peered":"2024-09-16T00:26:29.552587+0000","last_clean":"2024-09-16T00:26:29.552587+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:26:29.552587+0000","last_undegraded":"2024-09-16T00:26:29.552587+0000","last_fullsized":"2024-09-16T00:26:29.552587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.603716+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:00.603716+0000","last_peered":"2024-09-16T00:27:00.603716+0000","last_clean":"2024-09-16T00:27:00.603716+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:00.603716+0000","last_undegraded":"2024-09-16T00:27:00.603716+0000","last_fullsized":"2024-09-16T00:27:00.603716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969560+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:26:29.969560+0000","last_peered":"2024-09-16T00:26:29.969560+0000","last_clean":"2024-09-16T00:26:29.969560+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:26:29.969560+0000","last_undegraded":"2024-09-16T00:26:29.969560+0000","last_fullsized":"2024-09-16T00:26:29.969560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971257+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:26:29.971257+0000","last_peered":"2024-09-16T00:26:29.971257+0000","last_clean":"2024-09-16T00:26:29.971257+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:26:29.971257+0000","last_undegraded":"2024-09-16T00:26:29.971257+0000","last_fullsized":"2024-09-16T00:26:29.971257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:15.534188+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:27:15.534188+0000","last_peered":"2024-09-16T00:27:15.534188+0000","last_clean":"2024-09-16T00:27:15.534188+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:27:15.534188+0000","last_undegraded":"2024-09-16T00:27:15.534188+0000","last_fullsized":"2024-09-16T00:27:15.534188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969976+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:26:29.969976+0000","last_peered":"2024-09-16T00:26:29.969976+0000","last_clean":"2024-09-16T00:26:29.969976+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:26:29.969976+0000","last_undegraded":"2024-09-16T00:26:29.969976+0000","last_fullsized":"2024-09-16T00:26:29.969976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552308+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:26:29.552308+0000","last_peered":"2024-09-16T00:26:29.552308+0000","last_clean":"2024-09-16T00:26:29.552308+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:26:29.552308+0000","last_undegraded":"2024-09-16T00:26:29.552308+0000","last_fullsized":"2024-09-16T00:26:29.552308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969641+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:26:29.969641+0000","last_peered":"2024-09-16T00:26:29.969641+0000","last_clean":"2024-09-16T00:26:29.969641+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:26:29.969641+0000","last_undegraded":"2024-09-16T00:26:29.969641+0000","last_fullsized":"2024-09-16T00:26:29.969641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970322+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:26:29.970322+0000","last_peered":"2024-09-16T00:26:29.970322+0000","last_clean":"2024-09-16T00:26:29.970322+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:26:29.970322+0000","last_undegraded":"2024-09-16T00:26:29.970322+0000","last_fullsized":"2024-09-16T00:26:29.970322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971529+0000","last_change":"2024-09-16T00:26:25.300517+0000","last_active":"2024-09-16T00:26:29.971529+0000","last_peered":"2024-09-16T00:26:29.971529+0000","last_clean":"2024-09-16T00:26:29.971529+0000","last_became_active":"2024-09-16T00:26:25.300104+0000","last_became_peered":"2024-09-16T00:26:25.300104+0000","last_unstale":"2024-09-16T00:26:29.971529+0000","last_undegraded":"2024-09-16T00:26:29.971529+0000","last_fullsized":"2024-09-16T00:26:29.971529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:13.735689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969412+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:26:29.969412+0000","last_peered":"2024-09-16T00:26:29.969412+0000","last_clean":"2024-09-16T00:26:29.969412+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:26:29.969412+0000","last_undegraded":"2024-09-16T00:26:29.969412+0000","last_fullsized":"2024-09-16T00:26:29.969412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.607602+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:01.607602+0000","last_peered":"2024-09-16T00:27:01.607602+0000","last_clean":"2024-09-16T00:27:01.607602+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:01.607602+0000","last_undegraded":"2024-09-16T00:27:01.607602+0000","last_fullsized":"2024-09-16T00:27:01.607602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.608703+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:01.608703+0000","last_peered":"2024-09-16T00:27:01.608703+0000","last_clean":"2024-09-16T00:27:01.608703+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:01.608703+0000","last_undegraded":"2024-09-16T00:27:01.608703+0000","last_fullsized":"2024-09-16T00:27:01.608703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970603+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:26:29.970603+0000","last_peered":"2024-09-16T00:26:29.970603+0000","last_clean":"2024-09-16T00:26:29.970603+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:26:29.970603+0000","last_undegraded":"2024-09-16T00:26:29.970603+0000","last_fullsized":"2024-09-16T00:26:29.970603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290929+0000","last_change":"2024-09-16T00:26:25.314985+0000","last_active":"2024-09-16T00:26:29.290929+0000","last_peered":"2024-09-16T00:26:29.290929+0000","last_clean":"2024-09-16T00:26:29.290929+0000","last_became_active":"2024-09-16T00:26:25.314507+0000","last_became_peered":"2024-09-16T00:26:25.314507+0000","last_unstale":"2024-09-16T00:26:29.290929+0000","last_undegraded":"2024-09-16T00:26:29.290929+0000","last_fullsized":"2024-09-16T00:26:29.290929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:53:15.774916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:13.532053+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:27:13.532053+0000","last_peered":"2024-09-16T00:27:13.532053+0000","last_clean":"2024-09-16T00:27:13.532053+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:27:13.532053+0000","last_undegraded":"2024-09-16T00:27:13.532053+0000","last_fullsized":"2024-09-16T00:27:13.532053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971196+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:26:29.971196+0000","last_peered":"2024-09-16T00:26:29.971196+0000","last_clean":"2024-09-16T00:26:29.971196+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:26:29.971196+0000","last_undegraded":"2024-09-16T00:26:29.971196+0000","last_fullsized":"2024-09-16T00:26:29.971196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.291234+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:26:29.291234+0000","last_peered":"2024-09-16T00:26:29.291234+0000","last_clean":"2024-09-16T00:26:29.291234+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:26:29.291234+0000","last_undegraded":"2024-09-16T00:26:29.291234+0000","last_fullsized":"2024-09-16T00:26:29.291234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970095+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:26:29.970095+0000","last_peered":"2024-09-16T00:26:29.970095+0000","last_clean":"2024-09-16T00:26:29.970095+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:26:29.970095+0000","last_undegraded":"2024-09-16T00:26:29.970095+0000","last_fullsized":"2024-09-16T00:26:29.970095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.291077+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:26:29.291077+0000","last_peered":"2024-09-16T00:26:29.291077+0000","last_clean":"2024-09-16T00:26:29.291077+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:26:29.291077+0000","last_undegraded":"2024-09-16T00:26:29.291077+0000","last_fullsized":"2024-09-16T00:26:29.291077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290707+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:26:29.290707+0000","last_peered":"2024-09-16T00:26:29.290707+0000","last_clean":"2024-09-16T00:26:29.290707+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:26:29.290707+0000","last_undegraded":"2024-09-16T00:26:29.290707+0000","last_fullsized":"2024-09-16T00:26:29.290707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552252+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:26:29.552252+0000","last_peered":"2024-09-16T00:26:29.552252+0000","last_clean":"2024-09-16T00:26:29.552252+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:26:29.552252+0000","last_undegraded":"2024-09-16T00:26:29.552252+0000","last_fullsized":"2024-09-16T00:26:29.552252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289481+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:26:29.289481+0000","last_peered":"2024-09-16T00:26:29.289481+0000","last_clean":"2024-09-16T00:26:29.289481+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:26:29.289481+0000","last_undegraded":"2024-09-16T00:26:29.289481+0000","last_fullsized":"2024-09-16T00:26:29.289481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971411+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:26:29.971411+0000","last_peered":"2024-09-16T00:26:29.971411+0000","last_clean":"2024-09-16T00:26:29.971411+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:26:29.971411+0000","last_undegraded":"2024-09-16T00:26:29.971411+0000","last_fullsized":"2024-09-16T00:26:29.971411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.291359+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:26:29.291359+0000","last_peered":"2024-09-16T00:26:29.291359+0000","last_clean":"2024-09-16T00:26:29.291359+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:26:29.291359+0000","last_undegraded":"2024-09-16T00:26:29.291359+0000","last_fullsized":"2024-09-16T00:26:29.291359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969854+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:26:29.969854+0000","last_peered":"2024-09-16T00:26:29.969854+0000","last_clean":"2024-09-16T00:26:29.969854+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:26:29.969854+0000","last_undegraded":"2024-09-16T00:26:29.969854+0000","last_fullsized":"2024-09-16T00:26:29.969854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969867+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:26:29.969867+0000","last_peered":"2024-09-16T00:26:29.969867+0000","last_clean":"2024-09-16T00:26:29.969867+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:26:29.969867+0000","last_undegraded":"2024-09-16T00:26:29.969867+0000","last_fullsized":"2024-09-16T00:26:29.969867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970139+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:26:29.970139+0000","last_peered":"2024-09-16T00:26:29.970139+0000","last_clean":"2024-09-16T00:26:29.970139+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:26:29.970139+0000","last_undegraded":"2024-09-16T00:26:29.970139+0000","last_fullsized":"2024-09-16T00:26:29.970139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970875+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:26:29.970875+0000","last_peered":"2024-09-16T00:26:29.970875+0000","last_clean":"2024-09-16T00:26:29.970875+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:26:29.970875+0000","last_undegraded":"2024-09-16T00:26:29.970875+0000","last_fullsized":"2024-09-16T00:26:29.970875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289401+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:26:29.289401+0000","last_peered":"2024-09-16T00:26:29.289401+0000","last_clean":"2024-09-16T00:26:29.289401+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:26:29.289401+0000","last_undegraded":"2024-09-16T00:26:29.289401+0000","last_fullsized":"2024-09-16T00:26:29.289401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971221+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:26:29.971221+0000","last_peered":"2024-09-16T00:26:29.971221+0000","last_clean":"2024-09-16T00:26:29.971221+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:26:29.971221+0000","last_undegraded":"2024-09-16T00:26:29.971221+0000","last_fullsized":"2024-09-16T00:26:29.971221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.291307+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:26:29.291307+0000","last_peered":"2024-09-16T00:26:29.291307+0000","last_clean":"2024-09-16T00:26:29.291307+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:26:29.291307+0000","last_undegraded":"2024-09-16T00:26:29.291307+0000","last_fullsized":"2024-09-16T00:26:29.291307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970678+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:26:29.970678+0000","last_peered":"2024-09-16T00:26:29.970678+0000","last_clean":"2024-09-16T00:26:29.970678+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:26:29.970678+0000","last_undegraded":"2024-09-16T00:26:29.970678+0000","last_fullsized":"2024-09-16T00:26:29.970678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971627+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:26:29.971627+0000","last_peered":"2024-09-16T00:26:29.971627+0000","last_clean":"2024-09-16T00:26:29.971627+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:26:29.971627+0000","last_undegraded":"2024-09-16T00:26:29.971627+0000","last_fullsized":"2024-09-16T00:26:29.971627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969682+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:26:29.969682+0000","last_peered":"2024-09-16T00:26:29.969682+0000","last_clean":"2024-09-16T00:26:29.969682+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:26:29.969682+0000","last_undegraded":"2024-09-16T00:26:29.969682+0000","last_fullsized":"2024-09-16T00:26:29.969682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552125+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:26:29.552125+0000","last_peered":"2024-09-16T00:26:29.552125+0000","last_clean":"2024-09-16T00:26:29.552125+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:26:29.552125+0000","last_undegraded":"2024-09-16T00:26:29.552125+0000","last_fullsized":"2024-09-16T00:26:29.552125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552076+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:26:29.552076+0000","last_peered":"2024-09-16T00:26:29.552076+0000","last_clean":"2024-09-16T00:26:29.552076+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:26:29.552076+0000","last_undegraded":"2024-09-16T00:26:29.552076+0000","last_fullsized":"2024-09-16T00:26:29.552076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289339+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:26:29.289339+0000","last_peered":"2024-09-16T00:26:29.289339+0000","last_clean":"2024-09-16T00:26:29.289339+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:26:29.289339+0000","last_undegraded":"2024-09-16T00:26:29.289339+0000","last_fullsized":"2024-09-16T00:26:29.289339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971857+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:26:29.971857+0000","last_peered":"2024-09-16T00:26:29.971857+0000","last_clean":"2024-09-16T00:26:29.971857+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:26:29.971857+0000","last_undegraded":"2024-09-16T00:26:29.971857+0000","last_fullsized":"2024-09-16T00:26:29.971857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971136+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:26:29.971136+0000","last_peered":"2024-09-16T00:26:29.971136+0000","last_clean":"2024-09-16T00:26:29.971136+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:26:29.971136+0000","last_undegraded":"2024-09-16T00:26:29.971136+0000","last_fullsized":"2024-09-16T00:26:29.971136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290164+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:26:29.290164+0000","last_peered":"2024-09-16T00:26:29.290164+0000","last_clean":"2024-09-16T00:26:29.290164+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:26:29.290164+0000","last_undegraded":"2024-09-16T00:26:29.290164+0000","last_fullsized":"2024-09-16T00:26:29.290164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971446+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:26:29.971446+0000","last_peered":"2024-09-16T00:26:29.971446+0000","last_clean":"2024-09-16T00:26:29.971446+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:26:29.971446+0000","last_undegraded":"2024-09-16T00:26:29.971446+0000","last_fullsized":"2024-09-16T00:26:29.971446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969650+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:26:29.969650+0000","last_peered":"2024-09-16T00:26:29.969650+0000","last_clean":"2024-09-16T00:26:29.969650+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:26:29.969650+0000","last_undegraded":"2024-09-16T00:26:29.969650+0000","last_fullsized":"2024-09-16T00:26:29.969650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969984+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:26:29.969984+0000","last_peered":"2024-09-16T00:26:29.969984+0000","last_clean":"2024-09-16T00:26:29.969984+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:26:29.969984+0000","last_undegraded":"2024-09-16T00:26:29.969984+0000","last_fullsized":"2024-09-16T00:26:29.969984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289715+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:26:29.289715+0000","last_peered":"2024-09-16T00:26:29.289715+0000","last_clean":"2024-09-16T00:26:29.289715+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:26:29.289715+0000","last_undegraded":"2024-09-16T00:26:29.289715+0000","last_fullsized":"2024-09-16T00:26:29.289715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971208+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:26:29.971208+0000","last_peered":"2024-09-16T00:26:29.971208+0000","last_clean":"2024-09-16T00:26:29.971208+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:26:29.971208+0000","last_undegraded":"2024-09-16T00:26:29.971208+0000","last_fullsized":"2024-09-16T00:26:29.971208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971777+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:26:29.971777+0000","last_peered":"2024-09-16T00:26:29.971777+0000","last_clean":"2024-09-16T00:26:29.971777+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:26:29.971777+0000","last_undegraded":"2024-09-16T00:26:29.971777+0000","last_fullsized":"2024-09-16T00:26:29.971777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552116+0000","last_change":"2024-09-16T00:26:18.274217+0000","last_active":"2024-09-16T00:26:29.552116+0000","last_peered":"2024-09-16T00:26:29.552116+0000","last_clean":"2024-09-16T00:26:29.552116+0000","last_became_active":"2024-09-16T00:26:18.273929+0000","last_became_peered":"2024-09-16T00:26:18.273929+0000","last_unstale":"2024-09-16T00:26:29.552116+0000","last_undegraded":"2024-09-16T00:26:29.552116+0000","last_fullsized":"2024-09-16T00:26:29.552116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:31.633003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552254+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:26:29.552254+0000","last_peered":"2024-09-16T00:26:29.552254+0000","last_clean":"2024-09-16T00:26:29.552254+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:26:29.552254+0000","last_undegraded":"2024-09-16T00:26:29.552254+0000","last_fullsized":"2024-09-16T00:26:29.552254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971600+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:26:29.971600+0000","last_peered":"2024-09-16T00:26:29.971600+0000","last_clean":"2024-09-16T00:26:29.971600+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:26:29.971600+0000","last_undegraded":"2024-09-16T00:26:29.971600+0000","last_fullsized":"2024-09-16T00:26:29.971600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53101,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":110,"num_read_kb":114,"num_write":154,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":159303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514521,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":495845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743454,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1132211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":496001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36920,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708232,"statfs":{"total":95995035648,"available":95957229568,"internally_reserved":0,"allocated":1613824,"data_stored":1091912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":491257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712384,"statfs":{"total":95995035648,"available":95961481216,"internally_reserved":0,"allocated":1622016,"data_stored":1087500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":545005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":491451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":492910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":540265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":492645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":49742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":50676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:27:19.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:27:19.827 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:20.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:20 smithi001 ceph-mon[30934]: pgmap v316: 129 pgs: 129 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-09-16T00:27:20.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:20 smithi001 ceph-mon[30934]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:27:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.345+0000 7fcc69a50640 1 -- 172.21.15.1:0/1093778859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64106a80 msgr2=0x7fcc64106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.345+0000 7fcc69a50640 1 --2- 172.21.15.1:0/1093778859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64106a80 0x7fcc64106e60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009930 tx=0x7fcc5402a420 comp rx=0 tx=0).stop 2024-09-16T00:27:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.345+0000 7fcc69a50640 1 -- 172.21.15.1:0/1093778859 shutdown_connections 2024-09-16T00:27:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.345+0000 7fcc69a50640 1 --2- 172.21.15.1:0/1093778859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc64101420 0x7fcc6410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.345+0000 7fcc69a50640 1 --2- 172.21.15.1:0/1093778859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc64100a80 0x7fcc64100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.345+0000 7fcc69a50640 1 --2- 172.21.15.1:0/1093778859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64106a80 0x7fcc64106e60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.345+0000 7fcc69a50640 1 -- 172.21.15.1:0/1093778859 >> 172.21.15.1:0/1093778859 conn(0x7fcc640fc7a0 msgr2=0x7fcc640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.345+0000 7fcc69a50640 1 -- 172.21.15.1:0/1093778859 shutdown_connections 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.345+0000 7fcc69a50640 1 -- 172.21.15.1:0/1093778859 wait complete. 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc69a50640 1 Processor -- start 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc69a50640 1 -- start start 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc69a50640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64100a80 0x7fcc641984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc69a50640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc64101420 0x7fcc64198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc69a50640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc64106a80 0x7fcc6419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc69a50640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc64110e80 con 0x7fcc64100a80 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc69a50640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc64110d00 con 0x7fcc64106a80 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc69a50640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc64111000 con 0x7fcc64101420 2024-09-16T00:27:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc62ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64100a80 0x7fcc641984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc62ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64100a80 0x7fcc641984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45648/0 (socket says 172.21.15.1:45648) 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.346+0000 7fcc62ffd640 1 -- 172.21.15.1:0/3406631283 learned_addr learned my addr 172.21.15.1:0/3406631283 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc5bfff640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc64101420 0x7fcc64198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc637fe640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc64106a80 0x7fcc6419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc62ffd640 1 -- 172.21.15.1:0/3406631283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc64101420 msgr2=0x7fcc64198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc62ffd640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc64101420 0x7fcc64198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc62ffd640 1 -- 172.21.15.1:0/3406631283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc64106a80 msgr2=0x7fcc6419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc62ffd640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc64106a80 0x7fcc6419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc62ffd640 1 -- 172.21.15.1:0/3406631283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6419d4a0 con 0x7fcc64100a80 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc637fe640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc64106a80 0x7fcc6419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc62ffd640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64100a80 0x7fcc641984f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009230 tx=0x7fcc54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc68a4e640 1 -- 172.21.15.1:0/3406631283 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5403f030 con 0x7fcc64100a80 2024-09-16T00:27:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc68a4e640 1 -- 172.21.15.1:0/3406631283 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc54003ad0 con 0x7fcc64100a80 2024-09-16T00:27:20.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6419d730 con 0x7fcc64100a80 2024-09-16T00:27:20.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc68a4e640 1 -- 172.21.15.1:0/3406631283 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54035070 con 0x7fcc64100a80 2024-09-16T00:27:20.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.347+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc641a5070 con 0x7fcc64100a80 2024-09-16T00:27:20.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.348+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc641021c0 con 0x7fcc64100a80 2024-09-16T00:27:20.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.349+0000 7fcc68a4e640 1 -- 172.21.15.1:0/3406631283 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fcc54042030 con 0x7fcc64100a80 2024-09-16T00:27:20.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.350+0000 7fcc68a4e640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc500a32d0 0x7fcc500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:20.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.350+0000 7fcc68a4e640 1 -- 172.21.15.1:0/3406631283 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fcc540f02d0 con 0x7fcc64100a80 2024-09-16T00:27:20.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.350+0000 7fcc5bfff640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc500a32d0 0x7fcc500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:20.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.351+0000 7fcc5bfff640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc500a32d0 0x7fcc500a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcc640feb40 tx=0x7fcc4c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:20.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.354+0000 7fcc68a4e640 1 -- 172.21.15.1:0/3406631283 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc540bcb50 con 0x7fcc64100a80 2024-09-16T00:27:20.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:20 smithi153 ceph-mon[35538]: pgmap v316: 129 pgs: 129 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-09-16T00:27:20.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:20 smithi153 ceph-mon[35538]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:27:20.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.525+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc6406afe0 con 0x7fcc500a32d0 2024-09-16T00:27:20.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.533+0000 7fcc68a4e640 1 -- 172.21.15.1:0/3406631283 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340357 (secure 0 0 0) 0x7fcc6406afe0 con 0x7fcc500a32d0 2024-09-16T00:27:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:27:20.539 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:27:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc500a32d0 msgr2=0x7fcc500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc500a32d0 0x7fcc500a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcc640feb40 tx=0x7fcc4c00b390 comp rx=0 tx=0).stop 2024-09-16T00:27:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64100a80 msgr2=0x7fcc641984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64100a80 0x7fcc641984f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009230 tx=0x7fcc54004270 comp rx=0 tx=0).stop 2024-09-16T00:27:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 shutdown_connections 2024-09-16T00:27:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc64106a80 0x7fcc6419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc500a32d0 0x7fcc500a5790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc64101420 0x7fcc64198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 --2- 172.21.15.1:0/3406631283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc64100a80 0x7fcc641984f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:20.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 >> 172.21.15.1:0/3406631283 conn(0x7fcc640fc7a0 msgr2=0x7fcc6406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:20.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.536+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 shutdown_connections 2024-09-16T00:27:20.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:20.537+0000 7fcc69a50640 1 -- 172.21.15.1:0/3406631283 wait complete. 2024-09-16T00:27:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:20 smithi019 ceph-mon[35186]: pgmap v316: 129 pgs: 129 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-09-16T00:27:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:20 smithi019 ceph-mon[35186]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:27:21.188 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-16T00:27:20.322908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643703,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":246,"num_read_kb":223,"num_write":337,"num_write_kb":3039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375552,"kb_used_data":13824,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566272,"statfs":{"total":1151940427776,"available":1151555862528,"internally_reserved":0,"allocated":14155776,"data_stored":7873499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39279,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":70,"num_write":23,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.610128+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:27:01.610128+0000","last_peered":"2024-09-16T00:27:01.610128+0000","last_clean":"2024-09-16T00:27:01.610128+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:27:01.610128+0000","last_undegraded":"2024-09-16T00:27:01.610128+0000","last_fullsized":"2024-09-16T00:27:01.610128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969951+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:26:29.969951+0000","last_peered":"2024-09-16T00:26:29.969951+0000","last_clean":"2024-09-16T00:26:29.969951+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:26:29.969951+0000","last_undegraded":"2024-09-16T00:26:29.969951+0000","last_fullsized":"2024-09-16T00:26:29.969951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290691+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:26:29.290691+0000","last_peered":"2024-09-16T00:26:29.290691+0000","last_clean":"2024-09-16T00:26:29.290691+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:26:29.290691+0000","last_undegraded":"2024-09-16T00:26:29.290691+0000","last_fullsized":"2024-09-16T00:26:29.290691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970131+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:26:29.970131+0000","last_peered":"2024-09-16T00:26:29.970131+0000","last_clean":"2024-09-16T00:26:29.970131+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:26:29.970131+0000","last_undegraded":"2024-09-16T00:26:29.970131+0000","last_fullsized":"2024-09-16T00:26:29.970131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552533+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:26:29.552533+0000","last_peered":"2024-09-16T00:26:29.552533+0000","last_clean":"2024-09-16T00:26:29.552533+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:26:29.552533+0000","last_undegraded":"2024-09-16T00:26:29.552533+0000","last_fullsized":"2024-09-16T00:26:29.552533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970105+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:26:29.970105+0000","last_peered":"2024-09-16T00:26:29.970105+0000","last_clean":"2024-09-16T00:26:29.970105+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:26:29.970105+0000","last_undegraded":"2024-09-16T00:26:29.970105+0000","last_fullsized":"2024-09-16T00:26:29.970105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.608620+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:01.608620+0000","last_peered":"2024-09-16T00:27:01.608620+0000","last_clean":"2024-09-16T00:27:01.608620+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:01.608620+0000","last_undegraded":"2024-09-16T00:27:01.608620+0000","last_fullsized":"2024-09-16T00:27:01.608620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971638+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:26:29.971638+0000","last_peered":"2024-09-16T00:26:29.971638+0000","last_clean":"2024-09-16T00:26:29.971638+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:26:29.971638+0000","last_undegraded":"2024-09-16T00:26:29.971638+0000","last_fullsized":"2024-09-16T00:26:29.971638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971028+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:26:29.971028+0000","last_peered":"2024-09-16T00:26:29.971028+0000","last_clean":"2024-09-16T00:26:29.971028+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:26:29.971028+0000","last_undegraded":"2024-09-16T00:26:29.971028+0000","last_fullsized":"2024-09-16T00:26:29.971028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970207+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:26:29.970207+0000","last_peered":"2024-09-16T00:26:29.970207+0000","last_clean":"2024-09-16T00:26:29.970207+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:26:29.970207+0000","last_undegraded":"2024-09-16T00:26:29.970207+0000","last_fullsized":"2024-09-16T00:26:29.970207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970678+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:26:29.970678+0000","last_peered":"2024-09-16T00:26:29.970678+0000","last_clean":"2024-09-16T00:26:29.970678+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:26:29.970678+0000","last_undegraded":"2024-09-16T00:26:29.970678+0000","last_fullsized":"2024-09-16T00:26:29.970678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970756+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:26:29.970756+0000","last_peered":"2024-09-16T00:26:29.970756+0000","last_clean":"2024-09-16T00:26:29.970756+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:26:29.970756+0000","last_undegraded":"2024-09-16T00:26:29.970756+0000","last_fullsized":"2024-09-16T00:26:29.970756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970362+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:26:29.970362+0000","last_peered":"2024-09-16T00:26:29.970362+0000","last_clean":"2024-09-16T00:26:29.970362+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:26:29.970362+0000","last_undegraded":"2024-09-16T00:26:29.970362+0000","last_fullsized":"2024-09-16T00:26:29.970362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.604370+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:00.604370+0000","last_peered":"2024-09-16T00:27:00.604370+0000","last_clean":"2024-09-16T00:27:00.604370+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:00.604370+0000","last_undegraded":"2024-09-16T00:27:00.604370+0000","last_fullsized":"2024-09-16T00:27:00.604370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971345+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:26:29.971345+0000","last_peered":"2024-09-16T00:26:29.971345+0000","last_clean":"2024-09-16T00:26:29.971345+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:26:29.971345+0000","last_undegraded":"2024-09-16T00:26:29.971345+0000","last_fullsized":"2024-09-16T00:26:29.971345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969805+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:26:29.969805+0000","last_peered":"2024-09-16T00:26:29.969805+0000","last_clean":"2024-09-16T00:26:29.969805+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:26:29.969805+0000","last_undegraded":"2024-09-16T00:26:29.969805+0000","last_fullsized":"2024-09-16T00:26:29.969805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290960+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:26:29.290960+0000","last_peered":"2024-09-16T00:26:29.290960+0000","last_clean":"2024-09-16T00:26:29.290960+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:26:29.290960+0000","last_undegraded":"2024-09-16T00:26:29.290960+0000","last_fullsized":"2024-09-16T00:26:29.290960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971635+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:26:29.971635+0000","last_peered":"2024-09-16T00:26:29.971635+0000","last_clean":"2024-09-16T00:26:29.971635+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:26:29.971635+0000","last_undegraded":"2024-09-16T00:26:29.971635+0000","last_fullsized":"2024-09-16T00:26:29.971635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970212+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:26:29.970212+0000","last_peered":"2024-09-16T00:26:29.970212+0000","last_clean":"2024-09-16T00:26:29.970212+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:26:29.970212+0000","last_undegraded":"2024-09-16T00:26:29.970212+0000","last_fullsized":"2024-09-16T00:26:29.970212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.605861+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:00.605861+0000","last_peered":"2024-09-16T00:27:00.605861+0000","last_clean":"2024-09-16T00:27:00.605861+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:00.605861+0000","last_undegraded":"2024-09-16T00:27:00.605861+0000","last_fullsized":"2024-09-16T00:27:00.605861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970738+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:26:29.970738+0000","last_peered":"2024-09-16T00:26:29.970738+0000","last_clean":"2024-09-16T00:26:29.970738+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:26:29.970738+0000","last_undegraded":"2024-09-16T00:26:29.970738+0000","last_fullsized":"2024-09-16T00:26:29.970738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970609+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:26:29.970609+0000","last_peered":"2024-09-16T00:26:29.970609+0000","last_clean":"2024-09-16T00:26:29.970609+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:26:29.970609+0000","last_undegraded":"2024-09-16T00:26:29.970609+0000","last_fullsized":"2024-09-16T00:26:29.970609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.604570+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:00.604570+0000","last_peered":"2024-09-16T00:27:00.604570+0000","last_clean":"2024-09-16T00:27:00.604570+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:00.604570+0000","last_undegraded":"2024-09-16T00:27:00.604570+0000","last_fullsized":"2024-09-16T00:27:00.604570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971399+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:26:29.971399+0000","last_peered":"2024-09-16T00:26:29.971399+0000","last_clean":"2024-09-16T00:26:29.971399+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:26:29.971399+0000","last_undegraded":"2024-09-16T00:26:29.971399+0000","last_fullsized":"2024-09-16T00:26:29.971399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552385+0000","last_change":"2024-09-16T00:26:18.267540+0000","last_active":"2024-09-16T00:26:29.552385+0000","last_peered":"2024-09-16T00:26:29.552385+0000","last_clean":"2024-09-16T00:26:29.552385+0000","last_became_active":"2024-09-16T00:26:18.267230+0000","last_became_peered":"2024-09-16T00:26:18.267230+0000","last_unstale":"2024-09-16T00:26:29.552385+0000","last_undegraded":"2024-09-16T00:26:29.552385+0000","last_fullsized":"2024-09-16T00:26:29.552385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:03.127446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.603092+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:00.603092+0000","last_peered":"2024-09-16T00:27:00.603092+0000","last_clean":"2024-09-16T00:27:00.603092+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:00.603092+0000","last_undegraded":"2024-09-16T00:27:00.603092+0000","last_fullsized":"2024-09-16T00:27:00.603092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290772+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:26:29.290772+0000","last_peered":"2024-09-16T00:26:29.290772+0000","last_clean":"2024-09-16T00:26:29.290772+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:26:29.290772+0000","last_undegraded":"2024-09-16T00:26:29.290772+0000","last_fullsized":"2024-09-16T00:26:29.290772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969957+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:26:29.969957+0000","last_peered":"2024-09-16T00:26:29.969957+0000","last_clean":"2024-09-16T00:26:29.969957+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:26:29.969957+0000","last_undegraded":"2024-09-16T00:26:29.969957+0000","last_fullsized":"2024-09-16T00:26:29.969957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:13.284686+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:27:13.284686+0000","last_peered":"2024-09-16T00:27:13.284686+0000","last_clean":"2024-09-16T00:27:13.284686+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:27:13.284686+0000","last_undegraded":"2024-09-16T00:27:13.284686+0000","last_fullsized":"2024-09-16T00:27:13.284686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969341+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:26:29.969341+0000","last_peered":"2024-09-16T00:26:29.969341+0000","last_clean":"2024-09-16T00:26:29.969341+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:26:29.969341+0000","last_undegraded":"2024-09-16T00:26:29.969341+0000","last_fullsized":"2024-09-16T00:26:29.969341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289591+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:26:29.289591+0000","last_peered":"2024-09-16T00:26:29.289591+0000","last_clean":"2024-09-16T00:26:29.289591+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:26:29.289591+0000","last_undegraded":"2024-09-16T00:26:29.289591+0000","last_fullsized":"2024-09-16T00:26:29.289591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.602507+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:00.602507+0000","last_peered":"2024-09-16T00:27:00.602507+0000","last_clean":"2024-09-16T00:27:00.602507+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:00.602507+0000","last_undegraded":"2024-09-16T00:27:00.602507+0000","last_fullsized":"2024-09-16T00:27:00.602507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970529+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:26:29.970529+0000","last_peered":"2024-09-16T00:26:29.970529+0000","last_clean":"2024-09-16T00:26:29.970529+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:26:29.970529+0000","last_undegraded":"2024-09-16T00:26:29.970529+0000","last_fullsized":"2024-09-16T00:26:29.970529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289772+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:26:29.289772+0000","last_peered":"2024-09-16T00:26:29.289772+0000","last_clean":"2024-09-16T00:26:29.289772+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:26:29.289772+0000","last_undegraded":"2024-09-16T00:26:29.289772+0000","last_fullsized":"2024-09-16T00:26:29.289772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970481+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:26:29.970481+0000","last_peered":"2024-09-16T00:26:29.970481+0000","last_clean":"2024-09-16T00:26:29.970481+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:26:29.970481+0000","last_undegraded":"2024-09-16T00:26:29.970481+0000","last_fullsized":"2024-09-16T00:26:29.970481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971076+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:26:29.971076+0000","last_peered":"2024-09-16T00:26:29.971076+0000","last_clean":"2024-09-16T00:26:29.971076+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:26:29.971076+0000","last_undegraded":"2024-09-16T00:26:29.971076+0000","last_fullsized":"2024-09-16T00:26:29.971076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289710+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:26:29.289710+0000","last_peered":"2024-09-16T00:26:29.289710+0000","last_clean":"2024-09-16T00:26:29.289710+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:26:29.289710+0000","last_undegraded":"2024-09-16T00:26:29.289710+0000","last_fullsized":"2024-09-16T00:26:29.289710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970572+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:26:29.970572+0000","last_peered":"2024-09-16T00:26:29.970572+0000","last_clean":"2024-09-16T00:26:29.970572+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:26:29.970572+0000","last_undegraded":"2024-09-16T00:26:29.970572+0000","last_fullsized":"2024-09-16T00:26:29.970572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289482+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:26:29.289482+0000","last_peered":"2024-09-16T00:26:29.289482+0000","last_clean":"2024-09-16T00:26:29.289482+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:26:29.289482+0000","last_undegraded":"2024-09-16T00:26:29.289482+0000","last_fullsized":"2024-09-16T00:26:29.289482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969994+0000","last_change":"2024-09-16T00:26:25.295179+0000","last_active":"2024-09-16T00:26:29.969994+0000","last_peered":"2024-09-16T00:26:29.969994+0000","last_clean":"2024-09-16T00:26:29.969994+0000","last_became_active":"2024-09-16T00:26:25.294751+0000","last_became_peered":"2024-09-16T00:26:25.294751+0000","last_unstale":"2024-09-16T00:26:29.969994+0000","last_undegraded":"2024-09-16T00:26:29.969994+0000","last_fullsized":"2024-09-16T00:26:29.969994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:19.180985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:14.532309+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:27:14.532309+0000","last_peered":"2024-09-16T00:27:14.532309+0000","last_clean":"2024-09-16T00:27:14.532309+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:27:14.532309+0000","last_undegraded":"2024-09-16T00:27:14.532309+0000","last_fullsized":"2024-09-16T00:27:14.532309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289727+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:26:29.289727+0000","last_peered":"2024-09-16T00:26:29.289727+0000","last_clean":"2024-09-16T00:26:29.289727+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:26:29.289727+0000","last_undegraded":"2024-09-16T00:26:29.289727+0000","last_fullsized":"2024-09-16T00:26:29.289727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289050+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:26:29.289050+0000","last_peered":"2024-09-16T00:26:29.289050+0000","last_clean":"2024-09-16T00:26:29.289050+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:26:29.289050+0000","last_undegraded":"2024-09-16T00:26:29.289050+0000","last_fullsized":"2024-09-16T00:26:29.289050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.609137+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:01.609137+0000","last_peered":"2024-09-16T00:27:01.609137+0000","last_clean":"2024-09-16T00:27:01.609137+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:01.609137+0000","last_undegraded":"2024-09-16T00:27:01.609137+0000","last_fullsized":"2024-09-16T00:27:01.609137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970998+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:26:29.970998+0000","last_peered":"2024-09-16T00:26:29.970998+0000","last_clean":"2024-09-16T00:26:29.970998+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:26:29.970998+0000","last_undegraded":"2024-09-16T00:26:29.970998+0000","last_fullsized":"2024-09-16T00:26:29.970998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969686+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:26:29.969686+0000","last_peered":"2024-09-16T00:26:29.969686+0000","last_clean":"2024-09-16T00:26:29.969686+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:26:29.969686+0000","last_undegraded":"2024-09-16T00:26:29.969686+0000","last_fullsized":"2024-09-16T00:26:29.969686+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:02.511885+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:02.511885+0000","last_peered":"2024-09-16T00:27:02.511885+0000","last_clean":"2024-09-16T00:27:02.511885+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:02.511885+0000","last_undegraded":"2024-09-16T00:27:02.511885+0000","last_fullsized":"2024-09-16T00:27:02.511885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970170+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:26:29.970170+0000","last_peered":"2024-09-16T00:26:29.970170+0000","last_clean":"2024-09-16T00:26:29.970170+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:26:29.970170+0000","last_undegraded":"2024-09-16T00:26:29.970170+0000","last_fullsized":"2024-09-16T00:26:29.970170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971001+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:26:29.971001+0000","last_peered":"2024-09-16T00:26:29.971001+0000","last_clean":"2024-09-16T00:26:29.971001+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:26:29.971001+0000","last_undegraded":"2024-09-16T00:26:29.971001+0000","last_fullsized":"2024-09-16T00:26:29.971001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971253+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:26:29.971253+0000","last_peered":"2024-09-16T00:26:29.971253+0000","last_clean":"2024-09-16T00:26:29.971253+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:26:29.971253+0000","last_undegraded":"2024-09-16T00:26:29.971253+0000","last_fullsized":"2024-09-16T00:26:29.971253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.609685+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:01.609685+0000","last_peered":"2024-09-16T00:27:01.609685+0000","last_clean":"2024-09-16T00:27:01.609685+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:01.609685+0000","last_undegraded":"2024-09-16T00:27:01.609685+0000","last_fullsized":"2024-09-16T00:27:01.609685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970137+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:26:29.970137+0000","last_peered":"2024-09-16T00:26:29.970137+0000","last_clean":"2024-09-16T00:26:29.970137+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:26:29.970137+0000","last_undegraded":"2024-09-16T00:26:29.970137+0000","last_fullsized":"2024-09-16T00:26:29.970137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970495+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:26:29.970495+0000","last_peered":"2024-09-16T00:26:29.970495+0000","last_clean":"2024-09-16T00:26:29.970495+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:26:29.970495+0000","last_undegraded":"2024-09-16T00:26:29.970495+0000","last_fullsized":"2024-09-16T00:26:29.970495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971066+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:26:29.971066+0000","last_peered":"2024-09-16T00:26:29.971066+0000","last_clean":"2024-09-16T00:26:29.971066+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:26:29.971066+0000","last_undegraded":"2024-09-16T00:26:29.971066+0000","last_fullsized":"2024-09-16T00:26:29.971066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552289+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:26:29.552289+0000","last_peered":"2024-09-16T00:26:29.552289+0000","last_clean":"2024-09-16T00:26:29.552289+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:26:29.552289+0000","last_undegraded":"2024-09-16T00:26:29.552289+0000","last_fullsized":"2024-09-16T00:26:29.552289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970514+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:26:29.970514+0000","last_peered":"2024-09-16T00:26:29.970514+0000","last_clean":"2024-09-16T00:26:29.970514+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:26:29.970514+0000","last_undegraded":"2024-09-16T00:26:29.970514+0000","last_fullsized":"2024-09-16T00:26:29.970514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289740+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:26:29.289740+0000","last_peered":"2024-09-16T00:26:29.289740+0000","last_clean":"2024-09-16T00:26:29.289740+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:26:29.289740+0000","last_undegraded":"2024-09-16T00:26:29.289740+0000","last_fullsized":"2024-09-16T00:26:29.289740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969973+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:26:29.969973+0000","last_peered":"2024-09-16T00:26:29.969973+0000","last_clean":"2024-09-16T00:26:29.969973+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:26:29.969973+0000","last_undegraded":"2024-09-16T00:26:29.969973+0000","last_fullsized":"2024-09-16T00:26:29.969973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971037+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:26:29.971037+0000","last_peered":"2024-09-16T00:26:29.971037+0000","last_clean":"2024-09-16T00:26:29.971037+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:26:29.971037+0000","last_undegraded":"2024-09-16T00:26:29.971037+0000","last_fullsized":"2024-09-16T00:26:29.971037+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:02.512404+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:02.512404+0000","last_peered":"2024-09-16T00:27:02.512404+0000","last_clean":"2024-09-16T00:27:02.512404+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:02.512404+0000","last_undegraded":"2024-09-16T00:27:02.512404+0000","last_fullsized":"2024-09-16T00:27:02.512404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970610+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:26:29.970610+0000","last_peered":"2024-09-16T00:26:29.970610+0000","last_clean":"2024-09-16T00:26:29.970610+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:26:29.970610+0000","last_undegraded":"2024-09-16T00:26:29.970610+0000","last_fullsized":"2024-09-16T00:26:29.970610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970807+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:26:29.970807+0000","last_peered":"2024-09-16T00:26:29.970807+0000","last_clean":"2024-09-16T00:26:29.970807+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:26:29.970807+0000","last_undegraded":"2024-09-16T00:26:29.970807+0000","last_fullsized":"2024-09-16T00:26:29.970807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.610455+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:01.610455+0000","last_peered":"2024-09-16T00:27:01.610455+0000","last_clean":"2024-09-16T00:27:01.610455+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:01.610455+0000","last_undegraded":"2024-09-16T00:27:01.610455+0000","last_fullsized":"2024-09-16T00:27:01.610455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289638+0000","last_change":"2024-09-16T00:26:25.314911+0000","last_active":"2024-09-16T00:26:29.289638+0000","last_peered":"2024-09-16T00:26:29.289638+0000","last_clean":"2024-09-16T00:26:29.289638+0000","last_became_active":"2024-09-16T00:26:25.314429+0000","last_became_peered":"2024-09-16T00:26:25.314429+0000","last_unstale":"2024-09-16T00:26:29.289638+0000","last_undegraded":"2024-09-16T00:26:29.289638+0000","last_fullsized":"2024-09-16T00:26:29.289638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:57.436518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969825+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:26:29.969825+0000","last_peered":"2024-09-16T00:26:29.969825+0000","last_clean":"2024-09-16T00:26:29.969825+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:26:29.969825+0000","last_undegraded":"2024-09-16T00:26:29.969825+0000","last_fullsized":"2024-09-16T00:26:29.969825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:18.538912+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:27:18.538912+0000","last_peered":"2024-09-16T00:27:18.538912+0000","last_clean":"2024-09-16T00:27:18.538912+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:27:18.538912+0000","last_undegraded":"2024-09-16T00:27:18.538912+0000","last_fullsized":"2024-09-16T00:27:18.538912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970760+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:26:29.970760+0000","last_peered":"2024-09-16T00:26:29.970760+0000","last_clean":"2024-09-16T00:26:29.970760+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:26:29.970760+0000","last_undegraded":"2024-09-16T00:26:29.970760+0000","last_fullsized":"2024-09-16T00:26:29.970760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970211+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:26:29.970211+0000","last_peered":"2024-09-16T00:26:29.970211+0000","last_clean":"2024-09-16T00:26:29.970211+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:26:29.970211+0000","last_undegraded":"2024-09-16T00:26:29.970211+0000","last_fullsized":"2024-09-16T00:26:29.970211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.604616+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:00.604616+0000","last_peered":"2024-09-16T00:27:00.604616+0000","last_clean":"2024-09-16T00:27:00.604616+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:00.604616+0000","last_undegraded":"2024-09-16T00:27:00.604616+0000","last_fullsized":"2024-09-16T00:27:00.604616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971204+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:26:29.971204+0000","last_peered":"2024-09-16T00:26:29.971204+0000","last_clean":"2024-09-16T00:26:29.971204+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:26:29.971204+0000","last_undegraded":"2024-09-16T00:26:29.971204+0000","last_fullsized":"2024-09-16T00:26:29.971204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289254+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:26:29.289254+0000","last_peered":"2024-09-16T00:26:29.289254+0000","last_clean":"2024-09-16T00:26:29.289254+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:26:29.289254+0000","last_undegraded":"2024-09-16T00:26:29.289254+0000","last_fullsized":"2024-09-16T00:26:29.289254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.609640+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:01.609640+0000","last_peered":"2024-09-16T00:27:01.609640+0000","last_clean":"2024-09-16T00:27:01.609640+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:01.609640+0000","last_undegraded":"2024-09-16T00:27:01.609640+0000","last_fullsized":"2024-09-16T00:27:01.609640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970917+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:26:29.970917+0000","last_peered":"2024-09-16T00:26:29.970917+0000","last_clean":"2024-09-16T00:26:29.970917+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:26:29.970917+0000","last_undegraded":"2024-09-16T00:26:29.970917+0000","last_fullsized":"2024-09-16T00:26:29.970917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970908+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:26:29.970908+0000","last_peered":"2024-09-16T00:26:29.970908+0000","last_clean":"2024-09-16T00:26:29.970908+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:26:29.970908+0000","last_undegraded":"2024-09-16T00:26:29.970908+0000","last_fullsized":"2024-09-16T00:26:29.970908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'26","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:16.536670+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:27:16.536670+0000","last_peered":"2024-09-16T00:27:16.536670+0000","last_clean":"2024-09-16T00:27:16.536670+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:27:16.536670+0000","last_undegraded":"2024-09-16T00:27:16.536670+0000","last_fullsized":"2024-09-16T00:27:16.536670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":66,"num_write":27,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969660+0000","last_change":"2024-09-16T00:26:25.298202+0000","last_active":"2024-09-16T00:26:29.969660+0000","last_peered":"2024-09-16T00:26:29.969660+0000","last_clean":"2024-09-16T00:26:29.969660+0000","last_became_active":"2024-09-16T00:26:25.297336+0000","last_became_peered":"2024-09-16T00:26:25.297336+0000","last_unstale":"2024-09-16T00:26:29.969660+0000","last_undegraded":"2024-09-16T00:26:29.969660+0000","last_fullsized":"2024-09-16T00:26:29.969660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:44.405293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971761+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:26:29.971761+0000","last_peered":"2024-09-16T00:26:29.971761+0000","last_clean":"2024-09-16T00:26:29.971761+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:26:29.971761+0000","last_undegraded":"2024-09-16T00:26:29.971761+0000","last_fullsized":"2024-09-16T00:26:29.971761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.605642+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:00.605642+0000","last_peered":"2024-09-16T00:27:00.605642+0000","last_clean":"2024-09-16T00:27:00.605642+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:00.605642+0000","last_undegraded":"2024-09-16T00:27:00.605642+0000","last_fullsized":"2024-09-16T00:27:00.605642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971220+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:26:29.971220+0000","last_peered":"2024-09-16T00:26:29.971220+0000","last_clean":"2024-09-16T00:26:29.971220+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:26:29.971220+0000","last_undegraded":"2024-09-16T00:26:29.971220+0000","last_fullsized":"2024-09-16T00:26:29.971220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552587+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:26:29.552587+0000","last_peered":"2024-09-16T00:26:29.552587+0000","last_clean":"2024-09-16T00:26:29.552587+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:26:29.552587+0000","last_undegraded":"2024-09-16T00:26:29.552587+0000","last_fullsized":"2024-09-16T00:26:29.552587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:00.603716+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:00.603716+0000","last_peered":"2024-09-16T00:27:00.603716+0000","last_clean":"2024-09-16T00:27:00.603716+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:00.603716+0000","last_undegraded":"2024-09-16T00:27:00.603716+0000","last_fullsized":"2024-09-16T00:27:00.603716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969560+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:26:29.969560+0000","last_peered":"2024-09-16T00:26:29.969560+0000","last_clean":"2024-09-16T00:26:29.969560+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:26:29.969560+0000","last_undegraded":"2024-09-16T00:26:29.969560+0000","last_fullsized":"2024-09-16T00:26:29.969560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971257+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:26:29.971257+0000","last_peered":"2024-09-16T00:26:29.971257+0000","last_clean":"2024-09-16T00:26:29.971257+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:26:29.971257+0000","last_undegraded":"2024-09-16T00:26:29.971257+0000","last_fullsized":"2024-09-16T00:26:29.971257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:15.534188+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:27:15.534188+0000","last_peered":"2024-09-16T00:27:15.534188+0000","last_clean":"2024-09-16T00:27:15.534188+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:27:15.534188+0000","last_undegraded":"2024-09-16T00:27:15.534188+0000","last_fullsized":"2024-09-16T00:27:15.534188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969976+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:26:29.969976+0000","last_peered":"2024-09-16T00:26:29.969976+0000","last_clean":"2024-09-16T00:26:29.969976+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:26:29.969976+0000","last_undegraded":"2024-09-16T00:26:29.969976+0000","last_fullsized":"2024-09-16T00:26:29.969976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552308+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:26:29.552308+0000","last_peered":"2024-09-16T00:26:29.552308+0000","last_clean":"2024-09-16T00:26:29.552308+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:26:29.552308+0000","last_undegraded":"2024-09-16T00:26:29.552308+0000","last_fullsized":"2024-09-16T00:26:29.552308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969641+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:26:29.969641+0000","last_peered":"2024-09-16T00:26:29.969641+0000","last_clean":"2024-09-16T00:26:29.969641+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:26:29.969641+0000","last_undegraded":"2024-09-16T00:26:29.969641+0000","last_fullsized":"2024-09-16T00:26:29.969641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970322+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:26:29.970322+0000","last_peered":"2024-09-16T00:26:29.970322+0000","last_clean":"2024-09-16T00:26:29.970322+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:26:29.970322+0000","last_undegraded":"2024-09-16T00:26:29.970322+0000","last_fullsized":"2024-09-16T00:26:29.970322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971529+0000","last_change":"2024-09-16T00:26:25.300517+0000","last_active":"2024-09-16T00:26:29.971529+0000","last_peered":"2024-09-16T00:26:29.971529+0000","last_clean":"2024-09-16T00:26:29.971529+0000","last_became_active":"2024-09-16T00:26:25.300104+0000","last_became_peered":"2024-09-16T00:26:25.300104+0000","last_unstale":"2024-09-16T00:26:29.971529+0000","last_undegraded":"2024-09-16T00:26:29.971529+0000","last_fullsized":"2024-09-16T00:26:29.971529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:13.735689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969412+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:26:29.969412+0000","last_peered":"2024-09-16T00:26:29.969412+0000","last_clean":"2024-09-16T00:26:29.969412+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:26:29.969412+0000","last_undegraded":"2024-09-16T00:26:29.969412+0000","last_fullsized":"2024-09-16T00:26:29.969412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.607602+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:01.607602+0000","last_peered":"2024-09-16T00:27:01.607602+0000","last_clean":"2024-09-16T00:27:01.607602+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:01.607602+0000","last_undegraded":"2024-09-16T00:27:01.607602+0000","last_fullsized":"2024-09-16T00:27:01.607602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:01.608703+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:01.608703+0000","last_peered":"2024-09-16T00:27:01.608703+0000","last_clean":"2024-09-16T00:27:01.608703+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:01.608703+0000","last_undegraded":"2024-09-16T00:27:01.608703+0000","last_fullsized":"2024-09-16T00:27:01.608703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970603+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:26:29.970603+0000","last_peered":"2024-09-16T00:26:29.970603+0000","last_clean":"2024-09-16T00:26:29.970603+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:26:29.970603+0000","last_undegraded":"2024-09-16T00:26:29.970603+0000","last_fullsized":"2024-09-16T00:26:29.970603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290929+0000","last_change":"2024-09-16T00:26:25.314985+0000","last_active":"2024-09-16T00:26:29.290929+0000","last_peered":"2024-09-16T00:26:29.290929+0000","last_clean":"2024-09-16T00:26:29.290929+0000","last_became_active":"2024-09-16T00:26:25.314507+0000","last_became_peered":"2024-09-16T00:26:25.314507+0000","last_unstale":"2024-09-16T00:26:29.290929+0000","last_undegraded":"2024-09-16T00:26:29.290929+0000","last_fullsized":"2024-09-16T00:26:29.290929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:53:15.774916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:27:18.539704+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:27:18.539704+0000","last_peered":"2024-09-16T00:27:18.539704+0000","last_clean":"2024-09-16T00:27:18.539704+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:27:18.539704+0000","last_undegraded":"2024-09-16T00:27:18.539704+0000","last_fullsized":"2024-09-16T00:27:18.539704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971196+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:26:29.971196+0000","last_peered":"2024-09-16T00:26:29.971196+0000","last_clean":"2024-09-16T00:26:29.971196+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:26:29.971196+0000","last_undegraded":"2024-09-16T00:26:29.971196+0000","last_fullsized":"2024-09-16T00:26:29.971196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.291234+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:26:29.291234+0000","last_peered":"2024-09-16T00:26:29.291234+0000","last_clean":"2024-09-16T00:26:29.291234+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:26:29.291234+0000","last_undegraded":"2024-09-16T00:26:29.291234+0000","last_fullsized":"2024-09-16T00:26:29.291234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970095+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:26:29.970095+0000","last_peered":"2024-09-16T00:26:29.970095+0000","last_clean":"2024-09-16T00:26:29.970095+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:26:29.970095+0000","last_undegraded":"2024-09-16T00:26:29.970095+0000","last_fullsized":"2024-09-16T00:26:29.970095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.291077+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:26:29.291077+0000","last_peered":"2024-09-16T00:26:29.291077+0000","last_clean":"2024-09-16T00:26:29.291077+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:26:29.291077+0000","last_undegraded":"2024-09-16T00:26:29.291077+0000","last_fullsized":"2024-09-16T00:26:29.291077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290707+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:26:29.290707+0000","last_peered":"2024-09-16T00:26:29.290707+0000","last_clean":"2024-09-16T00:26:29.290707+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:26:29.290707+0000","last_undegraded":"2024-09-16T00:26:29.290707+0000","last_fullsized":"2024-09-16T00:26:29.290707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552252+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:26:29.552252+0000","last_peered":"2024-09-16T00:26:29.552252+0000","last_clean":"2024-09-16T00:26:29.552252+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:26:29.552252+0000","last_undegraded":"2024-09-16T00:26:29.552252+0000","last_fullsized":"2024-09-16T00:26:29.552252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289481+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:26:29.289481+0000","last_peered":"2024-09-16T00:26:29.289481+0000","last_clean":"2024-09-16T00:26:29.289481+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:26:29.289481+0000","last_undegraded":"2024-09-16T00:26:29.289481+0000","last_fullsized":"2024-09-16T00:26:29.289481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971411+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:26:29.971411+0000","last_peered":"2024-09-16T00:26:29.971411+0000","last_clean":"2024-09-16T00:26:29.971411+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:26:29.971411+0000","last_undegraded":"2024-09-16T00:26:29.971411+0000","last_fullsized":"2024-09-16T00:26:29.971411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.291359+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:26:29.291359+0000","last_peered":"2024-09-16T00:26:29.291359+0000","last_clean":"2024-09-16T00:26:29.291359+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:26:29.291359+0000","last_undegraded":"2024-09-16T00:26:29.291359+0000","last_fullsized":"2024-09-16T00:26:29.291359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969854+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:26:29.969854+0000","last_peered":"2024-09-16T00:26:29.969854+0000","last_clean":"2024-09-16T00:26:29.969854+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:26:29.969854+0000","last_undegraded":"2024-09-16T00:26:29.969854+0000","last_fullsized":"2024-09-16T00:26:29.969854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969867+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:26:29.969867+0000","last_peered":"2024-09-16T00:26:29.969867+0000","last_clean":"2024-09-16T00:26:29.969867+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:26:29.969867+0000","last_undegraded":"2024-09-16T00:26:29.969867+0000","last_fullsized":"2024-09-16T00:26:29.969867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970139+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:26:29.970139+0000","last_peered":"2024-09-16T00:26:29.970139+0000","last_clean":"2024-09-16T00:26:29.970139+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:26:29.970139+0000","last_undegraded":"2024-09-16T00:26:29.970139+0000","last_fullsized":"2024-09-16T00:26:29.970139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970875+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:26:29.970875+0000","last_peered":"2024-09-16T00:26:29.970875+0000","last_clean":"2024-09-16T00:26:29.970875+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:26:29.970875+0000","last_undegraded":"2024-09-16T00:26:29.970875+0000","last_fullsized":"2024-09-16T00:26:29.970875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289401+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:26:29.289401+0000","last_peered":"2024-09-16T00:26:29.289401+0000","last_clean":"2024-09-16T00:26:29.289401+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:26:29.289401+0000","last_undegraded":"2024-09-16T00:26:29.289401+0000","last_fullsized":"2024-09-16T00:26:29.289401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971221+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:26:29.971221+0000","last_peered":"2024-09-16T00:26:29.971221+0000","last_clean":"2024-09-16T00:26:29.971221+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:26:29.971221+0000","last_undegraded":"2024-09-16T00:26:29.971221+0000","last_fullsized":"2024-09-16T00:26:29.971221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.291307+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:26:29.291307+0000","last_peered":"2024-09-16T00:26:29.291307+0000","last_clean":"2024-09-16T00:26:29.291307+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:26:29.291307+0000","last_undegraded":"2024-09-16T00:26:29.291307+0000","last_fullsized":"2024-09-16T00:26:29.291307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.970678+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:26:29.970678+0000","last_peered":"2024-09-16T00:26:29.970678+0000","last_clean":"2024-09-16T00:26:29.970678+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:26:29.970678+0000","last_undegraded":"2024-09-16T00:26:29.970678+0000","last_fullsized":"2024-09-16T00:26:29.970678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971627+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:26:29.971627+0000","last_peered":"2024-09-16T00:26:29.971627+0000","last_clean":"2024-09-16T00:26:29.971627+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:26:29.971627+0000","last_undegraded":"2024-09-16T00:26:29.971627+0000","last_fullsized":"2024-09-16T00:26:29.971627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969682+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:26:29.969682+0000","last_peered":"2024-09-16T00:26:29.969682+0000","last_clean":"2024-09-16T00:26:29.969682+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:26:29.969682+0000","last_undegraded":"2024-09-16T00:26:29.969682+0000","last_fullsized":"2024-09-16T00:26:29.969682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552125+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:26:29.552125+0000","last_peered":"2024-09-16T00:26:29.552125+0000","last_clean":"2024-09-16T00:26:29.552125+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:26:29.552125+0000","last_undegraded":"2024-09-16T00:26:29.552125+0000","last_fullsized":"2024-09-16T00:26:29.552125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552076+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:26:29.552076+0000","last_peered":"2024-09-16T00:26:29.552076+0000","last_clean":"2024-09-16T00:26:29.552076+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:26:29.552076+0000","last_undegraded":"2024-09-16T00:26:29.552076+0000","last_fullsized":"2024-09-16T00:26:29.552076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289339+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:26:29.289339+0000","last_peered":"2024-09-16T00:26:29.289339+0000","last_clean":"2024-09-16T00:26:29.289339+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:26:29.289339+0000","last_undegraded":"2024-09-16T00:26:29.289339+0000","last_fullsized":"2024-09-16T00:26:29.289339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971857+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:26:29.971857+0000","last_peered":"2024-09-16T00:26:29.971857+0000","last_clean":"2024-09-16T00:26:29.971857+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:26:29.971857+0000","last_undegraded":"2024-09-16T00:26:29.971857+0000","last_fullsized":"2024-09-16T00:26:29.971857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971136+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:26:29.971136+0000","last_peered":"2024-09-16T00:26:29.971136+0000","last_clean":"2024-09-16T00:26:29.971136+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:26:29.971136+0000","last_undegraded":"2024-09-16T00:26:29.971136+0000","last_fullsized":"2024-09-16T00:26:29.971136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.290164+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:26:29.290164+0000","last_peered":"2024-09-16T00:26:29.290164+0000","last_clean":"2024-09-16T00:26:29.290164+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:26:29.290164+0000","last_undegraded":"2024-09-16T00:26:29.290164+0000","last_fullsized":"2024-09-16T00:26:29.290164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971446+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:26:29.971446+0000","last_peered":"2024-09-16T00:26:29.971446+0000","last_clean":"2024-09-16T00:26:29.971446+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:26:29.971446+0000","last_undegraded":"2024-09-16T00:26:29.971446+0000","last_fullsized":"2024-09-16T00:26:29.971446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969650+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:26:29.969650+0000","last_peered":"2024-09-16T00:26:29.969650+0000","last_clean":"2024-09-16T00:26:29.969650+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:26:29.969650+0000","last_undegraded":"2024-09-16T00:26:29.969650+0000","last_fullsized":"2024-09-16T00:26:29.969650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.969984+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:26:29.969984+0000","last_peered":"2024-09-16T00:26:29.969984+0000","last_clean":"2024-09-16T00:26:29.969984+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:26:29.969984+0000","last_undegraded":"2024-09-16T00:26:29.969984+0000","last_fullsized":"2024-09-16T00:26:29.969984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.289715+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:26:29.289715+0000","last_peered":"2024-09-16T00:26:29.289715+0000","last_clean":"2024-09-16T00:26:29.289715+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:26:29.289715+0000","last_undegraded":"2024-09-16T00:26:29.289715+0000","last_fullsized":"2024-09-16T00:26:29.289715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971208+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:26:29.971208+0000","last_peered":"2024-09-16T00:26:29.971208+0000","last_clean":"2024-09-16T00:26:29.971208+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:26:29.971208+0000","last_undegraded":"2024-09-16T00:26:29.971208+0000","last_fullsized":"2024-09-16T00:26:29.971208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971777+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:26:29.971777+0000","last_peered":"2024-09-16T00:26:29.971777+0000","last_clean":"2024-09-16T00:26:29.971777+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:26:29.971777+0000","last_undegraded":"2024-09-16T00:26:29.971777+0000","last_fullsized":"2024-09-16T00:26:29.971777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552116+0000","last_change":"2024-09-16T00:26:18.274217+0000","last_active":"2024-09-16T00:26:29.552116+0000","last_peered":"2024-09-16T00:26:29.552116+0000","last_clean":"2024-09-16T00:26:29.552116+0000","last_became_active":"2024-09-16T00:26:18.273929+0000","last_became_peered":"2024-09-16T00:26:18.273929+0000","last_unstale":"2024-09-16T00:26:29.552116+0000","last_undegraded":"2024-09-16T00:26:29.552116+0000","last_fullsized":"2024-09-16T00:26:29.552116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:31.633003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.552254+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:26:29.552254+0000","last_peered":"2024-09-16T00:26:29.552254+0000","last_clean":"2024-09-16T00:26:29.552254+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:26:29.552254+0000","last_undegraded":"2024-09-16T00:26:29.552254+0000","last_fullsized":"2024-09-16T00:26:29.552254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:26:29.971600+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:26:29.971600+0000","last_peered":"2024-09-16T00:26:29.971600+0000","last_clean":"2024-09-16T00:26:29.971600+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:26:29.971600+0000","last_undegraded":"2024-09-16T00:26:29.971600+0000","last_fullsized":"2024-09-16T00:26:29.971600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53101,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":120,"num_read_kb":114,"num_write":154,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":175623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514521,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":495845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743454,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1132211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":496001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36920,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708232,"statfs":{"total":95995035648,"available":95957229568,"internally_reserved":0,"allocated":1613824,"data_stored":1091912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":491257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790007,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712384,"statfs":{"total":95995035648,"available":95961481216,"internally_reserved":0,"allocated":1622016,"data_stored":1087500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":553165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":491451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":492910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":548425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":492645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":57902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":50676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:27:21.193 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:27:21.194 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T00:27:21.194 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T00:27:21.457 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.978+0000 7f7465ec3640 1 -- 172.21.15.1:0/3738732923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460102e00 msgr2=0x7f7460103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.978+0000 7f7465ec3640 1 --2- 172.21.15.1:0/3738732923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460102e00 0x7f7460103280 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7448009900 tx=0x7f744802a410 comp rx=0 tx=0).stop 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.978+0000 7f7465ec3640 1 -- 172.21.15.1:0/3738732923 shutdown_connections 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.978+0000 7f7465ec3640 1 --2- 172.21.15.1:0/3738732923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74601037c0 0x7f746010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.978+0000 7f7465ec3640 1 --2- 172.21.15.1:0/3738732923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460102e00 0x7f7460103280 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.978+0000 7f7465ec3640 1 --2- 172.21.15.1:0/3738732923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7460101c00 0x7f7460102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.978+0000 7f7465ec3640 1 -- 172.21.15.1:0/3738732923 >> 172.21.15.1:0/3738732923 conn(0x7f74600fd390 msgr2=0x7f74600ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.978+0000 7f7465ec3640 1 -- 172.21.15.1:0/3738732923 shutdown_connections 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.978+0000 7f7465ec3640 1 -- 172.21.15.1:0/3738732923 wait complete. 2024-09-16T00:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f7465ec3640 1 Processor -- start 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f7465ec3640 1 -- start start 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f7465ec3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460101c00 0x7f746019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f7465ec3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7460102e00 0x7f746019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f7465ec3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601037c0 0x7f74601a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f7465ec3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f746010d880 con 0x7f7460101c00 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f7465ec3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f746010d700 con 0x7f7460102e00 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f7465ec3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f746010da00 con 0x7f74601037c0 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460101c00 0x7f746019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460101c00 0x7f746019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45668/0 (socket says 172.21.15.1:45668) 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745f7fe640 1 -- 172.21.15.1:0/1851500413 learned_addr learned my addr 172.21.15.1:0/1851500413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745effd640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7460102e00 0x7f746019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745ffff640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601037c0 0x7f74601a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745f7fe640 1 -- 172.21.15.1:0/1851500413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601037c0 msgr2=0x7f74601a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745f7fe640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601037c0 0x7f74601a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745f7fe640 1 -- 172.21.15.1:0/1851500413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7460102e00 msgr2=0x7f746019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745f7fe640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7460102e00 0x7f746019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.979+0000 7f745f7fe640 1 -- 172.21.15.1:0/1851500413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74601a1d90 con 0x7f7460101c00 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.980+0000 7f745f7fe640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460101c00 0x7f746019a160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f745000e9c0 tx=0x7f745000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.980+0000 7f745cff9640 1 -- 172.21.15.1:0/1851500413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745000cd80 con 0x7f7460101c00 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.980+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74601a2080 con 0x7f7460101c00 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.980+0000 7f745cff9640 1 -- 172.21.15.1:0/1851500413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74500035f0 con 0x7f7460101c00 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.980+0000 7f745cff9640 1 -- 172.21.15.1:0/1851500413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450010690 con 0x7f7460101c00 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.980+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74601a2540 con 0x7f7460101c00 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.981+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7460102000 con 0x7f7460101c00 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.982+0000 7f745cff9640 1 -- 172.21.15.1:0/1851500413 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f74500037a0 con 0x7f7460101c00 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.983+0000 7f745cff9640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f744c0a3300 0x7f744c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.983+0000 7f745cff9640 1 -- 172.21.15.1:0/1851500413 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f74500d01c0 con 0x7f7460101c00 2024-09-16T00:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.983+0000 7f745effd640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f744c0a3300 0x7f744c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.983+0000 7f745effd640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f744c0a3300 0x7f744c0a57c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f74480314e0 tx=0x7f7448000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:21.987+0000 7f745cff9640 1 -- 172.21.15.1:0/1851500413 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f745009ca40 con 0x7f7460101c00 2024-09-16T00:27:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.232+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f74601a2b80 con 0x7f7460101c00 2024-09-16T00:27:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.234+0000 7f745cff9640 1 -- 172.21.15.1:0/1851500413 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f745009c340 con 0x7f7460101c00 2024-09-16T00:27:22.235 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-09-16T00:27:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQBqe+dm51X1DRAACn1pPSd/OFvy4te4AV5vsw== 2024-09-16T00:27:22.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f744c0a3300 msgr2=0x7f744c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:22.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f744c0a3300 0x7f744c0a57c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f74480314e0 tx=0x7f7448000bc0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460101c00 msgr2=0x7f746019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460101c00 0x7f746019a160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f745000e9c0 tx=0x7f745000ee80 comp rx=0 tx=0).stop 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 shutdown_connections 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f744c0a3300 0x7f744c0a57c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601037c0 0x7f74601a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7460102e00 0x7f746019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 --2- 172.21.15.1:0/1851500413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460101c00 0x7f746019a160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 >> 172.21.15.1:0/1851500413 conn(0x7f74600fd390 msgr2=0x7f74601085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 shutdown_connections 2024-09-16T00:27:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:22.236+0000 7f7465ec3640 1 -- 172.21.15.1:0/1851500413 wait complete. 2024-09-16T00:27:22.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:22 smithi001 ceph-mon[30934]: pgmap v317: 129 pgs: 129 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-16T00:27:22.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:22 smithi001 ceph-mon[30934]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:27:22.464 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:22 smithi153 ceph-mon[35538]: pgmap v317: 129 pgs: 129 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-16T00:27:22.465 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:22 smithi153 ceph-mon[35538]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:27:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:22 smithi019 ceph-mon[35186]: pgmap v317: 129 pgs: 129 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-16T00:27:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:22 smithi019 ceph-mon[35186]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:27:22.978 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:27:22.979 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T00:27:22.979 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T00:27:23.021 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T00:27:23.021 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T00:27:23.021 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T00:27:23.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:23 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1851500413' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:27:23.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:23 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1851500413' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:27:23.321 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:23.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:23 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1851500413' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:27:23.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:23 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1851500413' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:27:23.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:23 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1851500413' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:27:23.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:23 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1851500413' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:27:23.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.945+0000 7fc9f94df640 1 -- 172.21.15.1:0/1285377093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4106a80 msgr2=0x7fc9f4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:23.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.945+0000 7fc9f94df640 1 --2- 172.21.15.1:0/1285377093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4106a80 0x7fc9f4106e60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009930 tx=0x7fc9e402a420 comp rx=0 tx=0).stop 2024-09-16T00:27:23.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.945+0000 7fc9f94df640 1 -- 172.21.15.1:0/1285377093 shutdown_connections 2024-09-16T00:27:23.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.945+0000 7fc9f94df640 1 --2- 172.21.15.1:0/1285377093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f4101420 0x7fc9f410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:23.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.945+0000 7fc9f94df640 1 --2- 172.21.15.1:0/1285377093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9f4100a80 0x7fc9f4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:23.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.945+0000 7fc9f94df640 1 --2- 172.21.15.1:0/1285377093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4106a80 0x7fc9f4106e60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:23.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.945+0000 7fc9f94df640 1 -- 172.21.15.1:0/1285377093 >> 172.21.15.1:0/1285377093 conn(0x7fc9f40fc7a0 msgr2=0x7fc9f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:23.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.945+0000 7fc9f94df640 1 -- 172.21.15.1:0/1285377093 shutdown_connections 2024-09-16T00:27:23.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.945+0000 7fc9f94df640 1 -- 172.21.15.1:0/1285377093 wait complete. 2024-09-16T00:27:23.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f94df640 1 Processor -- start 2024-09-16T00:27:23.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f94df640 1 -- start start 2024-09-16T00:27:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f94df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4100a80 0x7fc9f419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f94df640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f4101420 0x7fc9f419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f94df640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9f4106a80 0x7fc9f419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f94df640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4110ea0 con 0x7fc9f4100a80 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f94df640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4110d20 con 0x7fc9f4101420 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f94df640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4111020 con 0x7fc9f4106a80 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4100a80 0x7fc9f419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4100a80 0x7fc9f419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45698/0 (socket says 172.21.15.1:45698) 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9f2ffd640 1 -- 172.21.15.1:0/3031717759 learned_addr learned my addr 172.21.15.1:0/3031717759 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.946+0000 7fc9ea7fc640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f4101420 0x7fc9f419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f37fe640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9f4106a80 0x7fc9f419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f2ffd640 1 -- 172.21.15.1:0/3031717759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9f4106a80 msgr2=0x7fc9f419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f2ffd640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9f4106a80 0x7fc9f419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f2ffd640 1 -- 172.21.15.1:0/3031717759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f4101420 msgr2=0x7fc9f419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f2ffd640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f4101420 0x7fc9f419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:23.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f2ffd640 1 -- 172.21.15.1:0/3031717759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f419f660 con 0x7fc9f4100a80 2024-09-16T00:27:23.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f37fe640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9f4106a80 0x7fc9f419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:23.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f2ffd640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4100a80 0x7fc9f419a6b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009230 tx=0x7fc9e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:23.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f0ff9640 1 -- 172.21.15.1:0/3031717759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e403f030 con 0x7fc9f4100a80 2024-09-16T00:27:23.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f419f8f0 con 0x7fc9f4100a80 2024-09-16T00:27:23.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f0ff9640 1 -- 172.21.15.1:0/3031717759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e4003ad0 con 0x7fc9f4100a80 2024-09-16T00:27:23.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f0ff9640 1 -- 172.21.15.1:0/3031717759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4035070 con 0x7fc9f4100a80 2024-09-16T00:27:23.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.947+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f41a7230 con 0x7fc9f4100a80 2024-09-16T00:27:23.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.948+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f4100ee0 con 0x7fc9f4100a80 2024-09-16T00:27:23.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.949+0000 7fc9f0ff9640 1 -- 172.21.15.1:0/3031717759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fc9e4042030 con 0x7fc9f4100a80 2024-09-16T00:27:23.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.950+0000 7fc9f0ff9640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9e00a32d0 0x7fc9e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:23.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.950+0000 7fc9f0ff9640 1 -- 172.21.15.1:0/3031717759 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7fc9e40f02c0 con 0x7fc9f4100a80 2024-09-16T00:27:23.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.950+0000 7fc9ea7fc640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9e00a32d0 0x7fc9e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:23.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.951+0000 7fc9ea7fc640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9e00a32d0 0x7fc9e00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc9f410d740 tx=0x7fc9dc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:23.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:23.954+0000 7fc9f0ff9640 1 -- 172.21.15.1:0/3031717759 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9e40bcb40 con 0x7fc9f4100a80 2024-09-16T00:27:24.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.175+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc9f41a7400 con 0x7fc9f4100a80 2024-09-16T00:27:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.179+0000 7fc9f0ff9640 1 -- 172.21.15.1:0/3031717759 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264040 (secure 0 0 0) 0x7fc9e40bc440 con 0x7fc9f4100a80 2024-09-16T00:27:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:27:24.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9e00a32d0 msgr2=0x7fc9e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9e00a32d0 0x7fc9e00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc9f410d740 tx=0x7fc9dc009110 comp rx=0 tx=0).stop 2024-09-16T00:27:24.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4100a80 msgr2=0x7fc9f419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4100a80 0x7fc9f419a6b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009230 tx=0x7fc9e4004270 comp rx=0 tx=0).stop 2024-09-16T00:27:24.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 shutdown_connections 2024-09-16T00:27:24.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9e00a32d0 0x7fc9e00a5790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9f4106a80 0x7fc9f419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f4101420 0x7fc9f419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 --2- 172.21.15.1:0/3031717759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9f4100a80 0x7fc9f419a6b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 >> 172.21.15.1:0/3031717759 conn(0x7fc9f40fc7a0 msgr2=0x7fc9f406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:24.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 shutdown_connections 2024-09-16T00:27:24.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:24.182+0000 7fc9f94df640 1 -- 172.21.15.1:0/3031717759 wait complete. 2024-09-16T00:27:24.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:24 smithi001 ceph-mon[30934]: pgmap v318: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-16T00:27:24.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:24 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3031717759' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:27:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:24 smithi019 ceph-mon[35186]: pgmap v318: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-16T00:27:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:24 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3031717759' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:27:24.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:24 smithi153 ceph-mon[35538]: pgmap v318: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-16T00:27:24.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:24 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3031717759' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:27:24.943 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":3146869121},{"type":"v1","addr":"172.21.15.1:6801","nonce":3146869121}]},"active_addr":"172.21.15.1:6801/3146869121","active_change":"2024-09-16T00:19:24.226129+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.1:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2593060091}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":4268092099}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1102762229}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1469515748}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":910297243}]}]} 2024-09-16T00:27:24.948 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T00:27:24.948 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T00:27:24.949 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:27:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:25.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-09-16T00:27:25.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-16T00:27:25.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]: dispatch 2024-09-16T00:27:25.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-09-16T00:27:25.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2024-09-16T00:27:25.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-16T00:27:25.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-16T00:27:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-09-16T00:27:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-16T00:27:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]: dispatch 2024-09-16T00:27:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-09-16T00:27:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2024-09-16T00:27:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-16T00:27:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-16T00:27:25.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-09-16T00:27:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-16T00:27:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]: dispatch 2024-09-16T00:27:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-09-16T00:27:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2024-09-16T00:27:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-16T00:27:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-16T00:27:25.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.720+0000 7f5153a28640 1 -- 172.21.15.1:0/2985923259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 msgr2=0x7f514c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:25.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.720+0000 7f5153a28640 1 --2- 172.21.15.1:0/2985923259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 0x7f514c10e600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5144009ca0 tx=0x7f514402a950 comp rx=0 tx=0).stop 2024-09-16T00:27:25.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.720+0000 7f5153a28640 1 -- 172.21.15.1:0/2985923259 shutdown_connections 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.720+0000 7f5153a28640 1 --2- 172.21.15.1:0/2985923259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 0x7f514c10e600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.720+0000 7f5153a28640 1 --2- 172.21.15.1:0/2985923259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f514c1036c0 0x7f514c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.720+0000 7f5153a28640 1 --2- 172.21.15.1:0/2985923259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f514c102da0 0x7f514c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.720+0000 7f5153a28640 1 -- 172.21.15.1:0/2985923259 >> 172.21.15.1:0/2985923259 conn(0x7f514c0fc7a0 msgr2=0x7f514c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.720+0000 7f5153a28640 1 -- 172.21.15.1:0/2985923259 shutdown_connections 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.720+0000 7f5153a28640 1 -- 172.21.15.1:0/2985923259 wait complete. 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5153a28640 1 Processor -- start 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5153a28640 1 -- start start 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5153a28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f514c102da0 0x7f514c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5153a28640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f514c1036c0 0x7f514c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5153a28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 0x7f514c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5153a28640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f514c111f60 con 0x7f514c10c180 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5153a28640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f514c111de0 con 0x7f514c102da0 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5153a28640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f514c1120e0 con 0x7f514c1036c0 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5151fc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 0x7f514c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5151fc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 0x7f514c19ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45718/0 (socket says 172.21.15.1:45718) 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5151fc5640 1 -- 172.21.15.1:0/1358727329 learned_addr learned my addr 172.21.15.1:0/1358727329 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f5150fc3640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f514c1036c0 0x7f514c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.721+0000 7f51517c4640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f514c102da0 0x7f514c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f5151fc5640 1 -- 172.21.15.1:0/1358727329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f514c1036c0 msgr2=0x7f514c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f5151fc5640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f514c1036c0 0x7f514c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f5151fc5640 1 -- 172.21.15.1:0/1358727329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f514c102da0 msgr2=0x7f514c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f5151fc5640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f514c102da0 0x7f514c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f5151fc5640 1 -- 172.21.15.1:0/1358727329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c19f550 con 0x7f514c10c180 2024-09-16T00:27:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f5150fc3640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f514c1036c0 0x7f514c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f5151fc5640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 0x7f514c19ee70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f514402ae70 tx=0x7f514402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f51427fc640 1 -- 172.21.15.1:0/1358727329 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144034040 con 0x7f514c10c180 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f51427fc640 1 -- 172.21.15.1:0/1358727329 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5144030920 con 0x7f514c10c180 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c19f7e0 con 0x7f514c10c180 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f51427fc640 1 -- 172.21.15.1:0/1358727329 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144045480 con 0x7f514c10c180 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.722+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c1a7120 con 0x7f514c10c180 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.723+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514c103180 con 0x7f514c10c180 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.724+0000 7f51427fc640 1 -- 172.21.15.1:0/1358727329 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f5144037020 con 0x7f514c10c180 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.725+0000 7f51427fc640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f51380a3350 0x7f51380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.725+0000 7f51517c4640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f51380a3350 0x7f51380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.725+0000 7f51427fc640 1 -- 172.21.15.1:0/1358727329 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9253+0+0 (secure 0 0 0) 0x7f51440f0db0 con 0x7f514c10c180 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.726+0000 7f51517c4640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f51380a3350 0x7f51380a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51480058c0 tx=0x7f514800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.730+0000 7f51427fc640 1 -- 172.21.15.1:0/1358727329 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51440bd680 con 0x7f514c10c180 2024-09-16T00:27:25.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.906+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f514c1a72f0 con 0x7f514c10c180 2024-09-16T00:27:25.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.908+0000 7f51427fc640 1 -- 172.21.15.1:0/1358727329 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24902 (secure 0 0 0) 0x7f51440bcf80 con 0x7f514c10c180 2024-09-16T00:27:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:27:25.910 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":90,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:27:25.238771+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:27:25.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f51380a3350 msgr2=0x7f51380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f51380a3350 0x7f51380a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51480058c0 tx=0x7f514800a4c0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 msgr2=0x7f514c19ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 0x7f514c19ee70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f514402ae70 tx=0x7f514402f5b0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 shutdown_connections 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c10c180 0x7f514c19ee70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f51380a3350 0x7f51380a5810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f514c1036c0 0x7f514c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 --2- 172.21.15.1:0/1358727329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f514c102da0 0x7f514c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 >> 172.21.15.1:0/1358727329 conn(0x7f514c0fc7a0 msgr2=0x7f514c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 shutdown_connections 2024-09-16T00:27:25.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:25.910+0000 7f5153a28640 1 -- 172.21.15.1:0/1358727329 wait complete. 2024-09-16T00:27:26.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: pgmap v319: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T00:27:26.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-09-16T00:27:26.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-16T00:27:26.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]': finished 2024-09-16T00:27:26.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-09-16T00:27:26.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2024-09-16T00:27:26.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-16T00:27:26.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-16T00:27:26.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:27:26.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:26 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1358727329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:27:26.586 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T00:27:26.586 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:27:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: pgmap v319: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T00:27:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-09-16T00:27:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-16T00:27:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]': finished 2024-09-16T00:27:26.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-09-16T00:27:26.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2024-09-16T00:27:26.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-16T00:27:26.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-16T00:27:26.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:27:26.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:26 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1358727329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:27:26.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: pgmap v319: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T00:27:26.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-09-16T00:27:26.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-16T00:27:26.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]': finished 2024-09-16T00:27:26.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-09-16T00:27:26.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2024-09-16T00:27:26.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-16T00:27:26.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-16T00:27:26.714 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:27:26.714 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:26 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1358727329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:27:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.454+0000 7f4fdce44640 1 -- 172.21.15.1:0/4184101105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd810c170 msgr2=0x7f4fd810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.454+0000 7f4fdce44640 1 --2- 172.21.15.1:0/4184101105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd810c170 0x7f4fd810e610 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009ca0 tx=0x7f4fd002a950 comp rx=0 tx=0).stop 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.454+0000 7f4fdce44640 1 -- 172.21.15.1:0/4184101105 shutdown_connections 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.454+0000 7f4fdce44640 1 --2- 172.21.15.1:0/4184101105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd810c170 0x7f4fd810e610 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.454+0000 7f4fdce44640 1 --2- 172.21.15.1:0/4184101105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd8103660 0x7f4fd810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.454+0000 7f4fdce44640 1 --2- 172.21.15.1:0/4184101105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd8102d40 0x7f4fd8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.454+0000 7f4fdce44640 1 -- 172.21.15.1:0/4184101105 >> 172.21.15.1:0/4184101105 conn(0x7f4fd80fc7a0 msgr2=0x7f4fd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.454+0000 7f4fdce44640 1 -- 172.21.15.1:0/4184101105 shutdown_connections 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.454+0000 7f4fdce44640 1 -- 172.21.15.1:0/4184101105 wait complete. 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fdce44640 1 Processor -- start 2024-09-16T00:27:27.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fdce44640 1 -- start start 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fdce44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd8102d40 0x7f4fd819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fdce44640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd8103660 0x7f4fd819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fdce44640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd810c170 0x7f4fd819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fdce44640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8111f50 con 0x7f4fd8102d40 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fdce44640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8111dd0 con 0x7f4fd810c170 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fdce44640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81120d0 con 0x7f4fd8103660 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd8102d40 0x7f4fd819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd8102d40 0x7f4fd819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45734/0 (socket says 172.21.15.1:45734) 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fd659c640 1 -- 172.21.15.1:0/3211082860 learned_addr learned my addr 172.21.15.1:0/3211082860 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:27.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fd6d9d640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd810c170 0x7f4fd819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fd659c640 1 -- 172.21.15.1:0/3211082860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd8103660 msgr2=0x7f4fd819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fd5d9b640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd8103660 0x7f4fd819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.455+0000 7f4fd659c640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd8103660 0x7f4fd819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.456+0000 7f4fd659c640 1 -- 172.21.15.1:0/3211082860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd810c170 msgr2=0x7f4fd819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.456+0000 7f4fd659c640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd810c170 0x7f4fd819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.456+0000 7f4fd659c640 1 -- 172.21.15.1:0/3211082860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd819f590 con 0x7f4fd8102d40 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.456+0000 7f4fd659c640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd8102d40 0x7f4fd819a5e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800d950 tx=0x7f4fc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.456+0000 7f4fc77fe640 1 -- 172.21.15.1:0/3211082860 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8017030 con 0x7f4fd8102d40 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.456+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd819f880 con 0x7f4fd8102d40 2024-09-16T00:27:27.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.456+0000 7f4fc77fe640 1 -- 172.21.15.1:0/3211082860 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc8004020 con 0x7f4fd8102d40 2024-09-16T00:27:27.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.456+0000 7f4fc77fe640 1 -- 172.21.15.1:0/3211082860 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc80047a0 con 0x7f4fd8102d40 2024-09-16T00:27:27.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.456+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a7160 con 0x7f4fd8102d40 2024-09-16T00:27:27.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.457+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd8103120 con 0x7f4fd8102d40 2024-09-16T00:27:27.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.458+0000 7f4fc77fe640 1 -- 172.21.15.1:0/3211082860 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f4fc80041d0 con 0x7f4fd8102d40 2024-09-16T00:27:27.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.459+0000 7f4fc77fe640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fc00a32d0 0x7f4fc00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.459+0000 7f4fd5d9b640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fc00a32d0 0x7f4fc00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.459+0000 7f4fc77fe640 1 -- 172.21.15.1:0/3211082860 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4fc80cff20 con 0x7f4fd8102d40 2024-09-16T00:27:27.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.459+0000 7f4fd5d9b640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fc00a32d0 0x7f4fc00a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fd810eb20 tx=0x7f4fcc005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:27.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.462+0000 7f4fc77fe640 1 -- 172.21.15.1:0/3211082860 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fc809c6e0 con 0x7f4fd8102d40 2024-09-16T00:27:27.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:27 smithi019 ceph-mon[35186]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:27:27.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.638+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fd819b940 con 0x7f4fd8102d40 2024-09-16T00:27:27.640 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:27 smithi001 ceph-mon[30934]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:27:27.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.640+0000 7f4fc77fe640 1 -- 172.21.15.1:0/3211082860 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24902 (secure 0 0 0) 0x7f4fc809bfe0 con 0x7f4fd8102d40 2024-09-16T00:27:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:27:27.642 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:27:26.243912+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:27:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fc00a32d0 msgr2=0x7f4fc00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fc00a32d0 0x7f4fc00a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fd810eb20 tx=0x7f4fcc005e50 comp rx=0 tx=0).stop 2024-09-16T00:27:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd8102d40 msgr2=0x7f4fd819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd8102d40 0x7f4fd819a5e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800d950 tx=0x7f4fc800de10 comp rx=0 tx=0).stop 2024-09-16T00:27:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 shutdown_connections 2024-09-16T00:27:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd810c170 0x7f4fd819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fc00a32d0 0x7f4fc00a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd8103660 0x7f4fd819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 --2- 172.21.15.1:0/3211082860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fd8102d40 0x7f4fd819a5e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 >> 172.21.15.1:0/3211082860 conn(0x7f4fd80fc7a0 msgr2=0x7f4fd80fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:27.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 shutdown_connections 2024-09-16T00:27:27.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:27.642+0000 7f4fdce44640 1 -- 172.21.15.1:0/3211082860 wait complete. 2024-09-16T00:27:27.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:27 smithi153 ceph-mon[35538]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:27:28.375 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T00:27:28.375 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T00:27:28.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T00:27:28.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T00:27:28.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T00:27:28.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T00:27:28.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T00:27:28.378 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T00:27:28.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-16T00:27:28.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-16T00:27:28.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-16T00:27:28.380 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-16T00:27:28.593 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:28 smithi001 ceph-mon[30934]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-16T00:27:28.594 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:28 smithi001 ceph-mon[30934]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:27:28.594 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:28 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3211082860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:27:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:28 smithi019 ceph-mon[35186]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-16T00:27:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:28 smithi019 ceph-mon[35186]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:27:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:28 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3211082860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:27:28.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:28 smithi153 ceph-mon[35538]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-16T00:27:28.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:28 smithi153 ceph-mon[35538]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:27:28.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:28 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3211082860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:27:29.103 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:29.112 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:30.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.014+0000 7f08808c9640 1 -- 172.21.15.1:0/2666000494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f087c1036e0 msgr2=0x7f087c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:30.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.014+0000 7f08808c9640 1 --2- 172.21.15.1:0/2666000494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f087c1036e0 0x7f087c107b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0870009930 tx=0x7f087002a420 comp rx=0 tx=0).stop 2024-09-16T00:27:30.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.014+0000 7f08808c9640 1 -- 172.21.15.1:0/2666000494 shutdown_connections 2024-09-16T00:27:30.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.014+0000 7f08808c9640 1 --2- 172.21.15.1:0/2666000494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f087c1082a0 0x7f087c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:30.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.014+0000 7f08808c9640 1 --2- 172.21.15.1:0/2666000494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f087c1036e0 0x7f087c107b90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:30.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.014+0000 7f08808c9640 1 --2- 172.21.15.1:0/2666000494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087c102d30 0x7f087c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:30.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.014+0000 7f08808c9640 1 -- 172.21.15.1:0/2666000494 >> 172.21.15.1:0/2666000494 conn(0x7f087c0fc7d0 msgr2=0x7f087c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:30.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.014+0000 7f08808c9640 1 -- 172.21.15.1:0/2666000494 shutdown_connections 2024-09-16T00:27:30.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.014+0000 7f08808c9640 1 -- 172.21.15.1:0/2666000494 wait complete. 2024-09-16T00:27:30.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f08808c9640 1 Processor -- start 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f08808c9640 1 -- start start 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f08808c9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087c102d30 0x7f087c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f08808c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f087c1036e0 0x7f087c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f08808c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f087c1082a0 0x7f087c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f08808c9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f087c112090 con 0x7f087c1036e0 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f08808c9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f087c111f10 con 0x7f087c1082a0 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f08808c9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f087c112210 con 0x7f087c102d30 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f0879d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f087c1036e0 0x7f087c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f0879d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f087c1036e0 0x7f087c198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45764/0 (socket says 172.21.15.1:45764) 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f0879d9b640 1 -- 172.21.15.1:0/4095053580 learned_addr learned my addr 172.21.15.1:0/4095053580 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:30.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.015+0000 7f087ad9d640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f087c1082a0 0x7f087c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:30.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f087a59c640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087c102d30 0x7f087c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:30.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f0879d9b640 1 -- 172.21.15.1:0/4095053580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087c102d30 msgr2=0x7f087c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:30.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f0879d9b640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f087c102d30 0x7f087c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:30.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f0879d9b640 1 -- 172.21.15.1:0/4095053580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f087c1082a0 msgr2=0x7f087c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:30.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f0879d9b640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f087c1082a0 0x7f087c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:30.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f0879d9b640 1 -- 172.21.15.1:0/4095053580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087c19d5a0 con 0x7f087c1036e0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f0879d9b640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f087c1036e0 0x7f087c198b30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f087002a940 tx=0x7f0870003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f086b7fe640 1 -- 172.21.15.1:0/4095053580 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087003f070 con 0x7f087c1036e0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f086b7fe640 1 -- 172.21.15.1:0/4095053580 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870031c90 con 0x7f087c1036e0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f086b7fe640 1 -- 172.21.15.1:0/4095053580 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870035030 con 0x7f087c1036e0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087c19d830 con 0x7f087c1036e0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.016+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087c1a5170 con 0x7f087c1036e0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.017+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f087c062980 con 0x7f087c1036e0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.018+0000 7f086b7fe640 1 -- 172.21.15.1:0/4095053580 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f0870042070 con 0x7f087c1036e0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.019+0000 7f086b7fe640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08640a32d0 0x7f08640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.019+0000 7f086b7fe640 1 -- 172.21.15.1:0/4095053580 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f08700f0670 con 0x7f087c1036e0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.019+0000 7f087a59c640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08640a32d0 0x7f08640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.019+0000 7f086b7fe640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] conn(0x7f08640ae720 0x7f08640b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:30.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.019+0000 7f086b7fe640 1 -- 172.21.15.1:0/4095053580 --> [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f087002fec0 con 0x7f08640ae720 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.019+0000 7f086b7fe640 1 -- 172.21.15.1:0/4095053580 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f08700f7050 con 0x7f087c1036e0 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.020+0000 7f087ad9d640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] conn(0x7f08640ae720 0x7f08640b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.020+0000 7f087a59c640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08640a32d0 0x7f08640a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f086c005fd0 tx=0x7f086c009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.020+0000 7f087ad9d640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] conn(0x7f08640ae720 0x7f08640b0bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.021+0000 7f086b7fe640 1 -- 172.21.15.1:0/4095053580 <== osd.1 v2:172.21.15.19:6800/3708883557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f087002fec0 con 0x7f08640ae720 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.035+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 --> [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f087c062e00 con 0x7f08640ae720 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.035+0000 7f086b7fe640 1 -- 172.21.15.1:0/4095053580 <== osd.1 v2:172.21.15.19:6800/3708883557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f087c062e00 con 0x7f08640ae720 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 >> [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] conn(0x7f08640ae720 msgr2=0x7f08640b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] conn(0x7f08640ae720 0x7f08640b0bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08640a32d0 msgr2=0x7f08640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08640a32d0 0x7f08640a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f086c005fd0 tx=0x7f086c009870 comp rx=0 tx=0).stop 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f087c1036e0 msgr2=0x7f087c198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:30.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 --2- 172.21.15.1:0/4095053580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f087c1036e0 0x7f087c198b30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f087002a940 tx=0x7f0870003b90 comp rx=0 tx=0).stop 2024-09-16T00:27:30.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f087ad9d640 1 -- 172.21.15.1:0/4095053580 reap_dead start 2024-09-16T00:27:30.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 shutdown_connections 2024-09-16T00:27:30.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 >> 172.21.15.1:0/4095053580 conn(0x7f087c0fc7d0 msgr2=0x7f087c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:30.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 shutdown_connections 2024-09-16T00:27:30.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:30.036+0000 7f08808c9640 1 -- 172.21.15.1:0/4095053580 wait complete. 2024-09-16T00:27:30.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:30 smithi019 ceph-mon[35186]: pgmap v323: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-16T00:27:30.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:30 smithi001 ceph-mon[30934]: pgmap v323: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-16T00:27:30.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:30 smithi153 ceph-mon[35538]: pgmap v323: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-16T00:27:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:30.757 INFO:teuthology.orchestra.run.smithi001.stdout:55834574927 2024-09-16T00:27:30.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.009+0000 7f496e25d640 1 -- 172.21.15.1:0/217795767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4968106f20 msgr2=0x7f496810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.009+0000 7f496e25d640 1 --2- 172.21.15.1:0/217795767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4968106f20 0x7f496810eaa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4960009ca0 tx=0x7f496002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.009+0000 7f496e25d640 1 -- 172.21.15.1:0/217795767 shutdown_connections 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.009+0000 7f496e25d640 1 --2- 172.21.15.1:0/217795767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4968106f20 0x7f496810eaa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.009+0000 7f496e25d640 1 --2- 172.21.15.1:0/217795767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49681027a0 0x7f49681067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.009+0000 7f496e25d640 1 --2- 172.21.15.1:0/217795767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968101df0 0x7f49681021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.009+0000 7f496e25d640 1 -- 172.21.15.1:0/217795767 >> 172.21.15.1:0/217795767 conn(0x7f49680fd8b0 msgr2=0x7f49680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.009+0000 7f496e25d640 1 -- 172.21.15.1:0/217795767 shutdown_connections 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.009+0000 7f496e25d640 1 -- 172.21.15.1:0/217795767 wait complete. 2024-09-16T00:27:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f496e25d640 1 Processor -- start 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f496e25d640 1 -- start start 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f496e25d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4968101df0 0x7f49680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f496e25d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49681027a0 0x7f4968074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f496e25d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968106f20 0x7f4968074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f496e25d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4968111f80 con 0x7f4968101df0 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f496e25d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4968111e00 con 0x7f49681027a0 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f496e25d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4968112100 con 0x7f4968106f20 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f49677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4968101df0 0x7f49680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f49677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4968101df0 0x7f49680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42702/0 (socket says 172.21.15.1:42702) 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.010+0000 7f49677fe640 1 -- 172.21.15.1:0/1876643435 learned_addr learned my addr 172.21.15.1:0/1876643435 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f495ffff640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49681027a0 0x7f4968074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f49677fe640 1 -- 172.21.15.1:0/1876643435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968106f20 msgr2=0x7f4968074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f4967fff640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968106f20 0x7f4968074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f49677fe640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968106f20 0x7f4968074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f49677fe640 1 -- 172.21.15.1:0/1876643435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49681027a0 msgr2=0x7f4968074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f49677fe640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49681027a0 0x7f4968074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f49677fe640 1 -- 172.21.15.1:0/1876643435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681a6850 con 0x7f4968101df0 2024-09-16T00:27:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f495ffff640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49681027a0 0x7f4968074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f49677fe640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4968101df0 0x7f49680744f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f495800d950 tx=0x7f495800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f49657fa640 1 -- 172.21.15.1:0/1876643435 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958017070 con 0x7f4968101df0 2024-09-16T00:27:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49681a6b40 con 0x7f4968101df0 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.011+0000 7f49657fa640 1 -- 172.21.15.1:0/1876643435 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4958009c30 con 0x7f4968101df0 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.012+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681a7080 con 0x7f4968101df0 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.012+0000 7f49657fa640 1 -- 172.21.15.1:0/1876643435 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800f6b0 con 0x7f4968101df0 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.013+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4968062900 con 0x7f4968101df0 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.015+0000 7f49657fa640 1 -- 172.21.15.1:0/1876643435 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f49580109d0 con 0x7f4968101df0 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.015+0000 7f49657fa640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49540a3420 0x7f49540a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.016+0000 7f495ffff640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49540a3420 0x7f49540a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.016+0000 7f49657fa640 1 -- 172.21.15.1:0/1876643435 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f49580cfa90 con 0x7f4968101df0 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.016+0000 7f49657fa640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] conn(0x7f49540ae870 0x7f49540b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.016+0000 7f49657fa640 1 -- 172.21.15.1:0/1876643435 --> [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4958004470 con 0x7f49540ae870 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.016+0000 7f49657fa640 1 -- 172.21.15.1:0/1876643435 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f49580d42c0 con 0x7f4968101df0 2024-09-16T00:27:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.016+0000 7f495ffff640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49540a3420 0x7f49540a58e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4968075f80 tx=0x7f4950005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.016+0000 7f4967fff640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] conn(0x7f49540ae870 0x7f49540b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.017+0000 7f4967fff640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] conn(0x7f49540ae870 0x7f49540b0d10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.018+0000 7f49657fa640 1 -- 172.21.15.1:0/1876643435 <== osd.4 v2:172.21.15.19:6808/616138863 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4958004470 con 0x7f49540ae870 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.031+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 --> [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4968062d80 con 0x7f49540ae870 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.032+0000 7f49657fa640 1 -- 172.21.15.1:0/1876643435 <== osd.4 v2:172.21.15.19:6808/616138863 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4968062d80 con 0x7f49540ae870 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.032+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 >> [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] conn(0x7f49540ae870 msgr2=0x7f49540b0d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.032+0000 7f496e25d640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] conn(0x7f49540ae870 0x7f49540b0d10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.033+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49540a3420 msgr2=0x7f49540a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.033+0000 7f496e25d640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49540a3420 0x7f49540a58e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4968075f80 tx=0x7f4950005f90 comp rx=0 tx=0).stop 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.033+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4968101df0 msgr2=0x7f49680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.033+0000 7f496e25d640 1 --2- 172.21.15.1:0/1876643435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4968101df0 0x7f49680744f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f495800d950 tx=0x7f495800de10 comp rx=0 tx=0).stop 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.033+0000 7f4967fff640 1 -- 172.21.15.1:0/1876643435 reap_dead start 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.033+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 shutdown_connections 2024-09-16T00:27:31.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.033+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 >> 172.21.15.1:0/1876643435 conn(0x7f49680fd8b0 msgr2=0x7f49680fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:31.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.033+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 shutdown_connections 2024-09-16T00:27:31.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:31.033+0000 7f496e25d640 1 -- 172.21.15.1:0/1876643435 wait complete. 2024-09-16T00:27:32.046 INFO:teuthology.orchestra.run.smithi001.stdout:137438953536 2024-09-16T00:27:32.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T00:27:32.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.254+0000 7f3dbc374640 1 -- 172.21.15.1:0/1565889066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db4106f20 msgr2=0x7f3db410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.254+0000 7f3dbc374640 1 --2- 172.21.15.1:0/1565889066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db4106f20 0x7f3db410eaa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009900 tx=0x7f3dac02a410 comp rx=0 tx=0).stop 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.254+0000 7f3dbc374640 1 -- 172.21.15.1:0/1565889066 shutdown_connections 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.254+0000 7f3dbc374640 1 --2- 172.21.15.1:0/1565889066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db4106f20 0x7f3db410eaa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.254+0000 7f3dbc374640 1 --2- 172.21.15.1:0/1565889066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db41027a0 0x7f3db41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.254+0000 7f3dbc374640 1 --2- 172.21.15.1:0/1565889066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db4101df0 0x7f3db41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.254+0000 7f3dbc374640 1 -- 172.21.15.1:0/1565889066 >> 172.21.15.1:0/1565889066 conn(0x7f3db40fd8f0 msgr2=0x7f3db40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.254+0000 7f3dbc374640 1 -- 172.21.15.1:0/1565889066 shutdown_connections 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.254+0000 7f3dbc374640 1 -- 172.21.15.1:0/1565889066 wait complete. 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dbc374640 1 Processor -- start 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dbc374640 1 -- start start 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dbc374640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db4101df0 0x7f3db41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dbc374640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db41027a0 0x7f3db41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dbc374640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db4106f20 0x7f3db419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dbc374640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4111f80 con 0x7f3db4101df0 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dbc374640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4111e00 con 0x7f3db41027a0 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dbc374640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4112100 con 0x7f3db4106f20 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dba110640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db4101df0 0x7f3db41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dba110640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db4101df0 0x7f3db41983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42722/0 (socket says 172.21.15.1:42722) 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.255+0000 7f3dba110640 1 -- 172.21.15.1:0/3594853220 learned_addr learned my addr 172.21.15.1:0/3594853220 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3db990f640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db41027a0 0x7f3db41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dba911640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db4106f20 0x7f3db419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dba110640 1 -- 172.21.15.1:0/3594853220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db4106f20 msgr2=0x7f3db419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dba110640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db4106f20 0x7f3db419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dba110640 1 -- 172.21.15.1:0/3594853220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db41027a0 msgr2=0x7f3db41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dba110640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db41027a0 0x7f3db41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dba110640 1 -- 172.21.15.1:0/3594853220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db419d350 con 0x7f3db4101df0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dba911640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db4106f20 0x7f3db419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dba110640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db4101df0 0x7f3db41983a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3db000cce0 tx=0x7f3db000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dab7fe640 1 -- 172.21.15.1:0/3594853220 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0004b20 con 0x7f3db4101df0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dab7fe640 1 -- 172.21.15.1:0/3594853220 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db0004cc0 con 0x7f3db4101df0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db419d640 con 0x7f3db4101df0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dab7fe640 1 -- 172.21.15.1:0/3594853220 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db000f8c0 con 0x7f3db4101df0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.256+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db41a4f20 con 0x7f3db4101df0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.257+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3db4062980 con 0x7f3db4101df0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.258+0000 7f3dab7fe640 1 -- 172.21.15.1:0/3594853220 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f3db000fa60 con 0x7f3db4101df0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.259+0000 7f3dab7fe640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32d0 0x7f3da00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.259+0000 7f3dab7fe640 1 -- 172.21.15.1:0/3594853220 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3db00d0460 con 0x7f3db4101df0 2024-09-16T00:27:32.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.259+0000 7f3db990f640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32d0 0x7f3da00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.259+0000 7f3dab7fe640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] conn(0x7f3da00ae720 0x7f3da00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.259+0000 7f3dab7fe640 1 -- 172.21.15.1:0/3594853220 --> [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3da00b1290 con 0x7f3da00ae720 2024-09-16T00:27:32.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.259+0000 7f3dab7fe640 1 -- 172.21.15.1:0/3594853220 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f3db00d0810 con 0x7f3db4101df0 2024-09-16T00:27:32.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.260+0000 7f3db990f640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32d0 0x7f3da00a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3db40fe1a0 tx=0x7f3da400c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:32.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.260+0000 7f3dba911640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] conn(0x7f3da00ae720 0x7f3da00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.260+0000 7f3dba911640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] conn(0x7f3da00ae720 0x7f3da00b0bc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:32.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.261+0000 7f3dab7fe640 1 -- 172.21.15.1:0/3594853220 <== osd.2 v2:172.21.15.153:6800/846294555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3da00b1290 con 0x7f3da00ae720 2024-09-16T00:27:32.273 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:32 smithi001 ceph-mon[30934]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 5 op/s 2024-09-16T00:27:32.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.276+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 --> [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3db4062e00 con 0x7f3da00ae720 2024-09-16T00:27:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dab7fe640 1 -- 172.21.15.1:0/3594853220 <== osd.2 v2:172.21.15.153:6800/846294555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3db4062e00 con 0x7f3da00ae720 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 >> [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] conn(0x7f3da00ae720 msgr2=0x7f3da00b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] conn(0x7f3da00ae720 0x7f3da00b0bc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32d0 msgr2=0x7f3da00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32d0 0x7f3da00a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3db40fe1a0 tx=0x7f3da400c4a0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db4101df0 msgr2=0x7f3db41983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 --2- 172.21.15.1:0/3594853220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db4101df0 0x7f3db41983a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3db000cce0 tx=0x7f3db000a620 comp rx=0 tx=0).stop 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dba911640 1 -- 172.21.15.1:0/3594853220 reap_dead start 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 shutdown_connections 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 >> 172.21.15.1:0/3594853220 conn(0x7f3db40fd8f0 msgr2=0x7f3db40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 shutdown_connections 2024-09-16T00:27:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:32.277+0000 7f3dbc374640 1 -- 172.21.15.1:0/3594853220 wait complete. 2024-09-16T00:27:32.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:32 smithi019 ceph-mon[35186]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 5 op/s 2024-09-16T00:27:32.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:32 smithi153 ceph-mon[35538]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 5 op/s 2024-09-16T00:27:32.970 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:32.970 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:32.971 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:32.989 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:33.016 INFO:teuthology.orchestra.run.smithi001.stdout:77309411402 2024-09-16T00:27:33.017 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T00:27:33.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:33 smithi001 ceph-mon[30934]: Health check update: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-16T00:27:33.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:33 smithi019 ceph-mon[35186]: Health check update: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-16T00:27:33.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:33 smithi153 ceph-mon[35538]: Health check update: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-16T00:27:34.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.130+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/203488220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d24106f20 msgr2=0x7f6d2410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.130+0000 7f6d2bc9a640 1 --2- 172.21.15.1:0/203488220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d24106f20 0x7f6d2410eaa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009ca0 tx=0x7f6d1402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.130+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/203488220 shutdown_connections 2024-09-16T00:27:34.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.130+0000 7f6d2bc9a640 1 --2- 172.21.15.1:0/203488220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d24106f20 0x7f6d2410eaa0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.130+0000 7f6d2bc9a640 1 --2- 172.21.15.1:0/203488220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d241027a0 0x7f6d241067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.130+0000 7f6d2bc9a640 1 --2- 172.21.15.1:0/203488220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d24101df0 0x7f6d241021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.130+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/203488220 >> 172.21.15.1:0/203488220 conn(0x7f6d240fd8f0 msgr2=0x7f6d240ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.130+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/203488220 shutdown_connections 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.130+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/203488220 wait complete. 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d2bc9a640 1 Processor -- start 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d2bc9a640 1 -- start start 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d2bc9a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d24101df0 0x7f6d240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d2bc9a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d241027a0 0x7f6d24074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d2bc9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d24106f20 0x7f6d24074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d2bc9a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24111f80 con 0x7f6d241027a0 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d2bc9a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24111e00 con 0x7f6d24106f20 2024-09-16T00:27:34.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d2bc9a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24112100 con 0x7f6d24101df0 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29235640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d241027a0 0x7f6d24074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29235640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d241027a0 0x7f6d24074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42740/0 (socket says 172.21.15.1:42740) 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29235640 1 -- 172.21.15.1:0/2754260496 learned_addr learned my addr 172.21.15.1:0/2754260496 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29a36640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d24101df0 0x7f6d240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d2a237640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d24106f20 0x7f6d24074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29235640 1 -- 172.21.15.1:0/2754260496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d24101df0 msgr2=0x7f6d240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29235640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d24101df0 0x7f6d240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29235640 1 -- 172.21.15.1:0/2754260496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d24106f20 msgr2=0x7f6d24074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29235640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d24106f20 0x7f6d24074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29235640 1 -- 172.21.15.1:0/2754260496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d241a68e0 con 0x7f6d241027a0 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.131+0000 7f6d29a36640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d24101df0 0x7f6d240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:34.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.132+0000 7f6d29235640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d241027a0 0x7f6d24074a30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800e9f0 tx=0x7f6d1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.132+0000 7f6d22ffd640 1 -- 172.21.15.1:0/2754260496 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800cd80 con 0x7f6d241027a0 2024-09-16T00:27:34.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.132+0000 7f6d22ffd640 1 -- 172.21.15.1:0/2754260496 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d18005500 con 0x7f6d241027a0 2024-09-16T00:27:34.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.132+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d241a6bd0 con 0x7f6d241027a0 2024-09-16T00:27:34.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.132+0000 7f6d22ffd640 1 -- 172.21.15.1:0/2754260496 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18010690 con 0x7f6d241027a0 2024-09-16T00:27:34.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.132+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d241a7110 con 0x7f6d241027a0 2024-09-16T00:27:34.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.133+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d240628c0 con 0x7f6d241027a0 2024-09-16T00:27:34.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.134+0000 7f6d22ffd640 1 -- 172.21.15.1:0/2754260496 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f6d180046c0 con 0x7f6d241027a0 2024-09-16T00:27:34.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.134+0000 7f6d22ffd640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6d0c0a3350 0x7f6d0c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.135+0000 7f6d22ffd640 1 -- 172.21.15.1:0/2754260496 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6d180d0740 con 0x7f6d241027a0 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.135+0000 7f6d29a36640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6d0c0a3350 0x7f6d0c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.135+0000 7f6d22ffd640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] conn(0x7f6d0c0ae7a0 0x7f6d0c0b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.135+0000 7f6d22ffd640 1 -- 172.21.15.1:0/2754260496 --> [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d1801aec0 con 0x7f6d0c0ae7a0 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.135+0000 7f6d22ffd640 1 -- 172.21.15.1:0/2754260496 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f6d180d0b90 con 0x7f6d241027a0 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.135+0000 7f6d29a36640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6d0c0a3350 0x7f6d0c0a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c005d00 tx=0x7f6d1c009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.135+0000 7f6d2a237640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] conn(0x7f6d0c0ae7a0 0x7f6d0c0b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.136+0000 7f6d2a237640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] conn(0x7f6d0c0ae7a0 0x7f6d0c0b0c40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.136+0000 7f6d22ffd640 1 -- 172.21.15.1:0/2754260496 <== osd.7 v2:172.21.15.19:6816/549185107 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d1801aec0 con 0x7f6d0c0ae7a0 2024-09-16T00:27:34.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.150+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 --> [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d24062d40 con 0x7f6d0c0ae7a0 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d22ffd640 1 -- 172.21.15.1:0/2754260496 <== osd.7 v2:172.21.15.19:6816/549185107 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d24062d40 con 0x7f6d0c0ae7a0 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 >> [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] conn(0x7f6d0c0ae7a0 msgr2=0x7f6d0c0b0c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] conn(0x7f6d0c0ae7a0 0x7f6d0c0b0c40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6d0c0a3350 msgr2=0x7f6d0c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6d0c0a3350 0x7f6d0c0a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c005d00 tx=0x7f6d1c009550 comp rx=0 tx=0).stop 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d241027a0 msgr2=0x7f6d24074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 --2- 172.21.15.1:0/2754260496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d241027a0 0x7f6d24074a30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800e9f0 tx=0x7f6d1800edf0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2a237640 1 -- 172.21.15.1:0/2754260496 reap_dead start 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 shutdown_connections 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 >> 172.21.15.1:0/2754260496 conn(0x7f6d240fd8f0 msgr2=0x7f6d240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 shutdown_connections 2024-09-16T00:27:34.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.151+0000 7f6d2bc9a640 1 -- 172.21.15.1:0/2754260496 wait complete. 2024-09-16T00:27:34.364 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:34 smithi001 ceph-mon[30934]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 895 B/s wr, 8 op/s 2024-09-16T00:27:34.365 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:34 smithi001 ceph-mon[30934]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-09-16T00:27:34.365 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:34 smithi001 ceph-mon[30934]: Cluster is now healthy 2024-09-16T00:27:34.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:34 smithi019 ceph-mon[35186]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 895 B/s wr, 8 op/s 2024-09-16T00:27:34.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:34 smithi019 ceph-mon[35186]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-09-16T00:27:34.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:34 smithi019 ceph-mon[35186]: Cluster is now healthy 2024-09-16T00:27:34.613 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:34 smithi153 ceph-mon[35538]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 895 B/s wr, 8 op/s 2024-09-16T00:27:34.613 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:34 smithi153 ceph-mon[35538]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-09-16T00:27:34.613 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:34 smithi153 ceph-mon[35538]: Cluster is now healthy 2024-09-16T00:27:34.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.801+0000 7effdf0a7640 1 -- 172.21.15.1:0/1612513438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd81027a0 msgr2=0x7effd81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.801+0000 7effdf0a7640 1 --2- 172.21.15.1:0/1612513438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd81027a0 0x7effd81067f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7effcc009900 tx=0x7effcc02a410 comp rx=0 tx=0).stop 2024-09-16T00:27:34.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.802+0000 7effdf0a7640 1 -- 172.21.15.1:0/1612513438 shutdown_connections 2024-09-16T00:27:34.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.802+0000 7effdf0a7640 1 --2- 172.21.15.1:0/1612513438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8106f20 0x7effd810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.802+0000 7effdf0a7640 1 --2- 172.21.15.1:0/1612513438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd81027a0 0x7effd81067f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.802+0000 7effdf0a7640 1 --2- 172.21.15.1:0/1612513438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effd8101df0 0x7effd81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.802+0000 7effdf0a7640 1 -- 172.21.15.1:0/1612513438 >> 172.21.15.1:0/1612513438 conn(0x7effd80fd8f0 msgr2=0x7effd80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.802+0000 7effdf0a7640 1 -- 172.21.15.1:0/1612513438 shutdown_connections 2024-09-16T00:27:34.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.802+0000 7effdf0a7640 1 -- 172.21.15.1:0/1612513438 wait complete. 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.802+0000 7effdf0a7640 1 Processor -- start 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.802+0000 7effdf0a7640 1 -- start start 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effdf0a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8101df0 0x7effd8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effdf0a7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd81027a0 0x7effd81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effdf0a7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effd8106f20 0x7effd819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effdf0a7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7effd8111fc0 con 0x7effd81027a0 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effdf0a7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7effd8111e40 con 0x7effd8101df0 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effdf0a7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7effd8112140 con 0x7effd8106f20 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effd7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd81027a0 0x7effd81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effdce43640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8101df0 0x7effd8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effd7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd81027a0 0x7effd81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42762/0 (socket says 172.21.15.1:42762) 2024-09-16T00:27:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effdd644640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effd8106f20 0x7effd819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effd7fff640 1 -- 172.21.15.1:0/2235404965 learned_addr learned my addr 172.21.15.1:0/2235404965 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effd7fff640 1 -- 172.21.15.1:0/2235404965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effd8106f20 msgr2=0x7effd819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effd7fff640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effd8106f20 0x7effd819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effd7fff640 1 -- 172.21.15.1:0/2235404965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8101df0 msgr2=0x7effd8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effd7fff640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8101df0 0x7effd8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.803+0000 7effd7fff640 1 -- 172.21.15.1:0/2235404965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd819d440 con 0x7effd81027a0 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.804+0000 7effd7fff640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd81027a0 0x7effd81989d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7effcc006010 tx=0x7effcc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.804+0000 7effd5ffb640 1 -- 172.21.15.1:0/2235404965 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc03f030 con 0x7effd81027a0 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.804+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd819d6d0 con 0x7effd81027a0 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.804+0000 7effd5ffb640 1 -- 172.21.15.1:0/2235404965 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effcc003ce0 con 0x7effd81027a0 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.804+0000 7effd5ffb640 1 -- 172.21.15.1:0/2235404965 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc035070 con 0x7effd81027a0 2024-09-16T00:27:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.804+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81a5010 con 0x7effd81027a0 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.805+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7effd8062980 con 0x7effd81027a0 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.806+0000 7effd5ffb640 1 -- 172.21.15.1:0/2235404965 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7effcc042030 con 0x7effd81027a0 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.807+0000 7effd5ffb640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7effc00a32d0 0x7effc00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.807+0000 7effd5ffb640 1 -- 172.21.15.1:0/2235404965 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7effcc0f0a50 con 0x7effd81027a0 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.807+0000 7effdce43640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7effc00a32d0 0x7effc00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.807+0000 7effd5ffb640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] conn(0x7effc00ae720 0x7effc00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.807+0000 7effd5ffb640 1 -- 172.21.15.1:0/2235404965 --> [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effcc040e70 con 0x7effc00ae720 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.807+0000 7effd5ffb640 1 -- 172.21.15.1:0/2235404965 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7effcc0f0ea0 con 0x7effd81027a0 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.807+0000 7effdd644640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] conn(0x7effc00ae720 0x7effc00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.807+0000 7effdce43640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7effc00a32d0 0x7effc00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7effc8005fd0 tx=0x7effc800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.808+0000 7effdd644640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] conn(0x7effc00ae720 0x7effc00b0bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.808+0000 7effd5ffb640 1 -- 172.21.15.1:0/2235404965 <== osd.6 v2:172.21.15.1:6818/2734761121 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7effcc040e70 con 0x7effc00ae720 2024-09-16T00:27:34.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.822+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 --> [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7effd8062d80 con 0x7effc00ae720 2024-09-16T00:27:34.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.822+0000 7effd5ffb640 1 -- 172.21.15.1:0/2235404965 <== osd.6 v2:172.21.15.1:6818/2734761121 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7effd8062d80 con 0x7effc00ae720 2024-09-16T00:27:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] conn(0x7effc00ae720 msgr2=0x7effc00b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] conn(0x7effc00ae720 0x7effc00b0bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7effc00a32d0 msgr2=0x7effc00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7effc00a32d0 0x7effc00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7effc8005fd0 tx=0x7effc800b390 comp rx=0 tx=0).stop 2024-09-16T00:27:34.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd81027a0 msgr2=0x7effd81989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 --2- 172.21.15.1:0/2235404965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd81027a0 0x7effd81989d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7effcc006010 tx=0x7effcc0042f0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdd644640 1 -- 172.21.15.1:0/2235404965 reap_dead start 2024-09-16T00:27:34.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 shutdown_connections 2024-09-16T00:27:34.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 >> 172.21.15.1:0/2235404965 conn(0x7effd80fd8f0 msgr2=0x7effd80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 shutdown_connections 2024-09-16T00:27:34.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:34.823+0000 7effdf0a7640 1 -- 172.21.15.1:0/2235404965 wait complete. 2024-09-16T00:27:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.021+0000 7fe7378c8640 1 -- 172.21.15.1:0/1392014278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe730102790 msgr2=0x7fe7301067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.021+0000 7fe7378c8640 1 --2- 172.21.15.1:0/1392014278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe730102790 0x7fe7301067e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe724009900 tx=0x7fe72402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:27:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.021+0000 7fe7378c8640 1 -- 172.21.15.1:0/1392014278 shutdown_connections 2024-09-16T00:27:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.021+0000 7fe7378c8640 1 --2- 172.21.15.1:0/1392014278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730106f10 0x7fe73010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.021+0000 7fe7378c8640 1 --2- 172.21.15.1:0/1392014278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe730102790 0x7fe7301067e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.021+0000 7fe7378c8640 1 --2- 172.21.15.1:0/1392014278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe730101de0 0x7fe7301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.021+0000 7fe7378c8640 1 -- 172.21.15.1:0/1392014278 >> 172.21.15.1:0/1392014278 conn(0x7fe7300fd8c0 msgr2=0x7fe7300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.021+0000 7fe7378c8640 1 -- 172.21.15.1:0/1392014278 shutdown_connections 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.021+0000 7fe7378c8640 1 -- 172.21.15.1:0/1392014278 wait complete. 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe7378c8640 1 Processor -- start 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe7378c8640 1 -- start start 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe7378c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe730101de0 0x7fe7301983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe7378c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730102790 0x7fe730198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe7378c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe730106f10 0x7fe73019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe7378c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe730111f70 con 0x7fe730101de0 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe7378c8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe730111df0 con 0x7fe730102790 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe7378c8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7301120f0 con 0x7fe730106f10 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe735664640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe730101de0 0x7fe7301983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe735664640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe730101de0 0x7fe7301983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42790/0 (socket says 172.21.15.1:42790) 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.022+0000 7fe735664640 1 -- 172.21.15.1:0/1014091891 learned_addr learned my addr 172.21.15.1:0/1014091891 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe735e65640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe730106f10 0x7fe73019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe734e63640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730102790 0x7fe730198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe735664640 1 -- 172.21.15.1:0/1014091891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe730106f10 msgr2=0x7fe73019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe735664640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe730106f10 0x7fe73019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe735664640 1 -- 172.21.15.1:0/1014091891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730102790 msgr2=0x7fe730198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe735664640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730102790 0x7fe730198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe735664640 1 -- 172.21.15.1:0/1014091891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73019d380 con 0x7fe730101de0 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe735664640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe730101de0 0x7fe7301983d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe72000d770 tx=0x7fe72000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe72e7fc640 1 -- 172.21.15.1:0/1014091891 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720017070 con 0x7fe730101de0 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73019d670 con 0x7fe730101de0 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe72e7fc640 1 -- 172.21.15.1:0/1014091891 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe720005500 con 0x7fe730101de0 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe72e7fc640 1 -- 172.21.15.1:0/1014091891 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720004e00 con 0x7fe730101de0 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.023+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7301a4f50 con 0x7fe730101de0 2024-09-16T00:27:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.024+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe730062940 con 0x7fe730101de0 2024-09-16T00:27:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.026+0000 7fe72e7fc640 1 -- 172.21.15.1:0/1014091891 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe720004fa0 con 0x7fe730101de0 2024-09-16T00:27:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.026+0000 7fe72e7fc640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7180a32d0 0x7fe7180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.026+0000 7fe734e63640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7180a32d0 0x7fe7180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.026+0000 7fe72e7fc640 1 -- 172.21.15.1:0/1014091891 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe7200cfc70 con 0x7fe730101de0 2024-09-16T00:27:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.026+0000 7fe72e7fc640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] conn(0x7fe7180ae720 0x7fe7180b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.026+0000 7fe72e7fc640 1 -- 172.21.15.1:0/1014091891 --> [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe72000fec0 con 0x7fe7180ae720 2024-09-16T00:27:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.026+0000 7fe72e7fc640 1 -- 172.21.15.1:0/1014091891 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe7200d00c0 con 0x7fe730101de0 2024-09-16T00:27:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.027+0000 7fe735e65640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] conn(0x7fe7180ae720 0x7fe7180b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.027+0000 7fe734e63640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7180a32d0 0x7fe7180a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe724009230 tx=0x7fe7240058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.027+0000 7fe735e65640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] conn(0x7fe7180ae720 0x7fe7180b0bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.028+0000 7fe72e7fc640 1 -- 172.21.15.1:0/1014091891 <== osd.9 v2:172.21.15.1:6826/4278887169 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe72000fec0 con 0x7fe7180ae720 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.041+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 --> [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe730062dc0 con 0x7fe7180ae720 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe72e7fc640 1 -- 172.21.15.1:0/1014091891 <== osd.9 v2:172.21.15.1:6826/4278887169 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe730062dc0 con 0x7fe7180ae720 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] conn(0x7fe7180ae720 msgr2=0x7fe7180b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] conn(0x7fe7180ae720 0x7fe7180b0bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7180a32d0 msgr2=0x7fe7180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7180a32d0 0x7fe7180a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe724009230 tx=0x7fe7240058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe730101de0 msgr2=0x7fe7301983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 --2- 172.21.15.1:0/1014091891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe730101de0 0x7fe7301983d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe72000d770 tx=0x7fe72000dc30 comp rx=0 tx=0).stop 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe735e65640 1 -- 172.21.15.1:0/1014091891 reap_dead start 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 shutdown_connections 2024-09-16T00:27:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 >> 172.21.15.1:0/1014091891 conn(0x7fe7300fd8c0 msgr2=0x7fe7300fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:35.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 shutdown_connections 2024-09-16T00:27:35.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:35.042+0000 7fe7378c8640 1 -- 172.21.15.1:0/1014091891 wait complete. 2024-09-16T00:27:36.188 INFO:teuthology.orchestra.run.smithi001.stdout:193273528374 2024-09-16T00:27:36.188 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T00:27:36.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:36 smithi001 ceph-mon[30934]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 789 B/s wr, 6 op/s 2024-09-16T00:27:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.435+0000 7fd0448fa640 1 -- 172.21.15.1:0/3844559839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0401011e0 msgr2=0x7fd040101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.435+0000 7fd0448fa640 1 --2- 172.21.15.1:0/3844559839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0401011e0 0x7fd040101640 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd034009900 tx=0x7fd03402a410 comp rx=0 tx=0).stop 2024-09-16T00:27:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.435+0000 7fd0448fa640 1 -- 172.21.15.1:0/3844559839 shutdown_connections 2024-09-16T00:27:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.435+0000 7fd0448fa640 1 --2- 172.21.15.1:0/3844559839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd040101b80 0x7fd04010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.435+0000 7fd0448fa640 1 --2- 172.21.15.1:0/3844559839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0401011e0 0x7fd040101640 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.435+0000 7fd0448fa640 1 --2- 172.21.15.1:0/3844559839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040105ab0 0x7fd040105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.435+0000 7fd0448fa640 1 -- 172.21.15.1:0/3844559839 >> 172.21.15.1:0/3844559839 conn(0x7fd0400fbee0 msgr2=0x7fd0400fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.435+0000 7fd0448fa640 1 -- 172.21.15.1:0/3844559839 shutdown_connections 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.435+0000 7fd0448fa640 1 -- 172.21.15.1:0/3844559839 wait complete. 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.436+0000 7fd0448fa640 1 Processor -- start 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.436+0000 7fd0448fa640 1 -- start start 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.436+0000 7fd0448fa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0401011e0 0x7fd040198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.436+0000 7fd0448fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040101b80 0x7fd0401988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.436+0000 7fd0448fa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd040105ab0 0x7fd04019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.436+0000 7fd0448fa640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd040110ed0 con 0x7fd0401011e0 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.436+0000 7fd0448fa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd040110d50 con 0x7fd040101b80 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.436+0000 7fd0448fa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd040111050 con 0x7fd040105ab0 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.436+0000 7fd03e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0401011e0 0x7fd040198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0401011e0 0x7fd040198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42808/0 (socket says 172.21.15.1:42808) 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03e59c640 1 -- 172.21.15.1:0/1243101207 learned_addr learned my addr 172.21.15.1:0/1243101207 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03dd9b640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040101b80 0x7fd0401988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03ed9d640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd040105ab0 0x7fd04019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03e59c640 1 -- 172.21.15.1:0/1243101207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd040105ab0 msgr2=0x7fd04019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03e59c640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd040105ab0 0x7fd04019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03e59c640 1 -- 172.21.15.1:0/1243101207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040101b80 msgr2=0x7fd0401988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03e59c640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040101b80 0x7fd0401988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03e59c640 1 -- 172.21.15.1:0/1243101207 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04019d330 con 0x7fd0401011e0 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03ed9d640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd040105ab0 0x7fd04019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:36.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd03e59c640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0401011e0 0x7fd040198380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd03000d950 tx=0x7fd03000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd02f7fe640 1 -- 172.21.15.1:0/1243101207 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030017070 con 0x7fd0401011e0 2024-09-16T00:27:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd02f7fe640 1 -- 172.21.15.1:0/1243101207 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd030009c30 con 0x7fd0401011e0 2024-09-16T00:27:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04019d620 con 0x7fd0401011e0 2024-09-16T00:27:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd02f7fe640 1 -- 172.21.15.1:0/1243101207 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03000f6b0 con 0x7fd0401011e0 2024-09-16T00:27:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.437+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0401a4f00 con 0x7fd0401011e0 2024-09-16T00:27:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.438+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd040062980 con 0x7fd0401011e0 2024-09-16T00:27:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.440+0000 7fd02f7fe640 1 -- 172.21.15.1:0/1243101207 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd0300109d0 con 0x7fd0401011e0 2024-09-16T00:27:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.440+0000 7fd02f7fe640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0280a32d0 0x7fd0280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.440+0000 7fd03dd9b640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0280a32d0 0x7fd0280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.440+0000 7fd02f7fe640 1 -- 172.21.15.1:0/1243101207 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd0300cff10 con 0x7fd0401011e0 2024-09-16T00:27:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.440+0000 7fd02f7fe640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] conn(0x7fd0280ae720 0x7fd0280b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.440+0000 7fd02f7fe640 1 -- 172.21.15.1:0/1243101207 --> [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd030016ec0 con 0x7fd0280ae720 2024-09-16T00:27:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.440+0000 7fd02f7fe640 1 -- 172.21.15.1:0/1243101207 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd0300d02c0 con 0x7fd0401011e0 2024-09-16T00:27:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.441+0000 7fd03ed9d640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] conn(0x7fd0280ae720 0x7fd0280b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.441+0000 7fd03dd9b640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0280a32d0 0x7fd0280a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd034009230 tx=0x7fd0340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.441+0000 7fd03ed9d640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] conn(0x7fd0280ae720 0x7fd0280b0bc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.442+0000 7fd02f7fe640 1 -- 172.21.15.1:0/1243101207 <== osd.0 v2:172.21.15.1:6802/2752149463 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd030016ec0 con 0x7fd0280ae720 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.455+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 --> [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd040062e00 con 0x7fd0280ae720 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd02f7fe640 1 -- 172.21.15.1:0/1243101207 <== osd.0 v2:172.21.15.1:6802/2752149463 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd040062e00 con 0x7fd0280ae720 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] conn(0x7fd0280ae720 msgr2=0x7fd0280b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] conn(0x7fd0280ae720 0x7fd0280b0bc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0280a32d0 msgr2=0x7fd0280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0280a32d0 0x7fd0280a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd034009230 tx=0x7fd0340058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0401011e0 msgr2=0x7fd040198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 --2- 172.21.15.1:0/1243101207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0401011e0 0x7fd040198380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd03000d950 tx=0x7fd03000de10 comp rx=0 tx=0).stop 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd03ed9d640 1 -- 172.21.15.1:0/1243101207 reap_dead start 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 shutdown_connections 2024-09-16T00:27:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 >> 172.21.15.1:0/1243101207 conn(0x7fd0400fbee0 msgr2=0x7fd040100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 shutdown_connections 2024-09-16T00:27:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:36.456+0000 7fd0448fa640 1 -- 172.21.15.1:0/1243101207 wait complete. 2024-09-16T00:27:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:36 smithi019 ceph-mon[35186]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 789 B/s wr, 6 op/s 2024-09-16T00:27:36.701 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:36.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:36 smithi153 ceph-mon[35538]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 789 B/s wr, 6 op/s 2024-09-16T00:27:37.136 INFO:teuthology.orchestra.run.smithi001.stdout:34359738453 2024-09-16T00:27:37.136 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T00:27:37.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.302+0000 7fe69c64a640 1 -- 172.21.15.1:0/3078374968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe694106a70 msgr2=0x7fe694106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.302+0000 7fe69c64a640 1 --2- 172.21.15.1:0/3078374968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe694106a70 0x7fe694106e50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe690009930 tx=0x7fe69002a420 comp rx=0 tx=0).stop 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.302+0000 7fe69c64a640 1 -- 172.21.15.1:0/3078374968 shutdown_connections 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.302+0000 7fe69c64a640 1 --2- 172.21.15.1:0/3078374968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe694101410 0x7fe69410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.302+0000 7fe69c64a640 1 --2- 172.21.15.1:0/3078374968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe694100a70 0x7fe694100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.302+0000 7fe69c64a640 1 --2- 172.21.15.1:0/3078374968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe694106a70 0x7fe694106e50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.302+0000 7fe69c64a640 1 -- 172.21.15.1:0/3078374968 >> 172.21.15.1:0/3078374968 conn(0x7fe6940fc790 msgr2=0x7fe6940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.302+0000 7fe69c64a640 1 -- 172.21.15.1:0/3078374968 shutdown_connections 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.302+0000 7fe69c64a640 1 -- 172.21.15.1:0/3078374968 wait complete. 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.303+0000 7fe69c64a640 1 Processor -- start 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.303+0000 7fe69c64a640 1 -- start start 2024-09-16T00:27:37.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69c64a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe694100a70 0x7fe694196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69c64a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe694101410 0x7fe6941967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69c64a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe694106a70 0x7fe69419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69c64a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410fdc0 con 0x7fe694106a70 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69c64a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410fc40 con 0x7fe694101410 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69c64a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410ff40 con 0x7fe694100a70 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69a3e6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe694100a70 0x7fe694196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69a3e6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe694100a70 0x7fe694196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:49124/0 (socket says 172.21.15.1:49124) 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69a3e6640 1 -- 172.21.15.1:0/2574157017 learned_addr learned my addr 172.21.15.1:0/2574157017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe699be5640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe694101410 0x7fe6941967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe69abe7640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe694106a70 0x7fe69419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe699be5640 1 -- 172.21.15.1:0/2574157017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe694100a70 msgr2=0x7fe694196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe699be5640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe694100a70 0x7fe694196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe699be5640 1 -- 172.21.15.1:0/2574157017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe694106a70 msgr2=0x7fe69419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe699be5640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe694106a70 0x7fe69419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.304+0000 7fe699be5640 1 -- 172.21.15.1:0/2574157017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69419b210 con 0x7fe694101410 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.305+0000 7fe699be5640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe694101410 0x7fe6941967a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe684009990 tx=0x7fe684009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.305+0000 7fe68b7fe640 1 -- 172.21.15.1:0/2574157017 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68400d030 con 0x7fe694101410 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.305+0000 7fe68b7fe640 1 -- 172.21.15.1:0/2574157017 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68400d1d0 con 0x7fe694101410 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.305+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69419b500 con 0x7fe694101410 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.305+0000 7fe68b7fe640 1 -- 172.21.15.1:0/2574157017 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684003850 con 0x7fe694101410 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.305+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941a2de0 con 0x7fe694101410 2024-09-16T00:27:37.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.306+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe694062980 con 0x7fe694101410 2024-09-16T00:27:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.307+0000 7fe68b7fe640 1 -- 172.21.15.1:0/2574157017 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe68400d380 con 0x7fe694101410 2024-09-16T00:27:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.308+0000 7fe68b7fe640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe6800a32d0 0x7fe6800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.308+0000 7fe68b7fe640 1 -- 172.21.15.1:0/2574157017 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe6840cf800 con 0x7fe694101410 2024-09-16T00:27:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.308+0000 7fe69a3e6640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe6800a32d0 0x7fe6800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.308+0000 7fe68b7fe640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] conn(0x7fe6800ae720 0x7fe6800b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.308+0000 7fe68b7fe640 1 -- 172.21.15.1:0/2574157017 --> [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe684005720 con 0x7fe6800ae720 2024-09-16T00:27:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.308+0000 7fe68b7fe640 1 -- 172.21.15.1:0/2574157017 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe6840cfc50 con 0x7fe694101410 2024-09-16T00:27:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.308+0000 7fe69abe7640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] conn(0x7fe6800ae720 0x7fe6800b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.309+0000 7fe69a3e6640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe6800a32d0 0x7fe6800a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe6900096c0 tx=0x7fe690005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.309+0000 7fe69abe7640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] conn(0x7fe6800ae720 0x7fe6800b0bc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.310+0000 7fe68b7fe640 1 -- 172.21.15.1:0/2574157017 <== osd.11 v2:172.21.15.153:6824/3642895016 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe684005720 con 0x7fe6800ae720 2024-09-16T00:27:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.324+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 --> [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe694062e00 con 0x7fe6800ae720 2024-09-16T00:27:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.325+0000 7fe68b7fe640 1 -- 172.21.15.1:0/2574157017 <== osd.11 v2:172.21.15.153:6824/3642895016 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe694062e00 con 0x7fe6800ae720 2024-09-16T00:27:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.325+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 >> [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] conn(0x7fe6800ae720 msgr2=0x7fe6800b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.325+0000 7fe69c64a640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] conn(0x7fe6800ae720 0x7fe6800b0bc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.325+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe6800a32d0 msgr2=0x7fe6800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.325+0000 7fe69c64a640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe6800a32d0 0x7fe6800a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe6900096c0 tx=0x7fe690005b50 comp rx=0 tx=0).stop 2024-09-16T00:27:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.325+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe694101410 msgr2=0x7fe6941967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.325+0000 7fe69c64a640 1 --2- 172.21.15.1:0/2574157017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe694101410 0x7fe6941967a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe684009990 tx=0x7fe684009e50 comp rx=0 tx=0).stop 2024-09-16T00:27:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.325+0000 7fe69abe7640 1 -- 172.21.15.1:0/2574157017 reap_dead start 2024-09-16T00:27:37.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.326+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 shutdown_connections 2024-09-16T00:27:37.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.326+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 >> 172.21.15.1:0/2574157017 conn(0x7fe6940fc790 msgr2=0x7fe6940fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:37.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.326+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 shutdown_connections 2024-09-16T00:27:37.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:37.326+0000 7fe69c64a640 1 -- 172.21.15.1:0/2574157017 wait complete. 2024-09-16T00:27:38.275 INFO:teuthology.orchestra.run.smithi001.stdout:214748364848 2024-09-16T00:27:38.275 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T00:27:38.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.443+0000 7fea83111640 1 -- 172.21.15.1:0/3516745216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7c1027a0 msgr2=0x7fea7c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.443+0000 7fea83111640 1 --2- 172.21.15.1:0/3516745216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7c1027a0 0x7fea7c1067f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fea68009900 tx=0x7fea6802a410 comp rx=0 tx=0).stop 2024-09-16T00:27:38.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.443+0000 7fea83111640 1 -- 172.21.15.1:0/3516745216 shutdown_connections 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.443+0000 7fea83111640 1 --2- 172.21.15.1:0/3516745216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea7c106f20 0x7fea7c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.443+0000 7fea83111640 1 --2- 172.21.15.1:0/3516745216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7c1027a0 0x7fea7c1067f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.443+0000 7fea83111640 1 --2- 172.21.15.1:0/3516745216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea7c101df0 0x7fea7c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.443+0000 7fea83111640 1 -- 172.21.15.1:0/3516745216 >> 172.21.15.1:0/3516745216 conn(0x7fea7c0fd8b0 msgr2=0x7fea7c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.443+0000 7fea83111640 1 -- 172.21.15.1:0/3516745216 shutdown_connections 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.444+0000 7fea83111640 1 -- 172.21.15.1:0/3516745216 wait complete. 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.444+0000 7fea83111640 1 Processor -- start 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.444+0000 7fea83111640 1 -- start start 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea83111640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea7c101df0 0x7fea7c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea83111640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7c1027a0 0x7fea7c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea83111640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea7c106f20 0x7fea7c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea83111640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c111fd0 con 0x7fea7c1027a0 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea83111640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c111e50 con 0x7fea7c106f20 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea83111640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c112150 con 0x7fea7c101df0 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7c1027a0 0x7fea7c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7c1027a0 0x7fea7c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42828/0 (socket says 172.21.15.1:42828) 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea7bfff640 1 -- 172.21.15.1:0/1555389055 learned_addr learned my addr 172.21.15.1:0/1555389055 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea80ead640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea7c101df0 0x7fea7c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea816ae640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea7c106f20 0x7fea7c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea7bfff640 1 -- 172.21.15.1:0/1555389055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea7c101df0 msgr2=0x7fea7c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea7bfff640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea7c101df0 0x7fea7c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea7bfff640 1 -- 172.21.15.1:0/1555389055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea7c106f20 msgr2=0x7fea7c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea7bfff640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea7c106f20 0x7fea7c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea7bfff640 1 -- 172.21.15.1:0/1555389055 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c19d3e0 con 0x7fea7c1027a0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.445+0000 7fea7bfff640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7c1027a0 0x7fea7c198970 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea68004770 tx=0x7fea68003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.446+0000 7fea79ffb640 1 -- 172.21.15.1:0/1555389055 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6803f070 con 0x7fea7c1027a0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.446+0000 7fea79ffb640 1 -- 172.21.15.1:0/1555389055 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea68031ca0 con 0x7fea7c1027a0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.446+0000 7fea79ffb640 1 -- 172.21.15.1:0/1555389055 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68035030 con 0x7fea7c1027a0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.446+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c19d670 con 0x7fea7c1027a0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.446+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c1a4fb0 con 0x7fea7c1027a0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.447+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea7c062980 con 0x7fea7c1027a0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.448+0000 7fea79ffb640 1 -- 172.21.15.1:0/1555389055 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fea68042070 con 0x7fea7c1027a0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.449+0000 7fea79ffb640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fea6c0a32d0 0x7fea6c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.449+0000 7fea79ffb640 1 -- 172.21.15.1:0/1555389055 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fea680f09b0 con 0x7fea7c1027a0 2024-09-16T00:27:38.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.449+0000 7fea80ead640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fea6c0a32d0 0x7fea6c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.449+0000 7fea79ffb640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] conn(0x7fea6c0ae720 0x7fea6c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.449+0000 7fea79ffb640 1 -- 172.21.15.1:0/1555389055 --> [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea68038e90 con 0x7fea6c0ae720 2024-09-16T00:27:38.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.449+0000 7fea79ffb640 1 -- 172.21.15.1:0/1555389055 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fea680f0d60 con 0x7fea7c1027a0 2024-09-16T00:27:38.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.449+0000 7fea80ead640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fea6c0a32d0 0x7fea6c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea70005d00 tx=0x7fea7000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:38.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.449+0000 7fea816ae640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] conn(0x7fea6c0ae720 0x7fea6c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.450+0000 7fea816ae640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] conn(0x7fea6c0ae720 0x7fea6c0b0bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:38.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.451+0000 7fea79ffb640 1 -- 172.21.15.1:0/1555389055 <== osd.5 v2:172.21.15.153:6808/394316446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea68038e90 con 0x7fea6c0ae720 2024-09-16T00:27:38.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.468+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 --> [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea7c062e00 con 0x7fea6c0ae720 2024-09-16T00:27:38.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.469+0000 7fea79ffb640 1 -- 172.21.15.1:0/1555389055 <== osd.5 v2:172.21.15.153:6808/394316446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea7c062e00 con 0x7fea6c0ae720 2024-09-16T00:27:38.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.469+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 >> [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] conn(0x7fea6c0ae720 msgr2=0x7fea6c0b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.469+0000 7fea83111640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] conn(0x7fea6c0ae720 0x7fea6c0b0bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.469+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fea6c0a32d0 msgr2=0x7fea6c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.469+0000 7fea83111640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fea6c0a32d0 0x7fea6c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea70005d00 tx=0x7fea7000b410 comp rx=0 tx=0).stop 2024-09-16T00:27:38.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.469+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7c1027a0 msgr2=0x7fea7c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.469+0000 7fea83111640 1 --2- 172.21.15.1:0/1555389055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7c1027a0 0x7fea7c198970 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea68004770 tx=0x7fea68003c10 comp rx=0 tx=0).stop 2024-09-16T00:27:38.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.470+0000 7fea816ae640 1 -- 172.21.15.1:0/1555389055 reap_dead start 2024-09-16T00:27:38.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.470+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 shutdown_connections 2024-09-16T00:27:38.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.470+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 >> 172.21.15.1:0/1555389055 conn(0x7fea7c0fd8b0 msgr2=0x7fea7c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:38.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.470+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 shutdown_connections 2024-09-16T00:27:38.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:38.470+0000 7fea83111640 1 -- 172.21.15.1:0/1555389055 wait complete. 2024-09-16T00:27:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:38 smithi019 ceph-mon[35186]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 711 B/s wr, 6 op/s 2024-09-16T00:27:38.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:38 smithi001 ceph-mon[30934]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 711 B/s wr, 6 op/s 2024-09-16T00:27:38.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:38 smithi153 ceph-mon[35538]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 711 B/s wr, 6 op/s 2024-09-16T00:27:39.208 INFO:teuthology.orchestra.run.smithi001.stdout:326417514526 2024-09-16T00:27:39.208 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-16T00:27:39.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 -- 172.21.15.1:0/139617609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0100e20 msgr2=0x7f1cf0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:39.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/139617609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0100e20 0x7f1cf0101200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009900 tx=0x7f1cec02a480 comp rx=0 tx=0).stop 2024-09-16T00:27:39.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 -- 172.21.15.1:0/139617609 shutdown_connections 2024-09-16T00:27:39.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/139617609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf0102ef0 0x7f1cf010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:39.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/139617609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf0102570 0x7f1cf01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:39.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/139617609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0100e20 0x7f1cf0101200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 -- 172.21.15.1:0/139617609 >> 172.21.15.1:0/139617609 conn(0x7f1cf00fc7d0 msgr2=0x7f1cf00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 -- 172.21.15.1:0/139617609 shutdown_connections 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 -- 172.21.15.1:0/139617609 wait complete. 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 Processor -- start 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.950+0000 7f1cf7a28640 1 -- start start 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf7a28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf0100e20 0x7f1cf0103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf7a28640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf0102570 0x7f1cf0103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf7a28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0102ef0 0x7f1cf01041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf7a28640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0110f80 con 0x7f1cf0102ef0 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf7a28640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0110e00 con 0x7f1cf0100e20 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf7a28640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0111100 con 0x7f1cf0102570 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf5fc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0102ef0 0x7f1cf01041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:39.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf5fc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0102ef0 0x7f1cf01041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42882/0 (socket says 172.21.15.1:42882) 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf5fc5640 1 -- 172.21.15.1:0/3504902215 learned_addr learned my addr 172.21.15.1:0/3504902215 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf57c4640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf0100e20 0x7f1cf0103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf4fc3640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf0102570 0x7f1cf0103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf5fc5640 1 -- 172.21.15.1:0/3504902215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf0102570 msgr2=0x7f1cf0103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf5fc5640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf0102570 0x7f1cf0103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf5fc5640 1 -- 172.21.15.1:0/3504902215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf0100e20 msgr2=0x7f1cf0103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf5fc5640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf0100e20 0x7f1cf0103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf5fc5640 1 -- 172.21.15.1:0/3504902215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf01a6a30 con 0x7f1cf0102ef0 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf5fc5640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0102ef0 0x7f1cf01041d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009fd0 tx=0x7f1ce800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:39.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1ce67fc640 1 -- 172.21.15.1:0/3504902215 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8017070 con 0x7f1cf0102ef0 2024-09-16T00:27:39.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1ce67fc640 1 -- 172.21.15.1:0/3504902215 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8003e50 con 0x7f1cf0102ef0 2024-09-16T00:27:39.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1ce67fc640 1 -- 172.21.15.1:0/3504902215 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80106c0 con 0x7f1cf0102ef0 2024-09-16T00:27:39.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.951+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf01a6d20 con 0x7f1cf0102ef0 2024-09-16T00:27:39.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.952+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf01a7260 con 0x7f1cf0102ef0 2024-09-16T00:27:39.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.952+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf0101200 con 0x7f1cf0102ef0 2024-09-16T00:27:39.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.954+0000 7f1ce67fc640 1 -- 172.21.15.1:0/3504902215 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f1ce8003970 con 0x7f1cf0102ef0 2024-09-16T00:27:39.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.954+0000 7f1ce67fc640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:39.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.954+0000 7f1ce67fc640 1 -- 172.21.15.1:0/3504902215 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1ce80cfaf0 con 0x7f1cf0102ef0 2024-09-16T00:27:39.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.954+0000 7f1cf57c4640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:39.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.955+0000 7f1cf57c4640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009230 tx=0x7f1cec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:39.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.959+0000 7f1ce67fc640 1 -- 172.21.15.1:0/3504902215 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ce809bc00 con 0x7f1cf0102ef0 2024-09-16T00:27:40.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.137+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1cf01a7490 con 0x7f1cf0102ef0 2024-09-16T00:27:40.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.137+0000 7f1ce67fc640 1 -- 172.21.15.1:0/3504902215 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ce809ff00 con 0x7f1cf0102ef0 2024-09-16T00:27:40.137 INFO:teuthology.orchestra.run.smithi001.stdout:55834574928 2024-09-16T00:27:40.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.139+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1cdc0a3350 msgr2=0x7f1cdc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.139+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009230 tx=0x7f1cec0058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.139+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0102ef0 msgr2=0x7f1cf01041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.139+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0102ef0 0x7f1cf01041d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009fd0 tx=0x7f1ce800b260 comp rx=0 tx=0).stop 2024-09-16T00:27:40.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.139+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 shutdown_connections 2024-09-16T00:27:40.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.139+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cf0102ef0 0x7f1cf01041d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.140+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.140+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cf0102570 0x7f1cf0103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.140+0000 7f1cf7a28640 1 --2- 172.21.15.1:0/3504902215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf0100e20 0x7f1cf0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.140+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 >> 172.21.15.1:0/3504902215 conn(0x7f1cf00fc7d0 msgr2=0x7f1cf006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:40.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.140+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 shutdown_connections 2024-09-16T00:27:40.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.140+0000 7f1cf7a28640 1 -- 172.21.15.1:0/3504902215 wait complete. 2024-09-16T00:27:40.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:40 smithi001 ceph-mon[30934]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:27:40.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:40 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3504902215' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:27:40.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.541+0000 7ffb68e3f640 1 -- 172.21.15.1:0/4022795895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb641036e0 msgr2=0x7ffb64107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.541+0000 7ffb68e3f640 1 --2- 172.21.15.1:0/4022795895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb641036e0 0x7ffb64107b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009930 tx=0x7ffb5802a420 comp rx=0 tx=0).stop 2024-09-16T00:27:40.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.541+0000 7ffb68e3f640 1 -- 172.21.15.1:0/4022795895 shutdown_connections 2024-09-16T00:27:40.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.541+0000 7ffb68e3f640 1 --2- 172.21.15.1:0/4022795895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb641082a0 0x7ffb6410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.541+0000 7ffb68e3f640 1 --2- 172.21.15.1:0/4022795895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb641036e0 0x7ffb64107b90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.541+0000 7ffb68e3f640 1 --2- 172.21.15.1:0/4022795895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64102d30 0x7ffb64103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.541+0000 7ffb68e3f640 1 -- 172.21.15.1:0/4022795895 >> 172.21.15.1:0/4022795895 conn(0x7ffb640fc7d0 msgr2=0x7ffb640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:40.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.541+0000 7ffb68e3f640 1 -- 172.21.15.1:0/4022795895 shutdown_connections 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.541+0000 7ffb68e3f640 1 -- 172.21.15.1:0/4022795895 wait complete. 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb68e3f640 1 Processor -- start 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb68e3f640 1 -- start start 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb68e3f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb64102d30 0x7ffb64100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb68e3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb641036e0 0x7ffb64100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb68e3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb641082a0 0x7ffb64101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb68e3f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64112060 con 0x7ffb641036e0 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb68e3f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64111ee0 con 0x7ffb641082a0 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb68e3f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641121e0 con 0x7ffb64102d30 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb61d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb641036e0 0x7ffb64100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb61d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb641036e0 0x7ffb64100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44828/0 (socket says 172.21.15.1:44828) 2024-09-16T00:27:40.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.542+0000 7ffb61d9b640 1 -- 172.21.15.1:0/2264894130 learned_addr learned my addr 172.21.15.1:0/2264894130 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb62d9d640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb641082a0 0x7ffb64101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb6259c640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb64102d30 0x7ffb64100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb61d9b640 1 -- 172.21.15.1:0/2264894130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb64102d30 msgr2=0x7ffb64100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb61d9b640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb64102d30 0x7ffb64100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb61d9b640 1 -- 172.21.15.1:0/2264894130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb641082a0 msgr2=0x7ffb64101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb61d9b640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb641082a0 0x7ffb64101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb61d9b640 1 -- 172.21.15.1:0/2264894130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb641a6b20 con 0x7ffb641036e0 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb62d9d640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb641082a0 0x7ffb64101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb61d9b640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb641036e0 0x7ffb64100ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005940 tx=0x7ffb580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb537fe640 1 -- 172.21.15.1:0/2264894130 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5803f030 con 0x7ffb641036e0 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb537fe640 1 -- 172.21.15.1:0/2264894130 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb58003ce0 con 0x7ffb641036e0 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb537fe640 1 -- 172.21.15.1:0/2264894130 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58035070 con 0x7ffb641036e0 2024-09-16T00:27:40.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb641a6db0 con 0x7ffb641036e0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.543+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb641a71e0 con 0x7ffb641036e0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.544+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb640628c0 con 0x7ffb641036e0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.545+0000 7ffb537fe640 1 -- 172.21.15.1:0/2264894130 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ffb58042030 con 0x7ffb641036e0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.546+0000 7ffb537fe640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb4c0a32d0 0x7ffb4c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.546+0000 7ffb537fe640 1 -- 172.21.15.1:0/2264894130 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ffb580f0aa0 con 0x7ffb641036e0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.546+0000 7ffb6259c640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb4c0a32d0 0x7ffb4c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.546+0000 7ffb537fe640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] conn(0x7ffb4c0ae720 0x7ffb4c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.546+0000 7ffb537fe640 1 -- 172.21.15.1:0/2264894130 --> [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb58032a80 con 0x7ffb4c0ae720 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.546+0000 7ffb537fe640 1 -- 172.21.15.1:0/2264894130 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ffb580f0ef0 con 0x7ffb641036e0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.547+0000 7ffb6259c640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb4c0a32d0 0x7ffb4c0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005fd0 tx=0x7ffb54005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.547+0000 7ffb62d9d640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] conn(0x7ffb4c0ae720 0x7ffb4c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.547+0000 7ffb62d9d640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] conn(0x7ffb4c0ae720 0x7ffb4c0b0bc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:40.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.548+0000 7ffb537fe640 1 -- 172.21.15.1:0/2264894130 <== osd.10 v2:172.21.15.19:6824/1721081370 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb58032a80 con 0x7ffb4c0ae720 2024-09-16T00:27:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.562+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 --> [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb64062cc0 con 0x7ffb4c0ae720 2024-09-16T00:27:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.562+0000 7ffb537fe640 1 -- 172.21.15.1:0/2264894130 <== osd.10 v2:172.21.15.19:6824/1721081370 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffb64062cc0 con 0x7ffb4c0ae720 2024-09-16T00:27:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.562+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 >> [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] conn(0x7ffb4c0ae720 msgr2=0x7ffb4c0b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.562+0000 7ffb68e3f640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] conn(0x7ffb4c0ae720 0x7ffb4c0b0bc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.563+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb4c0a32d0 msgr2=0x7ffb4c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.563+0000 7ffb68e3f640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb4c0a32d0 0x7ffb4c0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005fd0 tx=0x7ffb54005850 comp rx=0 tx=0).stop 2024-09-16T00:27:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.563+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb641036e0 msgr2=0x7ffb64100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.563+0000 7ffb68e3f640 1 --2- 172.21.15.1:0/2264894130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb641036e0 0x7ffb64100ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005940 tx=0x7ffb580042f0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.563+0000 7ffb62d9d640 1 -- 172.21.15.1:0/2264894130 reap_dead start 2024-09-16T00:27:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.563+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 shutdown_connections 2024-09-16T00:27:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.563+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 >> 172.21.15.1:0/2264894130 conn(0x7ffb640fc7d0 msgr2=0x7ffb64106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.563+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 shutdown_connections 2024-09-16T00:27:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:40.563+0000 7ffb68e3f640 1 -- 172.21.15.1:0/2264894130 wait complete. 2024-09-16T00:27:40.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:40 smithi019 ceph-mon[35186]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:27:40.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:40 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3504902215' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:27:40.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:40 smithi153 ceph-mon[35538]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:27:40.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:40 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3504902215' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:27:40.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.855+0000 7fd67ed17640 1 -- 172.21.15.1:0/3619302330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd67810c170 msgr2=0x7fd67810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.855+0000 7fd67ed17640 1 --2- 172.21.15.1:0/3619302330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd67810c170 0x7fd67810e610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd670009ca0 tx=0x7fd67002a950 comp rx=0 tx=0).stop 2024-09-16T00:27:40.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 -- 172.21.15.1:0/3619302330 shutdown_connections 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 --2- 172.21.15.1:0/3619302330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd67810c170 0x7fd67810e610 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 --2- 172.21.15.1:0/3619302330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678103660 0x7fd67810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 --2- 172.21.15.1:0/3619302330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678102d40 0x7fd678103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 -- 172.21.15.1:0/3619302330 >> 172.21.15.1:0/3619302330 conn(0x7fd6780fc7a0 msgr2=0x7fd6780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 -- 172.21.15.1:0/3619302330 shutdown_connections 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 -- 172.21.15.1:0/3619302330 wait complete. 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 Processor -- start 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 -- start start 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678102d40 0x7fd67819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678103660 0x7fd67819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd67810c170 0x7fd67819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd678111f30 con 0x7fd678102d40 2024-09-16T00:27:40.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd678111db0 con 0x7fd678103660 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.856+0000 7fd67ed17640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781120b0 con 0x7fd67810c170 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67cab3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678102d40 0x7fd67819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67cab3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678102d40 0x7fd67819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42854/0 (socket says 172.21.15.1:42854) 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67cab3640 1 -- 172.21.15.1:0/1818499127 learned_addr learned my addr 172.21.15.1:0/1818499127 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd677fff640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678103660 0x7fd67819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67d2b4640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd67810c170 0x7fd67819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67cab3640 1 -- 172.21.15.1:0/1818499127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd67810c170 msgr2=0x7fd67819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67cab3640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd67810c170 0x7fd67819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67cab3640 1 -- 172.21.15.1:0/1818499127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678103660 msgr2=0x7fd67819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67cab3640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678103660 0x7fd67819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67cab3640 1 -- 172.21.15.1:0/1818499127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67819f4f0 con 0x7fd678102d40 2024-09-16T00:27:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd677fff640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678103660 0x7fd67819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67cab3640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678102d40 0x7fd67819a5e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd66800c920 tx=0x7fd66800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd675ffb640 1 -- 172.21.15.1:0/1818499127 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668007c70 con 0x7fd678102d40 2024-09-16T00:27:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd675ffb640 1 -- 172.21.15.1:0/1818499127 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd668007e10 con 0x7fd678102d40 2024-09-16T00:27:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67819f7e0 con 0x7fd678102d40 2024-09-16T00:27:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.857+0000 7fd675ffb640 1 -- 172.21.15.1:0/1818499127 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668018890 con 0x7fd678102d40 2024-09-16T00:27:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.858+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6781a70c0 con 0x7fd678102d40 2024-09-16T00:27:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.858+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd678102450 con 0x7fd678102d40 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.860+0000 7fd675ffb640 1 -- 172.21.15.1:0/1818499127 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd668018a30 con 0x7fd678102d40 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.860+0000 7fd675ffb640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3350 0x7fd6600a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.861+0000 7fd677fff640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3350 0x7fd6600a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.861+0000 7fd675ffb640 1 -- 172.21.15.1:0/1818499127 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd6680d0490 con 0x7fd678102d40 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.861+0000 7fd675ffb640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] conn(0x7fd6600ae7a0 0x7fd6600b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.861+0000 7fd675ffb640 1 -- 172.21.15.1:0/1818499127 --> [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6600b1310 con 0x7fd6600ae7a0 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.861+0000 7fd675ffb640 1 -- 172.21.15.1:0/1818499127 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd6680d0840 con 0x7fd678102d40 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.861+0000 7fd677fff640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3350 0x7fd6600a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd67810ea60 tx=0x7fd66c005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.861+0000 7fd67d2b4640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] conn(0x7fd6600ae7a0 0x7fd6600b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.862+0000 7fd67d2b4640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] conn(0x7fd6600ae7a0 0x7fd6600b0c40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.863+0000 7fd675ffb640 1 -- 172.21.15.1:0/1818499127 <== osd.8 v2:172.21.15.153:6816/2981426956 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6600b1310 con 0x7fd6600ae7a0 2024-09-16T00:27:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.879+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 --> [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd67806b070 con 0x7fd6600ae7a0 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd675ffb640 1 -- 172.21.15.1:0/1818499127 <== osd.8 v2:172.21.15.153:6816/2981426956 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd67806b070 con 0x7fd6600ae7a0 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 >> [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] conn(0x7fd6600ae7a0 msgr2=0x7fd6600b0c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd67ed17640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] conn(0x7fd6600ae7a0 0x7fd6600b0c40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3350 msgr2=0x7fd6600a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd67ed17640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3350 0x7fd6600a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd67810ea60 tx=0x7fd66c005d50 comp rx=0 tx=0).stop 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678102d40 msgr2=0x7fd67819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd67ed17640 1 --2- 172.21.15.1:0/1818499127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678102d40 0x7fd67819a5e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd66800c920 tx=0x7fd66800cde0 comp rx=0 tx=0).stop 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd67d2b4640 1 -- 172.21.15.1:0/1818499127 reap_dead start 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 shutdown_connections 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.880+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 >> 172.21.15.1:0/1818499127 conn(0x7fd6780fc7a0 msgr2=0x7fd67810d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.881+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 shutdown_connections 2024-09-16T00:27:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:39.881+0000 7fd67ed17640 1 -- 172.21.15.1:0/1818499127 wait complete. 2024-09-16T00:27:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.406+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/495602079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fb4102db0 msgr2=0x7f6fb4103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.406+0000 7f6fb8bbe640 1 --2- 172.21.15.1:0/495602079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fb4102db0 0x7f6fb4103190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009930 tx=0x7f6fa402a420 comp rx=0 tx=0).stop 2024-09-16T00:27:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.407+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/495602079 shutdown_connections 2024-09-16T00:27:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.407+0000 7f6fb8bbe640 1 --2- 172.21.15.1:0/495602079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb41082e0 0x7f6fb410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.407+0000 7f6fb8bbe640 1 --2- 172.21.15.1:0/495602079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb4103760 0x7f6fb4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.407+0000 7f6fb8bbe640 1 --2- 172.21.15.1:0/495602079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fb4102db0 0x7f6fb4103190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.407+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/495602079 >> 172.21.15.1:0/495602079 conn(0x7f6fb40fc7d0 msgr2=0x7f6fb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.407+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/495602079 shutdown_connections 2024-09-16T00:27:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.407+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/495602079 wait complete. 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.407+0000 7f6fb8bbe640 1 Processor -- start 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.407+0000 7f6fb8bbe640 1 -- start start 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fb8bbe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb4102db0 0x7f6fb4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fb8bbe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fb4103760 0x7f6fb4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fb8bbe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb41082e0 0x7f6fb419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fb8bbe640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb4112110 con 0x7f6fb4103760 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fb8bbe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb4111f90 con 0x7f6fb4102db0 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fb8bbe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb4112290 con 0x7f6fb41082e0 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fabfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fb4103760 0x7f6fb4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fabfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fb4103760 0x7f6fb4198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44842/0 (socket says 172.21.15.1:44842) 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fabfff640 1 -- 172.21.15.1:0/1681212149 learned_addr learned my addr 172.21.15.1:0/1681212149 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fb259c640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb4102db0 0x7f6fb4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fb2d9d640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb41082e0 0x7f6fb419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fabfff640 1 -- 172.21.15.1:0/1681212149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb41082e0 msgr2=0x7f6fb419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fabfff640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fb41082e0 0x7f6fb419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fabfff640 1 -- 172.21.15.1:0/1681212149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb4102db0 msgr2=0x7f6fb4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fabfff640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb4102db0 0x7f6fb4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fabfff640 1 -- 172.21.15.1:0/1681212149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb419d5d0 con 0x7f6fb4103760 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fabfff640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fb4103760 0x7f6fb4198b60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00c970 tx=0x7f6f9c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fab7fe640 1 -- 172.21.15.1:0/1681212149 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c007bb0 con 0x7f6fb4103760 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fab7fe640 1 -- 172.21.15.1:0/1681212149 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9c007d50 con 0x7f6fb4103760 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb419d8c0 con 0x7f6fb4103760 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.408+0000 7f6fab7fe640 1 -- 172.21.15.1:0/1681212149 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c00f450 con 0x7f6fb4103760 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.409+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb41a51a0 con 0x7f6fb4103760 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.409+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6fb4062980 con 0x7f6fb4103760 2024-09-16T00:27:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.411+0000 7f6fab7fe640 1 -- 172.21.15.1:0/1681212149 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f6f9c004f20 con 0x7f6fb4103760 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.411+0000 7f6fab7fe640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fa00a32d0 0x7f6fa00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.411+0000 7f6fb259c640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fa00a32d0 0x7f6fa00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.412+0000 7f6fab7fe640 1 -- 172.21.15.1:0/1681212149 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6f9c0cff10 con 0x7f6fb4103760 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.412+0000 7f6fab7fe640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] conn(0x7f6fa00ae720 0x7f6fa00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.412+0000 7f6fab7fe640 1 -- 172.21.15.1:0/1681212149 --> [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f9c012ec0 con 0x7f6fa00ae720 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.412+0000 7f6fab7fe640 1 -- 172.21.15.1:0/1681212149 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f6f9c0d0360 con 0x7f6fb4103760 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.412+0000 7f6fb2d9d640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] conn(0x7f6fa00ae720 0x7f6fa00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.412+0000 7f6fb259c640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fa00a32d0 0x7f6fa00a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4006010 tx=0x7f6fa40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.412+0000 7f6fb2d9d640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] conn(0x7f6fa00ae720 0x7f6fa00b0bc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.413+0000 7f6fab7fe640 1 -- 172.21.15.1:0/1681212149 <== osd.3 v2:172.21.15.1:6810/2589582331 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f9c012ec0 con 0x7f6fa00ae720 2024-09-16T00:27:41.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.427+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 --> [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6fb4062d80 con 0x7f6fa00ae720 2024-09-16T00:27:41.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.427+0000 7f6fab7fe640 1 -- 172.21.15.1:0/1681212149 <== osd.3 v2:172.21.15.1:6810/2589582331 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6fb4062d80 con 0x7f6fa00ae720 2024-09-16T00:27:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.427+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] conn(0x7f6fa00ae720 msgr2=0x7f6fa00b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.427+0000 7f6fb8bbe640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] conn(0x7f6fa00ae720 0x7f6fa00b0bc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.427+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fa00a32d0 msgr2=0x7f6fa00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.427+0000 7f6fb8bbe640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fa00a32d0 0x7f6fa00a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4006010 tx=0x7f6fa40058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.427+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fb4103760 msgr2=0x7f6fb4198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.427+0000 7f6fb8bbe640 1 --2- 172.21.15.1:0/1681212149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fb4103760 0x7f6fb4198b60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00c970 tx=0x7f6f9c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:27:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.427+0000 7f6fb2d9d640 1 -- 172.21.15.1:0/1681212149 reap_dead start 2024-09-16T00:27:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.428+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 shutdown_connections 2024-09-16T00:27:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.428+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 >> 172.21.15.1:0/1681212149 conn(0x7f6fb40fc7d0 msgr2=0x7f6fb4105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.428+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 shutdown_connections 2024-09-16T00:27:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:41.428+0000 7f6fb8bbe640 1 -- 172.21.15.1:0/1681212149 wait complete. 2024-09-16T00:27:42.399 INFO:teuthology.orchestra.run.smithi001.stdout:244813135917 2024-09-16T00:27:42.400 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-16T00:27:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:42 smithi019 ceph-mon[35186]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-16T00:27:42.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:42 smithi001 ceph-mon[30934]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-16T00:27:42.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:42 smithi153 ceph-mon[35538]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-16T00:27:42.870 INFO:teuthology.orchestra.run.smithi001.stdout:158913790011 2024-09-16T00:27:42.870 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T00:27:43.844 INFO:teuthology.orchestra.run.smithi001.stdout:107374182470 2024-09-16T00:27:43.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T00:27:43.846 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574928 for osd.1 2024-09-16T00:27:43.846 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:44.283 INFO:teuthology.orchestra.run.smithi001.stdout:274877906983 2024-09-16T00:27:44.284 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-16T00:27:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:44 smithi019 ceph-mon[35186]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T00:27:44.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:44 smithi001 ceph-mon[30934]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T00:27:44.707 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:44.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:44 smithi153 ceph-mon[35538]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T00:27:44.714 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:44.719 INFO:teuthology.orchestra.run.smithi001.stdout:296352743459 2024-09-16T00:27:44.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-16T00:27:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:44.788 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 -- 172.21.15.1:0/3112603311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104100e20 msgr2=0x7f8104101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 --2- 172.21.15.1:0/3112603311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104100e20 0x7f8104101200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8100009900 tx=0x7f810002a480 comp rx=0 tx=0).stop 2024-09-16T00:27:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 -- 172.21.15.1:0/3112603311 shutdown_connections 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 --2- 172.21.15.1:0/3112603311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8104102ef0 0x7f810410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 --2- 172.21.15.1:0/3112603311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8104102570 0x7f81041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 --2- 172.21.15.1:0/3112603311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104100e20 0x7f8104101200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 -- 172.21.15.1:0/3112603311 >> 172.21.15.1:0/3112603311 conn(0x7f81040fc7d0 msgr2=0x7f81040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 -- 172.21.15.1:0/3112603311 shutdown_connections 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 -- 172.21.15.1:0/3112603311 wait complete. 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.463+0000 7f810c50e640 1 Processor -- start 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810c50e640 1 -- start start 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810c50e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8104100e20 0x7f8104198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810c50e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8104102570 0x7f8104198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810c50e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104102ef0 0x7f810419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810c50e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8104110f80 con 0x7f8104102ef0 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810c50e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8104110e00 con 0x7f8104102570 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810c50e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8104111100 con 0x7f8104100e20 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810aaab640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104102ef0 0x7f810419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810aaab640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104102ef0 0x7f810419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44868/0 (socket says 172.21.15.1:44868) 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810aaab640 1 -- 172.21.15.1:0/2351511131 learned_addr learned my addr 172.21.15.1:0/2351511131 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f8109aa9640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8104102570 0x7f8104198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810a2aa640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8104100e20 0x7f8104198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810aaab640 1 -- 172.21.15.1:0/2351511131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8104100e20 msgr2=0x7f8104198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810aaab640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8104100e20 0x7f8104198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810aaab640 1 -- 172.21.15.1:0/2351511131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8104102570 msgr2=0x7f8104198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810aaab640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8104102570 0x7f8104198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810aaab640 1 -- 172.21.15.1:0/2351511131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810419d4b0 con 0x7f8104102ef0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.464+0000 7f810aaab640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104102ef0 0x7f810419cdd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00cad0 tx=0x7f80fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.465+0000 7f80fb7fe640 1 -- 172.21.15.1:0/2351511131 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00ae20 con 0x7f8104102ef0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.465+0000 7f80fb7fe640 1 -- 172.21.15.1:0/2351511131 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc005020 con 0x7f8104102ef0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.465+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810419d7a0 con 0x7f8104102ef0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.465+0000 7f80fb7fe640 1 -- 172.21.15.1:0/2351511131 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00f6f0 con 0x7f8104102ef0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.465+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81041a5080 con 0x7f8104102ef0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.466+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8104101200 con 0x7f8104102ef0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.468+0000 7f80fb7fe640 1 -- 172.21.15.1:0/2351511131 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f80fc0051d0 con 0x7f8104102ef0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.468+0000 7f80fb7fe640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80f00a32d0 0x7f80f00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.468+0000 7f810a2aa640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80f00a32d0 0x7f80f00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.469+0000 7f80fb7fe640 1 -- 172.21.15.1:0/2351511131 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f80fc0d0200 con 0x7f8104102ef0 2024-09-16T00:27:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.469+0000 7f810a2aa640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80f00a32d0 0x7f80f00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8100000c00 tx=0x7f81000058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.472+0000 7f80fb7fe640 1 -- 172.21.15.1:0/2351511131 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80fc014030 con 0x7f8104102ef0 2024-09-16T00:27:45.605 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:45 smithi001 ceph-mon[30934]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:27:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.648+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8104199860 con 0x7f8104102ef0 2024-09-16T00:27:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.648+0000 7f80fb7fe640 1 -- 172.21.15.1:0/2351511131 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8104199860 con 0x7f8104102ef0 2024-09-16T00:27:45.649 INFO:teuthology.orchestra.run.smithi001.stdout:77309411405 2024-09-16T00:27:45.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.650+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80f00a32d0 msgr2=0x7f80f00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.650+0000 7f810c50e640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80f00a32d0 0x7f80f00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8100000c00 tx=0x7f81000058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.650+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104102ef0 msgr2=0x7f810419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.650+0000 7f810c50e640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104102ef0 0x7f810419cdd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00cad0 tx=0x7f80fc00cf90 comp rx=0 tx=0).stop 2024-09-16T00:27:45.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.650+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 shutdown_connections 2024-09-16T00:27:45.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.651+0000 7f810c50e640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104102ef0 0x7f810419cdd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.651+0000 7f810c50e640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8104102570 0x7f8104198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.651+0000 7f810c50e640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80f00a32d0 0x7f80f00a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.651+0000 7f810c50e640 1 --2- 172.21.15.1:0/2351511131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8104100e20 0x7f8104198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.651+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 >> 172.21.15.1:0/2351511131 conn(0x7f81040fc7d0 msgr2=0x7f810406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.651+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 shutdown_connections 2024-09-16T00:27:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.651+0000 7f810c50e640 1 -- 172.21.15.1:0/2351511131 wait complete. 2024-09-16T00:27:45.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.692+0000 7f7df1ba0640 1 -- 172.21.15.1:0/4285648819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec102bf0 msgr2=0x7f7dec102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.692+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/4285648819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec102bf0 0x7f7dec102fd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009900 tx=0x7f7de002a480 comp rx=0 tx=0).stop 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.692+0000 7f7df1ba0640 1 -- 172.21.15.1:0/4285648819 shutdown_connections 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.692+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/4285648819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dec10bfd0 0x7f7dec10e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.692+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/4285648819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dec103510 0x7f7dec10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.692+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/4285648819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec102bf0 0x7f7dec102fd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.692+0000 7f7df1ba0640 1 -- 172.21.15.1:0/4285648819 >> 172.21.15.1:0/4285648819 conn(0x7f7dec0fc610 msgr2=0x7f7dec0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.692+0000 7f7df1ba0640 1 -- 172.21.15.1:0/4285648819 shutdown_connections 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.692+0000 7f7df1ba0640 1 -- 172.21.15.1:0/4285648819 wait complete. 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df1ba0640 1 Processor -- start 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df1ba0640 1 -- start start 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df1ba0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dec102bf0 0x7f7dec1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df1ba0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dec103510 0x7f7dec1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df1ba0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec10bfd0 0x7f7dec19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df1ba0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec111e00 con 0x7f7dec10bfd0 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df1ba0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec111c80 con 0x7f7dec102bf0 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df1ba0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec111f80 con 0x7f7dec103510 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df139f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec10bfd0 0x7f7dec19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df139f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec10bfd0 0x7f7dec19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44888/0 (socket says 172.21.15.1:44888) 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df139f640 1 -- 172.21.15.1:0/3019407932 learned_addr learned my addr 172.21.15.1:0/3019407932 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df0b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dec102bf0 0x7f7dec1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7debfff640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dec103510 0x7f7dec1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df139f640 1 -- 172.21.15.1:0/3019407932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dec103510 msgr2=0x7f7dec1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df139f640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dec103510 0x7f7dec1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df139f640 1 -- 172.21.15.1:0/3019407932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dec102bf0 msgr2=0x7f7dec1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df139f640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dec102bf0 0x7f7dec1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df139f640 1 -- 172.21.15.1:0/3019407932 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec19d250 con 0x7f7dec10bfd0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.693+0000 7f7df0b9e640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dec102bf0 0x7f7dec1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.694+0000 7f7df139f640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec10bfd0 0x7f7dec19cb70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7de400cad0 tx=0x7f7de400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.694+0000 7f7de9ffb640 1 -- 172.21.15.1:0/3019407932 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de400ae20 con 0x7f7dec10bfd0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.694+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec19d540 con 0x7f7dec10bfd0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.694+0000 7f7de9ffb640 1 -- 172.21.15.1:0/3019407932 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de4005020 con 0x7f7dec10bfd0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.694+0000 7f7de9ffb640 1 -- 172.21.15.1:0/3019407932 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de400f6f0 con 0x7f7dec10bfd0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.694+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1a4e20 con 0x7f7dec10bfd0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.695+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec102fd0 con 0x7f7dec10bfd0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.696+0000 7f7de9ffb640 1 -- 172.21.15.1:0/3019407932 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f7de40051d0 con 0x7f7dec10bfd0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.697+0000 7f7de9ffb640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7ddc0a32d0 0x7f7ddc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.697+0000 7f7df0b9e640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7ddc0a32d0 0x7f7ddc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.697+0000 7f7de9ffb640 1 -- 172.21.15.1:0/3019407932 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7de40cfe80 con 0x7f7dec10bfd0 2024-09-16T00:27:45.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.697+0000 7f7df0b9e640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7ddc0a32d0 0x7f7ddc0a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000c00 tx=0x7f7de0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:45.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.701+0000 7f7de9ffb640 1 -- 172.21.15.1:0/3019407932 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7de4014030 con 0x7f7dec10bfd0 2024-09-16T00:27:45.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:45 smithi153 ceph-mon[35538]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:27:45.719 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:45.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:45 smithi019 ceph-mon[35186]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:27:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.879+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7dec02d7c0 con 0x7f7dec10bfd0 2024-09-16T00:27:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.880+0000 7f7de9ffb640 1 -- 172.21.15.1:0/3019407932 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7de409bf90 con 0x7f7dec10bfd0 2024-09-16T00:27:45.880 INFO:teuthology.orchestra.run.smithi001.stdout:34359738454 2024-09-16T00:27:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.882+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7ddc0a32d0 msgr2=0x7f7ddc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.882+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7ddc0a32d0 0x7f7ddc0a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000c00 tx=0x7f7de0031510 comp rx=0 tx=0).stop 2024-09-16T00:27:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.882+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec10bfd0 msgr2=0x7f7dec19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.882+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec10bfd0 0x7f7dec19cb70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7de400cad0 tx=0x7f7de400cf90 comp rx=0 tx=0).stop 2024-09-16T00:27:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.883+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 shutdown_connections 2024-09-16T00:27:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.883+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dec10bfd0 0x7f7dec19cb70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.883+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7ddc0a32d0 0x7f7ddc0a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.883+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dec103510 0x7f7dec1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.883+0000 7f7df1ba0640 1 --2- 172.21.15.1:0/3019407932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dec102bf0 0x7f7dec1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.883+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 >> 172.21.15.1:0/3019407932 conn(0x7f7dec0fc610 msgr2=0x7f7dec0fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.883+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 shutdown_connections 2024-09-16T00:27:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:45.883+0000 7f7df1ba0640 1 -- 172.21.15.1:0/3019407932 wait complete. 2024-09-16T00:27:46.596 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:46 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2351511131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:27:46.596 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:46 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3019407932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:27:46.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:46 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2351511131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:27:46.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:46 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3019407932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:27:46.626 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738454 for osd.0 2024-09-16T00:27:46.626 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:46.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:46 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2351511131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:27:46.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:46 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3019407932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:27:47.064 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411405 for osd.2 2024-09-16T00:27:47.064 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:47.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:47 smithi001 ceph-mon[30934]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:27:47.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:47 smithi019 ceph-mon[35186]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:27:47.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:47 smithi153 ceph-mon[35538]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:27:47.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.700+0000 7f62a132a640 1 -- 172.21.15.1:0/3685755096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 msgr2=0x7f629c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:47.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.700+0000 7f62a132a640 1 --2- 172.21.15.1:0/3685755096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 0x7f629c1021c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f628c009900 tx=0x7f628c02a410 comp rx=0 tx=0).stop 2024-09-16T00:27:47.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.700+0000 7f62a132a640 1 -- 172.21.15.1:0/3685755096 shutdown_connections 2024-09-16T00:27:47.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.700+0000 7f62a132a640 1 --2- 172.21.15.1:0/3685755096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f629c106f10 0x7f629c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.700+0000 7f62a132a640 1 --2- 172.21.15.1:0/3685755096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f629c102790 0x7f629c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.700+0000 7f62a132a640 1 --2- 172.21.15.1:0/3685755096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 0x7f629c1021c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.700+0000 7f62a132a640 1 -- 172.21.15.1:0/3685755096 >> 172.21.15.1:0/3685755096 conn(0x7f629c0fd8c0 msgr2=0x7f629c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.701+0000 7f62a132a640 1 -- 172.21.15.1:0/3685755096 shutdown_connections 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.701+0000 7f62a132a640 1 -- 172.21.15.1:0/3685755096 wait complete. 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.701+0000 7f62a132a640 1 Processor -- start 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.701+0000 7f62a132a640 1 -- start start 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.701+0000 7f62a132a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 0x7f629c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.701+0000 7f62a132a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f629c102790 0x7f629c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f62a132a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f629c106f10 0x7f629c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f62a132a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f629c111f70 con 0x7f629c101de0 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f62a132a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f629c111df0 con 0x7f629c106f10 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f62a132a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f629c1120f0 con 0x7f629c102790 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 0x7f629c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:47.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 0x7f629c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44900/0 (socket says 172.21.15.1:44900) 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629affd640 1 -- 172.21.15.1:0/3865949566 learned_addr learned my addr 172.21.15.1:0/3865949566 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629b7fe640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f629c106f10 0x7f629c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f6293fff640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f629c102790 0x7f629c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629affd640 1 -- 172.21.15.1:0/3865949566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f629c102790 msgr2=0x7f629c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629affd640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f629c102790 0x7f629c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629affd640 1 -- 172.21.15.1:0/3865949566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f629c106f10 msgr2=0x7f629c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629affd640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f629c106f10 0x7f629c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629affd640 1 -- 172.21.15.1:0/3865949566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c19d3f0 con 0x7f629c101de0 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f6293fff640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f629c102790 0x7f629c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f629affd640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 0x7f629c198440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f628c006010 tx=0x7f628c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f6298ff9640 1 -- 172.21.15.1:0/3865949566 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c03f030 con 0x7f629c101de0 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629c19d680 con 0x7f629c101de0 2024-09-16T00:27:47.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f6298ff9640 1 -- 172.21.15.1:0/3865949566 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f628c003ce0 con 0x7f629c101de0 2024-09-16T00:27:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.702+0000 7f6298ff9640 1 -- 172.21.15.1:0/3865949566 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c035070 con 0x7f629c101de0 2024-09-16T00:27:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.703+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629c074420 con 0x7f629c101de0 2024-09-16T00:27:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.704+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f629c1032c0 con 0x7f629c101de0 2024-09-16T00:27:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.705+0000 7f6298ff9640 1 -- 172.21.15.1:0/3865949566 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f628c042030 con 0x7f629c101de0 2024-09-16T00:27:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.705+0000 7f6298ff9640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62880a32d0 0x7f62880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.706+0000 7f6298ff9640 1 -- 172.21.15.1:0/3865949566 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f628c0f0530 con 0x7f629c101de0 2024-09-16T00:27:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.706+0000 7f6293fff640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62880a32d0 0x7f62880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.706+0000 7f6293fff640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62880a32d0 0x7f62880a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f629c0fdce0 tx=0x7f6284005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.710+0000 7f6298ff9640 1 -- 172.21.15.1:0/3865949566 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f628c0bcd50 con 0x7f629c101de0 2024-09-16T00:27:47.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.889+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f629c1997a0 con 0x7f629c101de0 2024-09-16T00:27:47.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.889+0000 7f6298ff9640 1 -- 172.21.15.1:0/3865949566 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f628c0bc650 con 0x7f629c101de0 2024-09-16T00:27:47.890 INFO:teuthology.orchestra.run.smithi001.stdout:137438953539 2024-09-16T00:27:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.891+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62880a32d0 msgr2=0x7f62880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.891+0000 7f62a132a640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62880a32d0 0x7f62880a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f629c0fdce0 tx=0x7f6284005e50 comp rx=0 tx=0).stop 2024-09-16T00:27:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.891+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 msgr2=0x7f629c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.891+0000 7f62a132a640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 0x7f629c198440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f628c006010 tx=0x7f628c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.891+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 shutdown_connections 2024-09-16T00:27:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.891+0000 7f62a132a640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f629c106f10 0x7f629c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.892+0000 7f62a132a640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62880a32d0 0x7f62880a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.892+0000 7f62a132a640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f629c102790 0x7f629c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.892+0000 7f62a132a640 1 --2- 172.21.15.1:0/3865949566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f629c101de0 0x7f629c198440 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:47.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.892+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 >> 172.21.15.1:0/3865949566 conn(0x7f629c0fd8c0 msgr2=0x7f629c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:47.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.892+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 shutdown_connections 2024-09-16T00:27:47.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.892+0000 7f62a132a640 1 -- 172.21.15.1:0/3865949566 wait complete. 2024-09-16T00:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.993+0000 7f0764495640 1 -- 172.21.15.1:0/4158762626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c100e20 msgr2=0x7f075c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.993+0000 7f0764495640 1 --2- 172.21.15.1:0/4158762626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c100e20 0x7f075c101200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0758009900 tx=0x7f075802a480 comp rx=0 tx=0).stop 2024-09-16T00:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.993+0000 7f0764495640 1 -- 172.21.15.1:0/4158762626 shutdown_connections 2024-09-16T00:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.993+0000 7f0764495640 1 --2- 172.21.15.1:0/4158762626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c102ef0 0x7f075c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.993+0000 7f0764495640 1 --2- 172.21.15.1:0/4158762626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c102570 0x7f075c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.993+0000 7f0764495640 1 --2- 172.21.15.1:0/4158762626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c100e20 0x7f075c101200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.993+0000 7f0764495640 1 -- 172.21.15.1:0/4158762626 >> 172.21.15.1:0/4158762626 conn(0x7f075c0fc7d0 msgr2=0x7f075c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.993+0000 7f0764495640 1 -- 172.21.15.1:0/4158762626 shutdown_connections 2024-09-16T00:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.993+0000 7f0764495640 1 -- 172.21.15.1:0/4158762626 wait complete. 2024-09-16T00:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.994+0000 7f0764495640 1 Processor -- start 2024-09-16T00:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.994+0000 7f0764495640 1 -- start start 2024-09-16T00:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.994+0000 7f0764495640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c100e20 0x7f075c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.994+0000 7f0764495640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c102570 0x7f075c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.994+0000 7f0764495640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c102ef0 0x7f075c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.994+0000 7f0764495640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f075c110f80 con 0x7f075c102570 2024-09-16T00:27:48.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.994+0000 7f0764495640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f075c110e00 con 0x7f075c100e20 2024-09-16T00:27:48.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.994+0000 7f0764495640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f075c111100 con 0x7f075c102ef0 2024-09-16T00:27:48.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0761a30640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c102570 0x7f075c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:48.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0761a30640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c102570 0x7f075c198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44914/0 (socket says 172.21.15.1:44914) 2024-09-16T00:27:48.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0761a30640 1 -- 172.21.15.1:0/4142434067 learned_addr learned my addr 172.21.15.1:0/4142434067 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:48.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0762231640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c100e20 0x7f075c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:48.271 INFO:teuthology.orchestra.run.smithi001.stdout:326417514527 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0762a32640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c102ef0 0x7f075c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0761a30640 1 -- 172.21.15.1:0/4142434067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c102ef0 msgr2=0x7f075c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0761a30640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c102ef0 0x7f075c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0761a30640 1 -- 172.21.15.1:0/4142434067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c100e20 msgr2=0x7f075c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0761a30640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c100e20 0x7f075c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0761a30640 1 -- 172.21.15.1:0/4142434067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c19d4f0 con 0x7f075c102570 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0761a30640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c102570 0x7f075c198a80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f074c00d950 tx=0x7f074c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f07537fe640 1 -- 172.21.15.1:0/4142434067 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c017070 con 0x7f075c102570 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f07537fe640 1 -- 172.21.15.1:0/4142434067 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074c009c30 con 0x7f075c102570 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0764495640 1 -- 172.21.15.1:0/4142434067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c19d7e0 con 0x7f075c102570 2024-09-16T00:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f07537fe640 1 -- 172.21.15.1:0/4142434067 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c00f6b0 con 0x7f075c102570 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.995+0000 7f0764495640 1 -- 172.21.15.1:0/4142434067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a50c0 con 0x7f075c102570 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.996+0000 7f07517fa640 1 -- 172.21.15.1:0/4142434067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0744005290 con 0x7f075c102570 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.998+0000 7f07537fe640 1 -- 172.21.15.1:0/4142434067 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f074c0109d0 con 0x7f075c102570 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:47.999+0000 7f07537fe640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f07480a32d0 0x7f07480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.000+0000 7f0762231640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f07480a32d0 0x7f07480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.000+0000 7f07537fe640 1 -- 172.21.15.1:0/4142434067 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f074c0cfae0 con 0x7f075c102570 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.000+0000 7f0762231640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f07480a32d0 0x7f07480a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0758009230 tx=0x7f07580058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.002+0000 7f07537fe640 1 -- 172.21.15.1:0/4142434067 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f074c009de0 con 0x7f075c102570 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.180+0000 7f07517fa640 1 -- 172.21.15.1:0/4142434067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f0744005850 con 0x7f075c102570 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.181+0000 7f07537fe640 1 -- 172.21.15.1:0/4142434067 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f074c09c2a0 con 0x7f075c102570 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 -- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f07480a32d0 msgr2=0x7f07480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f07480a32d0 0x7f07480a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0758009230 tx=0x7f07580058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 -- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c102570 msgr2=0x7f075c198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:48.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c102570 0x7f075c198a80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f074c00d950 tx=0x7f074c00de10 comp rx=0 tx=0).stop 2024-09-16T00:27:48.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 -- 172.21.15.1:0/4142434067 shutdown_connections 2024-09-16T00:27:48.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f07480a32d0 0x7f07480a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c102ef0 0x7f075c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f075c102570 0x7f075c198a80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 --2- 172.21.15.1:0/4142434067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c100e20 0x7f075c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:48.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 -- 172.21.15.1:0/4142434067 >> 172.21.15.1:0/4142434067 conn(0x7f075c0fc7d0 msgr2=0x7f075c06a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:48.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 -- 172.21.15.1:0/4142434067 shutdown_connections 2024-09-16T00:27:48.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:48.183+0000 7f07517fa640 1 -- 172.21.15.1:0/4142434067 wait complete. 2024-09-16T00:27:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:48 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3865949566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:27:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:48 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4142434067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:27:48.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:48 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3865949566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:27:48.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:48 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4142434067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:27:48.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:48 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3865949566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:27:48.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:48 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4142434067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:27:48.939 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514526 got 326417514527 for osd.11 2024-09-16T00:27:48.939 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:49.159 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:49.171 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:49.171 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:49.176 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:49.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:49 smithi001 ceph-mon[30934]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:27:49.653 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953536 got 137438953539 for osd.4 2024-09-16T00:27:49.653 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:49.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:49 smithi153 ceph-mon[35538]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:27:49.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:49 smithi019 ceph-mon[35186]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:27:50.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.657+0000 7f6de7ed9640 1 -- 172.21.15.1:0/1341436495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0100a80 msgr2=0x7f6de0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.657+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/1341436495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0100a80 0x7f6de0100ee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009930 tx=0x7f6dd002a420 comp rx=0 tx=0).stop 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.657+0000 7f6de7ed9640 1 -- 172.21.15.1:0/1341436495 shutdown_connections 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.657+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/1341436495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de0101420 0x7f6de010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.657+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/1341436495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0100a80 0x7f6de0100ee0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.657+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/1341436495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de0106a80 0x7f6de0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.657+0000 7f6de7ed9640 1 -- 172.21.15.1:0/1341436495 >> 172.21.15.1:0/1341436495 conn(0x7f6de00fc7a0 msgr2=0x7f6de00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.657+0000 7f6de7ed9640 1 -- 172.21.15.1:0/1341436495 shutdown_connections 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.657+0000 7f6de7ed9640 1 -- 172.21.15.1:0/1341436495 wait complete. 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de7ed9640 1 Processor -- start 2024-09-16T00:27:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de7ed9640 1 -- start start 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de7ed9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de0100a80 0x7f6de0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de7ed9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0101420 0x7f6de01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de7ed9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de0106a80 0x7f6de01a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de7ed9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0110ea0 con 0x7f6de0101420 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de7ed9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0110d20 con 0x7f6de0106a80 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de7ed9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0111020 con 0x7f6de0100a80 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de5474640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0101420 0x7f6de01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de5474640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0101420 0x7f6de01998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58582/0 (socket says 172.21.15.1:58582) 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de5474640 1 -- 172.21.15.1:0/2546447220 learned_addr learned my addr 172.21.15.1:0/2546447220 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de5c75640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de0100a80 0x7f6de0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de5474640 1 -- 172.21.15.1:0/2546447220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de0100a80 msgr2=0x7f6de0199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.658+0000 7f6de6476640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de0106a80 0x7f6de01a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6de5474640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de0100a80 0x7f6de0199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6de5474640 1 -- 172.21.15.1:0/2546447220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de0106a80 msgr2=0x7f6de01a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6de5474640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de0106a80 0x7f6de01a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6de5474640 1 -- 172.21.15.1:0/2546447220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de01a4930 con 0x7f6de0101420 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6de5c75640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de0100a80 0x7f6de0199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6de5474640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0101420 0x7f6de01998b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009230 tx=0x7f6dd0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6dd6ffd640 1 -- 172.21.15.1:0/2546447220 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd003f030 con 0x7f6de0101420 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6dd6ffd640 1 -- 172.21.15.1:0/2546447220 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd0003ad0 con 0x7f6de0101420 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de01a4b00 con 0x7f6de0101420 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6dd6ffd640 1 -- 172.21.15.1:0/2546447220 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0035070 con 0x7f6de0101420 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.659+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de01a4fb0 con 0x7f6de0101420 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.660+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de0100ee0 con 0x7f6de0101420 2024-09-16T00:27:50.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.661+0000 7f6dd6ffd640 1 -- 172.21.15.1:0/2546447220 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f6dd0042030 con 0x7f6de0101420 2024-09-16T00:27:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.662+0000 7f6dd6ffd640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6dcc0a3350 0x7f6dcc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.662+0000 7f6de5c75640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6dcc0a3350 0x7f6dcc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.662+0000 7f6dd6ffd640 1 -- 172.21.15.1:0/2546447220 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6dd00f0430 con 0x7f6de0101420 2024-09-16T00:27:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.662+0000 7f6de5c75640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6dcc0a3350 0x7f6dcc0a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006fd0 tx=0x7f6ddc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.666+0000 7f6dd6ffd640 1 -- 172.21.15.1:0/2546447220 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6dd00bcb90 con 0x7f6de0101420 2024-09-16T00:27:50.681 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:50.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.846+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6de0109010 con 0x7f6de0101420 2024-09-16T00:27:50.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.846+0000 7f6dd6ffd640 1 -- 172.21.15.1:0/2546447220 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6dd00bc490 con 0x7f6de0101420 2024-09-16T00:27:50.846 INFO:teuthology.orchestra.run.smithi001.stdout:193273528377 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6dcc0a3350 msgr2=0x7f6dcc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6dcc0a3350 0x7f6dcc0a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006fd0 tx=0x7f6ddc008040 comp rx=0 tx=0).stop 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0101420 msgr2=0x7f6de01998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0101420 0x7f6de01998b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009230 tx=0x7f6dd0004270 comp rx=0 tx=0).stop 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 shutdown_connections 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de0106a80 0x7f6de01a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de0101420 0x7f6de01998b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6dcc0a3350 0x7f6dcc0a5810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 --2- 172.21.15.1:0/2546447220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de0100a80 0x7f6de0199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 >> 172.21.15.1:0/2546447220 conn(0x7f6de00fc7a0 msgr2=0x7f6de00fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:50.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 shutdown_connections 2024-09-16T00:27:50.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.848+0000 7f6de7ed9640 1 -- 172.21.15.1:0/2546447220 wait complete. 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.931+0000 7ff14ef15640 1 -- 172.21.15.1:0/1044813319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148101de0 msgr2=0x7ff1481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.931+0000 7ff14ef15640 1 --2- 172.21.15.1:0/1044813319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148101de0 0x7ff1481021c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff138009900 tx=0x7ff13802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.931+0000 7ff14ef15640 1 -- 172.21.15.1:0/1044813319 shutdown_connections 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.931+0000 7ff14ef15640 1 --2- 172.21.15.1:0/1044813319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148106f10 0x7ff14810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.931+0000 7ff14ef15640 1 --2- 172.21.15.1:0/1044813319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff148102790 0x7ff1481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.931+0000 7ff14ef15640 1 --2- 172.21.15.1:0/1044813319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148101de0 0x7ff1481021c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.931+0000 7ff14ef15640 1 -- 172.21.15.1:0/1044813319 >> 172.21.15.1:0/1044813319 conn(0x7ff1480fd8c0 msgr2=0x7ff1480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.931+0000 7ff14ef15640 1 -- 172.21.15.1:0/1044813319 shutdown_connections 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.931+0000 7ff14ef15640 1 -- 172.21.15.1:0/1044813319 wait complete. 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14ef15640 1 Processor -- start 2024-09-16T00:27:50.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14ef15640 1 -- start start 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14ef15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148101de0 0x7ff148198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14ef15640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff148102790 0x7ff148198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14ef15640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148106f10 0x7ff14819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14ef15640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff148111fc0 con 0x7ff148106f10 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14ef15640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff148111e40 con 0x7ff148101de0 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14ef15640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff148112140 con 0x7ff148102790 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14d4b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148106f10 0x7ff14819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14d4b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148106f10 0x7ff14819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58594/0 (socket says 172.21.15.1:58594) 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.932+0000 7ff14d4b2640 1 -- 172.21.15.1:0/2202129353 learned_addr learned my addr 172.21.15.1:0/2202129353 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14ccb1640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148101de0 0x7ff148198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff147fff640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff148102790 0x7ff148198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14d4b2640 1 -- 172.21.15.1:0/2202129353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff148102790 msgr2=0x7ff148198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14d4b2640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff148102790 0x7ff148198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14d4b2640 1 -- 172.21.15.1:0/2202129353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148101de0 msgr2=0x7ff148198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14d4b2640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148101de0 0x7ff148198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14d4b2640 1 -- 172.21.15.1:0/2202129353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14819d2c0 con 0x7ff148106f10 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14ccb1640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148101de0 0x7ff148198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14d4b2640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148106f10 0x7ff14819cbe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff14000cce0 tx=0x7ff14000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff145ffb640 1 -- 172.21.15.1:0/2202129353 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14000f450 con 0x7ff148106f10 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14819d5b0 con 0x7ff148106f10 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff145ffb640 1 -- 172.21.15.1:0/2202129353 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff14000fad0 con 0x7ff148106f10 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff145ffb640 1 -- 172.21.15.1:0/2202129353 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140018c10 con 0x7ff148106f10 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.933+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1481a4e90 con 0x7ff148106f10 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.934+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1481021c0 con 0x7ff148106f10 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.935+0000 7ff145ffb640 1 -- 172.21.15.1:0/2202129353 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ff14000f5f0 con 0x7ff148106f10 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.936+0000 7ff145ffb640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1300a32d0 0x7ff1300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.936+0000 7ff14ccb1640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1300a32d0 0x7ff1300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.936+0000 7ff145ffb640 1 -- 172.21.15.1:0/2202129353 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff1400d05f0 con 0x7ff148106f10 2024-09-16T00:27:50.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.937+0000 7ff14ccb1640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1300a32d0 0x7ff1300a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff138005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:50.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.940+0000 7ff145ffb640 1 -- 172.21.15.1:0/2202129353 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff140014030 con 0x7ff148106f10 2024-09-16T00:27:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.119+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff1481a50f0 con 0x7ff148106f10 2024-09-16T00:27:51.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.119+0000 7ff145ffb640 1 -- 172.21.15.1:0/2202129353 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff14009cdb0 con 0x7ff148106f10 2024-09-16T00:27:51.120 INFO:teuthology.orchestra.run.smithi001.stdout:107374182472 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.121+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1300a32d0 msgr2=0x7ff1300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.121+0000 7ff14ef15640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1300a32d0 0x7ff1300a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff138005a20 comp rx=0 tx=0).stop 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.121+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148106f10 msgr2=0x7ff14819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.121+0000 7ff14ef15640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148106f10 0x7ff14819cbe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff14000cce0 tx=0x7ff14000a620 comp rx=0 tx=0).stop 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.121+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 shutdown_connections 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.121+0000 7ff14ef15640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff148106f10 0x7ff14819cbe0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.121+0000 7ff14ef15640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1300a32d0 0x7ff1300a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.121+0000 7ff14ef15640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff148102790 0x7ff148198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.121+0000 7ff14ef15640 1 --2- 172.21.15.1:0/2202129353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148101de0 0x7ff148198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.122+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 >> 172.21.15.1:0/2202129353 conn(0x7ff1480fd8c0 msgr2=0x7ff1480fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.122+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 shutdown_connections 2024-09-16T00:27:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:51.122+0000 7ff14ef15640 1 -- 172.21.15.1:0/2202129353 wait complete. 2024-09-16T00:27:51.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:51 smithi001 ceph-mon[30934]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:27:51.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:51 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/754027683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:27:51.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:51 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2546447220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:27:51.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:51 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2202129353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:27:51.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:51 smithi153 ceph-mon[35538]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:27:51.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:51 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/754027683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:27:51.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:51 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2546447220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:27:51.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:51 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2202129353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:27:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:51 smithi019 ceph-mon[35186]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:27:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:51 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/754027683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:27:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:51 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2546447220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:27:51.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:51 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2202129353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:27:51.997 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182470 got 107374182472 for osd.3 2024-09-16T00:27:51.997 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:52.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.391+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/1003972727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a281008d0 msgr2=0x7f0a28100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.391+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/1003972727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a281008d0 0x7f0a28100cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009ca0 tx=0x7f0a2002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.391+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/1003972727 shutdown_connections 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.391+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/1003972727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a2810e480 0x7f0a28110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.391+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/1003972727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a28101280 0x7f0a2810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.391+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/1003972727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a281008d0 0x7f0a28100cb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.391+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/1003972727 >> 172.21.15.1:0/1003972727 conn(0x7f0a280fc7a0 msgr2=0x7f0a280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.391+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/1003972727 shutdown_connections 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.391+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/1003972727 wait complete. 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2f9a6640 1 Processor -- start 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2f9a6640 1 -- start start 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2f9a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a281008d0 0x7f0a2819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:52.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2f9a6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a28101280 0x7f0a2819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2f9a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a2810e480 0x7f0a2819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2f9a6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28114190 con 0x7f0a2810e480 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2f9a6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28114010 con 0x7f0a281008d0 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2f9a6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28114310 con 0x7f0a28101280 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2df43640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a2810e480 0x7f0a2819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2df43640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a2810e480 0x7f0a2819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58604/0 (socket says 172.21.15.1:58604) 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2df43640 1 -- 172.21.15.1:0/223887666 learned_addr learned my addr 172.21.15.1:0/223887666 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2d742640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a281008d0 0x7f0a2819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.392+0000 7f0a2cf41640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a28101280 0x7f0a2819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a2df43640 1 -- 172.21.15.1:0/223887666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a28101280 msgr2=0x7f0a2819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a2df43640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a28101280 0x7f0a2819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a2df43640 1 -- 172.21.15.1:0/223887666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a281008d0 msgr2=0x7f0a2819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a2df43640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a281008d0 0x7f0a2819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a2df43640 1 -- 172.21.15.1:0/223887666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2819f620 con 0x7f0a2810e480 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a2d742640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a281008d0 0x7f0a2819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a2df43640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a2810e480 0x7f0a2819ef40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d6a0 tx=0x7f0a1800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a267fc640 1 -- 172.21.15.1:0/223887666 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18017070 con 0x7f0a2810e480 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a267fc640 1 -- 172.21.15.1:0/223887666 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a18009820 con 0x7f0a2810e480 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2819f910 con 0x7f0a2810e480 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a267fc640 1 -- 172.21.15.1:0/223887666 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18004ce0 con 0x7f0a2810e480 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.393+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a281a71f0 con 0x7f0a2810e480 2024-09-16T00:27:52.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.394+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a28100cb0 con 0x7f0a2810e480 2024-09-16T00:27:52.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.396+0000 7f0a267fc640 1 -- 172.21.15.1:0/223887666 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f0a180099d0 con 0x7f0a2810e480 2024-09-16T00:27:52.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.396+0000 7f0a267fc640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a100a3350 0x7f0a100a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:52.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.396+0000 7f0a267fc640 1 -- 172.21.15.1:0/223887666 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0a180cfae0 con 0x7f0a2810e480 2024-09-16T00:27:52.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.396+0000 7f0a2d742640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a100a3350 0x7f0a100a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:52.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.397+0000 7f0a2d742640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a100a3350 0x7f0a100a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0a20006010 tx=0x7f0a20005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:52.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.399+0000 7f0a267fc640 1 -- 172.21.15.1:0/223887666 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a1809bcd0 con 0x7f0a2810e480 2024-09-16T00:27:52.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.297+0000 7f49a0796640 1 -- 172.21.15.1:0/3947315314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074f40 msgr2=0x7f499810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.297+0000 7f49a0796640 1 --2- 172.21.15.1:0/3947315314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074f40 0x7f499810de80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4988009900 tx=0x7f498802a480 comp rx=0 tx=0).stop 2024-09-16T00:27:52.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 -- 172.21.15.1:0/3947315314 shutdown_connections 2024-09-16T00:27:52.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 --2- 172.21.15.1:0/3947315314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f499810e3c0 0x7f4998110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 --2- 172.21.15.1:0/3947315314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074f40 0x7f499810de80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 --2- 172.21.15.1:0/3947315314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998074620 0x7f4998074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 -- 172.21.15.1:0/3947315314 >> 172.21.15.1:0/3947315314 conn(0x7f49980fd8c0 msgr2=0x7f49980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:52.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 -- 172.21.15.1:0/3947315314 shutdown_connections 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 -- 172.21.15.1:0/3947315314 wait complete. 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 Processor -- start 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 -- start start 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074620 0x7f499819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998074f40 0x7f499819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f499810e3c0 0x7f499819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49981141b0 con 0x7f4998074620 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4998114030 con 0x7f499810e3c0 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.298+0000 7f49a0796640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4998114330 con 0x7f4998074f40 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499e532640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074620 0x7f499819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499e532640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074620 0x7f499819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58572/0 (socket says 172.21.15.1:58572) 2024-09-16T00:27:52.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499e532640 1 -- 172.21.15.1:0/754027683 learned_addr learned my addr 172.21.15.1:0/754027683 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499dd31640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998074f40 0x7f499819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499ed33640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f499810e3c0 0x7f499819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499e532640 1 -- 172.21.15.1:0/754027683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998074f40 msgr2=0x7f499819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499e532640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998074f40 0x7f499819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499e532640 1 -- 172.21.15.1:0/754027683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f499810e3c0 msgr2=0x7f499819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499e532640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f499810e3c0 0x7f499819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499e532640 1 -- 172.21.15.1:0/754027683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499819f640 con 0x7f4998074620 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499dd31640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998074f40 0x7f499819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f499e532640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074620 0x7f499819a690 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f499400c890 tx=0x7f499400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f498f7fe640 1 -- 172.21.15.1:0/754027683 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994004220 con 0x7f4998074620 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f498f7fe640 1 -- 172.21.15.1:0/754027683 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49940043c0 con 0x7f4998074620 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499819f930 con 0x7f4998074620 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f498f7fe640 1 -- 172.21.15.1:0/754027683 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49940049e0 con 0x7f4998074620 2024-09-16T00:27:52.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.299+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49981a7210 con 0x7f4998074620 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.300+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4998074a00 con 0x7f4998074620 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.301+0000 7f498f7fe640 1 -- 172.21.15.1:0/754027683 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f49940079b0 con 0x7f4998074620 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.302+0000 7f498f7fe640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49840a3350 0x7f49840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.303+0000 7f499dd31640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49840a3350 0x7f49840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.303+0000 7f498f7fe640 1 -- 172.21.15.1:0/754027683 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f49940cffb0 con 0x7f4998074620 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.303+0000 7f499dd31640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49840a3350 0x7f49840a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f499810f750 tx=0x7f4988005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.306+0000 7f498f7fe640 1 -- 172.21.15.1:0/754027683 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4994014030 con 0x7f4998074620 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.491+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f499819b9f0 con 0x7f4998074620 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.491+0000 7f498f7fe640 1 -- 172.21.15.1:0/754027683 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f499819b9f0 con 0x7f4998074620 2024-09-16T00:27:52.552 INFO:teuthology.orchestra.run.smithi001.stdout:244813135919 2024-09-16T00:27:52.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49840a3350 msgr2=0x7f49840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49840a3350 0x7f49840a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f499810f750 tx=0x7f4988005a20 comp rx=0 tx=0).stop 2024-09-16T00:27:52.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074620 msgr2=0x7f499819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074620 0x7f499819a690 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f499400c890 tx=0x7f499400cd50 comp rx=0 tx=0).stop 2024-09-16T00:27:52.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 shutdown_connections 2024-09-16T00:27:52.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f499810e3c0 0x7f499819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49840a3350 0x7f49840a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4998074f40 0x7f499819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 --2- 172.21.15.1:0/754027683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4998074620 0x7f499819a690 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 >> 172.21.15.1:0/754027683 conn(0x7f49980fd8c0 msgr2=0x7f49980fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:52.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 shutdown_connections 2024-09-16T00:27:52.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:50.494+0000 7f49a0796640 1 -- 172.21.15.1:0/754027683 wait complete. 2024-09-16T00:27:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.576+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0a2819b9d0 con 0x7f0a2810e480 2024-09-16T00:27:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.577+0000 7f0a267fc640 1 -- 172.21.15.1:0/223887666 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0a2819b9d0 con 0x7f0a2810e480 2024-09-16T00:27:52.577 INFO:teuthology.orchestra.run.smithi001.stdout:158913790014 2024-09-16T00:27:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a100a3350 msgr2=0x7f0a100a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a100a3350 0x7f0a100a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0a20006010 tx=0x7f0a20005f30 comp rx=0 tx=0).stop 2024-09-16T00:27:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a2810e480 msgr2=0x7f0a2819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a2810e480 0x7f0a2819ef40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d6a0 tx=0x7f0a1800db60 comp rx=0 tx=0).stop 2024-09-16T00:27:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 shutdown_connections 2024-09-16T00:27:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a2810e480 0x7f0a2819ef40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a100a3350 0x7f0a100a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a28101280 0x7f0a2819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 --2- 172.21.15.1:0/223887666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a281008d0 0x7f0a2819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 >> 172.21.15.1:0/223887666 conn(0x7f0a280fc7a0 msgr2=0x7f0a280feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 shutdown_connections 2024-09-16T00:27:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:52.579+0000 7f0a2f9a6640 1 -- 172.21.15.1:0/223887666 wait complete. 2024-09-16T00:27:53.077 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:53.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:53 smithi001 ceph-mon[30934]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:27:53.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:53 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/223887666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:27:53.635 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790011 got 158913790014 for osd.5 2024-09-16T00:27:53.635 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:53.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:53 smithi153 ceph-mon[35538]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:27:53.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:53 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/223887666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:27:53.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:53 smithi019 ceph-mon[35186]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:27:53.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:53 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/223887666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:27:54.481 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528374 got 193273528377 for osd.6 2024-09-16T00:27:54.482 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:54.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.698+0000 7f065c119640 1 -- 172.21.15.1:0/1160852862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654106a90 msgr2=0x7f0654106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:54.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.698+0000 7f065c119640 1 --2- 172.21.15.1:0/1160852862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654106a90 0x7f0654106e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0650009930 tx=0x7f065002a420 comp rx=0 tx=0).stop 2024-09-16T00:27:54.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.698+0000 7f065c119640 1 -- 172.21.15.1:0/1160852862 shutdown_connections 2024-09-16T00:27:54.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.698+0000 7f065c119640 1 --2- 172.21.15.1:0/1160852862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654101430 0x7f065410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.698+0000 7f065c119640 1 --2- 172.21.15.1:0/1160852862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654100a90 0x7f0654100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.698+0000 7f065c119640 1 --2- 172.21.15.1:0/1160852862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654106a90 0x7f0654106e70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.698+0000 7f065c119640 1 -- 172.21.15.1:0/1160852862 >> 172.21.15.1:0/1160852862 conn(0x7f06540fc7d0 msgr2=0x7f06540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:54.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.699+0000 7f065c119640 1 -- 172.21.15.1:0/1160852862 shutdown_connections 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.699+0000 7f065c119640 1 -- 172.21.15.1:0/1160852862 wait complete. 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.699+0000 7f065c119640 1 Processor -- start 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.699+0000 7f065c119640 1 -- start start 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f065c119640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654100a90 0x7f0654102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f065c119640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654101430 0x7f0654102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f065c119640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654106a90 0x7f06541030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f065c119640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0654110eb0 con 0x7f0654101430 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f065c119640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0654110d30 con 0x7f0654100a90 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f065c119640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0654111030 con 0x7f0654106a90 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f06596b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654101430 0x7f0654102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f0659eb5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654100a90 0x7f0654102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f06596b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654101430 0x7f0654102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58628/0 (socket says 172.21.15.1:58628) 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f06596b4640 1 -- 172.21.15.1:0/2749184074 learned_addr learned my addr 172.21.15.1:0/2749184074 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f065a6b6640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654106a90 0x7f06541030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f06596b4640 1 -- 172.21.15.1:0/2749184074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654106a90 msgr2=0x7f06541030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f06596b4640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654106a90 0x7f06541030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f06596b4640 1 -- 172.21.15.1:0/2749184074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654100a90 msgr2=0x7f0654102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f06596b4640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654100a90 0x7f0654102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f06596b4640 1 -- 172.21.15.1:0/2749184074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06541a6920 con 0x7f0654101430 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f065a6b6640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654106a90 0x7f06541030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.700+0000 7f06596b4640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654101430 0x7f0654102ba0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f064400b4a0 tx=0x7f064400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.701+0000 7f064affd640 1 -- 172.21.15.1:0/2749184074 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644014070 con 0x7f0654101430 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.701+0000 7f064affd640 1 -- 172.21.15.1:0/2749184074 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0644010af0 con 0x7f0654101430 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.701+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06541a6c10 con 0x7f0654101430 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.701+0000 7f064affd640 1 -- 172.21.15.1:0/2749184074 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644017d10 con 0x7f0654101430 2024-09-16T00:27:54.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.701+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06541a71a0 con 0x7f0654101430 2024-09-16T00:27:54.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.702+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0654100ef0 con 0x7f0654101430 2024-09-16T00:27:54.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.703+0000 7f064affd640 1 -- 172.21.15.1:0/2749184074 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f0644017070 con 0x7f0654101430 2024-09-16T00:27:54.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.704+0000 7f064affd640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f06400a3350 0x7f06400a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:54.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.704+0000 7f0659eb5640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f06400a3350 0x7f06400a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:54.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.704+0000 7f064affd640 1 -- 172.21.15.1:0/2749184074 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f06440cf0e0 con 0x7f0654101430 2024-09-16T00:27:54.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.704+0000 7f0659eb5640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f06400a3350 0x7f06400a5810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0650009900 tx=0x7f06500058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:54.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.708+0000 7f064affd640 1 -- 172.21.15.1:0/2749184074 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f064409b950 con 0x7f0654101430 2024-09-16T00:27:54.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.889+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f06541a73d0 con 0x7f0654101430 2024-09-16T00:27:54.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.890+0000 7f064affd640 1 -- 172.21.15.1:0/2749184074 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f064409b250 con 0x7f0654101430 2024-09-16T00:27:54.890 INFO:teuthology.orchestra.run.smithi001.stdout:296352743462 2024-09-16T00:27:54.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f06400a3350 msgr2=0x7f06400a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:54.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f06400a3350 0x7f06400a5810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0650009900 tx=0x7f06500058d0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654101430 msgr2=0x7f0654102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:54.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654101430 0x7f0654102ba0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f064400b4a0 tx=0x7f064400b960 comp rx=0 tx=0).stop 2024-09-16T00:27:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 shutdown_connections 2024-09-16T00:27:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f06400a3350 0x7f06400a5810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654106a90 0x7f06541030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0654101430 0x7f0654102ba0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 --2- 172.21.15.1:0/2749184074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654100a90 0x7f0654102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 >> 172.21.15.1:0/2749184074 conn(0x7f06540fc7d0 msgr2=0x7f0654104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 shutdown_connections 2024-09-16T00:27:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.892+0000 7f065c119640 1 -- 172.21.15.1:0/2749184074 wait complete. 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.927+0000 7f6343c1d640 1 -- 172.21.15.1:0/2826269007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c103740 msgr2=0x7f633c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.927+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2826269007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c103740 0x7f633c107b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f632c009930 tx=0x7f632c02a420 comp rx=0 tx=0).stop 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.927+0000 7f6343c1d640 1 -- 172.21.15.1:0/2826269007 shutdown_connections 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.927+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2826269007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f633c108290 0x7f633c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.927+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2826269007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c103740 0x7f633c107b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.927+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2826269007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f633c102d90 0x7f633c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.927+0000 7f6343c1d640 1 -- 172.21.15.1:0/2826269007 >> 172.21.15.1:0/2826269007 conn(0x7f633c0fc7d0 msgr2=0x7f633c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.927+0000 7f6343c1d640 1 -- 172.21.15.1:0/2826269007 shutdown_connections 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.927+0000 7f6343c1d640 1 -- 172.21.15.1:0/2826269007 wait complete. 2024-09-16T00:27:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f6343c1d640 1 Processor -- start 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f6343c1d640 1 -- start start 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f6343c1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f633c102d90 0x7f633c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f6343c1d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f633c103740 0x7f633c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f6343c1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c108290 0x7f633c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f6343c1d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f633c112050 con 0x7f633c108290 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f6343c1d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f633c111ed0 con 0x7f633c102d90 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f6343c1d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1121d0 con 0x7f633c103740 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f63421ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c108290 0x7f633c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f63421ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c108290 0x7f633c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58642/0 (socket says 172.21.15.1:58642) 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f63421ba640 1 -- 172.21.15.1:0/2258113841 learned_addr learned my addr 172.21.15.1:0/2258113841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.928+0000 7f63419b9640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f633c102d90 0x7f633c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f63411b8640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f633c103740 0x7f633c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f63421ba640 1 -- 172.21.15.1:0/2258113841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f633c103740 msgr2=0x7f633c19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f63421ba640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f633c103740 0x7f633c19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f63421ba640 1 -- 172.21.15.1:0/2258113841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f633c102d90 msgr2=0x7f633c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f63421ba640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f633c102d90 0x7f633c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f63421ba640 1 -- 172.21.15.1:0/2258113841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c19f6e0 con 0x7f633c108290 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f63419b9640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f633c102d90 0x7f633c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f63421ba640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c108290 0x7f633c19f000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f633400eaa0 tx=0x7f633400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f6332ffd640 1 -- 172.21.15.1:0/2258113841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633400cdb0 con 0x7f633c108290 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f6332ffd640 1 -- 172.21.15.1:0/2258113841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6334005020 con 0x7f633c108290 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c19f9d0 con 0x7f633c108290 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f6332ffd640 1 -- 172.21.15.1:0/2258113841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63340116b0 con 0x7f633c108290 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.929+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c1a72b0 con 0x7f633c108290 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.930+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633c103170 con 0x7f633c108290 2024-09-16T00:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.931+0000 7f6332ffd640 1 -- 172.21.15.1:0/2258113841 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f6334005560 con 0x7f633c108290 2024-09-16T00:27:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.932+0000 7f6332ffd640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f63280a3350 0x7f63280a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.932+0000 7f63419b9640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f63280a3350 0x7f63280a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.932+0000 7f6332ffd640 1 -- 172.21.15.1:0/2258113841 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f63340d00d0 con 0x7f633c108290 2024-09-16T00:27:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.933+0000 7f63419b9640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f63280a3350 0x7f63280a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f633800abe0 tx=0x7f633800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:54.937+0000 7f6332ffd640 1 -- 172.21.15.1:0/2258113841 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6334016030 con 0x7f633c108290 2024-09-16T00:27:55.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.113+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f633c10c2b0 con 0x7f633c108290 2024-09-16T00:27:55.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.114+0000 7f6332ffd640 1 -- 172.21.15.1:0/2258113841 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f633409c890 con 0x7f633c108290 2024-09-16T00:27:55.114 INFO:teuthology.orchestra.run.smithi001.stdout:274877906987 2024-09-16T00:27:55.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.115+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f63280a3350 msgr2=0x7f63280a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:55.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.115+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f63280a3350 0x7f63280a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f633800abe0 tx=0x7f633800a5c0 comp rx=0 tx=0).stop 2024-09-16T00:27:55.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.115+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c108290 msgr2=0x7f633c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:55.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.115+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c108290 0x7f633c19f000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f633400eaa0 tx=0x7f633400ef60 comp rx=0 tx=0).stop 2024-09-16T00:27:55.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.116+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 shutdown_connections 2024-09-16T00:27:55.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.116+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f633c108290 0x7f633c19f000 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:55.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.116+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f63280a3350 0x7f63280a5810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:55.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.116+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f633c103740 0x7f633c19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:55.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.116+0000 7f6343c1d640 1 --2- 172.21.15.1:0/2258113841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f633c102d90 0x7f633c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:55.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.116+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 >> 172.21.15.1:0/2258113841 conn(0x7f633c0fc7d0 msgr2=0x7f633c1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:55.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.116+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 shutdown_connections 2024-09-16T00:27:55.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:55.116+0000 7f6343c1d640 1 -- 172.21.15.1:0/2258113841 wait complete. 2024-09-16T00:27:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:55 smithi001 ceph-mon[30934]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:27:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:55 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2749184074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:27:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:55 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2258113841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:27:55.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:55 smithi153 ceph-mon[35538]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:27:55.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:55 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2749184074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:27:55.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:55 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2258113841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:27:55.831 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906983 got 274877906987 for osd.9 2024-09-16T00:27:55.831 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:55.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:55 smithi019 ceph-mon[35186]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:27:55.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:55 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2749184074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:27:55.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:55 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2258113841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:27:56.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.246+0000 7f3088da5640 1 -- 172.21.15.1:0/2996785857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 msgr2=0x7f3084101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:56.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.246+0000 7f3088da5640 1 --2- 172.21.15.1:0/2996785857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 0x7f3084101200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3074009900 tx=0x7f307402a480 comp rx=0 tx=0).stop 2024-09-16T00:27:56.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.246+0000 7f3088da5640 1 -- 172.21.15.1:0/2996785857 shutdown_connections 2024-09-16T00:27:56.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.246+0000 7f3088da5640 1 --2- 172.21.15.1:0/2996785857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3084102ef0 0x7f308410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.246+0000 7f3088da5640 1 --2- 172.21.15.1:0/2996785857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3084102570 0x7f30841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.246+0000 7f3088da5640 1 --2- 172.21.15.1:0/2996785857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 0x7f3084101200 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.246+0000 7f3088da5640 1 -- 172.21.15.1:0/2996785857 >> 172.21.15.1:0/2996785857 conn(0x7f30840fc7d0 msgr2=0x7f30840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.246+0000 7f3088da5640 1 -- 172.21.15.1:0/2996785857 shutdown_connections 2024-09-16T00:27:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.246+0000 7f3088da5640 1 -- 172.21.15.1:0/2996785857 wait complete. 2024-09-16T00:27:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f3088da5640 1 Processor -- start 2024-09-16T00:27:56.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f3088da5640 1 -- start start 2024-09-16T00:27:56.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f3088da5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 0x7f30841993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:56.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f3088da5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3084102570 0x7f30841998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:56.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f3088da5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3084102ef0 0x7f30841a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:56.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f3088da5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3084110fa0 con 0x7f3084100e20 2024-09-16T00:27:56.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f3088da5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3084110e20 con 0x7f3084102570 2024-09-16T00:27:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f3088da5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3084111120 con 0x7f3084102ef0 2024-09-16T00:27:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f308259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 0x7f30841993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f308259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 0x7f30841993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58650/0 (socket says 172.21.15.1:58650) 2024-09-16T00:27:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f308259c640 1 -- 172.21.15.1:0/3637673972 learned_addr learned my addr 172.21.15.1:0/3637673972 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:56.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f307bfff640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3084102570 0x7f30841998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:56.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f3082d9d640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3084102ef0 0x7f30841a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:56.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f308259c640 1 -- 172.21.15.1:0/3637673972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3084102ef0 msgr2=0x7f30841a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f308259c640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3084102ef0 0x7f30841a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f308259c640 1 -- 172.21.15.1:0/3637673972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3084102570 msgr2=0x7f30841998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f308259c640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3084102570 0x7f30841998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f308259c640 1 -- 172.21.15.1:0/3637673972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30841a4970 con 0x7f3084100e20 2024-09-16T00:27:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.247+0000 7f307bfff640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3084102570 0x7f30841998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.248+0000 7f308259c640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 0x7f30841993b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3074006010 tx=0x7f3074003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:56.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.248+0000 7f307b7fe640 1 -- 172.21.15.1:0/3637673972 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307403f070 con 0x7f3084100e20 2024-09-16T00:27:56.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.248+0000 7f307b7fe640 1 -- 172.21.15.1:0/3637673972 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3074031c30 con 0x7f3084100e20 2024-09-16T00:27:56.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.248+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30841a4b40 con 0x7f3084100e20 2024-09-16T00:27:56.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.248+0000 7f307b7fe640 1 -- 172.21.15.1:0/3637673972 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074035030 con 0x7f3084100e20 2024-09-16T00:27:56.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.248+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30841a4ff0 con 0x7f3084100e20 2024-09-16T00:27:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.249+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3084101200 con 0x7f3084100e20 2024-09-16T00:27:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.250+0000 7f307b7fe640 1 -- 172.21.15.1:0/3637673972 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f3074042070 con 0x7f3084100e20 2024-09-16T00:27:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.251+0000 7f307b7fe640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30700a3350 0x7f30700a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.251+0000 7f307bfff640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30700a3350 0x7f30700a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.251+0000 7f307b7fe640 1 -- 172.21.15.1:0/3637673972 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f30740f0790 con 0x7f3084100e20 2024-09-16T00:27:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.252+0000 7f307bfff640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30700a3350 0x7f30700a5810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f306c0058c0 tx=0x7f306c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:56.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.255+0000 7f307b7fe640 1 -- 172.21.15.1:0/3637673972 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3074031de0 con 0x7f3084100e20 2024-09-16T00:27:56.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.433+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3084109090 con 0x7f3084100e20 2024-09-16T00:27:56.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.433+0000 7f307b7fe640 1 -- 172.21.15.1:0/3637673972 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f30740bcef0 con 0x7f3084100e20 2024-09-16T00:27:56.434 INFO:teuthology.orchestra.run.smithi001.stdout:214748364853 2024-09-16T00:27:56.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30700a3350 msgr2=0x7f30700a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:56.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30700a3350 0x7f30700a5810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f306c0058c0 tx=0x7f306c005850 comp rx=0 tx=0).stop 2024-09-16T00:27:56.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 msgr2=0x7f30841993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:56.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 0x7f30841993b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3074006010 tx=0x7f3074003c10 comp rx=0 tx=0).stop 2024-09-16T00:27:56.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 shutdown_connections 2024-09-16T00:27:56.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30700a3350 0x7f30700a5810 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:56.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3084102ef0 0x7f30841a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:56.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3084102570 0x7f30841998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:56.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 --2- 172.21.15.1:0/3637673972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3084100e20 0x7f30841993b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:56.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.435+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 >> 172.21.15.1:0/3637673972 conn(0x7f30840fc7d0 msgr2=0x7f30840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:56.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.436+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 shutdown_connections 2024-09-16T00:27:56.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:56.436+0000 7f3088da5640 1 -- 172.21.15.1:0/3637673972 wait complete. 2024-09-16T00:27:56.760 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743459 got 296352743462 for osd.10 2024-09-16T00:27:56.761 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:57.548 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135917 got 244813135919 for osd.8 2024-09-16T00:27:57.548 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:57.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:57 smithi001 ceph-mon[30934]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:27:57.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:57 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3637673972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:27:57.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:57 smithi153 ceph-mon[35538]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:27:57.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:57 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3637673972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:27:57.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:57 smithi019 ceph-mon[35186]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:27:57.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:57 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3637673972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:27:57.871 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364848 got 214748364853 for osd.7 2024-09-16T00:27:57.872 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:57.872 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:27:57.872 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:27:58.135 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 -- 172.21.15.1:0/3366217956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 msgr2=0x7f26041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 --2- 172.21.15.1:0/3366217956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 0x7f26041021c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009900 tx=0x7f25f802a410 comp rx=0 tx=0).stop 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 -- 172.21.15.1:0/3366217956 shutdown_connections 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 --2- 172.21.15.1:0/3366217956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2604106f10 0x7f260410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 --2- 172.21.15.1:0/3366217956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2604102790 0x7f26041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 --2- 172.21.15.1:0/3366217956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 0x7f26041021c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 -- 172.21.15.1:0/3366217956 >> 172.21.15.1:0/3366217956 conn(0x7f26040fd8c0 msgr2=0x7f26040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 -- 172.21.15.1:0/3366217956 shutdown_connections 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 -- 172.21.15.1:0/3366217956 wait complete. 2024-09-16T00:27:58.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.651+0000 7f260ad51640 1 Processor -- start 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f260ad51640 1 -- start start 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f260ad51640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 0x7f26041983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f260ad51640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2604102790 0x7f2604198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f260ad51640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2604106f10 0x7f260419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f260ad51640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2604111f70 con 0x7f2604101de0 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f260ad51640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2604111df0 con 0x7f2604102790 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f260ad51640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26041120f0 con 0x7f2604106f10 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f2608aed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 0x7f26041983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f2608aed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 0x7f26041983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58668/0 (socket says 172.21.15.1:58668) 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f2608aed640 1 -- 172.21.15.1:0/332478399 learned_addr learned my addr 172.21.15.1:0/332478399 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f26092ee640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2604106f10 0x7f260419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f2603fff640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2604102790 0x7f2604198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f2608aed640 1 -- 172.21.15.1:0/332478399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2604106f10 msgr2=0x7f260419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f2608aed640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2604106f10 0x7f260419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f2608aed640 1 -- 172.21.15.1:0/332478399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2604102790 msgr2=0x7f2604198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f2608aed640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2604102790 0x7f2604198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.652+0000 7f2608aed640 1 -- 172.21.15.1:0/332478399 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260419d380 con 0x7f2604101de0 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.653+0000 7f2608aed640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 0x7f26041983d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f25f8005ab0 tx=0x7f25f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.653+0000 7f2601ffb640 1 -- 172.21.15.1:0/332478399 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f803f030 con 0x7f2604101de0 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.653+0000 7f2601ffb640 1 -- 172.21.15.1:0/332478399 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f8003ce0 con 0x7f2604101de0 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.653+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260419d610 con 0x7f2604101de0 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.653+0000 7f2601ffb640 1 -- 172.21.15.1:0/332478399 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8035070 con 0x7f2604101de0 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.653+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a4f50 con 0x7f2604101de0 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.654+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26041032c0 con 0x7f2604101de0 2024-09-16T00:27:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.655+0000 7f2601ffb640 1 -- 172.21.15.1:0/332478399 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f25f8042030 con 0x7f2604101de0 2024-09-16T00:27:58.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.656+0000 7f2601ffb640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a32d0 0x7f25f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.656+0000 7f2603fff640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a32d0 0x7f25f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.656+0000 7f2601ffb640 1 -- 172.21.15.1:0/332478399 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f25f80f0840 con 0x7f2604101de0 2024-09-16T00:27:58.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.656+0000 7f2603fff640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a32d0 0x7f25f40a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f26040fdce0 tx=0x7f25f000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:58.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.660+0000 7f2601ffb640 1 -- 172.21.15.1:0/332478399 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25f80bd000 con 0x7f2604101de0 2024-09-16T00:27:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.834+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2604062630 con 0x7f25f40a32d0 2024-09-16T00:27:58.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.844+0000 7f2601ffb640 1 -- 172.21.15.1:0/332478399 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340366 (secure 0 0 0) 0x7f2604062630 con 0x7f25f40a32d0 2024-09-16T00:27:58.849 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:27:58.850 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a32d0 msgr2=0x7f25f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a32d0 0x7f25f40a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f26040fdce0 tx=0x7f25f000a4c0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 msgr2=0x7f26041983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 0x7f26041983d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f25f8005ab0 tx=0x7f25f80042f0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 shutdown_connections 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a32d0 0x7f25f40a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2604106f10 0x7f260419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2604102790 0x7f2604198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 --2- 172.21.15.1:0/332478399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2604101de0 0x7f26041983d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 >> 172.21.15.1:0/332478399 conn(0x7f26040fd8c0 msgr2=0x7f26040febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:58.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 shutdown_connections 2024-09-16T00:27:58.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:27:58.849+0000 7f260ad51640 1 -- 172.21.15.1:0/332478399 wait complete. 2024-09-16T00:27:59.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:59 smithi001 ceph-mon[30934]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:27:59.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:27:59 smithi001 ceph-mon[30934]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:27:59.555 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-16T00:27:58.332984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656681,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":479,"num_read_kb":357,"num_write":349,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376032,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565792,"statfs":{"total":1151940427776,"available":1151555371008,"internally_reserved":0,"allocated":14569472,"data_stored":8125997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003367"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252813+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:27:26.252813+0000","last_peered":"2024-09-16T00:27:26.252813+0000","last_clean":"2024-09-16T00:27:26.252813+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:27:26.252813+0000","last_undegraded":"2024-09-16T00:27:26.252813+0000","last_fullsized":"2024-09-16T00:27:26.252813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247611+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:27:26.247611+0000","last_peered":"2024-09-16T00:27:26.247611+0000","last_clean":"2024-09-16T00:27:26.247611+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:27:26.247611+0000","last_undegraded":"2024-09-16T00:27:26.247611+0000","last_fullsized":"2024-09-16T00:27:26.247611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248105+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:27:26.248105+0000","last_peered":"2024-09-16T00:27:26.248105+0000","last_clean":"2024-09-16T00:27:26.248105+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:27:26.248105+0000","last_undegraded":"2024-09-16T00:27:26.248105+0000","last_fullsized":"2024-09-16T00:27:26.248105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247769+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:27:26.247769+0000","last_peered":"2024-09-16T00:27:26.247769+0000","last_clean":"2024-09-16T00:27:26.247769+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:27:26.247769+0000","last_undegraded":"2024-09-16T00:27:26.247769+0000","last_fullsized":"2024-09-16T00:27:26.247769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250371+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:27:26.250371+0000","last_peered":"2024-09-16T00:27:26.250371+0000","last_clean":"2024-09-16T00:27:26.250371+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:27:26.250371+0000","last_undegraded":"2024-09-16T00:27:26.250371+0000","last_fullsized":"2024-09-16T00:27:26.250371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355288+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:27:26.355288+0000","last_peered":"2024-09-16T00:27:26.355288+0000","last_clean":"2024-09-16T00:27:26.355288+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:27:26.355288+0000","last_undegraded":"2024-09-16T00:27:26.355288+0000","last_fullsized":"2024-09-16T00:27:26.355288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357791+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:26.357791+0000","last_peered":"2024-09-16T00:27:26.357791+0000","last_clean":"2024-09-16T00:27:26.357791+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:26.357791+0000","last_undegraded":"2024-09-16T00:27:26.357791+0000","last_fullsized":"2024-09-16T00:27:26.357791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355128+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:27:26.355128+0000","last_peered":"2024-09-16T00:27:26.355128+0000","last_clean":"2024-09-16T00:27:26.355128+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:27:26.355128+0000","last_undegraded":"2024-09-16T00:27:26.355128+0000","last_fullsized":"2024-09-16T00:27:26.355128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356501+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:27:26.356501+0000","last_peered":"2024-09-16T00:27:26.356501+0000","last_clean":"2024-09-16T00:27:26.356501+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:27:26.356501+0000","last_undegraded":"2024-09-16T00:27:26.356501+0000","last_fullsized":"2024-09-16T00:27:26.356501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252350+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:27:26.252350+0000","last_peered":"2024-09-16T00:27:26.252350+0000","last_clean":"2024-09-16T00:27:26.252350+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:27:26.252350+0000","last_undegraded":"2024-09-16T00:27:26.252350+0000","last_fullsized":"2024-09-16T00:27:26.252350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356056+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:27:26.356056+0000","last_peered":"2024-09-16T00:27:26.356056+0000","last_clean":"2024-09-16T00:27:26.356056+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:27:26.356056+0000","last_undegraded":"2024-09-16T00:27:26.356056+0000","last_fullsized":"2024-09-16T00:27:26.356056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355852+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:27:26.355852+0000","last_peered":"2024-09-16T00:27:26.355852+0000","last_clean":"2024-09-16T00:27:26.355852+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:27:26.355852+0000","last_undegraded":"2024-09-16T00:27:26.355852+0000","last_fullsized":"2024-09-16T00:27:26.355852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355783+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:27:26.355783+0000","last_peered":"2024-09-16T00:27:26.355783+0000","last_clean":"2024-09-16T00:27:26.355783+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:27:26.355783+0000","last_undegraded":"2024-09-16T00:27:26.355783+0000","last_fullsized":"2024-09-16T00:27:26.355783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357045+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:26.357045+0000","last_peered":"2024-09-16T00:27:26.357045+0000","last_clean":"2024-09-16T00:27:26.357045+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:26.357045+0000","last_undegraded":"2024-09-16T00:27:26.357045+0000","last_fullsized":"2024-09-16T00:27:26.357045+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356109+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:27:26.356109+0000","last_peered":"2024-09-16T00:27:26.356109+0000","last_clean":"2024-09-16T00:27:26.356109+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:27:26.356109+0000","last_undegraded":"2024-09-16T00:27:26.356109+0000","last_fullsized":"2024-09-16T00:27:26.356109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247463+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:27:26.247463+0000","last_peered":"2024-09-16T00:27:26.247463+0000","last_clean":"2024-09-16T00:27:26.247463+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:27:26.247463+0000","last_undegraded":"2024-09-16T00:27:26.247463+0000","last_fullsized":"2024-09-16T00:27:26.247463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248254+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:27:26.248254+0000","last_peered":"2024-09-16T00:27:26.248254+0000","last_clean":"2024-09-16T00:27:26.248254+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:27:26.248254+0000","last_undegraded":"2024-09-16T00:27:26.248254+0000","last_fullsized":"2024-09-16T00:27:26.248254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356229+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:27:26.356229+0000","last_peered":"2024-09-16T00:27:26.356229+0000","last_clean":"2024-09-16T00:27:26.356229+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:27:26.356229+0000","last_undegraded":"2024-09-16T00:27:26.356229+0000","last_fullsized":"2024-09-16T00:27:26.356229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252005+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:27:26.252005+0000","last_peered":"2024-09-16T00:27:26.252005+0000","last_clean":"2024-09-16T00:27:26.252005+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:27:26.252005+0000","last_undegraded":"2024-09-16T00:27:26.252005+0000","last_fullsized":"2024-09-16T00:27:26.252005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247471+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:26.247471+0000","last_peered":"2024-09-16T00:27:26.247471+0000","last_clean":"2024-09-16T00:27:26.247471+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:26.247471+0000","last_undegraded":"2024-09-16T00:27:26.247471+0000","last_fullsized":"2024-09-16T00:27:26.247471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252444+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:27:26.252444+0000","last_peered":"2024-09-16T00:27:26.252444+0000","last_clean":"2024-09-16T00:27:26.252444+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:27:26.252444+0000","last_undegraded":"2024-09-16T00:27:26.252444+0000","last_fullsized":"2024-09-16T00:27:26.252444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357153+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:27:26.357153+0000","last_peered":"2024-09-16T00:27:26.357153+0000","last_clean":"2024-09-16T00:27:26.357153+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:27:26.357153+0000","last_undegraded":"2024-09-16T00:27:26.357153+0000","last_fullsized":"2024-09-16T00:27:26.357153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355003+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:26.355003+0000","last_peered":"2024-09-16T00:27:26.355003+0000","last_clean":"2024-09-16T00:27:26.355003+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:26.355003+0000","last_undegraded":"2024-09-16T00:27:26.355003+0000","last_fullsized":"2024-09-16T00:27:26.355003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357278+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:27:26.357278+0000","last_peered":"2024-09-16T00:27:26.357278+0000","last_clean":"2024-09-16T00:27:26.357278+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:27:26.357278+0000","last_undegraded":"2024-09-16T00:27:26.357278+0000","last_fullsized":"2024-09-16T00:27:26.357278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.358639+0000","last_change":"2024-09-16T00:27:26.358639+0000","last_active":"2024-09-16T00:27:26.358639+0000","last_peered":"2024-09-16T00:27:26.358639+0000","last_clean":"2024-09-16T00:27:26.358639+0000","last_became_active":"2024-09-16T00:27:26.358163+0000","last_became_peered":"2024-09-16T00:27:26.358163+0000","last_unstale":"2024-09-16T00:27:26.358639+0000","last_undegraded":"2024-09-16T00:27:26.358639+0000","last_fullsized":"2024-09-16T00:27:26.358639+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:11.811436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356662+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:26.356662+0000","last_peered":"2024-09-16T00:27:26.356662+0000","last_clean":"2024-09-16T00:27:26.356662+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:26.356662+0000","last_undegraded":"2024-09-16T00:27:26.356662+0000","last_fullsized":"2024-09-16T00:27:26.356662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248194+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:27:26.248194+0000","last_peered":"2024-09-16T00:27:26.248194+0000","last_clean":"2024-09-16T00:27:26.248194+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:27:26.248194+0000","last_undegraded":"2024-09-16T00:27:26.248194+0000","last_fullsized":"2024-09-16T00:27:26.248194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247903+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:27:26.247903+0000","last_peered":"2024-09-16T00:27:26.247903+0000","last_clean":"2024-09-16T00:27:26.247903+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:27:26.247903+0000","last_undegraded":"2024-09-16T00:27:26.247903+0000","last_fullsized":"2024-09-16T00:27:26.247903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253036+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:27:26.253036+0000","last_peered":"2024-09-16T00:27:26.253036+0000","last_clean":"2024-09-16T00:27:26.253036+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:27:26.253036+0000","last_undegraded":"2024-09-16T00:27:26.253036+0000","last_fullsized":"2024-09-16T00:27:26.253036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247142+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:27:26.247142+0000","last_peered":"2024-09-16T00:27:26.247142+0000","last_clean":"2024-09-16T00:27:26.247142+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:27:26.247142+0000","last_undegraded":"2024-09-16T00:27:26.247142+0000","last_fullsized":"2024-09-16T00:27:26.247142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250262+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:27:26.250262+0000","last_peered":"2024-09-16T00:27:26.250262+0000","last_clean":"2024-09-16T00:27:26.250262+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:27:26.250262+0000","last_undegraded":"2024-09-16T00:27:26.250262+0000","last_fullsized":"2024-09-16T00:27:26.250262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357806+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:26.357806+0000","last_peered":"2024-09-16T00:27:26.357806+0000","last_clean":"2024-09-16T00:27:26.357806+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:26.357806+0000","last_undegraded":"2024-09-16T00:27:26.357806+0000","last_fullsized":"2024-09-16T00:27:26.357806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251726+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:27:26.251726+0000","last_peered":"2024-09-16T00:27:26.251726+0000","last_clean":"2024-09-16T00:27:26.251726+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:27:26.251726+0000","last_undegraded":"2024-09-16T00:27:26.251726+0000","last_fullsized":"2024-09-16T00:27:26.251726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251126+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:27:26.251126+0000","last_peered":"2024-09-16T00:27:26.251126+0000","last_clean":"2024-09-16T00:27:26.251126+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:27:26.251126+0000","last_undegraded":"2024-09-16T00:27:26.251126+0000","last_fullsized":"2024-09-16T00:27:26.251126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252405+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:27:26.252405+0000","last_peered":"2024-09-16T00:27:26.252405+0000","last_clean":"2024-09-16T00:27:26.252405+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:27:26.252405+0000","last_undegraded":"2024-09-16T00:27:26.252405+0000","last_fullsized":"2024-09-16T00:27:26.252405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356945+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:27:26.356945+0000","last_peered":"2024-09-16T00:27:26.356945+0000","last_clean":"2024-09-16T00:27:26.356945+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:27:26.356945+0000","last_undegraded":"2024-09-16T00:27:26.356945+0000","last_fullsized":"2024-09-16T00:27:26.356945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250926+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:27:26.250926+0000","last_peered":"2024-09-16T00:27:26.250926+0000","last_clean":"2024-09-16T00:27:26.250926+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:27:26.250926+0000","last_undegraded":"2024-09-16T00:27:26.250926+0000","last_fullsized":"2024-09-16T00:27:26.250926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357069+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:27:26.357069+0000","last_peered":"2024-09-16T00:27:26.357069+0000","last_clean":"2024-09-16T00:27:26.357069+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:27:26.357069+0000","last_undegraded":"2024-09-16T00:27:26.357069+0000","last_fullsized":"2024-09-16T00:27:26.357069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250021+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:27:26.250021+0000","last_peered":"2024-09-16T00:27:26.250021+0000","last_clean":"2024-09-16T00:27:26.250021+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:27:26.250021+0000","last_undegraded":"2024-09-16T00:27:26.250021+0000","last_fullsized":"2024-09-16T00:27:26.250021+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357501+0000","last_change":"2024-09-16T00:27:26.357501+0000","last_active":"2024-09-16T00:27:26.357501+0000","last_peered":"2024-09-16T00:27:26.357501+0000","last_clean":"2024-09-16T00:27:26.357501+0000","last_became_active":"2024-09-16T00:27:26.357108+0000","last_became_peered":"2024-09-16T00:27:26.357108+0000","last_unstale":"2024-09-16T00:27:26.357501+0000","last_undegraded":"2024-09-16T00:27:26.357501+0000","last_fullsized":"2024-09-16T00:27:26.357501+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:25.264059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:54.595870+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:27:54.595870+0000","last_peered":"2024-09-16T00:27:54.595870+0000","last_clean":"2024-09-16T00:27:54.595870+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:27:54.595870+0000","last_undegraded":"2024-09-16T00:27:54.595870+0000","last_fullsized":"2024-09-16T00:27:54.595870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250806+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:27:26.250806+0000","last_peered":"2024-09-16T00:27:26.250806+0000","last_clean":"2024-09-16T00:27:26.250806+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:27:26.250806+0000","last_undegraded":"2024-09-16T00:27:26.250806+0000","last_fullsized":"2024-09-16T00:27:26.250806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250353+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:27:26.250353+0000","last_peered":"2024-09-16T00:27:26.250353+0000","last_clean":"2024-09-16T00:27:26.250353+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:27:26.250353+0000","last_undegraded":"2024-09-16T00:27:26.250353+0000","last_fullsized":"2024-09-16T00:27:26.250353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357038+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:26.357038+0000","last_peered":"2024-09-16T00:27:26.357038+0000","last_clean":"2024-09-16T00:27:26.357038+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:26.357038+0000","last_undegraded":"2024-09-16T00:27:26.357038+0000","last_fullsized":"2024-09-16T00:27:26.357038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356072+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:27:26.356072+0000","last_peered":"2024-09-16T00:27:26.356072+0000","last_clean":"2024-09-16T00:27:26.356072+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:27:26.356072+0000","last_undegraded":"2024-09-16T00:27:26.356072+0000","last_fullsized":"2024-09-16T00:27:26.356072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247716+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:27:26.247716+0000","last_peered":"2024-09-16T00:27:26.247716+0000","last_clean":"2024-09-16T00:27:26.247716+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:27:26.247716+0000","last_undegraded":"2024-09-16T00:27:26.247716+0000","last_fullsized":"2024-09-16T00:27:26.247716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356998+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:26.356998+0000","last_peered":"2024-09-16T00:27:26.356998+0000","last_clean":"2024-09-16T00:27:26.356998+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:26.356998+0000","last_undegraded":"2024-09-16T00:27:26.356998+0000","last_fullsized":"2024-09-16T00:27:26.356998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357365+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:27:26.357365+0000","last_peered":"2024-09-16T00:27:26.357365+0000","last_clean":"2024-09-16T00:27:26.357365+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:27:26.357365+0000","last_undegraded":"2024-09-16T00:27:26.357365+0000","last_fullsized":"2024-09-16T00:27:26.357365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355664+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:27:26.355664+0000","last_peered":"2024-09-16T00:27:26.355664+0000","last_clean":"2024-09-16T00:27:26.355664+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:27:26.355664+0000","last_undegraded":"2024-09-16T00:27:26.355664+0000","last_fullsized":"2024-09-16T00:27:26.355664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356623+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:27:26.356623+0000","last_peered":"2024-09-16T00:27:26.356623+0000","last_clean":"2024-09-16T00:27:26.356623+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:27:26.356623+0000","last_undegraded":"2024-09-16T00:27:26.356623+0000","last_fullsized":"2024-09-16T00:27:26.356623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248787+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:26.248787+0000","last_peered":"2024-09-16T00:27:26.248787+0000","last_clean":"2024-09-16T00:27:26.248787+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:26.248787+0000","last_undegraded":"2024-09-16T00:27:26.248787+0000","last_fullsized":"2024-09-16T00:27:26.248787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355115+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:27:26.355115+0000","last_peered":"2024-09-16T00:27:26.355115+0000","last_clean":"2024-09-16T00:27:26.355115+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:27:26.355115+0000","last_undegraded":"2024-09-16T00:27:26.355115+0000","last_fullsized":"2024-09-16T00:27:26.355115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356948+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:27:26.356948+0000","last_peered":"2024-09-16T00:27:26.356948+0000","last_clean":"2024-09-16T00:27:26.356948+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:27:26.356948+0000","last_undegraded":"2024-09-16T00:27:26.356948+0000","last_fullsized":"2024-09-16T00:27:26.356948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356371+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:27:26.356371+0000","last_peered":"2024-09-16T00:27:26.356371+0000","last_clean":"2024-09-16T00:27:26.356371+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:27:26.356371+0000","last_undegraded":"2024-09-16T00:27:26.356371+0000","last_fullsized":"2024-09-16T00:27:26.356371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249484+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:27:26.249484+0000","last_peered":"2024-09-16T00:27:26.249484+0000","last_clean":"2024-09-16T00:27:26.249484+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:27:26.249484+0000","last_undegraded":"2024-09-16T00:27:26.249484+0000","last_fullsized":"2024-09-16T00:27:26.249484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357217+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:27:26.357217+0000","last_peered":"2024-09-16T00:27:26.357217+0000","last_clean":"2024-09-16T00:27:26.357217+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:27:26.357217+0000","last_undegraded":"2024-09-16T00:27:26.357217+0000","last_fullsized":"2024-09-16T00:27:26.357217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250922+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:27:26.250922+0000","last_peered":"2024-09-16T00:27:26.250922+0000","last_clean":"2024-09-16T00:27:26.250922+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:27:26.250922+0000","last_undegraded":"2024-09-16T00:27:26.250922+0000","last_fullsized":"2024-09-16T00:27:26.250922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354916+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:27:26.354916+0000","last_peered":"2024-09-16T00:27:26.354916+0000","last_clean":"2024-09-16T00:27:26.354916+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:27:26.354916+0000","last_undegraded":"2024-09-16T00:27:26.354916+0000","last_fullsized":"2024-09-16T00:27:26.354916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252521+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:27:26.252521+0000","last_peered":"2024-09-16T00:27:26.252521+0000","last_clean":"2024-09-16T00:27:26.252521+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:27:26.252521+0000","last_undegraded":"2024-09-16T00:27:26.252521+0000","last_fullsized":"2024-09-16T00:27:26.252521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357857+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:26.357857+0000","last_peered":"2024-09-16T00:27:26.357857+0000","last_clean":"2024-09-16T00:27:26.357857+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:26.357857+0000","last_undegraded":"2024-09-16T00:27:26.357857+0000","last_fullsized":"2024-09-16T00:27:26.357857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355120+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:27:26.355120+0000","last_peered":"2024-09-16T00:27:26.355120+0000","last_clean":"2024-09-16T00:27:26.355120+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:27:26.355120+0000","last_undegraded":"2024-09-16T00:27:26.355120+0000","last_fullsized":"2024-09-16T00:27:26.355120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355376+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:27:26.355376+0000","last_peered":"2024-09-16T00:27:26.355376+0000","last_clean":"2024-09-16T00:27:26.355376+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:27:26.355376+0000","last_undegraded":"2024-09-16T00:27:26.355376+0000","last_fullsized":"2024-09-16T00:27:26.355376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356933+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:26.356933+0000","last_peered":"2024-09-16T00:27:26.356933+0000","last_clean":"2024-09-16T00:27:26.356933+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:26.356933+0000","last_undegraded":"2024-09-16T00:27:26.356933+0000","last_fullsized":"2024-09-16T00:27:26.356933+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253824+0000","last_change":"2024-09-16T00:27:26.253824+0000","last_active":"2024-09-16T00:27:26.253824+0000","last_peered":"2024-09-16T00:27:26.253824+0000","last_clean":"2024-09-16T00:27:26.253824+0000","last_became_active":"2024-09-16T00:27:26.253206+0000","last_became_peered":"2024-09-16T00:27:26.253206+0000","last_unstale":"2024-09-16T00:27:26.253824+0000","last_undegraded":"2024-09-16T00:27:26.253824+0000","last_fullsized":"2024-09-16T00:27:26.253824+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:22.543369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355182+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:27:26.355182+0000","last_peered":"2024-09-16T00:27:26.355182+0000","last_clean":"2024-09-16T00:27:26.355182+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:27:26.355182+0000","last_undegraded":"2024-09-16T00:27:26.355182+0000","last_fullsized":"2024-09-16T00:27:26.355182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:53.595045+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:27:53.595045+0000","last_peered":"2024-09-16T00:27:53.595045+0000","last_clean":"2024-09-16T00:27:53.595045+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:27:53.595045+0000","last_undegraded":"2024-09-16T00:27:53.595045+0000","last_fullsized":"2024-09-16T00:27:53.595045+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252554+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:27:26.252554+0000","last_peered":"2024-09-16T00:27:26.252554+0000","last_clean":"2024-09-16T00:27:26.252554+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:27:26.252554+0000","last_undegraded":"2024-09-16T00:27:26.252554+0000","last_fullsized":"2024-09-16T00:27:26.252554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355755+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:27:26.355755+0000","last_peered":"2024-09-16T00:27:26.355755+0000","last_clean":"2024-09-16T00:27:26.355755+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:27:26.355755+0000","last_undegraded":"2024-09-16T00:27:26.355755+0000","last_fullsized":"2024-09-16T00:27:26.355755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356273+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:26.356273+0000","last_peered":"2024-09-16T00:27:26.356273+0000","last_clean":"2024-09-16T00:27:26.356273+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:26.356273+0000","last_undegraded":"2024-09-16T00:27:26.356273+0000","last_fullsized":"2024-09-16T00:27:26.356273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252536+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:27:26.252536+0000","last_peered":"2024-09-16T00:27:26.252536+0000","last_clean":"2024-09-16T00:27:26.252536+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:27:26.252536+0000","last_undegraded":"2024-09-16T00:27:26.252536+0000","last_fullsized":"2024-09-16T00:27:26.252536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249895+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:27:26.249895+0000","last_peered":"2024-09-16T00:27:26.249895+0000","last_clean":"2024-09-16T00:27:26.249895+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:27:26.249895+0000","last_undegraded":"2024-09-16T00:27:26.249895+0000","last_fullsized":"2024-09-16T00:27:26.249895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248436+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:26.248436+0000","last_peered":"2024-09-16T00:27:26.248436+0000","last_clean":"2024-09-16T00:27:26.248436+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:26.248436+0000","last_undegraded":"2024-09-16T00:27:26.248436+0000","last_fullsized":"2024-09-16T00:27:26.248436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355941+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:27:26.355941+0000","last_peered":"2024-09-16T00:27:26.355941+0000","last_clean":"2024-09-16T00:27:26.355941+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:27:26.355941+0000","last_undegraded":"2024-09-16T00:27:26.355941+0000","last_fullsized":"2024-09-16T00:27:26.355941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357408+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:27:26.357408+0000","last_peered":"2024-09-16T00:27:26.357408+0000","last_clean":"2024-09-16T00:27:26.357408+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:27:26.357408+0000","last_undegraded":"2024-09-16T00:27:26.357408+0000","last_fullsized":"2024-09-16T00:27:26.357408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'32","reported_seq":203,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:56.598620+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:27:56.598620+0000","last_peered":"2024-09-16T00:27:56.598620+0000","last_clean":"2024-09-16T00:27:56.598620+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:27:56.598620+0000","last_undegraded":"2024-09-16T00:27:56.598620+0000","last_fullsized":"2024-09-16T00:27:56.598620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":119,"num_read_kb":120,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.358804+0000","last_change":"2024-09-16T00:27:26.358804+0000","last_active":"2024-09-16T00:27:26.358804+0000","last_peered":"2024-09-16T00:27:26.358804+0000","last_clean":"2024-09-16T00:27:26.358804+0000","last_became_active":"2024-09-16T00:27:26.358317+0000","last_became_peered":"2024-09-16T00:27:26.358317+0000","last_unstale":"2024-09-16T00:27:26.358804+0000","last_undegraded":"2024-09-16T00:27:26.358804+0000","last_fullsized":"2024-09-16T00:27:26.358804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:38.642392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355783+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:27:26.355783+0000","last_peered":"2024-09-16T00:27:26.355783+0000","last_clean":"2024-09-16T00:27:26.355783+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:27:26.355783+0000","last_undegraded":"2024-09-16T00:27:26.355783+0000","last_fullsized":"2024-09-16T00:27:26.355783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356252+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:26.356252+0000","last_peered":"2024-09-16T00:27:26.356252+0000","last_clean":"2024-09-16T00:27:26.356252+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:26.356252+0000","last_undegraded":"2024-09-16T00:27:26.356252+0000","last_fullsized":"2024-09-16T00:27:26.356252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356754+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:27:26.356754+0000","last_peered":"2024-09-16T00:27:26.356754+0000","last_clean":"2024-09-16T00:27:26.356754+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:27:26.356754+0000","last_undegraded":"2024-09-16T00:27:26.356754+0000","last_fullsized":"2024-09-16T00:27:26.356754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250464+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:27:26.250464+0000","last_peered":"2024-09-16T00:27:26.250464+0000","last_clean":"2024-09-16T00:27:26.250464+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:27:26.250464+0000","last_undegraded":"2024-09-16T00:27:26.250464+0000","last_fullsized":"2024-09-16T00:27:26.250464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357185+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:26.357185+0000","last_peered":"2024-09-16T00:27:26.357185+0000","last_clean":"2024-09-16T00:27:26.357185+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:26.357185+0000","last_undegraded":"2024-09-16T00:27:26.357185+0000","last_fullsized":"2024-09-16T00:27:26.357185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252128+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:27:26.252128+0000","last_peered":"2024-09-16T00:27:26.252128+0000","last_clean":"2024-09-16T00:27:26.252128+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:27:26.252128+0000","last_undegraded":"2024-09-16T00:27:26.252128+0000","last_fullsized":"2024-09-16T00:27:26.252128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356388+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:27:26.356388+0000","last_peered":"2024-09-16T00:27:26.356388+0000","last_clean":"2024-09-16T00:27:26.356388+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:27:26.356388+0000","last_undegraded":"2024-09-16T00:27:26.356388+0000","last_fullsized":"2024-09-16T00:27:26.356388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:55.597069+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:27:55.597069+0000","last_peered":"2024-09-16T00:27:55.597069+0000","last_clean":"2024-09-16T00:27:55.597069+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:27:55.597069+0000","last_undegraded":"2024-09-16T00:27:55.597069+0000","last_fullsized":"2024-09-16T00:27:55.597069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247274+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:27:26.247274+0000","last_peered":"2024-09-16T00:27:26.247274+0000","last_clean":"2024-09-16T00:27:26.247274+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:27:26.247274+0000","last_undegraded":"2024-09-16T00:27:26.247274+0000","last_fullsized":"2024-09-16T00:27:26.247274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249921+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:27:26.249921+0000","last_peered":"2024-09-16T00:27:26.249921+0000","last_clean":"2024-09-16T00:27:26.249921+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:27:26.249921+0000","last_undegraded":"2024-09-16T00:27:26.249921+0000","last_fullsized":"2024-09-16T00:27:26.249921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251661+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:27:26.251661+0000","last_peered":"2024-09-16T00:27:26.251661+0000","last_clean":"2024-09-16T00:27:26.251661+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:27:26.251661+0000","last_undegraded":"2024-09-16T00:27:26.251661+0000","last_fullsized":"2024-09-16T00:27:26.251661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355360+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:27:26.355360+0000","last_peered":"2024-09-16T00:27:26.355360+0000","last_clean":"2024-09-16T00:27:26.355360+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:27:26.355360+0000","last_undegraded":"2024-09-16T00:27:26.355360+0000","last_fullsized":"2024-09-16T00:27:26.355360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359230+0000","last_change":"2024-09-16T00:27:26.359230+0000","last_active":"2024-09-16T00:27:26.359230+0000","last_peered":"2024-09-16T00:27:26.359230+0000","last_clean":"2024-09-16T00:27:26.359230+0000","last_became_active":"2024-09-16T00:27:26.358662+0000","last_became_peered":"2024-09-16T00:27:26.358662+0000","last_unstale":"2024-09-16T00:27:26.359230+0000","last_undegraded":"2024-09-16T00:27:26.359230+0000","last_fullsized":"2024-09-16T00:27:26.359230+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:52.111910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247271+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:27:26.247271+0000","last_peered":"2024-09-16T00:27:26.247271+0000","last_clean":"2024-09-16T00:27:26.247271+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:27:26.247271+0000","last_undegraded":"2024-09-16T00:27:26.247271+0000","last_fullsized":"2024-09-16T00:27:26.247271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250458+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:26.250458+0000","last_peered":"2024-09-16T00:27:26.250458+0000","last_clean":"2024-09-16T00:27:26.250458+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:26.250458+0000","last_undegraded":"2024-09-16T00:27:26.250458+0000","last_fullsized":"2024-09-16T00:27:26.250458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250892+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:26.250892+0000","last_peered":"2024-09-16T00:27:26.250892+0000","last_clean":"2024-09-16T00:27:26.250892+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:26.250892+0000","last_undegraded":"2024-09-16T00:27:26.250892+0000","last_fullsized":"2024-09-16T00:27:26.250892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355758+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:27:26.355758+0000","last_peered":"2024-09-16T00:27:26.355758+0000","last_clean":"2024-09-16T00:27:26.355758+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:27:26.355758+0000","last_undegraded":"2024-09-16T00:27:26.355758+0000","last_fullsized":"2024-09-16T00:27:26.355758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359360+0000","last_change":"2024-09-16T00:27:26.359360+0000","last_active":"2024-09-16T00:27:26.359360+0000","last_peered":"2024-09-16T00:27:26.359360+0000","last_clean":"2024-09-16T00:27:26.359360+0000","last_became_active":"2024-09-16T00:27:26.358462+0000","last_became_peered":"2024-09-16T00:27:26.358462+0000","last_unstale":"2024-09-16T00:27:26.359360+0000","last_undegraded":"2024-09-16T00:27:26.359360+0000","last_fullsized":"2024-09-16T00:27:26.359360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:38.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":98,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:53.594566+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:27:53.594566+0000","last_peered":"2024-09-16T00:27:53.594566+0000","last_clean":"2024-09-16T00:27:53.594566+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:27:53.594566+0000","last_undegraded":"2024-09-16T00:27:53.594566+0000","last_fullsized":"2024-09-16T00:27:53.594566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357068+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:27:26.357068+0000","last_peered":"2024-09-16T00:27:26.357068+0000","last_clean":"2024-09-16T00:27:26.357068+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:27:26.357068+0000","last_undegraded":"2024-09-16T00:27:26.357068+0000","last_fullsized":"2024-09-16T00:27:26.357068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248757+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:27:26.248757+0000","last_peered":"2024-09-16T00:27:26.248757+0000","last_clean":"2024-09-16T00:27:26.248757+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:27:26.248757+0000","last_undegraded":"2024-09-16T00:27:26.248757+0000","last_fullsized":"2024-09-16T00:27:26.248757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251876+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:27:26.251876+0000","last_peered":"2024-09-16T00:27:26.251876+0000","last_clean":"2024-09-16T00:27:26.251876+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:27:26.251876+0000","last_undegraded":"2024-09-16T00:27:26.251876+0000","last_fullsized":"2024-09-16T00:27:26.251876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248445+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:27:26.248445+0000","last_peered":"2024-09-16T00:27:26.248445+0000","last_clean":"2024-09-16T00:27:26.248445+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:27:26.248445+0000","last_undegraded":"2024-09-16T00:27:26.248445+0000","last_fullsized":"2024-09-16T00:27:26.248445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248197+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:27:26.248197+0000","last_peered":"2024-09-16T00:27:26.248197+0000","last_clean":"2024-09-16T00:27:26.248197+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:27:26.248197+0000","last_undegraded":"2024-09-16T00:27:26.248197+0000","last_fullsized":"2024-09-16T00:27:26.248197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249595+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:27:26.249595+0000","last_peered":"2024-09-16T00:27:26.249595+0000","last_clean":"2024-09-16T00:27:26.249595+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:27:26.249595+0000","last_undegraded":"2024-09-16T00:27:26.249595+0000","last_fullsized":"2024-09-16T00:27:26.249595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250176+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:27:26.250176+0000","last_peered":"2024-09-16T00:27:26.250176+0000","last_clean":"2024-09-16T00:27:26.250176+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:27:26.250176+0000","last_undegraded":"2024-09-16T00:27:26.250176+0000","last_fullsized":"2024-09-16T00:27:26.250176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252517+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:27:26.252517+0000","last_peered":"2024-09-16T00:27:26.252517+0000","last_clean":"2024-09-16T00:27:26.252517+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:27:26.252517+0000","last_undegraded":"2024-09-16T00:27:26.252517+0000","last_fullsized":"2024-09-16T00:27:26.252517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249157+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:27:26.249157+0000","last_peered":"2024-09-16T00:27:26.249157+0000","last_clean":"2024-09-16T00:27:26.249157+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:27:26.249157+0000","last_undegraded":"2024-09-16T00:27:26.249157+0000","last_fullsized":"2024-09-16T00:27:26.249157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247833+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:27:26.247833+0000","last_peered":"2024-09-16T00:27:26.247833+0000","last_clean":"2024-09-16T00:27:26.247833+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:27:26.247833+0000","last_undegraded":"2024-09-16T00:27:26.247833+0000","last_fullsized":"2024-09-16T00:27:26.247833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355240+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:27:26.355240+0000","last_peered":"2024-09-16T00:27:26.355240+0000","last_clean":"2024-09-16T00:27:26.355240+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:27:26.355240+0000","last_undegraded":"2024-09-16T00:27:26.355240+0000","last_fullsized":"2024-09-16T00:27:26.355240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247361+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:27:26.247361+0000","last_peered":"2024-09-16T00:27:26.247361+0000","last_clean":"2024-09-16T00:27:26.247361+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:27:26.247361+0000","last_undegraded":"2024-09-16T00:27:26.247361+0000","last_fullsized":"2024-09-16T00:27:26.247361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355378+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:27:26.355378+0000","last_peered":"2024-09-16T00:27:26.355378+0000","last_clean":"2024-09-16T00:27:26.355378+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:27:26.355378+0000","last_undegraded":"2024-09-16T00:27:26.355378+0000","last_fullsized":"2024-09-16T00:27:26.355378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251204+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:27:26.251204+0000","last_peered":"2024-09-16T00:27:26.251204+0000","last_clean":"2024-09-16T00:27:26.251204+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:27:26.251204+0000","last_undegraded":"2024-09-16T00:27:26.251204+0000","last_fullsized":"2024-09-16T00:27:26.251204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252574+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:27:26.252574+0000","last_peered":"2024-09-16T00:27:26.252574+0000","last_clean":"2024-09-16T00:27:26.252574+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:27:26.252574+0000","last_undegraded":"2024-09-16T00:27:26.252574+0000","last_fullsized":"2024-09-16T00:27:26.252574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248984+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:27:26.248984+0000","last_peered":"2024-09-16T00:27:26.248984+0000","last_clean":"2024-09-16T00:27:26.248984+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:27:26.248984+0000","last_undegraded":"2024-09-16T00:27:26.248984+0000","last_fullsized":"2024-09-16T00:27:26.248984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253029+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:27:26.253029+0000","last_peered":"2024-09-16T00:27:26.253029+0000","last_clean":"2024-09-16T00:27:26.253029+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:27:26.253029+0000","last_undegraded":"2024-09-16T00:27:26.253029+0000","last_fullsized":"2024-09-16T00:27:26.253029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252769+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:27:26.252769+0000","last_peered":"2024-09-16T00:27:26.252769+0000","last_clean":"2024-09-16T00:27:26.252769+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:27:26.252769+0000","last_undegraded":"2024-09-16T00:27:26.252769+0000","last_fullsized":"2024-09-16T00:27:26.252769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354886+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:27:26.354886+0000","last_peered":"2024-09-16T00:27:26.354886+0000","last_clean":"2024-09-16T00:27:26.354886+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:27:26.354886+0000","last_undegraded":"2024-09-16T00:27:26.354886+0000","last_fullsized":"2024-09-16T00:27:26.354886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250661+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:27:26.250661+0000","last_peered":"2024-09-16T00:27:26.250661+0000","last_clean":"2024-09-16T00:27:26.250661+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:27:26.250661+0000","last_undegraded":"2024-09-16T00:27:26.250661+0000","last_fullsized":"2024-09-16T00:27:26.250661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249464+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:27:26.249464+0000","last_peered":"2024-09-16T00:27:26.249464+0000","last_clean":"2024-09-16T00:27:26.249464+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:27:26.249464+0000","last_undegraded":"2024-09-16T00:27:26.249464+0000","last_fullsized":"2024-09-16T00:27:26.249464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251117+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:27:26.251117+0000","last_peered":"2024-09-16T00:27:26.251117+0000","last_clean":"2024-09-16T00:27:26.251117+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:27:26.251117+0000","last_undegraded":"2024-09-16T00:27:26.251117+0000","last_fullsized":"2024-09-16T00:27:26.251117+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357180+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:27:26.357180+0000","last_peered":"2024-09-16T00:27:26.357180+0000","last_clean":"2024-09-16T00:27:26.357180+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:27:26.357180+0000","last_undegraded":"2024-09-16T00:27:26.357180+0000","last_fullsized":"2024-09-16T00:27:26.357180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357085+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:27:26.357085+0000","last_peered":"2024-09-16T00:27:26.357085+0000","last_clean":"2024-09-16T00:27:26.357085+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:27:26.357085+0000","last_undegraded":"2024-09-16T00:27:26.357085+0000","last_fullsized":"2024-09-16T00:27:26.357085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251894+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:27:26.251894+0000","last_peered":"2024-09-16T00:27:26.251894+0000","last_clean":"2024-09-16T00:27:26.251894+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:27:26.251894+0000","last_undegraded":"2024-09-16T00:27:26.251894+0000","last_fullsized":"2024-09-16T00:27:26.251894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356608+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:27:26.356608+0000","last_peered":"2024-09-16T00:27:26.356608+0000","last_clean":"2024-09-16T00:27:26.356608+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:27:26.356608+0000","last_undegraded":"2024-09-16T00:27:26.356608+0000","last_fullsized":"2024-09-16T00:27:26.356608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252236+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:27:26.252236+0000","last_peered":"2024-09-16T00:27:26.252236+0000","last_clean":"2024-09-16T00:27:26.252236+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:27:26.252236+0000","last_undegraded":"2024-09-16T00:27:26.252236+0000","last_fullsized":"2024-09-16T00:27:26.252236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356543+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:27:26.356543+0000","last_peered":"2024-09-16T00:27:26.356543+0000","last_clean":"2024-09-16T00:27:26.356543+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:27:26.356543+0000","last_undegraded":"2024-09-16T00:27:26.356543+0000","last_fullsized":"2024-09-16T00:27:26.356543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250832+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:27:26.250832+0000","last_peered":"2024-09-16T00:27:26.250832+0000","last_clean":"2024-09-16T00:27:26.250832+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:27:26.250832+0000","last_undegraded":"2024-09-16T00:27:26.250832+0000","last_fullsized":"2024-09-16T00:27:26.250832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357007+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:27:26.357007+0000","last_peered":"2024-09-16T00:27:26.357007+0000","last_clean":"2024-09-16T00:27:26.357007+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:27:26.357007+0000","last_undegraded":"2024-09-16T00:27:26.357007+0000","last_fullsized":"2024-09-16T00:27:26.357007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357069+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:27:26.357069+0000","last_peered":"2024-09-16T00:27:26.357069+0000","last_clean":"2024-09-16T00:27:26.357069+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:27:26.357069+0000","last_undegraded":"2024-09-16T00:27:26.357069+0000","last_fullsized":"2024-09-16T00:27:26.357069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.358604+0000","last_change":"2024-09-16T00:27:26.358604+0000","last_active":"2024-09-16T00:27:26.358604+0000","last_peered":"2024-09-16T00:27:26.358604+0000","last_clean":"2024-09-16T00:27:26.358604+0000","last_became_active":"2024-09-16T00:27:26.358225+0000","last_became_peered":"2024-09-16T00:27:26.358225+0000","last_unstale":"2024-09-16T00:27:26.358604+0000","last_undegraded":"2024-09-16T00:27:26.358604+0000","last_fullsized":"2024-09-16T00:27:26.358604+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:21.806500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250120+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:27:26.250120+0000","last_peered":"2024-09-16T00:27:26.250120+0000","last_clean":"2024-09-16T00:27:26.250120+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:27:26.250120+0000","last_undegraded":"2024-09-16T00:27:26.250120+0000","last_fullsized":"2024-09-16T00:27:26.250120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252722+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:27:26.252722+0000","last_peered":"2024-09-16T00:27:26.252722+0000","last_clean":"2024-09-16T00:27:26.252722+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:27:26.252722+0000","last_undegraded":"2024-09-16T00:27:26.252722+0000","last_fullsized":"2024-09-16T00:27:26.252722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66079,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":353,"num_read_kb":248,"num_write":166,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514530,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":515002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1724416,"data_stored":1164346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906988,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":519334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135921,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1052672,"data_stored":515158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364853,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1646592,"data_stored":1111069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528379,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1032192,"data_stored":510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32808,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712344,"statfs":{"total":95995035648,"available":95961440256,"internally_reserved":0,"allocated":1654784,"data_stored":1106657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1114112,"data_stored":577140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":510608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1052672,"data_stored":512067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":572400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":511802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:27:59.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:27:59.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:59 smithi153 ceph-mon[35538]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:27:59.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:27:59 smithi153 ceph-mon[35538]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:27:59.819 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:59.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:59 smithi019 ceph-mon[35186]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:27:59.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:27:59 smithi019 ceph-mon[35186]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:00.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.346+0000 7f704d746640 1 -- 172.21.15.1:0/1412050248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048101400 msgr2=0x7f704810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:00.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.346+0000 7f704d746640 1 --2- 172.21.15.1:0/1412050248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048101400 0x7f704810d930 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7040009930 tx=0x7f704002a420 comp rx=0 tx=0).stop 2024-09-16T00:28:00.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.346+0000 7f704d746640 1 -- 172.21.15.1:0/1412050248 shutdown_connections 2024-09-16T00:28:00.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.346+0000 7f704d746640 1 --2- 172.21.15.1:0/1412050248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048101400 0x7f704810d930 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.346+0000 7f704d746640 1 --2- 172.21.15.1:0/1412050248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7048100a60 0x7f7048100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.346+0000 7f704d746640 1 --2- 172.21.15.1:0/1412050248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7048106a60 0x7f7048106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.346+0000 7f704d746640 1 -- 172.21.15.1:0/1412050248 >> 172.21.15.1:0/1412050248 conn(0x7f70480fc780 msgr2=0x7f70480feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:00.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.346+0000 7f704d746640 1 -- 172.21.15.1:0/1412050248 shutdown_connections 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.346+0000 7f704d746640 1 -- 172.21.15.1:0/1412050248 wait complete. 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f704d746640 1 Processor -- start 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f704d746640 1 -- start start 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f704d746640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7048100a60 0x7f7048102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f704d746640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7048101400 0x7f7048102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f704d746640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048106a60 0x7f70481030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f704d746640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7048110e60 con 0x7f7048106a60 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f704d746640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7048110ce0 con 0x7f7048101400 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f704d746640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7048110fe0 con 0x7f7048100a60 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f70477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048106a60 0x7f70481030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f70477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048106a60 0x7f70481030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39324/0 (socket says 172.21.15.1:39324) 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f70477fe640 1 -- 172.21.15.1:0/133111014 learned_addr learned my addr 172.21.15.1:0/133111014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f7046ffd640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7048100a60 0x7f7048102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:00.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.347+0000 7f70467fc640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7048101400 0x7f7048102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f70477fe640 1 -- 172.21.15.1:0/133111014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7048100a60 msgr2=0x7f7048102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f70477fe640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7048100a60 0x7f7048102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f70477fe640 1 -- 172.21.15.1:0/133111014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7048101400 msgr2=0x7f7048102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f70477fe640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7048101400 0x7f7048102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f70477fe640 1 -- 172.21.15.1:0/133111014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70481a6870 con 0x7f7048106a60 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f70477fe640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048106a60 0x7f70481030b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7040005ab0 tx=0x7f70400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f702ffff640 1 -- 172.21.15.1:0/133111014 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704003f030 con 0x7f7048106a60 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f702ffff640 1 -- 172.21.15.1:0/133111014 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7040003ce0 con 0x7f7048106a60 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f702ffff640 1 -- 172.21.15.1:0/133111014 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040035070 con 0x7f7048106a60 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70481a6b00 con 0x7f7048106a60 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.348+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70481a6fe0 con 0x7f7048106a60 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.349+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7048100ec0 con 0x7f7048106a60 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.351+0000 7f702ffff640 1 -- 172.21.15.1:0/133111014 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f7040042030 con 0x7f7048106a60 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.351+0000 7f702ffff640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70300a32d0 0x7f70300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:00.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.351+0000 7f702ffff640 1 -- 172.21.15.1:0/133111014 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f70400f0730 con 0x7f7048106a60 2024-09-16T00:28:00.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.351+0000 7f7046ffd640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70300a32d0 0x7f70300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:00.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.352+0000 7f7046ffd640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70300a32d0 0x7f70300a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7038007e40 tx=0x7f70380075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:00.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.358+0000 7f702ffff640 1 -- 172.21.15.1:0/133111014 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70400bcef0 con 0x7f7048106a60 2024-09-16T00:28:00.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.533+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f704806af40 con 0x7f70300a32d0 2024-09-16T00:28:00.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.541+0000 7f702ffff640 1 -- 172.21.15.1:0/133111014 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340367 (secure 0 0 0) 0x7f704806af40 con 0x7f70300a32d0 2024-09-16T00:28:00.545 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:00.546 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70300a32d0 msgr2=0x7f70300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70300a32d0 0x7f70300a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7038007e40 tx=0x7f70380075b0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048106a60 msgr2=0x7f70481030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048106a60 0x7f70481030b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7040005ab0 tx=0x7f70400042f0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 shutdown_connections 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7048106a60 0x7f70481030b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7048101400 0x7f7048102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70300a32d0 0x7f70300a5790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 --2- 172.21.15.1:0/133111014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7048100a60 0x7f7048102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 >> 172.21.15.1:0/133111014 conn(0x7f70480fc780 msgr2=0x7f704806a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 shutdown_connections 2024-09-16T00:28:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:00.545+0000 7f704d746640 1 -- 172.21.15.1:0/133111014 wait complete. 2024-09-16T00:28:01.364 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-16T00:28:00.333418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656681,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":489,"num_read_kb":357,"num_write":349,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376032,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565792,"statfs":{"total":1151940427776,"available":1151555371008,"internally_reserved":0,"allocated":14569472,"data_stored":8125997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003223"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252813+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:27:26.252813+0000","last_peered":"2024-09-16T00:27:26.252813+0000","last_clean":"2024-09-16T00:27:26.252813+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:27:26.252813+0000","last_undegraded":"2024-09-16T00:27:26.252813+0000","last_fullsized":"2024-09-16T00:27:26.252813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247611+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:27:26.247611+0000","last_peered":"2024-09-16T00:27:26.247611+0000","last_clean":"2024-09-16T00:27:26.247611+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:27:26.247611+0000","last_undegraded":"2024-09-16T00:27:26.247611+0000","last_fullsized":"2024-09-16T00:27:26.247611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248105+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:27:26.248105+0000","last_peered":"2024-09-16T00:27:26.248105+0000","last_clean":"2024-09-16T00:27:26.248105+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:27:26.248105+0000","last_undegraded":"2024-09-16T00:27:26.248105+0000","last_fullsized":"2024-09-16T00:27:26.248105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247769+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:27:26.247769+0000","last_peered":"2024-09-16T00:27:26.247769+0000","last_clean":"2024-09-16T00:27:26.247769+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:27:26.247769+0000","last_undegraded":"2024-09-16T00:27:26.247769+0000","last_fullsized":"2024-09-16T00:27:26.247769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250371+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:27:26.250371+0000","last_peered":"2024-09-16T00:27:26.250371+0000","last_clean":"2024-09-16T00:27:26.250371+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:27:26.250371+0000","last_undegraded":"2024-09-16T00:27:26.250371+0000","last_fullsized":"2024-09-16T00:27:26.250371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355288+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:27:26.355288+0000","last_peered":"2024-09-16T00:27:26.355288+0000","last_clean":"2024-09-16T00:27:26.355288+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:27:26.355288+0000","last_undegraded":"2024-09-16T00:27:26.355288+0000","last_fullsized":"2024-09-16T00:27:26.355288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357791+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:26.357791+0000","last_peered":"2024-09-16T00:27:26.357791+0000","last_clean":"2024-09-16T00:27:26.357791+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:26.357791+0000","last_undegraded":"2024-09-16T00:27:26.357791+0000","last_fullsized":"2024-09-16T00:27:26.357791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355128+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:27:26.355128+0000","last_peered":"2024-09-16T00:27:26.355128+0000","last_clean":"2024-09-16T00:27:26.355128+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:27:26.355128+0000","last_undegraded":"2024-09-16T00:27:26.355128+0000","last_fullsized":"2024-09-16T00:27:26.355128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356501+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:27:26.356501+0000","last_peered":"2024-09-16T00:27:26.356501+0000","last_clean":"2024-09-16T00:27:26.356501+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:27:26.356501+0000","last_undegraded":"2024-09-16T00:27:26.356501+0000","last_fullsized":"2024-09-16T00:27:26.356501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252350+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:27:26.252350+0000","last_peered":"2024-09-16T00:27:26.252350+0000","last_clean":"2024-09-16T00:27:26.252350+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:27:26.252350+0000","last_undegraded":"2024-09-16T00:27:26.252350+0000","last_fullsized":"2024-09-16T00:27:26.252350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356056+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:27:26.356056+0000","last_peered":"2024-09-16T00:27:26.356056+0000","last_clean":"2024-09-16T00:27:26.356056+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:27:26.356056+0000","last_undegraded":"2024-09-16T00:27:26.356056+0000","last_fullsized":"2024-09-16T00:27:26.356056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355852+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:27:26.355852+0000","last_peered":"2024-09-16T00:27:26.355852+0000","last_clean":"2024-09-16T00:27:26.355852+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:27:26.355852+0000","last_undegraded":"2024-09-16T00:27:26.355852+0000","last_fullsized":"2024-09-16T00:27:26.355852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355783+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:27:26.355783+0000","last_peered":"2024-09-16T00:27:26.355783+0000","last_clean":"2024-09-16T00:27:26.355783+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:27:26.355783+0000","last_undegraded":"2024-09-16T00:27:26.355783+0000","last_fullsized":"2024-09-16T00:27:26.355783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357045+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:26.357045+0000","last_peered":"2024-09-16T00:27:26.357045+0000","last_clean":"2024-09-16T00:27:26.357045+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:26.357045+0000","last_undegraded":"2024-09-16T00:27:26.357045+0000","last_fullsized":"2024-09-16T00:27:26.357045+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356109+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:27:26.356109+0000","last_peered":"2024-09-16T00:27:26.356109+0000","last_clean":"2024-09-16T00:27:26.356109+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:27:26.356109+0000","last_undegraded":"2024-09-16T00:27:26.356109+0000","last_fullsized":"2024-09-16T00:27:26.356109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247463+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:27:26.247463+0000","last_peered":"2024-09-16T00:27:26.247463+0000","last_clean":"2024-09-16T00:27:26.247463+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:27:26.247463+0000","last_undegraded":"2024-09-16T00:27:26.247463+0000","last_fullsized":"2024-09-16T00:27:26.247463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248254+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:27:26.248254+0000","last_peered":"2024-09-16T00:27:26.248254+0000","last_clean":"2024-09-16T00:27:26.248254+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:27:26.248254+0000","last_undegraded":"2024-09-16T00:27:26.248254+0000","last_fullsized":"2024-09-16T00:27:26.248254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356229+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:27:26.356229+0000","last_peered":"2024-09-16T00:27:26.356229+0000","last_clean":"2024-09-16T00:27:26.356229+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:27:26.356229+0000","last_undegraded":"2024-09-16T00:27:26.356229+0000","last_fullsized":"2024-09-16T00:27:26.356229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252005+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:27:26.252005+0000","last_peered":"2024-09-16T00:27:26.252005+0000","last_clean":"2024-09-16T00:27:26.252005+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:27:26.252005+0000","last_undegraded":"2024-09-16T00:27:26.252005+0000","last_fullsized":"2024-09-16T00:27:26.252005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247471+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:26.247471+0000","last_peered":"2024-09-16T00:27:26.247471+0000","last_clean":"2024-09-16T00:27:26.247471+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:26.247471+0000","last_undegraded":"2024-09-16T00:27:26.247471+0000","last_fullsized":"2024-09-16T00:27:26.247471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252444+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:27:26.252444+0000","last_peered":"2024-09-16T00:27:26.252444+0000","last_clean":"2024-09-16T00:27:26.252444+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:27:26.252444+0000","last_undegraded":"2024-09-16T00:27:26.252444+0000","last_fullsized":"2024-09-16T00:27:26.252444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357153+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:27:26.357153+0000","last_peered":"2024-09-16T00:27:26.357153+0000","last_clean":"2024-09-16T00:27:26.357153+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:27:26.357153+0000","last_undegraded":"2024-09-16T00:27:26.357153+0000","last_fullsized":"2024-09-16T00:27:26.357153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355003+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:26.355003+0000","last_peered":"2024-09-16T00:27:26.355003+0000","last_clean":"2024-09-16T00:27:26.355003+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:26.355003+0000","last_undegraded":"2024-09-16T00:27:26.355003+0000","last_fullsized":"2024-09-16T00:27:26.355003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357278+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:27:26.357278+0000","last_peered":"2024-09-16T00:27:26.357278+0000","last_clean":"2024-09-16T00:27:26.357278+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:27:26.357278+0000","last_undegraded":"2024-09-16T00:27:26.357278+0000","last_fullsized":"2024-09-16T00:27:26.357278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.358639+0000","last_change":"2024-09-16T00:27:26.358639+0000","last_active":"2024-09-16T00:27:26.358639+0000","last_peered":"2024-09-16T00:27:26.358639+0000","last_clean":"2024-09-16T00:27:26.358639+0000","last_became_active":"2024-09-16T00:27:26.358163+0000","last_became_peered":"2024-09-16T00:27:26.358163+0000","last_unstale":"2024-09-16T00:27:26.358639+0000","last_undegraded":"2024-09-16T00:27:26.358639+0000","last_fullsized":"2024-09-16T00:27:26.358639+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:11.811436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356662+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:26.356662+0000","last_peered":"2024-09-16T00:27:26.356662+0000","last_clean":"2024-09-16T00:27:26.356662+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:26.356662+0000","last_undegraded":"2024-09-16T00:27:26.356662+0000","last_fullsized":"2024-09-16T00:27:26.356662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248194+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:27:26.248194+0000","last_peered":"2024-09-16T00:27:26.248194+0000","last_clean":"2024-09-16T00:27:26.248194+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:27:26.248194+0000","last_undegraded":"2024-09-16T00:27:26.248194+0000","last_fullsized":"2024-09-16T00:27:26.248194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247903+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:27:26.247903+0000","last_peered":"2024-09-16T00:27:26.247903+0000","last_clean":"2024-09-16T00:27:26.247903+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:27:26.247903+0000","last_undegraded":"2024-09-16T00:27:26.247903+0000","last_fullsized":"2024-09-16T00:27:26.247903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253036+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:27:26.253036+0000","last_peered":"2024-09-16T00:27:26.253036+0000","last_clean":"2024-09-16T00:27:26.253036+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:27:26.253036+0000","last_undegraded":"2024-09-16T00:27:26.253036+0000","last_fullsized":"2024-09-16T00:27:26.253036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247142+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:27:26.247142+0000","last_peered":"2024-09-16T00:27:26.247142+0000","last_clean":"2024-09-16T00:27:26.247142+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:27:26.247142+0000","last_undegraded":"2024-09-16T00:27:26.247142+0000","last_fullsized":"2024-09-16T00:27:26.247142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250262+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:27:26.250262+0000","last_peered":"2024-09-16T00:27:26.250262+0000","last_clean":"2024-09-16T00:27:26.250262+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:27:26.250262+0000","last_undegraded":"2024-09-16T00:27:26.250262+0000","last_fullsized":"2024-09-16T00:27:26.250262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357806+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:26.357806+0000","last_peered":"2024-09-16T00:27:26.357806+0000","last_clean":"2024-09-16T00:27:26.357806+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:26.357806+0000","last_undegraded":"2024-09-16T00:27:26.357806+0000","last_fullsized":"2024-09-16T00:27:26.357806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251726+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:27:26.251726+0000","last_peered":"2024-09-16T00:27:26.251726+0000","last_clean":"2024-09-16T00:27:26.251726+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:27:26.251726+0000","last_undegraded":"2024-09-16T00:27:26.251726+0000","last_fullsized":"2024-09-16T00:27:26.251726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251126+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:27:26.251126+0000","last_peered":"2024-09-16T00:27:26.251126+0000","last_clean":"2024-09-16T00:27:26.251126+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:27:26.251126+0000","last_undegraded":"2024-09-16T00:27:26.251126+0000","last_fullsized":"2024-09-16T00:27:26.251126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252405+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:27:26.252405+0000","last_peered":"2024-09-16T00:27:26.252405+0000","last_clean":"2024-09-16T00:27:26.252405+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:27:26.252405+0000","last_undegraded":"2024-09-16T00:27:26.252405+0000","last_fullsized":"2024-09-16T00:27:26.252405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356945+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:27:26.356945+0000","last_peered":"2024-09-16T00:27:26.356945+0000","last_clean":"2024-09-16T00:27:26.356945+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:27:26.356945+0000","last_undegraded":"2024-09-16T00:27:26.356945+0000","last_fullsized":"2024-09-16T00:27:26.356945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250926+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:27:26.250926+0000","last_peered":"2024-09-16T00:27:26.250926+0000","last_clean":"2024-09-16T00:27:26.250926+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:27:26.250926+0000","last_undegraded":"2024-09-16T00:27:26.250926+0000","last_fullsized":"2024-09-16T00:27:26.250926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357069+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:27:26.357069+0000","last_peered":"2024-09-16T00:27:26.357069+0000","last_clean":"2024-09-16T00:27:26.357069+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:27:26.357069+0000","last_undegraded":"2024-09-16T00:27:26.357069+0000","last_fullsized":"2024-09-16T00:27:26.357069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250021+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:27:26.250021+0000","last_peered":"2024-09-16T00:27:26.250021+0000","last_clean":"2024-09-16T00:27:26.250021+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:27:26.250021+0000","last_undegraded":"2024-09-16T00:27:26.250021+0000","last_fullsized":"2024-09-16T00:27:26.250021+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357501+0000","last_change":"2024-09-16T00:27:26.357501+0000","last_active":"2024-09-16T00:27:26.357501+0000","last_peered":"2024-09-16T00:27:26.357501+0000","last_clean":"2024-09-16T00:27:26.357501+0000","last_became_active":"2024-09-16T00:27:26.357108+0000","last_became_peered":"2024-09-16T00:27:26.357108+0000","last_unstale":"2024-09-16T00:27:26.357501+0000","last_undegraded":"2024-09-16T00:27:26.357501+0000","last_fullsized":"2024-09-16T00:27:26.357501+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:25.264059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:54.595870+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:27:54.595870+0000","last_peered":"2024-09-16T00:27:54.595870+0000","last_clean":"2024-09-16T00:27:54.595870+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:27:54.595870+0000","last_undegraded":"2024-09-16T00:27:54.595870+0000","last_fullsized":"2024-09-16T00:27:54.595870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250806+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:27:26.250806+0000","last_peered":"2024-09-16T00:27:26.250806+0000","last_clean":"2024-09-16T00:27:26.250806+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:27:26.250806+0000","last_undegraded":"2024-09-16T00:27:26.250806+0000","last_fullsized":"2024-09-16T00:27:26.250806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250353+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:27:26.250353+0000","last_peered":"2024-09-16T00:27:26.250353+0000","last_clean":"2024-09-16T00:27:26.250353+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:27:26.250353+0000","last_undegraded":"2024-09-16T00:27:26.250353+0000","last_fullsized":"2024-09-16T00:27:26.250353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357038+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:26.357038+0000","last_peered":"2024-09-16T00:27:26.357038+0000","last_clean":"2024-09-16T00:27:26.357038+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:26.357038+0000","last_undegraded":"2024-09-16T00:27:26.357038+0000","last_fullsized":"2024-09-16T00:27:26.357038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356072+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:27:26.356072+0000","last_peered":"2024-09-16T00:27:26.356072+0000","last_clean":"2024-09-16T00:27:26.356072+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:27:26.356072+0000","last_undegraded":"2024-09-16T00:27:26.356072+0000","last_fullsized":"2024-09-16T00:27:26.356072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247716+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:27:26.247716+0000","last_peered":"2024-09-16T00:27:26.247716+0000","last_clean":"2024-09-16T00:27:26.247716+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:27:26.247716+0000","last_undegraded":"2024-09-16T00:27:26.247716+0000","last_fullsized":"2024-09-16T00:27:26.247716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356998+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:26.356998+0000","last_peered":"2024-09-16T00:27:26.356998+0000","last_clean":"2024-09-16T00:27:26.356998+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:26.356998+0000","last_undegraded":"2024-09-16T00:27:26.356998+0000","last_fullsized":"2024-09-16T00:27:26.356998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357365+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:27:26.357365+0000","last_peered":"2024-09-16T00:27:26.357365+0000","last_clean":"2024-09-16T00:27:26.357365+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:27:26.357365+0000","last_undegraded":"2024-09-16T00:27:26.357365+0000","last_fullsized":"2024-09-16T00:27:26.357365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355664+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:27:26.355664+0000","last_peered":"2024-09-16T00:27:26.355664+0000","last_clean":"2024-09-16T00:27:26.355664+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:27:26.355664+0000","last_undegraded":"2024-09-16T00:27:26.355664+0000","last_fullsized":"2024-09-16T00:27:26.355664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356623+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:27:26.356623+0000","last_peered":"2024-09-16T00:27:26.356623+0000","last_clean":"2024-09-16T00:27:26.356623+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:27:26.356623+0000","last_undegraded":"2024-09-16T00:27:26.356623+0000","last_fullsized":"2024-09-16T00:27:26.356623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248787+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:26.248787+0000","last_peered":"2024-09-16T00:27:26.248787+0000","last_clean":"2024-09-16T00:27:26.248787+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:26.248787+0000","last_undegraded":"2024-09-16T00:27:26.248787+0000","last_fullsized":"2024-09-16T00:27:26.248787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355115+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:27:26.355115+0000","last_peered":"2024-09-16T00:27:26.355115+0000","last_clean":"2024-09-16T00:27:26.355115+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:27:26.355115+0000","last_undegraded":"2024-09-16T00:27:26.355115+0000","last_fullsized":"2024-09-16T00:27:26.355115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356948+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:27:26.356948+0000","last_peered":"2024-09-16T00:27:26.356948+0000","last_clean":"2024-09-16T00:27:26.356948+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:27:26.356948+0000","last_undegraded":"2024-09-16T00:27:26.356948+0000","last_fullsized":"2024-09-16T00:27:26.356948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356371+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:27:26.356371+0000","last_peered":"2024-09-16T00:27:26.356371+0000","last_clean":"2024-09-16T00:27:26.356371+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:27:26.356371+0000","last_undegraded":"2024-09-16T00:27:26.356371+0000","last_fullsized":"2024-09-16T00:27:26.356371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249484+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:27:26.249484+0000","last_peered":"2024-09-16T00:27:26.249484+0000","last_clean":"2024-09-16T00:27:26.249484+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:27:26.249484+0000","last_undegraded":"2024-09-16T00:27:26.249484+0000","last_fullsized":"2024-09-16T00:27:26.249484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357217+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:27:26.357217+0000","last_peered":"2024-09-16T00:27:26.357217+0000","last_clean":"2024-09-16T00:27:26.357217+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:27:26.357217+0000","last_undegraded":"2024-09-16T00:27:26.357217+0000","last_fullsized":"2024-09-16T00:27:26.357217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250922+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:27:26.250922+0000","last_peered":"2024-09-16T00:27:26.250922+0000","last_clean":"2024-09-16T00:27:26.250922+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:27:26.250922+0000","last_undegraded":"2024-09-16T00:27:26.250922+0000","last_fullsized":"2024-09-16T00:27:26.250922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354916+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:27:26.354916+0000","last_peered":"2024-09-16T00:27:26.354916+0000","last_clean":"2024-09-16T00:27:26.354916+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:27:26.354916+0000","last_undegraded":"2024-09-16T00:27:26.354916+0000","last_fullsized":"2024-09-16T00:27:26.354916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252521+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:27:26.252521+0000","last_peered":"2024-09-16T00:27:26.252521+0000","last_clean":"2024-09-16T00:27:26.252521+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:27:26.252521+0000","last_undegraded":"2024-09-16T00:27:26.252521+0000","last_fullsized":"2024-09-16T00:27:26.252521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357857+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:26.357857+0000","last_peered":"2024-09-16T00:27:26.357857+0000","last_clean":"2024-09-16T00:27:26.357857+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:26.357857+0000","last_undegraded":"2024-09-16T00:27:26.357857+0000","last_fullsized":"2024-09-16T00:27:26.357857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355120+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:27:26.355120+0000","last_peered":"2024-09-16T00:27:26.355120+0000","last_clean":"2024-09-16T00:27:26.355120+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:27:26.355120+0000","last_undegraded":"2024-09-16T00:27:26.355120+0000","last_fullsized":"2024-09-16T00:27:26.355120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355376+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:27:26.355376+0000","last_peered":"2024-09-16T00:27:26.355376+0000","last_clean":"2024-09-16T00:27:26.355376+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:27:26.355376+0000","last_undegraded":"2024-09-16T00:27:26.355376+0000","last_fullsized":"2024-09-16T00:27:26.355376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356933+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:26.356933+0000","last_peered":"2024-09-16T00:27:26.356933+0000","last_clean":"2024-09-16T00:27:26.356933+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:26.356933+0000","last_undegraded":"2024-09-16T00:27:26.356933+0000","last_fullsized":"2024-09-16T00:27:26.356933+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253824+0000","last_change":"2024-09-16T00:27:26.253824+0000","last_active":"2024-09-16T00:27:26.253824+0000","last_peered":"2024-09-16T00:27:26.253824+0000","last_clean":"2024-09-16T00:27:26.253824+0000","last_became_active":"2024-09-16T00:27:26.253206+0000","last_became_peered":"2024-09-16T00:27:26.253206+0000","last_unstale":"2024-09-16T00:27:26.253824+0000","last_undegraded":"2024-09-16T00:27:26.253824+0000","last_fullsized":"2024-09-16T00:27:26.253824+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:22.543369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355182+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:27:26.355182+0000","last_peered":"2024-09-16T00:27:26.355182+0000","last_clean":"2024-09-16T00:27:26.355182+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:27:26.355182+0000","last_undegraded":"2024-09-16T00:27:26.355182+0000","last_fullsized":"2024-09-16T00:27:26.355182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:58.602003+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:27:58.602003+0000","last_peered":"2024-09-16T00:27:58.602003+0000","last_clean":"2024-09-16T00:27:58.602003+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:27:58.602003+0000","last_undegraded":"2024-09-16T00:27:58.602003+0000","last_fullsized":"2024-09-16T00:27:58.602003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252554+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:27:26.252554+0000","last_peered":"2024-09-16T00:27:26.252554+0000","last_clean":"2024-09-16T00:27:26.252554+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:27:26.252554+0000","last_undegraded":"2024-09-16T00:27:26.252554+0000","last_fullsized":"2024-09-16T00:27:26.252554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355755+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:27:26.355755+0000","last_peered":"2024-09-16T00:27:26.355755+0000","last_clean":"2024-09-16T00:27:26.355755+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:27:26.355755+0000","last_undegraded":"2024-09-16T00:27:26.355755+0000","last_fullsized":"2024-09-16T00:27:26.355755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356273+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:26.356273+0000","last_peered":"2024-09-16T00:27:26.356273+0000","last_clean":"2024-09-16T00:27:26.356273+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:26.356273+0000","last_undegraded":"2024-09-16T00:27:26.356273+0000","last_fullsized":"2024-09-16T00:27:26.356273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252536+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:27:26.252536+0000","last_peered":"2024-09-16T00:27:26.252536+0000","last_clean":"2024-09-16T00:27:26.252536+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:27:26.252536+0000","last_undegraded":"2024-09-16T00:27:26.252536+0000","last_fullsized":"2024-09-16T00:27:26.252536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249895+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:27:26.249895+0000","last_peered":"2024-09-16T00:27:26.249895+0000","last_clean":"2024-09-16T00:27:26.249895+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:27:26.249895+0000","last_undegraded":"2024-09-16T00:27:26.249895+0000","last_fullsized":"2024-09-16T00:27:26.249895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248436+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:26.248436+0000","last_peered":"2024-09-16T00:27:26.248436+0000","last_clean":"2024-09-16T00:27:26.248436+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:26.248436+0000","last_undegraded":"2024-09-16T00:27:26.248436+0000","last_fullsized":"2024-09-16T00:27:26.248436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355941+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:27:26.355941+0000","last_peered":"2024-09-16T00:27:26.355941+0000","last_clean":"2024-09-16T00:27:26.355941+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:27:26.355941+0000","last_undegraded":"2024-09-16T00:27:26.355941+0000","last_fullsized":"2024-09-16T00:27:26.355941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357408+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:27:26.357408+0000","last_peered":"2024-09-16T00:27:26.357408+0000","last_clean":"2024-09-16T00:27:26.357408+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:27:26.357408+0000","last_undegraded":"2024-09-16T00:27:26.357408+0000","last_fullsized":"2024-09-16T00:27:26.357408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'32","reported_seq":203,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:56.598620+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:27:56.598620+0000","last_peered":"2024-09-16T00:27:56.598620+0000","last_clean":"2024-09-16T00:27:56.598620+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:27:56.598620+0000","last_undegraded":"2024-09-16T00:27:56.598620+0000","last_fullsized":"2024-09-16T00:27:56.598620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":119,"num_read_kb":120,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.358804+0000","last_change":"2024-09-16T00:27:26.358804+0000","last_active":"2024-09-16T00:27:26.358804+0000","last_peered":"2024-09-16T00:27:26.358804+0000","last_clean":"2024-09-16T00:27:26.358804+0000","last_became_active":"2024-09-16T00:27:26.358317+0000","last_became_peered":"2024-09-16T00:27:26.358317+0000","last_unstale":"2024-09-16T00:27:26.358804+0000","last_undegraded":"2024-09-16T00:27:26.358804+0000","last_fullsized":"2024-09-16T00:27:26.358804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:38.642392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355783+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:27:26.355783+0000","last_peered":"2024-09-16T00:27:26.355783+0000","last_clean":"2024-09-16T00:27:26.355783+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:27:26.355783+0000","last_undegraded":"2024-09-16T00:27:26.355783+0000","last_fullsized":"2024-09-16T00:27:26.355783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356252+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:26.356252+0000","last_peered":"2024-09-16T00:27:26.356252+0000","last_clean":"2024-09-16T00:27:26.356252+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:26.356252+0000","last_undegraded":"2024-09-16T00:27:26.356252+0000","last_fullsized":"2024-09-16T00:27:26.356252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356754+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:27:26.356754+0000","last_peered":"2024-09-16T00:27:26.356754+0000","last_clean":"2024-09-16T00:27:26.356754+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:27:26.356754+0000","last_undegraded":"2024-09-16T00:27:26.356754+0000","last_fullsized":"2024-09-16T00:27:26.356754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250464+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:27:26.250464+0000","last_peered":"2024-09-16T00:27:26.250464+0000","last_clean":"2024-09-16T00:27:26.250464+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:27:26.250464+0000","last_undegraded":"2024-09-16T00:27:26.250464+0000","last_fullsized":"2024-09-16T00:27:26.250464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357185+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:26.357185+0000","last_peered":"2024-09-16T00:27:26.357185+0000","last_clean":"2024-09-16T00:27:26.357185+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:26.357185+0000","last_undegraded":"2024-09-16T00:27:26.357185+0000","last_fullsized":"2024-09-16T00:27:26.357185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252128+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:27:26.252128+0000","last_peered":"2024-09-16T00:27:26.252128+0000","last_clean":"2024-09-16T00:27:26.252128+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:27:26.252128+0000","last_undegraded":"2024-09-16T00:27:26.252128+0000","last_fullsized":"2024-09-16T00:27:26.252128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356388+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:27:26.356388+0000","last_peered":"2024-09-16T00:27:26.356388+0000","last_clean":"2024-09-16T00:27:26.356388+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:27:26.356388+0000","last_undegraded":"2024-09-16T00:27:26.356388+0000","last_fullsized":"2024-09-16T00:27:26.356388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:55.597069+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:27:55.597069+0000","last_peered":"2024-09-16T00:27:55.597069+0000","last_clean":"2024-09-16T00:27:55.597069+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:27:55.597069+0000","last_undegraded":"2024-09-16T00:27:55.597069+0000","last_fullsized":"2024-09-16T00:27:55.597069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247274+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:27:26.247274+0000","last_peered":"2024-09-16T00:27:26.247274+0000","last_clean":"2024-09-16T00:27:26.247274+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:27:26.247274+0000","last_undegraded":"2024-09-16T00:27:26.247274+0000","last_fullsized":"2024-09-16T00:27:26.247274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249921+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:27:26.249921+0000","last_peered":"2024-09-16T00:27:26.249921+0000","last_clean":"2024-09-16T00:27:26.249921+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:27:26.249921+0000","last_undegraded":"2024-09-16T00:27:26.249921+0000","last_fullsized":"2024-09-16T00:27:26.249921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251661+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:27:26.251661+0000","last_peered":"2024-09-16T00:27:26.251661+0000","last_clean":"2024-09-16T00:27:26.251661+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:27:26.251661+0000","last_undegraded":"2024-09-16T00:27:26.251661+0000","last_fullsized":"2024-09-16T00:27:26.251661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355360+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:27:26.355360+0000","last_peered":"2024-09-16T00:27:26.355360+0000","last_clean":"2024-09-16T00:27:26.355360+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:27:26.355360+0000","last_undegraded":"2024-09-16T00:27:26.355360+0000","last_fullsized":"2024-09-16T00:27:26.355360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359230+0000","last_change":"2024-09-16T00:27:26.359230+0000","last_active":"2024-09-16T00:27:26.359230+0000","last_peered":"2024-09-16T00:27:26.359230+0000","last_clean":"2024-09-16T00:27:26.359230+0000","last_became_active":"2024-09-16T00:27:26.358662+0000","last_became_peered":"2024-09-16T00:27:26.358662+0000","last_unstale":"2024-09-16T00:27:26.359230+0000","last_undegraded":"2024-09-16T00:27:26.359230+0000","last_fullsized":"2024-09-16T00:27:26.359230+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:52.111910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247271+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:27:26.247271+0000","last_peered":"2024-09-16T00:27:26.247271+0000","last_clean":"2024-09-16T00:27:26.247271+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:27:26.247271+0000","last_undegraded":"2024-09-16T00:27:26.247271+0000","last_fullsized":"2024-09-16T00:27:26.247271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250458+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:26.250458+0000","last_peered":"2024-09-16T00:27:26.250458+0000","last_clean":"2024-09-16T00:27:26.250458+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:26.250458+0000","last_undegraded":"2024-09-16T00:27:26.250458+0000","last_fullsized":"2024-09-16T00:27:26.250458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250892+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:26.250892+0000","last_peered":"2024-09-16T00:27:26.250892+0000","last_clean":"2024-09-16T00:27:26.250892+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:26.250892+0000","last_undegraded":"2024-09-16T00:27:26.250892+0000","last_fullsized":"2024-09-16T00:27:26.250892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355758+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:27:26.355758+0000","last_peered":"2024-09-16T00:27:26.355758+0000","last_clean":"2024-09-16T00:27:26.355758+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:27:26.355758+0000","last_undegraded":"2024-09-16T00:27:26.355758+0000","last_fullsized":"2024-09-16T00:27:26.355758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359360+0000","last_change":"2024-09-16T00:27:26.359360+0000","last_active":"2024-09-16T00:27:26.359360+0000","last_peered":"2024-09-16T00:27:26.359360+0000","last_clean":"2024-09-16T00:27:26.359360+0000","last_became_active":"2024-09-16T00:27:26.358462+0000","last_became_peered":"2024-09-16T00:27:26.358462+0000","last_unstale":"2024-09-16T00:27:26.359360+0000","last_undegraded":"2024-09-16T00:27:26.359360+0000","last_fullsized":"2024-09-16T00:27:26.359360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:38.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:58.602743+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:27:58.602743+0000","last_peered":"2024-09-16T00:27:58.602743+0000","last_clean":"2024-09-16T00:27:58.602743+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:27:58.602743+0000","last_undegraded":"2024-09-16T00:27:58.602743+0000","last_fullsized":"2024-09-16T00:27:58.602743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357068+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:27:26.357068+0000","last_peered":"2024-09-16T00:27:26.357068+0000","last_clean":"2024-09-16T00:27:26.357068+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:27:26.357068+0000","last_undegraded":"2024-09-16T00:27:26.357068+0000","last_fullsized":"2024-09-16T00:27:26.357068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248757+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:27:26.248757+0000","last_peered":"2024-09-16T00:27:26.248757+0000","last_clean":"2024-09-16T00:27:26.248757+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:27:26.248757+0000","last_undegraded":"2024-09-16T00:27:26.248757+0000","last_fullsized":"2024-09-16T00:27:26.248757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251876+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:27:26.251876+0000","last_peered":"2024-09-16T00:27:26.251876+0000","last_clean":"2024-09-16T00:27:26.251876+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:27:26.251876+0000","last_undegraded":"2024-09-16T00:27:26.251876+0000","last_fullsized":"2024-09-16T00:27:26.251876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248445+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:27:26.248445+0000","last_peered":"2024-09-16T00:27:26.248445+0000","last_clean":"2024-09-16T00:27:26.248445+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:27:26.248445+0000","last_undegraded":"2024-09-16T00:27:26.248445+0000","last_fullsized":"2024-09-16T00:27:26.248445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248197+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:27:26.248197+0000","last_peered":"2024-09-16T00:27:26.248197+0000","last_clean":"2024-09-16T00:27:26.248197+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:27:26.248197+0000","last_undegraded":"2024-09-16T00:27:26.248197+0000","last_fullsized":"2024-09-16T00:27:26.248197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249595+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:27:26.249595+0000","last_peered":"2024-09-16T00:27:26.249595+0000","last_clean":"2024-09-16T00:27:26.249595+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:27:26.249595+0000","last_undegraded":"2024-09-16T00:27:26.249595+0000","last_fullsized":"2024-09-16T00:27:26.249595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250176+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:27:26.250176+0000","last_peered":"2024-09-16T00:27:26.250176+0000","last_clean":"2024-09-16T00:27:26.250176+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:27:26.250176+0000","last_undegraded":"2024-09-16T00:27:26.250176+0000","last_fullsized":"2024-09-16T00:27:26.250176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252517+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:27:26.252517+0000","last_peered":"2024-09-16T00:27:26.252517+0000","last_clean":"2024-09-16T00:27:26.252517+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:27:26.252517+0000","last_undegraded":"2024-09-16T00:27:26.252517+0000","last_fullsized":"2024-09-16T00:27:26.252517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249157+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:27:26.249157+0000","last_peered":"2024-09-16T00:27:26.249157+0000","last_clean":"2024-09-16T00:27:26.249157+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:27:26.249157+0000","last_undegraded":"2024-09-16T00:27:26.249157+0000","last_fullsized":"2024-09-16T00:27:26.249157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247833+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:27:26.247833+0000","last_peered":"2024-09-16T00:27:26.247833+0000","last_clean":"2024-09-16T00:27:26.247833+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:27:26.247833+0000","last_undegraded":"2024-09-16T00:27:26.247833+0000","last_fullsized":"2024-09-16T00:27:26.247833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355240+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:27:26.355240+0000","last_peered":"2024-09-16T00:27:26.355240+0000","last_clean":"2024-09-16T00:27:26.355240+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:27:26.355240+0000","last_undegraded":"2024-09-16T00:27:26.355240+0000","last_fullsized":"2024-09-16T00:27:26.355240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247361+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:27:26.247361+0000","last_peered":"2024-09-16T00:27:26.247361+0000","last_clean":"2024-09-16T00:27:26.247361+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:27:26.247361+0000","last_undegraded":"2024-09-16T00:27:26.247361+0000","last_fullsized":"2024-09-16T00:27:26.247361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355378+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:27:26.355378+0000","last_peered":"2024-09-16T00:27:26.355378+0000","last_clean":"2024-09-16T00:27:26.355378+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:27:26.355378+0000","last_undegraded":"2024-09-16T00:27:26.355378+0000","last_fullsized":"2024-09-16T00:27:26.355378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251204+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:27:26.251204+0000","last_peered":"2024-09-16T00:27:26.251204+0000","last_clean":"2024-09-16T00:27:26.251204+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:27:26.251204+0000","last_undegraded":"2024-09-16T00:27:26.251204+0000","last_fullsized":"2024-09-16T00:27:26.251204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252574+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:27:26.252574+0000","last_peered":"2024-09-16T00:27:26.252574+0000","last_clean":"2024-09-16T00:27:26.252574+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:27:26.252574+0000","last_undegraded":"2024-09-16T00:27:26.252574+0000","last_fullsized":"2024-09-16T00:27:26.252574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248984+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:27:26.248984+0000","last_peered":"2024-09-16T00:27:26.248984+0000","last_clean":"2024-09-16T00:27:26.248984+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:27:26.248984+0000","last_undegraded":"2024-09-16T00:27:26.248984+0000","last_fullsized":"2024-09-16T00:27:26.248984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253029+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:27:26.253029+0000","last_peered":"2024-09-16T00:27:26.253029+0000","last_clean":"2024-09-16T00:27:26.253029+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:27:26.253029+0000","last_undegraded":"2024-09-16T00:27:26.253029+0000","last_fullsized":"2024-09-16T00:27:26.253029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252769+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:27:26.252769+0000","last_peered":"2024-09-16T00:27:26.252769+0000","last_clean":"2024-09-16T00:27:26.252769+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:27:26.252769+0000","last_undegraded":"2024-09-16T00:27:26.252769+0000","last_fullsized":"2024-09-16T00:27:26.252769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354886+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:27:26.354886+0000","last_peered":"2024-09-16T00:27:26.354886+0000","last_clean":"2024-09-16T00:27:26.354886+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:27:26.354886+0000","last_undegraded":"2024-09-16T00:27:26.354886+0000","last_fullsized":"2024-09-16T00:27:26.354886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250661+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:27:26.250661+0000","last_peered":"2024-09-16T00:27:26.250661+0000","last_clean":"2024-09-16T00:27:26.250661+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:27:26.250661+0000","last_undegraded":"2024-09-16T00:27:26.250661+0000","last_fullsized":"2024-09-16T00:27:26.250661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249464+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:27:26.249464+0000","last_peered":"2024-09-16T00:27:26.249464+0000","last_clean":"2024-09-16T00:27:26.249464+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:27:26.249464+0000","last_undegraded":"2024-09-16T00:27:26.249464+0000","last_fullsized":"2024-09-16T00:27:26.249464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251117+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:27:26.251117+0000","last_peered":"2024-09-16T00:27:26.251117+0000","last_clean":"2024-09-16T00:27:26.251117+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:27:26.251117+0000","last_undegraded":"2024-09-16T00:27:26.251117+0000","last_fullsized":"2024-09-16T00:27:26.251117+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357180+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:27:26.357180+0000","last_peered":"2024-09-16T00:27:26.357180+0000","last_clean":"2024-09-16T00:27:26.357180+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:27:26.357180+0000","last_undegraded":"2024-09-16T00:27:26.357180+0000","last_fullsized":"2024-09-16T00:27:26.357180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357085+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:27:26.357085+0000","last_peered":"2024-09-16T00:27:26.357085+0000","last_clean":"2024-09-16T00:27:26.357085+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:27:26.357085+0000","last_undegraded":"2024-09-16T00:27:26.357085+0000","last_fullsized":"2024-09-16T00:27:26.357085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251894+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:27:26.251894+0000","last_peered":"2024-09-16T00:27:26.251894+0000","last_clean":"2024-09-16T00:27:26.251894+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:27:26.251894+0000","last_undegraded":"2024-09-16T00:27:26.251894+0000","last_fullsized":"2024-09-16T00:27:26.251894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356608+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:27:26.356608+0000","last_peered":"2024-09-16T00:27:26.356608+0000","last_clean":"2024-09-16T00:27:26.356608+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:27:26.356608+0000","last_undegraded":"2024-09-16T00:27:26.356608+0000","last_fullsized":"2024-09-16T00:27:26.356608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252236+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:27:26.252236+0000","last_peered":"2024-09-16T00:27:26.252236+0000","last_clean":"2024-09-16T00:27:26.252236+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:27:26.252236+0000","last_undegraded":"2024-09-16T00:27:26.252236+0000","last_fullsized":"2024-09-16T00:27:26.252236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356543+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:27:26.356543+0000","last_peered":"2024-09-16T00:27:26.356543+0000","last_clean":"2024-09-16T00:27:26.356543+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:27:26.356543+0000","last_undegraded":"2024-09-16T00:27:26.356543+0000","last_fullsized":"2024-09-16T00:27:26.356543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250832+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:27:26.250832+0000","last_peered":"2024-09-16T00:27:26.250832+0000","last_clean":"2024-09-16T00:27:26.250832+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:27:26.250832+0000","last_undegraded":"2024-09-16T00:27:26.250832+0000","last_fullsized":"2024-09-16T00:27:26.250832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357007+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:27:26.357007+0000","last_peered":"2024-09-16T00:27:26.357007+0000","last_clean":"2024-09-16T00:27:26.357007+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:27:26.357007+0000","last_undegraded":"2024-09-16T00:27:26.357007+0000","last_fullsized":"2024-09-16T00:27:26.357007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357069+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:27:26.357069+0000","last_peered":"2024-09-16T00:27:26.357069+0000","last_clean":"2024-09-16T00:27:26.357069+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:27:26.357069+0000","last_undegraded":"2024-09-16T00:27:26.357069+0000","last_fullsized":"2024-09-16T00:27:26.357069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.358604+0000","last_change":"2024-09-16T00:27:26.358604+0000","last_active":"2024-09-16T00:27:26.358604+0000","last_peered":"2024-09-16T00:27:26.358604+0000","last_clean":"2024-09-16T00:27:26.358604+0000","last_became_active":"2024-09-16T00:27:26.358225+0000","last_became_peered":"2024-09-16T00:27:26.358225+0000","last_unstale":"2024-09-16T00:27:26.358604+0000","last_undegraded":"2024-09-16T00:27:26.358604+0000","last_fullsized":"2024-09-16T00:27:26.358604+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:21.806500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250120+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:27:26.250120+0000","last_peered":"2024-09-16T00:27:26.250120+0000","last_clean":"2024-09-16T00:27:26.250120+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:27:26.250120+0000","last_undegraded":"2024-09-16T00:27:26.250120+0000","last_fullsized":"2024-09-16T00:27:26.250120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252722+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:27:26.252722+0000","last_peered":"2024-09-16T00:27:26.252722+0000","last_clean":"2024-09-16T00:27:26.252722+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:27:26.252722+0000","last_undegraded":"2024-09-16T00:27:26.252722+0000","last_fullsized":"2024-09-16T00:27:26.252722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66079,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":363,"num_read_kb":248,"num_write":166,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514530,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":515002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1724416,"data_stored":1164346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906988,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":519334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135921,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1052672,"data_stored":515158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364854,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1646592,"data_stored":1111069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528379,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1032192,"data_stored":510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790016,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32808,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712344,"statfs":{"total":95995035648,"available":95961440256,"internally_reserved":0,"allocated":1654784,"data_stored":1106657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1114112,"data_stored":577140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":510608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1052672,"data_stored":512067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":572400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":511802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:28:01.370 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:28:01.370 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T00:28:01.370 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T00:28:01.370 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T00:28:01.631 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:01.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:01 smithi001 ceph-mon[30934]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:01.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:01 smithi001 ceph-mon[30934]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:01.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:01 smithi153 ceph-mon[35538]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:01.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:01 smithi153 ceph-mon[35538]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:01 smithi019 ceph-mon[35186]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:01 smithi019 ceph-mon[35186]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:02.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.121+0000 7f6cb5542640 1 -- 172.21.15.1:0/1217329286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 msgr2=0x7f6cb0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:02.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.121+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1217329286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 0x7f6cb0106e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009930 tx=0x7f6ca002a420 comp rx=0 tx=0).stop 2024-09-16T00:28:02.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.121+0000 7f6cb5542640 1 -- 172.21.15.1:0/1217329286 shutdown_connections 2024-09-16T00:28:02.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.121+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1217329286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cb0101420 0x7f6cb010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:02.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.121+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1217329286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb0100a80 0x7f6cb0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:02.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.121+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1217329286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 0x7f6cb0106e60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:02.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.121+0000 7f6cb5542640 1 -- 172.21.15.1:0/1217329286 >> 172.21.15.1:0/1217329286 conn(0x7f6cb00fc7a0 msgr2=0x7f6cb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.121+0000 7f6cb5542640 1 -- 172.21.15.1:0/1217329286 shutdown_connections 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.122+0000 7f6cb5542640 1 -- 172.21.15.1:0/1217329286 wait complete. 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.122+0000 7f6cb5542640 1 Processor -- start 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.122+0000 7f6cb5542640 1 -- start start 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6cb5542640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cb0100a80 0x7f6cb0102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6cb5542640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb0101420 0x7f6cb0102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6cb5542640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 0x7f6cb01030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6cb5542640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0110e80 con 0x7f6cb0106a80 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6cb5542640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0110d00 con 0x7f6cb0101420 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6cb5542640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0111000 con 0x7f6cb0100a80 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 0x7f6cb01030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 0x7f6cb01030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39350/0 (socket says 172.21.15.1:39350) 2024-09-16T00:28:02.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caf7fe640 1 -- 172.21.15.1:0/1213396674 learned_addr learned my addr 172.21.15.1:0/1213396674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6cae7fc640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb0101420 0x7f6cb0102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caeffd640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cb0100a80 0x7f6cb0102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caf7fe640 1 -- 172.21.15.1:0/1213396674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cb0100a80 msgr2=0x7f6cb0102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caf7fe640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cb0100a80 0x7f6cb0102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caf7fe640 1 -- 172.21.15.1:0/1213396674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb0101420 msgr2=0x7f6cb0102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caf7fe640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb0101420 0x7f6cb0102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caf7fe640 1 -- 172.21.15.1:0/1213396674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb01a6890 con 0x7f6cb0106a80 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6caf7fe640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 0x7f6cb01030d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800cce0 tx=0x7f6ca800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6c97fff640 1 -- 172.21.15.1:0/1213396674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800f450 con 0x7f6cb0106a80 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb01a6b80 con 0x7f6cb0106a80 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.123+0000 7f6c97fff640 1 -- 172.21.15.1:0/1213396674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ca800fad0 con 0x7f6cb0106a80 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.124+0000 7f6c97fff640 1 -- 172.21.15.1:0/1213396674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8018c10 con 0x7f6cb0106a80 2024-09-16T00:28:02.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.124+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb01a70c0 con 0x7f6cb0106a80 2024-09-16T00:28:02.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.124+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb0100ee0 con 0x7f6cb0106a80 2024-09-16T00:28:02.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.126+0000 7f6c97fff640 1 -- 172.21.15.1:0/1213396674 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f6ca800f5f0 con 0x7f6cb0106a80 2024-09-16T00:28:02.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.126+0000 7f6c97fff640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c980a32d0 0x7f6c980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:02.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.126+0000 7f6c97fff640 1 -- 172.21.15.1:0/1213396674 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6ca80d0580 con 0x7f6cb0106a80 2024-09-16T00:28:02.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.126+0000 7f6caeffd640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c980a32d0 0x7f6c980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:02.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.127+0000 7f6caeffd640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c980a32d0 0x7f6c980a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0000c00 tx=0x7f6ca0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:02.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.129+0000 7f6c97fff640 1 -- 172.21.15.1:0/1213396674 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ca8014030 con 0x7f6cb0106a80 2024-09-16T00:28:02.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.346+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6cb0103b80 con 0x7f6cb0106a80 2024-09-16T00:28:02.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.346+0000 7f6c97fff640 1 -- 172.21.15.1:0/1213396674 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6cb0103b80 con 0x7f6cb0106a80 2024-09-16T00:28:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:02.347 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T00:28:02.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c980a32d0 msgr2=0x7f6c980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:02.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c980a32d0 0x7f6c980a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0000c00 tx=0x7f6ca0005a20 comp rx=0 tx=0).stop 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 msgr2=0x7f6cb01030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 0x7f6cb01030d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800cce0 tx=0x7f6ca800a620 comp rx=0 tx=0).stop 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 shutdown_connections 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb0106a80 0x7f6cb01030d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb0101420 0x7f6cb0102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c980a32d0 0x7f6c980a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 --2- 172.21.15.1:0/1213396674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cb0100a80 0x7f6cb0102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 >> 172.21.15.1:0/1213396674 conn(0x7f6cb00fc7a0 msgr2=0x7f6cb006a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 shutdown_connections 2024-09-16T00:28:02.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:02.349+0000 7f6cb5542640 1 -- 172.21.15.1:0/1213396674 wait complete. 2024-09-16T00:28:02.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:02 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1213396674' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:28:02.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:02 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1213396674' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:28:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:02 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1213396674' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:28:03.160 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T00:28:03.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:03.422 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:03 smithi001 ceph-mon[30934]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:28:03.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:03 smithi153 ceph-mon[35538]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:28:03.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:03 smithi019 ceph-mon[35186]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:28:03.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.921+0000 7fcb8c455640 1 -- 172.21.15.1:0/4041667640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84106a80 msgr2=0x7fcb84106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:03.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.921+0000 7fcb8c455640 1 --2- 172.21.15.1:0/4041667640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84106a80 0x7fcb84106e60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009cd0 tx=0x7fcb7402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:28:03.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.921+0000 7fcb8c455640 1 -- 172.21.15.1:0/4041667640 shutdown_connections 2024-09-16T00:28:03.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.921+0000 7fcb8c455640 1 --2- 172.21.15.1:0/4041667640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb84101420 0x7fcb8410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:03.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.921+0000 7fcb8c455640 1 --2- 172.21.15.1:0/4041667640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84100a80 0x7fcb84100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.921+0000 7fcb8c455640 1 --2- 172.21.15.1:0/4041667640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84106a80 0x7fcb84106e60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.921+0000 7fcb8c455640 1 -- 172.21.15.1:0/4041667640 >> 172.21.15.1:0/4041667640 conn(0x7fcb840fc7a0 msgr2=0x7fcb840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.921+0000 7fcb8c455640 1 -- 172.21.15.1:0/4041667640 shutdown_connections 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.921+0000 7fcb8c455640 1 -- 172.21.15.1:0/4041667640 wait complete. 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8c455640 1 Processor -- start 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8c455640 1 -- start start 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8c455640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb84100a80 0x7fcb840fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8c455640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84101420 0x7fcb840fd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8c455640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84106a80 0x7fcb840fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8c455640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84110ea0 con 0x7fcb84101420 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8c455640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84110d20 con 0x7fcb84100a80 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8c455640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84111020 con 0x7fcb84106a80 2024-09-16T00:28:03.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb899f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84101420 0x7fcb840fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb899f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84101420 0x7fcb840fd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39378/0 (socket says 172.21.15.1:39378) 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb899f0640 1 -- 172.21.15.1:0/2802505406 learned_addr learned my addr 172.21.15.1:0/2802505406 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8a1f1640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb84100a80 0x7fcb840fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8a9f2640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84106a80 0x7fcb840fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb899f0640 1 -- 172.21.15.1:0/2802505406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84106a80 msgr2=0x7fcb840fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb899f0640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84106a80 0x7fcb840fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb899f0640 1 -- 172.21.15.1:0/2802505406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb84100a80 msgr2=0x7fcb840fcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb899f0640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb84100a80 0x7fcb840fcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb899f0640 1 -- 172.21.15.1:0/2802505406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb841a6980 con 0x7fcb84101420 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8a1f1640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb84100a80 0x7fcb840fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.922+0000 7fcb8a9f2640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84106a80 0x7fcb840fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.923+0000 7fcb899f0640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84101420 0x7fcb840fd310 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800b4a0 tx=0x7fcb7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.923+0000 7fcb837fe640 1 -- 172.21.15.1:0/2802505406 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78014070 con 0x7fcb84101420 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.923+0000 7fcb837fe640 1 -- 172.21.15.1:0/2802505406 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb78010af0 con 0x7fcb84101420 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.923+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb841a6c70 con 0x7fcb84101420 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.923+0000 7fcb837fe640 1 -- 172.21.15.1:0/2802505406 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78017d10 con 0x7fcb84101420 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.923+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb841a71b0 con 0x7fcb84101420 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.924+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb84100ee0 con 0x7fcb84101420 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.925+0000 7fcb837fe640 1 -- 172.21.15.1:0/2802505406 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fcb78017070 con 0x7fcb84101420 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.926+0000 7fcb837fe640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb6c0a3350 0x7fcb6c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.926+0000 7fcb8a1f1640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb6c0a3350 0x7fcb6c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.926+0000 7fcb837fe640 1 -- 172.21.15.1:0/2802505406 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fcb780cf200 con 0x7fcb84101420 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.926+0000 7fcb8a1f1640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb6c0a3350 0x7fcb6c0a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb74000f80 tx=0x7fcb74031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:03.930+0000 7fcb837fe640 1 -- 172.21.15.1:0/2802505406 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb7809ba70 con 0x7fcb84101420 2024-09-16T00:28:04.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.142+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb84109a70 con 0x7fcb84101420 2024-09-16T00:28:04.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.143+0000 7fcb837fe640 1 -- 172.21.15.1:0/2802505406 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11894 (secure 0 0 0) 0x7fcb7809b370 con 0x7fcb84101420 2024-09-16T00:28:04.143 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:04.143 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"btime":"2024-09-16T00:27:04:602627+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:27:04.334709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24470,24469]},"id":1}]} 2024-09-16T00:28:04.144 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 24 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb6c0a3350 msgr2=0x7fcb6c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb6c0a3350 0x7fcb6c0a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb74000f80 tx=0x7fcb74031430 comp rx=0 tx=0).stop 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84101420 msgr2=0x7fcb840fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84101420 0x7fcb840fd310 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800b4a0 tx=0x7fcb7800b960 comp rx=0 tx=0).stop 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 shutdown_connections 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb6c0a3350 0x7fcb6c0a5810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84106a80 0x7fcb840fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb84101420 0x7fcb840fd310 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 --2- 172.21.15.1:0/2802505406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb84100a80 0x7fcb840fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 >> 172.21.15.1:0/2802505406 conn(0x7fcb840fc7a0 msgr2=0x7fcb8406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:04.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 shutdown_connections 2024-09-16T00:28:04.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:04.145+0000 7fcb8c455640 1 -- 172.21.15.1:0/2802505406 wait complete. 2024-09-16T00:28:04.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:04 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2802505406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:04.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:04 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2802505406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:04.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:04 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2802505406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:04.956 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:05.218 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:05.615 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:05 smithi001 ceph-mon[30934]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:05.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:05 smithi153 ceph-mon[35538]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:05.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 -- 172.21.15.1:0/4146910212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4109120 msgr2=0x7fdbf4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:05.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 --2- 172.21.15.1:0/4146910212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4109120 0x7fdbf4110ca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009ca0 tx=0x7fdbec02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 -- 172.21.15.1:0/4146910212 shutdown_connections 2024-09-16T00:28:05.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 --2- 172.21.15.1:0/4146910212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4109120 0x7fdbf4110ca0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 --2- 172.21.15.1:0/4146910212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf41049a0 0x7fdbf41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 --2- 172.21.15.1:0/4146910212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf4103ff0 0x7fdbf41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 -- 172.21.15.1:0/4146910212 >> 172.21.15.1:0/4146910212 conn(0x7fdbf40ffa90 msgr2=0x7fdbf4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 -- 172.21.15.1:0/4146910212 shutdown_connections 2024-09-16T00:28:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 -- 172.21.15.1:0/4146910212 wait complete. 2024-09-16T00:28:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 Processor -- start 2024-09-16T00:28:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.698+0000 7fdbfac42640 1 -- start start 2024-09-16T00:28:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbfac42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4103ff0 0x7fdbf419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:05.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbfac42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf41049a0 0x7fdbf419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:05.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbfac42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf4109120 0x7fdbf419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:05.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbfac42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4114130 con 0x7fdbf4103ff0 2024-09-16T00:28:05.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbfac42640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4113fb0 con 0x7fdbf41049a0 2024-09-16T00:28:05.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbfac42640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41142b0 con 0x7fdbf4109120 2024-09-16T00:28:05.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf89de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4103ff0 0x7fdbf419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:05.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf89de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4103ff0 0x7fdbf419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39392/0 (socket says 172.21.15.1:39392) 2024-09-16T00:28:05.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf41049a0 0x7fdbf419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:05.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf89de640 1 -- 172.21.15.1:0/1160661232 learned_addr learned my addr 172.21.15.1:0/1160661232 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:05.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf91df640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf4109120 0x7fdbf419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:05.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf89de640 1 -- 172.21.15.1:0/1160661232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf4109120 msgr2=0x7fdbf419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:05.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf89de640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf4109120 0x7fdbf419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf89de640 1 -- 172.21.15.1:0/1160661232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf41049a0 msgr2=0x7fdbf419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:05.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf89de640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf41049a0 0x7fdbf419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf89de640 1 -- 172.21.15.1:0/1160661232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf419f500 con 0x7fdbf4103ff0 2024-09-16T00:28:05.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.699+0000 7fdbf89de640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4103ff0 0x7fdbf419a550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800c890 tx=0x7fdbe800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:05.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.700+0000 7fdbf1ffb640 1 -- 172.21.15.1:0/1160661232 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8004220 con 0x7fdbf4103ff0 2024-09-16T00:28:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.700+0000 7fdbf1ffb640 1 -- 172.21.15.1:0/1160661232 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe80043c0 con 0x7fdbf4103ff0 2024-09-16T00:28:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.700+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf419f7f0 con 0x7fdbf4103ff0 2024-09-16T00:28:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.700+0000 7fdbf1ffb640 1 -- 172.21.15.1:0/1160661232 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8004a00 con 0x7fdbf4103ff0 2024-09-16T00:28:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.700+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf41a70d0 con 0x7fdbf4103ff0 2024-09-16T00:28:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.701+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf41043d0 con 0x7fdbf4103ff0 2024-09-16T00:28:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.702+0000 7fdbf1ffb640 1 -- 172.21.15.1:0/1160661232 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fdbe80079b0 con 0x7fdbf4103ff0 2024-09-16T00:28:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.703+0000 7fdbf1ffb640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdbe40a3350 0x7fdbe40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.703+0000 7fdbf1ffb640 1 -- 172.21.15.1:0/1160661232 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdbe80cfaa0 con 0x7fdbf4103ff0 2024-09-16T00:28:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.703+0000 7fdbf3fff640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdbe40a3350 0x7fdbe40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.703+0000 7fdbf3fff640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdbe40a3350 0x7fdbe40a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdbf410a580 tx=0x7fdbe000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.707+0000 7fdbf1ffb640 1 -- 172.21.15.1:0/1160661232 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbe8014030 con 0x7fdbf4103ff0 2024-09-16T00:28:05.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:05 smithi019 ceph-mon[35186]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:05.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.918+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbf419b8b0 con 0x7fdbf4103ff0 2024-09-16T00:28:05.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.919+0000 7fdbf1ffb640 1 -- 172.21.15.1:0/1160661232 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11894 (secure 0 0 0) 0x7fdbe809c200 con 0x7fdbf4103ff0 2024-09-16T00:28:05.920 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:05.920 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"btime":"2024-09-16T00:27:04:602627+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:27:04.334709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24470,24469]},"id":1}]} 2024-09-16T00:28:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 24 2024-09-16T00:28:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdbe40a3350 msgr2=0x7fdbe40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdbe40a3350 0x7fdbe40a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdbf410a580 tx=0x7fdbe000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4103ff0 msgr2=0x7fdbf419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4103ff0 0x7fdbf419a550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800c890 tx=0x7fdbe800cc90 comp rx=0 tx=0).stop 2024-09-16T00:28:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 shutdown_connections 2024-09-16T00:28:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdbe40a3350 0x7fdbe40a5810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf4109120 0x7fdbf419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf41049a0 0x7fdbf419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 --2- 172.21.15.1:0/1160661232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf4103ff0 0x7fdbf419a550 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:05.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.921+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 >> 172.21.15.1:0/1160661232 conn(0x7fdbf40ffa90 msgr2=0x7fdbf41005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:05.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.922+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 shutdown_connections 2024-09-16T00:28:05.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:05.922+0000 7fdbfac42640 1 -- 172.21.15.1:0/1160661232 wait complete. 2024-09-16T00:28:06.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:06 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1160661232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:06.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:28:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:06 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1160661232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:06.758 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:06 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1160661232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:06.945 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:07.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.442+0000 7fd67f91f640 1 -- 172.21.15.1:0/2003541789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678106f10 msgr2=0x7fd67810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.442+0000 7fd67f91f640 1 --2- 172.21.15.1:0/2003541789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678106f10 0x7fd67810ea90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd670009ca0 tx=0x7fd67002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.442+0000 7fd67f91f640 1 -- 172.21.15.1:0/2003541789 shutdown_connections 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.442+0000 7fd67f91f640 1 --2- 172.21.15.1:0/2003541789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678106f10 0x7fd67810ea90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.442+0000 7fd67f91f640 1 --2- 172.21.15.1:0/2003541789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678102790 0x7fd6781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.442+0000 7fd67f91f640 1 --2- 172.21.15.1:0/2003541789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678101de0 0x7fd6781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.442+0000 7fd67f91f640 1 -- 172.21.15.1:0/2003541789 >> 172.21.15.1:0/2003541789 conn(0x7fd6780fd8c0 msgr2=0x7fd6780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.443+0000 7fd67f91f640 1 -- 172.21.15.1:0/2003541789 shutdown_connections 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.443+0000 7fd67f91f640 1 -- 172.21.15.1:0/2003541789 wait complete. 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.443+0000 7fd67f91f640 1 Processor -- start 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.443+0000 7fd67f91f640 1 -- start start 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.443+0000 7fd67f91f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101de0 0x7fd6780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67f91f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678102790 0x7fd678074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:07.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67f91f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678106f10 0x7fd678074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67f91f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd678111f70 con 0x7fd678101de0 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67f91f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd678111df0 con 0x7fd678102790 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67f91f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781120f0 con 0x7fd678106f10 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67d6bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101de0 0x7fd6780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67d6bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101de0 0x7fd6780744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39406/0 (socket says 172.21.15.1:39406) 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67d6bb640 1 -- 172.21.15.1:0/1496580797 learned_addr learned my addr 172.21.15.1:0/1496580797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67ceba640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678102790 0x7fd678074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67debc640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678106f10 0x7fd678074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67d6bb640 1 -- 172.21.15.1:0/1496580797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678106f10 msgr2=0x7fd678074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67d6bb640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678106f10 0x7fd678074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67d6bb640 1 -- 172.21.15.1:0/1496580797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678102790 msgr2=0x7fd678074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67d6bb640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678102790 0x7fd678074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67d6bb640 1 -- 172.21.15.1:0/1496580797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6781a6890 con 0x7fd678101de0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67d6bb640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101de0 0x7fd6780744f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c970 tx=0x7fd67400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd66e7fc640 1 -- 172.21.15.1:0/1496580797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674007bb0 con 0x7fd678101de0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6781a6b80 con 0x7fd678101de0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd66e7fc640 1 -- 172.21.15.1:0/1496580797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd674007d50 con 0x7fd678101de0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd66e7fc640 1 -- 172.21.15.1:0/1496580797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67400f450 con 0x7fd678101de0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.444+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6781a70c0 con 0x7fd678101de0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.445+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6781021c0 con 0x7fd678101de0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.447+0000 7fd66e7fc640 1 -- 172.21.15.1:0/1496580797 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd674005000 con 0x7fd678101de0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.447+0000 7fd66e7fc640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6640a32d0 0x7fd6640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.447+0000 7fd67ceba640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6640a32d0 0x7fd6640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:07.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.447+0000 7fd66e7fc640 1 -- 172.21.15.1:0/1496580797 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd6740cfcc0 con 0x7fd678101de0 2024-09-16T00:28:07.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.448+0000 7fd67ceba640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6640a32d0 0x7fd6640a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd678075f80 tx=0x7fd66800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:07.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.451+0000 7fd66e7fc640 1 -- 172.21.15.1:0/1496580797 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd674014030 con 0x7fd678101de0 2024-09-16T00:28:07.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.629+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd67810a150 con 0x7fd678101de0 2024-09-16T00:28:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:07 smithi001 ceph-mon[30934]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:28:07.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.630+0000 7fd66e7fc640 1 -- 172.21.15.1:0/1496580797 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24902 (secure 0 0 0) 0x7fd67409c480 con 0x7fd678101de0 2024-09-16T00:28:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:07.631 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:27:26.243912+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:28:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6640a32d0 msgr2=0x7fd6640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6640a32d0 0x7fd6640a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd678075f80 tx=0x7fd66800b390 comp rx=0 tx=0).stop 2024-09-16T00:28:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101de0 msgr2=0x7fd6780744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101de0 0x7fd6780744f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c970 tx=0x7fd67400ce30 comp rx=0 tx=0).stop 2024-09-16T00:28:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 shutdown_connections 2024-09-16T00:28:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6640a32d0 0x7fd6640a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678106f10 0x7fd678074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678102790 0x7fd678074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 --2- 172.21.15.1:0/1496580797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101de0 0x7fd6780744f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 >> 172.21.15.1:0/1496580797 conn(0x7fd6780fd8c0 msgr2=0x7fd6780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 shutdown_connections 2024-09-16T00:28:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:07.633+0000 7fd67f91f640 1 -- 172.21.15.1:0/1496580797 wait complete. 2024-09-16T00:28:07.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:07 smithi153 ceph-mon[35538]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:28:07.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:07 smithi019 ceph-mon[35186]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:28:08.287 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:08.546 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:08.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:08 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1496580797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:08.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:08 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1496580797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:08 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1496580797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:09.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.023+0000 7f7030542640 1 -- 172.21.15.1:0/4198190409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028106f10 msgr2=0x7f702810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:09.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.023+0000 7f7030542640 1 --2- 172.21.15.1:0/4198190409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028106f10 0x7f702810ea90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7020009ca0 tx=0x7f702002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 -- 172.21.15.1:0/4198190409 shutdown_connections 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 --2- 172.21.15.1:0/4198190409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028106f10 0x7f702810ea90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 --2- 172.21.15.1:0/4198190409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7028102790 0x7f70281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 --2- 172.21.15.1:0/4198190409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028101de0 0x7f70281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 -- 172.21.15.1:0/4198190409 >> 172.21.15.1:0/4198190409 conn(0x7f70280fd8c0 msgr2=0x7f70280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 -- 172.21.15.1:0/4198190409 shutdown_connections 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 -- 172.21.15.1:0/4198190409 wait complete. 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 Processor -- start 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 -- start start 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028101de0 0x7f7028198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028102790 0x7f7028198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7028106f10 0x7f702819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:09.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.024+0000 7f7030542640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7028111fc0 con 0x7f7028102790 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f7030542640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7028111e40 con 0x7f7028101de0 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f7030542640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7028112140 con 0x7f7028106f10 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702dadd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028102790 0x7f7028198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702dadd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028102790 0x7f7028198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39424/0 (socket says 172.21.15.1:39424) 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702dadd640 1 -- 172.21.15.1:0/2018608767 learned_addr learned my addr 172.21.15.1:0/2018608767 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702e2de640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028101de0 0x7f7028198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702eadf640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7028106f10 0x7f702819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702dadd640 1 -- 172.21.15.1:0/2018608767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7028106f10 msgr2=0x7f702819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702dadd640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7028106f10 0x7f702819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702dadd640 1 -- 172.21.15.1:0/2018608767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028101de0 msgr2=0x7f7028198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702dadd640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028101de0 0x7f7028198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702dadd640 1 -- 172.21.15.1:0/2018608767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702819d3c0 con 0x7f7028102790 2024-09-16T00:28:09.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f702dadd640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028102790 0x7f7028198950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f701800c920 tx=0x7f701800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f701f7fe640 1 -- 172.21.15.1:0/2018608767 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018007c70 con 0x7f7028102790 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f701f7fe640 1 -- 172.21.15.1:0/2018608767 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7018007e10 con 0x7f7028102790 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702819d6b0 con 0x7f7028102790 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f701f7fe640 1 -- 172.21.15.1:0/2018608767 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018018890 con 0x7f7028102790 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.025+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70281a4f90 con 0x7f7028102790 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.027+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70281021c0 con 0x7f7028102790 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.028+0000 7f701f7fe640 1 -- 172.21.15.1:0/2018608767 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f7018018a70 con 0x7f7028102790 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.028+0000 7f701f7fe640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70140a3350 0x7f70140a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.028+0000 7f702e2de640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70140a3350 0x7f70140a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.028+0000 7f701f7fe640 1 -- 172.21.15.1:0/2018608767 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f70180cff00 con 0x7f7028102790 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.029+0000 7f702e2de640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70140a3350 0x7f70140a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f702400bbe0 tx=0x7f702400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:09.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.033+0000 7f701f7fe640 1 -- 172.21.15.1:0/2018608767 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7018014030 con 0x7f7028102790 2024-09-16T00:28:09.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.245+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70281a5220 con 0x7f7028102790 2024-09-16T00:28:09.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.246+0000 7f701f7fe640 1 -- 172.21.15.1:0/2018608767 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11894 (secure 0 0 0) 0x7f701809c6c0 con 0x7f7028102790 2024-09-16T00:28:09.247 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:09.247 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"btime":"2024-09-16T00:27:04:602627+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:27:04.334709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24470,24469]},"id":1}]} 2024-09-16T00:28:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 24 2024-09-16T00:28:09.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70140a3350 msgr2=0x7f70140a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:09.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70140a3350 0x7f70140a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f702400bbe0 tx=0x7f702400b390 comp rx=0 tx=0).stop 2024-09-16T00:28:09.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028102790 msgr2=0x7f7028198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028102790 0x7f7028198950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f701800c920 tx=0x7f701800cde0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 shutdown_connections 2024-09-16T00:28:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70140a3350 0x7f70140a5810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7028106f10 0x7f702819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7028102790 0x7f7028198950 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 --2- 172.21.15.1:0/2018608767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028101de0 0x7f7028198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 >> 172.21.15.1:0/2018608767 conn(0x7f70280fd8c0 msgr2=0x7f70280ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 shutdown_connections 2024-09-16T00:28:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:09.248+0000 7f7030542640 1 -- 172.21.15.1:0/2018608767 wait complete. 2024-09-16T00:28:09.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:09 smithi001 ceph-mon[30934]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:28:09.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:09 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2018608767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:09.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:09 smithi153 ceph-mon[35538]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:28:09.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:09 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2018608767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:09 smithi019 ceph-mon[35186]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:28:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:09 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2018608767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:10.022 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:26:18.890037+0000', 'modified': '2024-09-16T00:27:04.334709+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24470, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/389176790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 389176790}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 389176790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6835/3078906320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 3078906320}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 3078906320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6837/3490951208', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 3490951208}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 3490951208}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.153:6839/1065022363', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 1065022363}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 1065022363}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.19:6837/928170641', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6836', 'nonce': 928170641}, {'type': 'v1', 'addr': '172.21.15.19:6837', 'nonce': 928170641}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6839/3056769092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 3056769092}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 3056769092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24506, 24470, 24469]} 2024-09-16T00:28:10.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-16T00:28:10.023 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T00:28:10.287 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.792+0000 7f3fd80d7640 1 -- 172.21.15.1:0/3712721113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 msgr2=0x7f3fd010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.792+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/3712721113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 0x7f3fd010df40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009900 tx=0x7f3fc002a410 comp rx=0 tx=0).stop 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.792+0000 7f3fd80d7640 1 -- 172.21.15.1:0/3712721113 shutdown_connections 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.792+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/3712721113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fd010e480 0x7f3fd0110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.792+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/3712721113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 0x7f3fd010df40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.792+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/3712721113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd01008d0 0x7f3fd0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.792+0000 7f3fd80d7640 1 -- 172.21.15.1:0/3712721113 >> 172.21.15.1:0/3712721113 conn(0x7f3fd00fc7a0 msgr2=0x7f3fd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 -- 172.21.15.1:0/3712721113 shutdown_connections 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 -- 172.21.15.1:0/3712721113 wait complete. 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 Processor -- start 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 -- start start 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd01008d0 0x7f3fd019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 0x7f3fd019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fd010e480 0x7f3fd019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0114200 con 0x7f3fd0101280 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0114080 con 0x7f3fd010e480 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.793+0000 7f3fd80d7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0114380 con 0x7f3fd01008d0 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5672640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 0x7f3fd019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5672640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 0x7f3fd019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50538/0 (socket says 172.21.15.1:50538) 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5672640 1 -- 172.21.15.1:0/1600219696 learned_addr learned my addr 172.21.15.1:0/1600219696 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd6674640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fd010e480 0x7f3fd019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd01008d0 0x7f3fd019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5672640 1 -- 172.21.15.1:0/1600219696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd01008d0 msgr2=0x7f3fd019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5672640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd01008d0 0x7f3fd019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5672640 1 -- 172.21.15.1:0/1600219696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fd010e480 msgr2=0x7f3fd019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5672640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fd010e480 0x7f3fd019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5672640 1 -- 172.21.15.1:0/1600219696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd019f6d0 con 0x7f3fd0101280 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd01008d0 0x7f3fd019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd5672640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 0x7f3fd019ac60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0006010 tx=0x7f3fc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fc6ffd640 1 -- 172.21.15.1:0/1600219696 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc003f030 con 0x7f3fd0101280 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fc6ffd640 1 -- 172.21.15.1:0/1600219696 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fc0003ce0 con 0x7f3fd0101280 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd80d7640 1 -- 172.21.15.1:0/1600219696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd019f960 con 0x7f3fd0101280 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fc6ffd640 1 -- 172.21.15.1:0/1600219696 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0035070 con 0x7f3fd0101280 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.794+0000 7f3fd80d7640 1 -- 172.21.15.1:0/1600219696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd01a72a0 con 0x7f3fd0101280 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.797+0000 7f3fc6ffd640 1 -- 172.21.15.1:0/1600219696 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f3fc0042030 con 0x7f3fd0101280 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.797+0000 7f3fda4b0740 1 Processor -- start 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.797+0000 7f3fda4b0740 1 -- start start 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 0x55cd2e8fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e856090 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e7fda70 con 0x55cd2e8f02f0 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e852050 con 0x55cd2e8fabb0 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8efdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50542/0 (socket says 172.21.15.1:50542) 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 -- 172.21.15.1:0/2907435464 learned_addr learned my addr 172.21.15.1:0/2907435464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fc6ffd640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fbc0a3350 0x7f3fbc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd5672640 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 0x55cd2e8fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 -- 172.21.15.1:0/2907435464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 msgr2=0x55cd2e8fcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 0x55cd2e8fcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 -- 172.21.15.1:0/2907435464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 msgr2=0x55cd2e8fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8fa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 -- 172.21.15.1:0/2907435464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2e80a130 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fc6ffd640 1 -- 172.21.15.1:0/1600219696 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3fc00f0490 con 0x7f3fd0101280 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd5672640 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 0x55cd2e8fcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fbc0a3350 0x7f3fbc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.798+0000 7f3fd6674640 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8efdb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8010510 tx=0x7f3fc80109d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fff6ae4d39539b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fbbfff640 1 -- 172.21.15.1:0/2907435464 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc801d400 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fbbfff640 1 -- 172.21.15.1:0/2907435464 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fc80134c0 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fbbfff640 1 -- 172.21.15.1:0/2907435464 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80118a0 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fd6674640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fbc0a3350 0x7f3fbc0a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8021000 tx=0x7f3fc801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2907435464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 msgr2=0x55cd2e8efdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8efdb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8010510 tx=0x7f3fc80109d0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2907435464 shutdown_connections 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 0x55cd2e8fcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2907435464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8efdb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.799+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2907435464 >> 172.21.15.1:0/2907435464 conn(0x55cd2e8ebf60 msgr2=0x55cd2e8ec340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2907435464 shutdown_connections 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2907435464 wait complete. 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 -- start start 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 Processor -- start 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 0x55cd2e8f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e8fd780 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e810f20 con 0x55cd2e8fabb0 2024-09-16T00:28:10.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e8fd900 con 0x55cd2e8f02f0 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fda4b0740 10 client.0 _ll_register_callbacks cb 0x55cd2e7da030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.800+0000 7f3fd5e73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5e73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8fa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50558/0 (socket says 172.21.15.1:50558) 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5e73640 1 -- 172.21.15.1:0/2583221178 learned_addr learned my addr 172.21.15.1:0/2583221178 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd6674640 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 0x55cd2e8f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5672640 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5e73640 1 -- 172.21.15.1:0/2583221178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 msgr2=0x55cd2e8f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5e73640 1 -- 172.21.15.1:0/2583221178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 msgr2=0x55cd2e8f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 0x55cd2e8f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5e73640 1 -- 172.21.15.1:0/2583221178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2e5c72b0 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5672640 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8fa450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00df00 tx=0x7f3fcc00f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc79effd6136570 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fb97fa640 1 -- 172.21.15.1:0/2583221178 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc018070 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fb97fa640 1 -- 172.21.15.1:0/2583221178 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3fcc014680 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2583221178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fc80134c0 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fb97fa640 1 -- 172.21.15.1:0/2583221178 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3fcc024050 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fda4b0740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.801+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2583221178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cd2e909070 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.802+0000 7f3fb97fa640 1 -- 172.21.15.1:0/2583221178 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fc80134c0 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.802+0000 7f3fb97fa640 1 -- 172.21.15.1:0/2583221178 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f3fcc013b70 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.802+0000 7f3fda4b0740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.802+0000 7f3fda4b0740 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.802+0000 7f3fda4b0740 20 client.15066 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:10.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.802+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cd2e90b650 0x55cd2e90cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.802+0000 7f3fda4b0740 4 client.15066 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.802+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2583221178 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd2e863600 con 0x55cd2e90b650 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.802+0000 7f3fd5672640 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cd2e90b650 0x55cd2e90cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.803+0000 7f3fd5672640 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cd2e90b650 0x55cd2e90cac0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ac19dbeafb26e70 server_cookie=c6ecabb91c4cab03 in_seq=0 out_seq=0 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.803+0000 7f3fb97fa640 10 client.15066 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.803+0000 7f3fb97fa640 1 -- 172.21.15.1:0/2583221178 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd2e863600 con 0x55cd2e90b650 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.803+0000 7f3fb97fa640 10 client.15066 handle_command_reply: tid=0 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.803+0000 7f3fda4b0740 1 client.15066 shutdown 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2583221178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 msgr2=0x55cd2e8fa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8fa450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00df00 tx=0x7f3fcc00f5b0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2583221178 shutdown_connections 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cd2e90b650 0x55cd2e90cac0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 0x55cd2e8f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/2583221178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8fa450 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fb97fa640 0 client.15066 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2583221178 >> 172.21.15.1:0/2583221178 conn(0x55cd2e8ebf60 msgr2=0x55cd2e8f1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2583221178 shutdown_connections 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 1 -- 172.21.15.1:0/2583221178 wait complete. 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.804+0000 7f3fda4b0740 20 client.15066 trim_cache size 0 max 16384 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.821+0000 7f3fda4b0740 1 Processor -- start 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fda4b0740 1 -- start start 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 0x55cd2e914da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e90ae00 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e90b4d0 con 0x55cd2e8fabb0 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e90bba0 con 0x55cd2e8f02f0 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5e73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5e73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50560/0 (socket says 172.21.15.1:50560) 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5e73640 1 -- 172.21.15.1:0/460605234 learned_addr learned my addr 172.21.15.1:0/460605234 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd6674640 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 0x55cd2e914da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5672640 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5e73640 1 -- 172.21.15.1:0/460605234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 msgr2=0x55cd2e8f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5e73640 1 -- 172.21.15.1:0/460605234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 msgr2=0x55cd2e914da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 0x55cd2e914da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5e73640 1 -- 172.21.15.1:0/460605234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2e8fd890 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5672640 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.822+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8f4a40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc006ca0 tx=0x7f3fcc00da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106e6ef942b61321 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fbb7fe640 1 -- 172.21.15.1:0/460605234 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc018040 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fbb7fe640 1 -- 172.21.15.1:0/460605234 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fcc01c520 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fbb7fe640 1 -- 172.21.15.1:0/460605234 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc006280 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fda4b0740 1 -- 172.21.15.1:0/460605234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 msgr2=0x55cd2e8f4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8f4a40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc006ca0 tx=0x7f3fcc00da70 comp rx=0 tx=0).stop 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fda4b0740 1 -- 172.21.15.1:0/460605234 shutdown_connections 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8fabb0 0x55cd2e914da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8f02f0 0x55cd2e8f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/460605234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e8f4a40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fda4b0740 1 -- 172.21.15.1:0/460605234 >> 172.21.15.1:0/460605234 conn(0x55cd2e8ebf60 msgr2=0x55cd2e8f6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fda4b0740 1 -- 172.21.15.1:0/460605234 shutdown_connections 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.823+0000 7f3fda4b0740 1 -- 172.21.15.1:0/460605234 wait complete. 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fda4b0740 1 -- start start 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fda4b0740 1 Processor -- start 2024-09-16T00:28:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e9102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e9107f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fda4b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 0x55cd2e973d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e90c940 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e90c270 con 0x55cd2e8f02f0 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fda4b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd2e90d010 con 0x55cd2e8fabb0 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fda4b0740 10 client.0 _ll_register_callbacks cb 0x7f3fcc00d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fd5e73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e9102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fd5e73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e9102b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50570/0 (socket says 172.21.15.1:50570) 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fd5e73640 1 -- 172.21.15.1:0/3667164934 learned_addr learned my addr 172.21.15.1:0/3667164934 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.824+0000 7f3fd5672640 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e9107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fd5e73640 1 -- 172.21.15.1:0/3667164934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 msgr2=0x55cd2e973d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 0x55cd2e973d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fd5e73640 1 -- 172.21.15.1:0/3667164934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 msgr2=0x55cd2e9107f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e9107f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fd5e73640 1 -- 172.21.15.1:0/3667164934 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2e974390 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fd5672640 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e9107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fd5e73640 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e9102b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00f5f0 tx=0x7f3fcc01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc427f717272b99b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fb97fa640 1 -- 172.21.15.1:0/3667164934 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc018040 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fda4b0740 1 -- 172.21.15.1:0/3667164934 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fcc01c520 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fb97fa640 1 -- 172.21.15.1:0/3667164934 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3fcc005bd0 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fb97fa640 1 -- 172.21.15.1:0/3667164934 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3fcc039050 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fda4b0740 10 client.15072 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fda4b0740 1 -- 172.21.15.1:0/3667164934 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cd2e9755a0 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fb97fa640 1 -- 172.21.15.1:0/3667164934 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fcc01c520 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fb97fa640 1 -- 172.21.15.1:0/3667164934 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f3fcc006740 con 0x55cd2e8ef9d0 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.825+0000 7f3fda4b0740 10 client.15072 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fda4b0740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fda4b0740 20 client.15072 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cd2e979e70 0x55cd2e978b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fda4b0740 4 client.15072 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fda4b0740 1 -- 172.21.15.1:0/3667164934 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "status"}) -- 0x55cd2e85db10 con 0x55cd2e979e70 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fd5672640 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cd2e979e70 0x55cd2e978b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fd5672640 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cd2e979e70 0x55cd2e978b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62a363f01f6060cc server_cookie=fd222fd2aa5ea1c8 in_seq=0 out_seq=0 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fb97fa640 10 client.15072 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fb97fa640 1 -- 172.21.15.1:0/3667164934 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cd2e85db10 con 0x55cd2e979e70 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.826+0000 7f3fb97fa640 10 client.15072 handle_command_reply: tid=0 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 client.15072 shutdown 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 -- 172.21.15.1:0/3667164934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 msgr2=0x55cd2e9102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:10.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e9102b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00f5f0 tx=0x7f3fcc01c900 comp rx=0 tx=0).stop 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 -- 172.21.15.1:0/3667164934 shutdown_connections 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd2e8fabb0 0x55cd2e973d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cd2e979e70 0x55cd2e978b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd2e8f02f0 0x55cd2e9107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 --2- 172.21.15.1:0/3667164934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd2e8ef9d0 0x55cd2e9102b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fb97fa640 0 client.15072 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 -- 172.21.15.1:0/3667164934 >> 172.21.15.1:0/3667164934 conn(0x55cd2e8ebf60 msgr2=0x55cd2e8f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 -- 172.21.15.1:0/3667164934 shutdown_connections 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 1 -- 172.21.15.1:0/3667164934 wait complete. 2024-09-16T00:28:10.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fda4b0740 20 client.15072 trim_cache size 0 max 16384 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24469, 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 69.227044911999997, 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 24, 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 89, 2024-09-16T00:28:10.845 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 131.947614684 2024-09-16T00:28:10.846 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:28:10.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fd80d7640 1 -- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fbc0a3350 msgr2=0x7f3fbc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:10.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fbc0a3350 0x7f3fbc0a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8021000 tx=0x7f3fc801e000 comp rx=0 tx=0).stop 2024-09-16T00:28:10.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fd80d7640 1 -- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 msgr2=0x7f3fd019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:10.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.827+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 0x7f3fd019ac60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0006010 tx=0x7f3fc00042f0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.828+0000 7f3fd80d7640 1 -- 172.21.15.1:0/1600219696 shutdown_connections 2024-09-16T00:28:10.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.828+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fd010e480 0x7f3fd019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.828+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0101280 0x7f3fd019ac60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.828+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fbc0a3350 0x7f3fbc0a5810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.828+0000 7f3fd80d7640 1 --2- 172.21.15.1:0/1600219696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd01008d0 0x7f3fd019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:10.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.828+0000 7f3fd80d7640 1 -- 172.21.15.1:0/1600219696 >> 172.21.15.1:0/1600219696 conn(0x7f3fd00fc7a0 msgr2=0x7f3fd00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:10.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.828+0000 7f3fd80d7640 1 -- 172.21.15.1:0/1600219696 shutdown_connections 2024-09-16T00:28:10.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:10.828+0000 7f3fd80d7640 1 -- 172.21.15.1:0/1600219696 wait complete. 2024-09-16T00:28:11.669 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T00:28:11.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:11 smithi001 ceph-mon[30934]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:11.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:11 smithi153 ceph-mon[35538]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:11.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:11 smithi019 ceph-mon[35186]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:11.927 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 -- 172.21.15.1:0/4199699814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4108290 msgr2=0x7fb3d410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/4199699814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4108290 0x7fb3d410ec30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009cd0 tx=0x7fb3cc02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 -- 172.21.15.1:0/4199699814 shutdown_connections 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/4199699814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4108290 0x7fb3d410ec30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/4199699814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3d4103740 0x7fb3d4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/4199699814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3d4102d90 0x7fb3d4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 -- 172.21.15.1:0/4199699814 >> 172.21.15.1:0/4199699814 conn(0x7fb3d40fc7d0 msgr2=0x7fb3d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 -- 172.21.15.1:0/4199699814 shutdown_connections 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 -- 172.21.15.1:0/4199699814 wait complete. 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 Processor -- start 2024-09-16T00:28:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.412+0000 7fb3dabe4640 1 -- start start 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3dabe4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4102d90 0x7fb3d41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3dabe4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3d4103740 0x7fb3d4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3dabe4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3d4108290 0x7fb3d419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3dabe4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d41120d0 con 0x7fb3d4102d90 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3dabe4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d4111f50 con 0x7fb3d4108290 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3dabe4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d4112250 con 0x7fb3d4103740 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d8980640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4102d90 0x7fb3d41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d8980640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4102d90 0x7fb3d41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50602/0 (socket says 172.21.15.1:50602) 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d8980640 1 -- 172.21.15.1:0/3797250544 learned_addr learned my addr 172.21.15.1:0/3797250544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d9181640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3d4108290 0x7fb3d419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d3fff640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3d4103740 0x7fb3d4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d8980640 1 -- 172.21.15.1:0/3797250544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3d4103740 msgr2=0x7fb3d4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d8980640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3d4103740 0x7fb3d4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d8980640 1 -- 172.21.15.1:0/3797250544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3d4108290 msgr2=0x7fb3d419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d8980640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3d4108290 0x7fb3d419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d8980640 1 -- 172.21.15.1:0/3797250544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d419d560 con 0x7fb3d4102d90 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.413+0000 7fb3d8980640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4102d90 0x7fb3d41985b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800b4a0 tx=0x7fb3c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.414+0000 7fb3d1ffb640 1 -- 172.21.15.1:0/3797250544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8014070 con 0x7fb3d4102d90 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.414+0000 7fb3dabe4640 1 -- 172.21.15.1:0/3797250544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d419d850 con 0x7fb3d4102d90 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.414+0000 7fb3d1ffb640 1 -- 172.21.15.1:0/3797250544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c8010af0 con 0x7fb3d4102d90 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.414+0000 7fb3d1ffb640 1 -- 172.21.15.1:0/3797250544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8017d10 con 0x7fb3d4102d90 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.414+0000 7fb3dabe4640 1 -- 172.21.15.1:0/3797250544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d41a5130 con 0x7fb3d4102d90 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.416+0000 7fb3d1ffb640 1 -- 172.21.15.1:0/3797250544 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fb3c8017070 con 0x7fb3d4102d90 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.416+0000 7fb3dcfbd740 1 Processor -- start 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.416+0000 7fb3dcfbd740 1 -- start start 2024-09-16T00:28:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 0x55c51fb41db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb422f0 0x55c51fb4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 0x55c51fb4efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c51faa8090 con 0x55c51fb422f0 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c51fa4fa70 con 0x55c51fb4cbb0 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c51faa4050 con 0x55c51fb419d0 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d1ffb640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb3c40a32d0 0x7fb3c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d8980640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb422f0 0x55c51fb4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d8980640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb422f0 0x55c51fb4c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50604/0 (socket says 172.21.15.1:50604) 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d8980640 1 -- 172.21.15.1:0/2477403824 learned_addr learned my addr 172.21.15.1:0/2477403824 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d3fff640 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 0x55c51fb4efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d9181640 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 0x55c51fb41db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d9181640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb3c40a32d0 0x7fb3c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d1ffb640 1 -- 172.21.15.1:0/3797250544 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb3c80cf0b0 con 0x7fb3d4102d90 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d8980640 1 -- 172.21.15.1:0/2477403824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 msgr2=0x55c51fb41db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d8980640 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 0x55c51fb41db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d8980640 1 -- 172.21.15.1:0/2477403824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 msgr2=0x55c51fb4efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d8980640 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 0x55c51fb4efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d8980640 1 -- 172.21.15.1:0/2477403824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51fa5c130 con 0x55c51fb422f0 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d3fff640 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 0x55c51fb4efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.417+0000 7fb3d8980640 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb422f0 0x55c51fb4c670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800be60 tx=0x7fb3c80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb710f48e98489be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3d9181640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb3c40a32d0 0x7fb3c40a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc004ab0 tx=0x7fb3cc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3beffd640 1 -- 172.21.15.1:0/2477403824 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c801e020 con 0x55c51fb422f0 2024-09-16T00:28:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3beffd640 1 -- 172.21.15.1:0/2477403824 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c809f630 con 0x55c51fb422f0 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3beffd640 1 -- 172.21.15.1:0/2477403824 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8010e00 con 0x55c51fb422f0 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/2477403824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb422f0 msgr2=0x55c51fb4c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb422f0 0x55c51fb4c670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800be60 tx=0x7fb3c80047d0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/2477403824 shutdown_connections 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 0x55c51fb4efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb422f0 0x55c51fb4c670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/2477403824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 0x55c51fb41db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/2477403824 >> 172.21.15.1:0/2477403824 conn(0x55c51fb3df60 msgr2=0x55c51fb3e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.418+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/2477403824 shutdown_connections 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.419+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/2477403824 wait complete. 2024-09-16T00:28:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.419+0000 7fb3dcfbd740 1 -- start start 2024-09-16T00:28:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.419+0000 7fb3dcfbd740 1 Processor -- start 2024-09-16T00:28:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.419+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb419d0 0x55c51fb4c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.419+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 0x55c51fb49720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.419+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.419+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c51fb4f780 con 0x55c51fb4cbb0 2024-09-16T00:28:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.419+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c51fa62f20 con 0x55c51fb419d0 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.419+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c51fb4f900 con 0x55c51fb422f0 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3dcfbd740 10 client.0 _ll_register_callbacks cb 0x55c51fa2c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d9181640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d9181640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50616/0 (socket says 172.21.15.1:50616) 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d9181640 1 -- 172.21.15.1:0/229985746 learned_addr learned my addr 172.21.15.1:0/229985746 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d8980640 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb419d0 0x55c51fb4c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d3fff640 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 0x55c51fb49720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d9181640 1 -- 172.21.15.1:0/229985746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 msgr2=0x55c51fb49720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d9181640 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 0x55c51fb49720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d9181640 1 -- 172.21.15.1:0/229985746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb419d0 msgr2=0x55c51fb4c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d9181640 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb419d0 0x55c51fb4c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d9181640 1 -- 172.21.15.1:0/229985746 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51f8192b0 con 0x55c51fb4cbb0 2024-09-16T00:28:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d3fff640 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 0x55c51fb49720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3d9181640 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc035000 tx=0x7fb3cc02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452ef0a504b74c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3b3fff640 1 -- 172.21.15.1:0/229985746 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc0307a0 con 0x55c51fb4cbb0 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3b3fff640 1 -- 172.21.15.1:0/229985746 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb3cc030940 con 0x55c51fb4cbb0 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/229985746 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3c809f630 con 0x55c51fb4cbb0 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.420+0000 7fb3b3fff640 1 -- 172.21.15.1:0/229985746 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb3cc030b40 con 0x55c51fb4cbb0 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.421+0000 7fb3dcfbd740 10 client.15084 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.421+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/229985746 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c51fb5b030 con 0x55c51fb4cbb0 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.423+0000 7fb3b3fff640 1 -- 172.21.15.1:0/229985746 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c809f630 con 0x55c51fb4cbb0 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.423+0000 7fb3b3fff640 1 -- 172.21.15.1:0/229985746 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fb3cc048040 con 0x55c51fb4cbb0 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.423+0000 7fb3dcfbd740 10 client.15084 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.423+0000 7fb3dcfbd740 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.423+0000 7fb3dcfbd740 20 client.15084 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.423+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55c51fb5d750 0x55c51fb5ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.423+0000 7fb3dcfbd740 4 client.15084 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.423+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/229985746 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c51fab5600 con 0x55c51fb5d750 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.423+0000 7fb3d8980640 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55c51fb5d750 0x55c51fb5ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.424+0000 7fb3d8980640 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55c51fb5d750 0x55c51fb5ebc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcc1255fa5f860ae server_cookie=81773235eb28344e in_seq=0 out_seq=0 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.424+0000 7fb3b3fff640 10 client.15084 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3b3fff640 1 -- 172.21.15.1:0/229985746 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c51fab5600 con 0x55c51fb5d750 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3b3fff640 10 client.15084 handle_command_reply: tid=0 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 client.15084 shutdown 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/229985746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 msgr2=0x55c51fb49c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc035000 tx=0x7fb3cc02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/229985746 shutdown_connections 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55c51fb5d750 0x55c51fb5ebc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 0x55c51fb49720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/229985746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb419d0 0x55c51fb4c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/229985746 >> 172.21.15.1:0/229985746 conn(0x55c51fb3df60 msgr2=0x55c51fb43c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3b3fff640 0 client.15084 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:28:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/229985746 shutdown_connections 2024-09-16T00:28:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/229985746 wait complete. 2024-09-16T00:28:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.425+0000 7fb3dcfbd740 20 client.15084 trim_cache size 0 max 16384 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.443+0000 7fb3dcfbd740 1 Processor -- start 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.443+0000 7fb3dcfbd740 1 -- start start 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 0x55c51fb5d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb422f0 0x55c51fb496e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c51fb5cf00 con 0x55c51fb4cbb0 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c51fb5d5d0 con 0x55c51fb422f0 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c51fb5dca0 con 0x55c51fb419d0 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d9181640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d9181640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50620/0 (socket says 172.21.15.1:50620) 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d9181640 1 -- 172.21.15.1:0/419584881 learned_addr learned my addr 172.21.15.1:0/419584881 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d3fff640 1 --2- 172.21.15.1:0/419584881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb422f0 0x55c51fb496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d8980640 1 --2- 172.21.15.1:0/419584881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 0x55c51fb5d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d9181640 1 -- 172.21.15.1:0/419584881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 msgr2=0x55c51fb5d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d9181640 1 --2- 172.21.15.1:0/419584881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 0x55c51fb5d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d9181640 1 -- 172.21.15.1:0/419584881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb422f0 msgr2=0x55c51fb496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d9181640 1 --2- 172.21.15.1:0/419584881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb422f0 0x55c51fb496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d9181640 1 -- 172.21.15.1:0/419584881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51fb3dd90 con 0x55c51fb4cbb0 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.444+0000 7fb3d9181640 1 --2- 172.21.15.1:0/419584881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009860 tx=0x7fb3cc005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caa318e208f0038 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3be7fc640 1 -- 172.21.15.1:0/419584881 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc003c80 con 0x55c51fb4cbb0 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3be7fc640 1 -- 172.21.15.1:0/419584881 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3cc036070 con 0x55c51fb4cbb0 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3be7fc640 1 -- 172.21.15.1:0/419584881 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc004d60 con 0x55c51fb4cbb0 2024-09-16T00:28:12.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/419584881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 msgr2=0x55c51fb49c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/419584881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009860 tx=0x7fb3cc005cf0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/419584881 shutdown_connections 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/419584881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb4cbb0 0x55c51fb49c20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/419584881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb422f0 0x55c51fb496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/419584881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb419d0 0x55c51fb5d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/419584881 >> 172.21.15.1:0/419584881 conn(0x55c51fb3df60 msgr2=0x55c51fb38ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/419584881 shutdown_connections 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.445+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/419584881 wait complete. 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3dcfbd740 1 -- start start 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3dcfbd740 1 Processor -- start 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb419d0 0x55c51fb61dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 0x55c51fb62310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3dcfbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 0x55c51fb62850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c51fb5ea40 con 0x55c51fb419d0 2024-09-16T00:28:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c51fb5e370 con 0x55c51fb4cbb0 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3dcfbd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c51fb5f110 con 0x55c51fb422f0 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3dcfbd740 10 client.0 _ll_register_callbacks cb 0x7fb3cc009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3d8980640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb419d0 0x55c51fb61dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3d8980640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb419d0 0x55c51fb61dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50622/0 (socket says 172.21.15.1:50622) 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.446+0000 7fb3d8980640 1 -- 172.21.15.1:0/801656704 learned_addr learned my addr 172.21.15.1:0/801656704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3d9181640 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 0x55c51fb62850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3d3fff640 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 0x55c51fb62310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3d8980640 1 -- 172.21.15.1:0/801656704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 msgr2=0x55c51fb62310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3d8980640 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 0x55c51fb62310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3d8980640 1 -- 172.21.15.1:0/801656704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 msgr2=0x55c51fb62850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3d8980640 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 0x55c51fb62850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3d8980640 1 -- 172.21.15.1:0/801656704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51fbc60a0 con 0x55c51fb419d0 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3d8980640 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb419d0 0x55c51fb61dd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80cc7f0 tx=0x7fb3c809b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d714b715c5d3ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3b3fff640 1 -- 172.21.15.1:0/801656704 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c801e040 con 0x55c51fb419d0 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3b3fff640 1 -- 172.21.15.1:0/801656704 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb3c80979a0 con 0x55c51fb419d0 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/801656704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3cc036070 con 0x55c51fb419d0 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3b3fff640 1 -- 172.21.15.1:0/801656704 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb3c809f630 con 0x55c51fb419d0 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3dcfbd740 10 client.15090 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.447+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/801656704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c51fbc7340 con 0x55c51fb419d0 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3b3fff640 1 -- 172.21.15.1:0/801656704 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3cc036070 con 0x55c51fb419d0 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3b3fff640 1 -- 172.21.15.1:0/801656704 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fb3c80bb0b0 con 0x55c51fb419d0 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3dcfbd740 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3dcfbd740 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3dcfbd740 20 client.15090 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55c51fbcb950 0x55c51fbca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3dcfbd740 4 client.15090 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:28:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/801656704 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "status"}) -- 0x55c51faafb10 con 0x55c51fbcb950 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3d3fff640 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55c51fbcb950 0x55c51fbca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3b3fff640 10 client.15090 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.448+0000 7fb3d3fff640 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55c51fbcb950 0x55c51fbca660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b01f4ab8ec4b0653 server_cookie=f1a2338dd9485b0a in_seq=0 out_seq=0 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3b3fff640 1 -- 172.21.15.1:0/801656704 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c51faafb10 con 0x55c51fbcb950 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3b3fff640 10 client.15090 handle_command_reply: tid=0 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 client.15090 shutdown 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/801656704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb419d0 msgr2=0x55c51fb61dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb419d0 0x55c51fb61dd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80cc7f0 tx=0x7fb3c809b4b0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/801656704 shutdown_connections 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c51fb4cbb0 0x55c51fb62850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c51fb422f0 0x55c51fb62310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55c51fbcb950 0x55c51fbca660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 --2- 172.21.15.1:0/801656704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c51fb419d0 0x55c51fb61dd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3b3fff640 0 client.15090 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:28:12.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/801656704 >> 172.21.15.1:0/801656704 conn(0x55c51fb3df60 msgr2=0x55c51fb496e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/801656704 shutdown_connections 2024-09-16T00:28:12.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 1 -- 172.21.15.1:0/801656704 wait complete. 2024-09-16T00:28:12.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.449+0000 7fb3dcfbd740 20 client.15090 trim_cache size 0 max 16384 2024-09-16T00:28:12.458 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:28:12.458 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24470, 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 71.855159235000002, 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 24, 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 89, 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 136.31103777999999 2024-09-16T00:28:12.459 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:28:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 -- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb3c40a32d0 msgr2=0x7fb3c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb3c40a32d0 0x7fb3c40a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc004ab0 tx=0x7fb3cc004a40 comp rx=0 tx=0).stop 2024-09-16T00:28:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 -- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4102d90 msgr2=0x7fb3d41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4102d90 0x7fb3d41985b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800b4a0 tx=0x7fb3c800b960 comp rx=0 tx=0).stop 2024-09-16T00:28:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 -- 172.21.15.1:0/3797250544 shutdown_connections 2024-09-16T00:28:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3d4108290 0x7fb3d419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb3c40a32d0 0x7fb3c40a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3d4103740 0x7fb3d4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 --2- 172.21.15.1:0/3797250544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4102d90 0x7fb3d41985b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 -- 172.21.15.1:0/3797250544 >> 172.21.15.1:0/3797250544 conn(0x7fb3d40fc7d0 msgr2=0x7fb3d4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 -- 172.21.15.1:0/3797250544 shutdown_connections 2024-09-16T00:28:12.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:12.450+0000 7fb3dabe4640 1 -- 172.21.15.1:0/3797250544 wait complete. 2024-09-16T00:28:13.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:28:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:13.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:13 smithi001 ceph-mon[30934]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:28:13.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:13 smithi153 ceph-mon[35538]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:28:13.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:13 smithi019 ceph-mon[35186]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:28:13.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 -- 172.21.15.1:0/2160705497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc106a80 msgr2=0x7fd5bc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:13.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/2160705497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc106a80 0x7fd5bc106e60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009930 tx=0x7fd5b002a420 comp rx=0 tx=0).stop 2024-09-16T00:28:13.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 -- 172.21.15.1:0/2160705497 shutdown_connections 2024-09-16T00:28:13.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/2160705497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc101420 0x7fd5bc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:13.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/2160705497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5bc100a80 0x7fd5bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:13.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/2160705497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc106a80 0x7fd5bc106e60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:13.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 -- 172.21.15.1:0/2160705497 >> 172.21.15.1:0/2160705497 conn(0x7fd5bc0fc7a0 msgr2=0x7fd5bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:13.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 -- 172.21.15.1:0/2160705497 shutdown_connections 2024-09-16T00:28:13.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 -- 172.21.15.1:0/2160705497 wait complete. 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 Processor -- start 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.979+0000 7fd5c3c23640 1 -- start start 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c3c23640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc100a80 0x7fd5bc198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c3c23640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc101420 0x7fd5bc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c3c23640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5bc106a80 0x7fd5bc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c3c23640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc110e80 con 0x7fd5bc100a80 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c3c23640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc110d00 con 0x7fd5bc106a80 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c3c23640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc111000 con 0x7fd5bc101420 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c19bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc100a80 0x7fd5bc198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c19bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc100a80 0x7fd5bc198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50648/0 (socket says 172.21.15.1:50648) 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c19bf640 1 -- 172.21.15.1:0/1445710006 learned_addr learned my addr 172.21.15.1:0/1445710006 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c11be640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc101420 0x7fd5bc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c21c0640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5bc106a80 0x7fd5bc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c19bf640 1 -- 172.21.15.1:0/1445710006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc101420 msgr2=0x7fd5bc198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc101420 0x7fd5bc198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c19bf640 1 -- 172.21.15.1:0/1445710006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5bc106a80 msgr2=0x7fd5bc19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5bc106a80 0x7fd5bc19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c19bf640 1 -- 172.21.15.1:0/1445710006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc19d3f0 con 0x7fd5bc100a80 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c21c0640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5bc106a80 0x7fd5bc19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc100a80 0x7fd5bc198440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009230 tx=0x7fd5b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5aeffd640 1 -- 172.21.15.1:0/1445710006 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b003f030 con 0x7fd5bc100a80 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c3c23640 1 -- 172.21.15.1:0/1445710006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc19d680 con 0x7fd5bc100a80 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5aeffd640 1 -- 172.21.15.1:0/1445710006 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b0003ad0 con 0x7fd5bc100a80 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5aeffd640 1 -- 172.21.15.1:0/1445710006 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0035070 con 0x7fd5bc100a80 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.980+0000 7fd5c3c23640 1 -- 172.21.15.1:0/1445710006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc1a4fc0 con 0x7fd5bc100a80 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.983+0000 7fd5aeffd640 1 -- 172.21.15.1:0/1445710006 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd5b0042030 con 0x7fd5bc100a80 2024-09-16T00:28:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.983+0000 7fd5c5ffc740 1 Processor -- start 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.983+0000 7fd5c5ffc740 1 -- start start 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e2fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e301d0 0x564385e3ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 0x564385e413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564385d96090 con 0x564385e2f8b0 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564385d3da70 con 0x564385e3efc0 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564385d92050 con 0x564385e301d0 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c21c0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e2fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5aeffd640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5a80a32d0 0x7fd5a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c21c0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e2fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50664/0 (socket says 172.21.15.1:50664) 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c21c0640 1 -- 172.21.15.1:0/1303310875 learned_addr learned my addr 172.21.15.1:0/1303310875 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c11be640 1 --2- 172.21.15.1:0/1303310875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 0x564385e413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/1303310875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e301d0 0x564385e3ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c21c0640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5a80a32d0 0x7fd5a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5aeffd640 1 -- 172.21.15.1:0/1445710006 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd5b00f0210 con 0x7fd5bc100a80 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c11be640 1 -- 172.21.15.1:0/1303310875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e301d0 msgr2=0x564385e3ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c11be640 1 --2- 172.21.15.1:0/1303310875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e301d0 0x564385e3ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c11be640 1 -- 172.21.15.1:0/1303310875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 msgr2=0x564385e2fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c11be640 1 --2- 172.21.15.1:0/1303310875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e2fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c11be640 1 -- 172.21.15.1:0/1303310875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564385d4a130 con 0x564385e3efc0 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.984+0000 7fd5c11be640 1 --2- 172.21.15.1:0/1303310875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 0x564385e413b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009b40 tx=0x7fd5b80113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd733c925b51dd14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5c21c0640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5a80a32d0 0x7fd5a80a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40092f0 tx=0x7fd5b40099c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5a7fff640 1 -- 172.21.15.1:0/1303310875 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8011e10 con 0x564385e3efc0 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5a7fff640 1 -- 172.21.15.1:0/1303310875 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b8012930 con 0x564385e3efc0 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5a7fff640 1 -- 172.21.15.1:0/1303310875 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800f970 con 0x564385e3efc0 2024-09-16T00:28:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/1303310875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 msgr2=0x564385e413b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/1303310875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 0x564385e413b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009b40 tx=0x7fd5b80113e0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/1303310875 shutdown_connections 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/1303310875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 0x564385e413b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/1303310875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e301d0 0x564385e3ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/1303310875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e2fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.985+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/1303310875 >> 172.21.15.1:0/1303310875 conn(0x564385e2bf60 msgr2=0x564385e2c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/1303310875 shutdown_connections 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/1303310875 wait complete. 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 -- start start 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 Processor -- start 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 0x564385e3e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 0x564385e3bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e3efc0 0x564385e3c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564385e41b90 con 0x564385e3efc0 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564385d50f20 con 0x564385e301d0 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564385e41d10 con 0x564385e2f8b0 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.986+0000 7fd5c5ffc740 10 client.0 _ll_register_callbacks cb 0x564385d1a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c21c0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e3efc0 0x564385e3c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c21c0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e3efc0 0x564385e3c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50674/0 (socket says 172.21.15.1:50674) 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c21c0640 1 -- 172.21.15.1:0/672177703 learned_addr learned my addr 172.21.15.1:0/672177703 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c11be640 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 0x564385e3bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 0x564385e3e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c21c0640 1 -- 172.21.15.1:0/672177703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 msgr2=0x564385e3e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c21c0640 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 0x564385e3e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c21c0640 1 -- 172.21.15.1:0/672177703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 msgr2=0x564385e3bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c21c0640 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 0x564385e3bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c21c0640 1 -- 172.21.15.1:0/672177703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564385b072b0 con 0x564385e3efc0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c21c0640 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e3efc0 0x564385e3c060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4011db0 tx=0x7fd5b4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2359e0e1360ecb99 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5a57fa640 1 -- 172.21.15.1:0/672177703 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b401f450 con 0x564385e3efc0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5a57fa640 1 -- 172.21.15.1:0/672177703 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd5b401f5f0 con 0x564385e3efc0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/672177703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b8012930 con 0x564385e3efc0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5a57fa640 1 -- 172.21.15.1:0/672177703 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd5b4032050 con 0x564385e3efc0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c5ffc740 10 client.15102 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.987+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/672177703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564385e4d450 con 0x564385e3efc0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.988+0000 7fd5a57fa640 1 -- 172.21.15.1:0/672177703 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b8012930 con 0x564385e3efc0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.988+0000 7fd5a57fa640 1 -- 172.21.15.1:0/672177703 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fd5b4029c20 con 0x564385e3efc0 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.988+0000 7fd5c5ffc740 10 client.15102 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.988+0000 7fd5c5ffc740 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.988+0000 7fd5c5ffc740 20 client.15102 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.988+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564385e4fac0 0x564385e50f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.988+0000 7fd5c5ffc740 4 client.15102 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.988+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/672177703 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564385da3600 con 0x564385e4fac0 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.988+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564385e4fac0 0x564385e50f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.989+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564385e4fac0 0x564385e50f30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8626c3861d446be2 server_cookie=e11f0a40f53bb132 in_seq=0 out_seq=0 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.989+0000 7fd5a57fa640 10 client.15102 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.989+0000 7fd5a57fa640 1 -- 172.21.15.1:0/672177703 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564385da3600 con 0x564385e4fac0 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.989+0000 7fd5a57fa640 10 client.15102 handle_command_reply: tid=0 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.989+0000 7fd5c5ffc740 1 client.15102 shutdown 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/672177703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e3efc0 msgr2=0x564385e3c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e3efc0 0x564385e3c060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4011db0 tx=0x7fd5b4009680 comp rx=0 tx=0).stop 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/672177703 shutdown_connections 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564385e4fac0 0x564385e50f30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e3efc0 0x564385e3c060 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 0x564385e3bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/672177703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 0x564385e3e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/672177703 >> 172.21.15.1:0/672177703 conn(0x564385e2bf60 msgr2=0x564385e31cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5a57fa640 0 client.15102 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/672177703 shutdown_connections 2024-09-16T00:28:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/672177703 wait complete. 2024-09-16T00:28:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:13.990+0000 7fd5c5ffc740 20 client.15102 trim_cache size 0 max 16384 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.008+0000 7fd5c5ffc740 1 Processor -- start 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.008+0000 7fd5c5ffc740 1 -- start start 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 0x564385e4fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e301d0 0x564385e50860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 0x564385e4f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564385e4f270 con 0x564385e301d0 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564385e4f940 con 0x564385e3efc0 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564385e50010 con 0x564385e2f8b0 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c11be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e301d0 0x564385e50860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c11be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e301d0 0x564385e50860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50688/0 (socket says 172.21.15.1:50688) 2024-09-16T00:28:14.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c11be640 1 -- 172.21.15.1:0/2821794428 learned_addr learned my addr 172.21.15.1:0/2821794428 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c21c0640 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 0x564385e4f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 0x564385e4fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c11be640 1 -- 172.21.15.1:0/2821794428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 msgr2=0x564385e4fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c11be640 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 0x564385e4fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c11be640 1 -- 172.21.15.1:0/2821794428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 msgr2=0x564385e4f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c11be640 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 0x564385e4f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c11be640 1 -- 172.21.15.1:0/2821794428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564385e2ffc0 con 0x564385e301d0 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 0x564385e4fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5c11be640 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e301d0 0x564385e50860 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8000c00 tx=0x7fd5b8009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb54fde6a49a88b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.009+0000 7fd5a77fe640 1 -- 172.21.15.1:0/2821794428 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8029020 con 0x564385e301d0 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5a77fe640 1 -- 172.21.15.1:0/2821794428 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b8011e50 con 0x564385e301d0 2024-09-16T00:28:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5a77fe640 1 -- 172.21.15.1:0/2821794428 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8012450 con 0x564385e301d0 2024-09-16T00:28:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/2821794428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e301d0 msgr2=0x564385e50860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e301d0 0x564385e50860 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8000c00 tx=0x7fd5b8009570 comp rx=0 tx=0).stop 2024-09-16T00:28:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/2821794428 shutdown_connections 2024-09-16T00:28:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e3efc0 0x564385e4f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e301d0 0x564385e50860 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/2821794428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e2f8b0 0x564385e4fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/2821794428 >> 172.21.15.1:0/2821794428 conn(0x564385e2bf60 msgr2=0x564385e2c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.010+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/2821794428 shutdown_connections 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.011+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/2821794428 wait complete. 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.011+0000 7fd5c5ffc740 1 -- start start 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.011+0000 7fd5c5ffc740 1 Processor -- start 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.011+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e52290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 0x564385e527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c5ffc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e3efc0 0x564385eb81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564385e50db0 con 0x564385e2f8b0 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564385e506e0 con 0x564385e301d0 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c5ffc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564385e51480 con 0x564385e3efc0 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c5ffc740 10 client.0 _ll_register_callbacks cb 0x7fd5b4009240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c19bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e52290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c19bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e52290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50694/0 (socket says 172.21.15.1:50694) 2024-09-16T00:28:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c19bf640 1 -- 172.21.15.1:0/420404629 learned_addr learned my addr 172.21.15.1:0/420404629 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c11be640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 0x564385e527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c21c0640 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e3efc0 0x564385eb81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c19bf640 1 -- 172.21.15.1:0/420404629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e3efc0 msgr2=0x564385eb81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e3efc0 0x564385eb81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c19bf640 1 -- 172.21.15.1:0/420404629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 msgr2=0x564385e527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 0x564385e527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c19bf640 1 -- 172.21.15.1:0/420404629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564385eb8700 con 0x564385e2f8b0 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5c19bf640 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e52290 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00b8ca0 tx=0x7fd5b008b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d64844c4ec2b999 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.012+0000 7fd5a57fa640 1 -- 172.21.15.1:0/420404629 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0049040 con 0x564385e2f8b0 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5a57fa640 1 -- 172.21.15.1:0/420404629 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd5b003bd40 con 0x564385e2f8b0 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/420404629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b8011e50 con 0x564385e2f8b0 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5a57fa640 1 -- 172.21.15.1:0/420404629 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd5b00bc9d0 con 0x564385e2f8b0 2024-09-16T00:28:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5c5ffc740 10 client.15108 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/420404629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564385eb97c0 con 0x564385e2f8b0 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5a57fa640 1 -- 172.21.15.1:0/420404629 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b8011e50 con 0x564385e2f8b0 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5a57fa640 1 -- 172.21.15.1:0/420404629 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fd5b00b8940 con 0x564385e2f8b0 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5c5ffc740 10 client.15108 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5c5ffc740 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5c5ffc740 20 client.15108 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564385ebe3d0 0x564385ebd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5c5ffc740 4 client.15108 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.013+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/420404629 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "status"}) -- 0x564385d9db10 con 0x564385ebe3d0 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.014+0000 7fd5c11be640 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564385ebe3d0 0x564385ebd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.014+0000 7fd5a57fa640 10 client.15108 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.014+0000 7fd5c11be640 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564385ebe3d0 0x564385ebd0e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e876fd72163025b server_cookie=305b0f6a9336a92f in_seq=0 out_seq=0 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.014+0000 7fd5a57fa640 1 -- 172.21.15.1:0/420404629 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564385d9db10 con 0x564385ebe3d0 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.014+0000 7fd5a57fa640 10 client.15108 handle_command_reply: tid=0 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.014+0000 7fd5c5ffc740 1 client.15108 shutdown 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.014+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/420404629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 msgr2=0x564385e52290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.014+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e52290 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00b8ca0 tx=0x7fd5b008b010 comp rx=0 tx=0).stop 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/420404629 shutdown_connections 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564385ebe3d0 0x564385ebd0e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564385e3efc0 0x564385eb81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564385e301d0 0x564385e527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5a57fa640 0 client.15108 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5c5ffc740 1 --2- 172.21.15.1:0/420404629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564385e2f8b0 0x564385e52290 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/420404629 >> 172.21.15.1:0/420404629 conn(0x564385e2bf60 msgr2=0x564385e3e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/420404629 shutdown_connections 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5c5ffc740 1 -- 172.21.15.1:0/420404629 wait complete. 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.015+0000 7fd5c5ffc740 20 client.15108 trim_cache size 0 max 16384 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24506, 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:28:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:28:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:28:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 114.74341153500001, 2024-09-16T00:28:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 24, 2024-09-16T00:28:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-09-16T00:28:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 89, 2024-09-16T00:28:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 126.087766397 2024-09-16T00:28:14.021 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 -- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5a80a32d0 msgr2=0x7fd5a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5a80a32d0 0x7fd5a80a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40092f0 tx=0x7fd5b40099c0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 -- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc100a80 msgr2=0x7fd5bc198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc100a80 0x7fd5bc198440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009230 tx=0x7fd5b0004270 comp rx=0 tx=0).stop 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 -- 172.21.15.1:0/1445710006 shutdown_connections 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5bc106a80 0x7fd5bc19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd5a80a32d0 0x7fd5a80a5790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc101420 0x7fd5bc198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 --2- 172.21.15.1:0/1445710006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5bc100a80 0x7fd5bc198440 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 -- 172.21.15.1:0/1445710006 >> 172.21.15.1:0/1445710006 conn(0x7fd5bc0fc7a0 msgr2=0x7fd5bc104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:14.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 -- 172.21.15.1:0/1445710006 shutdown_connections 2024-09-16T00:28:14.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:14.016+0000 7fd5c3c23640 1 -- 172.21.15.1:0/1445710006 wait complete. 2024-09-16T00:28:14.843 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T00:28:14.843 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:28:14.853 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-16T00:28:14.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T00:28:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:15.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:15 smithi001 ceph-mon[30934]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:15.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.586+0000 7f1c68b81640 1 -- 172.21.15.1:0/159682220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c641037c0 msgr2=0x7f1c6410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.586+0000 7f1c68b81640 1 --2- 172.21.15.1:0/159682220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c641037c0 0x7f1c6410a050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009ca0 tx=0x7f1c5c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.587+0000 7f1c68b81640 1 -- 172.21.15.1:0/159682220 shutdown_connections 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.587+0000 7f1c68b81640 1 --2- 172.21.15.1:0/159682220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c641037c0 0x7f1c6410a050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.587+0000 7f1c68b81640 1 --2- 172.21.15.1:0/159682220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c64102e00 0x7f1c64103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.587+0000 7f1c68b81640 1 --2- 172.21.15.1:0/159682220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c64101c00 0x7f1c64102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.587+0000 7f1c68b81640 1 -- 172.21.15.1:0/159682220 >> 172.21.15.1:0/159682220 conn(0x7f1c640fd390 msgr2=0x7f1c640ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.587+0000 7f1c68b81640 1 -- 172.21.15.1:0/159682220 shutdown_connections 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.587+0000 7f1c68b81640 1 -- 172.21.15.1:0/159682220 wait complete. 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.587+0000 7f1c68b81640 1 Processor -- start 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.587+0000 7f1c68b81640 1 -- start start 2024-09-16T00:28:15.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c68b81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c64101c00 0x7f1c64075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c68b81640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c64102e00 0x7f1c640745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c68b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c641037c0 0x7f1c64074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c68b81640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6410d880 con 0x7f1c64101c00 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c68b81640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6410d700 con 0x7f1c641037c0 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c68b81640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6410da00 con 0x7f1c64102e00 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c64101c00 0x7f1c64075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c64101c00 0x7f1c64075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50726/0 (socket says 172.21.15.1:50726) 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c6259c640 1 -- 172.21.15.1:0/1532108612 learned_addr learned my addr 172.21.15.1:0/1532108612 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c62d9d640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c641037c0 0x7f1c64074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c61d9b640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c64102e00 0x7f1c640745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c6259c640 1 -- 172.21.15.1:0/1532108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c64102e00 msgr2=0x7f1c640745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c6259c640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c64102e00 0x7f1c640745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c6259c640 1 -- 172.21.15.1:0/1532108612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c641037c0 msgr2=0x7f1c64074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c6259c640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c641037c0 0x7f1c64074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c6259c640 1 -- 172.21.15.1:0/1532108612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c640753e0 con 0x7f1c64101c00 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c61d9b640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c64102e00 0x7f1c640745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c6259c640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c64101c00 0x7f1c64075f90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c537fe640 1 -- 172.21.15.1:0/1532108612 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54017020 con 0x7f1c64101c00 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c537fe640 1 -- 172.21.15.1:0/1532108612 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c54009c30 con 0x7f1c64101c00 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c641a1fd0 con 0x7f1c64101c00 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.588+0000 7f1c537fe640 1 -- 172.21.15.1:0/1532108612 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400f740 con 0x7f1c64101c00 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.589+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c641a2480 con 0x7f1c64101c00 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.589+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c64108f80 con 0x7f1c64101c00 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.591+0000 7f1c537fe640 1 -- 172.21.15.1:0/1532108612 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f1c5401a020 con 0x7f1c64101c00 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.591+0000 7f1c537fe640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c4c0a3350 0x7f1c4c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.592+0000 7f1c61d9b640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c4c0a3350 0x7f1c4c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:15.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.592+0000 7f1c537fe640 1 -- 172.21.15.1:0/1532108612 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1c540cfbc0 con 0x7f1c64101c00 2024-09-16T00:28:15.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.592+0000 7f1c61d9b640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c4c0a3350 0x7f1c4c0a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005fd0 tx=0x7f1c580094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:15.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.595+0000 7f1c537fe640 1 -- 172.21.15.1:0/1532108612 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c54021070 con 0x7f1c64101c00 2024-09-16T00:28:15.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:15 smithi153 ceph-mon[35538]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.789+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1c64062630 con 0x7f1c4c0a3350 2024-09-16T00:28:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.790+0000 7f1c537fe640 1 -- 172.21.15.1:0/1532108612 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1c64062630 con 0x7f1c4c0a3350 2024-09-16T00:28:15.790 INFO:teuthology.orchestra.run.smithi001.stdout:Backend: cephadm 2024-09-16T00:28:15.790 INFO:teuthology.orchestra.run.smithi001.stdout:Available: Yes 2024-09-16T00:28:15.791 INFO:teuthology.orchestra.run.smithi001.stdout:Paused: No 2024-09-16T00:28:15.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c4c0a3350 msgr2=0x7f1c4c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:15.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c4c0a3350 0x7f1c4c0a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005fd0 tx=0x7f1c580094d0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c64101c00 msgr2=0x7f1c64075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:15.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c64101c00 0x7f1c64075f90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400de10 comp rx=0 tx=0).stop 2024-09-16T00:28:15.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 shutdown_connections 2024-09-16T00:28:15.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c641037c0 0x7f1c64074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c4c0a3350 0x7f1c4c0a5810 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c64102e00 0x7f1c640745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 --2- 172.21.15.1:0/1532108612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c64101c00 0x7f1c64075f90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.792+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 >> 172.21.15.1:0/1532108612 conn(0x7f1c640fd390 msgr2=0x7f1c64106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:15.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.793+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 shutdown_connections 2024-09-16T00:28:15.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:15.793+0000 7f1c68b81640 1 -- 172.21.15.1:0/1532108612 wait complete. 2024-09-16T00:28:15.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:15 smithi019 ceph-mon[35186]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:16.647 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T00:28:16.692 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:16 smithi019 ceph-mon[35186]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:16.692 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:16 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:16.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:16 smithi001 ceph-mon[30934]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:16.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:16 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:16.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:16 smithi153 ceph-mon[35538]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:16.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:16 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:17.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:17 smithi001 ceph-mon[30934]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:28:17.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:17 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:17.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:17 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:17.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:17 smithi153 ceph-mon[35538]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:28:17.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:17 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:17.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:17 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:17.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:17 smithi019 ceph-mon[35186]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:28:17.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:17 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:17.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:17 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:18.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.074+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/1773548980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc100ad0 msgr2=0x7fd3dc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.074+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/1773548980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc100ad0 0x7fd3dc100ed0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009980 tx=0x7fd3cc02c620 comp rx=0 tx=0).stop 2024-09-16T00:28:18.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.074+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/1773548980 shutdown_connections 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.074+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/1773548980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc102690 0x7fd3dc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.074+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/1773548980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3dc101cd0 0x7fd3dc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.074+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/1773548980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc100ad0 0x7fd3dc100ed0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.074+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/1773548980 >> 172.21.15.1:0/1773548980 conn(0x7fd3dc0fc280 msgr2=0x7fd3dc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.074+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/1773548980 shutdown_connections 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.074+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/1773548980 wait complete. 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3e1e4a640 1 Processor -- start 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3e1e4a640 1 -- start start 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3e1e4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc100ad0 0x7fd3dc195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3e1e4a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc101cd0 0x7fd3dc196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3e1e4a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3dc102690 0x7fd3dc19d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3e1e4a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc10b660 con 0x7fd3dc101cd0 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3e1e4a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc10b4e0 con 0x7fd3dc100ad0 2024-09-16T00:28:18.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3e1e4a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc10b7e0 con 0x7fd3dc102690 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc101cd0 0x7fd3dc196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc101cd0 0x7fd3dc196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50746/0 (socket says 172.21.15.1:50746) 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3daffd640 1 -- 172.21.15.1:0/2517570857 learned_addr learned my addr 172.21.15.1:0/2517570857 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.075+0000 7fd3dbfff640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3dc102690 0x7fd3dc19d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3db7fe640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc100ad0 0x7fd3dc195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3daffd640 1 -- 172.21.15.1:0/2517570857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3dc102690 msgr2=0x7fd3dc19d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3daffd640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3dc102690 0x7fd3dc19d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3daffd640 1 -- 172.21.15.1:0/2517570857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc100ad0 msgr2=0x7fd3dc195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3daffd640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc100ad0 0x7fd3dc195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3daffd640 1 -- 172.21.15.1:0/2517570857 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc19d930 con 0x7fd3dc101cd0 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3daffd640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc101cd0 0x7fd3dc196250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d770 tx=0x7fd3d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3d8ff9640 1 -- 172.21.15.1:0/2517570857 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0017070 con 0x7fd3dc101cd0 2024-09-16T00:28:18.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3d8ff9640 1 -- 172.21.15.1:0/2517570857 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d0005500 con 0x7fd3dc101cd0 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc19dc20 con 0x7fd3dc101cd0 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3d8ff9640 1 -- 172.21.15.1:0/2517570857 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0004e00 con 0x7fd3dc101cd0 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.076+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc19e130 con 0x7fd3dc101cd0 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.077+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3dc107f20 con 0x7fd3dc101cd0 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.079+0000 7fd3d8ff9640 1 -- 172.21.15.1:0/2517570857 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd3d0004fa0 con 0x7fd3dc101cd0 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.080+0000 7fd3d8ff9640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3c40a3350 0x7fd3c40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.080+0000 7fd3d8ff9640 1 -- 172.21.15.1:0/2517570857 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd3d00cf820 con 0x7fd3dc101cd0 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.080+0000 7fd3db7fe640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3c40a3350 0x7fd3c40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.081+0000 7fd3db7fe640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3c40a3350 0x7fd3c40a5810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc005ab0 tx=0x7fd3cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:18.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.086+0000 7fd3d8ff9640 1 -- 172.21.15.1:0/2517570857 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3d009b930 con 0x7fd3dc101cd0 2024-09-16T00:28:18.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.283+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd3dc062630 con 0x7fd3c40a3350 2024-09-16T00:28:18.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.296+0000 7fd3d8ff9640 1 -- 172.21.15.1:0/2517570857 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7fd3dc062630 con 0x7fd3c40a3350 2024-09-16T00:28:18.297 INFO:teuthology.orchestra.run.smithi001.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:28:18.297 INFO:teuthology.orchestra.run.smithi001.stdout:mds.a smithi001 running (2m) 62s ago 2m 13.6M - 19.1.1-326-g8921c8ef f1455bd0219c ca4a96a20d49 2024-09-16T00:28:18.297 INFO:teuthology.orchestra.run.smithi001.stdout:mds.b smithi019 running (2m) 0s ago 2m 29.1M - 19.1.1-326-g8921c8ef f1455bd0219c 09e23b1983f3 2024-09-16T00:28:18.297 INFO:teuthology.orchestra.run.smithi001.stdout:mds.c smithi153 running (2m) 0s ago 2m 19.7M - 19.1.1-326-g8921c8ef f1455bd0219c 3e37f8eed2c9 2024-09-16T00:28:18.297 INFO:teuthology.orchestra.run.smithi001.stdout:mds.d smithi001 running (2m) 62s ago 2m 14.4M - 19.1.1-326-g8921c8ef f1455bd0219c d05f3acf2a8b 2024-09-16T00:28:18.297 INFO:teuthology.orchestra.run.smithi001.stdout:mds.e smithi019 running (2m) 0s ago 2m 18.4M - 19.1.1-326-g8921c8ef f1455bd0219c c8d7c23eff53 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mds.f smithi153 running (2m) 0s ago 2m 19.9M - 19.1.1-326-g8921c8ef f1455bd0219c 6a5b145e9c1f 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mds.g smithi001 running (2m) 62s ago 2m 14.3M - 19.1.1-326-g8921c8ef f1455bd0219c 20f03c28ee4e 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mds.h smithi019 running (2m) 0s ago 2m 18.5M - 19.1.1-326-g8921c8ef f1455bd0219c ce175bee7926 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mds.i smithi153 running (2m) 0s ago 2m 18.9M - 19.1.1-326-g8921c8ef f1455bd0219c f0b544629d6c 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mds.j smithi001 running (2m) 62s ago 2m 13.5M - 19.1.1-326-g8921c8ef f1455bd0219c 5c3cdfa00a05 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mds.k smithi019 running (2m) 0s ago 2m 14.8M - 19.1.1-326-g8921c8ef f1455bd0219c c29345750f6f 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mds.l smithi153 running (2m) 0s ago 2m 14.0M - 19.1.1-326-g8921c8ef f1455bd0219c b0883f0ab622 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.x smithi001 *:9283,8765 running (9m) 62s ago 9m 509M - 19.1.1-326-g8921c8ef f1455bd0219c 7c193014c932 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.y smithi019 *:8443,8765 running (7m) 0s ago 7m 444M - 19.1.1-326-g8921c8ef f1455bd0219c cf88e9079cd1 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.z smithi153 *:8443,8765 running (7m) 0s ago 7m 442M - 19.1.1-326-g8921c8ef f1455bd0219c a677ab130ae5 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a smithi001 running (9m) 62s ago 9m 51.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 196e28c9c2a5 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b smithi019 running (8m) 0s ago 8m 44.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c bffa471eec8c 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c smithi153 running (8m) 0s ago 8m 41.1M 2048M 19.1.1-326-g8921c8ef f1455bd0219c bad8b46e65d3 2024-09-16T00:28:18.298 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 smithi001 running (7m) 62s ago 7m 47.5M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 4ee55519adc1 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 smithi019 running (7m) 0s ago 7m 50.2M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 049dbd9fb3df 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 smithi153 running (6m) 0s ago 6m 57.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 4aa8e5bda815 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 smithi001 running (6m) 62s ago 6m 44.8M 4273M 19.1.1-326-g8921c8ef f1455bd0219c e24288000a4a 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 smithi019 running (5m) 0s ago 5m 44.1M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 396246fa8bd7 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 smithi153 running (5m) 0s ago 5m 57.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c e9a1a8ac2eff 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 smithi001 running (5m) 62s ago 5m 49.7M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 48718c9375da 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 smithi019 running (4m) 0s ago 4m 54.2M 4273M 19.1.1-326-g8921c8ef f1455bd0219c cf3867af6ca6 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 smithi153 running (4m) 0s ago 4m 52.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 0ee8f6047d79 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 smithi001 running (3m) 62s ago 3m 43.0M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 2d6304517744 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 smithi019 running (3m) 0s ago 3m 46.9M 4273M 19.1.1-326-g8921c8ef f1455bd0219c c95447b66108 2024-09-16T00:28:18.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 smithi153 running (3m) 0s ago 3m 50.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 45f672136646 2024-09-16T00:28:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.298+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3c40a3350 msgr2=0x7fd3c40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.298+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3c40a3350 0x7fd3c40a5810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc005ab0 tx=0x7fd3cc000bc0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.298+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc101cd0 msgr2=0x7fd3dc196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.298+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc101cd0 0x7fd3dc196250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d770 tx=0x7fd3d000dc30 comp rx=0 tx=0).stop 2024-09-16T00:28:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.299+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 shutdown_connections 2024-09-16T00:28:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.299+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3c40a3350 0x7fd3c40a5810 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.299+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3dc102690 0x7fd3dc19d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.299+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3dc101cd0 0x7fd3dc196250 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.299+0000 7fd3e1e4a640 1 --2- 172.21.15.1:0/2517570857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc100ad0 0x7fd3dc195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.299+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 >> 172.21.15.1:0/2517570857 conn(0x7fd3dc0fc280 msgr2=0x7fd3dc1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.299+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 shutdown_connections 2024-09-16T00:28:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:18.299+0000 7fd3e1e4a640 1 -- 172.21.15.1:0/2517570857 wait complete. 2024-09-16T00:28:18.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:18.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:18 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:18.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:18 smithi019 ceph-mon[35186]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:18.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:18.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:18 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:18.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:18 smithi001 ceph-mon[30934]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:18.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:18.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:18 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:18.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:18 smithi153 ceph-mon[35538]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:19.229 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T00:28:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:19.922 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:19 smithi001 ceph-mon[30934]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:28:19.922 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:19.922 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:19.922 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:19.923 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:19.923 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:19.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:19 smithi153 ceph-mon[35538]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:28:19.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:19.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:20.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.005+0000 7fe9f13f9640 1 -- 172.21.15.1:0/278396729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec104fd0 msgr2=0x7fe9ec1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.005+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/278396729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec104fd0 0x7fe9ec1073c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009900 tx=0x7fe9d402a410 comp rx=0 tx=0).stop 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.005+0000 7fe9f13f9640 1 -- 172.21.15.1:0/278396729 shutdown_connections 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.005+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/278396729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9ec107900 0x7fe9ec109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.005+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/278396729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec104fd0 0x7fe9ec1073c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.005+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/278396729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ec1026a0 0x7fe9ec104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.005+0000 7fe9f13f9640 1 -- 172.21.15.1:0/278396729 >> 172.21.15.1:0/278396729 conn(0x7fe9ec0fc2a0 msgr2=0x7fe9ec0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 -- 172.21.15.1:0/278396729 shutdown_connections 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 -- 172.21.15.1:0/278396729 wait complete. 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 Processor -- start 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 -- start start 2024-09-16T00:28:20.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec1026a0 0x7fe9ec197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9ec104fd0 0x7fe9ec1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ec107900 0x7fe9ec19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec10d8a0 con 0x7fe9ec1026a0 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec10d720 con 0x7fe9ec107900 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.006+0000 7fe9f13f9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec10da20 con 0x7fe9ec104fd0 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec1026a0 0x7fe9ec197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec1026a0 0x7fe9ec197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50752/0 (socket says 172.21.15.1:50752) 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eaffd640 1 -- 172.21.15.1:0/3297705612 learned_addr learned my addr 172.21.15.1:0/3297705612 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eb7fe640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ec107900 0x7fe9ec19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9e27fc640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9ec104fd0 0x7fe9ec1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:20.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eaffd640 1 -- 172.21.15.1:0/3297705612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9ec104fd0 msgr2=0x7fe9ec1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eaffd640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9ec104fd0 0x7fe9ec1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eaffd640 1 -- 172.21.15.1:0/3297705612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ec107900 msgr2=0x7fe9ec19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eaffd640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ec107900 0x7fe9ec19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eaffd640 1 -- 172.21.15.1:0/3297705612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec19fb90 con 0x7fe9ec1026a0 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eb7fe640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ec107900 0x7fe9ec19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9eaffd640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec1026a0 0x7fe9ec197f60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00d950 tx=0x7fe9dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9e8ff9640 1 -- 172.21.15.1:0/3297705612 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc017030 con 0x7fe9ec1026a0 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec19fe80 con 0x7fe9ec1026a0 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9e8ff9640 1 -- 172.21.15.1:0/3297705612 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9dc004020 con 0x7fe9ec1026a0 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9e8ff9640 1 -- 172.21.15.1:0/3297705612 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0047a0 con 0x7fe9ec1026a0 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.007+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec1a02d0 con 0x7fe9ec1026a0 2024-09-16T00:28:20.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.008+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec0fdd90 con 0x7fe9ec1026a0 2024-09-16T00:28:20.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.010+0000 7fe9e8ff9640 1 -- 172.21.15.1:0/3297705612 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe9dc0041d0 con 0x7fe9ec1026a0 2024-09-16T00:28:20.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.010+0000 7fe9e8ff9640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9d80a3300 0x7fe9d80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:20.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.010+0000 7fe9e27fc640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9d80a3300 0x7fe9d80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:20.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.010+0000 7fe9e8ff9640 1 -- 172.21.15.1:0/3297705612 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe9dc0cf9f0 con 0x7fe9ec1026a0 2024-09-16T00:28:20.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.011+0000 7fe9e27fc640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9d80a3300 0x7fe9d80a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4006010 tx=0x7fe9d4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:20.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.014+0000 7fe9e8ff9640 1 -- 172.21.15.1:0/3297705612 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9dc09c150 con 0x7fe9ec1026a0 2024-09-16T00:28:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:19 smithi019 ceph-mon[35186]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:28:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:28:20.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.203+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe9ec062630 con 0x7fe9d80a3300 2024-09-16T00:28:20.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.209+0000 7fe9e8ff9640 1 -- 172.21.15.1:0/3297705612 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe9ec062630 con 0x7fe9d80a3300 2024-09-16T00:28:20.209 INFO:teuthology.orchestra.run.smithi001.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T00:28:20.210 INFO:teuthology.orchestra.run.smithi001.stdout:mds.all 12/12 2s ago 2m smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-16T00:28:20.210 INFO:teuthology.orchestra.run.smithi001.stdout:mgr 3/3 2s ago 8m smithi001=x;smithi019=y;smithi153=z;count:3 2024-09-16T00:28:20.210 INFO:teuthology.orchestra.run.smithi001.stdout:mon 3/3 2s ago 8m smithi001:172.21.15.1=a;smithi019:172.21.15.19=b;smithi153:172.21.15.153=c;count:3 2024-09-16T00:28:20.210 INFO:teuthology.orchestra.run.smithi001.stdout:osd 12 2s ago - 2024-09-16T00:28:20.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9d80a3300 msgr2=0x7fe9d80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:20.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9d80a3300 0x7fe9d80a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4006010 tx=0x7fe9d4000bc0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec1026a0 msgr2=0x7fe9ec197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:20.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec1026a0 0x7fe9ec197f60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00d950 tx=0x7fe9dc00de10 comp rx=0 tx=0).stop 2024-09-16T00:28:20.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 shutdown_connections 2024-09-16T00:28:20.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ec107900 0x7fe9ec19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9d80a3300 0x7fe9d80a57c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9ec104fd0 0x7fe9ec1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 --2- 172.21.15.1:0/3297705612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9ec1026a0 0x7fe9ec197f60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:20.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 >> 172.21.15.1:0/3297705612 conn(0x7fe9ec0fc2a0 msgr2=0x7fe9ec103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:20.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 shutdown_connections 2024-09-16T00:28:20.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:20.212+0000 7fe9f13f9640 1 -- 172.21.15.1:0/3297705612 wait complete. 2024-09-16T00:28:20.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:20 smithi001 ceph-mon[30934]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:20.933 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T00:28:20.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:20 smithi153 ceph-mon[35538]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:21.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:20 smithi019 ceph-mon[35186]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:21.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:21 smithi001 ceph-mon[30934]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.715+0000 7fefad22c640 1 -- 172.21.15.1:0/4230567004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa8101d10 msgr2=0x7fefa8102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.715+0000 7fefad22c640 1 --2- 172.21.15.1:0/4230567004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa8101d10 0x7fefa8102190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fef90009900 tx=0x7fef9002a410 comp rx=0 tx=0).stop 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.715+0000 7fefad22c640 1 -- 172.21.15.1:0/4230567004 shutdown_connections 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.715+0000 7fefad22c640 1 --2- 172.21.15.1:0/4230567004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa81026d0 0x7fefa8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.715+0000 7fefad22c640 1 --2- 172.21.15.1:0/4230567004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa8101d10 0x7fefa8102190 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.715+0000 7fefad22c640 1 --2- 172.21.15.1:0/4230567004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa8100b10 0x7fefa8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.715+0000 7fefad22c640 1 -- 172.21.15.1:0/4230567004 >> 172.21.15.1:0/4230567004 conn(0x7fefa80fc2a0 msgr2=0x7fefa80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.715+0000 7fefad22c640 1 -- 172.21.15.1:0/4230567004 shutdown_connections 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.715+0000 7fefad22c640 1 -- 172.21.15.1:0/4230567004 wait complete. 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.716+0000 7fefad22c640 1 Processor -- start 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.716+0000 7fefad22c640 1 -- start start 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefad22c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa8100b10 0x7fefa8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefad22c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa8101d10 0x7fefa8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefad22c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa81026d0 0x7fefa819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefad22c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fefa810c770 con 0x7fefa81026d0 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefad22c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fefa810c5f0 con 0x7fefa8101d10 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefad22c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fefa810c8f0 con 0x7fefa8100b10 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa81026d0 0x7fefa819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa81026d0 0x7fefa819f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48048/0 (socket says 172.21.15.1:48048) 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa759e640 1 -- 172.21.15.1:0/76045126 learned_addr learned my addr 172.21.15.1:0/76045126 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa6d9d640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa8100b10 0x7fefa8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa659c640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa8101d10 0x7fefa8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa759e640 1 -- 172.21.15.1:0/76045126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa8100b10 msgr2=0x7fefa8197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa759e640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa8100b10 0x7fefa8197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa759e640 1 -- 172.21.15.1:0/76045126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa8101d10 msgr2=0x7fefa8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa759e640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa8101d10 0x7fefa8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa759e640 1 -- 172.21.15.1:0/76045126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa819fb70 con 0x7fefa81026d0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.717+0000 7fefa759e640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa81026d0 0x7fefa819f500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fefa000eaa0 tx=0x7fefa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.718+0000 7fef9ffff640 1 -- 172.21.15.1:0/76045126 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa000cdb0 con 0x7fefa81026d0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.718+0000 7fef9ffff640 1 -- 172.21.15.1:0/76045126 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa0005020 con 0x7fefa81026d0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.718+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa819fe60 con 0x7fefa81026d0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.718+0000 7fef9ffff640 1 -- 172.21.15.1:0/76045126 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa00116b0 con 0x7fefa81026d0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.718+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a02c0 con 0x7fefa81026d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.719+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa81a0550 con 0x7fefa81026d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.720+0000 7fef9ffff640 1 -- 172.21.15.1:0/76045126 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fefa0005560 con 0x7fefa81026d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.721+0000 7fef9ffff640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fef940a3300 0x7fef940a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.721+0000 7fefa6d9d640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fef940a3300 0x7fef940a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.721+0000 7fef9ffff640 1 -- 172.21.15.1:0/76045126 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fefa00d03b0 con 0x7fefa81026d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.721+0000 7fefa6d9d640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fef940a3300 0x7fef940a57c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fefa8101b70 tx=0x7fef9800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.724+0000 7fef9ffff640 1 -- 172.21.15.1:0/76045126 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefa0016030 con 0x7fefa81026d0 2024-09-16T00:28:21.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.905+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fefa81a0720 con 0x7fef940a3300 2024-09-16T00:28:21.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.907+0000 7fef9ffff640 1 -- 172.21.15.1:0/76045126 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fefa81a0720 con 0x7fef940a3300 2024-09-16T00:28:21.907 INFO:teuthology.orchestra.run.smithi001.stdout:HOST ADDR LABELS STATUS 2024-09-16T00:28:21.907 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 172.21.15.1 2024-09-16T00:28:21.907 INFO:teuthology.orchestra.run.smithi001.stdout:smithi019 172.21.15.19 2024-09-16T00:28:21.907 INFO:teuthology.orchestra.run.smithi001.stdout:smithi153 172.21.15.153 2024-09-16T00:28:21.908 INFO:teuthology.orchestra.run.smithi001.stdout:3 hosts in cluster 2024-09-16T00:28:21.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fef940a3300 msgr2=0x7fef940a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fef940a3300 0x7fef940a57c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fefa8101b70 tx=0x7fef9800b390 comp rx=0 tx=0).stop 2024-09-16T00:28:21.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa81026d0 msgr2=0x7fefa819f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa81026d0 0x7fefa819f500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fefa000eaa0 tx=0x7fefa000ef60 comp rx=0 tx=0).stop 2024-09-16T00:28:21.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 shutdown_connections 2024-09-16T00:28:21.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefa81026d0 0x7fefa819f500 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa8101d10 0x7fefa8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fef940a3300 0x7fef940a57c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 --2- 172.21.15.1:0/76045126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa8100b10 0x7fefa8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.909+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 >> 172.21.15.1:0/76045126 conn(0x7fefa80fc2a0 msgr2=0x7fefa80fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:21.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.910+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 shutdown_connections 2024-09-16T00:28:21.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:21.910+0000 7fefad22c640 1 -- 172.21.15.1:0/76045126 wait complete. 2024-09-16T00:28:21.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:21 smithi153 ceph-mon[35538]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:21 smithi019 ceph-mon[35186]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:22.653 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T00:28:22.912 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:22.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:22 smithi001 ceph-mon[30934]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:22.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:22 smithi153 ceph-mon[35538]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:23.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:22 smithi019 ceph-mon[35186]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:23.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 -- 172.21.15.1:0/4160676211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 msgr2=0x7fe238109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4160676211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 0x7fe238109d10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe230009ca0 tx=0x7fe23002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 -- 172.21.15.1:0/4160676211 shutdown_connections 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4160676211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 0x7fe238109d10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4160676211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe238104f90 0x7fe238107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4160676211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe238102660 0x7fe238104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 -- 172.21.15.1:0/4160676211 >> 172.21.15.1:0/4160676211 conn(0x7fe2380fc280 msgr2=0x7fe2380fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 -- 172.21.15.1:0/4160676211 shutdown_connections 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 -- 172.21.15.1:0/4160676211 wait complete. 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.433+0000 7fe23efe1640 1 Processor -- start 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23efe1640 1 -- start start 2024-09-16T00:28:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23efe1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe238102660 0x7fe238197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23efe1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe238104f90 0x7fe238198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23efe1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 0x7fe23819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23efe1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe23810d880 con 0x7fe2381078c0 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23efe1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe23810d700 con 0x7fe238104f90 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23efe1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe23810da00 con 0x7fe238102660 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23d57e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 0x7fe23819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23cd7d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe238102660 0x7fe238197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23cd7d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe238102660 0x7fe238197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:54694/0 (socket says 172.21.15.1:54694) 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23d57e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 0x7fe23819f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48056/0 (socket says 172.21.15.1:48056) 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23cd7d640 1 -- 172.21.15.1:0/4164923767 learned_addr learned my addr 172.21.15.1:0/4164923767 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe237fff640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe238104f90 0x7fe238198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23d57e640 1 -- 172.21.15.1:0/4164923767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe238102660 msgr2=0x7fe238197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23d57e640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe238102660 0x7fe238197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23d57e640 1 -- 172.21.15.1:0/4164923767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe238104f90 msgr2=0x7fe238198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23d57e640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe238104f90 0x7fe238198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.434+0000 7fe23d57e640 1 -- 172.21.15.1:0/4164923767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23819fb10 con 0x7fe2381078c0 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.435+0000 7fe23d57e640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 0x7fe23819f4a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe230031ae0 tx=0x7fe230031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.435+0000 7fe235ffb640 1 -- 172.21.15.1:0/4164923767 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230003c30 con 0x7fe2381078c0 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.435+0000 7fe235ffb640 1 -- 172.21.15.1:0/4164923767 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe230003dd0 con 0x7fe2381078c0 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.435+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23819fda0 con 0x7fe2381078c0 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.435+0000 7fe235ffb640 1 -- 172.21.15.1:0/4164923767 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23002fd40 con 0x7fe2381078c0 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.435+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2381a0250 con 0x7fe2381078c0 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.436+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2380fdd70 con 0x7fe2381078c0 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.437+0000 7fe235ffb640 1 -- 172.21.15.1:0/4164923767 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe230004050 con 0x7fe2381078c0 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.438+0000 7fe235ffb640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe2200a3300 0x7fe2200a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.438+0000 7fe23cd7d640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe2200a3300 0x7fe2200a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.438+0000 7fe235ffb640 1 -- 172.21.15.1:0/4164923767 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe2300f0fe0 con 0x7fe2381078c0 2024-09-16T00:28:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.438+0000 7fe23cd7d640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe2200a3300 0x7fe2200a57c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe22800bca0 tx=0x7fe22800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.442+0000 7fe235ffb640 1 -- 172.21.15.1:0/4164923767 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe2300bd8b0 con 0x7fe2381078c0 2024-09-16T00:28:23.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.627+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe238062630 con 0x7fe2200a3300 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.629+0000 7fe235ffb640 1 -- 172.21.15.1:0/4164923767 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe238062630 con 0x7fe2200a3300 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi001.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513000D480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi001.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi001.stdout:smithi153 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300MV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe2200a3300 msgr2=0x7fe2200a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe2200a3300 0x7fe2200a57c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe22800bca0 tx=0x7fe22800b410 comp rx=0 tx=0).stop 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 msgr2=0x7fe23819f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 0x7fe23819f4a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe230031ae0 tx=0x7fe230031ee0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 shutdown_connections 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2381078c0 0x7fe23819f4a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe238104f90 0x7fe238198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe2200a3300 0x7fe2200a57c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 --2- 172.21.15.1:0/4164923767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe238102660 0x7fe238197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 >> 172.21.15.1:0/4164923767 conn(0x7fe2380fc280 msgr2=0x7fe238102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 shutdown_connections 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:23.631+0000 7fe23efe1640 1 -- 172.21.15.1:0/4164923767 wait complete. 2024-09-16T00:28:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:23 smithi001 ceph-mon[30934]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:28:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:23 smithi153 ceph-mon[35538]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:28:24.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:23 smithi019 ceph-mon[35186]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:28:24.347 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:28:24.355 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-16T00:28:24.355 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T00:28:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:24.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:24 smithi001 ceph-mon[30934]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:24.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:24 smithi153 ceph-mon[35538]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:25.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:24 smithi019 ceph-mon[35186]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:25.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 -- 172.21.15.1:0/307984057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 msgr2=0x7fb030100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 --2- 172.21.15.1:0/307984057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 0x7fb030100f10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb020009900 tx=0x7fb02002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 -- 172.21.15.1:0/307984057 shutdown_connections 2024-09-16T00:28:25.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 --2- 172.21.15.1:0/307984057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0301026d0 0x7fb030108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 --2- 172.21.15.1:0/307984057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030101d10 0x7fb030102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 --2- 172.21.15.1:0/307984057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 0x7fb030100f10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 -- 172.21.15.1:0/307984057 >> 172.21.15.1:0/307984057 conn(0x7fb0300fc2a0 msgr2=0x7fb0300fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 -- 172.21.15.1:0/307984057 shutdown_connections 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 -- 172.21.15.1:0/307984057 wait complete. 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.123+0000 7fb03527c640 1 Processor -- start 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb03527c640 1 -- start start 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb03527c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 0x7fb030197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb03527c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030101d10 0x7fb030198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb03527c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0301026d0 0x7fb03019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb03527c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb03010c770 con 0x7fb030100b10 2024-09-16T00:28:25.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb03527c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb03010c5f0 con 0x7fb0301026d0 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb03527c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb03010c8f0 con 0x7fb030101d10 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 0x7fb030197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 0x7fb030197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48060/0 (socket says 172.21.15.1:48060) 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02effd640 1 -- 172.21.15.1:0/39253080 learned_addr learned my addr 172.21.15.1:0/39253080 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02e7fc640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030101d10 0x7fb030198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02f7fe640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0301026d0 0x7fb03019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02effd640 1 -- 172.21.15.1:0/39253080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030101d10 msgr2=0x7fb030198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02effd640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030101d10 0x7fb030198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02effd640 1 -- 172.21.15.1:0/39253080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0301026d0 msgr2=0x7fb03019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02effd640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0301026d0 0x7fb03019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02effd640 1 -- 172.21.15.1:0/39253080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03019fb20 con 0x7fb030100b10 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02f7fe640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0301026d0 0x7fb03019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.124+0000 7fb02effd640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 0x7fb030197ef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb020031610 tx=0x7fb0200043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.125+0000 7fb017fff640 1 -- 172.21.15.1:0/39253080 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02003f030 con 0x7fb030100b10 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.125+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03019fdb0 con 0x7fb030100b10 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.125+0000 7fb017fff640 1 -- 172.21.15.1:0/39253080 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb020003ce0 con 0x7fb030100b10 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.125+0000 7fb017fff640 1 -- 172.21.15.1:0/39253080 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020035070 con 0x7fb030100b10 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.125+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0301a0200 con 0x7fb030100b10 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.126+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0301a04f0 con 0x7fb030100b10 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.127+0000 7fb017fff640 1 -- 172.21.15.1:0/39253080 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fb020042030 con 0x7fb030100b10 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.127+0000 7fb017fff640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb0180a3270 0x7fb0180a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.128+0000 7fb02e7fc640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb0180a3270 0x7fb0180a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.128+0000 7fb017fff640 1 -- 172.21.15.1:0/39253080 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb0200f06d0 con 0x7fb030100b10 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.128+0000 7fb02e7fc640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb0180a3270 0x7fb0180a5730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb024005e10 tx=0x7fb024005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.131+0000 7fb017fff640 1 -- 172.21.15.1:0/39253080 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0200bcf90 con 0x7fb030100b10 2024-09-16T00:28:25.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.360+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb0301a07e0 con 0x7fb030100b10 2024-09-16T00:28:25.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.361+0000 7fb017fff640 1 -- 172.21.15.1:0/39253080 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3076 (secure 0 0 0) 0x7fb0200bc890 con 0x7fb030100b10 2024-09-16T00:28:25.361 INFO:teuthology.orchestra.run.smithi001.stdout:e24 2024-09-16T00:28:25.363 INFO:teuthology.orchestra.run.smithi001.stdout:btime 2024-09-16T00:27:04:602627+0000 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:legacy client fscid: 1 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:fs_name cephfs 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 24 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-16T00:26:18.890037+0000 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-09-16T00:27:04.334709+0000 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:tableserver 0 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:root 0 2024-09-16T00:28:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:session_timeout 300 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:session_autoclose 300 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:max_file_size 1099511627776 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:max_xattr_size 65536 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:required_client_features {} 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure 0 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure_osd_epoch 0 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:max_mds 3 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:in 0,1,2 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:up {0=24506,1=24470,2=24469} 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:failed 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:damaged 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:stopped 2024-09-16T00:28:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:data_pools [3,4] 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:metadata_pool 2 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:inline_data disabled 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:balancer 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:bal_rank_mask -1 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:standby_count_wanted 0 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:qdb_cluster leader: 24506 members: 24506,24470,24469 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b{0:24506} state up:active seq 4 addr [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c{0:24451} state up:standby-replay seq 1 addr [v2:172.21.15.153:6836/3490951208,v1:172.21.15.153:6837/3490951208] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.h{1:24470} state up:active seq 18 addr [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.f{1:24445} state up:standby-replay seq 1 addr [v2:172.21.15.153:6834/3078906320,v1:172.21.15.153:6835/3078906320] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.i{2:24469} state up:active seq 17 addr [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.e{2:24443} state up:standby-replay seq 1 addr [v2:172.21.15.19:6834/389176790,v1:172.21.15.19:6835/389176790] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:25.366 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:25.367 INFO:teuthology.orchestra.run.smithi001.stdout:Standby daemons: 2024-09-16T00:28:25.367 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:25.367 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.j{-1:14532} state up:standby seq 1 addr [v2:172.21.15.1:6834/2811424948,v1:172.21.15.1:6835/2811424948] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.367 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d{-1:14544} state up:standby seq 1 addr [v2:172.21.15.1:6836/436431990,v1:172.21.15.1:6837/436431990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.367 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.g{-1:14607} state up:standby seq 1 addr [v2:172.21.15.1:6838/4066219801,v1:172.21.15.1:6839/4066219801] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.367 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a{-1:14613} state up:standby seq 1 addr [v2:172.21.15.1:6840/208912636,v1:172.21.15.1:6841/208912636] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.367 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.l{-1:24409} state up:standby seq 1 addr [v2:172.21.15.153:6832/1581415887,v1:172.21.15.153:6833/1581415887] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.367 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.k{-1:24425} state up:standby seq 1 addr [v2:172.21.15.19:6832/1257631294,v1:172.21.15.19:6833/1257631294] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:28:25.368 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 24 2024-09-16T00:28:25.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb0180a3270 msgr2=0x7fb0180a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb0180a3270 0x7fb0180a5730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb024005e10 tx=0x7fb024005da0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 msgr2=0x7fb030197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 0x7fb030197ef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb020031610 tx=0x7fb0200043b0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 shutdown_connections 2024-09-16T00:28:25.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0301026d0 0x7fb03019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb0180a3270 0x7fb0180a5730 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030101d10 0x7fb030198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 --2- 172.21.15.1:0/39253080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb030100b10 0x7fb030197ef0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.363+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 >> 172.21.15.1:0/39253080 conn(0x7fb0300fc2a0 msgr2=0x7fb03006a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.364+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 shutdown_connections 2024-09-16T00:28:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:25.364+0000 7fb03527c640 1 -- 172.21.15.1:0/39253080 wait complete. 2024-09-16T00:28:25.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:25 smithi001 ceph-mon[30934]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:25.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:25 smithi001 ceph-mon[30934]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:28:25.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:25 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/39253080' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:28:25.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:25 smithi153 ceph-mon[35538]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:25.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:25 smithi153 ceph-mon[35538]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:28:25.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:25 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/39253080' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:28:26.017 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-16T00:28:26.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:25 smithi019 ceph-mon[35186]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:26.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:25 smithi019 ceph-mon[35186]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:28:26.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:25 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/39253080' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:28:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.823+0000 7f3848ea4640 1 -- 172.21.15.1:0/1424493311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 msgr2=0x7f384410bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.823+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1424493311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 0x7f384410bf40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f383c009ca0 tx=0x7f383c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.823+0000 7f3848ea4640 1 -- 172.21.15.1:0/1424493311 shutdown_connections 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.823+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1424493311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 0x7f384410bf40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.823+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1424493311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3844074c50 0x7f38440750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.823+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1424493311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3844076220 0x7f3844074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.823+0000 7f3848ea4640 1 -- 172.21.15.1:0/1424493311 >> 172.21.15.1:0/1424493311 conn(0x7f38440fd390 msgr2=0x7f38440ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.823+0000 7f3848ea4640 1 -- 172.21.15.1:0/1424493311 shutdown_connections 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.823+0000 7f3848ea4640 1 -- 172.21.15.1:0/1424493311 wait complete. 2024-09-16T00:28:26.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3848ea4640 1 Processor -- start 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3848ea4640 1 -- start start 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3848ea4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3844074c50 0x7f384419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3848ea4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3844076220 0x7f384419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3848ea4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 0x7f38441a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3848ea4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f384410fa80 con 0x7f3844109a80 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3848ea4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f384410f900 con 0x7f3844074c50 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3848ea4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f384410fc00 con 0x7f3844076220 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3842d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 0x7f38441a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f384259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3844074c50 0x7f384419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3842d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 0x7f38441a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48076/0 (socket says 172.21.15.1:48076) 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f3842d9d640 1 -- 172.21.15.1:0/1431880790 learned_addr learned my addr 172.21.15.1:0/1431880790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.824+0000 7f383bfff640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3844076220 0x7f384419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:26.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f3842d9d640 1 -- 172.21.15.1:0/1431880790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3844076220 msgr2=0x7f384419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f3842d9d640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3844076220 0x7f384419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f3842d9d640 1 -- 172.21.15.1:0/1431880790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3844074c50 msgr2=0x7f384419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f3842d9d640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3844074c50 0x7f384419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f3842d9d640 1 -- 172.21.15.1:0/1431880790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38441a1cb0 con 0x7f3844109a80 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f384259c640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3844074c50 0x7f384419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f3842d9d640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 0x7f38441a1640 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f383c005e30 tx=0x7f383c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f383b7fe640 1 -- 172.21.15.1:0/1431880790 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c003c30 con 0x7f3844109a80 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f383b7fe640 1 -- 172.21.15.1:0/1431880790 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c003dd0 con 0x7f3844109a80 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38441a1f40 con 0x7f3844109a80 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f383b7fe640 1 -- 172.21.15.1:0/1431880790 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c02fde0 con 0x7f3844109a80 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.825+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38441a23f0 con 0x7f3844109a80 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.826+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3844108800 con 0x7f3844109a80 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.828+0000 7f383b7fe640 1 -- 172.21.15.1:0/1431880790 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f383c004050 con 0x7f3844109a80 2024-09-16T00:28:26.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.829+0000 7f383b7fe640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f38300a3270 0x7f38300a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:26.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.829+0000 7f384259c640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f38300a3270 0x7f38300a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:26.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.829+0000 7f383b7fe640 1 -- 172.21.15.1:0/1431880790 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f383c0f1160 con 0x7f3844109a80 2024-09-16T00:28:26.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.829+0000 7f384259c640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f38300a3270 0x7f38300a5730 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3844076600 tx=0x7f383400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:26.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:26.832+0000 7f383b7fe640 1 -- 172.21.15.1:0/1431880790 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f383c0bda20 con 0x7f3844109a80 2024-09-16T00:28:27.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.007+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f384402d7c0 con 0x7f3844109a80 2024-09-16T00:28:27.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.008+0000 7f383b7fe640 1 -- 172.21.15.1:0/1431880790 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5692 (secure 0 0 0) 0x7f383c0bd320 con 0x7f3844109a80 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 91 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-16T00:18:28.249157+0000 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-09-16T00:27:26.243912+0000 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 27 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2024-09-16T00:28:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client luminous 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release squid 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 12 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6802/2752149463,v1:172.21.15.1:6803/2752149463] [v2:172.21.15.1:6804/2752149463,v1:172.21.15.1:6805/2752149463] exists,up ff1bfb29-43e6-4e65-8d96-715b74e01491 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6800/3708883557,v1:172.21.15.19:6801/3708883557] [v2:172.21.15.19:6802/3708883557,v1:172.21.15.19:6803/3708883557] exists,up 6449fa21-488a-46cb-9e6c-1391e84c0f35 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6800/846294555,v1:172.21.15.153:6801/846294555] [v2:172.21.15.153:6802/846294555,v1:172.21.15.153:6803/846294555] exists,up 1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6810/2589582331,v1:172.21.15.1:6811/2589582331] [v2:172.21.15.1:6812/2589582331,v1:172.21.15.1:6813/2589582331] exists,up 926a23b3-e3d6-41bf-856e-226f69d8915b 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 32 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6808/616138863,v1:172.21.15.19:6809/616138863] [v2:172.21.15.19:6810/616138863,v1:172.21.15.19:6811/616138863] exists,up e8536388-50a8-4dd3-acd2-bd8a71f8cfad 2024-09-16T00:28:27.010 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 37 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6808/394316446,v1:172.21.15.153:6809/394316446] [v2:172.21.15.153:6810/394316446,v1:172.21.15.153:6811/394316446] exists,up 617ccae2-59c9-45bd-83f0-59b1f3555020 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 45 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6818/2734761121,v1:172.21.15.1:6819/2734761121] [v2:172.21.15.1:6820/2734761121,v1:172.21.15.1:6821/2734761121] exists,up b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 50 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6816/549185107,v1:172.21.15.19:6817/549185107] [v2:172.21.15.19:6818/549185107,v1:172.21.15.19:6819/549185107] exists,up 9860f5c4-575b-402c-98a2-3436d37b4cfa 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 up in weight 1 up_from 57 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6816/2981426956,v1:172.21.15.153:6817/2981426956] [v2:172.21.15.153:6818/2981426956,v1:172.21.15.153:6819/2981426956] exists,up 2371d5a7-e6c3-415f-b6d3-35df1a4c3a27 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 up in weight 1 up_from 64 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6826/4278887169,v1:172.21.15.1:6827/4278887169] [v2:172.21.15.1:6828/4278887169,v1:172.21.15.1:6829/4278887169] exists,up 92377e46-86e8-49b1-91b3-8180013968bb 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 up in weight 1 up_from 69 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6824/1721081370,v1:172.21.15.19:6825/1721081370] [v2:172.21.15.19:6826/1721081370,v1:172.21.15.19:6827/1721081370] exists,up 74fcd28a-7ae8-4dca-a314-b22a8c871db8 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 up in weight 1 up_from 76 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6824/3642895016,v1:172.21.15.153:6825/3642895016] [v2:172.21.15.153:6826/3642895016,v1:172.21.15.153:6827/3642895016] exists,up cf7afb8e-9c11-460a-960d-57f1eba3ead6 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.4 [2,11] 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.14 [2,4] 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.1c [8,11] 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.0 [4,8,5,9] 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.6 [3,7] 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.17 [4,8] 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.21 [4,7] 2024-09-16T00:28:27.011 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.36 [4,9] 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.37 [5,6] 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.38 [3,9,5,6] 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.3d [1,8] 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.12 [5,10] 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.16 [9,10] 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.1c [9,8] 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.1f [9,8] 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/178572347 expires 2024-09-17T00:19:24.226024+0000 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/3932831772 expires 2024-09-17T00:19:24.226024+0000 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/3251079055 expires 2024-09-17T00:19:24.226024+0000 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/3191294475 expires 2024-09-17T00:19:24.226024+0000 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/178572347 expires 2024-09-17T00:19:24.226024+0000 2024-09-16T00:28:27.012 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1613718962 expires 2024-09-17T00:19:00.598574+0000 2024-09-16T00:28:27.013 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/786826717 expires 2024-09-17T00:19:00.598574+0000 2024-09-16T00:28:27.013 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/786826717 expires 2024-09-17T00:19:00.598574+0000 2024-09-16T00:28:27.013 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/639100255 expires 2024-09-17T00:19:00.598574+0000 2024-09-16T00:28:27.013 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/365027668 expires 2024-09-17T00:19:00.598574+0000 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f38300a3270 msgr2=0x7f38300a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f38300a3270 0x7f38300a5730 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3844076600 tx=0x7f383400b390 comp rx=0 tx=0).stop 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 msgr2=0x7f38441a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 0x7f38441a1640 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f383c005e30 tx=0x7f383c031ee0 comp rx=0 tx=0).stop 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 shutdown_connections 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844109a80 0x7f38441a1640 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f38300a3270 0x7f38300a5730 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3844076220 0x7f384419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 --2- 172.21.15.1:0/1431880790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3844074c50 0x7f384419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.010+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 >> 172.21.15.1:0/1431880790 conn(0x7f38440fd390 msgr2=0x7f38441059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.011+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 shutdown_connections 2024-09-16T00:28:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:27.011+0000 7f3848ea4640 1 -- 172.21.15.1:0/1431880790 wait complete. 2024-09-16T00:28:27.662 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-16T00:28:27.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:27.933 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:27.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:27 smithi001 ceph-mon[30934]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:28:27.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:27 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1431880790' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:28:27.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:27 smithi153 ceph-mon[35538]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:28:27.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:27 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1431880790' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:28:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:27 smithi019 ceph-mon[35186]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:28:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:27 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1431880790' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:28:28.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 -- 172.21.15.1:0/205109309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 msgr2=0x7f9ab410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:28.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 --2- 172.21.15.1:0/205109309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 0x7f9ab410da40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009ca0 tx=0x7f9aac02a950 comp rx=0 tx=0).stop 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 -- 172.21.15.1:0/205109309 shutdown_connections 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 --2- 172.21.15.1:0/205109309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 0x7f9ab410da40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 --2- 172.21.15.1:0/205109309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab4102570 0x7f9ab41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 --2- 172.21.15.1:0/205109309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab4100e20 0x7f9ab4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 -- 172.21.15.1:0/205109309 >> 172.21.15.1:0/205109309 conn(0x7f9ab40fc7d0 msgr2=0x7f9ab40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 -- 172.21.15.1:0/205109309 shutdown_connections 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 -- 172.21.15.1:0/205109309 wait complete. 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.431+0000 7f9abb998640 1 Processor -- start 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9abb998640 1 -- start start 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9abb998640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab4100e20 0x7f9ab40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9abb998640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab4102570 0x7f9ab40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:28.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9abb998640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 0x7f9ab40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9abb998640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4110f80 con 0x7f9ab4102ef0 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9abb998640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4110e00 con 0x7f9ab4100e20 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9abb998640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4111100 con 0x7f9ab4102570 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9f35640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 0x7f9ab40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9f35640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 0x7f9ab40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48096/0 (socket says 172.21.15.1:48096) 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9f35640 1 -- 172.21.15.1:0/2578026392 learned_addr learned my addr 172.21.15.1:0/2578026392 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9734640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab4100e20 0x7f9ab40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab8f33640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab4102570 0x7f9ab40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9f35640 1 -- 172.21.15.1:0/2578026392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab4102570 msgr2=0x7f9ab40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9f35640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab4102570 0x7f9ab40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9f35640 1 -- 172.21.15.1:0/2578026392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab4100e20 msgr2=0x7f9ab40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9f35640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab4100e20 0x7f9ab40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9f35640 1 -- 172.21.15.1:0/2578026392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab41a6950 con 0x7f9ab4102ef0 2024-09-16T00:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab8f33640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab4102570 0x7f9ab40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9734640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab4100e20 0x7f9ab40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:28.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.432+0000 7f9ab9f35640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 0x7f9ab40fd700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9aac02ae70 tx=0x7f9aac02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:28.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.433+0000 7f9aaa7fc640 1 -- 172.21.15.1:0/2578026392 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac034040 con 0x7f9ab4102ef0 2024-09-16T00:28:28.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.433+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab41a6be0 con 0x7f9ab4102ef0 2024-09-16T00:28:28.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.433+0000 7f9aaa7fc640 1 -- 172.21.15.1:0/2578026392 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aac030920 con 0x7f9ab4102ef0 2024-09-16T00:28:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.433+0000 7f9aaa7fc640 1 -- 172.21.15.1:0/2578026392 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac045480 con 0x7f9ab4102ef0 2024-09-16T00:28:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.433+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab41a7110 con 0x7f9ab4102ef0 2024-09-16T00:28:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.434+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab4101200 con 0x7f9ab4102ef0 2024-09-16T00:28:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.435+0000 7f9aaa7fc640 1 -- 172.21.15.1:0/2578026392 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9aac037020 con 0x7f9ab4102ef0 2024-09-16T00:28:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.436+0000 7f9aaa7fc640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9aa00a3240 0x7f9aa00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.436+0000 7f9ab9734640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9aa00a3240 0x7f9aa00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.436+0000 7f9aaa7fc640 1 -- 172.21.15.1:0/2578026392 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9aac0f0ed0 con 0x7f9ab4102ef0 2024-09-16T00:28:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.436+0000 7f9ab9734640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9aa00a3240 0x7f9aa00a5700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000abe0 tx=0x7f9ab000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:28.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.439+0000 7f9aaa7fc640 1 -- 172.21.15.1:0/2578026392 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9aac0811b0 con 0x7f9ab4102ef0 2024-09-16T00:28:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.651+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ab40fe1b0 con 0x7f9ab4102ef0 2024-09-16T00:28:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.651+0000 7f9aaa7fc640 1 -- 172.21.15.1:0/2578026392 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11894 (secure 0 0 0) 0x7f9ab40fe1b0 con 0x7f9ab4102ef0 2024-09-16T00:28:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:28.652 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"btime":"2024-09-16T00:27:04:602627+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:27:04.334709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24470,24469]},"id":1}]} 2024-09-16T00:28:28.652 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 24 2024-09-16T00:28:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.653+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9aa00a3240 msgr2=0x7f9aa00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.653+0000 7f9abb998640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9aa00a3240 0x7f9aa00a5700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000abe0 tx=0x7f9ab000a5c0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.653+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 msgr2=0x7f9ab40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.653+0000 7f9abb998640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 0x7f9ab40fd700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9aac02ae70 tx=0x7f9aac02f5d0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.654+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 shutdown_connections 2024-09-16T00:28:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.654+0000 7f9abb998640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4102ef0 0x7f9ab40fd700 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.654+0000 7f9abb998640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9aa00a3240 0x7f9aa00a5700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.654+0000 7f9abb998640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab4102570 0x7f9ab40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.654+0000 7f9abb998640 1 --2- 172.21.15.1:0/2578026392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab4100e20 0x7f9ab40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:28.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.654+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 >> 172.21.15.1:0/2578026392 conn(0x7f9ab40fc7d0 msgr2=0x7f9ab406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:28.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.654+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 shutdown_connections 2024-09-16T00:28:28.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:28.654+0000 7f9abb998640 1 -- 172.21.15.1:0/2578026392 wait complete. 2024-09-16T00:28:29.383 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:29.640 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:29 smithi001 ceph-mon[30934]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:29 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2578026392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:29 smithi153 ceph-mon[35538]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:29 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2578026392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:29 smithi019 ceph-mon[35186]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:29 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2578026392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.174+0000 7f094edf9640 1 -- 172.21.15.1:0/3966981857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948102790 msgr2=0x7f09481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.174+0000 7f094edf9640 1 --2- 172.21.15.1:0/3966981857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948102790 0x7f09481067e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0938009900 tx=0x7f093802a410 comp rx=0 tx=0).stop 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.174+0000 7f094edf9640 1 -- 172.21.15.1:0/3966981857 shutdown_connections 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.174+0000 7f094edf9640 1 --2- 172.21.15.1:0/3966981857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948106f10 0x7f094810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.174+0000 7f094edf9640 1 --2- 172.21.15.1:0/3966981857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948102790 0x7f09481067e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.174+0000 7f094edf9640 1 --2- 172.21.15.1:0/3966981857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0948101de0 0x7f09481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.174+0000 7f094edf9640 1 -- 172.21.15.1:0/3966981857 >> 172.21.15.1:0/3966981857 conn(0x7f09480fd8c0 msgr2=0x7f09480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.174+0000 7f094edf9640 1 -- 172.21.15.1:0/3966981857 shutdown_connections 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.174+0000 7f094edf9640 1 -- 172.21.15.1:0/3966981857 wait complete. 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094edf9640 1 Processor -- start 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094edf9640 1 -- start start 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094edf9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948101de0 0x7f0948198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094edf9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948102790 0x7f0948198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094edf9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0948106f10 0x7f094819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094edf9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0948111f30 con 0x7f0948101de0 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094edf9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0948111db0 con 0x7f0948102790 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094edf9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09481120b0 con 0x7f0948106f10 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094cb95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948101de0 0x7f0948198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094cb95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948101de0 0x7f0948198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48114/0 (socket says 172.21.15.1:48114) 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094cb95640 1 -- 172.21.15.1:0/237621382 learned_addr learned my addr 172.21.15.1:0/237621382 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f093ffff640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948102790 0x7f0948198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094d396640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0948106f10 0x7f094819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094cb95640 1 -- 172.21.15.1:0/237621382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0948106f10 msgr2=0x7f094819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094cb95640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0948106f10 0x7f094819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094cb95640 1 -- 172.21.15.1:0/237621382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948102790 msgr2=0x7f0948198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094cb95640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948102790 0x7f0948198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f094cb95640 1 -- 172.21.15.1:0/237621382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094819d3e0 con 0x7f0948101de0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.175+0000 7f093ffff640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948102790 0x7f0948198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.176+0000 7f094cb95640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948101de0 0x7f0948198430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f094400d950 tx=0x7f094400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.176+0000 7f093dffb640 1 -- 172.21.15.1:0/237621382 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944017070 con 0x7f0948101de0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.176+0000 7f093dffb640 1 -- 172.21.15.1:0/237621382 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0944009c30 con 0x7f0948101de0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.176+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094819d6d0 con 0x7f0948101de0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.176+0000 7f093dffb640 1 -- 172.21.15.1:0/237621382 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094400f6b0 con 0x7f0948101de0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.176+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09481a4fb0 con 0x7f0948101de0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.177+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09481021c0 con 0x7f0948101de0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.178+0000 7f093dffb640 1 -- 172.21.15.1:0/237621382 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f09440109d0 con 0x7f0948101de0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.179+0000 7f093dffb640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09300a3240 0x7f09300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.179+0000 7f093ffff640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09300a3240 0x7f09300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.179+0000 7f093dffb640 1 -- 172.21.15.1:0/237621382 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f09440cf8f0 con 0x7f0948101de0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.179+0000 7f093ffff640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09300a3240 0x7f09300a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0938009230 tx=0x7f09380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.183+0000 7f093dffb640 1 -- 172.21.15.1:0/237621382 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0944009de0 con 0x7f0948101de0 2024-09-16T00:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.397+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f094810a0f0 con 0x7f0948101de0 2024-09-16T00:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.397+0000 7f093dffb640 1 -- 172.21.15.1:0/237621382 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11894 (secure 0 0 0) 0x7f094409bbc0 con 0x7f0948101de0 2024-09-16T00:28:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:30.398 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"btime":"2024-09-16T00:27:04:602627+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:27:04.334709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24470,24469]},"id":1}]} 2024-09-16T00:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 24 2024-09-16T00:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09300a3240 msgr2=0x7f09300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09300a3240 0x7f09300a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0938009230 tx=0x7f09380058d0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948101de0 msgr2=0x7f0948198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948101de0 0x7f0948198430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f094400d950 tx=0x7f094400de10 comp rx=0 tx=0).stop 2024-09-16T00:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 shutdown_connections 2024-09-16T00:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09300a3240 0x7f09300a5700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0948106f10 0x7f094819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948102790 0x7f0948198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 --2- 172.21.15.1:0/237621382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0948101de0 0x7f0948198430 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 >> 172.21.15.1:0/237621382 conn(0x7f09480fd8c0 msgr2=0x7f09480fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 shutdown_connections 2024-09-16T00:28:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:30.400+0000 7f094edf9640 1 -- 172.21.15.1:0/237621382 wait complete. 2024-09-16T00:28:30.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:30 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/237621382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:30.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:30 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/237621382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:31.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:30 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/237621382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:31.148 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:28:31.411 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:31.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:31 smithi001 ceph-mon[30934]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:31.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.933+0000 7f983e8f1640 1 -- 172.21.15.1:0/3861414016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 msgr2=0x7f983810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:31.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.933+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3861414016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 0x7f983810e600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9830009ca0 tx=0x7f983002a950 comp rx=0 tx=0).stop 2024-09-16T00:28:31.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.933+0000 7f983e8f1640 1 -- 172.21.15.1:0/3861414016 shutdown_connections 2024-09-16T00:28:31.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.933+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3861414016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 0x7f983810e600 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:31.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.933+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3861414016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98381036c0 0x7f983810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:31.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.933+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3861414016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9838102da0 0x7f9838103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:31.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.933+0000 7f983e8f1640 1 -- 172.21.15.1:0/3861414016 >> 172.21.15.1:0/3861414016 conn(0x7f98380fc7a0 msgr2=0x7f98380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:31.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.933+0000 7f983e8f1640 1 -- 172.21.15.1:0/3861414016 shutdown_connections 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.933+0000 7f983e8f1640 1 -- 172.21.15.1:0/3861414016 wait complete. 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.934+0000 7f983e8f1640 1 Processor -- start 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.934+0000 7f983e8f1640 1 -- start start 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.934+0000 7f983e8f1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9838102da0 0x7f9838198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.934+0000 7f983e8f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98381036c0 0x7f9838198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.934+0000 7f983e8f1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 0x7f983819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.934+0000 7f983e8f1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9838112070 con 0x7f983810c180 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.934+0000 7f983e8f1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9838111ef0 con 0x7f9838102da0 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.934+0000 7f983e8f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98381121f0 con 0x7f98381036c0 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.934+0000 7f983ce8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 0x7f983819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983ce8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 0x7f983819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42940/0 (socket says 172.21.15.1:42940) 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983ce8e640 1 -- 172.21.15.1:0/3004028846 learned_addr learned my addr 172.21.15.1:0/3004028846 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f98377fe640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98381036c0 0x7f9838198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f9837fff640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9838102da0 0x7f9838198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983ce8e640 1 -- 172.21.15.1:0/3004028846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98381036c0 msgr2=0x7f9838198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983ce8e640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98381036c0 0x7f9838198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983ce8e640 1 -- 172.21.15.1:0/3004028846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9838102da0 msgr2=0x7f9838198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983ce8e640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9838102da0 0x7f9838198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983ce8e640 1 -- 172.21.15.1:0/3004028846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983819d520 con 0x7f983810c180 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f98377fe640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98381036c0 0x7f9838198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f9837fff640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9838102da0 0x7f9838198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983ce8e640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 0x7f983819ce40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f98300098d0 tx=0x7f9830003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f98357fa640 1 -- 172.21.15.1:0/3004028846 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830034040 con 0x7f983810c180 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f98357fa640 1 -- 172.21.15.1:0/3004028846 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983002fe20 con 0x7f983810c180 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983819d7b0 con 0x7f983810c180 2024-09-16T00:28:31.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f98357fa640 1 -- 172.21.15.1:0/3004028846 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830039450 con 0x7f983810c180 2024-09-16T00:28:31.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.935+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381a50f0 con 0x7f983810c180 2024-09-16T00:28:31.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.937+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9838103200 con 0x7f983810c180 2024-09-16T00:28:31.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.937+0000 7f98357fa640 1 -- 172.21.15.1:0/3004028846 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9830037020 con 0x7f983810c180 2024-09-16T00:28:31.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.938+0000 7f98357fa640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98200a32c0 0x7f98200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:31.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.939+0000 7f9837fff640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98200a32c0 0x7f98200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:31.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.939+0000 7f98357fa640 1 -- 172.21.15.1:0/3004028846 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f98300f0e00 con 0x7f983810c180 2024-09-16T00:28:31.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.939+0000 7f9837fff640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98200a32c0 0x7f98200a5780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9828009930 tx=0x7f9828008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:31.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:31.943+0000 7f98357fa640 1 -- 172.21.15.1:0/3004028846 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98300bd720 con 0x7f983810c180 2024-09-16T00:28:31.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:31 smithi153 ceph-mon[35538]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:32.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:31 smithi019 ceph-mon[35186]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:32.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.119+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9838107200 con 0x7f983810c180 2024-09-16T00:28:32.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.121+0000 7f98357fa640 1 -- 172.21.15.1:0/3004028846 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24902 (secure 0 0 0) 0x7f98300bd020 con 0x7f983810c180 2024-09-16T00:28:32.121 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:32.122 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:27:26.243912+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:28:32.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98200a32c0 msgr2=0x7f98200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:32.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98200a32c0 0x7f98200a5780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9828009930 tx=0x7f9828008040 comp rx=0 tx=0).stop 2024-09-16T00:28:32.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 msgr2=0x7f983819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:32.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 0x7f983819ce40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f98300098d0 tx=0x7f9830003f10 comp rx=0 tx=0).stop 2024-09-16T00:28:32.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 shutdown_connections 2024-09-16T00:28:32.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f983810c180 0x7f983819ce40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:32.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98200a32c0 0x7f98200a5780 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:32.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98381036c0 0x7f9838198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:32.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 --2- 172.21.15.1:0/3004028846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9838102da0 0x7f9838198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:32.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 >> 172.21.15.1:0/3004028846 conn(0x7f98380fc7a0 msgr2=0x7f98380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:32.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 shutdown_connections 2024-09-16T00:28:32.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:32.123+0000 7f983e8f1640 1 -- 172.21.15.1:0/3004028846 wait complete. 2024-09-16T00:28:32.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:32 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3004028846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:32.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:26:18.890037+0000', 'modified': '2024-09-16T00:27:04.334709+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24470, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/389176790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 389176790}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 389176790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6835/3078906320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 3078906320}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 3078906320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6837/3490951208', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 3490951208}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 3490951208}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.153:6839/1065022363', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 1065022363}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 1065022363}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.19:6837/928170641', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6836', 'nonce': 928170641}, {'type': 'v1', 'addr': '172.21.15.19:6837', 'nonce': 928170641}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6839/3056769092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 3056769092}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 3056769092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24506, 24470, 24469]} 2024-09-16T00:28:32.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-16T00:28:32.833 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T00:28:32.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:32 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3004028846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:33.092 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:33.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:32 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3004028846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:33.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.607+0000 7faf9f0f4640 1 -- 172.21.15.1:0/3466930754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102570 msgr2=0x7faf981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.607+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/3466930754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102570 0x7faf981029b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf88009900 tx=0x7faf8802a480 comp rx=0 tx=0).stop 2024-09-16T00:28:33.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.607+0000 7faf9f0f4640 1 -- 172.21.15.1:0/3466930754 shutdown_connections 2024-09-16T00:28:33.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.607+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/3466930754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf98102ef0 0x7faf9810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.607+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/3466930754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102570 0x7faf981029b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.607+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/3466930754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf98100e20 0x7faf98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.607+0000 7faf9f0f4640 1 -- 172.21.15.1:0/3466930754 >> 172.21.15.1:0/3466930754 conn(0x7faf980fc7d0 msgr2=0x7faf980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.607+0000 7faf9f0f4640 1 -- 172.21.15.1:0/3466930754 shutdown_connections 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.607+0000 7faf9f0f4640 1 -- 172.21.15.1:0/3466930754 wait complete. 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9f0f4640 1 Processor -- start 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9f0f4640 1 -- start start 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9f0f4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf98100e20 0x7faf98197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9f0f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf98102570 0x7faf98197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9f0f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102ef0 0x7faf9819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9f0f4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf98110fc0 con 0x7faf98102ef0 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9f0f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf98110e40 con 0x7faf98102570 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9f0f4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faf98111140 con 0x7faf98100e20 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9d691640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102ef0 0x7faf9819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9d691640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102ef0 0x7faf9819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42968/0 (socket says 172.21.15.1:42968) 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9d691640 1 -- 172.21.15.1:0/246535914 learned_addr learned my addr 172.21.15.1:0/246535914 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.608+0000 7faf9ce90640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf98100e20 0x7faf98197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf8ffff640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf98102570 0x7faf98197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf9d691640 1 -- 172.21.15.1:0/246535914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf98100e20 msgr2=0x7faf98197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf9d691640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf98100e20 0x7faf98197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf9d691640 1 -- 172.21.15.1:0/246535914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf98102570 msgr2=0x7faf98197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf9d691640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf98102570 0x7faf98197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf9d691640 1 -- 172.21.15.1:0/246535914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9819c3f0 con 0x7faf98102ef0 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf9ce90640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf98100e20 0x7faf98197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf8ffff640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf98102570 0x7faf98197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf9d691640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102ef0 0x7faf9819bd10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf9000b970 tx=0x7faf9000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf8dffb640 1 -- 172.21.15.1:0/246535914 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf90019070 con 0x7faf98102ef0 2024-09-16T00:28:33.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf8dffb640 1 -- 172.21.15.1:0/246535914 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf9000cb40 con 0x7faf98102ef0 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf9f0f4640 1 -- 172.21.15.1:0/246535914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9819c6e0 con 0x7faf98102ef0 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf8dffb640 1 -- 172.21.15.1:0/246535914 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf90007650 con 0x7faf98102ef0 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.609+0000 7faf9f0f4640 1 -- 172.21.15.1:0/246535914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf981a3fc0 con 0x7faf98102ef0 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.611+0000 7faf8dffb640 1 -- 172.21.15.1:0/246535914 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7faf9001c070 con 0x7faf98102ef0 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7fafa14cd740 1 Processor -- start 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7fafa14cd740 1 -- start start 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c217380fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 0x55c21738b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 0x55c21738e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7fafa14cd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c2172e5090 con 0x55c217380be0 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7fafa14cd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c21728ca70 con 0x55c21738bdc0 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7fafa14cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c2172e1050 con 0x55c217381500 2024-09-16T00:28:33.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7faf8dffb640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf800a3240 0x7faf800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c217380fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c217380fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42970/0 (socket says 172.21.15.1:42970) 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7faf8ffff640 1 -- 172.21.15.1:0/2821117735 learned_addr learned my addr 172.21.15.1:0/2821117735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7faf9d691640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf800a3240 0x7faf800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7faf8dffb640 1 -- 172.21.15.1:0/246535914 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7faf900cf400 con 0x7faf98102ef0 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7faf9ce90640 1 --2- 172.21.15.1:0/2821117735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 0x55c21738b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7faf8ffff640 1 --2- 172.21.15.1:0/2821117735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 0x55c21738e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.612+0000 7faf8ffff640 1 -- 172.21.15.1:0/2821117735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 msgr2=0x55c21738b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.613+0000 7faf8ffff640 1 --2- 172.21.15.1:0/2821117735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 0x55c21738b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.613+0000 7faf8ffff640 1 -- 172.21.15.1:0/2821117735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 msgr2=0x55c21738e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.613+0000 7faf8ffff640 1 --2- 172.21.15.1:0/2821117735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 0x55c21738e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.613+0000 7faf8ffff640 1 -- 172.21.15.1:0/2821117735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c217299130 con 0x55c217380be0 2024-09-16T00:28:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.613+0000 7faf8ffff640 1 --2- 172.21.15.1:0/2821117735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c217380fc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faf88005ab0 tx=0x7faf88031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75275b53811de10d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.613+0000 7faf9d691640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf800a3240 0x7faf800a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faf90097a20 tx=0x7faf90090aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.613+0000 7faf7effd640 1 -- 172.21.15.1:0/2821117735 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88041020 con 0x55c217380be0 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.613+0000 7faf7effd640 1 -- 172.21.15.1:0/2821117735 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf880046c0 con 0x55c217380be0 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.613+0000 7faf7effd640 1 -- 172.21.15.1:0/2821117735 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88035070 con 0x55c217380be0 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 -- 172.21.15.1:0/2821117735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 msgr2=0x55c217380fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 --2- 172.21.15.1:0/2821117735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c217380fc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faf88005ab0 tx=0x7faf88031440 comp rx=0 tx=0).stop 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 -- 172.21.15.1:0/2821117735 shutdown_connections 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 --2- 172.21.15.1:0/2821117735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 0x55c21738e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 --2- 172.21.15.1:0/2821117735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 0x55c21738b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 --2- 172.21.15.1:0/2821117735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c217380fc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 -- 172.21.15.1:0/2821117735 >> 172.21.15.1:0/2821117735 conn(0x55c21737d170 msgr2=0x55c21737d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 -- 172.21.15.1:0/2821117735 shutdown_connections 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 -- 172.21.15.1:0/2821117735 wait complete. 2024-09-16T00:28:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 -- start start 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.614+0000 7fafa14cd740 1 Processor -- start 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21738b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 0x55c2173889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 0x55c217388ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7fafa14cd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c21738e990 con 0x55c217380be0 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7fafa14cd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c21729ff20 con 0x55c21738bdc0 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7fafa14cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c21738eb10 con 0x55c217381500 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7fafa14cd740 10 client.0 _ll_register_callbacks cb 0x55c217269030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21738b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21738b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42976/0 (socket says 172.21.15.1:42976) 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 -- 172.21.15.1:0/1141327998 learned_addr learned my addr 172.21.15.1:0/1141327998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf9ce90640 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 0x55c2173889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 0x55c217388ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 -- 172.21.15.1:0/1141327998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 msgr2=0x55c2173889b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 0x55c2173889b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 -- 172.21.15.1:0/1141327998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 msgr2=0x55c217388ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 0x55c217388ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 -- 172.21.15.1:0/1141327998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2170562b0 con 0x55c217380be0 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf9ce90640 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 0x55c2173889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf8ffff640 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21738b660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faf8802a4c0 tx=0x7faf88005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8b84077ae8cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.615+0000 7faf77fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7faf77fff640 1 -- 172.21.15.1:0/1141327998 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88035030 con 0x55c217380be0 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7faf77fff640 1 -- 172.21.15.1:0/1141327998 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7faf88031740 con 0x55c217380be0 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7fafa14cd740 1 -- 172.21.15.1:0/1141327998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf880046c0 con 0x55c217380be0 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7faf77fff640 1 -- 172.21.15.1:0/1141327998 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf88041040 con 0x55c217380be0 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7fafa14cd740 10 client.15180 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7fafa14cd740 1 -- 172.21.15.1:0/1141327998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c21739a2b0 con 0x55c217380be0 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7faf77fff640 1 -- 172.21.15.1:0/1141327998 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf880046c0 con 0x55c217380be0 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7faf77fff640 1 -- 172.21.15.1:0/1141327998 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7faf88003cd0 con 0x55c217380be0 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7fafa14cd740 10 client.15180 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7fafa14cd740 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7fafa14cd740 20 client.15180 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55c21739c3e0 0x55c21739d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7fafa14cd740 4 client.15180 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.616+0000 7fafa14cd740 1 -- 172.21.15.1:0/1141327998 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2172f2600 con 0x55c21739c3e0 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.617+0000 7faf9ce90640 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55c21739c3e0 0x55c21739d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.617+0000 7faf9ce90640 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55c21739c3e0 0x55c21739d850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17d967c1a4f09e32 server_cookie=99858db83822c9da in_seq=0 out_seq=0 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.617+0000 7faf77fff640 10 client.15180 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:28:33.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7faf77fff640 1 -- 172.21.15.1:0/1141327998 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2172f2600 con 0x55c21739c3e0 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7faf77fff640 10 client.15180 handle_command_reply: tid=0 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 client.15180 shutdown 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 -- 172.21.15.1:0/1141327998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 msgr2=0x55c21738b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21738b660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faf8802a4c0 tx=0x7faf88005d50 comp rx=0 tx=0).stop 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 -- 172.21.15.1:0/1141327998 shutdown_connections 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55c21739c3e0 0x55c21739d850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21738bdc0 0x55c217388ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7faf77fff640 0 client.15180 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217381500 0x55c2173889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1141327998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21738b660 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 -- 172.21.15.1:0/1141327998 >> 172.21.15.1:0/1141327998 conn(0x55c21737d170 msgr2=0x55c217382ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 -- 172.21.15.1:0/1141327998 shutdown_connections 2024-09-16T00:28:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 1 -- 172.21.15.1:0/1141327998 wait complete. 2024-09-16T00:28:33.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.618+0000 7fafa14cd740 20 client.15180 trim_cache size 0 max 16384 2024-09-16T00:28:33.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.636+0000 7fafa14cd740 1 Processor -- start 2024-09-16T00:28:33.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.636+0000 7fafa14cd740 1 -- start start 2024-09-16T00:28:33.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217380be0 0x55c21739d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c217387280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c21738bdc0 0x55c2173877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7fafa14cd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c21739bb90 con 0x55c21738bdc0 2024-09-16T00:28:33.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7fafa14cd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c21739c260 con 0x55c217381500 2024-09-16T00:28:33.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7fafa14cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c21739c930 con 0x55c217380be0 2024-09-16T00:28:33.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c21738bdc0 0x55c2173877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c21738bdc0 0x55c2173877c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42986/0 (socket says 172.21.15.1:42986) 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 -- 172.21.15.1:0/4087513755 learned_addr learned my addr 172.21.15.1:0/4087513755 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf9ce90640 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c217387280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217380be0 0x55c21739d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 -- 172.21.15.1:0/4087513755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217380be0 msgr2=0x55c21739d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217380be0 0x55c21739d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 -- 172.21.15.1:0/4087513755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 msgr2=0x55c217387280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c217387280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 -- 172.21.15.1:0/4087513755 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c217381330 con 0x55c21738bdc0 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf9ce90640 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c217387280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf8ffff640 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c21738bdc0 0x55c2173877c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faf880040d0 tx=0x7faf880042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c36113a1272ab79 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.637+0000 7faf7e7fc640 1 -- 172.21.15.1:0/4087513755 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88035070 con 0x55c21738bdc0 2024-09-16T00:28:33.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7faf7e7fc640 1 -- 172.21.15.1:0/4087513755 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf8804b030 con 0x55c21738bdc0 2024-09-16T00:28:33.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7faf7e7fc640 1 -- 172.21.15.1:0/4087513755 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88003850 con 0x55c21738bdc0 2024-09-16T00:28:33.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7fafa14cd740 1 -- 172.21.15.1:0/4087513755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c21738bdc0 msgr2=0x55c2173877c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7fafa14cd740 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c21738bdc0 0x55c2173877c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faf880040d0 tx=0x7faf880042e0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7fafa14cd740 1 -- 172.21.15.1:0/4087513755 shutdown_connections 2024-09-16T00:28:33.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7fafa14cd740 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c21738bdc0 0x55c2173877c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7fafa14cd740 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c217387280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7fafa14cd740 1 --2- 172.21.15.1:0/4087513755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c217380be0 0x55c21739d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7fafa14cd740 1 -- 172.21.15.1:0/4087513755 >> 172.21.15.1:0/4087513755 conn(0x55c21737d170 msgr2=0x55c217375ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7fafa14cd740 1 -- 172.21.15.1:0/4087513755 shutdown_connections 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.638+0000 7fafa14cd740 1 -- 172.21.15.1:0/4087513755 wait complete. 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7fafa14cd740 1 -- start start 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7fafa14cd740 1 Processor -- start 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21739eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c21739f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7fafa14cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c21738bdc0 0x55c217404ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7fafa14cd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c21739d6d0 con 0x55c217380be0 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7fafa14cd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c21739d000 con 0x55c217381500 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7fafa14cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c21739dda0 con 0x55c21738bdc0 2024-09-16T00:28:33.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7fafa14cd740 10 client.0 _ll_register_callbacks cb 0x7faf9400b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21739eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21739eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42992/0 (socket says 172.21.15.1:42992) 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7faf8ffff640 1 -- 172.21.15.1:0/1195266087 learned_addr learned my addr 172.21.15.1:0/1195266087 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.639+0000 7faf8ffff640 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c21738bdc0 0x55c217404ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf9ce90640 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c21739f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf8ffff640 1 -- 172.21.15.1:0/1195266087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c21738bdc0 msgr2=0x55c217404ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf8ffff640 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c21738bdc0 0x55c217404ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf8ffff640 1 -- 172.21.15.1:0/1195266087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 msgr2=0x55c21739f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf8ffff640 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c21739f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf8ffff640 1 -- 172.21.15.1:0/1195266087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c217405020 con 0x55c217380be0 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf9ce90640 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c21739f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf8ffff640 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21739eff0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faf88004430 tx=0x7faf8803ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3459c18a0ffc48e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:33.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf77fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf77fff640 1 -- 172.21.15.1:0/1195266087 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88061060 con 0x55c217380be0 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7fafa14cd740 1 -- 172.21.15.1:0/1195266087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf8804b030 con 0x55c217380be0 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf77fff640 1 -- 172.21.15.1:0/1195266087 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7faf880037f0 con 0x55c217380be0 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf77fff640 1 -- 172.21.15.1:0/1195266087 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf88035040 con 0x55c217380be0 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7fafa14cd740 10 client.15186 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7fafa14cd740 1 -- 172.21.15.1:0/1195266087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c2174062a0 con 0x55c217380be0 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.640+0000 7faf77fff640 1 -- 172.21.15.1:0/1195266087 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf8804b030 con 0x55c217380be0 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7faf77fff640 1 -- 172.21.15.1:0/1195266087 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7faf88046420 con 0x55c217380be0 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7fafa14cd740 10 client.15186 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7fafa14cd740 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7fafa14cd740 20 client.15186 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55c21740a660 0x55c217409370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7fafa14cd740 4 client.15186 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7fafa14cd740 1 -- 172.21.15.1:0/1195266087 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "status"}) -- 0x55c2172ecb10 con 0x55c21740a660 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7faf9ce90640 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55c21740a660 0x55c217409370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7faf9ce90640 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55c21740a660 0x55c217409370 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b70e68b1c9c0226a server_cookie=62d3eae70277c964 in_seq=0 out_seq=0 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.641+0000 7faf77fff640 10 client.15186 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7faf77fff640 1 -- 172.21.15.1:0/1195266087 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c2172ecb10 con 0x55c21740a660 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7faf77fff640 10 client.15186 handle_command_reply: tid=0 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 client.15186 shutdown 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 -- 172.21.15.1:0/1195266087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 msgr2=0x55c21739eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21739eff0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faf88004430 tx=0x7faf8803ae40 comp rx=0 tx=0).stop 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 -- 172.21.15.1:0/1195266087 shutdown_connections 2024-09-16T00:28:33.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c21738bdc0 0x55c217404ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55c21740a660 0x55c217409370 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c217381500 0x55c21739f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 --2- 172.21.15.1:0/1195266087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c217380be0 0x55c21739eff0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7faf77fff640 0 client.15186 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:28:33.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 -- 172.21.15.1:0/1195266087 >> 172.21.15.1:0/1195266087 conn(0x55c21737d170 msgr2=0x55c217379800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:33.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 -- 172.21.15.1:0/1195266087 shutdown_connections 2024-09-16T00:28:33.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 1 -- 172.21.15.1:0/1195266087 wait complete. 2024-09-16T00:28:33.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.642+0000 7fafa14cd740 20 client.15186 trim_cache size 0 max 16384 2024-09-16T00:28:33.648 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24469, 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 92.042207583000007, 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 24, 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 89, 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 154.76277217500001 2024-09-16T00:28:33.649 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:28:33.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 -- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf800a3240 msgr2=0x7faf800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf800a3240 0x7faf800a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faf90097a20 tx=0x7faf90090aa0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 -- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102ef0 msgr2=0x7faf9819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102ef0 0x7faf9819bd10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf9000b970 tx=0x7faf9000be30 comp rx=0 tx=0).stop 2024-09-16T00:28:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 -- 172.21.15.1:0/246535914 shutdown_connections 2024-09-16T00:28:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf98102ef0 0x7faf9819bd10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf98102570 0x7faf98197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf800a3240 0x7faf800a5700 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 --2- 172.21.15.1:0/246535914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf98100e20 0x7faf98197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 -- 172.21.15.1:0/246535914 >> 172.21.15.1:0/246535914 conn(0x7faf980fc7d0 msgr2=0x7faf980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 -- 172.21.15.1:0/246535914 shutdown_connections 2024-09-16T00:28:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:33.643+0000 7faf9f0f4640 1 -- 172.21.15.1:0/246535914 wait complete. 2024-09-16T00:28:33.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:33 smithi001 ceph-mon[30934]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:28:33.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:33 smithi153 ceph-mon[35538]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:28:34.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:33 smithi019 ceph-mon[35186]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:28:34.410 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T00:28:34.670 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.206+0000 7fe097e91640 1 -- 172.21.15.1:0/1889018439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090102790 msgr2=0x7fe0901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.206+0000 7fe097e91640 1 --2- 172.21.15.1:0/1889018439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090102790 0x7fe0901067e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe084009900 tx=0x7fe08402a410 comp rx=0 tx=0).stop 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.206+0000 7fe097e91640 1 -- 172.21.15.1:0/1889018439 shutdown_connections 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.206+0000 7fe097e91640 1 --2- 172.21.15.1:0/1889018439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe090106f10 0x7fe09010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.206+0000 7fe097e91640 1 --2- 172.21.15.1:0/1889018439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090102790 0x7fe0901067e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.206+0000 7fe097e91640 1 --2- 172.21.15.1:0/1889018439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe090101de0 0x7fe0901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.206+0000 7fe097e91640 1 -- 172.21.15.1:0/1889018439 >> 172.21.15.1:0/1889018439 conn(0x7fe0900fd8c0 msgr2=0x7fe0900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.206+0000 7fe097e91640 1 -- 172.21.15.1:0/1889018439 shutdown_connections 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe097e91640 1 -- 172.21.15.1:0/1889018439 wait complete. 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe097e91640 1 Processor -- start 2024-09-16T00:28:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe097e91640 1 -- start start 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe097e91640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090101de0 0x7fe0901983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe097e91640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe090102790 0x7fe0901988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe097e91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe090106f10 0x7fe09019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe097e91640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe090111f70 con 0x7fe090101de0 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe097e91640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe090111df0 con 0x7fe090106f10 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe097e91640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0901120f0 con 0x7fe090102790 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.207+0000 7fe095c2d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090101de0 0x7fe0901983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe095c2d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090101de0 0x7fe0901983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43002/0 (socket says 172.21.15.1:43002) 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe095c2d640 1 -- 172.21.15.1:0/2074765041 learned_addr learned my addr 172.21.15.1:0/2074765041 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe09642e640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe090106f10 0x7fe09019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe095c2d640 1 -- 172.21.15.1:0/2074765041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe090102790 msgr2=0x7fe0901988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe095c2d640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe090102790 0x7fe0901988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe095c2d640 1 -- 172.21.15.1:0/2074765041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe090106f10 msgr2=0x7fe09019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe095c2d640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe090106f10 0x7fe09019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe095c2d640 1 -- 172.21.15.1:0/2074765041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09019d360 con 0x7fe090101de0 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe095c2d640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090101de0 0x7fe0901983b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00c890 tx=0x7fe08c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe082ffd640 1 -- 172.21.15.1:0/2074765041 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004220 con 0x7fe090101de0 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe097e91640 1 -- 172.21.15.1:0/2074765041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09019d650 con 0x7fe090101de0 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe082ffd640 1 -- 172.21.15.1:0/2074765041 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe08c0043c0 con 0x7fe090101de0 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe082ffd640 1 -- 172.21.15.1:0/2074765041 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004a00 con 0x7fe090101de0 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.208+0000 7fe097e91640 1 -- 172.21.15.1:0/2074765041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0901a4f30 con 0x7fe090101de0 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.210+0000 7fe082ffd640 1 -- 172.21.15.1:0/2074765041 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe08c0079b0 con 0x7fe090101de0 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09a26a740 1 Processor -- start 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09a26a740 1 -- start start 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820ae9d0 0x55e7820aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820b9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 0x55e7820bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09a26a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e782015090 con 0x55e7820ae9d0 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09a26a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e781fbca70 con 0x55e7820b9bb0 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09a26a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e782011050 con 0x55e7820af2f0 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe082ffd640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe07c0a3240 0x7fe07c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09642e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820ae9d0 0x55e7820aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09642e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820ae9d0 0x55e7820aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43006/0 (socket says 172.21.15.1:43006) 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09642e640 1 -- 172.21.15.1:0/3972328778 learned_addr learned my addr 172.21.15.1:0/3972328778 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.211+0000 7fe09642e640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe07c0a3240 0x7fe07c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe082ffd640 1 -- 172.21.15.1:0/2074765041 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe08c0cfb20 con 0x7fe090101de0 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe095c2d640 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe09542c640 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 0x55e7820bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe09642e640 1 -- 172.21.15.1:0/3972328778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 msgr2=0x55e7820b9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe09642e640 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820b9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe09642e640 1 -- 172.21.15.1:0/3972328778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 msgr2=0x55e7820bbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe09642e640 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 0x55e7820bbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe09642e640 1 -- 172.21.15.1:0/3972328778 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e781fc9130 con 0x55e7820ae9d0 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe095c2d640 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820b9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe09642e640 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820ae9d0 0x55e7820aedb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe088012cd0 tx=0x7fe08800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9c865ab2002ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe09642e640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe07c0a3240 0x7fe07c0a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe08800fc30 tx=0x7fe08801a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe07bfff640 1 -- 172.21.15.1:0/3972328778 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088020070 con 0x55e7820ae9d0 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe07bfff640 1 -- 172.21.15.1:0/3972328778 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe08800aaa0 con 0x55e7820ae9d0 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.212+0000 7fe07bfff640 1 -- 172.21.15.1:0/3972328778 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0880113e0 con 0x55e7820ae9d0 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.213+0000 7fe09a26a740 1 -- 172.21.15.1:0/3972328778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820ae9d0 msgr2=0x55e7820aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.213+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820ae9d0 0x55e7820aedb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe088012cd0 tx=0x7fe08800f730 comp rx=0 tx=0).stop 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.213+0000 7fe09a26a740 1 -- 172.21.15.1:0/3972328778 shutdown_connections 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.213+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 0x55e7820bbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.213+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820b9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.213+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3972328778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820ae9d0 0x55e7820aedb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.213+0000 7fe09a26a740 1 -- 172.21.15.1:0/3972328778 >> 172.21.15.1:0/3972328778 conn(0x55e7820aaf60 msgr2=0x55e7820ab340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.213+0000 7fe09a26a740 1 -- 172.21.15.1:0/3972328778 shutdown_connections 2024-09-16T00:28:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.213+0000 7fe09a26a740 1 -- 172.21.15.1:0/3972328778 wait complete. 2024-09-16T00:28:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09a26a740 1 -- start start 2024-09-16T00:28:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09a26a740 1 Processor -- start 2024-09-16T00:28:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820ae9d0 0x55e7820b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820af2f0 0x55e7820b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 0x55e7820b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09a26a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e7820c0b90 con 0x55e7820af2f0 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09a26a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e781fcff20 con 0x55e7820b9bb0 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09a26a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e7820c0d10 con 0x55e7820ae9d0 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09a26a740 10 client.0 _ll_register_callbacks cb 0x55e781f99030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09542c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820af2f0 0x55e7820b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09542c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820af2f0 0x55e7820b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43012/0 (socket says 172.21.15.1:43012) 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.214+0000 7fe09542c640 1 -- 172.21.15.1:0/3932499519 learned_addr learned my addr 172.21.15.1:0/3932499519 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe095c2d640 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820ae9d0 0x55e7820b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09642e640 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 0x55e7820b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09542c640 1 -- 172.21.15.1:0/3932499519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820ae9d0 msgr2=0x55e7820b9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09542c640 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820ae9d0 0x55e7820b9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09542c640 1 -- 172.21.15.1:0/3932499519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 msgr2=0x55e7820b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09542c640 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 0x55e7820b6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09542c640 1 -- 172.21.15.1:0/3932499519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e781d862b0 con 0x55e7820af2f0 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09542c640 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820af2f0 0x55e7820b6850 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe084033000 tx=0x7fe0840317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e1a75b37e19cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe0797fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe0797fa640 1 -- 172.21.15.1:0/3932499519 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084040030 con 0x55e7820af2f0 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe0797fa640 1 -- 172.21.15.1:0/3932499519 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe0840319d0 con 0x55e7820af2f0 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09a26a740 1 -- 172.21.15.1:0/3932499519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe08800aaa0 con 0x55e7820af2f0 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe0797fa640 1 -- 172.21.15.1:0/3932499519 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe08404d050 con 0x55e7820af2f0 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09a26a740 10 client.15198 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe09a26a740 1 -- 172.21.15.1:0/3932499519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7820cc560 con 0x55e7820af2f0 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.215+0000 7fe0797fa640 1 -- 172.21.15.1:0/3932499519 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe08800aaa0 con 0x55e7820af2f0 2024-09-16T00:28:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe0797fa640 1 -- 172.21.15.1:0/3932499519 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fe08403a040 con 0x55e7820af2f0 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe09a26a740 10 client.15198 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe09a26a740 10 client.15198 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe09a26a740 20 client.15198 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e7820cec40 0x55e7820d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe09a26a740 4 client.15198 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe09a26a740 1 -- 172.21.15.1:0/3932499519 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e782022600 con 0x55e7820cec40 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe095c2d640 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e7820cec40 0x55e7820d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe095c2d640 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e7820cec40 0x55e7820d00b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=835c2e40a452b26e server_cookie=8c359c22766724d6 in_seq=0 out_seq=0 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.216+0000 7fe0797fa640 10 client.15198 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe0797fa640 1 -- 172.21.15.1:0/3932499519 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e782022600 con 0x55e7820cec40 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe0797fa640 10 client.15198 handle_command_reply: tid=0 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 client.15198 shutdown 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 -- 172.21.15.1:0/3932499519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820af2f0 msgr2=0x55e7820b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820af2f0 0x55e7820b6850 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe084033000 tx=0x7fe0840317e0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 -- 172.21.15.1:0/3932499519 shutdown_connections 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820b9bb0 0x55e7820b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e7820cec40 0x55e7820d00b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820af2f0 0x55e7820b6850 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3932499519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820ae9d0 0x55e7820b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 -- 172.21.15.1:0/3932499519 >> 172.21.15.1:0/3932499519 conn(0x55e7820aaf60 msgr2=0x55e7820b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe0797fa640 0 client.15198 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 -- 172.21.15.1:0/3932499519 shutdown_connections 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.217+0000 7fe09a26a740 1 -- 172.21.15.1:0/3932499519 wait complete. 2024-09-16T00:28:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.218+0000 7fe09a26a740 20 client.15198 trim_cache size 0 max 16384 2024-09-16T00:28:35.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09a26a740 1 Processor -- start 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09a26a740 1 -- start start 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 0x55e7820d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09a26a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e7820ce3f0 con 0x55e7820b9bb0 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09a26a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e7820ceac0 con 0x55e7820ae9d0 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09a26a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e7820cf190 con 0x55e7820af2f0 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09642e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09642e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43024/0 (socket says 172.21.15.1:43024) 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09642e640 1 -- 172.21.15.1:0/3809957355 learned_addr learned my addr 172.21.15.1:0/3809957355 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe09542c640 1 --2- 172.21.15.1:0/3809957355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.236+0000 7fe095c2d640 1 --2- 172.21.15.1:0/3809957355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 0x55e7820d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.237+0000 7fe09642e640 1 -- 172.21.15.1:0/3809957355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 msgr2=0x55e7820b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.237+0000 7fe09642e640 1 --2- 172.21.15.1:0/3809957355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.237+0000 7fe09642e640 1 -- 172.21.15.1:0/3809957355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 msgr2=0x55e7820d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.237+0000 7fe09642e640 1 --2- 172.21.15.1:0/3809957355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 0x55e7820d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.237+0000 7fe09642e640 1 -- 172.21.15.1:0/3809957355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7820b4da0 con 0x55e7820b9bb0 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.237+0000 7fe09642e640 1 --2- 172.21.15.1:0/3809957355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d8390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe088015090 tx=0x7fe088012440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d3be72ebad5f66 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.237+0000 7fe07b7fe640 1 -- 172.21.15.1:0/3809957355 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088020040 con 0x55e7820b9bb0 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.237+0000 7fe07b7fe640 1 -- 172.21.15.1:0/3809957355 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0880049c0 con 0x55e7820b9bb0 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.237+0000 7fe07b7fe640 1 -- 172.21.15.1:0/3809957355 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088005680 con 0x55e7820b9bb0 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 -- 172.21.15.1:0/3809957355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 msgr2=0x55e7820d8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3809957355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d8390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe088015090 tx=0x7fe088012440 comp rx=0 tx=0).stop 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 -- 172.21.15.1:0/3809957355 shutdown_connections 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3809957355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d8390 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3809957355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3809957355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 0x55e7820d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 -- 172.21.15.1:0/3809957355 >> 172.21.15.1:0/3809957355 conn(0x55e7820aaf60 msgr2=0x55e7820b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 -- 172.21.15.1:0/3809957355 shutdown_connections 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 -- 172.21.15.1:0/3809957355 wait complete. 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.238+0000 7fe09a26a740 1 -- start start 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09a26a740 1 Processor -- start 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 0x55e7820d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09a26a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09a26a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e7820cff30 con 0x55e7820b9bb0 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09a26a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e7820cf860 con 0x55e7820ae9d0 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09a26a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e7820d0600 con 0x55e7820af2f0 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09a26a740 10 client.0 _ll_register_callbacks cb 0x7fe084009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:35.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09642e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09642e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43040/0 (socket says 172.21.15.1:43040) 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09642e640 1 -- 172.21.15.1:0/3236892709 learned_addr learned my addr 172.21.15.1:0/3236892709 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe095c2d640 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 0x55e7820d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09542c640 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09642e640 1 -- 172.21.15.1:0/3236892709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 msgr2=0x55e7820d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09642e640 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09642e640 1 -- 172.21.15.1:0/3236892709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 msgr2=0x55e7820d87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09642e640 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 0x55e7820d87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.239+0000 7fe09642e640 1 -- 172.21.15.1:0/3236892709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7820d25a0 con 0x55e7820b9bb0 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.240+0000 7fe09642e640 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d2060 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe088012cd0 tx=0x7fe088012960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1975061a404309 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.240+0000 7fe0797fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:35.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.240+0000 7fe0797fa640 1 -- 172.21.15.1:0/3236892709 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08803b060 con 0x55e7820b9bb0 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.240+0000 7fe0797fa640 1 -- 172.21.15.1:0/3236892709 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe088004d70 con 0x55e7820b9bb0 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.240+0000 7fe09a26a740 1 -- 172.21.15.1:0/3236892709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0880049c0 con 0x55e7820b9bb0 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.240+0000 7fe0797fa640 1 -- 172.21.15.1:0/3236892709 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe088020050 con 0x55e7820b9bb0 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.240+0000 7fe09a26a740 10 client.15204 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.240+0000 7fe09a26a740 1 -- 172.21.15.1:0/3236892709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7820d7040 con 0x55e7820b9bb0 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.240+0000 7fe0797fa640 1 -- 172.21.15.1:0/3236892709 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0880049c0 con 0x55e7820b9bb0 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe0797fa640 1 -- 172.21.15.1:0/3236892709 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fe08802c070 con 0x55e7820b9bb0 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe09a26a740 10 client.15204 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe09a26a740 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe09a26a740 20 client.15204 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e78213d570 0x55e78213c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe09a26a740 4 client.15204 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe09a26a740 1 -- 172.21.15.1:0/3236892709 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "status"}) -- 0x55e78201cb10 con 0x55e78213d570 2024-09-16T00:28:35.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe095c2d640 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e78213d570 0x55e78213c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe0797fa640 10 client.15204 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe095c2d640 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e78213d570 0x55e78213c280 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e966185bb5fd9e7e server_cookie=a7764b48f0bad711 in_seq=0 out_seq=0 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe0797fa640 1 -- 172.21.15.1:0/3236892709 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e78201cb10 con 0x55e78213d570 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.241+0000 7fe0797fa640 10 client.15204 handle_command_reply: tid=0 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 client.15204 shutdown 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 -- 172.21.15.1:0/3236892709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 msgr2=0x55e7820d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d2060 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe088012cd0 tx=0x7fe088012960 comp rx=0 tx=0).stop 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 -- 172.21.15.1:0/3236892709 shutdown_connections 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e78213d570 0x55e78213c280 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7820b9bb0 0x55e7820d2060 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e7820af2f0 0x55e7820d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 --2- 172.21.15.1:0/3236892709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7820ae9d0 0x55e7820d87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe0797fa640 0 client.15204 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:28:35.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 -- 172.21.15.1:0/3236892709 >> 172.21.15.1:0/3236892709 conn(0x55e7820aaf60 msgr2=0x55e7820b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 -- 172.21.15.1:0/3236892709 shutdown_connections 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 1 -- 172.21.15.1:0/3236892709 wait complete. 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe09a26a740 20 client.15204 trim_cache size 0 max 16384 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24470, 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:28:35.254 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:28:35.255 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 94.648886102000006, 2024-09-16T00:28:35.255 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 24, 2024-09-16T00:28:35.255 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-09-16T00:28:35.255 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 89, 2024-09-16T00:28:35.255 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 159.10476406399999 2024-09-16T00:28:35.255 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:28:35.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe097e91640 1 -- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe07c0a3240 msgr2=0x7fe07c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:35.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe097e91640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe07c0a3240 0x7fe07c0a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe08800fc30 tx=0x7fe08801a000 comp rx=0 tx=0).stop 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe097e91640 1 -- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090101de0 msgr2=0x7fe0901983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.242+0000 7fe097e91640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090101de0 0x7fe0901983b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00c890 tx=0x7fe08c00cd50 comp rx=0 tx=0).stop 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.243+0000 7fe097e91640 1 -- 172.21.15.1:0/2074765041 shutdown_connections 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.243+0000 7fe097e91640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe090106f10 0x7fe09019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.243+0000 7fe097e91640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe07c0a3240 0x7fe07c0a5700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.243+0000 7fe097e91640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe090102790 0x7fe0901988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.243+0000 7fe097e91640 1 --2- 172.21.15.1:0/2074765041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe090101de0 0x7fe0901983b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.243+0000 7fe097e91640 1 -- 172.21.15.1:0/2074765041 >> 172.21.15.1:0/2074765041 conn(0x7fe0900fd8c0 msgr2=0x7fe0900fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.243+0000 7fe097e91640 1 -- 172.21.15.1:0/2074765041 shutdown_connections 2024-09-16T00:28:35.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:35.243+0000 7fe097e91640 1 -- 172.21.15.1:0/2074765041 wait complete. 2024-09-16T00:28:35.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:35 smithi001 ceph-mon[30934]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:28:35.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:35 smithi153 ceph-mon[35538]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:28:35.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:28:36.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:35 smithi019 ceph-mon[35186]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:28:36.230 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:36.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.741+0000 7f5ca4768640 1 -- 172.21.15.1:0/4009427707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c106f10 msgr2=0x7f5c9c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.741+0000 7f5ca4768640 1 --2- 172.21.15.1:0/4009427707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c106f10 0x7f5c9c10ea90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009ca0 tx=0x7f5c9402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.741+0000 7f5ca4768640 1 -- 172.21.15.1:0/4009427707 shutdown_connections 2024-09-16T00:28:36.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.741+0000 7f5ca4768640 1 --2- 172.21.15.1:0/4009427707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c106f10 0x7f5c9c10ea90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.741+0000 7f5ca4768640 1 --2- 172.21.15.1:0/4009427707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c9c102790 0x7f5c9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.741+0000 7f5ca4768640 1 --2- 172.21.15.1:0/4009427707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9c101de0 0x7f5c9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.741+0000 7f5ca4768640 1 -- 172.21.15.1:0/4009427707 >> 172.21.15.1:0/4009427707 conn(0x7f5c9c0fd8c0 msgr2=0x7f5c9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.741+0000 7f5ca4768640 1 -- 172.21.15.1:0/4009427707 shutdown_connections 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.741+0000 7f5ca4768640 1 -- 172.21.15.1:0/4009427707 wait complete. 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca4768640 1 Processor -- start 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca4768640 1 -- start start 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca4768640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9c101de0 0x7f5c9c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca4768640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c102790 0x7f5c9c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca4768640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c9c106f10 0x7f5c9c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca4768640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c111f70 con 0x7f5c9c102790 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca4768640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c111df0 con 0x7f5c9c101de0 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca4768640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1120f0 con 0x7f5c9c106f10 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca1d03640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c102790 0x7f5c9c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca1d03640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c102790 0x7f5c9c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43068/0 (socket says 172.21.15.1:43068) 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca1d03640 1 -- 172.21.15.1:0/2311102627 learned_addr learned my addr 172.21.15.1:0/2311102627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca2504640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9c101de0 0x7f5c9c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c9c106f10 0x7f5c9c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca1d03640 1 -- 172.21.15.1:0/2311102627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c9c106f10 msgr2=0x7f5c9c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c9c106f10 0x7f5c9c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca1d03640 1 -- 172.21.15.1:0/2311102627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9c101de0 msgr2=0x7f5c9c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9c101de0 0x7f5c9c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.742+0000 7f5ca1d03640 1 -- 172.21.15.1:0/2311102627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c1a6870 con 0x7f5c9c102790 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.743+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c9c106f10 0x7f5c9c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.743+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c102790 0x7f5c9c074a30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d770 tx=0x7f5c8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.743+0000 7f5c937fe640 1 -- 172.21.15.1:0/2311102627 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c017070 con 0x7f5c9c102790 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.743+0000 7f5c937fe640 1 -- 172.21.15.1:0/2311102627 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8c005500 con 0x7f5c9c102790 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.743+0000 7f5ca4768640 1 -- 172.21.15.1:0/2311102627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c1a6b60 con 0x7f5c9c102790 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.743+0000 7f5c937fe640 1 -- 172.21.15.1:0/2311102627 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c004e00 con 0x7f5c9c102790 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.743+0000 7f5ca4768640 1 -- 172.21.15.1:0/2311102627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c1a70a0 con 0x7f5c9c102790 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.745+0000 7f5c937fe640 1 -- 172.21.15.1:0/2311102627 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f5c8c004fa0 con 0x7f5c9c102790 2024-09-16T00:28:36.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.745+0000 7f5ca6b41740 1 Processor -- start 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca6b41740 1 -- start start 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fb9d0 0x55f86f8fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f906670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 0x55f86f908fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f86f862090 con 0x55f86f8fc2f0 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f86f809a70 con 0x55f86f906bb0 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f86f85e050 con 0x55f86f8fb9d0 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2504640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f906670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2504640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f906670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43082/0 (socket says 172.21.15.1:43082) 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2504640 1 -- 172.21.15.1:0/3949519644 learned_addr learned my addr 172.21.15.1:0/3949519644 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fb9d0 0x55f86f8fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5c937fe640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c880a32c0 0x7f5c880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 0x55f86f908fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2504640 1 -- 172.21.15.1:0/3949519644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fb9d0 msgr2=0x55f86f8fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2504640 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fb9d0 0x55f86f8fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2504640 1 -- 172.21.15.1:0/3949519644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 msgr2=0x55f86f908fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2504640 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 0x55f86f908fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2504640 1 -- 172.21.15.1:0/3949519644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f86f816130 con 0x55f86f8fc2f0 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.746+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fb9d0 0x55f86f8fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.747+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c880a32c0 0x7f5c880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.747+0000 7f5c937fe640 1 -- 172.21.15.1:0/2311102627 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5c8c0cf6d0 con 0x7f5c9c102790 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.747+0000 7f5ca2504640 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f906670 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800bd10 tx=0x7f5c980107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd63923a529f9794 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.747+0000 7f5c90ff9640 1 -- 172.21.15.1:0/3949519644 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98004030 con 0x55f86f8fc2f0 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.747+0000 7f5c90ff9640 1 -- 172.21.15.1:0/3949519644 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c980041d0 con 0x55f86f8fc2f0 2024-09-16T00:28:36.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.747+0000 7f5c90ff9640 1 -- 172.21.15.1:0/3949519644 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98011820 con 0x55f86f8fc2f0 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3949519644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 msgr2=0x55f86f906670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c880a32c0 0x7f5c880a5780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5c94005e30 tx=0x7f5c940095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f906670 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800bd10 tx=0x7f5c980107a0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3949519644 shutdown_connections 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 0x55f86f908fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f906670 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3949519644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fb9d0 0x55f86f8fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3949519644 >> 172.21.15.1:0/3949519644 conn(0x55f86f8f7f60 msgr2=0x55f86f8f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3949519644 shutdown_connections 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3949519644 wait complete. 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 -- start start 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.748+0000 7f5ca6b41740 1 Processor -- start 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fb9d0 0x55f86f906400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f903750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f903c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f86f909780 con 0x55f86f8fc2f0 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f86f81cf20 con 0x55f86f8fb9d0 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f86f909900 con 0x55f86f906bb0 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca6b41740 10 client.0 _ll_register_callbacks cb 0x55f86f7e6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca1d03640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f903750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca1d03640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f903750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43096/0 (socket says 172.21.15.1:43096) 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca1d03640 1 -- 172.21.15.1:0/1594334870 learned_addr learned my addr 172.21.15.1:0/1594334870 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca2504640 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fb9d0 0x55f86f906400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f903c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca1d03640 1 -- 172.21.15.1:0/1594334870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 msgr2=0x55f86f903c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f903c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca1d03640 1 -- 172.21.15.1:0/1594334870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fb9d0 msgr2=0x55f86f906400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fb9d0 0x55f86f906400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca1d03640 1 -- 172.21.15.1:0/1594334870 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f86f5d32b0 con 0x55f86f8fc2f0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f903c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f903750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0ccd70 tx=0x7f5c8c09b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4d3edbb580e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5c85ffb640 1 -- 172.21.15.1:0/1594334870 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c020070 con 0x55f86f8fc2f0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5c85ffb640 1 -- 172.21.15.1:0/1594334870 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5c8c099b90 con 0x55f86f8fc2f0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1594334870 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c980041d0 con 0x55f86f8fc2f0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.749+0000 7f5c85ffb640 1 -- 172.21.15.1:0/1594334870 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c8c09bf90 con 0x55f86f8fc2f0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5ca6b41740 10 client.15216 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1594334870 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f86f9150d0 con 0x55f86f8fc2f0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5c85ffb640 1 -- 172.21.15.1:0/1594334870 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c980041d0 con 0x55f86f8fc2f0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5c85ffb640 1 -- 172.21.15.1:0/1594334870 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f5c8c097f00 con 0x55f86f8fc2f0 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5ca6b41740 10 client.15216 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:36.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5ca6b41740 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5ca6b41740 20 client.15216 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f86f9176d0 0x55f86f918b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5ca6b41740 4 client.15216 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1594334870 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f86f86f600 con 0x55f86f9176d0 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.750+0000 7f5ca2504640 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f86f9176d0 0x55f86f918b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.751+0000 7f5ca2504640 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f86f9176d0 0x55f86f918b40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc2b40e986bc14cb server_cookie=e8125787f81ba57f in_seq=0 out_seq=0 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.751+0000 7f5c85ffb640 10 client.15216 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.751+0000 7f5c85ffb640 1 -- 172.21.15.1:0/1594334870 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f86f86f600 con 0x55f86f9176d0 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.751+0000 7f5c85ffb640 10 client.15216 handle_command_reply: tid=0 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.751+0000 7f5ca6b41740 1 client.15216 shutdown 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1594334870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 msgr2=0x55f86f903750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f903750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0ccd70 tx=0x7f5c8c09b1f0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1594334870 shutdown_connections 2024-09-16T00:28:36.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f86f9176d0 0x55f86f918b40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f903c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fc2f0 0x55f86f903750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5c85ffb640 0 client.15216 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1594334870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fb9d0 0x55f86f906400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1594334870 >> 172.21.15.1:0/1594334870 conn(0x55f86f8f7f60 msgr2=0x55f86f8fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1594334870 shutdown_connections 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1594334870 wait complete. 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.752+0000 7f5ca6b41740 20 client.15216 trim_cache size 0 max 16384 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca6b41740 1 Processor -- start 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca6b41740 1 -- start start 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f918b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fc2f0 0x55f86f903010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f902020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f86f916e80 con 0x55f86f8fb9d0 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f86f917550 con 0x55f86f8fc2f0 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f86f917c20 con 0x55f86f906bb0 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca2504640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f918b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca2504640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f918b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43102/0 (socket says 172.21.15.1:43102) 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca2504640 1 -- 172.21.15.1:0/1764088821 learned_addr learned my addr 172.21.15.1:0/1764088821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.770+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fc2f0 0x55f86f903010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f902020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5ca2504640 1 -- 172.21.15.1:0/1764088821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 msgr2=0x55f86f902020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5ca2504640 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f902020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5ca2504640 1 -- 172.21.15.1:0/1764088821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fc2f0 msgr2=0x55f86f903010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5ca2504640 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fc2f0 0x55f86f903010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5ca2504640 1 -- 172.21.15.1:0/1764088821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f86f906770 con 0x55f86f8fb9d0 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f902020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fc2f0 0x55f86f903010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:36.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5ca2504640 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f918b40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b3d0 tx=0x7f5c98011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e737effa79cbcc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5c87fff640 1 -- 172.21.15.1:0/1764088821 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c980035f0 con 0x55f86f8fb9d0 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5c87fff640 1 -- 172.21.15.1:0/1764088821 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c98005040 con 0x55f86f8fb9d0 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.771+0000 7f5c87fff640 1 -- 172.21.15.1:0/1764088821 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98011710 con 0x55f86f8fb9d0 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1764088821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 msgr2=0x55f86f918b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f918b40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b3d0 tx=0x7f5c98011440 comp rx=0 tx=0).stop 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1764088821 shutdown_connections 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f906bb0 0x55f86f902020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f8fc2f0 0x55f86f903010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/1764088821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f918b40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1764088821 >> 172.21.15.1:0/1764088821 conn(0x55f86f8f7f60 msgr2=0x55f86f8f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1764088821 shutdown_connections 2024-09-16T00:28:36.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 -- 172.21.15.1:0/1764088821 wait complete. 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 -- start start 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 Processor -- start 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f921120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fc2f0 0x55f86f921660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 0x55f86f91c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f86f9189c0 con 0x55f86f8fb9d0 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f86f9182f0 con 0x55f86f906bb0 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.772+0000 7f5ca6b41740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f86f919090 con 0x55f86f8fc2f0 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca6b41740 10 client.0 _ll_register_callbacks cb 0x7f5c8c097c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2504640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f921120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2504640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f921120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43116/0 (socket says 172.21.15.1:43116) 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2504640 1 -- 172.21.15.1:0/3137797149 learned_addr learned my addr 172.21.15.1:0/3137797149 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:36.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2d05640 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 0x55f86f91c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fc2f0 0x55f86f921660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2504640 1 -- 172.21.15.1:0/3137797149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fc2f0 msgr2=0x55f86f921660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2504640 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fc2f0 0x55f86f921660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2504640 1 -- 172.21.15.1:0/3137797149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 msgr2=0x55f86f91c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2504640 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 0x55f86f91c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2504640 1 -- 172.21.15.1:0/3137797149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f86f91cd80 con 0x55f86f8fb9d0 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca2504640 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f921120 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5c98010ca0 tx=0x7f5c98003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b253a19457de7689 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5c85ffb640 1 -- 172.21.15.1:0/3137797149 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9803f060 con 0x55f86f8fb9d0 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5c85ffb640 1 -- 172.21.15.1:0/3137797149 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5c98005400 con 0x55f86f8fb9d0 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3137797149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c98005040 con 0x55f86f8fb9d0 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5c85ffb640 1 -- 172.21.15.1:0/3137797149 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c98005600 con 0x55f86f8fb9d0 2024-09-16T00:28:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.773+0000 7f5ca6b41740 10 client.15222 fetch_fsmap learned FSMap version 24 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3137797149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f86f981930 con 0x55f86f8fb9d0 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5c85ffb640 1 -- 172.21.15.1:0/3137797149 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c98005040 con 0x55f86f8fb9d0 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5c85ffb640 1 -- 172.21.15.1:0/3137797149 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f5c98011ab0 con 0x55f86f8fb9d0 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5ca6b41740 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5ca6b41740 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5ca6b41740 20 client.15222 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f86f985d90 0x55f86f984aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5ca6b41740 4 client.15222 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3137797149 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "status"}) -- 0x55f86f869b10 con 0x55f86f985d90 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.774+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f86f985d90 0x55f86f984aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5c85ffb640 10 client.15222 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca1d03640 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f86f985d90 0x55f86f984aa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb3b57915d0653d server_cookie=1fe08e371fafe4e5 in_seq=0 out_seq=0 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5c85ffb640 1 -- 172.21.15.1:0/3137797149 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55f86f869b10 con 0x55f86f985d90 2024-09-16T00:28:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5c85ffb640 10 client.15222 handle_command_reply: tid=0 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 client.15222 shutdown 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3137797149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 msgr2=0x55f86f921120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f921120 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5c98010ca0 tx=0x7f5c98003f60 comp rx=0 tx=0).stop 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3137797149 shutdown_connections 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f86f985d90 0x55f86f984aa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f86f906bb0 0x55f86f91c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f86f8fc2f0 0x55f86f921660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5c85ffb640 0 client.15222 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 --2- 172.21.15.1:0/3137797149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f86f8fb9d0 0x55f86f921120 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3137797149 >> 172.21.15.1:0/3137797149 conn(0x55f86f8f7f60 msgr2=0x55f86f901bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3137797149 shutdown_connections 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 1 -- 172.21.15.1:0/3137797149 wait complete. 2024-09-16T00:28:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.775+0000 7f5ca6b41740 20 client.15222 trim_cache size 0 max 16384 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24506, 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 137.504873221, 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 24, 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-09-16T00:28:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 89, 2024-09-16T00:28:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 148.849227587 2024-09-16T00:28:36.795 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:28:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 -- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c880a32c0 msgr2=0x7f5c880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c880a32c0 0x7f5c880a5780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5c94005e30 tx=0x7f5c940095c0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 -- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c102790 msgr2=0x7f5c9c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c102790 0x7f5c9c074a30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d770 tx=0x7f5c8c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 -- 172.21.15.1:0/2311102627 shutdown_connections 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5c880a32c0 0x7f5c880a5780 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c9c106f10 0x7f5c9c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c9c102790 0x7f5c9c074a30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 --2- 172.21.15.1:0/2311102627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9c101de0 0x7f5c9c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 -- 172.21.15.1:0/2311102627 >> 172.21.15.1:0/2311102627 conn(0x7f5c9c0fd8c0 msgr2=0x7f5c9c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 -- 172.21.15.1:0/2311102627 shutdown_connections 2024-09-16T00:28:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:36.776+0000 7f5ca4768640 1 -- 172.21.15.1:0/2311102627 wait complete. 2024-09-16T00:28:37.601 INFO:teuthology.run_tasks:Running task sequential... 2024-09-16T00:28:37.611 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-16T00:28:37.611 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-16T00:28:37.611 INFO:teuthology.task.print:Enabling mgr modules 2024-09-16T00:28:37.611 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T00:28:37.657 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T00:28:37.657 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-16T00:28:37.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:37.658 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-16T00:28:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-16T00:28:37.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:37.677 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:28:37.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:37.747 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-09-16T00:28:37.836 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:37 smithi001 ceph-mon[30934]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:37.847 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-16T00:28:37.847 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:28:37.847 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:28:37.847 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:28:37.924 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:37 smithi019 ceph-mon[35186]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:37.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:37 smithi153 ceph-mon[35538]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:38.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.620+0000 7f4912ce1640 1 -- 172.21.15.1:0/895008362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102570 msgr2=0x7f490c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.620+0000 7f4912ce1640 1 --2- 172.21.15.1:0/895008362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102570 0x7f490c1029b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f402a480 comp rx=0 tx=0).stop 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.620+0000 7f4912ce1640 1 -- 172.21.15.1:0/895008362 shutdown_connections 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.620+0000 7f4912ce1640 1 --2- 172.21.15.1:0/895008362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c102ef0 0x7f490c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.620+0000 7f4912ce1640 1 --2- 172.21.15.1:0/895008362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102570 0x7f490c1029b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.620+0000 7f4912ce1640 1 --2- 172.21.15.1:0/895008362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c100e20 0x7f490c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.620+0000 7f4912ce1640 1 -- 172.21.15.1:0/895008362 >> 172.21.15.1:0/895008362 conn(0x7f490c0fc7d0 msgr2=0x7f490c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.620+0000 7f4912ce1640 1 -- 172.21.15.1:0/895008362 shutdown_connections 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.620+0000 7f4912ce1640 1 -- 172.21.15.1:0/895008362 wait complete. 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f4912ce1640 1 Processor -- start 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f4912ce1640 1 -- start start 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f4912ce1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c100e20 0x7f490c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f4912ce1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c102570 0x7f490c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f4912ce1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102ef0 0x7f490c1a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f4912ce1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f490c110f80 con 0x7f490c102ef0 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f4912ce1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f490c110e00 con 0x7f490c102570 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f4912ce1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f490c111100 con 0x7f490c100e20 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f491127e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102ef0 0x7f490c1a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f491127e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102ef0 0x7f490c1a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43130/0 (socket says 172.21.15.1:43130) 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f491127e640 1 -- 172.21.15.1:0/4253421243 learned_addr learned my addr 172.21.15.1:0/4253421243 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f490bfff640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c102570 0x7f490c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f4910a7d640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c100e20 0x7f490c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f491127e640 1 -- 172.21.15.1:0/4253421243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c100e20 msgr2=0x7f490c199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f491127e640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c100e20 0x7f490c199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f491127e640 1 -- 172.21.15.1:0/4253421243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c102570 msgr2=0x7f490c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.621+0000 7f491127e640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c102570 0x7f490c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.622+0000 7f491127e640 1 -- 172.21.15.1:0/4253421243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c1a4950 con 0x7f490c102ef0 2024-09-16T00:28:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.622+0000 7f491127e640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102ef0 0x7f490c1a4410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f490400eb10 tx=0x7f490400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.622+0000 7f4909ffb640 1 -- 172.21.15.1:0/4253421243 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904004b20 con 0x7f490c102ef0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.622+0000 7f4909ffb640 1 -- 172.21.15.1:0/4253421243 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4904004cc0 con 0x7f490c102ef0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.622+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c1a4b20 con 0x7f490c102ef0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.622+0000 7f4909ffb640 1 -- 172.21.15.1:0/4253421243 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904010760 con 0x7f490c102ef0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.622+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a4fe0 con 0x7f490c102ef0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.623+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c101200 con 0x7f490c102ef0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.624+0000 7f4909ffb640 1 -- 172.21.15.1:0/4253421243 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4904005020 con 0x7f490c102ef0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.625+0000 7f4909ffb640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48f80a3240 0x7f48f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.625+0000 7f4910a7d640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48f80a3240 0x7f48f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.625+0000 7f4909ffb640 1 -- 172.21.15.1:0/4253421243 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f49040d0310 con 0x7f490c102ef0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.626+0000 7f4910a7d640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48f80a3240 0x7f48f80a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0058c0 tx=0x7f48fc005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.630+0000 7f4909ffb640 1 -- 172.21.15.1:0/4253421243 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f490409cbd0 con 0x7f490c102ef0 2024-09-16T00:28:38.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.842+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f490c109090 con 0x7f490c102ef0 2024-09-16T00:28:38.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.843+0000 7f4909ffb640 1 -- 172.21.15.1:0/4253421243 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f490409c4d0 con 0x7f490c102ef0 2024-09-16T00:28:38.844 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T00:28:38.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48f80a3240 msgr2=0x7f48f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:38.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48f80a3240 0x7f48f80a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0058c0 tx=0x7f48fc005c60 comp rx=0 tx=0).stop 2024-09-16T00:28:38.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102ef0 msgr2=0x7f490c1a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:38.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102ef0 0x7f490c1a4410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f490400eb10 tx=0x7f490400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 shutdown_connections 2024-09-16T00:28:38.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f490c102ef0 0x7f490c1a4410 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c102570 0x7f490c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f48f80a3240 0x7f48f80a5700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 --2- 172.21.15.1:0/4253421243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c100e20 0x7f490c199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 >> 172.21.15.1:0/4253421243 conn(0x7f490c0fc7d0 msgr2=0x7f490c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 shutdown_connections 2024-09-16T00:28:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:38.846+0000 7f4912ce1640 1 -- 172.21.15.1:0/4253421243 wait complete. 2024-09-16T00:28:39.527 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T00:28:39.527 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:28:39.527 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-09-16T00:28:39.527 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T00:28:39.527 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:28:39.527 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T00:28:39.527 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:28:39.528 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:28:39.528 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:28:39.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:39.528 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-09-16T00:28:39.547 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:28:39.547 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:28:39.547 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:28:39.547 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:28:39.547 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-09-16T00:28:39.547 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42180sec preferred_lft 42180sec 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-09-16T00:28:39.548 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:28:39.549 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:28:39.549 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:28:39.549 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-09-16T00:28:39.549 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:28:39.549 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:28:39.549 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:28:39.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:28:39.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-09-16T00:28:39.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-09-16T00:28:39.550 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:28:39.550 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:28:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:28:39.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:39.696 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:28:39.721 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-09-16T00:28:39.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:39.723 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-09-16T00:28:39.780 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-09-16T00:28:39.780 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:28:39.781 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T00:28:39.781 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T00:28:39.781 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:28:39.781 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:28:39.781 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:28:39.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T00:28:39.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T00:28:39.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T00:28:39.781 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:28:39.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:28:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:39 smithi001 ceph-mon[30934]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:28:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:39 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4253421243' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:28:39.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:28:39.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:39.911 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:39.928 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-09-16T00:28:39.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:39 smithi153 ceph-mon[35538]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:28:39.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:39 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4253421243' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:28:39.983 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:28:39.983 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:28:39.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:28:39.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T00:28:39.983 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:28:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:28:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:28:40.103 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T00:28:40.103 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:28:40.104 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:28:40.104 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T00:28:40.104 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:28:40.104 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T00:28:40.104 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T00:28:40.104 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T00:28:40.104 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:28:40.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:39 smithi019 ceph-mon[35186]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:28:40.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:39 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4253421243' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:28:40.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:28:40.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:28:40.225 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:28:40.225 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-16T00:28:40.225 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-09-16T00:28:40.226 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:28:40.226 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-16T00:28:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:28:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:28:40.305 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:28:40.305 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T00:28:40.321 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:28:40.321 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:28:40.321 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:28:40.377 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-16T00:28:40.377 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-16T00:28:40.377 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/4c1f3cd4-b1c5-443e-970a-b5a707971cba 2024-09-16T00:28:40.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:40.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/4c1f3cd4-b1c5-443e-970a-b5a707971cba /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-16T00:28:40.462 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-16T00:28:40.462 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-16T00:28:40.597 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-16T00:28:40.598 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-16T00:28:40.598 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:3300,172.21.15.19:3300,172.21.15.153:3300" 2024-09-16T00:28:40.598 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: trying mount with new device syntax: 0@1226ed86-73c1-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/4c1f3cd4-b1c5-443e-970a-b5a707971cba 2024-09-16T00:28:40.598 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.1:3300/172.21.15.19:3300/172.21.15.153:3300" will pass to kernel 2024-09-16T00:28:40.598 INFO:teuthology.orchestra.run.smithi001.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T00:28:40.599 INFO:teuthology.orchestra.run.smithi001.stdout: You just mounted a file system that supports labels which does not 2024-09-16T00:28:40.599 INFO:teuthology.orchestra.run.smithi001.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T00:28:40.599 INFO:teuthology.orchestra.run.smithi001.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T00:28:40.599 INFO:teuthology.orchestra.run.smithi001.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T00:28:40.600 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T00:28:40.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:28:40.601 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:28:40.627 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:28:40.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:28:40.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:28:40.725 INFO:teuthology.orchestra.run.smithi001.stdout:client24953 2024-09-16T00:28:40.726 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:28:40.726 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:28:40.742 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:28:40.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:28:40.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:28:40.840 INFO:teuthology.orchestra.run.smithi001.stdout:client24953 2024-09-16T00:28:40.841 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:28:40.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:28:40.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:28:40.898 INFO:teuthology.orchestra.run.smithi001.stdout:client24953 2024-09-16T00:28:40.898 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1226ed86-73c1-11ef-bceb-c7b262605968.client24953/status') 2024-09-16T00:28:40.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:28:40.958 INFO:teuthology.orchestra.run.smithi001.stdout:instance: client.24953 (3)172.21.15.1:0/919228023 2024-09-16T00:28:40.958 INFO:teuthology.orchestra.run.smithi001.stdout:blocklisted: false 2024-09-16T00:28:40.958 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records in 2024-09-16T00:28:40.958 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records out 2024-09-16T00:28:40.958 INFO:teuthology.orchestra.run.smithi001.stderr:69 bytes copied, 2.7148e-05 s, 2.5 MB/s 2024-09-16T00:28:40.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:28:40.965 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T00:28:40.976 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T00:28:40.976 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-16T00:28:40.976 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-16T00:28:41.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.079+0000 7f7029c65640 1 -- 172.21.15.1:0/3296367021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7024100a40 msgr2=0x7f7024100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.079+0000 7f7029c65640 1 --2- 172.21.15.1:0/3296367021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7024100a40 0x7f7024100e40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7014009900 tx=0x7f701402a410 comp rx=0 tx=0).stop 2024-09-16T00:28:41.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.079+0000 7f7029c65640 1 -- 172.21.15.1:0/3296367021 shutdown_connections 2024-09-16T00:28:41.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.079+0000 7f7029c65640 1 --2- 172.21.15.1:0/3296367021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70241025f0 0x7f7024108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.079+0000 7f7029c65640 1 --2- 172.21.15.1:0/3296367021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024101c30 0x7f70241020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.079+0000 7f7029c65640 1 --2- 172.21.15.1:0/3296367021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7024100a40 0x7f7024100e40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.079+0000 7f7029c65640 1 -- 172.21.15.1:0/3296367021 >> 172.21.15.1:0/3296367021 conn(0x7f70240fc1c0 msgr2=0x7f70240fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.079+0000 7f7029c65640 1 -- 172.21.15.1:0/3296367021 shutdown_connections 2024-09-16T00:28:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.079+0000 7f7029c65640 1 -- 172.21.15.1:0/3296367021 wait complete. 2024-09-16T00:28:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.080+0000 7f7029c65640 1 Processor -- start 2024-09-16T00:28:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.080+0000 7f7029c65640 1 -- start start 2024-09-16T00:28:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7029c65640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7024100a40 0x7f70241049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7029c65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024101c30 0x7f7024103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7029c65640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70241025f0 0x7f7024103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7029c65640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f702410c6a0 con 0x7f70241025f0 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7029c65640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f702410c520 con 0x7f7024101c30 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7029c65640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f702410c820 con 0x7f7024100a40 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7023fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70241025f0 0x7f7024103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7023fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70241025f0 0x7f7024103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60152/0 (socket says 172.21.15.1:60152) 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7023fff640 1 -- 172.21.15.1:0/3048854785 learned_addr learned my addr 172.21.15.1:0/3048854785 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f70237fe640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7024100a40 0x7f70241049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7022ffd640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024101c30 0x7f7024103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7023fff640 1 -- 172.21.15.1:0/3048854785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7024100a40 msgr2=0x7f70241049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7023fff640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7024100a40 0x7f70241049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7023fff640 1 -- 172.21.15.1:0/3048854785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024101c30 msgr2=0x7f7024103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7023fff640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024101c30 0x7f7024103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7023fff640 1 -- 172.21.15.1:0/3048854785 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7024103de0 con 0x7f70241025f0 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7022ffd640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024101c30 0x7f7024103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7023fff640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70241025f0 0x7f7024103550 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f701c00cce0 tx=0x7f701c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7020ff9640 1 -- 172.21.15.1:0/3048854785 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c00f450 con 0x7f70241025f0 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7020ff9640 1 -- 172.21.15.1:0/3048854785 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701c00fad0 con 0x7f70241025f0 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.081+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70241a1f60 con 0x7f70241025f0 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.082+0000 7f7020ff9640 1 -- 172.21.15.1:0/3048854785 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c018c10 con 0x7f70241025f0 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.082+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70241a2370 con 0x7f70241025f0 2024-09-16T00:28:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.082+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70241a2690 con 0x7f70241025f0 2024-09-16T00:28:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.084+0000 7f7020ff9640 1 -- 172.21.15.1:0/3048854785 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f701c00f5f0 con 0x7f70241025f0 2024-09-16T00:28:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.084+0000 7f7020ff9640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f700c0a3240 0x7f700c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.084+0000 7f70237fe640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f700c0a3240 0x7f700c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.084+0000 7f7020ff9640 1 -- 172.21.15.1:0/3048854785 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f701c0d0460 con 0x7f70241025f0 2024-09-16T00:28:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.085+0000 7f70237fe640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f700c0a3240 0x7f700c0a5700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7014006010 tx=0x7f7014000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.089+0000 7f7020ff9640 1 -- 172.21.15.1:0/3048854785 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f701c014030 con 0x7f70241025f0 2024-09-16T00:28:41.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.304+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f70241a29f0 con 0x7f70241025f0 2024-09-16T00:28:41.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.667+0000 7f7020ff9640 1 -- 172.21.15.1:0/3048854785 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f701c09cd80 con 0x7f70241025f0 2024-09-16T00:28:41.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f700c0a3240 msgr2=0x7f700c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f700c0a3240 0x7f700c0a5700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7014006010 tx=0x7f7014000bc0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70241025f0 msgr2=0x7f7024103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70241025f0 0x7f7024103550 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f701c00cce0 tx=0x7f701c00a620 comp rx=0 tx=0).stop 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 shutdown_connections 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70241025f0 0x7f7024103550 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024101c30 0x7f7024103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f700c0a3240 0x7f700c0a5700 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 --2- 172.21.15.1:0/3048854785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7024100a40 0x7f70241049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 >> 172.21.15.1:0/3048854785 conn(0x7f70240fc1c0 msgr2=0x7f7024069a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 shutdown_connections 2024-09-16T00:28:41.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.670+0000 7f7029c65640 1 -- 172.21.15.1:0/3048854785 wait complete. 2024-09-16T00:28:41.685 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-16T00:28:41.696 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-16T00:28:41.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:28:41.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.851+0000 7f801fffd640 1 -- 172.21.15.1:0/2054246826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018100a50 msgr2=0x7f8018100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.851+0000 7f801fffd640 1 --2- 172.21.15.1:0/2054246826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018100a50 0x7f8018100e50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8014009900 tx=0x7f801402a410 comp rx=0 tx=0).stop 2024-09-16T00:28:41.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.851+0000 7f801fffd640 1 -- 172.21.15.1:0/2054246826 shutdown_connections 2024-09-16T00:28:41.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.851+0000 7f801fffd640 1 --2- 172.21.15.1:0/2054246826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8018102600 0x7f8018108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.851+0000 7f801fffd640 1 --2- 172.21.15.1:0/2054246826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8018101c40 0x7f80181020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.851+0000 7f801fffd640 1 --2- 172.21.15.1:0/2054246826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018100a50 0x7f8018100e50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.851+0000 7f801fffd640 1 -- 172.21.15.1:0/2054246826 >> 172.21.15.1:0/2054246826 conn(0x7f80180fc190 msgr2=0x7f80180fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:41.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.851+0000 7f801fffd640 1 -- 172.21.15.1:0/2054246826 shutdown_connections 2024-09-16T00:28:41.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.851+0000 7f801fffd640 1 -- 172.21.15.1:0/2054246826 wait complete. 2024-09-16T00:28:41.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801fffd640 1 Processor -- start 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801fffd640 1 -- start start 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801fffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8018100a50 0x7f8018197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801fffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8018101c40 0x7f8018198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801fffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018102600 0x7f801819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801fffd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f801810c6d0 con 0x7f8018102600 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801fffd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f801810c550 con 0x7f8018101c40 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801fffd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f801810c850 con 0x7f8018100a50 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801e59a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018102600 0x7f801819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801e59a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018102600 0x7f801819f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60178/0 (socket says 172.21.15.1:60178) 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.852+0000 7f801e59a640 1 -- 172.21.15.1:0/2411763083 learned_addr learned my addr 172.21.15.1:0/2411763083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801d598640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8018101c40 0x7f8018198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801dd99640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8018100a50 0x7f8018197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801e59a640 1 -- 172.21.15.1:0/2411763083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8018100a50 msgr2=0x7f8018197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801e59a640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8018100a50 0x7f8018197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801e59a640 1 -- 172.21.15.1:0/2411763083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8018101c40 msgr2=0x7f8018198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801e59a640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8018101c40 0x7f8018198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801e59a640 1 -- 172.21.15.1:0/2411763083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801819fb30 con 0x7f8018102600 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801d598640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8018101c40 0x7f8018198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801e59a640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018102600 0x7f801819f4c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f801000eaa0 tx=0x7f801000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f800effd640 1 -- 172.21.15.1:0/2411763083 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801000cdb0 con 0x7f8018102600 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f800effd640 1 -- 172.21.15.1:0/2411763083 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8010005020 con 0x7f8018102600 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801819fe20 con 0x7f8018102600 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f800effd640 1 -- 172.21.15.1:0/2411763083 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100116b0 con 0x7f8018102600 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.853+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80181a02e0 con 0x7f8018102600 2024-09-16T00:28:41.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.854+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80181a05a0 con 0x7f8018102600 2024-09-16T00:28:41.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.855+0000 7f800effd640 1 -- 172.21.15.1:0/2411763083 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8010005560 con 0x7f8018102600 2024-09-16T00:28:41.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.856+0000 7f800effd640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80040a3250 0x7f80040a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.856+0000 7f800effd640 1 -- 172.21.15.1:0/2411763083 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f80100d0370 con 0x7f8018102600 2024-09-16T00:28:41.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.856+0000 7f801dd99640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80040a3250 0x7f80040a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.857+0000 7f801dd99640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80040a3250 0x7f80040a5710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8014000c00 tx=0x7f80140315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:41.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:41.859+0000 7f800effd640 1 -- 172.21.15.1:0/2411763083 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8010016030 con 0x7f8018102600 2024-09-16T00:28:41.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:41 smithi001 ceph-mon[30934]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:41.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:41 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3048854785' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T00:28:41.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:41 smithi153 ceph-mon[35538]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:41.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:41 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3048854785' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T00:28:42.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.036+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80181a0810 con 0x7f8018102600 2024-09-16T00:28:42.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.037+0000 7f800effd640 1 -- 172.21.15.1:0/2411763083 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24902 (secure 0 0 0) 0x7f801009cc90 con 0x7f8018102600 2024-09-16T00:28:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:42.039 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:27:26.243912+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80040a3250 msgr2=0x7f80040a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80040a3250 0x7f80040a5710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8014000c00 tx=0x7f80140315d0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018102600 msgr2=0x7f801819f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018102600 0x7f801819f4c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f801000eaa0 tx=0x7f801000ef60 comp rx=0 tx=0).stop 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 shutdown_connections 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8018102600 0x7f801819f4c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8018101c40 0x7f8018198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f80040a3250 0x7f80040a5710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 --2- 172.21.15.1:0/2411763083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8018100a50 0x7f8018197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.039+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 >> 172.21.15.1:0/2411763083 conn(0x7f80180fc190 msgr2=0x7f801806a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.040+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 shutdown_connections 2024-09-16T00:28:42.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.040+0000 7f801fffd640 1 -- 172.21.15.1:0/2411763083 wait complete. 2024-09-16T00:28:42.052 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:21:40.721155+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:26:14.641676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:26:17.242600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-16T00:26:23.721714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-16T00:28:42.053 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T00:28:42.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:41 smithi019 ceph-mon[35186]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:28:42.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:41 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3048854785' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T00:28:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.158+0000 7f4002d42640 1 -- 172.21.15.1:0/2928021315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc104f00 msgr2=0x7f3ffc1072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.158+0000 7f4002d42640 1 --2- 172.21.15.1:0/2928021315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc104f00 0x7f3ffc1072f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009900 tx=0x7f3fe802a410 comp rx=0 tx=0).stop 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.158+0000 7f4002d42640 1 -- 172.21.15.1:0/2928021315 shutdown_connections 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.158+0000 7f4002d42640 1 --2- 172.21.15.1:0/2928021315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc107830 0x7f3ffc109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.158+0000 7f4002d42640 1 --2- 172.21.15.1:0/2928021315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc104f00 0x7f3ffc1072f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.158+0000 7f4002d42640 1 --2- 172.21.15.1:0/2928021315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ffc1025d0 0x7f3ffc1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.158+0000 7f4002d42640 1 -- 172.21.15.1:0/2928021315 >> 172.21.15.1:0/2928021315 conn(0x7f3ffc0fc190 msgr2=0x7f3ffc0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.158+0000 7f4002d42640 1 -- 172.21.15.1:0/2928021315 shutdown_connections 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.158+0000 7f4002d42640 1 -- 172.21.15.1:0/2928021315 wait complete. 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f4002d42640 1 Processor -- start 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f4002d42640 1 -- start start 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f4002d42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc1025d0 0x7f3ffc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f4002d42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ffc104f00 0x7f3ffc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f4002d42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc107830 0x7f3ffc19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f4002d42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc10d7e0 con 0x7f3ffc107830 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f4002d42640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc10d660 con 0x7f3ffc1025d0 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f4002d42640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc10d960 con 0x7f3ffc104f00 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f40012df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc107830 0x7f3ffc19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f40012df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc107830 0x7f3ffc19f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60190/0 (socket says 172.21.15.1:60190) 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f40012df640 1 -- 172.21.15.1:0/2297071567 learned_addr learned my addr 172.21.15.1:0/2297071567 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f4000ade640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc1025d0 0x7f3ffc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f3ffbfff640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ffc104f00 0x7f3ffc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.159+0000 7f40012df640 1 -- 172.21.15.1:0/2297071567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ffc104f00 msgr2=0x7f3ffc1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f40012df640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ffc104f00 0x7f3ffc1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f40012df640 1 -- 172.21.15.1:0/2297071567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc1025d0 msgr2=0x7f3ffc197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f40012df640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc1025d0 0x7f3ffc197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f40012df640 1 -- 172.21.15.1:0/2297071567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc19fb80 con 0x7f3ffc107830 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f4000ade640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc1025d0 0x7f3ffc197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f40012df640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc107830 0x7f3ffc19f510 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e9d0 tx=0x7f3ff400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f3ff9ffb640 1 -- 172.21.15.1:0/2297071567 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400cd80 con 0x7f3ffc107830 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f3ff9ffb640 1 -- 172.21.15.1:0/2297071567 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff4005020 con 0x7f3ffc107830 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc19fe70 con 0x7f3ffc107830 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f3ff9ffb640 1 -- 172.21.15.1:0/2297071567 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40116d0 con 0x7f3ffc107830 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.160+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1a02e0 con 0x7f3ffc107830 2024-09-16T00:28:42.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.161+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc0fdc80 con 0x7f3ffc107830 2024-09-16T00:28:42.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.162+0000 7f3ff9ffb640 1 -- 172.21.15.1:0/2297071567 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3ff40051d0 con 0x7f3ffc107830 2024-09-16T00:28:42.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.163+0000 7f3ff9ffb640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fec0a3250 0x7f3fec0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.163+0000 7f3ff9ffb640 1 -- 172.21.15.1:0/2297071567 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3ff40d03b0 con 0x7f3ffc107830 2024-09-16T00:28:42.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.163+0000 7f4000ade640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fec0a3250 0x7f3fec0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.164+0000 7f4000ade640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fec0a3250 0x7f3fec0a5710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000abe0 tx=0x7f3ff000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.167+0000 7f3ff9ffb640 1 -- 172.21.15.1:0/2297071567 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ff4016030 con 0x7f3ffc107830 2024-09-16T00:28:42.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.346+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ffc062e90 con 0x7f3ffc107830 2024-09-16T00:28:42.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.347+0000 7f3ff9ffb640 1 -- 172.21.15.1:0/2297071567 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f3ff40bc030 con 0x7f3ffc107830 2024-09-16T00:28:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-09-16T00:28:42.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fec0a3250 msgr2=0x7f3fec0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fec0a3250 0x7f3fec0a5710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000abe0 tx=0x7f3ff000a5c0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc107830 msgr2=0x7f3ffc19f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc107830 0x7f3ffc19f510 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e9d0 tx=0x7f3ff400ee90 comp rx=0 tx=0).stop 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 shutdown_connections 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ffc107830 0x7f3ffc19f510 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fec0a3250 0x7f3fec0a5710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ffc104f00 0x7f3ffc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 --2- 172.21.15.1:0/2297071567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc1025d0 0x7f3ffc197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 >> 172.21.15.1:0/2297071567 conn(0x7f3ffc0fc190 msgr2=0x7f3ffc1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 shutdown_connections 2024-09-16T00:28:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.349+0000 7f4002d42640 1 -- 172.21.15.1:0/2297071567 wait complete. 2024-09-16T00:28:42.362 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 -- 172.21.15.1:0/4018726766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 msgr2=0x7f731c108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 --2- 172.21.15.1:0/4018726766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 0x7f731c108e60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7314009ca0 tx=0x7f731402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 -- 172.21.15.1:0/4018726766 shutdown_connections 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 --2- 172.21.15.1:0/4018726766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 0x7f731c108e60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 --2- 172.21.15.1:0/4018726766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f731c101c10 0x7f731c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 --2- 172.21.15.1:0/4018726766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f731c100a20 0x7f731c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 -- 172.21.15.1:0/4018726766 >> 172.21.15.1:0/4018726766 conn(0x7f731c0fc180 msgr2=0x7f731c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 -- 172.21.15.1:0/4018726766 shutdown_connections 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 -- 172.21.15.1:0/4018726766 wait complete. 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 Processor -- start 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 -- start start 2024-09-16T00:28:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f731c100a20 0x7f731c104940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f731c101c10 0x7f731c102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 0x7f731c103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f731c10c6a0 con 0x7f731c1025d0 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f731c10c520 con 0x7f731c100a20 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.468+0000 7f7323f6f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f731c10c820 con 0x7f731c101c10 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732250c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 0x7f731c103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732250c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 0x7f731c103530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60200/0 (socket says 172.21.15.1:60200) 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732250c640 1 -- 172.21.15.1:0/3372446840 learned_addr learned my addr 172.21.15.1:0/3372446840 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f7321d0b640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f731c100a20 0x7f731c104940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732150a640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f731c101c10 0x7f731c102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732250c640 1 -- 172.21.15.1:0/3372446840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f731c101c10 msgr2=0x7f731c102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732250c640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f731c101c10 0x7f731c102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732250c640 1 -- 172.21.15.1:0/3372446840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f731c100a20 msgr2=0x7f731c104940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732250c640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f731c100a20 0x7f731c104940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732250c640 1 -- 172.21.15.1:0/3372446840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c103d50 con 0x7f731c1025d0 2024-09-16T00:28:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f732250c640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 0x7f731c103530 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7314005e30 tx=0x7f7314031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f7312ffd640 1 -- 172.21.15.1:0/3372446840 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314003b70 con 0x7f731c1025d0 2024-09-16T00:28:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f7312ffd640 1 -- 172.21.15.1:0/3372446840 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7314003d10 con 0x7f731c1025d0 2024-09-16T00:28:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731c1a1e50 con 0x7f731c1025d0 2024-09-16T00:28:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f7312ffd640 1 -- 172.21.15.1:0/3372446840 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731402fd30 con 0x7f731c1025d0 2024-09-16T00:28:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.469+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731c1a2360 con 0x7f731c1025d0 2024-09-16T00:28:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.470+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731c1a25c0 con 0x7f731c1025d0 2024-09-16T00:28:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.471+0000 7f7312ffd640 1 -- 172.21.15.1:0/3372446840 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f7314004050 con 0x7f731c1025d0 2024-09-16T00:28:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.472+0000 7f7312ffd640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73080a32d0 0x7f73080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.472+0000 7f7321d0b640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73080a32d0 0x7f73080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.472+0000 7f7312ffd640 1 -- 172.21.15.1:0/3372446840 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f73140f1050 con 0x7f731c1025d0 2024-09-16T00:28:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.473+0000 7f7321d0b640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73080a32d0 0x7f73080a5790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7318005fd0 tx=0x7f731800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.476+0000 7f7312ffd640 1 -- 172.21.15.1:0/3372446840 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73140bd970 con 0x7f731c1025d0 2024-09-16T00:28:42.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.656+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f731c1a2790 con 0x7f731c1025d0 2024-09-16T00:28:42.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.656+0000 7f7312ffd640 1 -- 172.21.15.1:0/3372446840 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f73140bd270 con 0x7f731c1025d0 2024-09-16T00:28:42.658 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.658+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73080a32d0 msgr2=0x7f73080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.658+0000 7f7323f6f640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73080a32d0 0x7f73080a5790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7318005fd0 tx=0x7f731800b390 comp rx=0 tx=0).stop 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.658+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 msgr2=0x7f731c103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.658+0000 7f7323f6f640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 0x7f731c103530 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7314005e30 tx=0x7f7314031ee0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.659+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 shutdown_connections 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.659+0000 7f7323f6f640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f731c1025d0 0x7f731c103530 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.659+0000 7f7323f6f640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73080a32d0 0x7f73080a5790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.659+0000 7f7323f6f640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f731c101c10 0x7f731c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.659+0000 7f7323f6f640 1 --2- 172.21.15.1:0/3372446840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f731c100a20 0x7f731c104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.659+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 >> 172.21.15.1:0/3372446840 conn(0x7f731c0fc180 msgr2=0x7f731c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.659+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 shutdown_connections 2024-09-16T00:28:42.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.659+0000 7f7323f6f640 1 -- 172.21.15.1:0/3372446840 wait complete. 2024-09-16T00:28:42.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T00:28:42.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.777+0000 7efc500e6640 1 -- 172.21.15.1:0/3492465156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 msgr2=0x7efc481007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.777+0000 7efc500e6640 1 --2- 172.21.15.1:0/3492465156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 0x7efc481007c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7efc44009900 tx=0x7efc4402a410 comp rx=0 tx=0).stop 2024-09-16T00:28:42.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.777+0000 7efc500e6640 1 -- 172.21.15.1:0/3492465156 shutdown_connections 2024-09-16T00:28:42.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.777+0000 7efc500e6640 1 --2- 172.21.15.1:0/3492465156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc481016a0 0x7efc48101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.777+0000 7efc500e6640 1 --2- 172.21.15.1:0/3492465156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc48100d00 0x7efc48101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.777+0000 7efc500e6640 1 --2- 172.21.15.1:0/3492465156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 0x7efc481007c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.777+0000 7efc500e6640 1 -- 172.21.15.1:0/3492465156 >> 172.21.15.1:0/3492465156 conn(0x7efc480fc190 msgr2=0x7efc480fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.777+0000 7efc500e6640 1 -- 172.21.15.1:0/3492465156 shutdown_connections 2024-09-16T00:28:42.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.777+0000 7efc500e6640 1 -- 172.21.15.1:0/3492465156 wait complete. 2024-09-16T00:28:42.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc500e6640 1 Processor -- start 2024-09-16T00:28:42.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc500e6640 1 -- start start 2024-09-16T00:28:42.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc500e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 0x7efc4819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc500e6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc48100d00 0x7efc4819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc500e6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc481016a0 0x7efc481a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc500e6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efc4810f9a0 con 0x7efc481003c0 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc500e6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc4810f820 con 0x7efc481016a0 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc500e6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efc4810fb20 con 0x7efc48100d00 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc4de82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 0x7efc4819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc4de82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 0x7efc4819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60226/0 (socket says 172.21.15.1:60226) 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc4de82640 1 -- 172.21.15.1:0/1928985272 learned_addr learned my addr 172.21.15.1:0/1928985272 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc4d681640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc48100d00 0x7efc4819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.778+0000 7efc4e683640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc481016a0 0x7efc481a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc4de82640 1 -- 172.21.15.1:0/1928985272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc48100d00 msgr2=0x7efc4819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc4de82640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc48100d00 0x7efc4819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc4de82640 1 -- 172.21.15.1:0/1928985272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc481016a0 msgr2=0x7efc481a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc4de82640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc481016a0 0x7efc481a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc4de82640 1 -- 172.21.15.1:0/1928985272 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc481a1c50 con 0x7efc481003c0 2024-09-16T00:28:42.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc4d681640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc48100d00 0x7efc4819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:42.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc4de82640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 0x7efc4819a030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efc44005940 tx=0x7efc44004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc3effd640 1 -- 172.21.15.1:0/1928985272 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4403f030 con 0x7efc481003c0 2024-09-16T00:28:42.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc3effd640 1 -- 172.21.15.1:0/1928985272 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc44003ce0 con 0x7efc481003c0 2024-09-16T00:28:42.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc481a1ee0 con 0x7efc481003c0 2024-09-16T00:28:42.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc3effd640 1 -- 172.21.15.1:0/1928985272 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44035070 con 0x7efc481003c0 2024-09-16T00:28:42.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.779+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc481a2340 con 0x7efc481003c0 2024-09-16T00:28:42.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.780+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4810b110 con 0x7efc481003c0 2024-09-16T00:28:42.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.781+0000 7efc3effd640 1 -- 172.21.15.1:0/1928985272 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7efc44042030 con 0x7efc481003c0 2024-09-16T00:28:42.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.782+0000 7efc3effd640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc340a32d0 0x7efc340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.782+0000 7efc4d681640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc340a32d0 0x7efc340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.782+0000 7efc3effd640 1 -- 172.21.15.1:0/1928985272 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efc440f0710 con 0x7efc481003c0 2024-09-16T00:28:42.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.783+0000 7efc4d681640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc340a32d0 0x7efc340a5790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc4819b540 tx=0x7efc3800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.786+0000 7efc3effd640 1 -- 172.21.15.1:0/1928985272 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc440bd030 con 0x7efc481003c0 2024-09-16T00:28:42.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:42 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3048854785' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T00:28:42.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:42 smithi001 ceph-mon[30934]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:28:42.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:42 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2411763083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:42.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:42 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2297071567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:28:42.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:42 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3372446840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:28:42.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:42 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3048854785' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T00:28:42.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:42 smithi153 ceph-mon[35538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:28:42.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:42 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2411763083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:42.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:42 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2297071567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:28:42.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:42 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3372446840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:28:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.963+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efc48062e90 con 0x7efc481003c0 2024-09-16T00:28:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.964+0000 7efc3effd640 1 -- 172.21.15.1:0/1928985272 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7efc440bc930 con 0x7efc481003c0 2024-09-16T00:28:42.965 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-09-16T00:28:42.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc340a32d0 msgr2=0x7efc340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc340a32d0 0x7efc340a5790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc4819b540 tx=0x7efc3800a5c0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 msgr2=0x7efc4819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 0x7efc4819a030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efc44005940 tx=0x7efc44004360 comp rx=0 tx=0).stop 2024-09-16T00:28:42.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 shutdown_connections 2024-09-16T00:28:42.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc481016a0 0x7efc481a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efc340a32d0 0x7efc340a5790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc48100d00 0x7efc4819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 --2- 172.21.15.1:0/1928985272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc481003c0 0x7efc4819a030 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 >> 172.21.15.1:0/1928985272 conn(0x7efc480fc190 msgr2=0x7efc480fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 shutdown_connections 2024-09-16T00:28:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:42.966+0000 7efc500e6640 1 -- 172.21.15.1:0/1928985272 wait complete. 2024-09-16T00:28:42.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-16T00:28:43.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.083+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3435023986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0101b30 msgr2=0x7f09c0101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.083+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3435023986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0101b30 0x7f09c0101f30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009900 tx=0x7f09b002a410 comp rx=0 tx=0).stop 2024-09-16T00:28:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.083+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3435023986 shutdown_connections 2024-09-16T00:28:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.083+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3435023986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c01036e0 0x7f09c0109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.083+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3435023986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c0102d20 0x7f09c01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.083+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3435023986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0101b30 0x7f09c0101f30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.083+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3435023986 >> 172.21.15.1:0/3435023986 conn(0x7f09c00fd270 msgr2=0x7f09c00ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.083+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3435023986 shutdown_connections 2024-09-16T00:28:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.083+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3435023986 wait complete. 2024-09-16T00:28:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09c5ae4640 1 Processor -- start 2024-09-16T00:28:43.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09c5ae4640 1 -- start start 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09c5ae4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c0101b30 0x7f09c0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09c5ae4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0102d20 0x7f09c01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09c5ae4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c01036e0 0x7f09c019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09c5ae4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f09c010d790 con 0x7f09c0102d20 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09c5ae4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09c010d610 con 0x7f09c01036e0 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09c5ae4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09c010d910 con 0x7f09c0101b30 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0102d20 0x7f09c01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0102d20 0x7f09c01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60244/0 (socket says 172.21.15.1:60244) 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09beffd640 1 -- 172.21.15.1:0/3006391317 learned_addr learned my addr 172.21.15.1:0/3006391317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09bffff640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c01036e0 0x7f09c019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09beffd640 1 -- 172.21.15.1:0/3006391317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c0101b30 msgr2=0x7f09c0197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:43.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.084+0000 7f09bf7fe640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c0101b30 0x7f09c0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09beffd640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c0101b30 0x7f09c0197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09beffd640 1 -- 172.21.15.1:0/3006391317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c01036e0 msgr2=0x7f09c019f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09beffd640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c01036e0 0x7f09c019f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09beffd640 1 -- 172.21.15.1:0/3006391317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c019fa90 con 0x7f09c0102d20 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09bffff640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c01036e0 0x7f09c019f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09beffd640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0102d20 0x7f09c01983b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f09b400d950 tx=0x7f09b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09bcff9640 1 -- 172.21.15.1:0/3006391317 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4017070 con 0x7f09c0102d20 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09bcff9640 1 -- 172.21.15.1:0/3006391317 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4009c30 con 0x7f09c0102d20 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c019fd80 con 0x7f09c0102d20 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09bcff9640 1 -- 172.21.15.1:0/3006391317 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b400f6b0 con 0x7f09c0102d20 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.085+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c01a0270 con 0x7f09c0102d20 2024-09-16T00:28:43.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.086+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c0108fc0 con 0x7f09c0102d20 2024-09-16T00:28:43.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.087+0000 7f09bcff9640 1 -- 172.21.15.1:0/3006391317 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f09b40109d0 con 0x7f09c0102d20 2024-09-16T00:28:43.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.088+0000 7f09bcff9640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09a80a3250 0x7f09a80a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:43.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.088+0000 7f09bf7fe640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09a80a3250 0x7f09a80a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:43.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.088+0000 7f09bcff9640 1 -- 172.21.15.1:0/3006391317 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f09b40cfdb0 con 0x7f09c0102d20 2024-09-16T00:28:43.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.089+0000 7f09bf7fe640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09a80a3250 0x7f09a80a5710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09b0031610 tx=0x7f09b0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:43.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.092+0000 7f09bcff9640 1 -- 172.21.15.1:0/3006391317 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09b40d4070 con 0x7f09c0102d20 2024-09-16T00:28:43.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:42 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3048854785' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T00:28:43.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:42 smithi019 ceph-mon[35186]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:28:43.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:42 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2411763083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:43.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:42 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2297071567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:28:43.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:42 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3372446840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:28:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.269+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f09c0062e90 con 0x7f09c0102d20 2024-09-16T00:28:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.270+0000 7f09bcff9640 1 -- 172.21.15.1:0/3006391317 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f09b4009de0 con 0x7f09c0102d20 2024-09-16T00:28:43.271 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-09-16T00:28:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.271+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09a80a3250 msgr2=0x7f09a80a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.271+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09a80a3250 0x7f09a80a5710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09b0031610 tx=0x7f09b0000bc0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0102d20 msgr2=0x7f09c01983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0102d20 0x7f09c01983b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f09b400d950 tx=0x7f09b400de10 comp rx=0 tx=0).stop 2024-09-16T00:28:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 shutdown_connections 2024-09-16T00:28:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c01036e0 0x7f09c019f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c0102d20 0x7f09c01983b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09a80a3250 0x7f09a80a5710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 --2- 172.21.15.1:0/3006391317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c0101b30 0x7f09c0197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 >> 172.21.15.1:0/3006391317 conn(0x7f09c00fd270 msgr2=0x7f09c0106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 shutdown_connections 2024-09-16T00:28:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:43.272+0000 7f09c5ae4640 1 -- 172.21.15.1:0/3006391317 wait complete. 2024-09-16T00:28:43.284 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-16T00:28:43.284 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:43.541 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:43.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:43 smithi001 ceph-mon[30934]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:28:43.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:43 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1928985272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:28:43.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:43 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3006391317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:28:43.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:43 smithi153 ceph-mon[35538]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:28:43.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:43 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1928985272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:28:43.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:43 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3006391317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:28:44.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.064+0000 7f99f104b640 1 -- 172.21.15.1:0/483747022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec100e20 msgr2=0x7f99ec101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.064+0000 7f99f104b640 1 --2- 172.21.15.1:0/483747022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec100e20 0x7f99ec101200 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009900 tx=0x7f99dc02a480 comp rx=0 tx=0).stop 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.064+0000 7f99f104b640 1 -- 172.21.15.1:0/483747022 shutdown_connections 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.064+0000 7f99f104b640 1 --2- 172.21.15.1:0/483747022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec102ef0 0x7f99ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.064+0000 7f99f104b640 1 --2- 172.21.15.1:0/483747022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99ec102570 0x7f99ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.064+0000 7f99f104b640 1 --2- 172.21.15.1:0/483747022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec100e20 0x7f99ec101200 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.064+0000 7f99f104b640 1 -- 172.21.15.1:0/483747022 >> 172.21.15.1:0/483747022 conn(0x7f99ec0fc7d0 msgr2=0x7f99ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.064+0000 7f99f104b640 1 -- 172.21.15.1:0/483747022 shutdown_connections 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.064+0000 7f99f104b640 1 -- 172.21.15.1:0/483747022 wait complete. 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99f104b640 1 Processor -- start 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99f104b640 1 -- start start 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99f104b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99ec100e20 0x7f99ec198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99f104b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec102570 0x7f99ec198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:44.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99f104b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec102ef0 0x7f99ec19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99f104b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec1120a0 con 0x7f99ec102ef0 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99f104b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec111f20 con 0x7f99ec100e20 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99f104b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec112220 con 0x7f99ec102570 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99eb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec102ef0 0x7f99ec19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99eb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec102ef0 0x7f99ec19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60262/0 (socket says 172.21.15.1:60262) 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.065+0000 7f99eb59e640 1 -- 172.21.15.1:0/2426571856 learned_addr learned my addr 172.21.15.1:0/2426571856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99ead9d640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99ec100e20 0x7f99ec198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99e259c640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec102570 0x7f99ec198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99eb59e640 1 -- 172.21.15.1:0/2426571856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec102570 msgr2=0x7f99ec198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99eb59e640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec102570 0x7f99ec198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99eb59e640 1 -- 172.21.15.1:0/2426571856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99ec100e20 msgr2=0x7f99ec198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:44.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99eb59e640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99ec100e20 0x7f99ec198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99eb59e640 1 -- 172.21.15.1:0/2426571856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ec19d4b0 con 0x7f99ec102ef0 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99eb59e640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec102ef0 0x7f99ec19cdd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f99e400eaa0 tx=0x7f99e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99e8d99640 1 -- 172.21.15.1:0/2426571856 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e400cdb0 con 0x7f99ec102ef0 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ec19d7a0 con 0x7f99ec102ef0 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99e8d99640 1 -- 172.21.15.1:0/2426571856 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e4005020 con 0x7f99ec102ef0 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99e8d99640 1 -- 172.21.15.1:0/2426571856 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e40116d0 con 0x7f99ec102ef0 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.066+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ec1a5080 con 0x7f99ec102ef0 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.067+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99ec101200 con 0x7f99ec102ef0 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.068+0000 7f99e8d99640 1 -- 172.21.15.1:0/2426571856 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f99e40054d0 con 0x7f99ec102ef0 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.069+0000 7f99e8d99640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99d80a3240 0x7f99d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:44.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.069+0000 7f99e8d99640 1 -- 172.21.15.1:0/2426571856 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f99e40cffc0 con 0x7f99ec102ef0 2024-09-16T00:28:44.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.069+0000 7f99ead9d640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99d80a3240 0x7f99d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:44.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.070+0000 7f99ead9d640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99d80a3240 0x7f99d80a5700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f99dc005ab0 tx=0x7f99dc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:44.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.073+0000 7f99e8d99640 1 -- 172.21.15.1:0/2426571856 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99e4016030 con 0x7f99ec102ef0 2024-09-16T00:28:44.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:43 smithi019 ceph-mon[35186]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:28:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:43 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1928985272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:28:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:43 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3006391317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:28:44.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.284+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99ec109090 con 0x7f99ec102ef0 2024-09-16T00:28:44.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.285+0000 7f99e8d99640 1 -- 172.21.15.1:0/2426571856 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f99e40bae20 con 0x7f99ec102ef0 2024-09-16T00:28:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:44.286 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:28:44.287 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:28:44.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99d80a3240 msgr2=0x7f99d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:44.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99d80a3240 0x7f99d80a5700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f99dc005ab0 tx=0x7f99dc005a20 comp rx=0 tx=0).stop 2024-09-16T00:28:44.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec102ef0 msgr2=0x7f99ec19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:44.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec102ef0 0x7f99ec19cdd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f99e400eaa0 tx=0x7f99e400ef60 comp rx=0 tx=0).stop 2024-09-16T00:28:44.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 shutdown_connections 2024-09-16T00:28:44.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99ec102ef0 0x7f99ec19cdd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99d80a3240 0x7f99d80a5700 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec102570 0x7f99ec198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 --2- 172.21.15.1:0/2426571856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99ec100e20 0x7f99ec198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 >> 172.21.15.1:0/2426571856 conn(0x7f99ec0fc7d0 msgr2=0x7f99ec1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:44.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 shutdown_connections 2024-09-16T00:28:44.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:44.287+0000 7f99f104b640 1 -- 172.21.15.1:0/2426571856 wait complete. 2024-09-16T00:28:44.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:44 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2426571856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:44.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:44 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2426571856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:45.000 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-16T00:28:45.001 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-16T00:28:45.001 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:28:45.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:44 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2426571856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:45.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 -- 172.21.15.1:0/3057916585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20101c40 msgr2=0x7f3a201020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/3057916585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20101c40 0x7f3a201020c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009900 tx=0x7f3a1402a410 comp rx=0 tx=0).stop 2024-09-16T00:28:45.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 -- 172.21.15.1:0/3057916585 shutdown_connections 2024-09-16T00:28:45.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/3057916585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a20102600 0x7f3a20108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/3057916585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20101c40 0x7f3a201020c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/3057916585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a20100a50 0x7f3a20100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 -- 172.21.15.1:0/3057916585 >> 172.21.15.1:0/3057916585 conn(0x7f3a200fc1f0 msgr2=0x7f3a200fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:45.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 -- 172.21.15.1:0/3057916585 shutdown_connections 2024-09-16T00:28:45.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 -- 172.21.15.1:0/3057916585 wait complete. 2024-09-16T00:28:45.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 Processor -- start 2024-09-16T00:28:45.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.107+0000 7f3a24aa5640 1 -- start start 2024-09-16T00:28:45.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a24aa5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20100a50 0x7f3a20196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a24aa5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a20101c40 0x7f3a20197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a24aa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a20102600 0x7f3a2019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a24aa5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2010c6d0 con 0x7f3a20100a50 2024-09-16T00:28:45.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a24aa5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2010c550 con 0x7f3a20102600 2024-09-16T00:28:45.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a24aa5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2010c850 con 0x7f3a20101c40 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20100a50 0x7f3a20196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20100a50 0x7f3a20196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60270/0 (socket says 172.21.15.1:60270) 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1e59c640 1 -- 172.21.15.1:0/831498900 learned_addr learned my addr 172.21.15.1:0/831498900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1dd9b640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a20101c40 0x7f3a20197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1ed9d640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a20102600 0x7f3a2019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1e59c640 1 -- 172.21.15.1:0/831498900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a20101c40 msgr2=0x7f3a20197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1e59c640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a20101c40 0x7f3a20197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1e59c640 1 -- 172.21.15.1:0/831498900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a20102600 msgr2=0x7f3a2019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1e59c640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a20102600 0x7f3a2019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1e59c640 1 -- 172.21.15.1:0/831498900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2019e920 con 0x7f3a20100a50 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1ed9d640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a20102600 0x7f3a2019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:45.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a1e59c640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20100a50 0x7f3a20196d10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000ccb0 tx=0x7f3a1000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:45.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a0f7fe640 1 -- 172.21.15.1:0/831498900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10003b70 con 0x7f3a20100a50 2024-09-16T00:28:45.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a0f7fe640 1 -- 172.21.15.1:0/831498900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1000ae50 con 0x7f3a20100a50 2024-09-16T00:28:45.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.108+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2019ec10 con 0x7f3a20100a50 2024-09-16T00:28:45.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.109+0000 7f3a0f7fe640 1 -- 172.21.15.1:0/831498900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000f810 con 0x7f3a20100a50 2024-09-16T00:28:45.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.109+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2019f160 con 0x7f3a20100a50 2024-09-16T00:28:45.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.110+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a20107e40 con 0x7f3a20100a50 2024-09-16T00:28:45.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.111+0000 7f3a0f7fe640 1 -- 172.21.15.1:0/831498900 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3a1000f9b0 con 0x7f3a20100a50 2024-09-16T00:28:45.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.112+0000 7f3a0f7fe640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3a080a3250 0x7f3a080a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.112+0000 7f3a0f7fe640 1 -- 172.21.15.1:0/831498900 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3a100cff20 con 0x7f3a20100a50 2024-09-16T00:28:45.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.112+0000 7f3a1dd9b640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3a080a3250 0x7f3a080a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.112+0000 7f3a1dd9b640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3a080a3250 0x7f3a080a5710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a140314e0 tx=0x7f3a14000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:45.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.116+0000 7f3a0f7fe640 1 -- 172.21.15.1:0/831498900 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a10014070 con 0x7f3a20100a50 2024-09-16T00:28:45.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.289+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a20062630 con 0x7f3a080a3250 2024-09-16T00:28:45.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.298+0000 7f3a0f7fe640 1 -- 172.21.15.1:0/831498900 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340483 (secure 0 0 0) 0x7f3a20062630 con 0x7f3a080a3250 2024-09-16T00:28:45.304 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:45.306 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:28:45.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3a080a3250 msgr2=0x7f3a080a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3a080a3250 0x7f3a080a5710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a140314e0 tx=0x7f3a14000bc0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20100a50 msgr2=0x7f3a20196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20100a50 0x7f3a20196d10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000ccb0 tx=0x7f3a1000a620 comp rx=0 tx=0).stop 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 shutdown_connections 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a20102600 0x7f3a2019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3a080a3250 0x7f3a080a5710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a20101c40 0x7f3a20197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 --2- 172.21.15.1:0/831498900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a20100a50 0x7f3a20196d10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 >> 172.21.15.1:0/831498900 conn(0x7f3a200fc1f0 msgr2=0x7f3a20105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 shutdown_connections 2024-09-16T00:28:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.302+0000 7f3a24aa5640 1 -- 172.21.15.1:0/831498900 wait complete. 2024-09-16T00:28:45.315 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-16T00:28:44.346084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657379,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":752,"num_read_kb":495,"num_write":359,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380128,"kb_used_data":14240,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561696,"statfs":{"total":1151940427776,"available":1151551176704,"internally_reserved":0,"allocated":14581760,"data_stored":8127783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003579"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252865+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:27:26.252865+0000","last_peered":"2024-09-16T00:27:26.252865+0000","last_clean":"2024-09-16T00:27:26.252865+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:27:26.252865+0000","last_undegraded":"2024-09-16T00:27:26.252865+0000","last_fullsized":"2024-09-16T00:27:26.252865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322537+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:28:24.322537+0000","last_peered":"2024-09-16T00:28:24.322537+0000","last_clean":"2024-09-16T00:28:24.322537+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:28:24.322537+0000","last_undegraded":"2024-09-16T00:28:24.322537+0000","last_fullsized":"2024-09-16T00:28:24.322537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248105+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:27:26.248105+0000","last_peered":"2024-09-16T00:27:26.248105+0000","last_clean":"2024-09-16T00:27:26.248105+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:27:26.248105+0000","last_undegraded":"2024-09-16T00:27:26.248105+0000","last_fullsized":"2024-09-16T00:27:26.248105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247830+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:27:26.247830+0000","last_peered":"2024-09-16T00:27:26.247830+0000","last_clean":"2024-09-16T00:27:26.247830+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:27:26.247830+0000","last_undegraded":"2024-09-16T00:27:26.247830+0000","last_fullsized":"2024-09-16T00:27:26.247830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336305+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:28:24.336305+0000","last_peered":"2024-09-16T00:28:24.336305+0000","last_clean":"2024-09-16T00:28:24.336305+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:28:24.336305+0000","last_undegraded":"2024-09-16T00:28:24.336305+0000","last_fullsized":"2024-09-16T00:28:24.336305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355288+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:27:26.355288+0000","last_peered":"2024-09-16T00:27:26.355288+0000","last_clean":"2024-09-16T00:27:26.355288+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:27:26.355288+0000","last_undegraded":"2024-09-16T00:27:26.355288+0000","last_fullsized":"2024-09-16T00:27:26.355288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357791+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:26.357791+0000","last_peered":"2024-09-16T00:27:26.357791+0000","last_clean":"2024-09-16T00:27:26.357791+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:26.357791+0000","last_undegraded":"2024-09-16T00:27:26.357791+0000","last_fullsized":"2024-09-16T00:27:26.357791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.335269+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:28:24.335269+0000","last_peered":"2024-09-16T00:28:24.335269+0000","last_clean":"2024-09-16T00:28:24.335269+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:28:24.335269+0000","last_undegraded":"2024-09-16T00:28:24.335269+0000","last_fullsized":"2024-09-16T00:28:24.335269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356501+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:27:26.356501+0000","last_peered":"2024-09-16T00:27:26.356501+0000","last_clean":"2024-09-16T00:27:26.356501+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:27:26.356501+0000","last_undegraded":"2024-09-16T00:27:26.356501+0000","last_fullsized":"2024-09-16T00:27:26.356501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252350+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:27:26.252350+0000","last_peered":"2024-09-16T00:27:26.252350+0000","last_clean":"2024-09-16T00:27:26.252350+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:27:26.252350+0000","last_undegraded":"2024-09-16T00:27:26.252350+0000","last_fullsized":"2024-09-16T00:27:26.252350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336415+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:28:24.336415+0000","last_peered":"2024-09-16T00:28:24.336415+0000","last_clean":"2024-09-16T00:28:24.336415+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:28:24.336415+0000","last_undegraded":"2024-09-16T00:28:24.336415+0000","last_fullsized":"2024-09-16T00:28:24.336415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355852+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:27:26.355852+0000","last_peered":"2024-09-16T00:27:26.355852+0000","last_clean":"2024-09-16T00:27:26.355852+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:27:26.355852+0000","last_undegraded":"2024-09-16T00:27:26.355852+0000","last_fullsized":"2024-09-16T00:27:26.355852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338941+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:28:24.338941+0000","last_peered":"2024-09-16T00:28:24.338941+0000","last_clean":"2024-09-16T00:28:24.338941+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:28:24.338941+0000","last_undegraded":"2024-09-16T00:28:24.338941+0000","last_fullsized":"2024-09-16T00:28:24.338941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357104+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:26.357104+0000","last_peered":"2024-09-16T00:27:26.357104+0000","last_clean":"2024-09-16T00:27:26.357104+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:26.357104+0000","last_undegraded":"2024-09-16T00:27:26.357104+0000","last_fullsized":"2024-09-16T00:27:26.357104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356176+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:27:26.356176+0000","last_peered":"2024-09-16T00:27:26.356176+0000","last_clean":"2024-09-16T00:27:26.356176+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:27:26.356176+0000","last_undegraded":"2024-09-16T00:27:26.356176+0000","last_fullsized":"2024-09-16T00:27:26.356176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327188+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:28:24.327188+0000","last_peered":"2024-09-16T00:28:24.327188+0000","last_clean":"2024-09-16T00:28:24.327188+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:28:24.327188+0000","last_undegraded":"2024-09-16T00:28:24.327188+0000","last_fullsized":"2024-09-16T00:28:24.327188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248306+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:27:26.248306+0000","last_peered":"2024-09-16T00:27:26.248306+0000","last_clean":"2024-09-16T00:27:26.248306+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:27:26.248306+0000","last_undegraded":"2024-09-16T00:27:26.248306+0000","last_fullsized":"2024-09-16T00:27:26.248306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356287+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:27:26.356287+0000","last_peered":"2024-09-16T00:27:26.356287+0000","last_clean":"2024-09-16T00:27:26.356287+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:27:26.356287+0000","last_undegraded":"2024-09-16T00:27:26.356287+0000","last_fullsized":"2024-09-16T00:27:26.356287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324477+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:28:24.324477+0000","last_peered":"2024-09-16T00:28:24.324477+0000","last_clean":"2024-09-16T00:28:24.324477+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:28:24.324477+0000","last_undegraded":"2024-09-16T00:28:24.324477+0000","last_fullsized":"2024-09-16T00:28:24.324477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247518+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:26.247518+0000","last_peered":"2024-09-16T00:27:26.247518+0000","last_clean":"2024-09-16T00:27:26.247518+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:26.247518+0000","last_undegraded":"2024-09-16T00:27:26.247518+0000","last_fullsized":"2024-09-16T00:27:26.247518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252497+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:27:26.252497+0000","last_peered":"2024-09-16T00:27:26.252497+0000","last_clean":"2024-09-16T00:27:26.252497+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:27:26.252497+0000","last_undegraded":"2024-09-16T00:27:26.252497+0000","last_fullsized":"2024-09-16T00:27:26.252497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.312122+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:28:24.312122+0000","last_peered":"2024-09-16T00:28:24.312122+0000","last_clean":"2024-09-16T00:28:24.312122+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:28:24.312122+0000","last_undegraded":"2024-09-16T00:28:24.312122+0000","last_fullsized":"2024-09-16T00:28:24.312122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355055+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:26.355055+0000","last_peered":"2024-09-16T00:27:26.355055+0000","last_clean":"2024-09-16T00:27:26.355055+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:26.355055+0000","last_undegraded":"2024-09-16T00:27:26.355055+0000","last_fullsized":"2024-09-16T00:27:26.355055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357331+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:27:26.357331+0000","last_peered":"2024-09-16T00:27:26.357331+0000","last_clean":"2024-09-16T00:27:26.357331+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:27:26.357331+0000","last_undegraded":"2024-09-16T00:27:26.357331+0000","last_fullsized":"2024-09-16T00:27:26.357331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334895+0000","last_change":"2024-09-16T00:27:26.358639+0000","last_active":"2024-09-16T00:28:24.334895+0000","last_peered":"2024-09-16T00:28:24.334895+0000","last_clean":"2024-09-16T00:28:24.334895+0000","last_became_active":"2024-09-16T00:27:26.358163+0000","last_became_peered":"2024-09-16T00:27:26.358163+0000","last_unstale":"2024-09-16T00:28:24.334895+0000","last_undegraded":"2024-09-16T00:28:24.334895+0000","last_fullsized":"2024-09-16T00:28:24.334895+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:11.811436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356716+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:26.356716+0000","last_peered":"2024-09-16T00:27:26.356716+0000","last_clean":"2024-09-16T00:27:26.356716+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:26.356716+0000","last_undegraded":"2024-09-16T00:27:26.356716+0000","last_fullsized":"2024-09-16T00:27:26.356716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248397+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:27:26.248397+0000","last_peered":"2024-09-16T00:27:26.248397+0000","last_clean":"2024-09-16T00:27:26.248397+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:27:26.248397+0000","last_undegraded":"2024-09-16T00:27:26.248397+0000","last_fullsized":"2024-09-16T00:27:26.248397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332074+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:28:24.332074+0000","last_peered":"2024-09-16T00:28:24.332074+0000","last_clean":"2024-09-16T00:28:24.332074+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:28:24.332074+0000","last_undegraded":"2024-09-16T00:28:24.332074+0000","last_fullsized":"2024-09-16T00:28:24.332074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329964+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:28:24.329964+0000","last_peered":"2024-09-16T00:28:24.329964+0000","last_clean":"2024-09-16T00:28:24.329964+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:28:24.329964+0000","last_undegraded":"2024-09-16T00:28:24.329964+0000","last_fullsized":"2024-09-16T00:28:24.329964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247207+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:27:26.247207+0000","last_peered":"2024-09-16T00:27:26.247207+0000","last_clean":"2024-09-16T00:27:26.247207+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:27:26.247207+0000","last_undegraded":"2024-09-16T00:27:26.247207+0000","last_fullsized":"2024-09-16T00:27:26.247207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.325854+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:28:24.325854+0000","last_peered":"2024-09-16T00:28:24.325854+0000","last_clean":"2024-09-16T00:28:24.325854+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:28:24.325854+0000","last_undegraded":"2024-09-16T00:28:24.325854+0000","last_fullsized":"2024-09-16T00:28:24.325854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357960+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:26.357960+0000","last_peered":"2024-09-16T00:27:26.357960+0000","last_clean":"2024-09-16T00:27:26.357960+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:26.357960+0000","last_undegraded":"2024-09-16T00:27:26.357960+0000","last_fullsized":"2024-09-16T00:27:26.357960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251785+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:27:26.251785+0000","last_peered":"2024-09-16T00:27:26.251785+0000","last_clean":"2024-09-16T00:27:26.251785+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:27:26.251785+0000","last_undegraded":"2024-09-16T00:27:26.251785+0000","last_fullsized":"2024-09-16T00:27:26.251785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329144+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:28:24.329144+0000","last_peered":"2024-09-16T00:28:24.329144+0000","last_clean":"2024-09-16T00:28:24.329144+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:28:24.329144+0000","last_undegraded":"2024-09-16T00:28:24.329144+0000","last_fullsized":"2024-09-16T00:28:24.329144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252455+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:27:26.252455+0000","last_peered":"2024-09-16T00:27:26.252455+0000","last_clean":"2024-09-16T00:27:26.252455+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:27:26.252455+0000","last_undegraded":"2024-09-16T00:27:26.252455+0000","last_fullsized":"2024-09-16T00:27:26.252455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357048+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:27:26.357048+0000","last_peered":"2024-09-16T00:27:26.357048+0000","last_clean":"2024-09-16T00:27:26.357048+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:27:26.357048+0000","last_undegraded":"2024-09-16T00:27:26.357048+0000","last_fullsized":"2024-09-16T00:27:26.357048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321720+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:28:24.321720+0000","last_peered":"2024-09-16T00:28:24.321720+0000","last_clean":"2024-09-16T00:28:24.321720+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:28:24.321720+0000","last_undegraded":"2024-09-16T00:28:24.321720+0000","last_fullsized":"2024-09-16T00:28:24.321720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357297+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:27:26.357297+0000","last_peered":"2024-09-16T00:27:26.357297+0000","last_clean":"2024-09-16T00:27:26.357297+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:27:26.357297+0000","last_undegraded":"2024-09-16T00:27:26.357297+0000","last_fullsized":"2024-09-16T00:27:26.357297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250069+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:27:26.250069+0000","last_peered":"2024-09-16T00:27:26.250069+0000","last_clean":"2024-09-16T00:27:26.250069+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:27:26.250069+0000","last_undegraded":"2024-09-16T00:27:26.250069+0000","last_fullsized":"2024-09-16T00:27:26.250069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.305108+0000","last_change":"2024-09-16T00:27:26.357501+0000","last_active":"2024-09-16T00:28:24.305108+0000","last_peered":"2024-09-16T00:28:24.305108+0000","last_clean":"2024-09-16T00:28:24.305108+0000","last_became_active":"2024-09-16T00:27:26.357108+0000","last_became_peered":"2024-09-16T00:27:26.357108+0000","last_unstale":"2024-09-16T00:28:24.305108+0000","last_undegraded":"2024-09-16T00:28:24.305108+0000","last_fullsized":"2024-09-16T00:28:24.305108+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:25.264059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":140,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:39.661319+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:28:39.661319+0000","last_peered":"2024-09-16T00:28:39.661319+0000","last_clean":"2024-09-16T00:28:39.661319+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:28:39.661319+0000","last_undegraded":"2024-09-16T00:28:39.661319+0000","last_fullsized":"2024-09-16T00:28:39.661319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250857+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:27:26.250857+0000","last_peered":"2024-09-16T00:27:26.250857+0000","last_clean":"2024-09-16T00:27:26.250857+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:27:26.250857+0000","last_undegraded":"2024-09-16T00:27:26.250857+0000","last_fullsized":"2024-09-16T00:27:26.250857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326885+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:28:24.326885+0000","last_peered":"2024-09-16T00:28:24.326885+0000","last_clean":"2024-09-16T00:28:24.326885+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:28:24.326885+0000","last_undegraded":"2024-09-16T00:28:24.326885+0000","last_fullsized":"2024-09-16T00:28:24.326885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357096+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:26.357096+0000","last_peered":"2024-09-16T00:27:26.357096+0000","last_clean":"2024-09-16T00:27:26.357096+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:26.357096+0000","last_undegraded":"2024-09-16T00:27:26.357096+0000","last_fullsized":"2024-09-16T00:27:26.357096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356120+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:27:26.356120+0000","last_peered":"2024-09-16T00:27:26.356120+0000","last_clean":"2024-09-16T00:27:26.356120+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:27:26.356120+0000","last_undegraded":"2024-09-16T00:27:26.356120+0000","last_fullsized":"2024-09-16T00:27:26.356120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336576+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:28:24.336576+0000","last_peered":"2024-09-16T00:28:24.336576+0000","last_clean":"2024-09-16T00:28:24.336576+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:28:24.336576+0000","last_undegraded":"2024-09-16T00:28:24.336576+0000","last_fullsized":"2024-09-16T00:28:24.336576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357054+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:26.357054+0000","last_peered":"2024-09-16T00:27:26.357054+0000","last_clean":"2024-09-16T00:27:26.357054+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:26.357054+0000","last_undegraded":"2024-09-16T00:27:26.357054+0000","last_fullsized":"2024-09-16T00:27:26.357054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357365+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:27:26.357365+0000","last_peered":"2024-09-16T00:27:26.357365+0000","last_clean":"2024-09-16T00:27:26.357365+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:27:26.357365+0000","last_undegraded":"2024-09-16T00:27:26.357365+0000","last_fullsized":"2024-09-16T00:27:26.357365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355730+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:27:26.355730+0000","last_peered":"2024-09-16T00:27:26.355730+0000","last_clean":"2024-09-16T00:27:26.355730+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:27:26.355730+0000","last_undegraded":"2024-09-16T00:27:26.355730+0000","last_fullsized":"2024-09-16T00:27:26.355730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338444+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:28:24.338444+0000","last_peered":"2024-09-16T00:28:24.338444+0000","last_clean":"2024-09-16T00:28:24.338444+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:28:24.338444+0000","last_undegraded":"2024-09-16T00:28:24.338444+0000","last_fullsized":"2024-09-16T00:28:24.338444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248838+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:26.248838+0000","last_peered":"2024-09-16T00:27:26.248838+0000","last_clean":"2024-09-16T00:27:26.248838+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:26.248838+0000","last_undegraded":"2024-09-16T00:27:26.248838+0000","last_fullsized":"2024-09-16T00:27:26.248838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355174+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:27:26.355174+0000","last_peered":"2024-09-16T00:27:26.355174+0000","last_clean":"2024-09-16T00:27:26.355174+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:27:26.355174+0000","last_undegraded":"2024-09-16T00:27:26.355174+0000","last_fullsized":"2024-09-16T00:27:26.355174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331484+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:28:24.331484+0000","last_peered":"2024-09-16T00:28:24.331484+0000","last_clean":"2024-09-16T00:28:24.331484+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:28:24.331484+0000","last_undegraded":"2024-09-16T00:28:24.331484+0000","last_fullsized":"2024-09-16T00:28:24.331484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356422+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:27:26.356422+0000","last_peered":"2024-09-16T00:27:26.356422+0000","last_clean":"2024-09-16T00:27:26.356422+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:27:26.356422+0000","last_undegraded":"2024-09-16T00:27:26.356422+0000","last_fullsized":"2024-09-16T00:27:26.356422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249539+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:27:26.249539+0000","last_peered":"2024-09-16T00:27:26.249539+0000","last_clean":"2024-09-16T00:27:26.249539+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:27:26.249539+0000","last_undegraded":"2024-09-16T00:27:26.249539+0000","last_fullsized":"2024-09-16T00:27:26.249539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.333565+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:28:24.333565+0000","last_peered":"2024-09-16T00:28:24.333565+0000","last_clean":"2024-09-16T00:28:24.333565+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:28:24.333565+0000","last_undegraded":"2024-09-16T00:28:24.333565+0000","last_fullsized":"2024-09-16T00:28:24.333565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250975+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:27:26.250975+0000","last_peered":"2024-09-16T00:27:26.250975+0000","last_clean":"2024-09-16T00:27:26.250975+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:27:26.250975+0000","last_undegraded":"2024-09-16T00:27:26.250975+0000","last_fullsized":"2024-09-16T00:27:26.250975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354985+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:27:26.354985+0000","last_peered":"2024-09-16T00:27:26.354985+0000","last_clean":"2024-09-16T00:27:26.354985+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:27:26.354985+0000","last_undegraded":"2024-09-16T00:27:26.354985+0000","last_fullsized":"2024-09-16T00:27:26.354985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.319032+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:28:24.319032+0000","last_peered":"2024-09-16T00:28:24.319032+0000","last_clean":"2024-09-16T00:28:24.319032+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:28:24.319032+0000","last_undegraded":"2024-09-16T00:28:24.319032+0000","last_fullsized":"2024-09-16T00:28:24.319032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357916+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:26.357916+0000","last_peered":"2024-09-16T00:27:26.357916+0000","last_clean":"2024-09-16T00:27:26.357916+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:26.357916+0000","last_undegraded":"2024-09-16T00:27:26.357916+0000","last_fullsized":"2024-09-16T00:27:26.357916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355183+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:27:26.355183+0000","last_peered":"2024-09-16T00:27:26.355183+0000","last_clean":"2024-09-16T00:27:26.355183+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:27:26.355183+0000","last_undegraded":"2024-09-16T00:27:26.355183+0000","last_fullsized":"2024-09-16T00:27:26.355183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336143+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:28:24.336143+0000","last_peered":"2024-09-16T00:28:24.336143+0000","last_clean":"2024-09-16T00:28:24.336143+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:28:24.336143+0000","last_undegraded":"2024-09-16T00:28:24.336143+0000","last_fullsized":"2024-09-16T00:28:24.336143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356998+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:26.356998+0000","last_peered":"2024-09-16T00:27:26.356998+0000","last_clean":"2024-09-16T00:27:26.356998+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:26.356998+0000","last_undegraded":"2024-09-16T00:27:26.356998+0000","last_fullsized":"2024-09-16T00:27:26.356998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253824+0000","last_change":"2024-09-16T00:27:26.253824+0000","last_active":"2024-09-16T00:27:26.253824+0000","last_peered":"2024-09-16T00:27:26.253824+0000","last_clean":"2024-09-16T00:27:26.253824+0000","last_became_active":"2024-09-16T00:27:26.253206+0000","last_became_peered":"2024-09-16T00:27:26.253206+0000","last_unstale":"2024-09-16T00:27:26.253824+0000","last_undegraded":"2024-09-16T00:27:26.253824+0000","last_fullsized":"2024-09-16T00:27:26.253824+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:22.543369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332839+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:28:24.332839+0000","last_peered":"2024-09-16T00:28:24.332839+0000","last_clean":"2024-09-16T00:28:24.332839+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:28:24.332839+0000","last_undegraded":"2024-09-16T00:28:24.332839+0000","last_fullsized":"2024-09-16T00:28:24.332839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":140,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:39.661782+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:28:39.661782+0000","last_peered":"2024-09-16T00:28:39.661782+0000","last_clean":"2024-09-16T00:28:39.661782+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:28:39.661782+0000","last_undegraded":"2024-09-16T00:28:39.661782+0000","last_fullsized":"2024-09-16T00:28:39.661782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252554+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:27:26.252554+0000","last_peered":"2024-09-16T00:27:26.252554+0000","last_clean":"2024-09-16T00:27:26.252554+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:27:26.252554+0000","last_undegraded":"2024-09-16T00:27:26.252554+0000","last_fullsized":"2024-09-16T00:27:26.252554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.335224+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:28:24.335224+0000","last_peered":"2024-09-16T00:28:24.335224+0000","last_clean":"2024-09-16T00:28:24.335224+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:28:24.335224+0000","last_undegraded":"2024-09-16T00:28:24.335224+0000","last_fullsized":"2024-09-16T00:28:24.335224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356320+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:26.356320+0000","last_peered":"2024-09-16T00:27:26.356320+0000","last_clean":"2024-09-16T00:27:26.356320+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:26.356320+0000","last_undegraded":"2024-09-16T00:27:26.356320+0000","last_fullsized":"2024-09-16T00:27:26.356320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252536+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:27:26.252536+0000","last_peered":"2024-09-16T00:27:26.252536+0000","last_clean":"2024-09-16T00:27:26.252536+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:27:26.252536+0000","last_undegraded":"2024-09-16T00:27:26.252536+0000","last_fullsized":"2024-09-16T00:27:26.252536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330126+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:28:24.330126+0000","last_peered":"2024-09-16T00:28:24.330126+0000","last_clean":"2024-09-16T00:28:24.330126+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:28:24.330126+0000","last_undegraded":"2024-09-16T00:28:24.330126+0000","last_fullsized":"2024-09-16T00:28:24.330126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248474+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:26.248474+0000","last_peered":"2024-09-16T00:27:26.248474+0000","last_clean":"2024-09-16T00:27:26.248474+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:26.248474+0000","last_undegraded":"2024-09-16T00:27:26.248474+0000","last_fullsized":"2024-09-16T00:27:26.248474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355941+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:27:26.355941+0000","last_peered":"2024-09-16T00:27:26.355941+0000","last_clean":"2024-09-16T00:27:26.355941+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:27:26.355941+0000","last_undegraded":"2024-09-16T00:27:26.355941+0000","last_fullsized":"2024-09-16T00:27:26.355941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.310802+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:28:24.310802+0000","last_peered":"2024-09-16T00:28:24.310802+0000","last_clean":"2024-09-16T00:28:24.310802+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:28:24.310802+0000","last_undegraded":"2024-09-16T00:28:24.310802+0000","last_fullsized":"2024-09-16T00:28:24.310802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":301,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:41.667628+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:28:41.667628+0000","last_peered":"2024-09-16T00:28:41.667628+0000","last_clean":"2024-09-16T00:28:41.667628+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:28:41.667628+0000","last_undegraded":"2024-09-16T00:28:41.667628+0000","last_fullsized":"2024-09-16T00:28:41.667628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62633,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":211,"num_read_kb":167,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.358804+0000","last_change":"2024-09-16T00:27:26.358804+0000","last_active":"2024-09-16T00:27:26.358804+0000","last_peered":"2024-09-16T00:27:26.358804+0000","last_clean":"2024-09-16T00:27:26.358804+0000","last_became_active":"2024-09-16T00:27:26.358317+0000","last_became_peered":"2024-09-16T00:27:26.358317+0000","last_unstale":"2024-09-16T00:27:26.358804+0000","last_undegraded":"2024-09-16T00:27:26.358804+0000","last_fullsized":"2024-09-16T00:27:26.358804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:38.642392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336469+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:28:24.336469+0000","last_peered":"2024-09-16T00:28:24.336469+0000","last_clean":"2024-09-16T00:28:24.336469+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:28:24.336469+0000","last_undegraded":"2024-09-16T00:28:24.336469+0000","last_fullsized":"2024-09-16T00:28:24.336469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356252+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:26.356252+0000","last_peered":"2024-09-16T00:27:26.356252+0000","last_clean":"2024-09-16T00:27:26.356252+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:26.356252+0000","last_undegraded":"2024-09-16T00:27:26.356252+0000","last_fullsized":"2024-09-16T00:27:26.356252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356754+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:27:26.356754+0000","last_peered":"2024-09-16T00:27:26.356754+0000","last_clean":"2024-09-16T00:27:26.356754+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:27:26.356754+0000","last_undegraded":"2024-09-16T00:27:26.356754+0000","last_fullsized":"2024-09-16T00:27:26.356754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320427+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:28:24.320427+0000","last_peered":"2024-09-16T00:28:24.320427+0000","last_clean":"2024-09-16T00:28:24.320427+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:28:24.320427+0000","last_undegraded":"2024-09-16T00:28:24.320427+0000","last_fullsized":"2024-09-16T00:28:24.320427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357185+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:26.357185+0000","last_peered":"2024-09-16T00:27:26.357185+0000","last_clean":"2024-09-16T00:27:26.357185+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:26.357185+0000","last_undegraded":"2024-09-16T00:27:26.357185+0000","last_fullsized":"2024-09-16T00:27:26.357185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252128+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:27:26.252128+0000","last_peered":"2024-09-16T00:27:26.252128+0000","last_clean":"2024-09-16T00:27:26.252128+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:27:26.252128+0000","last_undegraded":"2024-09-16T00:27:26.252128+0000","last_fullsized":"2024-09-16T00:27:26.252128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336828+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:28:24.336828+0000","last_peered":"2024-09-16T00:28:24.336828+0000","last_clean":"2024-09-16T00:28:24.336828+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:28:24.336828+0000","last_undegraded":"2024-09-16T00:28:24.336828+0000","last_fullsized":"2024-09-16T00:28:24.336828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":142,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:41.667633+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:28:41.667633+0000","last_peered":"2024-09-16T00:28:41.667633+0000","last_clean":"2024-09-16T00:28:41.667633+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:28:41.667633+0000","last_undegraded":"2024-09-16T00:28:41.667633+0000","last_fullsized":"2024-09-16T00:28:41.667633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247274+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:27:26.247274+0000","last_peered":"2024-09-16T00:27:26.247274+0000","last_clean":"2024-09-16T00:27:26.247274+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:27:26.247274+0000","last_undegraded":"2024-09-16T00:27:26.247274+0000","last_fullsized":"2024-09-16T00:27:26.247274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250057+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:27:26.250057+0000","last_peered":"2024-09-16T00:27:26.250057+0000","last_clean":"2024-09-16T00:27:26.250057+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:27:26.250057+0000","last_undegraded":"2024-09-16T00:27:26.250057+0000","last_fullsized":"2024-09-16T00:27:26.250057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331295+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:28:24.331295+0000","last_peered":"2024-09-16T00:28:24.331295+0000","last_clean":"2024-09-16T00:28:24.331295+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:28:24.331295+0000","last_undegraded":"2024-09-16T00:28:24.331295+0000","last_fullsized":"2024-09-16T00:28:24.331295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355360+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:27:26.355360+0000","last_peered":"2024-09-16T00:27:26.355360+0000","last_clean":"2024-09-16T00:27:26.355360+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:27:26.355360+0000","last_undegraded":"2024-09-16T00:27:26.355360+0000","last_fullsized":"2024-09-16T00:27:26.355360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.360047+0000","last_change":"2024-09-16T00:27:26.359230+0000","last_active":"2024-09-16T00:27:26.360047+0000","last_peered":"2024-09-16T00:27:26.360047+0000","last_clean":"2024-09-16T00:27:26.360047+0000","last_became_active":"2024-09-16T00:27:26.358662+0000","last_became_peered":"2024-09-16T00:27:26.358662+0000","last_unstale":"2024-09-16T00:27:26.360047+0000","last_undegraded":"2024-09-16T00:27:26.360047+0000","last_fullsized":"2024-09-16T00:27:26.360047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:52.111910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331664+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:28:24.331664+0000","last_peered":"2024-09-16T00:28:24.331664+0000","last_clean":"2024-09-16T00:28:24.331664+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:28:24.331664+0000","last_undegraded":"2024-09-16T00:28:24.331664+0000","last_fullsized":"2024-09-16T00:28:24.331664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250512+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:26.250512+0000","last_peered":"2024-09-16T00:27:26.250512+0000","last_clean":"2024-09-16T00:27:26.250512+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:26.250512+0000","last_undegraded":"2024-09-16T00:27:26.250512+0000","last_fullsized":"2024-09-16T00:27:26.250512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250892+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:26.250892+0000","last_peered":"2024-09-16T00:27:26.250892+0000","last_clean":"2024-09-16T00:27:26.250892+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:26.250892+0000","last_undegraded":"2024-09-16T00:27:26.250892+0000","last_fullsized":"2024-09-16T00:27:26.250892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332482+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:28:24.332482+0000","last_peered":"2024-09-16T00:28:24.332482+0000","last_clean":"2024-09-16T00:28:24.332482+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:28:24.332482+0000","last_undegraded":"2024-09-16T00:28:24.332482+0000","last_fullsized":"2024-09-16T00:28:24.332482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359360+0000","last_change":"2024-09-16T00:27:26.359360+0000","last_active":"2024-09-16T00:27:26.359360+0000","last_peered":"2024-09-16T00:27:26.359360+0000","last_clean":"2024-09-16T00:27:26.359360+0000","last_became_active":"2024-09-16T00:27:26.358462+0000","last_became_peered":"2024-09-16T00:27:26.358462+0000","last_unstale":"2024-09-16T00:27:26.359360+0000","last_undegraded":"2024-09-16T00:27:26.359360+0000","last_fullsized":"2024-09-16T00:27:26.359360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:38.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:38.662940+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:28:38.662940+0000","last_peered":"2024-09-16T00:28:38.662940+0000","last_clean":"2024-09-16T00:28:38.662940+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:28:38.662940+0000","last_undegraded":"2024-09-16T00:28:38.662940+0000","last_fullsized":"2024-09-16T00:28:38.662940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320803+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:28:24.320803+0000","last_peered":"2024-09-16T00:28:24.320803+0000","last_clean":"2024-09-16T00:28:24.320803+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:28:24.320803+0000","last_undegraded":"2024-09-16T00:28:24.320803+0000","last_fullsized":"2024-09-16T00:28:24.320803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248757+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:27:26.248757+0000","last_peered":"2024-09-16T00:27:26.248757+0000","last_clean":"2024-09-16T00:27:26.248757+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:27:26.248757+0000","last_undegraded":"2024-09-16T00:27:26.248757+0000","last_fullsized":"2024-09-16T00:27:26.248757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321717+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:28:24.321717+0000","last_peered":"2024-09-16T00:28:24.321717+0000","last_clean":"2024-09-16T00:28:24.321717+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:28:24.321717+0000","last_undegraded":"2024-09-16T00:28:24.321717+0000","last_fullsized":"2024-09-16T00:28:24.321717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321331+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:28:24.321331+0000","last_peered":"2024-09-16T00:28:24.321331+0000","last_clean":"2024-09-16T00:28:24.321331+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:28:24.321331+0000","last_undegraded":"2024-09-16T00:28:24.321331+0000","last_fullsized":"2024-09-16T00:28:24.321331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329213+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:28:24.329213+0000","last_peered":"2024-09-16T00:28:24.329213+0000","last_clean":"2024-09-16T00:28:24.329213+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:28:24.329213+0000","last_undegraded":"2024-09-16T00:28:24.329213+0000","last_fullsized":"2024-09-16T00:28:24.329213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326265+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:28:24.326265+0000","last_peered":"2024-09-16T00:28:24.326265+0000","last_clean":"2024-09-16T00:28:24.326265+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:28:24.326265+0000","last_undegraded":"2024-09-16T00:28:24.326265+0000","last_fullsized":"2024-09-16T00:28:24.326265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327220+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:28:24.327220+0000","last_peered":"2024-09-16T00:28:24.327220+0000","last_clean":"2024-09-16T00:28:24.327220+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:28:24.327220+0000","last_undegraded":"2024-09-16T00:28:24.327220+0000","last_fullsized":"2024-09-16T00:28:24.327220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.316594+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:28:24.316594+0000","last_peered":"2024-09-16T00:28:24.316594+0000","last_clean":"2024-09-16T00:28:24.316594+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:28:24.316594+0000","last_undegraded":"2024-09-16T00:28:24.316594+0000","last_fullsized":"2024-09-16T00:28:24.316594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337588+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:28:24.337588+0000","last_peered":"2024-09-16T00:28:24.337588+0000","last_clean":"2024-09-16T00:28:24.337588+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:28:24.337588+0000","last_undegraded":"2024-09-16T00:28:24.337588+0000","last_fullsized":"2024-09-16T00:28:24.337588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322996+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:28:24.322996+0000","last_peered":"2024-09-16T00:28:24.322996+0000","last_clean":"2024-09-16T00:28:24.322996+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:28:24.322996+0000","last_undegraded":"2024-09-16T00:28:24.322996+0000","last_fullsized":"2024-09-16T00:28:24.322996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322343+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:28:24.322343+0000","last_peered":"2024-09-16T00:28:24.322343+0000","last_clean":"2024-09-16T00:28:24.322343+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:28:24.322343+0000","last_undegraded":"2024-09-16T00:28:24.322343+0000","last_fullsized":"2024-09-16T00:28:24.322343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330356+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:28:24.330356+0000","last_peered":"2024-09-16T00:28:24.330356+0000","last_clean":"2024-09-16T00:28:24.330356+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:28:24.330356+0000","last_undegraded":"2024-09-16T00:28:24.330356+0000","last_fullsized":"2024-09-16T00:28:24.330356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337238+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:28:24.337238+0000","last_peered":"2024-09-16T00:28:24.337238+0000","last_clean":"2024-09-16T00:28:24.337238+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:28:24.337238+0000","last_undegraded":"2024-09-16T00:28:24.337238+0000","last_fullsized":"2024-09-16T00:28:24.337238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330609+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:28:24.330609+0000","last_peered":"2024-09-16T00:28:24.330609+0000","last_clean":"2024-09-16T00:28:24.330609+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:28:24.330609+0000","last_undegraded":"2024-09-16T00:28:24.330609+0000","last_fullsized":"2024-09-16T00:28:24.330609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.317884+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:28:24.317884+0000","last_peered":"2024-09-16T00:28:24.317884+0000","last_clean":"2024-09-16T00:28:24.317884+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:28:24.317884+0000","last_undegraded":"2024-09-16T00:28:24.317884+0000","last_fullsized":"2024-09-16T00:28:24.317884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329300+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:28:24.329300+0000","last_peered":"2024-09-16T00:28:24.329300+0000","last_clean":"2024-09-16T00:28:24.329300+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:28:24.329300+0000","last_undegraded":"2024-09-16T00:28:24.329300+0000","last_fullsized":"2024-09-16T00:28:24.329300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337159+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:28:24.337159+0000","last_peered":"2024-09-16T00:28:24.337159+0000","last_clean":"2024-09-16T00:28:24.337159+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:28:24.337159+0000","last_undegraded":"2024-09-16T00:28:24.337159+0000","last_fullsized":"2024-09-16T00:28:24.337159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336933+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:28:24.336933+0000","last_peered":"2024-09-16T00:28:24.336933+0000","last_clean":"2024-09-16T00:28:24.336933+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:28:24.336933+0000","last_undegraded":"2024-09-16T00:28:24.336933+0000","last_fullsized":"2024-09-16T00:28:24.336933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337483+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:28:24.337483+0000","last_peered":"2024-09-16T00:28:24.337483+0000","last_clean":"2024-09-16T00:28:24.337483+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:28:24.337483+0000","last_undegraded":"2024-09-16T00:28:24.337483+0000","last_fullsized":"2024-09-16T00:28:24.337483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326582+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:28:24.326582+0000","last_peered":"2024-09-16T00:28:24.326582+0000","last_clean":"2024-09-16T00:28:24.326582+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:28:24.326582+0000","last_undegraded":"2024-09-16T00:28:24.326582+0000","last_fullsized":"2024-09-16T00:28:24.326582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326496+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:28:24.326496+0000","last_peered":"2024-09-16T00:28:24.326496+0000","last_clean":"2024-09-16T00:28:24.326496+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:28:24.326496+0000","last_undegraded":"2024-09-16T00:28:24.326496+0000","last_fullsized":"2024-09-16T00:28:24.326496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322821+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:28:24.322821+0000","last_peered":"2024-09-16T00:28:24.322821+0000","last_clean":"2024-09-16T00:28:24.322821+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:28:24.322821+0000","last_undegraded":"2024-09-16T00:28:24.322821+0000","last_fullsized":"2024-09-16T00:28:24.322821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.319623+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:28:24.319623+0000","last_peered":"2024-09-16T00:28:24.319623+0000","last_clean":"2024-09-16T00:28:24.319623+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:28:24.319623+0000","last_undegraded":"2024-09-16T00:28:24.319623+0000","last_fullsized":"2024-09-16T00:28:24.319623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338146+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:28:24.338146+0000","last_peered":"2024-09-16T00:28:24.338146+0000","last_clean":"2024-09-16T00:28:24.338146+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:28:24.338146+0000","last_undegraded":"2024-09-16T00:28:24.338146+0000","last_fullsized":"2024-09-16T00:28:24.338146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327928+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:28:24.327928+0000","last_peered":"2024-09-16T00:28:24.327928+0000","last_clean":"2024-09-16T00:28:24.327928+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:28:24.327928+0000","last_undegraded":"2024-09-16T00:28:24.327928+0000","last_fullsized":"2024-09-16T00:28:24.327928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.333220+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:28:24.333220+0000","last_peered":"2024-09-16T00:28:24.333220+0000","last_clean":"2024-09-16T00:28:24.333220+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:28:24.333220+0000","last_undegraded":"2024-09-16T00:28:24.333220+0000","last_fullsized":"2024-09-16T00:28:24.333220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324496+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:28:24.324496+0000","last_peered":"2024-09-16T00:28:24.324496+0000","last_clean":"2024-09-16T00:28:24.324496+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:28:24.324496+0000","last_undegraded":"2024-09-16T00:28:24.324496+0000","last_fullsized":"2024-09-16T00:28:24.324496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334178+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:28:24.334178+0000","last_peered":"2024-09-16T00:28:24.334178+0000","last_clean":"2024-09-16T00:28:24.334178+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:28:24.334178+0000","last_undegraded":"2024-09-16T00:28:24.334178+0000","last_fullsized":"2024-09-16T00:28:24.334178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327306+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:28:24.327306+0000","last_peered":"2024-09-16T00:28:24.327306+0000","last_clean":"2024-09-16T00:28:24.327306+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:28:24.327306+0000","last_undegraded":"2024-09-16T00:28:24.327306+0000","last_fullsized":"2024-09-16T00:28:24.327306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.309954+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:28:24.309954+0000","last_peered":"2024-09-16T00:28:24.309954+0000","last_clean":"2024-09-16T00:28:24.309954+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:28:24.309954+0000","last_undegraded":"2024-09-16T00:28:24.309954+0000","last_fullsized":"2024-09-16T00:28:24.309954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320484+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:28:24.320484+0000","last_peered":"2024-09-16T00:28:24.320484+0000","last_clean":"2024-09-16T00:28:24.320484+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:28:24.320484+0000","last_undegraded":"2024-09-16T00:28:24.320484+0000","last_fullsized":"2024-09-16T00:28:24.320484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.318969+0000","last_change":"2024-09-16T00:27:26.358604+0000","last_active":"2024-09-16T00:28:24.318969+0000","last_peered":"2024-09-16T00:28:24.318969+0000","last_clean":"2024-09-16T00:28:24.318969+0000","last_became_active":"2024-09-16T00:27:26.358225+0000","last_became_peered":"2024-09-16T00:27:26.358225+0000","last_unstale":"2024-09-16T00:28:24.318969+0000","last_undegraded":"2024-09-16T00:28:24.318969+0000","last_fullsized":"2024-09-16T00:28:24.318969+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:21.806500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326711+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:28:24.326711+0000","last_peered":"2024-09-16T00:28:24.326711+0000","last_clean":"2024-09-16T00:28:24.326711+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:28:24.326711+0000","last_undegraded":"2024-09-16T00:28:24.326711+0000","last_fullsized":"2024-09-16T00:28:24.326711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336940+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:28:24.336940+0000","last_peered":"2024-09-16T00:28:24.336940+0000","last_clean":"2024-09-16T00:28:24.336940+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:28:24.336940+0000","last_undegraded":"2024-09-16T00:28:24.336940+0000","last_fullsized":"2024-09-16T00:28:24.336940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66777,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":626,"num_read_kb":386,"num_write":176,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":515002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1728512,"data_stored":1164903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906997,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":519334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":515158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364862,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1646592,"data_stored":1111210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528388,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1106798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":577389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":510608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":512516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":572649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":511802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:28:45.321 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:28:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.457+0000 7f972508d640 1 -- 172.21.15.1:0/959283917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 msgr2=0x7f9720109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.457+0000 7f972508d640 1 --2- 172.21.15.1:0/959283917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 0x7f9720109ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9718009ca0 tx=0x7f971802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.457+0000 7f972508d640 1 -- 172.21.15.1:0/959283917 shutdown_connections 2024-09-16T00:28:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.457+0000 7f972508d640 1 --2- 172.21.15.1:0/959283917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 0x7f9720109ce0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.457+0000 7f972508d640 1 --2- 172.21.15.1:0/959283917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720104fc0 0x7f97201073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.457+0000 7f972508d640 1 --2- 172.21.15.1:0/959283917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9720102690 0x7f9720104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.457+0000 7f972508d640 1 -- 172.21.15.1:0/959283917 >> 172.21.15.1:0/959283917 conn(0x7f97200fc190 msgr2=0x7f97200fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:45.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.457+0000 7f972508d640 1 -- 172.21.15.1:0/959283917 shutdown_connections 2024-09-16T00:28:45.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.457+0000 7f972508d640 1 -- 172.21.15.1:0/959283917 wait complete. 2024-09-16T00:28:45.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.458+0000 7f972508d640 1 Processor -- start 2024-09-16T00:28:45.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.458+0000 7f972508d640 1 -- start start 2024-09-16T00:28:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.458+0000 7f972508d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9720102690 0x7f9720102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.458+0000 7f972508d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720104fc0 0x7f9720100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.458+0000 7f972508d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 0x7f9720100b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.458+0000 7f972508d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f972010d800 con 0x7f97201078f0 2024-09-16T00:28:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.458+0000 7f972508d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f972010d680 con 0x7f9720104fc0 2024-09-16T00:28:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.458+0000 7f972508d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f972010d980 con 0x7f9720102690 2024-09-16T00:28:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 0x7f9720100b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 0x7f9720100b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60296/0 (socket says 172.21.15.1:60296) 2024-09-16T00:28:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971f59e640 1 -- 172.21.15.1:0/1867537617 learned_addr learned my addr 172.21.15.1:0/1867537617 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971ed9d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9720102690 0x7f9720102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971e59c640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720104fc0 0x7f9720100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971f59e640 1 -- 172.21.15.1:0/1867537617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9720102690 msgr2=0x7f9720102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971f59e640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9720102690 0x7f9720102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971f59e640 1 -- 172.21.15.1:0/1867537617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720104fc0 msgr2=0x7f9720100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971f59e640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720104fc0 0x7f9720100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971f59e640 1 -- 172.21.15.1:0/1867537617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9720101410 con 0x7f97201078f0 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971ed9d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9720102690 0x7f9720102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f971f59e640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 0x7f9720100b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9718031ae0 tx=0x7f9718031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f9707fff640 1 -- 172.21.15.1:0/1867537617 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718003b70 con 0x7f97201078f0 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f9707fff640 1 -- 172.21.15.1:0/1867537617 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9718003d10 con 0x7f97201078f0 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97201a1f80 con 0x7f97201078f0 2024-09-16T00:28:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f9707fff640 1 -- 172.21.15.1:0/1867537617 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971802fc40 con 0x7f97201078f0 2024-09-16T00:28:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.459+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97201a2440 con 0x7f97201078f0 2024-09-16T00:28:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.460+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97200fdc80 con 0x7f97201078f0 2024-09-16T00:28:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.462+0000 7f9707fff640 1 -- 172.21.15.1:0/1867537617 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9718004050 con 0x7f97201078f0 2024-09-16T00:28:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.462+0000 7f9707fff640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97080a32d0 0x7f97080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.462+0000 7f971ed9d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97080a32d0 0x7f97080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.462+0000 7f9707fff640 1 -- 172.21.15.1:0/1867537617 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f97180f0e30 con 0x7f97201078f0 2024-09-16T00:28:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.463+0000 7f971ed9d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97080a32d0 0x7f97080a5790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9710009c20 tx=0x7f9710005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:45.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.466+0000 7f9707fff640 1 -- 172.21.15.1:0/1867537617 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97180bd750 con 0x7f97201078f0 2024-09-16T00:28:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.640+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9720062630 con 0x7f97080a32d0 2024-09-16T00:28:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.648+0000 7f9707fff640 1 -- 172.21.15.1:0/1867537617 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340483 (secure 0 0 0) 0x7f9720062630 con 0x7f97080a32d0 2024-09-16T00:28:45.653 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:45.655 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:28:45.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97080a32d0 msgr2=0x7f97080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97080a32d0 0x7f97080a5790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9710009c20 tx=0x7f9710005f90 comp rx=0 tx=0).stop 2024-09-16T00:28:45.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 msgr2=0x7f9720100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 0x7f9720100b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9718031ae0 tx=0x7f9718031ee0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 shutdown_connections 2024-09-16T00:28:45.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97201078f0 0x7f9720100b80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720104fc0 0x7f9720100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97080a32d0 0x7f97080a5790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 --2- 172.21.15.1:0/1867537617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9720102690 0x7f9720102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 >> 172.21.15.1:0/1867537617 conn(0x7f97200fc190 msgr2=0x7f9720102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:45.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 shutdown_connections 2024-09-16T00:28:45.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.652+0000 7f972508d640 1 -- 172.21.15.1:0/1867537617 wait complete. 2024-09-16T00:28:45.665 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-16T00:28:44.346084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657379,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":752,"num_read_kb":495,"num_write":359,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380128,"kb_used_data":14240,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561696,"statfs":{"total":1151940427776,"available":1151551176704,"internally_reserved":0,"allocated":14581760,"data_stored":8127783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003579"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252865+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:27:26.252865+0000","last_peered":"2024-09-16T00:27:26.252865+0000","last_clean":"2024-09-16T00:27:26.252865+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:27:26.252865+0000","last_undegraded":"2024-09-16T00:27:26.252865+0000","last_fullsized":"2024-09-16T00:27:26.252865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322537+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:28:24.322537+0000","last_peered":"2024-09-16T00:28:24.322537+0000","last_clean":"2024-09-16T00:28:24.322537+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:28:24.322537+0000","last_undegraded":"2024-09-16T00:28:24.322537+0000","last_fullsized":"2024-09-16T00:28:24.322537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248105+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:27:26.248105+0000","last_peered":"2024-09-16T00:27:26.248105+0000","last_clean":"2024-09-16T00:27:26.248105+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:27:26.248105+0000","last_undegraded":"2024-09-16T00:27:26.248105+0000","last_fullsized":"2024-09-16T00:27:26.248105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247830+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:27:26.247830+0000","last_peered":"2024-09-16T00:27:26.247830+0000","last_clean":"2024-09-16T00:27:26.247830+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:27:26.247830+0000","last_undegraded":"2024-09-16T00:27:26.247830+0000","last_fullsized":"2024-09-16T00:27:26.247830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336305+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:28:24.336305+0000","last_peered":"2024-09-16T00:28:24.336305+0000","last_clean":"2024-09-16T00:28:24.336305+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:28:24.336305+0000","last_undegraded":"2024-09-16T00:28:24.336305+0000","last_fullsized":"2024-09-16T00:28:24.336305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355288+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:27:26.355288+0000","last_peered":"2024-09-16T00:27:26.355288+0000","last_clean":"2024-09-16T00:27:26.355288+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:27:26.355288+0000","last_undegraded":"2024-09-16T00:27:26.355288+0000","last_fullsized":"2024-09-16T00:27:26.355288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357791+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:26.357791+0000","last_peered":"2024-09-16T00:27:26.357791+0000","last_clean":"2024-09-16T00:27:26.357791+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:26.357791+0000","last_undegraded":"2024-09-16T00:27:26.357791+0000","last_fullsized":"2024-09-16T00:27:26.357791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.335269+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:28:24.335269+0000","last_peered":"2024-09-16T00:28:24.335269+0000","last_clean":"2024-09-16T00:28:24.335269+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:28:24.335269+0000","last_undegraded":"2024-09-16T00:28:24.335269+0000","last_fullsized":"2024-09-16T00:28:24.335269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356501+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:27:26.356501+0000","last_peered":"2024-09-16T00:27:26.356501+0000","last_clean":"2024-09-16T00:27:26.356501+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:27:26.356501+0000","last_undegraded":"2024-09-16T00:27:26.356501+0000","last_fullsized":"2024-09-16T00:27:26.356501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252350+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:27:26.252350+0000","last_peered":"2024-09-16T00:27:26.252350+0000","last_clean":"2024-09-16T00:27:26.252350+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:27:26.252350+0000","last_undegraded":"2024-09-16T00:27:26.252350+0000","last_fullsized":"2024-09-16T00:27:26.252350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336415+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:28:24.336415+0000","last_peered":"2024-09-16T00:28:24.336415+0000","last_clean":"2024-09-16T00:28:24.336415+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:28:24.336415+0000","last_undegraded":"2024-09-16T00:28:24.336415+0000","last_fullsized":"2024-09-16T00:28:24.336415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355852+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:27:26.355852+0000","last_peered":"2024-09-16T00:27:26.355852+0000","last_clean":"2024-09-16T00:27:26.355852+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:27:26.355852+0000","last_undegraded":"2024-09-16T00:27:26.355852+0000","last_fullsized":"2024-09-16T00:27:26.355852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338941+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:28:24.338941+0000","last_peered":"2024-09-16T00:28:24.338941+0000","last_clean":"2024-09-16T00:28:24.338941+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:28:24.338941+0000","last_undegraded":"2024-09-16T00:28:24.338941+0000","last_fullsized":"2024-09-16T00:28:24.338941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357104+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:26.357104+0000","last_peered":"2024-09-16T00:27:26.357104+0000","last_clean":"2024-09-16T00:27:26.357104+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:26.357104+0000","last_undegraded":"2024-09-16T00:27:26.357104+0000","last_fullsized":"2024-09-16T00:27:26.357104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356176+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:27:26.356176+0000","last_peered":"2024-09-16T00:27:26.356176+0000","last_clean":"2024-09-16T00:27:26.356176+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:27:26.356176+0000","last_undegraded":"2024-09-16T00:27:26.356176+0000","last_fullsized":"2024-09-16T00:27:26.356176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327188+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:28:24.327188+0000","last_peered":"2024-09-16T00:28:24.327188+0000","last_clean":"2024-09-16T00:28:24.327188+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:28:24.327188+0000","last_undegraded":"2024-09-16T00:28:24.327188+0000","last_fullsized":"2024-09-16T00:28:24.327188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248306+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:27:26.248306+0000","last_peered":"2024-09-16T00:27:26.248306+0000","last_clean":"2024-09-16T00:27:26.248306+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:27:26.248306+0000","last_undegraded":"2024-09-16T00:27:26.248306+0000","last_fullsized":"2024-09-16T00:27:26.248306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356287+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:27:26.356287+0000","last_peered":"2024-09-16T00:27:26.356287+0000","last_clean":"2024-09-16T00:27:26.356287+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:27:26.356287+0000","last_undegraded":"2024-09-16T00:27:26.356287+0000","last_fullsized":"2024-09-16T00:27:26.356287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324477+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:28:24.324477+0000","last_peered":"2024-09-16T00:28:24.324477+0000","last_clean":"2024-09-16T00:28:24.324477+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:28:24.324477+0000","last_undegraded":"2024-09-16T00:28:24.324477+0000","last_fullsized":"2024-09-16T00:28:24.324477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247518+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:26.247518+0000","last_peered":"2024-09-16T00:27:26.247518+0000","last_clean":"2024-09-16T00:27:26.247518+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:26.247518+0000","last_undegraded":"2024-09-16T00:27:26.247518+0000","last_fullsized":"2024-09-16T00:27:26.247518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252497+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:27:26.252497+0000","last_peered":"2024-09-16T00:27:26.252497+0000","last_clean":"2024-09-16T00:27:26.252497+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:27:26.252497+0000","last_undegraded":"2024-09-16T00:27:26.252497+0000","last_fullsized":"2024-09-16T00:27:26.252497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.312122+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:28:24.312122+0000","last_peered":"2024-09-16T00:28:24.312122+0000","last_clean":"2024-09-16T00:28:24.312122+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:28:24.312122+0000","last_undegraded":"2024-09-16T00:28:24.312122+0000","last_fullsized":"2024-09-16T00:28:24.312122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355055+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:26.355055+0000","last_peered":"2024-09-16T00:27:26.355055+0000","last_clean":"2024-09-16T00:27:26.355055+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:26.355055+0000","last_undegraded":"2024-09-16T00:27:26.355055+0000","last_fullsized":"2024-09-16T00:27:26.355055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357331+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:27:26.357331+0000","last_peered":"2024-09-16T00:27:26.357331+0000","last_clean":"2024-09-16T00:27:26.357331+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:27:26.357331+0000","last_undegraded":"2024-09-16T00:27:26.357331+0000","last_fullsized":"2024-09-16T00:27:26.357331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334895+0000","last_change":"2024-09-16T00:27:26.358639+0000","last_active":"2024-09-16T00:28:24.334895+0000","last_peered":"2024-09-16T00:28:24.334895+0000","last_clean":"2024-09-16T00:28:24.334895+0000","last_became_active":"2024-09-16T00:27:26.358163+0000","last_became_peered":"2024-09-16T00:27:26.358163+0000","last_unstale":"2024-09-16T00:28:24.334895+0000","last_undegraded":"2024-09-16T00:28:24.334895+0000","last_fullsized":"2024-09-16T00:28:24.334895+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:11.811436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356716+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:26.356716+0000","last_peered":"2024-09-16T00:27:26.356716+0000","last_clean":"2024-09-16T00:27:26.356716+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:26.356716+0000","last_undegraded":"2024-09-16T00:27:26.356716+0000","last_fullsized":"2024-09-16T00:27:26.356716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248397+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:27:26.248397+0000","last_peered":"2024-09-16T00:27:26.248397+0000","last_clean":"2024-09-16T00:27:26.248397+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:27:26.248397+0000","last_undegraded":"2024-09-16T00:27:26.248397+0000","last_fullsized":"2024-09-16T00:27:26.248397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332074+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:28:24.332074+0000","last_peered":"2024-09-16T00:28:24.332074+0000","last_clean":"2024-09-16T00:28:24.332074+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:28:24.332074+0000","last_undegraded":"2024-09-16T00:28:24.332074+0000","last_fullsized":"2024-09-16T00:28:24.332074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329964+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:28:24.329964+0000","last_peered":"2024-09-16T00:28:24.329964+0000","last_clean":"2024-09-16T00:28:24.329964+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:28:24.329964+0000","last_undegraded":"2024-09-16T00:28:24.329964+0000","last_fullsized":"2024-09-16T00:28:24.329964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247207+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:27:26.247207+0000","last_peered":"2024-09-16T00:27:26.247207+0000","last_clean":"2024-09-16T00:27:26.247207+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:27:26.247207+0000","last_undegraded":"2024-09-16T00:27:26.247207+0000","last_fullsized":"2024-09-16T00:27:26.247207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.325854+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:28:24.325854+0000","last_peered":"2024-09-16T00:28:24.325854+0000","last_clean":"2024-09-16T00:28:24.325854+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:28:24.325854+0000","last_undegraded":"2024-09-16T00:28:24.325854+0000","last_fullsized":"2024-09-16T00:28:24.325854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357960+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:26.357960+0000","last_peered":"2024-09-16T00:27:26.357960+0000","last_clean":"2024-09-16T00:27:26.357960+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:26.357960+0000","last_undegraded":"2024-09-16T00:27:26.357960+0000","last_fullsized":"2024-09-16T00:27:26.357960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251785+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:27:26.251785+0000","last_peered":"2024-09-16T00:27:26.251785+0000","last_clean":"2024-09-16T00:27:26.251785+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:27:26.251785+0000","last_undegraded":"2024-09-16T00:27:26.251785+0000","last_fullsized":"2024-09-16T00:27:26.251785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329144+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:28:24.329144+0000","last_peered":"2024-09-16T00:28:24.329144+0000","last_clean":"2024-09-16T00:28:24.329144+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:28:24.329144+0000","last_undegraded":"2024-09-16T00:28:24.329144+0000","last_fullsized":"2024-09-16T00:28:24.329144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252455+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:27:26.252455+0000","last_peered":"2024-09-16T00:27:26.252455+0000","last_clean":"2024-09-16T00:27:26.252455+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:27:26.252455+0000","last_undegraded":"2024-09-16T00:27:26.252455+0000","last_fullsized":"2024-09-16T00:27:26.252455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357048+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:27:26.357048+0000","last_peered":"2024-09-16T00:27:26.357048+0000","last_clean":"2024-09-16T00:27:26.357048+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:27:26.357048+0000","last_undegraded":"2024-09-16T00:27:26.357048+0000","last_fullsized":"2024-09-16T00:27:26.357048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321720+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:28:24.321720+0000","last_peered":"2024-09-16T00:28:24.321720+0000","last_clean":"2024-09-16T00:28:24.321720+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:28:24.321720+0000","last_undegraded":"2024-09-16T00:28:24.321720+0000","last_fullsized":"2024-09-16T00:28:24.321720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357297+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:27:26.357297+0000","last_peered":"2024-09-16T00:27:26.357297+0000","last_clean":"2024-09-16T00:27:26.357297+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:27:26.357297+0000","last_undegraded":"2024-09-16T00:27:26.357297+0000","last_fullsized":"2024-09-16T00:27:26.357297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250069+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:27:26.250069+0000","last_peered":"2024-09-16T00:27:26.250069+0000","last_clean":"2024-09-16T00:27:26.250069+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:27:26.250069+0000","last_undegraded":"2024-09-16T00:27:26.250069+0000","last_fullsized":"2024-09-16T00:27:26.250069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.305108+0000","last_change":"2024-09-16T00:27:26.357501+0000","last_active":"2024-09-16T00:28:24.305108+0000","last_peered":"2024-09-16T00:28:24.305108+0000","last_clean":"2024-09-16T00:28:24.305108+0000","last_became_active":"2024-09-16T00:27:26.357108+0000","last_became_peered":"2024-09-16T00:27:26.357108+0000","last_unstale":"2024-09-16T00:28:24.305108+0000","last_undegraded":"2024-09-16T00:28:24.305108+0000","last_fullsized":"2024-09-16T00:28:24.305108+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:25.264059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":140,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:39.661319+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:28:39.661319+0000","last_peered":"2024-09-16T00:28:39.661319+0000","last_clean":"2024-09-16T00:28:39.661319+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:28:39.661319+0000","last_undegraded":"2024-09-16T00:28:39.661319+0000","last_fullsized":"2024-09-16T00:28:39.661319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250857+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:27:26.250857+0000","last_peered":"2024-09-16T00:27:26.250857+0000","last_clean":"2024-09-16T00:27:26.250857+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:27:26.250857+0000","last_undegraded":"2024-09-16T00:27:26.250857+0000","last_fullsized":"2024-09-16T00:27:26.250857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326885+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:28:24.326885+0000","last_peered":"2024-09-16T00:28:24.326885+0000","last_clean":"2024-09-16T00:28:24.326885+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:28:24.326885+0000","last_undegraded":"2024-09-16T00:28:24.326885+0000","last_fullsized":"2024-09-16T00:28:24.326885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357096+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:26.357096+0000","last_peered":"2024-09-16T00:27:26.357096+0000","last_clean":"2024-09-16T00:27:26.357096+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:26.357096+0000","last_undegraded":"2024-09-16T00:27:26.357096+0000","last_fullsized":"2024-09-16T00:27:26.357096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356120+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:27:26.356120+0000","last_peered":"2024-09-16T00:27:26.356120+0000","last_clean":"2024-09-16T00:27:26.356120+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:27:26.356120+0000","last_undegraded":"2024-09-16T00:27:26.356120+0000","last_fullsized":"2024-09-16T00:27:26.356120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336576+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:28:24.336576+0000","last_peered":"2024-09-16T00:28:24.336576+0000","last_clean":"2024-09-16T00:28:24.336576+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:28:24.336576+0000","last_undegraded":"2024-09-16T00:28:24.336576+0000","last_fullsized":"2024-09-16T00:28:24.336576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357054+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:26.357054+0000","last_peered":"2024-09-16T00:27:26.357054+0000","last_clean":"2024-09-16T00:27:26.357054+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:26.357054+0000","last_undegraded":"2024-09-16T00:27:26.357054+0000","last_fullsized":"2024-09-16T00:27:26.357054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357365+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:27:26.357365+0000","last_peered":"2024-09-16T00:27:26.357365+0000","last_clean":"2024-09-16T00:27:26.357365+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:27:26.357365+0000","last_undegraded":"2024-09-16T00:27:26.357365+0000","last_fullsized":"2024-09-16T00:27:26.357365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355730+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:27:26.355730+0000","last_peered":"2024-09-16T00:27:26.355730+0000","last_clean":"2024-09-16T00:27:26.355730+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:27:26.355730+0000","last_undegraded":"2024-09-16T00:27:26.355730+0000","last_fullsized":"2024-09-16T00:27:26.355730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338444+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:28:24.338444+0000","last_peered":"2024-09-16T00:28:24.338444+0000","last_clean":"2024-09-16T00:28:24.338444+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:28:24.338444+0000","last_undegraded":"2024-09-16T00:28:24.338444+0000","last_fullsized":"2024-09-16T00:28:24.338444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248838+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:26.248838+0000","last_peered":"2024-09-16T00:27:26.248838+0000","last_clean":"2024-09-16T00:27:26.248838+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:26.248838+0000","last_undegraded":"2024-09-16T00:27:26.248838+0000","last_fullsized":"2024-09-16T00:27:26.248838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355174+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:27:26.355174+0000","last_peered":"2024-09-16T00:27:26.355174+0000","last_clean":"2024-09-16T00:27:26.355174+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:27:26.355174+0000","last_undegraded":"2024-09-16T00:27:26.355174+0000","last_fullsized":"2024-09-16T00:27:26.355174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331484+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:28:24.331484+0000","last_peered":"2024-09-16T00:28:24.331484+0000","last_clean":"2024-09-16T00:28:24.331484+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:28:24.331484+0000","last_undegraded":"2024-09-16T00:28:24.331484+0000","last_fullsized":"2024-09-16T00:28:24.331484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356422+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:27:26.356422+0000","last_peered":"2024-09-16T00:27:26.356422+0000","last_clean":"2024-09-16T00:27:26.356422+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:27:26.356422+0000","last_undegraded":"2024-09-16T00:27:26.356422+0000","last_fullsized":"2024-09-16T00:27:26.356422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249539+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:27:26.249539+0000","last_peered":"2024-09-16T00:27:26.249539+0000","last_clean":"2024-09-16T00:27:26.249539+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:27:26.249539+0000","last_undegraded":"2024-09-16T00:27:26.249539+0000","last_fullsized":"2024-09-16T00:27:26.249539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.333565+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:28:24.333565+0000","last_peered":"2024-09-16T00:28:24.333565+0000","last_clean":"2024-09-16T00:28:24.333565+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:28:24.333565+0000","last_undegraded":"2024-09-16T00:28:24.333565+0000","last_fullsized":"2024-09-16T00:28:24.333565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250975+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:27:26.250975+0000","last_peered":"2024-09-16T00:27:26.250975+0000","last_clean":"2024-09-16T00:27:26.250975+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:27:26.250975+0000","last_undegraded":"2024-09-16T00:27:26.250975+0000","last_fullsized":"2024-09-16T00:27:26.250975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354985+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:27:26.354985+0000","last_peered":"2024-09-16T00:27:26.354985+0000","last_clean":"2024-09-16T00:27:26.354985+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:27:26.354985+0000","last_undegraded":"2024-09-16T00:27:26.354985+0000","last_fullsized":"2024-09-16T00:27:26.354985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.319032+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:28:24.319032+0000","last_peered":"2024-09-16T00:28:24.319032+0000","last_clean":"2024-09-16T00:28:24.319032+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:28:24.319032+0000","last_undegraded":"2024-09-16T00:28:24.319032+0000","last_fullsized":"2024-09-16T00:28:24.319032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357916+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:26.357916+0000","last_peered":"2024-09-16T00:27:26.357916+0000","last_clean":"2024-09-16T00:27:26.357916+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:26.357916+0000","last_undegraded":"2024-09-16T00:27:26.357916+0000","last_fullsized":"2024-09-16T00:27:26.357916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355183+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:27:26.355183+0000","last_peered":"2024-09-16T00:27:26.355183+0000","last_clean":"2024-09-16T00:27:26.355183+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:27:26.355183+0000","last_undegraded":"2024-09-16T00:27:26.355183+0000","last_fullsized":"2024-09-16T00:27:26.355183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336143+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:28:24.336143+0000","last_peered":"2024-09-16T00:28:24.336143+0000","last_clean":"2024-09-16T00:28:24.336143+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:28:24.336143+0000","last_undegraded":"2024-09-16T00:28:24.336143+0000","last_fullsized":"2024-09-16T00:28:24.336143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356998+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:26.356998+0000","last_peered":"2024-09-16T00:27:26.356998+0000","last_clean":"2024-09-16T00:27:26.356998+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:26.356998+0000","last_undegraded":"2024-09-16T00:27:26.356998+0000","last_fullsized":"2024-09-16T00:27:26.356998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253824+0000","last_change":"2024-09-16T00:27:26.253824+0000","last_active":"2024-09-16T00:27:26.253824+0000","last_peered":"2024-09-16T00:27:26.253824+0000","last_clean":"2024-09-16T00:27:26.253824+0000","last_became_active":"2024-09-16T00:27:26.253206+0000","last_became_peered":"2024-09-16T00:27:26.253206+0000","last_unstale":"2024-09-16T00:27:26.253824+0000","last_undegraded":"2024-09-16T00:27:26.253824+0000","last_fullsized":"2024-09-16T00:27:26.253824+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:22.543369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332839+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:28:24.332839+0000","last_peered":"2024-09-16T00:28:24.332839+0000","last_clean":"2024-09-16T00:28:24.332839+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:28:24.332839+0000","last_undegraded":"2024-09-16T00:28:24.332839+0000","last_fullsized":"2024-09-16T00:28:24.332839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":140,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:39.661782+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:28:39.661782+0000","last_peered":"2024-09-16T00:28:39.661782+0000","last_clean":"2024-09-16T00:28:39.661782+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:28:39.661782+0000","last_undegraded":"2024-09-16T00:28:39.661782+0000","last_fullsized":"2024-09-16T00:28:39.661782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252554+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:27:26.252554+0000","last_peered":"2024-09-16T00:27:26.252554+0000","last_clean":"2024-09-16T00:27:26.252554+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:27:26.252554+0000","last_undegraded":"2024-09-16T00:27:26.252554+0000","last_fullsized":"2024-09-16T00:27:26.252554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.335224+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:28:24.335224+0000","last_peered":"2024-09-16T00:28:24.335224+0000","last_clean":"2024-09-16T00:28:24.335224+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:28:24.335224+0000","last_undegraded":"2024-09-16T00:28:24.335224+0000","last_fullsized":"2024-09-16T00:28:24.335224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356320+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:26.356320+0000","last_peered":"2024-09-16T00:27:26.356320+0000","last_clean":"2024-09-16T00:27:26.356320+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:26.356320+0000","last_undegraded":"2024-09-16T00:27:26.356320+0000","last_fullsized":"2024-09-16T00:27:26.356320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252536+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:27:26.252536+0000","last_peered":"2024-09-16T00:27:26.252536+0000","last_clean":"2024-09-16T00:27:26.252536+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:27:26.252536+0000","last_undegraded":"2024-09-16T00:27:26.252536+0000","last_fullsized":"2024-09-16T00:27:26.252536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330126+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:28:24.330126+0000","last_peered":"2024-09-16T00:28:24.330126+0000","last_clean":"2024-09-16T00:28:24.330126+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:28:24.330126+0000","last_undegraded":"2024-09-16T00:28:24.330126+0000","last_fullsized":"2024-09-16T00:28:24.330126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248474+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:26.248474+0000","last_peered":"2024-09-16T00:27:26.248474+0000","last_clean":"2024-09-16T00:27:26.248474+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:26.248474+0000","last_undegraded":"2024-09-16T00:27:26.248474+0000","last_fullsized":"2024-09-16T00:27:26.248474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355941+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:27:26.355941+0000","last_peered":"2024-09-16T00:27:26.355941+0000","last_clean":"2024-09-16T00:27:26.355941+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:27:26.355941+0000","last_undegraded":"2024-09-16T00:27:26.355941+0000","last_fullsized":"2024-09-16T00:27:26.355941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.310802+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:28:24.310802+0000","last_peered":"2024-09-16T00:28:24.310802+0000","last_clean":"2024-09-16T00:28:24.310802+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:28:24.310802+0000","last_undegraded":"2024-09-16T00:28:24.310802+0000","last_fullsized":"2024-09-16T00:28:24.310802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":301,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:41.667628+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:28:41.667628+0000","last_peered":"2024-09-16T00:28:41.667628+0000","last_clean":"2024-09-16T00:28:41.667628+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:28:41.667628+0000","last_undegraded":"2024-09-16T00:28:41.667628+0000","last_fullsized":"2024-09-16T00:28:41.667628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62633,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":211,"num_read_kb":167,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.358804+0000","last_change":"2024-09-16T00:27:26.358804+0000","last_active":"2024-09-16T00:27:26.358804+0000","last_peered":"2024-09-16T00:27:26.358804+0000","last_clean":"2024-09-16T00:27:26.358804+0000","last_became_active":"2024-09-16T00:27:26.358317+0000","last_became_peered":"2024-09-16T00:27:26.358317+0000","last_unstale":"2024-09-16T00:27:26.358804+0000","last_undegraded":"2024-09-16T00:27:26.358804+0000","last_fullsized":"2024-09-16T00:27:26.358804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:38.642392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336469+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:28:24.336469+0000","last_peered":"2024-09-16T00:28:24.336469+0000","last_clean":"2024-09-16T00:28:24.336469+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:28:24.336469+0000","last_undegraded":"2024-09-16T00:28:24.336469+0000","last_fullsized":"2024-09-16T00:28:24.336469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356252+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:26.356252+0000","last_peered":"2024-09-16T00:27:26.356252+0000","last_clean":"2024-09-16T00:27:26.356252+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:26.356252+0000","last_undegraded":"2024-09-16T00:27:26.356252+0000","last_fullsized":"2024-09-16T00:27:26.356252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356754+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:27:26.356754+0000","last_peered":"2024-09-16T00:27:26.356754+0000","last_clean":"2024-09-16T00:27:26.356754+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:27:26.356754+0000","last_undegraded":"2024-09-16T00:27:26.356754+0000","last_fullsized":"2024-09-16T00:27:26.356754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320427+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:28:24.320427+0000","last_peered":"2024-09-16T00:28:24.320427+0000","last_clean":"2024-09-16T00:28:24.320427+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:28:24.320427+0000","last_undegraded":"2024-09-16T00:28:24.320427+0000","last_fullsized":"2024-09-16T00:28:24.320427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357185+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:26.357185+0000","last_peered":"2024-09-16T00:27:26.357185+0000","last_clean":"2024-09-16T00:27:26.357185+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:26.357185+0000","last_undegraded":"2024-09-16T00:27:26.357185+0000","last_fullsized":"2024-09-16T00:27:26.357185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252128+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:27:26.252128+0000","last_peered":"2024-09-16T00:27:26.252128+0000","last_clean":"2024-09-16T00:27:26.252128+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:27:26.252128+0000","last_undegraded":"2024-09-16T00:27:26.252128+0000","last_fullsized":"2024-09-16T00:27:26.252128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336828+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:28:24.336828+0000","last_peered":"2024-09-16T00:28:24.336828+0000","last_clean":"2024-09-16T00:28:24.336828+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:28:24.336828+0000","last_undegraded":"2024-09-16T00:28:24.336828+0000","last_fullsized":"2024-09-16T00:28:24.336828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":142,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:41.667633+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:28:41.667633+0000","last_peered":"2024-09-16T00:28:41.667633+0000","last_clean":"2024-09-16T00:28:41.667633+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:28:41.667633+0000","last_undegraded":"2024-09-16T00:28:41.667633+0000","last_fullsized":"2024-09-16T00:28:41.667633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247274+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:27:26.247274+0000","last_peered":"2024-09-16T00:27:26.247274+0000","last_clean":"2024-09-16T00:27:26.247274+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:27:26.247274+0000","last_undegraded":"2024-09-16T00:27:26.247274+0000","last_fullsized":"2024-09-16T00:27:26.247274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250057+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:27:26.250057+0000","last_peered":"2024-09-16T00:27:26.250057+0000","last_clean":"2024-09-16T00:27:26.250057+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:27:26.250057+0000","last_undegraded":"2024-09-16T00:27:26.250057+0000","last_fullsized":"2024-09-16T00:27:26.250057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331295+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:28:24.331295+0000","last_peered":"2024-09-16T00:28:24.331295+0000","last_clean":"2024-09-16T00:28:24.331295+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:28:24.331295+0000","last_undegraded":"2024-09-16T00:28:24.331295+0000","last_fullsized":"2024-09-16T00:28:24.331295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355360+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:27:26.355360+0000","last_peered":"2024-09-16T00:27:26.355360+0000","last_clean":"2024-09-16T00:27:26.355360+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:27:26.355360+0000","last_undegraded":"2024-09-16T00:27:26.355360+0000","last_fullsized":"2024-09-16T00:27:26.355360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.360047+0000","last_change":"2024-09-16T00:27:26.359230+0000","last_active":"2024-09-16T00:27:26.360047+0000","last_peered":"2024-09-16T00:27:26.360047+0000","last_clean":"2024-09-16T00:27:26.360047+0000","last_became_active":"2024-09-16T00:27:26.358662+0000","last_became_peered":"2024-09-16T00:27:26.358662+0000","last_unstale":"2024-09-16T00:27:26.360047+0000","last_undegraded":"2024-09-16T00:27:26.360047+0000","last_fullsized":"2024-09-16T00:27:26.360047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:52.111910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331664+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:28:24.331664+0000","last_peered":"2024-09-16T00:28:24.331664+0000","last_clean":"2024-09-16T00:28:24.331664+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:28:24.331664+0000","last_undegraded":"2024-09-16T00:28:24.331664+0000","last_fullsized":"2024-09-16T00:28:24.331664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250512+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:26.250512+0000","last_peered":"2024-09-16T00:27:26.250512+0000","last_clean":"2024-09-16T00:27:26.250512+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:26.250512+0000","last_undegraded":"2024-09-16T00:27:26.250512+0000","last_fullsized":"2024-09-16T00:27:26.250512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250892+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:26.250892+0000","last_peered":"2024-09-16T00:27:26.250892+0000","last_clean":"2024-09-16T00:27:26.250892+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:26.250892+0000","last_undegraded":"2024-09-16T00:27:26.250892+0000","last_fullsized":"2024-09-16T00:27:26.250892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332482+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:28:24.332482+0000","last_peered":"2024-09-16T00:28:24.332482+0000","last_clean":"2024-09-16T00:28:24.332482+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:28:24.332482+0000","last_undegraded":"2024-09-16T00:28:24.332482+0000","last_fullsized":"2024-09-16T00:28:24.332482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359360+0000","last_change":"2024-09-16T00:27:26.359360+0000","last_active":"2024-09-16T00:27:26.359360+0000","last_peered":"2024-09-16T00:27:26.359360+0000","last_clean":"2024-09-16T00:27:26.359360+0000","last_became_active":"2024-09-16T00:27:26.358462+0000","last_became_peered":"2024-09-16T00:27:26.358462+0000","last_unstale":"2024-09-16T00:27:26.359360+0000","last_undegraded":"2024-09-16T00:27:26.359360+0000","last_fullsized":"2024-09-16T00:27:26.359360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:38.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:38.662940+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:28:38.662940+0000","last_peered":"2024-09-16T00:28:38.662940+0000","last_clean":"2024-09-16T00:28:38.662940+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:28:38.662940+0000","last_undegraded":"2024-09-16T00:28:38.662940+0000","last_fullsized":"2024-09-16T00:28:38.662940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320803+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:28:24.320803+0000","last_peered":"2024-09-16T00:28:24.320803+0000","last_clean":"2024-09-16T00:28:24.320803+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:28:24.320803+0000","last_undegraded":"2024-09-16T00:28:24.320803+0000","last_fullsized":"2024-09-16T00:28:24.320803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248757+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:27:26.248757+0000","last_peered":"2024-09-16T00:27:26.248757+0000","last_clean":"2024-09-16T00:27:26.248757+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:27:26.248757+0000","last_undegraded":"2024-09-16T00:27:26.248757+0000","last_fullsized":"2024-09-16T00:27:26.248757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321717+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:28:24.321717+0000","last_peered":"2024-09-16T00:28:24.321717+0000","last_clean":"2024-09-16T00:28:24.321717+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:28:24.321717+0000","last_undegraded":"2024-09-16T00:28:24.321717+0000","last_fullsized":"2024-09-16T00:28:24.321717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321331+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:28:24.321331+0000","last_peered":"2024-09-16T00:28:24.321331+0000","last_clean":"2024-09-16T00:28:24.321331+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:28:24.321331+0000","last_undegraded":"2024-09-16T00:28:24.321331+0000","last_fullsized":"2024-09-16T00:28:24.321331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329213+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:28:24.329213+0000","last_peered":"2024-09-16T00:28:24.329213+0000","last_clean":"2024-09-16T00:28:24.329213+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:28:24.329213+0000","last_undegraded":"2024-09-16T00:28:24.329213+0000","last_fullsized":"2024-09-16T00:28:24.329213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326265+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:28:24.326265+0000","last_peered":"2024-09-16T00:28:24.326265+0000","last_clean":"2024-09-16T00:28:24.326265+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:28:24.326265+0000","last_undegraded":"2024-09-16T00:28:24.326265+0000","last_fullsized":"2024-09-16T00:28:24.326265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327220+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:28:24.327220+0000","last_peered":"2024-09-16T00:28:24.327220+0000","last_clean":"2024-09-16T00:28:24.327220+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:28:24.327220+0000","last_undegraded":"2024-09-16T00:28:24.327220+0000","last_fullsized":"2024-09-16T00:28:24.327220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.316594+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:28:24.316594+0000","last_peered":"2024-09-16T00:28:24.316594+0000","last_clean":"2024-09-16T00:28:24.316594+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:28:24.316594+0000","last_undegraded":"2024-09-16T00:28:24.316594+0000","last_fullsized":"2024-09-16T00:28:24.316594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337588+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:28:24.337588+0000","last_peered":"2024-09-16T00:28:24.337588+0000","last_clean":"2024-09-16T00:28:24.337588+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:28:24.337588+0000","last_undegraded":"2024-09-16T00:28:24.337588+0000","last_fullsized":"2024-09-16T00:28:24.337588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322996+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:28:24.322996+0000","last_peered":"2024-09-16T00:28:24.322996+0000","last_clean":"2024-09-16T00:28:24.322996+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:28:24.322996+0000","last_undegraded":"2024-09-16T00:28:24.322996+0000","last_fullsized":"2024-09-16T00:28:24.322996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322343+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:28:24.322343+0000","last_peered":"2024-09-16T00:28:24.322343+0000","last_clean":"2024-09-16T00:28:24.322343+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:28:24.322343+0000","last_undegraded":"2024-09-16T00:28:24.322343+0000","last_fullsized":"2024-09-16T00:28:24.322343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330356+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:28:24.330356+0000","last_peered":"2024-09-16T00:28:24.330356+0000","last_clean":"2024-09-16T00:28:24.330356+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:28:24.330356+0000","last_undegraded":"2024-09-16T00:28:24.330356+0000","last_fullsized":"2024-09-16T00:28:24.330356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337238+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:28:24.337238+0000","last_peered":"2024-09-16T00:28:24.337238+0000","last_clean":"2024-09-16T00:28:24.337238+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:28:24.337238+0000","last_undegraded":"2024-09-16T00:28:24.337238+0000","last_fullsized":"2024-09-16T00:28:24.337238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330609+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:28:24.330609+0000","last_peered":"2024-09-16T00:28:24.330609+0000","last_clean":"2024-09-16T00:28:24.330609+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:28:24.330609+0000","last_undegraded":"2024-09-16T00:28:24.330609+0000","last_fullsized":"2024-09-16T00:28:24.330609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.317884+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:28:24.317884+0000","last_peered":"2024-09-16T00:28:24.317884+0000","last_clean":"2024-09-16T00:28:24.317884+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:28:24.317884+0000","last_undegraded":"2024-09-16T00:28:24.317884+0000","last_fullsized":"2024-09-16T00:28:24.317884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329300+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:28:24.329300+0000","last_peered":"2024-09-16T00:28:24.329300+0000","last_clean":"2024-09-16T00:28:24.329300+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:28:24.329300+0000","last_undegraded":"2024-09-16T00:28:24.329300+0000","last_fullsized":"2024-09-16T00:28:24.329300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337159+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:28:24.337159+0000","last_peered":"2024-09-16T00:28:24.337159+0000","last_clean":"2024-09-16T00:28:24.337159+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:28:24.337159+0000","last_undegraded":"2024-09-16T00:28:24.337159+0000","last_fullsized":"2024-09-16T00:28:24.337159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336933+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:28:24.336933+0000","last_peered":"2024-09-16T00:28:24.336933+0000","last_clean":"2024-09-16T00:28:24.336933+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:28:24.336933+0000","last_undegraded":"2024-09-16T00:28:24.336933+0000","last_fullsized":"2024-09-16T00:28:24.336933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337483+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:28:24.337483+0000","last_peered":"2024-09-16T00:28:24.337483+0000","last_clean":"2024-09-16T00:28:24.337483+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:28:24.337483+0000","last_undegraded":"2024-09-16T00:28:24.337483+0000","last_fullsized":"2024-09-16T00:28:24.337483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326582+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:28:24.326582+0000","last_peered":"2024-09-16T00:28:24.326582+0000","last_clean":"2024-09-16T00:28:24.326582+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:28:24.326582+0000","last_undegraded":"2024-09-16T00:28:24.326582+0000","last_fullsized":"2024-09-16T00:28:24.326582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326496+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:28:24.326496+0000","last_peered":"2024-09-16T00:28:24.326496+0000","last_clean":"2024-09-16T00:28:24.326496+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:28:24.326496+0000","last_undegraded":"2024-09-16T00:28:24.326496+0000","last_fullsized":"2024-09-16T00:28:24.326496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322821+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:28:24.322821+0000","last_peered":"2024-09-16T00:28:24.322821+0000","last_clean":"2024-09-16T00:28:24.322821+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:28:24.322821+0000","last_undegraded":"2024-09-16T00:28:24.322821+0000","last_fullsized":"2024-09-16T00:28:24.322821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.319623+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:28:24.319623+0000","last_peered":"2024-09-16T00:28:24.319623+0000","last_clean":"2024-09-16T00:28:24.319623+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:28:24.319623+0000","last_undegraded":"2024-09-16T00:28:24.319623+0000","last_fullsized":"2024-09-16T00:28:24.319623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338146+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:28:24.338146+0000","last_peered":"2024-09-16T00:28:24.338146+0000","last_clean":"2024-09-16T00:28:24.338146+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:28:24.338146+0000","last_undegraded":"2024-09-16T00:28:24.338146+0000","last_fullsized":"2024-09-16T00:28:24.338146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327928+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:28:24.327928+0000","last_peered":"2024-09-16T00:28:24.327928+0000","last_clean":"2024-09-16T00:28:24.327928+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:28:24.327928+0000","last_undegraded":"2024-09-16T00:28:24.327928+0000","last_fullsized":"2024-09-16T00:28:24.327928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.333220+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:28:24.333220+0000","last_peered":"2024-09-16T00:28:24.333220+0000","last_clean":"2024-09-16T00:28:24.333220+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:28:24.333220+0000","last_undegraded":"2024-09-16T00:28:24.333220+0000","last_fullsized":"2024-09-16T00:28:24.333220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324496+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:28:24.324496+0000","last_peered":"2024-09-16T00:28:24.324496+0000","last_clean":"2024-09-16T00:28:24.324496+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:28:24.324496+0000","last_undegraded":"2024-09-16T00:28:24.324496+0000","last_fullsized":"2024-09-16T00:28:24.324496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334178+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:28:24.334178+0000","last_peered":"2024-09-16T00:28:24.334178+0000","last_clean":"2024-09-16T00:28:24.334178+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:28:24.334178+0000","last_undegraded":"2024-09-16T00:28:24.334178+0000","last_fullsized":"2024-09-16T00:28:24.334178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327306+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:28:24.327306+0000","last_peered":"2024-09-16T00:28:24.327306+0000","last_clean":"2024-09-16T00:28:24.327306+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:28:24.327306+0000","last_undegraded":"2024-09-16T00:28:24.327306+0000","last_fullsized":"2024-09-16T00:28:24.327306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.309954+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:28:24.309954+0000","last_peered":"2024-09-16T00:28:24.309954+0000","last_clean":"2024-09-16T00:28:24.309954+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:28:24.309954+0000","last_undegraded":"2024-09-16T00:28:24.309954+0000","last_fullsized":"2024-09-16T00:28:24.309954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320484+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:28:24.320484+0000","last_peered":"2024-09-16T00:28:24.320484+0000","last_clean":"2024-09-16T00:28:24.320484+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:28:24.320484+0000","last_undegraded":"2024-09-16T00:28:24.320484+0000","last_fullsized":"2024-09-16T00:28:24.320484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.318969+0000","last_change":"2024-09-16T00:27:26.358604+0000","last_active":"2024-09-16T00:28:24.318969+0000","last_peered":"2024-09-16T00:28:24.318969+0000","last_clean":"2024-09-16T00:28:24.318969+0000","last_became_active":"2024-09-16T00:27:26.358225+0000","last_became_peered":"2024-09-16T00:27:26.358225+0000","last_unstale":"2024-09-16T00:28:24.318969+0000","last_undegraded":"2024-09-16T00:28:24.318969+0000","last_fullsized":"2024-09-16T00:28:24.318969+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:21.806500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326711+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:28:24.326711+0000","last_peered":"2024-09-16T00:28:24.326711+0000","last_clean":"2024-09-16T00:28:24.326711+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:28:24.326711+0000","last_undegraded":"2024-09-16T00:28:24.326711+0000","last_fullsized":"2024-09-16T00:28:24.326711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336940+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:28:24.336940+0000","last_peered":"2024-09-16T00:28:24.336940+0000","last_clean":"2024-09-16T00:28:24.336940+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:28:24.336940+0000","last_undegraded":"2024-09-16T00:28:24.336940+0000","last_fullsized":"2024-09-16T00:28:24.336940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66777,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":626,"num_read_kb":386,"num_write":176,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":515002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1728512,"data_stored":1164903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906997,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":519334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":515158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364862,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1646592,"data_stored":1111210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528388,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1106798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":577389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":510608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":512516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":572649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":511802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:28:45.671 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-16T00:28:45.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:28:45.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.809+0000 7ff3b19bd640 1 -- 172.21.15.1:0/3669997911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 msgr2=0x7ff3ac101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.809+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/3669997911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 0x7ff3ac101f40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009900 tx=0x7ff39c02a410 comp rx=0 tx=0).stop 2024-09-16T00:28:45.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.809+0000 7ff3b19bd640 1 -- 172.21.15.1:0/3669997911 shutdown_connections 2024-09-16T00:28:45.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.809+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/3669997911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1036f0 0x7ff3ac109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.809+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/3669997911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3ac102d30 0x7ff3ac1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.809+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/3669997911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 0x7ff3ac101f40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.809+0000 7ff3b19bd640 1 -- 172.21.15.1:0/3669997911 >> 172.21.15.1:0/3669997911 conn(0x7ff3ac0fd2e0 msgr2=0x7ff3ac0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:45.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.809+0000 7ff3b19bd640 1 -- 172.21.15.1:0/3669997911 shutdown_connections 2024-09-16T00:28:45.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.809+0000 7ff3b19bd640 1 -- 172.21.15.1:0/3669997911 wait complete. 2024-09-16T00:28:45.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3b19bd640 1 Processor -- start 2024-09-16T00:28:45.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3b19bd640 1 -- start start 2024-09-16T00:28:45.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3b19bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 0x7ff3ac197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3b19bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac102d30 0x7ff3ac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3b19bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3ac1036f0 0x7ff3ac19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3b19bd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10d820 con 0x7ff3ac101b40 2024-09-16T00:28:45.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3b19bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10d6a0 con 0x7ff3ac102d30 2024-09-16T00:28:45.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3b19bd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10d9a0 con 0x7ff3ac1036f0 2024-09-16T00:28:45.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3aaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 0x7ff3ac197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3aaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 0x7ff3ac197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60326/0 (socket says 172.21.15.1:60326) 2024-09-16T00:28:45.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3aaffd640 1 -- 172.21.15.1:0/4094325162 learned_addr learned my addr 172.21.15.1:0/4094325162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:45.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3ab7fe640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3ac1036f0 0x7ff3ac19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3a3fff640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac102d30 0x7ff3ac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3aaffd640 1 -- 172.21.15.1:0/4094325162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3ac1036f0 msgr2=0x7ff3ac19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3aaffd640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3ac1036f0 0x7ff3ac19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3aaffd640 1 -- 172.21.15.1:0/4094325162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac102d30 msgr2=0x7ff3ac198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.810+0000 7ff3aaffd640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac102d30 0x7ff3ac198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.811+0000 7ff3aaffd640 1 -- 172.21.15.1:0/4094325162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac19fb30 con 0x7ff3ac101b40 2024-09-16T00:28:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.811+0000 7ff3a3fff640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac102d30 0x7ff3ac198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.811+0000 7ff3aaffd640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 0x7ff3ac197f10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff39c031610 tx=0x7ff39c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.811+0000 7ff3b09bb640 1 -- 172.21.15.1:0/4094325162 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c03f030 con 0x7ff3ac101b40 2024-09-16T00:28:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.811+0000 7ff3b09bb640 1 -- 172.21.15.1:0/4094325162 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39c003ce0 con 0x7ff3ac101b40 2024-09-16T00:28:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.811+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac19fdc0 con 0x7ff3ac101b40 2024-09-16T00:28:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.811+0000 7ff3b09bb640 1 -- 172.21.15.1:0/4094325162 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c035070 con 0x7ff3ac101b40 2024-09-16T00:28:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.811+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac1a0270 con 0x7ff3ac101b40 2024-09-16T00:28:45.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.812+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ac108f80 con 0x7ff3ac101b40 2024-09-16T00:28:45.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.813+0000 7ff3b09bb640 1 -- 172.21.15.1:0/4094325162 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff39c042030 con 0x7ff3ac101b40 2024-09-16T00:28:45.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.814+0000 7ff3b09bb640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3980a3250 0x7ff3980a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:45.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.814+0000 7ff3b09bb640 1 -- 172.21.15.1:0/4094325162 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff39c0f03d0 con 0x7ff3ac101b40 2024-09-16T00:28:45.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.814+0000 7ff3a3fff640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3980a3250 0x7ff3980a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:45.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.814+0000 7ff3a3fff640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3980a3250 0x7ff3980a5710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac199420 tx=0x7ff394008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:45.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.818+0000 7ff3b09bb640 1 -- 172.21.15.1:0/4094325162 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff39c0bcd90 con 0x7ff3ac101b40 2024-09-16T00:28:45.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:45 smithi001 ceph-mon[30934]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:28:45.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:45 smithi001 ceph-mon[30934]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:45.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:45 smithi153 ceph-mon[35538]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:28:45.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:45 smithi153 ceph-mon[35538]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:45.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:45.992+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3ac062630 con 0x7ff3980a3250 2024-09-16T00:28:46.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.000+0000 7ff3b09bb640 1 -- 172.21.15.1:0/4094325162 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340483 (secure 0 0 0) 0x7ff3ac062630 con 0x7ff3980a3250 2024-09-16T00:28:46.005 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:46.007 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:28:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3980a3250 msgr2=0x7ff3980a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3980a3250 0x7ff3980a5710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac199420 tx=0x7ff394008040 comp rx=0 tx=0).stop 2024-09-16T00:28:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 msgr2=0x7ff3ac197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 0x7ff3ac197f10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff39c031610 tx=0x7ff39c0043b0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 shutdown_connections 2024-09-16T00:28:46.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff3980a3250 0x7ff3980a5710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3ac1036f0 0x7ff3ac19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac102d30 0x7ff3ac198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 --2- 172.21.15.1:0/4094325162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3ac101b40 0x7ff3ac197f10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 >> 172.21.15.1:0/4094325162 conn(0x7ff3ac0fd2e0 msgr2=0x7ff3ac106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:46.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 shutdown_connections 2024-09-16T00:28:46.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.004+0000 7ff3b19bd640 1 -- 172.21.15.1:0/4094325162 wait complete. 2024-09-16T00:28:46.016 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-16T00:28:44.346084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657379,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":752,"num_read_kb":495,"num_write":359,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380128,"kb_used_data":14240,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561696,"statfs":{"total":1151940427776,"available":1151551176704,"internally_reserved":0,"allocated":14581760,"data_stored":8127783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003579"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252865+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:27:26.252865+0000","last_peered":"2024-09-16T00:27:26.252865+0000","last_clean":"2024-09-16T00:27:26.252865+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:27:26.252865+0000","last_undegraded":"2024-09-16T00:27:26.252865+0000","last_fullsized":"2024-09-16T00:27:26.252865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322537+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:28:24.322537+0000","last_peered":"2024-09-16T00:28:24.322537+0000","last_clean":"2024-09-16T00:28:24.322537+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:28:24.322537+0000","last_undegraded":"2024-09-16T00:28:24.322537+0000","last_fullsized":"2024-09-16T00:28:24.322537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248105+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:27:26.248105+0000","last_peered":"2024-09-16T00:27:26.248105+0000","last_clean":"2024-09-16T00:27:26.248105+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:27:26.248105+0000","last_undegraded":"2024-09-16T00:27:26.248105+0000","last_fullsized":"2024-09-16T00:27:26.248105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247830+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:27:26.247830+0000","last_peered":"2024-09-16T00:27:26.247830+0000","last_clean":"2024-09-16T00:27:26.247830+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:27:26.247830+0000","last_undegraded":"2024-09-16T00:27:26.247830+0000","last_fullsized":"2024-09-16T00:27:26.247830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336305+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:28:24.336305+0000","last_peered":"2024-09-16T00:28:24.336305+0000","last_clean":"2024-09-16T00:28:24.336305+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:28:24.336305+0000","last_undegraded":"2024-09-16T00:28:24.336305+0000","last_fullsized":"2024-09-16T00:28:24.336305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355288+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:27:26.355288+0000","last_peered":"2024-09-16T00:27:26.355288+0000","last_clean":"2024-09-16T00:27:26.355288+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:27:26.355288+0000","last_undegraded":"2024-09-16T00:27:26.355288+0000","last_fullsized":"2024-09-16T00:27:26.355288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357791+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:26.357791+0000","last_peered":"2024-09-16T00:27:26.357791+0000","last_clean":"2024-09-16T00:27:26.357791+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:26.357791+0000","last_undegraded":"2024-09-16T00:27:26.357791+0000","last_fullsized":"2024-09-16T00:27:26.357791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.335269+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:28:24.335269+0000","last_peered":"2024-09-16T00:28:24.335269+0000","last_clean":"2024-09-16T00:28:24.335269+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:28:24.335269+0000","last_undegraded":"2024-09-16T00:28:24.335269+0000","last_fullsized":"2024-09-16T00:28:24.335269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356501+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:27:26.356501+0000","last_peered":"2024-09-16T00:27:26.356501+0000","last_clean":"2024-09-16T00:27:26.356501+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:27:26.356501+0000","last_undegraded":"2024-09-16T00:27:26.356501+0000","last_fullsized":"2024-09-16T00:27:26.356501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252350+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:27:26.252350+0000","last_peered":"2024-09-16T00:27:26.252350+0000","last_clean":"2024-09-16T00:27:26.252350+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:27:26.252350+0000","last_undegraded":"2024-09-16T00:27:26.252350+0000","last_fullsized":"2024-09-16T00:27:26.252350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336415+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:28:24.336415+0000","last_peered":"2024-09-16T00:28:24.336415+0000","last_clean":"2024-09-16T00:28:24.336415+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:28:24.336415+0000","last_undegraded":"2024-09-16T00:28:24.336415+0000","last_fullsized":"2024-09-16T00:28:24.336415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355852+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:27:26.355852+0000","last_peered":"2024-09-16T00:27:26.355852+0000","last_clean":"2024-09-16T00:27:26.355852+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:27:26.355852+0000","last_undegraded":"2024-09-16T00:27:26.355852+0000","last_fullsized":"2024-09-16T00:27:26.355852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338941+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:28:24.338941+0000","last_peered":"2024-09-16T00:28:24.338941+0000","last_clean":"2024-09-16T00:28:24.338941+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:28:24.338941+0000","last_undegraded":"2024-09-16T00:28:24.338941+0000","last_fullsized":"2024-09-16T00:28:24.338941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357104+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:26.357104+0000","last_peered":"2024-09-16T00:27:26.357104+0000","last_clean":"2024-09-16T00:27:26.357104+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:26.357104+0000","last_undegraded":"2024-09-16T00:27:26.357104+0000","last_fullsized":"2024-09-16T00:27:26.357104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356176+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:27:26.356176+0000","last_peered":"2024-09-16T00:27:26.356176+0000","last_clean":"2024-09-16T00:27:26.356176+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:27:26.356176+0000","last_undegraded":"2024-09-16T00:27:26.356176+0000","last_fullsized":"2024-09-16T00:27:26.356176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327188+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:28:24.327188+0000","last_peered":"2024-09-16T00:28:24.327188+0000","last_clean":"2024-09-16T00:28:24.327188+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:28:24.327188+0000","last_undegraded":"2024-09-16T00:28:24.327188+0000","last_fullsized":"2024-09-16T00:28:24.327188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248306+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:27:26.248306+0000","last_peered":"2024-09-16T00:27:26.248306+0000","last_clean":"2024-09-16T00:27:26.248306+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:27:26.248306+0000","last_undegraded":"2024-09-16T00:27:26.248306+0000","last_fullsized":"2024-09-16T00:27:26.248306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356287+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:27:26.356287+0000","last_peered":"2024-09-16T00:27:26.356287+0000","last_clean":"2024-09-16T00:27:26.356287+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:27:26.356287+0000","last_undegraded":"2024-09-16T00:27:26.356287+0000","last_fullsized":"2024-09-16T00:27:26.356287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324477+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:28:24.324477+0000","last_peered":"2024-09-16T00:28:24.324477+0000","last_clean":"2024-09-16T00:28:24.324477+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:28:24.324477+0000","last_undegraded":"2024-09-16T00:28:24.324477+0000","last_fullsized":"2024-09-16T00:28:24.324477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247518+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:26.247518+0000","last_peered":"2024-09-16T00:27:26.247518+0000","last_clean":"2024-09-16T00:27:26.247518+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:26.247518+0000","last_undegraded":"2024-09-16T00:27:26.247518+0000","last_fullsized":"2024-09-16T00:27:26.247518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252497+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:27:26.252497+0000","last_peered":"2024-09-16T00:27:26.252497+0000","last_clean":"2024-09-16T00:27:26.252497+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:27:26.252497+0000","last_undegraded":"2024-09-16T00:27:26.252497+0000","last_fullsized":"2024-09-16T00:27:26.252497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.312122+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:28:24.312122+0000","last_peered":"2024-09-16T00:28:24.312122+0000","last_clean":"2024-09-16T00:28:24.312122+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:28:24.312122+0000","last_undegraded":"2024-09-16T00:28:24.312122+0000","last_fullsized":"2024-09-16T00:28:24.312122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355055+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:26.355055+0000","last_peered":"2024-09-16T00:27:26.355055+0000","last_clean":"2024-09-16T00:27:26.355055+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:26.355055+0000","last_undegraded":"2024-09-16T00:27:26.355055+0000","last_fullsized":"2024-09-16T00:27:26.355055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357331+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:27:26.357331+0000","last_peered":"2024-09-16T00:27:26.357331+0000","last_clean":"2024-09-16T00:27:26.357331+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:27:26.357331+0000","last_undegraded":"2024-09-16T00:27:26.357331+0000","last_fullsized":"2024-09-16T00:27:26.357331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334895+0000","last_change":"2024-09-16T00:27:26.358639+0000","last_active":"2024-09-16T00:28:24.334895+0000","last_peered":"2024-09-16T00:28:24.334895+0000","last_clean":"2024-09-16T00:28:24.334895+0000","last_became_active":"2024-09-16T00:27:26.358163+0000","last_became_peered":"2024-09-16T00:27:26.358163+0000","last_unstale":"2024-09-16T00:28:24.334895+0000","last_undegraded":"2024-09-16T00:28:24.334895+0000","last_fullsized":"2024-09-16T00:28:24.334895+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:11.811436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356716+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:26.356716+0000","last_peered":"2024-09-16T00:27:26.356716+0000","last_clean":"2024-09-16T00:27:26.356716+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:26.356716+0000","last_undegraded":"2024-09-16T00:27:26.356716+0000","last_fullsized":"2024-09-16T00:27:26.356716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248397+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:27:26.248397+0000","last_peered":"2024-09-16T00:27:26.248397+0000","last_clean":"2024-09-16T00:27:26.248397+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:27:26.248397+0000","last_undegraded":"2024-09-16T00:27:26.248397+0000","last_fullsized":"2024-09-16T00:27:26.248397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332074+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:28:24.332074+0000","last_peered":"2024-09-16T00:28:24.332074+0000","last_clean":"2024-09-16T00:28:24.332074+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:28:24.332074+0000","last_undegraded":"2024-09-16T00:28:24.332074+0000","last_fullsized":"2024-09-16T00:28:24.332074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329964+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:28:24.329964+0000","last_peered":"2024-09-16T00:28:24.329964+0000","last_clean":"2024-09-16T00:28:24.329964+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:28:24.329964+0000","last_undegraded":"2024-09-16T00:28:24.329964+0000","last_fullsized":"2024-09-16T00:28:24.329964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247207+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:27:26.247207+0000","last_peered":"2024-09-16T00:27:26.247207+0000","last_clean":"2024-09-16T00:27:26.247207+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:27:26.247207+0000","last_undegraded":"2024-09-16T00:27:26.247207+0000","last_fullsized":"2024-09-16T00:27:26.247207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.325854+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:28:24.325854+0000","last_peered":"2024-09-16T00:28:24.325854+0000","last_clean":"2024-09-16T00:28:24.325854+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:28:24.325854+0000","last_undegraded":"2024-09-16T00:28:24.325854+0000","last_fullsized":"2024-09-16T00:28:24.325854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357960+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:26.357960+0000","last_peered":"2024-09-16T00:27:26.357960+0000","last_clean":"2024-09-16T00:27:26.357960+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:26.357960+0000","last_undegraded":"2024-09-16T00:27:26.357960+0000","last_fullsized":"2024-09-16T00:27:26.357960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251785+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:27:26.251785+0000","last_peered":"2024-09-16T00:27:26.251785+0000","last_clean":"2024-09-16T00:27:26.251785+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:27:26.251785+0000","last_undegraded":"2024-09-16T00:27:26.251785+0000","last_fullsized":"2024-09-16T00:27:26.251785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329144+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:28:24.329144+0000","last_peered":"2024-09-16T00:28:24.329144+0000","last_clean":"2024-09-16T00:28:24.329144+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:28:24.329144+0000","last_undegraded":"2024-09-16T00:28:24.329144+0000","last_fullsized":"2024-09-16T00:28:24.329144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252455+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:27:26.252455+0000","last_peered":"2024-09-16T00:27:26.252455+0000","last_clean":"2024-09-16T00:27:26.252455+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:27:26.252455+0000","last_undegraded":"2024-09-16T00:27:26.252455+0000","last_fullsized":"2024-09-16T00:27:26.252455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357048+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:27:26.357048+0000","last_peered":"2024-09-16T00:27:26.357048+0000","last_clean":"2024-09-16T00:27:26.357048+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:27:26.357048+0000","last_undegraded":"2024-09-16T00:27:26.357048+0000","last_fullsized":"2024-09-16T00:27:26.357048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321720+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:28:24.321720+0000","last_peered":"2024-09-16T00:28:24.321720+0000","last_clean":"2024-09-16T00:28:24.321720+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:28:24.321720+0000","last_undegraded":"2024-09-16T00:28:24.321720+0000","last_fullsized":"2024-09-16T00:28:24.321720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357297+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:27:26.357297+0000","last_peered":"2024-09-16T00:27:26.357297+0000","last_clean":"2024-09-16T00:27:26.357297+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:27:26.357297+0000","last_undegraded":"2024-09-16T00:27:26.357297+0000","last_fullsized":"2024-09-16T00:27:26.357297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250069+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:27:26.250069+0000","last_peered":"2024-09-16T00:27:26.250069+0000","last_clean":"2024-09-16T00:27:26.250069+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:27:26.250069+0000","last_undegraded":"2024-09-16T00:27:26.250069+0000","last_fullsized":"2024-09-16T00:27:26.250069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.305108+0000","last_change":"2024-09-16T00:27:26.357501+0000","last_active":"2024-09-16T00:28:24.305108+0000","last_peered":"2024-09-16T00:28:24.305108+0000","last_clean":"2024-09-16T00:28:24.305108+0000","last_became_active":"2024-09-16T00:27:26.357108+0000","last_became_peered":"2024-09-16T00:27:26.357108+0000","last_unstale":"2024-09-16T00:28:24.305108+0000","last_undegraded":"2024-09-16T00:28:24.305108+0000","last_fullsized":"2024-09-16T00:28:24.305108+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:25.264059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":140,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:39.661319+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:28:39.661319+0000","last_peered":"2024-09-16T00:28:39.661319+0000","last_clean":"2024-09-16T00:28:39.661319+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:28:39.661319+0000","last_undegraded":"2024-09-16T00:28:39.661319+0000","last_fullsized":"2024-09-16T00:28:39.661319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250857+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:27:26.250857+0000","last_peered":"2024-09-16T00:27:26.250857+0000","last_clean":"2024-09-16T00:27:26.250857+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:27:26.250857+0000","last_undegraded":"2024-09-16T00:27:26.250857+0000","last_fullsized":"2024-09-16T00:27:26.250857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326885+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:28:24.326885+0000","last_peered":"2024-09-16T00:28:24.326885+0000","last_clean":"2024-09-16T00:28:24.326885+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:28:24.326885+0000","last_undegraded":"2024-09-16T00:28:24.326885+0000","last_fullsized":"2024-09-16T00:28:24.326885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357096+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:26.357096+0000","last_peered":"2024-09-16T00:27:26.357096+0000","last_clean":"2024-09-16T00:27:26.357096+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:26.357096+0000","last_undegraded":"2024-09-16T00:27:26.357096+0000","last_fullsized":"2024-09-16T00:27:26.357096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356120+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:27:26.356120+0000","last_peered":"2024-09-16T00:27:26.356120+0000","last_clean":"2024-09-16T00:27:26.356120+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:27:26.356120+0000","last_undegraded":"2024-09-16T00:27:26.356120+0000","last_fullsized":"2024-09-16T00:27:26.356120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336576+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:28:24.336576+0000","last_peered":"2024-09-16T00:28:24.336576+0000","last_clean":"2024-09-16T00:28:24.336576+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:28:24.336576+0000","last_undegraded":"2024-09-16T00:28:24.336576+0000","last_fullsized":"2024-09-16T00:28:24.336576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357054+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:26.357054+0000","last_peered":"2024-09-16T00:27:26.357054+0000","last_clean":"2024-09-16T00:27:26.357054+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:26.357054+0000","last_undegraded":"2024-09-16T00:27:26.357054+0000","last_fullsized":"2024-09-16T00:27:26.357054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357365+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:27:26.357365+0000","last_peered":"2024-09-16T00:27:26.357365+0000","last_clean":"2024-09-16T00:27:26.357365+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:27:26.357365+0000","last_undegraded":"2024-09-16T00:27:26.357365+0000","last_fullsized":"2024-09-16T00:27:26.357365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355730+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:27:26.355730+0000","last_peered":"2024-09-16T00:27:26.355730+0000","last_clean":"2024-09-16T00:27:26.355730+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:27:26.355730+0000","last_undegraded":"2024-09-16T00:27:26.355730+0000","last_fullsized":"2024-09-16T00:27:26.355730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338444+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:28:24.338444+0000","last_peered":"2024-09-16T00:28:24.338444+0000","last_clean":"2024-09-16T00:28:24.338444+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:28:24.338444+0000","last_undegraded":"2024-09-16T00:28:24.338444+0000","last_fullsized":"2024-09-16T00:28:24.338444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248838+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:26.248838+0000","last_peered":"2024-09-16T00:27:26.248838+0000","last_clean":"2024-09-16T00:27:26.248838+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:26.248838+0000","last_undegraded":"2024-09-16T00:27:26.248838+0000","last_fullsized":"2024-09-16T00:27:26.248838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355174+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:27:26.355174+0000","last_peered":"2024-09-16T00:27:26.355174+0000","last_clean":"2024-09-16T00:27:26.355174+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:27:26.355174+0000","last_undegraded":"2024-09-16T00:27:26.355174+0000","last_fullsized":"2024-09-16T00:27:26.355174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331484+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:28:24.331484+0000","last_peered":"2024-09-16T00:28:24.331484+0000","last_clean":"2024-09-16T00:28:24.331484+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:28:24.331484+0000","last_undegraded":"2024-09-16T00:28:24.331484+0000","last_fullsized":"2024-09-16T00:28:24.331484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356422+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:27:26.356422+0000","last_peered":"2024-09-16T00:27:26.356422+0000","last_clean":"2024-09-16T00:27:26.356422+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:27:26.356422+0000","last_undegraded":"2024-09-16T00:27:26.356422+0000","last_fullsized":"2024-09-16T00:27:26.356422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249539+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:27:26.249539+0000","last_peered":"2024-09-16T00:27:26.249539+0000","last_clean":"2024-09-16T00:27:26.249539+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:27:26.249539+0000","last_undegraded":"2024-09-16T00:27:26.249539+0000","last_fullsized":"2024-09-16T00:27:26.249539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.333565+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:28:24.333565+0000","last_peered":"2024-09-16T00:28:24.333565+0000","last_clean":"2024-09-16T00:28:24.333565+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:28:24.333565+0000","last_undegraded":"2024-09-16T00:28:24.333565+0000","last_fullsized":"2024-09-16T00:28:24.333565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250975+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:27:26.250975+0000","last_peered":"2024-09-16T00:27:26.250975+0000","last_clean":"2024-09-16T00:27:26.250975+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:27:26.250975+0000","last_undegraded":"2024-09-16T00:27:26.250975+0000","last_fullsized":"2024-09-16T00:27:26.250975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354985+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:27:26.354985+0000","last_peered":"2024-09-16T00:27:26.354985+0000","last_clean":"2024-09-16T00:27:26.354985+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:27:26.354985+0000","last_undegraded":"2024-09-16T00:27:26.354985+0000","last_fullsized":"2024-09-16T00:27:26.354985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.319032+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:28:24.319032+0000","last_peered":"2024-09-16T00:28:24.319032+0000","last_clean":"2024-09-16T00:28:24.319032+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:28:24.319032+0000","last_undegraded":"2024-09-16T00:28:24.319032+0000","last_fullsized":"2024-09-16T00:28:24.319032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357916+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:26.357916+0000","last_peered":"2024-09-16T00:27:26.357916+0000","last_clean":"2024-09-16T00:27:26.357916+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:26.357916+0000","last_undegraded":"2024-09-16T00:27:26.357916+0000","last_fullsized":"2024-09-16T00:27:26.357916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355183+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:27:26.355183+0000","last_peered":"2024-09-16T00:27:26.355183+0000","last_clean":"2024-09-16T00:27:26.355183+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:27:26.355183+0000","last_undegraded":"2024-09-16T00:27:26.355183+0000","last_fullsized":"2024-09-16T00:27:26.355183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336143+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:28:24.336143+0000","last_peered":"2024-09-16T00:28:24.336143+0000","last_clean":"2024-09-16T00:28:24.336143+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:28:24.336143+0000","last_undegraded":"2024-09-16T00:28:24.336143+0000","last_fullsized":"2024-09-16T00:28:24.336143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356998+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:26.356998+0000","last_peered":"2024-09-16T00:27:26.356998+0000","last_clean":"2024-09-16T00:27:26.356998+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:26.356998+0000","last_undegraded":"2024-09-16T00:27:26.356998+0000","last_fullsized":"2024-09-16T00:27:26.356998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253824+0000","last_change":"2024-09-16T00:27:26.253824+0000","last_active":"2024-09-16T00:27:26.253824+0000","last_peered":"2024-09-16T00:27:26.253824+0000","last_clean":"2024-09-16T00:27:26.253824+0000","last_became_active":"2024-09-16T00:27:26.253206+0000","last_became_peered":"2024-09-16T00:27:26.253206+0000","last_unstale":"2024-09-16T00:27:26.253824+0000","last_undegraded":"2024-09-16T00:27:26.253824+0000","last_fullsized":"2024-09-16T00:27:26.253824+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:22.543369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332839+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:28:24.332839+0000","last_peered":"2024-09-16T00:28:24.332839+0000","last_clean":"2024-09-16T00:28:24.332839+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:28:24.332839+0000","last_undegraded":"2024-09-16T00:28:24.332839+0000","last_fullsized":"2024-09-16T00:28:24.332839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":140,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:39.661782+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:28:39.661782+0000","last_peered":"2024-09-16T00:28:39.661782+0000","last_clean":"2024-09-16T00:28:39.661782+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:28:39.661782+0000","last_undegraded":"2024-09-16T00:28:39.661782+0000","last_fullsized":"2024-09-16T00:28:39.661782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252554+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:27:26.252554+0000","last_peered":"2024-09-16T00:27:26.252554+0000","last_clean":"2024-09-16T00:27:26.252554+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:27:26.252554+0000","last_undegraded":"2024-09-16T00:27:26.252554+0000","last_fullsized":"2024-09-16T00:27:26.252554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.335224+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:28:24.335224+0000","last_peered":"2024-09-16T00:28:24.335224+0000","last_clean":"2024-09-16T00:28:24.335224+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:28:24.335224+0000","last_undegraded":"2024-09-16T00:28:24.335224+0000","last_fullsized":"2024-09-16T00:28:24.335224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356320+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:26.356320+0000","last_peered":"2024-09-16T00:27:26.356320+0000","last_clean":"2024-09-16T00:27:26.356320+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:26.356320+0000","last_undegraded":"2024-09-16T00:27:26.356320+0000","last_fullsized":"2024-09-16T00:27:26.356320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252536+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:27:26.252536+0000","last_peered":"2024-09-16T00:27:26.252536+0000","last_clean":"2024-09-16T00:27:26.252536+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:27:26.252536+0000","last_undegraded":"2024-09-16T00:27:26.252536+0000","last_fullsized":"2024-09-16T00:27:26.252536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330126+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:28:24.330126+0000","last_peered":"2024-09-16T00:28:24.330126+0000","last_clean":"2024-09-16T00:28:24.330126+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:28:24.330126+0000","last_undegraded":"2024-09-16T00:28:24.330126+0000","last_fullsized":"2024-09-16T00:28:24.330126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248474+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:26.248474+0000","last_peered":"2024-09-16T00:27:26.248474+0000","last_clean":"2024-09-16T00:27:26.248474+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:26.248474+0000","last_undegraded":"2024-09-16T00:27:26.248474+0000","last_fullsized":"2024-09-16T00:27:26.248474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355941+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:27:26.355941+0000","last_peered":"2024-09-16T00:27:26.355941+0000","last_clean":"2024-09-16T00:27:26.355941+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:27:26.355941+0000","last_undegraded":"2024-09-16T00:27:26.355941+0000","last_fullsized":"2024-09-16T00:27:26.355941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.310802+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:28:24.310802+0000","last_peered":"2024-09-16T00:28:24.310802+0000","last_clean":"2024-09-16T00:28:24.310802+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:28:24.310802+0000","last_undegraded":"2024-09-16T00:28:24.310802+0000","last_fullsized":"2024-09-16T00:28:24.310802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":301,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:41.667628+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:28:41.667628+0000","last_peered":"2024-09-16T00:28:41.667628+0000","last_clean":"2024-09-16T00:28:41.667628+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:28:41.667628+0000","last_undegraded":"2024-09-16T00:28:41.667628+0000","last_fullsized":"2024-09-16T00:28:41.667628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62633,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":211,"num_read_kb":167,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.358804+0000","last_change":"2024-09-16T00:27:26.358804+0000","last_active":"2024-09-16T00:27:26.358804+0000","last_peered":"2024-09-16T00:27:26.358804+0000","last_clean":"2024-09-16T00:27:26.358804+0000","last_became_active":"2024-09-16T00:27:26.358317+0000","last_became_peered":"2024-09-16T00:27:26.358317+0000","last_unstale":"2024-09-16T00:27:26.358804+0000","last_undegraded":"2024-09-16T00:27:26.358804+0000","last_fullsized":"2024-09-16T00:27:26.358804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:38.642392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336469+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:28:24.336469+0000","last_peered":"2024-09-16T00:28:24.336469+0000","last_clean":"2024-09-16T00:28:24.336469+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:28:24.336469+0000","last_undegraded":"2024-09-16T00:28:24.336469+0000","last_fullsized":"2024-09-16T00:28:24.336469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356252+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:26.356252+0000","last_peered":"2024-09-16T00:27:26.356252+0000","last_clean":"2024-09-16T00:27:26.356252+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:26.356252+0000","last_undegraded":"2024-09-16T00:27:26.356252+0000","last_fullsized":"2024-09-16T00:27:26.356252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356754+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:27:26.356754+0000","last_peered":"2024-09-16T00:27:26.356754+0000","last_clean":"2024-09-16T00:27:26.356754+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:27:26.356754+0000","last_undegraded":"2024-09-16T00:27:26.356754+0000","last_fullsized":"2024-09-16T00:27:26.356754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320427+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:28:24.320427+0000","last_peered":"2024-09-16T00:28:24.320427+0000","last_clean":"2024-09-16T00:28:24.320427+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:28:24.320427+0000","last_undegraded":"2024-09-16T00:28:24.320427+0000","last_fullsized":"2024-09-16T00:28:24.320427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357185+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:26.357185+0000","last_peered":"2024-09-16T00:27:26.357185+0000","last_clean":"2024-09-16T00:27:26.357185+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:26.357185+0000","last_undegraded":"2024-09-16T00:27:26.357185+0000","last_fullsized":"2024-09-16T00:27:26.357185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252128+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:27:26.252128+0000","last_peered":"2024-09-16T00:27:26.252128+0000","last_clean":"2024-09-16T00:27:26.252128+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:27:26.252128+0000","last_undegraded":"2024-09-16T00:27:26.252128+0000","last_fullsized":"2024-09-16T00:27:26.252128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336828+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:28:24.336828+0000","last_peered":"2024-09-16T00:28:24.336828+0000","last_clean":"2024-09-16T00:28:24.336828+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:28:24.336828+0000","last_undegraded":"2024-09-16T00:28:24.336828+0000","last_fullsized":"2024-09-16T00:28:24.336828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":142,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:41.667633+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:28:41.667633+0000","last_peered":"2024-09-16T00:28:41.667633+0000","last_clean":"2024-09-16T00:28:41.667633+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:28:41.667633+0000","last_undegraded":"2024-09-16T00:28:41.667633+0000","last_fullsized":"2024-09-16T00:28:41.667633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247274+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:27:26.247274+0000","last_peered":"2024-09-16T00:27:26.247274+0000","last_clean":"2024-09-16T00:27:26.247274+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:27:26.247274+0000","last_undegraded":"2024-09-16T00:27:26.247274+0000","last_fullsized":"2024-09-16T00:27:26.247274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250057+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:27:26.250057+0000","last_peered":"2024-09-16T00:27:26.250057+0000","last_clean":"2024-09-16T00:27:26.250057+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:27:26.250057+0000","last_undegraded":"2024-09-16T00:27:26.250057+0000","last_fullsized":"2024-09-16T00:27:26.250057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331295+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:28:24.331295+0000","last_peered":"2024-09-16T00:28:24.331295+0000","last_clean":"2024-09-16T00:28:24.331295+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:28:24.331295+0000","last_undegraded":"2024-09-16T00:28:24.331295+0000","last_fullsized":"2024-09-16T00:28:24.331295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355360+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:27:26.355360+0000","last_peered":"2024-09-16T00:27:26.355360+0000","last_clean":"2024-09-16T00:27:26.355360+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:27:26.355360+0000","last_undegraded":"2024-09-16T00:27:26.355360+0000","last_fullsized":"2024-09-16T00:27:26.355360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.360047+0000","last_change":"2024-09-16T00:27:26.359230+0000","last_active":"2024-09-16T00:27:26.360047+0000","last_peered":"2024-09-16T00:27:26.360047+0000","last_clean":"2024-09-16T00:27:26.360047+0000","last_became_active":"2024-09-16T00:27:26.358662+0000","last_became_peered":"2024-09-16T00:27:26.358662+0000","last_unstale":"2024-09-16T00:27:26.360047+0000","last_undegraded":"2024-09-16T00:27:26.360047+0000","last_fullsized":"2024-09-16T00:27:26.360047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:52.111910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331664+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:28:24.331664+0000","last_peered":"2024-09-16T00:28:24.331664+0000","last_clean":"2024-09-16T00:28:24.331664+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:28:24.331664+0000","last_undegraded":"2024-09-16T00:28:24.331664+0000","last_fullsized":"2024-09-16T00:28:24.331664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250512+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:26.250512+0000","last_peered":"2024-09-16T00:27:26.250512+0000","last_clean":"2024-09-16T00:27:26.250512+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:26.250512+0000","last_undegraded":"2024-09-16T00:27:26.250512+0000","last_fullsized":"2024-09-16T00:27:26.250512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250892+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:26.250892+0000","last_peered":"2024-09-16T00:27:26.250892+0000","last_clean":"2024-09-16T00:27:26.250892+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:26.250892+0000","last_undegraded":"2024-09-16T00:27:26.250892+0000","last_fullsized":"2024-09-16T00:27:26.250892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332482+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:28:24.332482+0000","last_peered":"2024-09-16T00:28:24.332482+0000","last_clean":"2024-09-16T00:28:24.332482+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:28:24.332482+0000","last_undegraded":"2024-09-16T00:28:24.332482+0000","last_fullsized":"2024-09-16T00:28:24.332482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359360+0000","last_change":"2024-09-16T00:27:26.359360+0000","last_active":"2024-09-16T00:27:26.359360+0000","last_peered":"2024-09-16T00:27:26.359360+0000","last_clean":"2024-09-16T00:27:26.359360+0000","last_became_active":"2024-09-16T00:27:26.358462+0000","last_became_peered":"2024-09-16T00:27:26.358462+0000","last_unstale":"2024-09-16T00:27:26.359360+0000","last_undegraded":"2024-09-16T00:27:26.359360+0000","last_fullsized":"2024-09-16T00:27:26.359360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:38.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:38.662940+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:28:38.662940+0000","last_peered":"2024-09-16T00:28:38.662940+0000","last_clean":"2024-09-16T00:28:38.662940+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:28:38.662940+0000","last_undegraded":"2024-09-16T00:28:38.662940+0000","last_fullsized":"2024-09-16T00:28:38.662940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320803+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:28:24.320803+0000","last_peered":"2024-09-16T00:28:24.320803+0000","last_clean":"2024-09-16T00:28:24.320803+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:28:24.320803+0000","last_undegraded":"2024-09-16T00:28:24.320803+0000","last_fullsized":"2024-09-16T00:28:24.320803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248757+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:27:26.248757+0000","last_peered":"2024-09-16T00:27:26.248757+0000","last_clean":"2024-09-16T00:27:26.248757+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:27:26.248757+0000","last_undegraded":"2024-09-16T00:27:26.248757+0000","last_fullsized":"2024-09-16T00:27:26.248757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321717+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:28:24.321717+0000","last_peered":"2024-09-16T00:28:24.321717+0000","last_clean":"2024-09-16T00:28:24.321717+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:28:24.321717+0000","last_undegraded":"2024-09-16T00:28:24.321717+0000","last_fullsized":"2024-09-16T00:28:24.321717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321331+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:28:24.321331+0000","last_peered":"2024-09-16T00:28:24.321331+0000","last_clean":"2024-09-16T00:28:24.321331+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:28:24.321331+0000","last_undegraded":"2024-09-16T00:28:24.321331+0000","last_fullsized":"2024-09-16T00:28:24.321331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329213+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:28:24.329213+0000","last_peered":"2024-09-16T00:28:24.329213+0000","last_clean":"2024-09-16T00:28:24.329213+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:28:24.329213+0000","last_undegraded":"2024-09-16T00:28:24.329213+0000","last_fullsized":"2024-09-16T00:28:24.329213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326265+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:28:24.326265+0000","last_peered":"2024-09-16T00:28:24.326265+0000","last_clean":"2024-09-16T00:28:24.326265+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:28:24.326265+0000","last_undegraded":"2024-09-16T00:28:24.326265+0000","last_fullsized":"2024-09-16T00:28:24.326265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327220+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:28:24.327220+0000","last_peered":"2024-09-16T00:28:24.327220+0000","last_clean":"2024-09-16T00:28:24.327220+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:28:24.327220+0000","last_undegraded":"2024-09-16T00:28:24.327220+0000","last_fullsized":"2024-09-16T00:28:24.327220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.316594+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:28:24.316594+0000","last_peered":"2024-09-16T00:28:24.316594+0000","last_clean":"2024-09-16T00:28:24.316594+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:28:24.316594+0000","last_undegraded":"2024-09-16T00:28:24.316594+0000","last_fullsized":"2024-09-16T00:28:24.316594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337588+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:28:24.337588+0000","last_peered":"2024-09-16T00:28:24.337588+0000","last_clean":"2024-09-16T00:28:24.337588+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:28:24.337588+0000","last_undegraded":"2024-09-16T00:28:24.337588+0000","last_fullsized":"2024-09-16T00:28:24.337588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322996+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:28:24.322996+0000","last_peered":"2024-09-16T00:28:24.322996+0000","last_clean":"2024-09-16T00:28:24.322996+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:28:24.322996+0000","last_undegraded":"2024-09-16T00:28:24.322996+0000","last_fullsized":"2024-09-16T00:28:24.322996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322343+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:28:24.322343+0000","last_peered":"2024-09-16T00:28:24.322343+0000","last_clean":"2024-09-16T00:28:24.322343+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:28:24.322343+0000","last_undegraded":"2024-09-16T00:28:24.322343+0000","last_fullsized":"2024-09-16T00:28:24.322343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330356+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:28:24.330356+0000","last_peered":"2024-09-16T00:28:24.330356+0000","last_clean":"2024-09-16T00:28:24.330356+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:28:24.330356+0000","last_undegraded":"2024-09-16T00:28:24.330356+0000","last_fullsized":"2024-09-16T00:28:24.330356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337238+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:28:24.337238+0000","last_peered":"2024-09-16T00:28:24.337238+0000","last_clean":"2024-09-16T00:28:24.337238+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:28:24.337238+0000","last_undegraded":"2024-09-16T00:28:24.337238+0000","last_fullsized":"2024-09-16T00:28:24.337238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330609+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:28:24.330609+0000","last_peered":"2024-09-16T00:28:24.330609+0000","last_clean":"2024-09-16T00:28:24.330609+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:28:24.330609+0000","last_undegraded":"2024-09-16T00:28:24.330609+0000","last_fullsized":"2024-09-16T00:28:24.330609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.317884+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:28:24.317884+0000","last_peered":"2024-09-16T00:28:24.317884+0000","last_clean":"2024-09-16T00:28:24.317884+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:28:24.317884+0000","last_undegraded":"2024-09-16T00:28:24.317884+0000","last_fullsized":"2024-09-16T00:28:24.317884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329300+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:28:24.329300+0000","last_peered":"2024-09-16T00:28:24.329300+0000","last_clean":"2024-09-16T00:28:24.329300+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:28:24.329300+0000","last_undegraded":"2024-09-16T00:28:24.329300+0000","last_fullsized":"2024-09-16T00:28:24.329300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337159+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:28:24.337159+0000","last_peered":"2024-09-16T00:28:24.337159+0000","last_clean":"2024-09-16T00:28:24.337159+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:28:24.337159+0000","last_undegraded":"2024-09-16T00:28:24.337159+0000","last_fullsized":"2024-09-16T00:28:24.337159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336933+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:28:24.336933+0000","last_peered":"2024-09-16T00:28:24.336933+0000","last_clean":"2024-09-16T00:28:24.336933+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:28:24.336933+0000","last_undegraded":"2024-09-16T00:28:24.336933+0000","last_fullsized":"2024-09-16T00:28:24.336933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337483+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:28:24.337483+0000","last_peered":"2024-09-16T00:28:24.337483+0000","last_clean":"2024-09-16T00:28:24.337483+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:28:24.337483+0000","last_undegraded":"2024-09-16T00:28:24.337483+0000","last_fullsized":"2024-09-16T00:28:24.337483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326582+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:28:24.326582+0000","last_peered":"2024-09-16T00:28:24.326582+0000","last_clean":"2024-09-16T00:28:24.326582+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:28:24.326582+0000","last_undegraded":"2024-09-16T00:28:24.326582+0000","last_fullsized":"2024-09-16T00:28:24.326582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326496+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:28:24.326496+0000","last_peered":"2024-09-16T00:28:24.326496+0000","last_clean":"2024-09-16T00:28:24.326496+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:28:24.326496+0000","last_undegraded":"2024-09-16T00:28:24.326496+0000","last_fullsized":"2024-09-16T00:28:24.326496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322821+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:28:24.322821+0000","last_peered":"2024-09-16T00:28:24.322821+0000","last_clean":"2024-09-16T00:28:24.322821+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:28:24.322821+0000","last_undegraded":"2024-09-16T00:28:24.322821+0000","last_fullsized":"2024-09-16T00:28:24.322821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.319623+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:28:24.319623+0000","last_peered":"2024-09-16T00:28:24.319623+0000","last_clean":"2024-09-16T00:28:24.319623+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:28:24.319623+0000","last_undegraded":"2024-09-16T00:28:24.319623+0000","last_fullsized":"2024-09-16T00:28:24.319623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338146+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:28:24.338146+0000","last_peered":"2024-09-16T00:28:24.338146+0000","last_clean":"2024-09-16T00:28:24.338146+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:28:24.338146+0000","last_undegraded":"2024-09-16T00:28:24.338146+0000","last_fullsized":"2024-09-16T00:28:24.338146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327928+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:28:24.327928+0000","last_peered":"2024-09-16T00:28:24.327928+0000","last_clean":"2024-09-16T00:28:24.327928+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:28:24.327928+0000","last_undegraded":"2024-09-16T00:28:24.327928+0000","last_fullsized":"2024-09-16T00:28:24.327928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.333220+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:28:24.333220+0000","last_peered":"2024-09-16T00:28:24.333220+0000","last_clean":"2024-09-16T00:28:24.333220+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:28:24.333220+0000","last_undegraded":"2024-09-16T00:28:24.333220+0000","last_fullsized":"2024-09-16T00:28:24.333220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324496+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:28:24.324496+0000","last_peered":"2024-09-16T00:28:24.324496+0000","last_clean":"2024-09-16T00:28:24.324496+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:28:24.324496+0000","last_undegraded":"2024-09-16T00:28:24.324496+0000","last_fullsized":"2024-09-16T00:28:24.324496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334178+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:28:24.334178+0000","last_peered":"2024-09-16T00:28:24.334178+0000","last_clean":"2024-09-16T00:28:24.334178+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:28:24.334178+0000","last_undegraded":"2024-09-16T00:28:24.334178+0000","last_fullsized":"2024-09-16T00:28:24.334178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327306+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:28:24.327306+0000","last_peered":"2024-09-16T00:28:24.327306+0000","last_clean":"2024-09-16T00:28:24.327306+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:28:24.327306+0000","last_undegraded":"2024-09-16T00:28:24.327306+0000","last_fullsized":"2024-09-16T00:28:24.327306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.309954+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:28:24.309954+0000","last_peered":"2024-09-16T00:28:24.309954+0000","last_clean":"2024-09-16T00:28:24.309954+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:28:24.309954+0000","last_undegraded":"2024-09-16T00:28:24.309954+0000","last_fullsized":"2024-09-16T00:28:24.309954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320484+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:28:24.320484+0000","last_peered":"2024-09-16T00:28:24.320484+0000","last_clean":"2024-09-16T00:28:24.320484+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:28:24.320484+0000","last_undegraded":"2024-09-16T00:28:24.320484+0000","last_fullsized":"2024-09-16T00:28:24.320484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.318969+0000","last_change":"2024-09-16T00:27:26.358604+0000","last_active":"2024-09-16T00:28:24.318969+0000","last_peered":"2024-09-16T00:28:24.318969+0000","last_clean":"2024-09-16T00:28:24.318969+0000","last_became_active":"2024-09-16T00:27:26.358225+0000","last_became_peered":"2024-09-16T00:27:26.358225+0000","last_unstale":"2024-09-16T00:28:24.318969+0000","last_undegraded":"2024-09-16T00:28:24.318969+0000","last_fullsized":"2024-09-16T00:28:24.318969+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:21.806500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326711+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:28:24.326711+0000","last_peered":"2024-09-16T00:28:24.326711+0000","last_clean":"2024-09-16T00:28:24.326711+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:28:24.326711+0000","last_undegraded":"2024-09-16T00:28:24.326711+0000","last_fullsized":"2024-09-16T00:28:24.326711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336940+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:28:24.336940+0000","last_peered":"2024-09-16T00:28:24.336940+0000","last_clean":"2024-09-16T00:28:24.336940+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:28:24.336940+0000","last_undegraded":"2024-09-16T00:28:24.336940+0000","last_fullsized":"2024-09-16T00:28:24.336940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66777,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":626,"num_read_kb":386,"num_write":176,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":515002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1728512,"data_stored":1164903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906997,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":519334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":515158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364862,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1646592,"data_stored":1111210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528388,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1106798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":577389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":510608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":512516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":572649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":511802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:28:46.025 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:45 smithi019 ceph-mon[35186]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:28:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:45 smithi019 ceph-mon[35186]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:46.315 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:46.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.802+0000 7f2ad2108640 1 -- 172.21.15.1:0/2195678461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 msgr2=0x7f2acc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:46.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.802+0000 7f2ad2108640 1 --2- 172.21.15.1:0/2195678461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 0x7f2acc1029b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009900 tx=0x7f2ac002a480 comp rx=0 tx=0).stop 2024-09-16T00:28:46.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.802+0000 7f2ad2108640 1 -- 172.21.15.1:0/2195678461 shutdown_connections 2024-09-16T00:28:46.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.802+0000 7f2ad2108640 1 --2- 172.21.15.1:0/2195678461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc102ef0 0x7f2acc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.802+0000 7f2ad2108640 1 --2- 172.21.15.1:0/2195678461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 0x7f2acc1029b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.802+0000 7f2ad2108640 1 --2- 172.21.15.1:0/2195678461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2acc100e20 0x7f2acc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.802+0000 7f2ad2108640 1 -- 172.21.15.1:0/2195678461 >> 172.21.15.1:0/2195678461 conn(0x7f2acc0fc7d0 msgr2=0x7f2acc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.802+0000 7f2ad2108640 1 -- 172.21.15.1:0/2195678461 shutdown_connections 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.802+0000 7f2ad2108640 1 -- 172.21.15.1:0/2195678461 wait complete. 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.803+0000 7f2ad2108640 1 Processor -- start 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.803+0000 7f2ad2108640 1 -- start start 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.803+0000 7f2ad2108640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc100e20 0x7f2acc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2ad2108640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 0x7f2acc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2ad2108640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2acc102ef0 0x7f2acc1a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2ad2108640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc110f80 con 0x7f2acc102570 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2ad2108640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc110e00 con 0x7f2acc100e20 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2ad2108640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc111100 con 0x7f2acc102ef0 2024-09-16T00:28:46.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 0x7f2acc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 0x7f2acc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60338/0 (socket says 172.21.15.1:60338) 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acaffd640 1 -- 172.21.15.1:0/674463693 learned_addr learned my addr 172.21.15.1:0/674463693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acb7fe640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc100e20 0x7f2acc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acbfff640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2acc102ef0 0x7f2acc1a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acaffd640 1 -- 172.21.15.1:0/674463693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2acc102ef0 msgr2=0x7f2acc1a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acaffd640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2acc102ef0 0x7f2acc1a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acaffd640 1 -- 172.21.15.1:0/674463693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc100e20 msgr2=0x7f2acc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acaffd640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc100e20 0x7f2acc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acaffd640 1 -- 172.21.15.1:0/674463693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc1a4910 con 0x7f2acc102570 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2acaffd640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 0x7f2acc199890 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0006010 tx=0x7f2ac0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2ac8ff9640 1 -- 172.21.15.1:0/674463693 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac003f070 con 0x7f2acc102570 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2ac8ff9640 1 -- 172.21.15.1:0/674463693 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac0031c30 con 0x7f2acc102570 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.804+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2acc1a4ae0 con 0x7f2acc102570 2024-09-16T00:28:46.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.805+0000 7f2ac8ff9640 1 -- 172.21.15.1:0/674463693 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0035030 con 0x7f2acc102570 2024-09-16T00:28:46.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.805+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2acc1a4f30 con 0x7f2acc102570 2024-09-16T00:28:46.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.806+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2acc101200 con 0x7f2acc102570 2024-09-16T00:28:46.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.808+0000 7f2ac8ff9640 1 -- 172.21.15.1:0/674463693 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2ac0042070 con 0x7f2acc102570 2024-09-16T00:28:46.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.808+0000 7f2ac8ff9640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2ab40a3240 0x7f2ab40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:46.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.808+0000 7f2acb7fe640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2ab40a3240 0x7f2ab40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:46.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.808+0000 7f2ac8ff9640 1 -- 172.21.15.1:0/674463693 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2ac00f0990 con 0x7f2acc102570 2024-09-16T00:28:46.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.809+0000 7f2acb7fe640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2ab40a3240 0x7f2ab40a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2abc005fd0 tx=0x7f2abc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:46.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:46.812+0000 7f2ac8ff9640 1 -- 172.21.15.1:0/674463693 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ac0031de0 con 0x7f2acc102570 2024-09-16T00:28:46.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:46 smithi001 ceph-mon[30934]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:46.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:46 smithi001 ceph-mon[30934]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:46.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:46 smithi153 ceph-mon[35538]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:46.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:46 smithi153 ceph-mon[35538]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:47.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.030+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2acc1090c0 con 0x7f2acc102570 2024-09-16T00:28:47.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.030+0000 7f2ac8ff9640 1 -- 172.21.15.1:0/674463693 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f2ac00bd2b0 con 0x7f2acc102570 2024-09-16T00:28:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:47.032 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:28:47.033 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2ab40a3240 msgr2=0x7f2ab40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2ab40a3240 0x7f2ab40a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2abc005fd0 tx=0x7f2abc005850 comp rx=0 tx=0).stop 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 msgr2=0x7f2acc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 0x7f2acc199890 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0006010 tx=0x7f2ac0003c10 comp rx=0 tx=0).stop 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 shutdown_connections 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2ab40a3240 0x7f2ab40a5700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2acc102ef0 0x7f2acc1a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2acc102570 0x7f2acc199890 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 --2- 172.21.15.1:0/674463693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc100e20 0x7f2acc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.032+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 >> 172.21.15.1:0/674463693 conn(0x7f2acc0fc7d0 msgr2=0x7f2acc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.033+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 shutdown_connections 2024-09-16T00:28:47.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:47.033+0000 7f2ad2108640 1 -- 172.21.15.1:0/674463693 wait complete. 2024-09-16T00:28:47.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:46 smithi019 ceph-mon[35186]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:47.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:46 smithi019 ceph-mon[35186]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:47.750 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:47 smithi001 ceph-mon[30934]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:47.750 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:47 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/674463693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:47.795 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:28:47.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:47 smithi153 ceph-mon[35538]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:47.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:47 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/674463693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:48.054 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:48.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:47 smithi019 ceph-mon[35186]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:28:48.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:47 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/674463693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:48.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.583+0000 7fbc88b9d640 1 -- 172.21.15.1:0/899727928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 msgr2=0x7fbc841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:48.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.583+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/899727928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 0x7fbc841029b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009900 tx=0x7fbc6c02a480 comp rx=0 tx=0).stop 2024-09-16T00:28:48.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.583+0000 7fbc88b9d640 1 -- 172.21.15.1:0/899727928 shutdown_connections 2024-09-16T00:28:48.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.583+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/899727928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc84102ef0 0x7fbc8410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:48.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.583+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/899727928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 0x7fbc841029b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:48.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.583+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/899727928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc84100e20 0x7fbc84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:48.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.583+0000 7fbc88b9d640 1 -- 172.21.15.1:0/899727928 >> 172.21.15.1:0/899727928 conn(0x7fbc840fc7d0 msgr2=0x7fbc840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:48.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.583+0000 7fbc88b9d640 1 -- 172.21.15.1:0/899727928 shutdown_connections 2024-09-16T00:28:48.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.583+0000 7fbc88b9d640 1 -- 172.21.15.1:0/899727928 wait complete. 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.584+0000 7fbc88b9d640 1 Processor -- start 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.584+0000 7fbc88b9d640 1 -- start start 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.584+0000 7fbc88b9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc84100e20 0x7fbc841974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.584+0000 7fbc88b9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 0x7fbc841979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.584+0000 7fbc88b9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc84102ef0 0x7fbc8419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.584+0000 7fbc88b9d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84110ff0 con 0x7fbc84102570 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.584+0000 7fbc88b9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84110e70 con 0x7fbc84102ef0 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.584+0000 7fbc88b9d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84111170 con 0x7fbc84100e20 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 0x7fbc841979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 0x7fbc841979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60364/0 (socket says 172.21.15.1:60364) 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7bfff640 1 -- 172.21.15.1:0/742675328 learned_addr learned my addr 172.21.15.1:0/742675328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc82d9d640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc84102ef0 0x7fbc8419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc8259c640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc84100e20 0x7fbc841974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7bfff640 1 -- 172.21.15.1:0/742675328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc84100e20 msgr2=0x7fbc841974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7bfff640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc84100e20 0x7fbc841974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7bfff640 1 -- 172.21.15.1:0/742675328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc84102ef0 msgr2=0x7fbc8419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7bfff640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc84102ef0 0x7fbc8419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7bfff640 1 -- 172.21.15.1:0/742675328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8419c460 con 0x7fbc84102570 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7bfff640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 0x7fbc841979f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c02aa70 tx=0x7fbc6c003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7b7fe640 1 -- 172.21.15.1:0/742675328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c03f070 con 0x7fbc84102570 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7b7fe640 1 -- 172.21.15.1:0/742675328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc6c031c20 con 0x7fbc84102570 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8419c6f0 con 0x7fbc84102570 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc7b7fe640 1 -- 172.21.15.1:0/742675328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c035030 con 0x7fbc84102570 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.585+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc841a4030 con 0x7fbc84102570 2024-09-16T00:28:48.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.586+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc841021b0 con 0x7fbc84102570 2024-09-16T00:28:48.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.588+0000 7fbc7b7fe640 1 -- 172.21.15.1:0/742675328 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fbc6c042070 con 0x7fbc84102570 2024-09-16T00:28:48.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.589+0000 7fbc7b7fe640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc700a3240 0x7fbc700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:48.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.589+0000 7fbc7b7fe640 1 -- 172.21.15.1:0/742675328 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbc6c0f0a30 con 0x7fbc84102570 2024-09-16T00:28:48.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.589+0000 7fbc8259c640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc700a3240 0x7fbc700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:48.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.589+0000 7fbc8259c640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc700a3240 0x7fbc700a5700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400bbe0 tx=0x7fbc7400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:48.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.592+0000 7fbc7b7fe640 1 -- 172.21.15.1:0/742675328 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc6c031dd0 con 0x7fbc84102570 2024-09-16T00:28:48.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.770+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc841a4200 con 0x7fbc84102570 2024-09-16T00:28:48.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.771+0000 7fbc7b7fe640 1 -- 172.21.15.1:0/742675328 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24902 (secure 0 0 0) 0x7fbc6c0bcca0 con 0x7fbc84102570 2024-09-16T00:28:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:48.773 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:27:26.243912+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:28:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.773+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc700a3240 msgr2=0x7fbc700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.773+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc700a3240 0x7fbc700a5700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400bbe0 tx=0x7fbc7400b390 comp rx=0 tx=0).stop 2024-09-16T00:28:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.773+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 msgr2=0x7fbc841979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.773+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 0x7fbc841979f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c02aa70 tx=0x7fbc6c003b60 comp rx=0 tx=0).stop 2024-09-16T00:28:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.773+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 shutdown_connections 2024-09-16T00:28:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.774+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc84102ef0 0x7fbc8419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.774+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc84102570 0x7fbc841979f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.774+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc700a3240 0x7fbc700a5700 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:48.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.774+0000 7fbc88b9d640 1 --2- 172.21.15.1:0/742675328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc84100e20 0x7fbc841974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:48.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.774+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 >> 172.21.15.1:0/742675328 conn(0x7fbc840fc7d0 msgr2=0x7fbc841068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:48.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.774+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 shutdown_connections 2024-09-16T00:28:48.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:48.774+0000 7fbc88b9d640 1 -- 172.21.15.1:0/742675328 wait complete. 2024-09-16T00:28:49.515 DEBUG:tasks.fwd_scrub:Yielding 2024-09-16T00:28:49.515 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-16T00:28:49.530 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:28:49.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:28:49.531 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-16T00:28:49.531 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:28:49.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:28:49.808 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:49.808 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:49.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:49 smithi001 ceph-mon[30934]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-16T00:28:49.931 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:49 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/742675328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:49.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:49 smithi153 ceph-mon[35538]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-16T00:28:49.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:49 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/742675328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:50.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:49 smithi019 ceph-mon[35186]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-16T00:28:50.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:49 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/742675328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.367+0000 7f9b45fe1640 1 -- 172.21.15.1:0/3095339347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40101de0 msgr2=0x7f9b401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.367+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/3095339347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40101de0 0x7f9b401021c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009ca0 tx=0x7f9b3002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.367+0000 7f9b45fe1640 1 -- 172.21.15.1:0/3095339347 shutdown_connections 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.367+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/3095339347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b40106f10 0x7f9b4010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.367+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/3095339347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b40102790 0x7f9b401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.367+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/3095339347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40101de0 0x7f9b401021c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.367+0000 7f9b45fe1640 1 -- 172.21.15.1:0/3095339347 >> 172.21.15.1:0/3095339347 conn(0x7f9b400fd8c0 msgr2=0x7f9b400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.367+0000 7f9b45fe1640 1 -- 172.21.15.1:0/3095339347 shutdown_connections 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.367+0000 7f9b45fe1640 1 -- 172.21.15.1:0/3095339347 wait complete. 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.368+0000 7f9b45fe1640 1 Processor -- start 2024-09-16T00:28:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.368+0000 7f9b45fe1640 1 -- start start 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.368+0000 7f9b45fe1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b40101de0 0x7f9b401983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b45fe1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b40102790 0x7f9b40198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b45fe1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40106f10 0x7f9b4019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b45fe1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40111f70 con 0x7f9b40106f10 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b45fe1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40111df0 con 0x7f9b40101de0 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b45fe1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401120f0 con 0x7f9b40102790 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40106f10 0x7f9b4019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40106f10 0x7f9b4019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38418/0 (socket says 172.21.15.1:38418) 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3ffff640 1 -- 172.21.15.1:0/2751442415 learned_addr learned my addr 172.21.15.1:0/2751442415 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b40101de0 0x7f9b401983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3effd640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b40102790 0x7f9b40198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3ffff640 1 -- 172.21.15.1:0/2751442415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b40102790 msgr2=0x7f9b40198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3ffff640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b40102790 0x7f9b40198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3ffff640 1 -- 172.21.15.1:0/2751442415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b40101de0 msgr2=0x7f9b401983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3ffff640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b40101de0 0x7f9b401983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3ffff640 1 -- 172.21.15.1:0/2751442415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4019d380 con 0x7f9b40106f10 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3effd640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b40102790 0x7f9b40198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3ffff640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40106f10 0x7f9b4019cca0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c970 tx=0x7f9b3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3cff9640 1 -- 172.21.15.1:0/2751442415 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38007bb0 con 0x7f9b40106f10 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3cff9640 1 -- 172.21.15.1:0/2751442415 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b38007d50 con 0x7f9b40106f10 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b3cff9640 1 -- 172.21.15.1:0/2751442415 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38018800 con 0x7f9b40106f10 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.369+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4019d670 con 0x7f9b40106f10 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.370+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b401a4f50 con 0x7f9b40106f10 2024-09-16T00:28:50.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.370+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b401021c0 con 0x7f9b40106f10 2024-09-16T00:28:50.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.372+0000 7f9b3cff9640 1 -- 172.21.15.1:0/2751442415 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9b3800fa50 con 0x7f9b40106f10 2024-09-16T00:28:50.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.372+0000 7f9b3cff9640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9b280a3240 0x7f9b280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:50.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.372+0000 7f9b3f7fe640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9b280a3240 0x7f9b280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:50.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.372+0000 7f9b3cff9640 1 -- 172.21.15.1:0/2751442415 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9b380cfd10 con 0x7f9b40106f10 2024-09-16T00:28:50.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.373+0000 7f9b3f7fe640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9b280a3240 0x7f9b280a5700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006010 tx=0x7f9b30009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:50.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.377+0000 7f9b3cff9640 1 -- 172.21.15.1:0/2751442415 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b38014030 con 0x7f9b40106f10 2024-09-16T00:28:50.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.549+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b40062630 con 0x7f9b280a3240 2024-09-16T00:28:50.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.558+0000 7f9b3cff9640 1 -- 172.21.15.1:0/2751442415 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340487 (secure 0 0 0) 0x7f9b40062630 con 0x7f9b280a3240 2024-09-16T00:28:50.562 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:50.564 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:28:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9b280a3240 msgr2=0x7f9b280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9b280a3240 0x7f9b280a5700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006010 tx=0x7f9b30009360 comp rx=0 tx=0).stop 2024-09-16T00:28:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40106f10 msgr2=0x7f9b4019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40106f10 0x7f9b4019cca0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c970 tx=0x7f9b3800ce30 comp rx=0 tx=0).stop 2024-09-16T00:28:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 shutdown_connections 2024-09-16T00:28:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b40106f10 0x7f9b4019cca0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9b280a3240 0x7f9b280a5700 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b40102790 0x7f9b40198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 --2- 172.21.15.1:0/2751442415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b40101de0 0x7f9b401983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 >> 172.21.15.1:0/2751442415 conn(0x7f9b400fd8c0 msgr2=0x7f9b400febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 shutdown_connections 2024-09-16T00:28:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:50.561+0000 7f9b45fe1640 1 -- 172.21.15.1:0/2751442415 wait complete. 2024-09-16T00:28:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.202+0000 7f9e16b53640 1 -- 172.21.15.1:0/4231127759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 msgr2=0x7f9e10107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.202+0000 7f9e16b53640 1 --2- 172.21.15.1:0/4231127759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 0x7f9e10107b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009930 tx=0x7f9df802a420 comp rx=0 tx=0).stop 2024-09-16T00:28:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.203+0000 7f9e16b53640 1 -- 172.21.15.1:0/4231127759 shutdown_connections 2024-09-16T00:28:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.203+0000 7f9e16b53640 1 --2- 172.21.15.1:0/4231127759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e101082a0 0x7f9e1010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.203+0000 7f9e16b53640 1 --2- 172.21.15.1:0/4231127759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 0x7f9e10107b90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.203+0000 7f9e16b53640 1 --2- 172.21.15.1:0/4231127759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e10102d30 0x7f9e10103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.203+0000 7f9e16b53640 1 -- 172.21.15.1:0/4231127759 >> 172.21.15.1:0/4231127759 conn(0x7f9e100fc7d0 msgr2=0x7f9e100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.203+0000 7f9e16b53640 1 -- 172.21.15.1:0/4231127759 shutdown_connections 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.203+0000 7f9e16b53640 1 -- 172.21.15.1:0/4231127759 wait complete. 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.203+0000 7f9e16b53640 1 Processor -- start 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.203+0000 7f9e16b53640 1 -- start start 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e16b53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e10102d30 0x7f9e101985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e16b53640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 0x7f9e10198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e16b53640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e101082a0 0x7f9e1019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e16b53640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101120a0 con 0x7f9e101036e0 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e16b53640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10111f20 con 0x7f9e10102d30 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e16b53640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10112220 con 0x7f9e101082a0 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e0ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 0x7f9e10198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e0ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 0x7f9e10198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38442/0 (socket says 172.21.15.1:38442) 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e0ffff640 1 -- 172.21.15.1:0/2836241646 learned_addr learned my addr 172.21.15.1:0/2836241646 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e148ef640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e10102d30 0x7f9e101985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e150f0640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e101082a0 0x7f9e1019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e0ffff640 1 -- 172.21.15.1:0/2836241646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e101082a0 msgr2=0x7f9e1019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e101082a0 0x7f9e1019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e0ffff640 1 -- 172.21.15.1:0/2836241646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e10102d30 msgr2=0x7f9e101985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e10102d30 0x7f9e101985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e0ffff640 1 -- 172.21.15.1:0/2836241646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1019d570 con 0x7f9e101036e0 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e148ef640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e10102d30 0x7f9e101985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.204+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 0x7f9e10198b00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9df8005940 tx=0x7f9df80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:51.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.205+0000 7f9e0dffb640 1 -- 172.21.15.1:0/2836241646 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df803f030 con 0x7f9e101036e0 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.205+0000 7f9e0dffb640 1 -- 172.21.15.1:0/2836241646 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df8003ce0 con 0x7f9e101036e0 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.205+0000 7f9e16b53640 1 -- 172.21.15.1:0/2836241646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1019d800 con 0x7f9e101036e0 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.205+0000 7f9e0dffb640 1 -- 172.21.15.1:0/2836241646 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8035070 con 0x7f9e101036e0 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.205+0000 7f9e16b53640 1 -- 172.21.15.1:0/2836241646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e101a5140 con 0x7f9e101036e0 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.207+0000 7f9e0dffb640 1 -- 172.21.15.1:0/2836241646 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9df8042030 con 0x7f9e101036e0 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e18f2c740 1 Processor -- start 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e18f2c740 1 -- start start 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e66b60 0x55f775e66f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e67480 0x55f775e71800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e71d40 0x55f775e74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f775dcc280 con 0x55f775e66b60 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f775d73b10 con 0x55f775e71d40 2024-09-16T00:28:51.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f775dc8240 con 0x55f775e67480 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e150f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e66b60 0x55f775e66f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e150f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e66b60 0x55f775e66f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38452/0 (socket says 172.21.15.1:38452) 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e150f0640 1 -- 172.21.15.1:0/2000382973 learned_addr learned my addr 172.21.15.1:0/2000382973 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.208+0000 7f9e0dffb640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dfc0a3240 0x7f9dfc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e148ef640 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e67480 0x55f775e71800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e150f0640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dfc0a3240 0x7f9dfc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e0dffb640 1 -- 172.21.15.1:0/2836241646 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9df80f01f0 con 0x7f9e101036e0 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e71d40 0x55f775e74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e150f0640 1 -- 172.21.15.1:0/2000382973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e67480 msgr2=0x55f775e71800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e150f0640 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e67480 0x55f775e71800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e150f0640 1 -- 172.21.15.1:0/2000382973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e71d40 msgr2=0x55f775e74130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e150f0640 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e71d40 0x55f775e74130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e150f0640 1 -- 172.21.15.1:0/2000382973 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f775d801f0 con 0x55f775e66b60 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e71d40 0x55f775e74130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e150f0640 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e66b60 0x55f775e66f40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9e00014a60 tx=0x7f9e00014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff445a39c3f19abe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e06ffd640 1 -- 172.21.15.1:0/2000382973 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0001ec50 con 0x55f775e66b60 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e06ffd640 1 -- 172.21.15.1:0/2000382973 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e00023070 con 0x55f775e66b60 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e06ffd640 1 -- 172.21.15.1:0/2000382973 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000055c0 con 0x55f775e66b60 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.209+0000 7f9e150f0640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dfc0a3240 0x7f9dfc0a5700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9e00022000 tx=0x7f9e0001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2000382973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e66b60 msgr2=0x55f775e66f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e66b60 0x55f775e66f40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9e00014a60 tx=0x7f9e00014f20 comp rx=0 tx=0).stop 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2000382973 shutdown_connections 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e71d40 0x55f775e74130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e67480 0x55f775e71800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2000382973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e66b60 0x55f775e66f40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2000382973 >> 172.21.15.1:0/2000382973 conn(0x55f775e630f0 msgr2=0x55f775e634d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2000382973 shutdown_connections 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2000382973 wait complete. 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 -- start start 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.210+0000 7f9e18f2c740 1 Processor -- start 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 0x55f775e6ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 0x55f775e6f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e6f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f775e74910 con 0x55f775e71d40 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f775d86f10 con 0x55f775e67480 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f775e74a90 con 0x55f775e66b60 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e18f2c740 10 client.0 _ll_register_callbacks cb 0x55f775d4fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e150f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e6f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e150f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e6f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38462/0 (socket says 172.21.15.1:38462) 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e150f0640 1 -- 172.21.15.1:0/2351383944 learned_addr learned my addr 172.21.15.1:0/2351383944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 0x55f775e6f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e148ef640 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 0x55f775e6ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e150f0640 1 -- 172.21.15.1:0/2351383944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 msgr2=0x55f775e6ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e150f0640 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 0x55f775e6ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e150f0640 1 -- 172.21.15.1:0/2351383944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 msgr2=0x55f775e6f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e150f0640 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 0x55f775e6f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.211+0000 7f9e150f0640 1 -- 172.21.15.1:0/2351383944 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f775b3d6a0 con 0x55f775e71d40 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9e150f0640 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e6f660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005f40 tx=0x7f9e000128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a233d6896d814a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9deffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9deffff640 1 -- 172.21.15.1:0/2351383944 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00037060 con 0x55f775e71d40 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9deffff640 1 -- 172.21.15.1:0/2351383944 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9e00005890 con 0x55f775e71d40 2024-09-16T00:28:51.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2351383944 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e00023070 con 0x55f775e71d40 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9deffff640 1 -- 172.21.15.1:0/2351383944 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e0003e050 con 0x55f775e71d40 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9e18f2c740 10 client.15318 fetch_fsmap learned FSMap version 25 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2351383944 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f775e804e0 con 0x55f775e71d40 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9deffff640 1 -- 172.21.15.1:0/2351383944 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e00023070 con 0x55f775e71d40 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9deffff640 1 -- 172.21.15.1:0/2351383944 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9e0000f040 con 0x55f775e71d40 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9e18f2c740 10 client.15318 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9e18f2c740 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.212+0000 7f9e18f2c740 20 client.15318 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.213+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f775e818e0 0x55f775e83cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.213+0000 7f9e18f2c740 4 client.15318 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.213+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2351383944 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f775dd97f0 con 0x55f775e818e0 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.213+0000 7f9e148ef640 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f775e818e0 0x55f775e83cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.213+0000 7f9deffff640 10 client.15318 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.213+0000 7f9e148ef640 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f775e818e0 0x55f775e83cd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aa93d4d8f4b205f server_cookie=e356a5d48deb6841 in_seq=0 out_seq=0 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9deffff640 1 -- 172.21.15.1:0/2351383944 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f775dd97f0 con 0x55f775e818e0 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9deffff640 10 client.15318 handle_command_reply: tid=0 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 client.15318 shutdown 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2351383944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 msgr2=0x55f775e6f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e6f660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005f40 tx=0x7f9e000128d0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2351383944 shutdown_connections 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f775e818e0 0x55f775e83cd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e6f660 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 0x55f775e6f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/2351383944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 0x55f775e6ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2351383944 >> 172.21.15.1:0/2351383944 conn(0x55f775e630f0 msgr2=0x55f775e62270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9deffff640 0 client.15318 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:51.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2351383944 shutdown_connections 2024-09-16T00:28:51.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 1 -- 172.21.15.1:0/2351383944 wait complete. 2024-09-16T00:28:51.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.214+0000 7f9e18f2c740 20 client.15318 trim_cache size 0 max 16384 2024-09-16T00:28:51.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e18f2c740 1 Processor -- start 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e18f2c740 1 -- start start 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e66b60 0x55f775e695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e67480 0x55f775e892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e71d40 0x55f775e897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101b1740 con 0x55f775e67480 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101af010 con 0x55f775e66b60 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101ace60 con 0x55f775e71d40 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e0ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e67480 0x55f775e892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e0ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e67480 0x55f775e892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38470/0 (socket says 172.21.15.1:38470) 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e0ffff640 1 -- 172.21.15.1:0/893213652 learned_addr learned my addr 172.21.15.1:0/893213652 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e148ef640 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e66b60 0x55f775e695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e150f0640 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e71d40 0x55f775e897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e0ffff640 1 -- 172.21.15.1:0/893213652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e71d40 msgr2=0x55f775e897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e71d40 0x55f775e897e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e0ffff640 1 -- 172.21.15.1:0/893213652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e66b60 msgr2=0x55f775e695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e66b60 0x55f775e695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e0ffff640 1 -- 172.21.15.1:0/893213652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f775e5e830 con 0x55f775e67480 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.232+0000 7f9e150f0640 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e71d40 0x55f775e897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.233+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e67480 0x55f775e892a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9df80bc0b0 tx=0x7f9df803b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a0a90858f0ed30 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.233+0000 7f9e067fc640 1 -- 172.21.15.1:0/893213652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8049040 con 0x55f775e67480 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.233+0000 7f9e067fc640 1 -- 172.21.15.1:0/893213652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df80bc3b0 con 0x55f775e67480 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.233+0000 7f9e067fc640 1 -- 172.21.15.1:0/893213652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df80bc610 con 0x55f775e67480 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.233+0000 7f9e18f2c740 1 -- 172.21.15.1:0/893213652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e67480 msgr2=0x55f775e892a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.233+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e67480 0x55f775e892a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9df80bc0b0 tx=0x7f9df803b6b0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 -- 172.21.15.1:0/893213652 shutdown_connections 2024-09-16T00:28:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e71d40 0x55f775e897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e67480 0x55f775e892a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/893213652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e66b60 0x55f775e695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 -- 172.21.15.1:0/893213652 >> 172.21.15.1:0/893213652 conn(0x55f775e630f0 msgr2=0x55f775e70b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 -- 172.21.15.1:0/893213652 shutdown_connections 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 -- 172.21.15.1:0/893213652 wait complete. 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 -- start start 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 Processor -- start 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 0x55f775e7a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 0x55f775e75820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e75dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101ad4d0 con 0x55f775e71d40 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101ae940 con 0x55f775e67480 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101adba0 con 0x55f775e66b60 2024-09-16T00:28:51.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.234+0000 7f9e18f2c740 10 client.0 _ll_register_callbacks cb 0x7f9e08008390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:51.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e150f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e75dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e150f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e75dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38480/0 (socket says 172.21.15.1:38480) 2024-09-16T00:28:51.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e150f0640 1 -- 172.21.15.1:0/1139101941 learned_addr learned my addr 172.21.15.1:0/1139101941 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:51.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e0ffff640 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 0x55f775e75820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e148ef640 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 0x55f775e7a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e150f0640 1 -- 172.21.15.1:0/1139101941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 msgr2=0x55f775e7a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:51.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e150f0640 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 0x55f775e7a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e150f0640 1 -- 172.21.15.1:0/1139101941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 msgr2=0x55f775e75820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:51.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e150f0640 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 0x55f775e75820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e150f0640 1 -- 172.21.15.1:0/1139101941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f775e672b0 con 0x55f775e71d40 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e150f0640 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e75dc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009dc0 tx=0x7f9e00005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e131da211b0ba9ab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9deffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9deffff640 1 -- 172.21.15.1:0/1139101941 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0002b020 con 0x55f775e71d40 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9deffff640 1 -- 172.21.15.1:0/1139101941 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9e00009a00 con 0x55f775e71d40 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.235+0000 7f9e18f2c740 1 -- 172.21.15.1:0/1139101941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df80bc3b0 con 0x55f775e71d40 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9deffff640 1 -- 172.21.15.1:0/1139101941 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e00037040 con 0x55f775e71d40 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9e18f2c740 10 client.15324 fetch_fsmap learned FSMap version 25 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9e18f2c740 1 -- 172.21.15.1:0/1139101941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f775ef0520 con 0x55f775e71d40 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9deffff640 1 -- 172.21.15.1:0/1139101941 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df80bc3b0 con 0x55f775e71d40 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9deffff640 1 -- 172.21.15.1:0/1139101941 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9e00004b60 con 0x55f775e71d40 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9e18f2c740 10 client.15324 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9e18f2c740 10 client.15324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9e18f2c740 20 client.15324 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f775ef0c30 0x55f775ef3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9e18f2c740 4 client.15324 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9e18f2c740 1 -- 172.21.15.1:0/1139101941 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f775dd3d00 con 0x55f775ef0c30 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.236+0000 7f9e148ef640 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f775ef0c30 0x55f775ef3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.237+0000 7f9e148ef640 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f775ef0c30 0x55f775ef3050 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a1271819974c2b server_cookie=267e430cbccf71e5 in_seq=0 out_seq=0 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.237+0000 7f9deffff640 10 client.15324 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9deffff640 1 -- 172.21.15.1:0/1139101941 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f775dd3d00 con 0x55f775ef0c30 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9deffff640 10 client.15324 handle_command_reply: tid=0 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 client.15324 shutdown 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 -- 172.21.15.1:0/1139101941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 msgr2=0x55f775e75dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e75dc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009dc0 tx=0x7f9e00005b40 comp rx=0 tx=0).stop 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 -- 172.21.15.1:0/1139101941 shutdown_connections 2024-09-16T00:28:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f775ef0c30 0x55f775ef3050 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9deffff640 0 client.15324 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f775e71d40 0x55f775e75dc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f775e67480 0x55f775e75820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 --2- 172.21.15.1:0/1139101941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f775e66b60 0x55f775e7a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 -- 172.21.15.1:0/1139101941 >> 172.21.15.1:0/1139101941 conn(0x55f775e630f0 msgr2=0x55f775e61a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 -- 172.21.15.1:0/1139101941 shutdown_connections 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 1 -- 172.21.15.1:0/1139101941 wait complete. 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.238+0000 7f9e18f2c740 20 client.15324 trim_cache size 0 max 16384 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:28:51.252 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:28:51.253 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "200d5172-c5f3-43ed-9a59-c6430ce048ff", 2024-09-16T00:28:51.253 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:28:51.253 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 -- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dfc0a3240 msgr2=0x7f9dfc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dfc0a3240 0x7f9dfc0a5700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9e00022000 tx=0x7f9e0001e000 comp rx=0 tx=0).stop 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 -- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 msgr2=0x7f9e10198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 0x7f9e10198b00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9df8005940 tx=0x7f9df80042f0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 -- 172.21.15.1:0/2836241646 shutdown_connections 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dfc0a3240 0x7f9dfc0a5700 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e101082a0 0x7f9e1019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e101036e0 0x7f9e10198b00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 --2- 172.21.15.1:0/2836241646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e10102d30 0x7f9e101985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 -- 172.21.15.1:0/2836241646 >> 172.21.15.1:0/2836241646 conn(0x7f9e100fc7d0 msgr2=0x7f9e10106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 -- 172.21.15.1:0/2836241646 shutdown_connections 2024-09-16T00:28:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:51.239+0000 7f9e16b53640 1 -- 172.21.15.1:0/2836241646 wait complete. 2024-09-16T00:28:51.914 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:51 smithi001 ceph-mon[30934]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:28:51.915 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:51 smithi001 ceph-mon[30934]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:51.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:28:51.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:51 smithi153 ceph-mon[35538]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:28:51.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:51 smithi153 ceph-mon[35538]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:52.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:51 smithi019 ceph-mon[35186]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:28:52.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:51 smithi019 ceph-mon[35186]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:52.454 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-16T00:28:50.347430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663060,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":794,"num_read_kb":517,"num_write":363,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380140,"kb_used_data":14252,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561684,"statfs":{"total":1151940427776,"available":1151551164416,"internally_reserved":0,"allocated":14594048,"data_stored":8145134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":38,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002893"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252865+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:27:26.252865+0000","last_peered":"2024-09-16T00:27:26.252865+0000","last_clean":"2024-09-16T00:27:26.252865+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:27:26.252865+0000","last_undegraded":"2024-09-16T00:27:26.252865+0000","last_fullsized":"2024-09-16T00:27:26.252865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322537+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:28:24.322537+0000","last_peered":"2024-09-16T00:28:24.322537+0000","last_clean":"2024-09-16T00:28:24.322537+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:28:24.322537+0000","last_undegraded":"2024-09-16T00:28:24.322537+0000","last_fullsized":"2024-09-16T00:28:24.322537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248105+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:27:26.248105+0000","last_peered":"2024-09-16T00:27:26.248105+0000","last_clean":"2024-09-16T00:27:26.248105+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:27:26.248105+0000","last_undegraded":"2024-09-16T00:27:26.248105+0000","last_fullsized":"2024-09-16T00:27:26.248105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247830+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:27:26.247830+0000","last_peered":"2024-09-16T00:27:26.247830+0000","last_clean":"2024-09-16T00:27:26.247830+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:27:26.247830+0000","last_undegraded":"2024-09-16T00:27:26.247830+0000","last_fullsized":"2024-09-16T00:27:26.247830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336305+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:28:24.336305+0000","last_peered":"2024-09-16T00:28:24.336305+0000","last_clean":"2024-09-16T00:28:24.336305+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:28:24.336305+0000","last_undegraded":"2024-09-16T00:28:24.336305+0000","last_fullsized":"2024-09-16T00:28:24.336305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355341+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:27:26.355341+0000","last_peered":"2024-09-16T00:27:26.355341+0000","last_clean":"2024-09-16T00:27:26.355341+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:27:26.355341+0000","last_undegraded":"2024-09-16T00:27:26.355341+0000","last_fullsized":"2024-09-16T00:27:26.355341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357885+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:26.357885+0000","last_peered":"2024-09-16T00:27:26.357885+0000","last_clean":"2024-09-16T00:27:26.357885+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:26.357885+0000","last_undegraded":"2024-09-16T00:27:26.357885+0000","last_fullsized":"2024-09-16T00:27:26.357885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337468+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:28:24.337468+0000","last_peered":"2024-09-16T00:28:24.337468+0000","last_clean":"2024-09-16T00:28:24.337468+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:28:24.337468+0000","last_undegraded":"2024-09-16T00:28:24.337468+0000","last_fullsized":"2024-09-16T00:28:24.337468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356501+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:27:26.356501+0000","last_peered":"2024-09-16T00:27:26.356501+0000","last_clean":"2024-09-16T00:27:26.356501+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:27:26.356501+0000","last_undegraded":"2024-09-16T00:27:26.356501+0000","last_fullsized":"2024-09-16T00:27:26.356501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252350+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:27:26.252350+0000","last_peered":"2024-09-16T00:27:26.252350+0000","last_clean":"2024-09-16T00:27:26.252350+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:27:26.252350+0000","last_undegraded":"2024-09-16T00:27:26.252350+0000","last_fullsized":"2024-09-16T00:27:26.252350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336557+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:28:24.336557+0000","last_peered":"2024-09-16T00:28:24.336557+0000","last_clean":"2024-09-16T00:28:24.336557+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:28:24.336557+0000","last_undegraded":"2024-09-16T00:28:24.336557+0000","last_fullsized":"2024-09-16T00:28:24.336557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355852+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:27:26.355852+0000","last_peered":"2024-09-16T00:27:26.355852+0000","last_clean":"2024-09-16T00:27:26.355852+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:27:26.355852+0000","last_undegraded":"2024-09-16T00:27:26.355852+0000","last_fullsized":"2024-09-16T00:27:26.355852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338941+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:28:24.338941+0000","last_peered":"2024-09-16T00:28:24.338941+0000","last_clean":"2024-09-16T00:28:24.338941+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:28:24.338941+0000","last_undegraded":"2024-09-16T00:28:24.338941+0000","last_fullsized":"2024-09-16T00:28:24.338941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357104+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:26.357104+0000","last_peered":"2024-09-16T00:27:26.357104+0000","last_clean":"2024-09-16T00:27:26.357104+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:26.357104+0000","last_undegraded":"2024-09-16T00:27:26.357104+0000","last_fullsized":"2024-09-16T00:27:26.357104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356176+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:27:26.356176+0000","last_peered":"2024-09-16T00:27:26.356176+0000","last_clean":"2024-09-16T00:27:26.356176+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:27:26.356176+0000","last_undegraded":"2024-09-16T00:27:26.356176+0000","last_fullsized":"2024-09-16T00:27:26.356176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327188+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:28:24.327188+0000","last_peered":"2024-09-16T00:28:24.327188+0000","last_clean":"2024-09-16T00:28:24.327188+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:28:24.327188+0000","last_undegraded":"2024-09-16T00:28:24.327188+0000","last_fullsized":"2024-09-16T00:28:24.327188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248306+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:27:26.248306+0000","last_peered":"2024-09-16T00:27:26.248306+0000","last_clean":"2024-09-16T00:27:26.248306+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:27:26.248306+0000","last_undegraded":"2024-09-16T00:27:26.248306+0000","last_fullsized":"2024-09-16T00:27:26.248306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356287+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:27:26.356287+0000","last_peered":"2024-09-16T00:27:26.356287+0000","last_clean":"2024-09-16T00:27:26.356287+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:27:26.356287+0000","last_undegraded":"2024-09-16T00:27:26.356287+0000","last_fullsized":"2024-09-16T00:27:26.356287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324477+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:28:24.324477+0000","last_peered":"2024-09-16T00:28:24.324477+0000","last_clean":"2024-09-16T00:28:24.324477+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:28:24.324477+0000","last_undegraded":"2024-09-16T00:28:24.324477+0000","last_fullsized":"2024-09-16T00:28:24.324477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247518+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:26.247518+0000","last_peered":"2024-09-16T00:27:26.247518+0000","last_clean":"2024-09-16T00:27:26.247518+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:26.247518+0000","last_undegraded":"2024-09-16T00:27:26.247518+0000","last_fullsized":"2024-09-16T00:27:26.247518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252497+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:27:26.252497+0000","last_peered":"2024-09-16T00:27:26.252497+0000","last_clean":"2024-09-16T00:27:26.252497+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:27:26.252497+0000","last_undegraded":"2024-09-16T00:27:26.252497+0000","last_fullsized":"2024-09-16T00:27:26.252497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.312122+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:28:24.312122+0000","last_peered":"2024-09-16T00:28:24.312122+0000","last_clean":"2024-09-16T00:28:24.312122+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:28:24.312122+0000","last_undegraded":"2024-09-16T00:28:24.312122+0000","last_fullsized":"2024-09-16T00:28:24.312122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355055+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:26.355055+0000","last_peered":"2024-09-16T00:27:26.355055+0000","last_clean":"2024-09-16T00:27:26.355055+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:26.355055+0000","last_undegraded":"2024-09-16T00:27:26.355055+0000","last_fullsized":"2024-09-16T00:27:26.355055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357331+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:27:26.357331+0000","last_peered":"2024-09-16T00:27:26.357331+0000","last_clean":"2024-09-16T00:27:26.357331+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:27:26.357331+0000","last_undegraded":"2024-09-16T00:27:26.357331+0000","last_fullsized":"2024-09-16T00:27:26.357331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334895+0000","last_change":"2024-09-16T00:27:26.358639+0000","last_active":"2024-09-16T00:28:24.334895+0000","last_peered":"2024-09-16T00:28:24.334895+0000","last_clean":"2024-09-16T00:28:24.334895+0000","last_became_active":"2024-09-16T00:27:26.358163+0000","last_became_peered":"2024-09-16T00:27:26.358163+0000","last_unstale":"2024-09-16T00:28:24.334895+0000","last_undegraded":"2024-09-16T00:28:24.334895+0000","last_fullsized":"2024-09-16T00:28:24.334895+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:11.811436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356716+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:26.356716+0000","last_peered":"2024-09-16T00:27:26.356716+0000","last_clean":"2024-09-16T00:27:26.356716+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:26.356716+0000","last_undegraded":"2024-09-16T00:27:26.356716+0000","last_fullsized":"2024-09-16T00:27:26.356716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248397+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:27:26.248397+0000","last_peered":"2024-09-16T00:27:26.248397+0000","last_clean":"2024-09-16T00:27:26.248397+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:27:26.248397+0000","last_undegraded":"2024-09-16T00:27:26.248397+0000","last_fullsized":"2024-09-16T00:27:26.248397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332074+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:28:24.332074+0000","last_peered":"2024-09-16T00:28:24.332074+0000","last_clean":"2024-09-16T00:28:24.332074+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:28:24.332074+0000","last_undegraded":"2024-09-16T00:28:24.332074+0000","last_fullsized":"2024-09-16T00:28:24.332074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:26.231276+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:28:26.231276+0000","last_peered":"2024-09-16T00:28:26.231276+0000","last_clean":"2024-09-16T00:28:26.231276+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:28:26.231276+0000","last_undegraded":"2024-09-16T00:28:26.231276+0000","last_fullsized":"2024-09-16T00:28:26.231276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247207+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:27:26.247207+0000","last_peered":"2024-09-16T00:27:26.247207+0000","last_clean":"2024-09-16T00:27:26.247207+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:27:26.247207+0000","last_undegraded":"2024-09-16T00:27:26.247207+0000","last_fullsized":"2024-09-16T00:27:26.247207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.325854+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:28:24.325854+0000","last_peered":"2024-09-16T00:28:24.325854+0000","last_clean":"2024-09-16T00:28:24.325854+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:28:24.325854+0000","last_undegraded":"2024-09-16T00:28:24.325854+0000","last_fullsized":"2024-09-16T00:28:24.325854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357960+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:26.357960+0000","last_peered":"2024-09-16T00:27:26.357960+0000","last_clean":"2024-09-16T00:27:26.357960+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:26.357960+0000","last_undegraded":"2024-09-16T00:27:26.357960+0000","last_fullsized":"2024-09-16T00:27:26.357960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251785+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:27:26.251785+0000","last_peered":"2024-09-16T00:27:26.251785+0000","last_clean":"2024-09-16T00:27:26.251785+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:27:26.251785+0000","last_undegraded":"2024-09-16T00:27:26.251785+0000","last_fullsized":"2024-09-16T00:27:26.251785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329144+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:28:24.329144+0000","last_peered":"2024-09-16T00:28:24.329144+0000","last_clean":"2024-09-16T00:28:24.329144+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:28:24.329144+0000","last_undegraded":"2024-09-16T00:28:24.329144+0000","last_fullsized":"2024-09-16T00:28:24.329144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252455+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:27:26.252455+0000","last_peered":"2024-09-16T00:27:26.252455+0000","last_clean":"2024-09-16T00:27:26.252455+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:27:26.252455+0000","last_undegraded":"2024-09-16T00:27:26.252455+0000","last_fullsized":"2024-09-16T00:27:26.252455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357048+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:27:26.357048+0000","last_peered":"2024-09-16T00:27:26.357048+0000","last_clean":"2024-09-16T00:27:26.357048+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:27:26.357048+0000","last_undegraded":"2024-09-16T00:27:26.357048+0000","last_fullsized":"2024-09-16T00:27:26.357048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321720+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:28:24.321720+0000","last_peered":"2024-09-16T00:28:24.321720+0000","last_clean":"2024-09-16T00:28:24.321720+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:28:24.321720+0000","last_undegraded":"2024-09-16T00:28:24.321720+0000","last_fullsized":"2024-09-16T00:28:24.321720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357297+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:27:26.357297+0000","last_peered":"2024-09-16T00:27:26.357297+0000","last_clean":"2024-09-16T00:27:26.357297+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:27:26.357297+0000","last_undegraded":"2024-09-16T00:27:26.357297+0000","last_fullsized":"2024-09-16T00:27:26.357297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250069+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:27:26.250069+0000","last_peered":"2024-09-16T00:27:26.250069+0000","last_clean":"2024-09-16T00:27:26.250069+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:27:26.250069+0000","last_undegraded":"2024-09-16T00:27:26.250069+0000","last_fullsized":"2024-09-16T00:27:26.250069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.305108+0000","last_change":"2024-09-16T00:27:26.357501+0000","last_active":"2024-09-16T00:28:24.305108+0000","last_peered":"2024-09-16T00:28:24.305108+0000","last_clean":"2024-09-16T00:28:24.305108+0000","last_became_active":"2024-09-16T00:27:26.357108+0000","last_became_peered":"2024-09-16T00:27:26.357108+0000","last_unstale":"2024-09-16T00:28:24.305108+0000","last_undegraded":"2024-09-16T00:28:24.305108+0000","last_fullsized":"2024-09-16T00:28:24.305108+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:25.264059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:44.668016+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:28:44.668016+0000","last_peered":"2024-09-16T00:28:44.668016+0000","last_clean":"2024-09-16T00:28:44.668016+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:28:44.668016+0000","last_undegraded":"2024-09-16T00:28:44.668016+0000","last_fullsized":"2024-09-16T00:28:44.668016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250857+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:27:26.250857+0000","last_peered":"2024-09-16T00:27:26.250857+0000","last_clean":"2024-09-16T00:27:26.250857+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:27:26.250857+0000","last_undegraded":"2024-09-16T00:27:26.250857+0000","last_fullsized":"2024-09-16T00:27:26.250857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326885+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:28:24.326885+0000","last_peered":"2024-09-16T00:28:24.326885+0000","last_clean":"2024-09-16T00:28:24.326885+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:28:24.326885+0000","last_undegraded":"2024-09-16T00:28:24.326885+0000","last_fullsized":"2024-09-16T00:28:24.326885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357096+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:26.357096+0000","last_peered":"2024-09-16T00:27:26.357096+0000","last_clean":"2024-09-16T00:27:26.357096+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:26.357096+0000","last_undegraded":"2024-09-16T00:27:26.357096+0000","last_fullsized":"2024-09-16T00:27:26.357096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356120+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:27:26.356120+0000","last_peered":"2024-09-16T00:27:26.356120+0000","last_clean":"2024-09-16T00:27:26.356120+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:27:26.356120+0000","last_undegraded":"2024-09-16T00:27:26.356120+0000","last_fullsized":"2024-09-16T00:27:26.356120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336576+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:28:24.336576+0000","last_peered":"2024-09-16T00:28:24.336576+0000","last_clean":"2024-09-16T00:28:24.336576+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:28:24.336576+0000","last_undegraded":"2024-09-16T00:28:24.336576+0000","last_fullsized":"2024-09-16T00:28:24.336576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357054+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:26.357054+0000","last_peered":"2024-09-16T00:27:26.357054+0000","last_clean":"2024-09-16T00:27:26.357054+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:26.357054+0000","last_undegraded":"2024-09-16T00:27:26.357054+0000","last_fullsized":"2024-09-16T00:27:26.357054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357365+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:27:26.357365+0000","last_peered":"2024-09-16T00:27:26.357365+0000","last_clean":"2024-09-16T00:27:26.357365+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:27:26.357365+0000","last_undegraded":"2024-09-16T00:27:26.357365+0000","last_fullsized":"2024-09-16T00:27:26.357365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355730+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:27:26.355730+0000","last_peered":"2024-09-16T00:27:26.355730+0000","last_clean":"2024-09-16T00:27:26.355730+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:27:26.355730+0000","last_undegraded":"2024-09-16T00:27:26.355730+0000","last_fullsized":"2024-09-16T00:27:26.355730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338444+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:28:24.338444+0000","last_peered":"2024-09-16T00:28:24.338444+0000","last_clean":"2024-09-16T00:28:24.338444+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:28:24.338444+0000","last_undegraded":"2024-09-16T00:28:24.338444+0000","last_fullsized":"2024-09-16T00:28:24.338444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248838+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:26.248838+0000","last_peered":"2024-09-16T00:27:26.248838+0000","last_clean":"2024-09-16T00:27:26.248838+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:26.248838+0000","last_undegraded":"2024-09-16T00:27:26.248838+0000","last_fullsized":"2024-09-16T00:27:26.248838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355174+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:27:26.355174+0000","last_peered":"2024-09-16T00:27:26.355174+0000","last_clean":"2024-09-16T00:27:26.355174+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:27:26.355174+0000","last_undegraded":"2024-09-16T00:27:26.355174+0000","last_fullsized":"2024-09-16T00:27:26.355174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331484+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:28:24.331484+0000","last_peered":"2024-09-16T00:28:24.331484+0000","last_clean":"2024-09-16T00:28:24.331484+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:28:24.331484+0000","last_undegraded":"2024-09-16T00:28:24.331484+0000","last_fullsized":"2024-09-16T00:28:24.331484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356422+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:27:26.356422+0000","last_peered":"2024-09-16T00:27:26.356422+0000","last_clean":"2024-09-16T00:27:26.356422+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:27:26.356422+0000","last_undegraded":"2024-09-16T00:27:26.356422+0000","last_fullsized":"2024-09-16T00:27:26.356422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249539+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:27:26.249539+0000","last_peered":"2024-09-16T00:27:26.249539+0000","last_clean":"2024-09-16T00:27:26.249539+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:27:26.249539+0000","last_undegraded":"2024-09-16T00:27:26.249539+0000","last_fullsized":"2024-09-16T00:27:26.249539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.333565+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:28:24.333565+0000","last_peered":"2024-09-16T00:28:24.333565+0000","last_clean":"2024-09-16T00:28:24.333565+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:28:24.333565+0000","last_undegraded":"2024-09-16T00:28:24.333565+0000","last_fullsized":"2024-09-16T00:28:24.333565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250975+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:27:26.250975+0000","last_peered":"2024-09-16T00:27:26.250975+0000","last_clean":"2024-09-16T00:27:26.250975+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:27:26.250975+0000","last_undegraded":"2024-09-16T00:27:26.250975+0000","last_fullsized":"2024-09-16T00:27:26.250975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354985+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:27:26.354985+0000","last_peered":"2024-09-16T00:27:26.354985+0000","last_clean":"2024-09-16T00:27:26.354985+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:27:26.354985+0000","last_undegraded":"2024-09-16T00:27:26.354985+0000","last_fullsized":"2024-09-16T00:27:26.354985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.319032+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:28:24.319032+0000","last_peered":"2024-09-16T00:28:24.319032+0000","last_clean":"2024-09-16T00:28:24.319032+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:28:24.319032+0000","last_undegraded":"2024-09-16T00:28:24.319032+0000","last_fullsized":"2024-09-16T00:28:24.319032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357916+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:26.357916+0000","last_peered":"2024-09-16T00:27:26.357916+0000","last_clean":"2024-09-16T00:27:26.357916+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:26.357916+0000","last_undegraded":"2024-09-16T00:27:26.357916+0000","last_fullsized":"2024-09-16T00:27:26.357916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355183+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:27:26.355183+0000","last_peered":"2024-09-16T00:27:26.355183+0000","last_clean":"2024-09-16T00:27:26.355183+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:27:26.355183+0000","last_undegraded":"2024-09-16T00:27:26.355183+0000","last_fullsized":"2024-09-16T00:27:26.355183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336143+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:28:24.336143+0000","last_peered":"2024-09-16T00:28:24.336143+0000","last_clean":"2024-09-16T00:28:24.336143+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:28:24.336143+0000","last_undegraded":"2024-09-16T00:28:24.336143+0000","last_fullsized":"2024-09-16T00:28:24.336143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356998+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:26.356998+0000","last_peered":"2024-09-16T00:27:26.356998+0000","last_clean":"2024-09-16T00:27:26.356998+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:26.356998+0000","last_undegraded":"2024-09-16T00:27:26.356998+0000","last_fullsized":"2024-09-16T00:27:26.356998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.253824+0000","last_change":"2024-09-16T00:27:26.253824+0000","last_active":"2024-09-16T00:27:26.253824+0000","last_peered":"2024-09-16T00:27:26.253824+0000","last_clean":"2024-09-16T00:27:26.253824+0000","last_became_active":"2024-09-16T00:27:26.253206+0000","last_became_peered":"2024-09-16T00:27:26.253206+0000","last_unstale":"2024-09-16T00:27:26.253824+0000","last_undegraded":"2024-09-16T00:27:26.253824+0000","last_fullsized":"2024-09-16T00:27:26.253824+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:22.543369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332839+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:28:24.332839+0000","last_peered":"2024-09-16T00:28:24.332839+0000","last_clean":"2024-09-16T00:28:24.332839+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:28:24.332839+0000","last_undegraded":"2024-09-16T00:28:24.332839+0000","last_fullsized":"2024-09-16T00:28:24.332839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:48.674171+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:28:48.674171+0000","last_peered":"2024-09-16T00:28:48.674171+0000","last_clean":"2024-09-16T00:28:48.674171+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:28:48.674171+0000","last_undegraded":"2024-09-16T00:28:48.674171+0000","last_fullsized":"2024-09-16T00:28:48.674171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252554+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:27:26.252554+0000","last_peered":"2024-09-16T00:27:26.252554+0000","last_clean":"2024-09-16T00:27:26.252554+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:27:26.252554+0000","last_undegraded":"2024-09-16T00:27:26.252554+0000","last_fullsized":"2024-09-16T00:27:26.252554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.335224+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:28:24.335224+0000","last_peered":"2024-09-16T00:28:24.335224+0000","last_clean":"2024-09-16T00:28:24.335224+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:28:24.335224+0000","last_undegraded":"2024-09-16T00:28:24.335224+0000","last_fullsized":"2024-09-16T00:28:24.335224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356320+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:26.356320+0000","last_peered":"2024-09-16T00:27:26.356320+0000","last_clean":"2024-09-16T00:27:26.356320+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:26.356320+0000","last_undegraded":"2024-09-16T00:27:26.356320+0000","last_fullsized":"2024-09-16T00:27:26.356320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252594+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:27:26.252594+0000","last_peered":"2024-09-16T00:27:26.252594+0000","last_clean":"2024-09-16T00:27:26.252594+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:27:26.252594+0000","last_undegraded":"2024-09-16T00:27:26.252594+0000","last_fullsized":"2024-09-16T00:27:26.252594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330126+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:28:24.330126+0000","last_peered":"2024-09-16T00:28:24.330126+0000","last_clean":"2024-09-16T00:28:24.330126+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:28:24.330126+0000","last_undegraded":"2024-09-16T00:28:24.330126+0000","last_fullsized":"2024-09-16T00:28:24.330126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248474+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:26.248474+0000","last_peered":"2024-09-16T00:27:26.248474+0000","last_clean":"2024-09-16T00:27:26.248474+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:26.248474+0000","last_undegraded":"2024-09-16T00:27:26.248474+0000","last_fullsized":"2024-09-16T00:27:26.248474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355990+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:27:26.355990+0000","last_peered":"2024-09-16T00:27:26.355990+0000","last_clean":"2024-09-16T00:27:26.355990+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:27:26.355990+0000","last_undegraded":"2024-09-16T00:27:26.355990+0000","last_fullsized":"2024-09-16T00:27:26.355990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.311816+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:28:24.311816+0000","last_peered":"2024-09-16T00:28:24.311816+0000","last_clean":"2024-09-16T00:28:24.311816+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:28:24.311816+0000","last_undegraded":"2024-09-16T00:28:24.311816+0000","last_fullsized":"2024-09-16T00:28:24.311816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":313,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:46.675247+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:28:46.675247+0000","last_peered":"2024-09-16T00:28:46.675247+0000","last_clean":"2024-09-16T00:28:46.675247+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:28:46.675247+0000","last_undegraded":"2024-09-16T00:28:46.675247+0000","last_fullsized":"2024-09-16T00:28:46.675247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68006,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":222,"num_read_kb":178,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359395+0000","last_change":"2024-09-16T00:27:26.358804+0000","last_active":"2024-09-16T00:27:26.359395+0000","last_peered":"2024-09-16T00:27:26.359395+0000","last_clean":"2024-09-16T00:27:26.359395+0000","last_became_active":"2024-09-16T00:27:26.358317+0000","last_became_peered":"2024-09-16T00:27:26.358317+0000","last_unstale":"2024-09-16T00:27:26.359395+0000","last_undegraded":"2024-09-16T00:27:26.359395+0000","last_fullsized":"2024-09-16T00:27:26.359395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:38.642392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336469+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:28:24.336469+0000","last_peered":"2024-09-16T00:28:24.336469+0000","last_clean":"2024-09-16T00:28:24.336469+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:28:24.336469+0000","last_undegraded":"2024-09-16T00:28:24.336469+0000","last_fullsized":"2024-09-16T00:28:24.336469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356307+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:26.356307+0000","last_peered":"2024-09-16T00:27:26.356307+0000","last_clean":"2024-09-16T00:27:26.356307+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:26.356307+0000","last_undegraded":"2024-09-16T00:27:26.356307+0000","last_fullsized":"2024-09-16T00:27:26.356307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356802+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:27:26.356802+0000","last_peered":"2024-09-16T00:27:26.356802+0000","last_clean":"2024-09-16T00:27:26.356802+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:27:26.356802+0000","last_undegraded":"2024-09-16T00:27:26.356802+0000","last_fullsized":"2024-09-16T00:27:26.356802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320577+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:28:24.320577+0000","last_peered":"2024-09-16T00:28:24.320577+0000","last_clean":"2024-09-16T00:28:24.320577+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:28:24.320577+0000","last_undegraded":"2024-09-16T00:28:24.320577+0000","last_fullsized":"2024-09-16T00:28:24.320577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357240+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:26.357240+0000","last_peered":"2024-09-16T00:27:26.357240+0000","last_clean":"2024-09-16T00:27:26.357240+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:26.357240+0000","last_undegraded":"2024-09-16T00:27:26.357240+0000","last_fullsized":"2024-09-16T00:27:26.357240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252178+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:27:26.252178+0000","last_peered":"2024-09-16T00:27:26.252178+0000","last_clean":"2024-09-16T00:27:26.252178+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:27:26.252178+0000","last_undegraded":"2024-09-16T00:27:26.252178+0000","last_fullsized":"2024-09-16T00:27:26.252178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336948+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:28:24.336948+0000","last_peered":"2024-09-16T00:28:24.336948+0000","last_clean":"2024-09-16T00:28:24.336948+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:28:24.336948+0000","last_undegraded":"2024-09-16T00:28:24.336948+0000","last_fullsized":"2024-09-16T00:28:24.336948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":147,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:45.673811+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:28:45.673811+0000","last_peered":"2024-09-16T00:28:45.673811+0000","last_clean":"2024-09-16T00:28:45.673811+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:28:45.673811+0000","last_undegraded":"2024-09-16T00:28:45.673811+0000","last_fullsized":"2024-09-16T00:28:45.673811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247326+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:27:26.247326+0000","last_peered":"2024-09-16T00:27:26.247326+0000","last_clean":"2024-09-16T00:27:26.247326+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:27:26.247326+0000","last_undegraded":"2024-09-16T00:27:26.247326+0000","last_fullsized":"2024-09-16T00:27:26.247326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250057+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:27:26.250057+0000","last_peered":"2024-09-16T00:27:26.250057+0000","last_clean":"2024-09-16T00:27:26.250057+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:27:26.250057+0000","last_undegraded":"2024-09-16T00:27:26.250057+0000","last_fullsized":"2024-09-16T00:27:26.250057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331405+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:28:24.331405+0000","last_peered":"2024-09-16T00:28:24.331405+0000","last_clean":"2024-09-16T00:28:24.331405+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:28:24.331405+0000","last_undegraded":"2024-09-16T00:28:24.331405+0000","last_fullsized":"2024-09-16T00:28:24.331405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355413+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:27:26.355413+0000","last_peered":"2024-09-16T00:27:26.355413+0000","last_clean":"2024-09-16T00:27:26.355413+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:27:26.355413+0000","last_undegraded":"2024-09-16T00:27:26.355413+0000","last_fullsized":"2024-09-16T00:27:26.355413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.360047+0000","last_change":"2024-09-16T00:27:26.359230+0000","last_active":"2024-09-16T00:27:26.360047+0000","last_peered":"2024-09-16T00:27:26.360047+0000","last_clean":"2024-09-16T00:27:26.360047+0000","last_became_active":"2024-09-16T00:27:26.358662+0000","last_became_peered":"2024-09-16T00:27:26.358662+0000","last_unstale":"2024-09-16T00:27:26.360047+0000","last_undegraded":"2024-09-16T00:27:26.360047+0000","last_fullsized":"2024-09-16T00:27:26.360047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:52.111910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331664+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:28:24.331664+0000","last_peered":"2024-09-16T00:28:24.331664+0000","last_clean":"2024-09-16T00:28:24.331664+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:28:24.331664+0000","last_undegraded":"2024-09-16T00:28:24.331664+0000","last_fullsized":"2024-09-16T00:28:24.331664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250512+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:26.250512+0000","last_peered":"2024-09-16T00:27:26.250512+0000","last_clean":"2024-09-16T00:27:26.250512+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:26.250512+0000","last_undegraded":"2024-09-16T00:27:26.250512+0000","last_fullsized":"2024-09-16T00:27:26.250512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250964+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:26.250964+0000","last_peered":"2024-09-16T00:27:26.250964+0000","last_clean":"2024-09-16T00:27:26.250964+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:26.250964+0000","last_undegraded":"2024-09-16T00:27:26.250964+0000","last_fullsized":"2024-09-16T00:27:26.250964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334137+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:28:24.334137+0000","last_peered":"2024-09-16T00:28:24.334137+0000","last_clean":"2024-09-16T00:28:24.334137+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:28:24.334137+0000","last_undegraded":"2024-09-16T00:28:24.334137+0000","last_fullsized":"2024-09-16T00:28:24.334137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359360+0000","last_change":"2024-09-16T00:27:26.359360+0000","last_active":"2024-09-16T00:27:26.359360+0000","last_peered":"2024-09-16T00:27:26.359360+0000","last_clean":"2024-09-16T00:27:26.359360+0000","last_became_active":"2024-09-16T00:27:26.358462+0000","last_became_peered":"2024-09-16T00:27:26.358462+0000","last_unstale":"2024-09-16T00:27:26.359360+0000","last_undegraded":"2024-09-16T00:27:26.359360+0000","last_fullsized":"2024-09-16T00:27:26.359360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:38.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:49.680040+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:28:49.680040+0000","last_peered":"2024-09-16T00:28:49.680040+0000","last_clean":"2024-09-16T00:28:49.680040+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:28:49.680040+0000","last_undegraded":"2024-09-16T00:28:49.680040+0000","last_fullsized":"2024-09-16T00:28:49.680040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320803+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:28:24.320803+0000","last_peered":"2024-09-16T00:28:24.320803+0000","last_clean":"2024-09-16T00:28:24.320803+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:28:24.320803+0000","last_undegraded":"2024-09-16T00:28:24.320803+0000","last_fullsized":"2024-09-16T00:28:24.320803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248757+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:27:26.248757+0000","last_peered":"2024-09-16T00:27:26.248757+0000","last_clean":"2024-09-16T00:27:26.248757+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:27:26.248757+0000","last_undegraded":"2024-09-16T00:27:26.248757+0000","last_fullsized":"2024-09-16T00:27:26.248757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321717+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:28:24.321717+0000","last_peered":"2024-09-16T00:28:24.321717+0000","last_clean":"2024-09-16T00:28:24.321717+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:28:24.321717+0000","last_undegraded":"2024-09-16T00:28:24.321717+0000","last_fullsized":"2024-09-16T00:28:24.321717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321331+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:28:24.321331+0000","last_peered":"2024-09-16T00:28:24.321331+0000","last_clean":"2024-09-16T00:28:24.321331+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:28:24.321331+0000","last_undegraded":"2024-09-16T00:28:24.321331+0000","last_fullsized":"2024-09-16T00:28:24.321331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329213+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:28:24.329213+0000","last_peered":"2024-09-16T00:28:24.329213+0000","last_clean":"2024-09-16T00:28:24.329213+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:28:24.329213+0000","last_undegraded":"2024-09-16T00:28:24.329213+0000","last_fullsized":"2024-09-16T00:28:24.329213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326265+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:28:24.326265+0000","last_peered":"2024-09-16T00:28:24.326265+0000","last_clean":"2024-09-16T00:28:24.326265+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:28:24.326265+0000","last_undegraded":"2024-09-16T00:28:24.326265+0000","last_fullsized":"2024-09-16T00:28:24.326265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327220+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:28:24.327220+0000","last_peered":"2024-09-16T00:28:24.327220+0000","last_clean":"2024-09-16T00:28:24.327220+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:28:24.327220+0000","last_undegraded":"2024-09-16T00:28:24.327220+0000","last_fullsized":"2024-09-16T00:28:24.327220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.316594+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:28:24.316594+0000","last_peered":"2024-09-16T00:28:24.316594+0000","last_clean":"2024-09-16T00:28:24.316594+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:28:24.316594+0000","last_undegraded":"2024-09-16T00:28:24.316594+0000","last_fullsized":"2024-09-16T00:28:24.316594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337588+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:28:24.337588+0000","last_peered":"2024-09-16T00:28:24.337588+0000","last_clean":"2024-09-16T00:28:24.337588+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:28:24.337588+0000","last_undegraded":"2024-09-16T00:28:24.337588+0000","last_fullsized":"2024-09-16T00:28:24.337588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322996+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:28:24.322996+0000","last_peered":"2024-09-16T00:28:24.322996+0000","last_clean":"2024-09-16T00:28:24.322996+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:28:24.322996+0000","last_undegraded":"2024-09-16T00:28:24.322996+0000","last_fullsized":"2024-09-16T00:28:24.322996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322343+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:28:24.322343+0000","last_peered":"2024-09-16T00:28:24.322343+0000","last_clean":"2024-09-16T00:28:24.322343+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:28:24.322343+0000","last_undegraded":"2024-09-16T00:28:24.322343+0000","last_fullsized":"2024-09-16T00:28:24.322343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330356+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:28:24.330356+0000","last_peered":"2024-09-16T00:28:24.330356+0000","last_clean":"2024-09-16T00:28:24.330356+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:28:24.330356+0000","last_undegraded":"2024-09-16T00:28:24.330356+0000","last_fullsized":"2024-09-16T00:28:24.330356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337238+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:28:24.337238+0000","last_peered":"2024-09-16T00:28:24.337238+0000","last_clean":"2024-09-16T00:28:24.337238+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:28:24.337238+0000","last_undegraded":"2024-09-16T00:28:24.337238+0000","last_fullsized":"2024-09-16T00:28:24.337238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330609+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:28:24.330609+0000","last_peered":"2024-09-16T00:28:24.330609+0000","last_clean":"2024-09-16T00:28:24.330609+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:28:24.330609+0000","last_undegraded":"2024-09-16T00:28:24.330609+0000","last_fullsized":"2024-09-16T00:28:24.330609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.317884+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:28:24.317884+0000","last_peered":"2024-09-16T00:28:24.317884+0000","last_clean":"2024-09-16T00:28:24.317884+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:28:24.317884+0000","last_undegraded":"2024-09-16T00:28:24.317884+0000","last_fullsized":"2024-09-16T00:28:24.317884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329300+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:28:24.329300+0000","last_peered":"2024-09-16T00:28:24.329300+0000","last_clean":"2024-09-16T00:28:24.329300+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:28:24.329300+0000","last_undegraded":"2024-09-16T00:28:24.329300+0000","last_fullsized":"2024-09-16T00:28:24.329300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337159+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:28:24.337159+0000","last_peered":"2024-09-16T00:28:24.337159+0000","last_clean":"2024-09-16T00:28:24.337159+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:28:24.337159+0000","last_undegraded":"2024-09-16T00:28:24.337159+0000","last_fullsized":"2024-09-16T00:28:24.337159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336933+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:28:24.336933+0000","last_peered":"2024-09-16T00:28:24.336933+0000","last_clean":"2024-09-16T00:28:24.336933+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:28:24.336933+0000","last_undegraded":"2024-09-16T00:28:24.336933+0000","last_fullsized":"2024-09-16T00:28:24.336933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337483+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:28:24.337483+0000","last_peered":"2024-09-16T00:28:24.337483+0000","last_clean":"2024-09-16T00:28:24.337483+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:28:24.337483+0000","last_undegraded":"2024-09-16T00:28:24.337483+0000","last_fullsized":"2024-09-16T00:28:24.337483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326582+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:28:24.326582+0000","last_peered":"2024-09-16T00:28:24.326582+0000","last_clean":"2024-09-16T00:28:24.326582+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:28:24.326582+0000","last_undegraded":"2024-09-16T00:28:24.326582+0000","last_fullsized":"2024-09-16T00:28:24.326582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326496+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:28:24.326496+0000","last_peered":"2024-09-16T00:28:24.326496+0000","last_clean":"2024-09-16T00:28:24.326496+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:28:24.326496+0000","last_undegraded":"2024-09-16T00:28:24.326496+0000","last_fullsized":"2024-09-16T00:28:24.326496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326067+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:28:24.326067+0000","last_peered":"2024-09-16T00:28:24.326067+0000","last_clean":"2024-09-16T00:28:24.326067+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:28:24.326067+0000","last_undegraded":"2024-09-16T00:28:24.326067+0000","last_fullsized":"2024-09-16T00:28:24.326067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320083+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:28:24.320083+0000","last_peered":"2024-09-16T00:28:24.320083+0000","last_clean":"2024-09-16T00:28:24.320083+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:28:24.320083+0000","last_undegraded":"2024-09-16T00:28:24.320083+0000","last_fullsized":"2024-09-16T00:28:24.320083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338288+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:28:24.338288+0000","last_peered":"2024-09-16T00:28:24.338288+0000","last_clean":"2024-09-16T00:28:24.338288+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:28:24.338288+0000","last_undegraded":"2024-09-16T00:28:24.338288+0000","last_fullsized":"2024-09-16T00:28:24.338288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.328897+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:28:24.328897+0000","last_peered":"2024-09-16T00:28:24.328897+0000","last_clean":"2024-09-16T00:28:24.328897+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:28:24.328897+0000","last_undegraded":"2024-09-16T00:28:24.328897+0000","last_fullsized":"2024-09-16T00:28:24.328897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334467+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:28:24.334467+0000","last_peered":"2024-09-16T00:28:24.334467+0000","last_clean":"2024-09-16T00:28:24.334467+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:28:24.334467+0000","last_undegraded":"2024-09-16T00:28:24.334467+0000","last_fullsized":"2024-09-16T00:28:24.334467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324496+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:28:24.324496+0000","last_peered":"2024-09-16T00:28:24.324496+0000","last_clean":"2024-09-16T00:28:24.324496+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:28:24.324496+0000","last_undegraded":"2024-09-16T00:28:24.324496+0000","last_fullsized":"2024-09-16T00:28:24.324496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334306+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:28:24.334306+0000","last_peered":"2024-09-16T00:28:24.334306+0000","last_clean":"2024-09-16T00:28:24.334306+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:28:24.334306+0000","last_undegraded":"2024-09-16T00:28:24.334306+0000","last_fullsized":"2024-09-16T00:28:24.334306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327306+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:28:24.327306+0000","last_peered":"2024-09-16T00:28:24.327306+0000","last_clean":"2024-09-16T00:28:24.327306+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:28:24.327306+0000","last_undegraded":"2024-09-16T00:28:24.327306+0000","last_fullsized":"2024-09-16T00:28:24.327306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.309954+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:28:24.309954+0000","last_peered":"2024-09-16T00:28:24.309954+0000","last_clean":"2024-09-16T00:28:24.309954+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:28:24.309954+0000","last_undegraded":"2024-09-16T00:28:24.309954+0000","last_fullsized":"2024-09-16T00:28:24.309954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320484+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:28:24.320484+0000","last_peered":"2024-09-16T00:28:24.320484+0000","last_clean":"2024-09-16T00:28:24.320484+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:28:24.320484+0000","last_undegraded":"2024-09-16T00:28:24.320484+0000","last_fullsized":"2024-09-16T00:28:24.320484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.318969+0000","last_change":"2024-09-16T00:27:26.358604+0000","last_active":"2024-09-16T00:28:24.318969+0000","last_peered":"2024-09-16T00:28:24.318969+0000","last_clean":"2024-09-16T00:28:24.318969+0000","last_became_active":"2024-09-16T00:27:26.358225+0000","last_became_peered":"2024-09-16T00:27:26.358225+0000","last_unstale":"2024-09-16T00:28:24.318969+0000","last_undegraded":"2024-09-16T00:28:24.318969+0000","last_fullsized":"2024-09-16T00:28:24.318969+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:21.806500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326711+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:28:24.326711+0000","last_peered":"2024-09-16T00:28:24.326711+0000","last_clean":"2024-09-16T00:28:24.326711+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:28:24.326711+0000","last_undegraded":"2024-09-16T00:28:24.326711+0000","last_fullsized":"2024-09-16T00:28:24.326711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336940+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:28:24.336940+0000","last_peered":"2024-09-16T00:28:24.336940+0000","last_clean":"2024-09-16T00:28:24.336940+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:28:24.336940+0000","last_undegraded":"2024-09-16T00:28:24.336940+0000","last_fullsized":"2024-09-16T00:28:24.336940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72458,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":668,"num_read_kb":408,"num_write":180,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":217374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":515002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743473,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1732608,"data_stored":1170276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906998,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":519334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":515158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364864,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1646592,"data_stored":1111518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528389,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1107106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":583070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":510608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":512516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":578330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":511802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:28:52.463 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:28:52.542 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:52.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:52 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:28:52.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:52 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:52.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:52 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:28:52.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:52 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:53.013 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:52 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:28:53.014 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:52 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:53.014 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:52 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:28:53.014 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:52 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 -- 172.21.15.1:0/3416379350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc102d40 msgr2=0x7faccc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 --2- 172.21.15.1:0/3416379350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc102d40 0x7faccc103120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7facbc009ca0 tx=0x7facbc02a950 comp rx=0 tx=0).stop 2024-09-16T00:28:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 -- 172.21.15.1:0/3416379350 shutdown_connections 2024-09-16T00:28:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 --2- 172.21.15.1:0/3416379350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faccc10c170 0x7faccc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 --2- 172.21.15.1:0/3416379350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faccc103660 0x7faccc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 --2- 172.21.15.1:0/3416379350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc102d40 0x7faccc103120 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 -- 172.21.15.1:0/3416379350 >> 172.21.15.1:0/3416379350 conn(0x7faccc0fc7a0 msgr2=0x7faccc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 -- 172.21.15.1:0/3416379350 shutdown_connections 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 -- 172.21.15.1:0/3416379350 wait complete. 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.098+0000 7facd202a640 1 Processor -- start 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7facd202a640 1 -- start start 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7facd202a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faccc102d40 0x7faccc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7facd202a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faccc103660 0x7faccc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7facd202a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc10c170 0x7faccc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7facd202a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faccc111f70 con 0x7faccc103660 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7facd202a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faccc111df0 con 0x7faccc10c170 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7facd202a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1120f0 con 0x7faccc102d40 2024-09-16T00:28:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faccc103660 0x7faccc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faccc103660 0x7faccc198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38508/0 (socket says 172.21.15.1:38508) 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccaffd640 1 -- 172.21.15.1:0/1121226344 learned_addr learned my addr 172.21.15.1:0/1121226344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccb7fe640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faccc102d40 0x7faccc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccbfff640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc10c170 0x7faccc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccaffd640 1 -- 172.21.15.1:0/1121226344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faccc102d40 msgr2=0x7faccc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccaffd640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faccc102d40 0x7faccc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccaffd640 1 -- 172.21.15.1:0/1121226344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc10c170 msgr2=0x7faccc19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccaffd640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc10c170 0x7faccc19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccaffd640 1 -- 172.21.15.1:0/1121226344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc19d4a0 con 0x7faccc103660 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.099+0000 7faccbfff640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc10c170 0x7faccc19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.100+0000 7faccaffd640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faccc103660 0x7faccc198a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7facc000e9f0 tx=0x7facc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.100+0000 7facc8ff9640 1 -- 172.21.15.1:0/1121226344 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc000ccc0 con 0x7faccc103660 2024-09-16T00:28:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.100+0000 7facd202a640 1 -- 172.21.15.1:0/1121226344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc19d790 con 0x7faccc103660 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.100+0000 7facc8ff9640 1 -- 172.21.15.1:0/1121226344 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc0010910 con 0x7faccc103660 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.100+0000 7facc8ff9640 1 -- 172.21.15.1:0/1121226344 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0018920 con 0x7faccc103660 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.100+0000 7facd202a640 1 -- 172.21.15.1:0/1121226344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc1a5070 con 0x7faccc103660 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.102+0000 7facc8ff9640 1 -- 172.21.15.1:0/1121226344 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7facc0010430 con 0x7faccc103660 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.102+0000 7facd4403740 1 Processor -- start 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.102+0000 7facd4403740 1 -- start start 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7facd4403740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cb360 0x564e267cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7facd4403740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267da550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7facd4403740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267daa90 0x564e267dce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7facd4403740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564e267312e0 con 0x564e267cb360 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7facd4403740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564e266d8ce0 con 0x564e267cbc80 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7facd4403740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e2672d2a0 con 0x564e267daa90 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cb360 0x564e267cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cb360 0x564e267cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38516/0 (socket says 172.21.15.1:38516) 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccbfff640 1 -- 172.21.15.1:0/482740088 learned_addr learned my addr 172.21.15.1:0/482740088 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7facc8ff9640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7facb40a3240 0x7facb40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccb7fe640 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267da550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccaffd640 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267daa90 0x564e267dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccbfff640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7facb40a3240 0x7facb40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7facc8ff9640 1 -- 172.21.15.1:0/1121226344 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7facc00d0190 con 0x7faccc103660 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccb7fe640 1 -- 172.21.15.1:0/482740088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267daa90 msgr2=0x564e267dce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccb7fe640 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267daa90 0x564e267dce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccb7fe640 1 -- 172.21.15.1:0/482740088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cb360 msgr2=0x564e267cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccb7fe640 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cb360 0x564e267cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccb7fe640 1 -- 172.21.15.1:0/482740088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e266bd440 con 0x564e267cbc80 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.103+0000 7faccbfff640 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cb360 0x564e267cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.104+0000 7faccbfff640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7facb40a3240 0x7facb40a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7facc4014a40 tx=0x7facc4004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.104+0000 7faccb7fe640 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267da550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7facbc000f80 tx=0x7facbc005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b9ed3eee27e133 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.104+0000 7facb1ffb640 1 -- 172.21.15.1:0/482740088 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc034040 con 0x564e267cbc80 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.104+0000 7facb1ffb640 1 -- 172.21.15.1:0/482740088 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facbc02fb40 con 0x564e267cbc80 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.104+0000 7facb1ffb640 1 -- 172.21.15.1:0/482740088 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc004330 con 0x564e267cbc80 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 -- 172.21.15.1:0/482740088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 msgr2=0x564e267da550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267da550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7facbc000f80 tx=0x7facbc005f60 comp rx=0 tx=0).stop 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 -- 172.21.15.1:0/482740088 shutdown_connections 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267daa90 0x564e267dce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267da550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 --2- 172.21.15.1:0/482740088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cb360 0x564e267cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 -- 172.21.15.1:0/482740088 >> 172.21.15.1:0/482740088 conn(0x564e267c7a30 msgr2=0x564e267c7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 -- 172.21.15.1:0/482740088 shutdown_connections 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 -- 172.21.15.1:0/482740088 wait complete. 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.105+0000 7facd4403740 1 -- start start 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7facd4403740 1 Processor -- start 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7facd4403740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cb360 0x564e267da3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7facd4403740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7facd4403740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267daa90 0x564e267d7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7facd4403740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564e267dd600 con 0x564e267daa90 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7facd4403740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564e266ec050 con 0x564e267cbc80 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7facd4403740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e267dd780 con 0x564e267cb360 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7facd4403740 10 client.0 _ll_register_callbacks cb 0x564e2672c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:53.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267daa90 0x564e267d7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267daa90 0x564e267d7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38526/0 (socket says 172.21.15.1:38526) 2024-09-16T00:28:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccbfff640 1 -- 172.21.15.1:0/2568533822 learned_addr learned my addr 172.21.15.1:0/2568533822 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccb7fe640 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cb360 0x564e267da3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccaffd640 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccbfff640 1 -- 172.21.15.1:0/2568533822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cb360 msgr2=0x564e267da3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccbfff640 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cb360 0x564e267da3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccbfff640 1 -- 172.21.15.1:0/2568533822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 msgr2=0x564e267d7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccbfff640 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267d7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.106+0000 7faccbfff640 1 -- 172.21.15.1:0/2568533822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e264a23c0 con 0x564e267daa90 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7faccaffd640 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267d7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7faccbfff640 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267daa90 0x564e267d7ba0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7facc4004390 tx=0x7facc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a707a39330ec3a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7facaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7facaaffd640 1 -- 172.21.15.1:0/2568533822 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4025070 con 0x564e267daa90 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7facd4403740 1 -- 172.21.15.1:0/2568533822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facbc02fb40 con 0x564e267daa90 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7facaaffd640 1 -- 172.21.15.1:0/2568533822 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7facc4003ac0 con 0x564e267daa90 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7facaaffd640 1 -- 172.21.15.1:0/2568533822 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facc4033050 con 0x564e267daa90 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7facd4403740 10 client.15336 fetch_fsmap learned FSMap version 25 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7facd4403740 1 -- 172.21.15.1:0/2568533822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e267e9070 con 0x564e267daa90 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.107+0000 7facaaffd640 1 -- 172.21.15.1:0/2568533822 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facbc02fb40 con 0x564e267daa90 2024-09-16T00:28:53.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7facaaffd640 1 -- 172.21.15.1:0/2568533822 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7facc400f040 con 0x564e267daa90 2024-09-16T00:28:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7facd4403740 10 client.15336 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:28:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7facd4403740 10 client.15336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7facd4403740 20 client.15336 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7facd4403740 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e267ea530 0x564e267ec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7facd4403740 4 client.15336 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7facd4403740 1 -- 172.21.15.1:0/2568533822 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e2673e850 con 0x564e267ea530 2024-09-16T00:28:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7faccb7fe640 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e267ea530 0x564e267ec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.118 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:52 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:28:53.118 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:52 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:53.118 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:52 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:28:53.118 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:52 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7facaaffd640 10 client.15336 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.108+0000 7faccb7fe640 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e267ea530 0x564e267ec980 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424c3624a9bd9ea7 server_cookie=11d05739c2dc38b1 in_seq=0 out_seq=0 2024-09-16T00:28:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facaaffd640 1 -- 172.21.15.1:0/2568533822 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e2673e850 con 0x564e267ea530 2024-09-16T00:28:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facaaffd640 10 client.15336 handle_command_reply: tid=0 2024-09-16T00:28:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facd4403740 1 client.15336 shutdown 2024-09-16T00:28:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facd4403740 1 -- 172.21.15.1:0/2568533822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267daa90 msgr2=0x564e267d7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facd4403740 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267daa90 0x564e267d7ba0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7facc4004390 tx=0x7facc40046c0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facd4403740 1 -- 172.21.15.1:0/2568533822 shutdown_connections 2024-09-16T00:28:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facd4403740 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e267ea530 0x564e267ec980 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facd4403740 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267daa90 0x564e267d7ba0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facd4403740 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cbc80 0x564e267d7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facd4403740 1 --2- 172.21.15.1:0/2568533822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cb360 0x564e267da3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.109+0000 7facd4403740 1 -- 172.21.15.1:0/2568533822 >> 172.21.15.1:0/2568533822 conn(0x564e267c7a30 msgr2=0x564e267cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.110+0000 7facaaffd640 0 client.15336 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.110+0000 7facd4403740 1 -- 172.21.15.1:0/2568533822 shutdown_connections 2024-09-16T00:28:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.110+0000 7facd4403740 1 -- 172.21.15.1:0/2568533822 wait complete. 2024-09-16T00:28:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.110+0000 7facd4403740 20 client.15336 trim_cache size 0 max 16384 2024-09-16T00:28:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.127+0000 7facd4403740 1 Processor -- start 2024-09-16T00:28:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.127+0000 7facd4403740 1 -- start start 2024-09-16T00:28:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.127+0000 7facd4403740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267c7a30 0x564e267e68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.127+0000 7facd4403740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.127+0000 7facd4403740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cbc80 0x564e267f2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.127+0000 7facd4403740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1af540 con 0x564e267c7a30 2024-09-16T00:28:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.127+0000 7facd4403740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1ace10 con 0x564e267cb360 2024-09-16T00:28:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.127+0000 7facd4403740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1aac60 con 0x564e267cbc80 2024-09-16T00:28:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267c7a30 0x564e267e68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267d5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:49048/0 (socket says 172.21.15.1:49048) 2024-09-16T00:28:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccaffd640 1 -- 172.21.15.1:0/3355820341 learned_addr learned my addr 172.21.15.1:0/3355820341 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccbfff640 1 --2- 172.21.15.1:0/3355820341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cbc80 0x564e267f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccaffd640 1 -- 172.21.15.1:0/3355820341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cbc80 msgr2=0x564e267f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccaffd640 1 --2- 172.21.15.1:0/3355820341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cbc80 0x564e267f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccaffd640 1 -- 172.21.15.1:0/3355820341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267c7a30 msgr2=0x564e267e68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccaffd640 1 --2- 172.21.15.1:0/3355820341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267c7a30 0x564e267e68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccaffd640 1 -- 172.21.15.1:0/3355820341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e267d6610 con 0x564e267cb360 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccb7fe640 1 --2- 172.21.15.1:0/3355820341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267c7a30 0x564e267e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.128+0000 7faccaffd640 1 --2- 172.21.15.1:0/3355820341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267d5f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7facc000cf10 tx=0x7facc00bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5340cfdfbb1d12bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.129+0000 7facb17fa640 1 -- 172.21.15.1:0/3355820341 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc00a4fc0 con 0x564e267cb360 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.129+0000 7facb17fa640 1 -- 172.21.15.1:0/3355820341 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc0010dc0 con 0x564e267cb360 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.129+0000 7facb17fa640 1 -- 172.21.15.1:0/3355820341 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0018ac0 con 0x564e267cb360 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.129+0000 7facd4403740 1 -- 172.21.15.1:0/3355820341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 msgr2=0x564e267d5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.129+0000 7facd4403740 1 --2- 172.21.15.1:0/3355820341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267d5f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7facc000cf10 tx=0x7facc00bc040 comp rx=0 tx=0).stop 2024-09-16T00:28:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.129+0000 7facd4403740 1 -- 172.21.15.1:0/3355820341 shutdown_connections 2024-09-16T00:28:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.129+0000 7facd4403740 1 --2- 172.21.15.1:0/3355820341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267cbc80 0x564e267f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.130+0000 7facd4403740 1 --2- 172.21.15.1:0/3355820341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267d5f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.130+0000 7facd4403740 1 --2- 172.21.15.1:0/3355820341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267c7a30 0x564e267e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.130+0000 7facd4403740 1 -- 172.21.15.1:0/3355820341 >> 172.21.15.1:0/3355820341 conn(0x564e267d7d00 msgr2=0x564e267d2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.130+0000 7facd4403740 1 -- 172.21.15.1:0/3355820341 shutdown_connections 2024-09-16T00:28:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.130+0000 7facd4403740 1 -- 172.21.15.1:0/3355820341 wait complete. 2024-09-16T00:28:53.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.130+0000 7facd4403740 1 -- start start 2024-09-16T00:28:53.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.130+0000 7facd4403740 1 Processor -- start 2024-09-16T00:28:53.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7facd4403740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267c7a30 0x564e267de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7facd4403740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267ded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7facd4403740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cbc80 0x564e267df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7facd4403740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1ab2d0 con 0x564e267cbc80 2024-09-16T00:28:53.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7facd4403740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1ac740 con 0x564e267cb360 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7facd4403740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1ab9a0 con 0x564e267c7a30 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7facd4403740 10 client.0 _ll_register_callbacks cb 0x7facc4014f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267ded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267ded30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:49054/0 (socket says 172.21.15.1:49054) 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccaffd640 1 -- 172.21.15.1:0/3555956849 learned_addr learned my addr 172.21.15.1:0/3555956849 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccb7fe640 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267c7a30 0x564e267de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccbfff640 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cbc80 0x564e267df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccaffd640 1 -- 172.21.15.1:0/3555956849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267c7a30 msgr2=0x564e267de7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccaffd640 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267c7a30 0x564e267de7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccaffd640 1 -- 172.21.15.1:0/3555956849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cbc80 msgr2=0x564e267df270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccaffd640 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cbc80 0x564e267df270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccaffd640 1 -- 172.21.15.1:0/3555956849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e26854270 con 0x564e267cb360 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccbfff640 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cbc80 0x564e267df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7faccaffd640 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267ded30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7facc009cc00 tx=0x7facc0005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52f7f5a8de8d64a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.131+0000 7facaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facaaffd640 1 -- 172.21.15.1:0/3555956849 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc00b4020 con 0x564e267cb360 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facaaffd640 1 -- 172.21.15.1:0/3555956849 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7facc0098a30 con 0x564e267cb360 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facd4403740 1 -- 172.21.15.1:0/3555956849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facc0010dc0 con 0x564e267cb360 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facaaffd640 1 -- 172.21.15.1:0/3555956849 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facc00a4fc0 con 0x564e267cb360 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facd4403740 10 client.25043 fetch_fsmap learned FSMap version 25 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facd4403740 1 -- 172.21.15.1:0/3555956849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e26855540 con 0x564e267cb360 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facaaffd640 1 -- 172.21.15.1:0/3555956849 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc0010dc0 con 0x564e267cb360 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facaaffd640 1 -- 172.21.15.1:0/3555956849 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7facc009b7a0 con 0x564e267cb360 2024-09-16T00:28:53.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facd4403740 10 client.25043 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:28:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facd4403740 10 client.25043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.132+0000 7facd4403740 20 client.25043 populate_metadata read hostname 'smithi001' 2024-09-16T00:28:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.133+0000 7facd4403740 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e267daa90 0x564e26857d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.133+0000 7facd4403740 4 client.25043 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:28:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.133+0000 7facd4403740 1 -- 172.21.15.1:0/3555956849 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e26738d60 con 0x564e267daa90 2024-09-16T00:28:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.133+0000 7faccb7fe640 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e267daa90 0x564e26857d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.133+0000 7faccb7fe640 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e267daa90 0x564e26857d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a77e71e81edc71e server_cookie=6c7c8fea19a80f2a in_seq=0 out_seq=0 2024-09-16T00:28:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.133+0000 7facaaffd640 10 client.25043 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.133+0000 7facaaffd640 1 -- 172.21.15.1:0/3555956849 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564e26738d60 con 0x564e267daa90 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.133+0000 7facaaffd640 10 client.25043 handle_command_reply: tid=0 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.133+0000 7facd4403740 1 client.25043 shutdown 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 -- 172.21.15.1:0/3555956849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 msgr2=0x564e267ded30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267ded30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7facc009cc00 tx=0x7facc0005c50 comp rx=0 tx=0).stop 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 -- 172.21.15.1:0/3555956849 shutdown_connections 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e267daa90 0x564e26857d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e267cbc80 0x564e267df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e267cb360 0x564e267ded30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 --2- 172.21.15.1:0/3555956849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e267c7a30 0x564e267de7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 -- 172.21.15.1:0/3555956849 >> 172.21.15.1:0/3555956849 conn(0x564e267d7d00 msgr2=0x564e267d2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facaaffd640 0 client.25043 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 -- 172.21.15.1:0/3555956849 shutdown_connections 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 1 -- 172.21.15.1:0/3555956849 wait complete. 2024-09-16T00:28:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.134+0000 7facd4403740 20 client.25043 trim_cache size 0 max 16384 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout: "200d5172-c5f3-43ed-9a59-c6430ce048ff": { 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "200d5172-c5f3-43ed-9a59-c6430ce048ff", 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:28:53.144 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:28:53.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 -- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7facb40a3240 msgr2=0x7facb40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7facb40a3240 0x7facb40a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7facc4014a40 tx=0x7facc4004730 comp rx=0 tx=0).stop 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 -- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faccc103660 msgr2=0x7faccc198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faccc103660 0x7faccc198a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7facc000e9f0 tx=0x7facc000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 -- 172.21.15.1:0/1121226344 shutdown_connections 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc10c170 0x7faccc19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faccc103660 0x7faccc198a30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7facb40a3240 0x7facb40a5700 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 --2- 172.21.15.1:0/1121226344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faccc102d40 0x7faccc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 -- 172.21.15.1:0/1121226344 >> 172.21.15.1:0/1121226344 conn(0x7faccc0fc7a0 msgr2=0x7faccc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 -- 172.21.15.1:0/1121226344 shutdown_connections 2024-09-16T00:28:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:53.135+0000 7facd202a640 1 -- 172.21.15.1:0/1121226344 wait complete. 2024-09-16T00:28:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:53.920 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:53 smithi001 ceph-mon[30934]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-16T00:28:53.950 INFO:tasks.cephfs.filesystem:scrub status for tag:200d5172-c5f3-43ed-9a59-c6430ce048ff - {'path': '/', 'tag': '200d5172-c5f3-43ed-9a59-c6430ce048ff', 'options': 'recursive,force'} 2024-09-16T00:28:53.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:53 smithi153 ceph-mon[35538]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-16T00:28:54.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:53 smithi019 ceph-mon[35186]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-16T00:28:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 -- 172.21.15.1:0/3247588771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea810c180 msgr2=0x7efea810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 --2- 172.21.15.1:0/3247588771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea810c180 0x7efea810e610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efea0009ca0 tx=0x7efea002a950 comp rx=0 tx=0).stop 2024-09-16T00:28:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 -- 172.21.15.1:0/3247588771 shutdown_connections 2024-09-16T00:28:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 --2- 172.21.15.1:0/3247588771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea810c180 0x7efea810e610 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 --2- 172.21.15.1:0/3247588771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efea8103700 0x7efea810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 --2- 172.21.15.1:0/3247588771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea8102de0 0x7efea81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 -- 172.21.15.1:0/3247588771 >> 172.21.15.1:0/3247588771 conn(0x7efea80fc7a0 msgr2=0x7efea80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 -- 172.21.15.1:0/3247588771 shutdown_connections 2024-09-16T00:28:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 -- 172.21.15.1:0/3247588771 wait complete. 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.434+0000 7efeadb6e640 1 Processor -- start 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efeadb6e640 1 -- start start 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efeadb6e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea8102de0 0x7efea81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efeadb6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efea8103700 0x7efea8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efeadb6e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea810c180 0x7efea819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efeadb6e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efea8111f70 con 0x7efea8102de0 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efeadb6e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efea8111df0 con 0x7efea8103700 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efeadb6e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efea81120f0 con 0x7efea810c180 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea8102de0 0x7efea81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea8102de0 0x7efea81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38574/0 (socket says 172.21.15.1:38574) 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea77fe640 1 -- 172.21.15.1:0/766154385 learned_addr learned my addr 172.21.15.1:0/766154385 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea6ffd640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efea8103700 0x7efea8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea7fff640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea810c180 0x7efea819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea77fe640 1 -- 172.21.15.1:0/766154385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea810c180 msgr2=0x7efea819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea77fe640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea810c180 0x7efea819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea77fe640 1 -- 172.21.15.1:0/766154385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efea8103700 msgr2=0x7efea8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea77fe640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efea8103700 0x7efea8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.435+0000 7efea77fe640 1 -- 172.21.15.1:0/766154385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea819d3a0 con 0x7efea8102de0 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.436+0000 7efea77fe640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea8102de0 0x7efea81983f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efe9800d950 tx=0x7efe9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.436+0000 7efea4ff9640 1 -- 172.21.15.1:0/766154385 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98017070 con 0x7efea8102de0 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.436+0000 7efea4ff9640 1 -- 172.21.15.1:0/766154385 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe98009c30 con 0x7efea8102de0 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.436+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea819d690 con 0x7efea8102de0 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.436+0000 7efea4ff9640 1 -- 172.21.15.1:0/766154385 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9800f6b0 con 0x7efea8102de0 2024-09-16T00:28:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.436+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea81a4f70 con 0x7efea8102de0 2024-09-16T00:28:54.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.437+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea81031c0 con 0x7efea8102de0 2024-09-16T00:28:54.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.439+0000 7efea4ff9640 1 -- 172.21.15.1:0/766154385 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7efe980109d0 con 0x7efea8102de0 2024-09-16T00:28:54.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.439+0000 7efea4ff9640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efe900a3240 0x7efe900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:54.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.440+0000 7efea6ffd640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efe900a3240 0x7efe900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:54.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.440+0000 7efea4ff9640 1 -- 172.21.15.1:0/766154385 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efe980cf8a0 con 0x7efea8102de0 2024-09-16T00:28:54.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.440+0000 7efea6ffd640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efe900a3240 0x7efe900a5700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efea80fed30 tx=0x7efe9c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:54.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.443+0000 7efea4ff9640 1 -- 172.21.15.1:0/766154385 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe98009de0 con 0x7efea8102de0 2024-09-16T00:28:54.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.616+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efea8062630 con 0x7efe900a3240 2024-09-16T00:28:54.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.624+0000 7efea4ff9640 1 -- 172.21.15.1:0/766154385 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340486 (secure 0 0 0) 0x7efea8062630 con 0x7efe900a3240 2024-09-16T00:28:54.628 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:54.630 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:28:54.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efe900a3240 msgr2=0x7efe900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:54.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efe900a3240 0x7efe900a5700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efea80fed30 tx=0x7efe9c00a2d0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea8102de0 msgr2=0x7efea81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:54.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea8102de0 0x7efea81983f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efe9800d950 tx=0x7efe9800de10 comp rx=0 tx=0).stop 2024-09-16T00:28:54.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 shutdown_connections 2024-09-16T00:28:54.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efe900a3240 0x7efe900a5700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea810c180 0x7efea819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efea8103700 0x7efea8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 --2- 172.21.15.1:0/766154385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea8102de0 0x7efea81983f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:54.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 >> 172.21.15.1:0/766154385 conn(0x7efea80fc7a0 msgr2=0x7efea80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:54.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 shutdown_connections 2024-09-16T00:28:54.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:54.629+0000 7efeadb6e640 1 -- 172.21.15.1:0/766154385 wait complete. 2024-09-16T00:28:55.380 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-16T00:28:54.348630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663060,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":815,"num_read_kb":533,"num_write":363,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380140,"kb_used_data":14252,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561684,"statfs":{"total":1151940427776,"available":1151551164416,"internally_reserved":0,"allocated":14594048,"data_stored":8145134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":38,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003073"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252865+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:27:26.252865+0000","last_peered":"2024-09-16T00:27:26.252865+0000","last_clean":"2024-09-16T00:27:26.252865+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:27:26.252865+0000","last_undegraded":"2024-09-16T00:27:26.252865+0000","last_fullsized":"2024-09-16T00:27:26.252865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322537+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:28:24.322537+0000","last_peered":"2024-09-16T00:28:24.322537+0000","last_clean":"2024-09-16T00:28:24.322537+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:28:24.322537+0000","last_undegraded":"2024-09-16T00:28:24.322537+0000","last_fullsized":"2024-09-16T00:28:24.322537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248176+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:27:26.248176+0000","last_peered":"2024-09-16T00:27:26.248176+0000","last_clean":"2024-09-16T00:27:26.248176+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:27:26.248176+0000","last_undegraded":"2024-09-16T00:27:26.248176+0000","last_fullsized":"2024-09-16T00:27:26.248176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247830+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:27:26.247830+0000","last_peered":"2024-09-16T00:27:26.247830+0000","last_clean":"2024-09-16T00:27:26.247830+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:27:26.247830+0000","last_undegraded":"2024-09-16T00:27:26.247830+0000","last_fullsized":"2024-09-16T00:27:26.247830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336305+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:28:24.336305+0000","last_peered":"2024-09-16T00:28:24.336305+0000","last_clean":"2024-09-16T00:28:24.336305+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:28:24.336305+0000","last_undegraded":"2024-09-16T00:28:24.336305+0000","last_fullsized":"2024-09-16T00:28:24.336305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355341+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:27:26.355341+0000","last_peered":"2024-09-16T00:27:26.355341+0000","last_clean":"2024-09-16T00:27:26.355341+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:27:26.355341+0000","last_undegraded":"2024-09-16T00:27:26.355341+0000","last_fullsized":"2024-09-16T00:27:26.355341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357885+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:26.357885+0000","last_peered":"2024-09-16T00:27:26.357885+0000","last_clean":"2024-09-16T00:27:26.357885+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:26.357885+0000","last_undegraded":"2024-09-16T00:27:26.357885+0000","last_fullsized":"2024-09-16T00:27:26.357885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337468+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:28:24.337468+0000","last_peered":"2024-09-16T00:28:24.337468+0000","last_clean":"2024-09-16T00:28:24.337468+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:28:24.337468+0000","last_undegraded":"2024-09-16T00:28:24.337468+0000","last_fullsized":"2024-09-16T00:28:24.337468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356501+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:27:26.356501+0000","last_peered":"2024-09-16T00:27:26.356501+0000","last_clean":"2024-09-16T00:27:26.356501+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:27:26.356501+0000","last_undegraded":"2024-09-16T00:27:26.356501+0000","last_fullsized":"2024-09-16T00:27:26.356501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252350+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:27:26.252350+0000","last_peered":"2024-09-16T00:27:26.252350+0000","last_clean":"2024-09-16T00:27:26.252350+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:27:26.252350+0000","last_undegraded":"2024-09-16T00:27:26.252350+0000","last_fullsized":"2024-09-16T00:27:26.252350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336557+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:28:24.336557+0000","last_peered":"2024-09-16T00:28:24.336557+0000","last_clean":"2024-09-16T00:28:24.336557+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:28:24.336557+0000","last_undegraded":"2024-09-16T00:28:24.336557+0000","last_fullsized":"2024-09-16T00:28:24.336557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355852+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:27:26.355852+0000","last_peered":"2024-09-16T00:27:26.355852+0000","last_clean":"2024-09-16T00:27:26.355852+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:27:26.355852+0000","last_undegraded":"2024-09-16T00:27:26.355852+0000","last_fullsized":"2024-09-16T00:27:26.355852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338941+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:28:24.338941+0000","last_peered":"2024-09-16T00:28:24.338941+0000","last_clean":"2024-09-16T00:28:24.338941+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:28:24.338941+0000","last_undegraded":"2024-09-16T00:28:24.338941+0000","last_fullsized":"2024-09-16T00:28:24.338941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357104+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:26.357104+0000","last_peered":"2024-09-16T00:27:26.357104+0000","last_clean":"2024-09-16T00:27:26.357104+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:26.357104+0000","last_undegraded":"2024-09-16T00:27:26.357104+0000","last_fullsized":"2024-09-16T00:27:26.357104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356176+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:27:26.356176+0000","last_peered":"2024-09-16T00:27:26.356176+0000","last_clean":"2024-09-16T00:27:26.356176+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:27:26.356176+0000","last_undegraded":"2024-09-16T00:27:26.356176+0000","last_fullsized":"2024-09-16T00:27:26.356176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327188+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:28:24.327188+0000","last_peered":"2024-09-16T00:28:24.327188+0000","last_clean":"2024-09-16T00:28:24.327188+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:28:24.327188+0000","last_undegraded":"2024-09-16T00:28:24.327188+0000","last_fullsized":"2024-09-16T00:28:24.327188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248306+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:27:26.248306+0000","last_peered":"2024-09-16T00:27:26.248306+0000","last_clean":"2024-09-16T00:27:26.248306+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:27:26.248306+0000","last_undegraded":"2024-09-16T00:27:26.248306+0000","last_fullsized":"2024-09-16T00:27:26.248306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356287+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:27:26.356287+0000","last_peered":"2024-09-16T00:27:26.356287+0000","last_clean":"2024-09-16T00:27:26.356287+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:27:26.356287+0000","last_undegraded":"2024-09-16T00:27:26.356287+0000","last_fullsized":"2024-09-16T00:27:26.356287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324477+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:28:24.324477+0000","last_peered":"2024-09-16T00:28:24.324477+0000","last_clean":"2024-09-16T00:28:24.324477+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:28:24.324477+0000","last_undegraded":"2024-09-16T00:28:24.324477+0000","last_fullsized":"2024-09-16T00:28:24.324477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247518+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:26.247518+0000","last_peered":"2024-09-16T00:27:26.247518+0000","last_clean":"2024-09-16T00:27:26.247518+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:26.247518+0000","last_undegraded":"2024-09-16T00:27:26.247518+0000","last_fullsized":"2024-09-16T00:27:26.247518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252497+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:27:26.252497+0000","last_peered":"2024-09-16T00:27:26.252497+0000","last_clean":"2024-09-16T00:27:26.252497+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:27:26.252497+0000","last_undegraded":"2024-09-16T00:27:26.252497+0000","last_fullsized":"2024-09-16T00:27:26.252497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.312122+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:28:24.312122+0000","last_peered":"2024-09-16T00:28:24.312122+0000","last_clean":"2024-09-16T00:28:24.312122+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:28:24.312122+0000","last_undegraded":"2024-09-16T00:28:24.312122+0000","last_fullsized":"2024-09-16T00:28:24.312122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355055+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:26.355055+0000","last_peered":"2024-09-16T00:27:26.355055+0000","last_clean":"2024-09-16T00:27:26.355055+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:26.355055+0000","last_undegraded":"2024-09-16T00:27:26.355055+0000","last_fullsized":"2024-09-16T00:27:26.355055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357331+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:27:26.357331+0000","last_peered":"2024-09-16T00:27:26.357331+0000","last_clean":"2024-09-16T00:27:26.357331+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:27:26.357331+0000","last_undegraded":"2024-09-16T00:27:26.357331+0000","last_fullsized":"2024-09-16T00:27:26.357331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334895+0000","last_change":"2024-09-16T00:27:26.358639+0000","last_active":"2024-09-16T00:28:24.334895+0000","last_peered":"2024-09-16T00:28:24.334895+0000","last_clean":"2024-09-16T00:28:24.334895+0000","last_became_active":"2024-09-16T00:27:26.358163+0000","last_became_peered":"2024-09-16T00:27:26.358163+0000","last_unstale":"2024-09-16T00:28:24.334895+0000","last_undegraded":"2024-09-16T00:28:24.334895+0000","last_fullsized":"2024-09-16T00:28:24.334895+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:11.811436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356716+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:26.356716+0000","last_peered":"2024-09-16T00:27:26.356716+0000","last_clean":"2024-09-16T00:27:26.356716+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:26.356716+0000","last_undegraded":"2024-09-16T00:27:26.356716+0000","last_fullsized":"2024-09-16T00:27:26.356716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248397+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:27:26.248397+0000","last_peered":"2024-09-16T00:27:26.248397+0000","last_clean":"2024-09-16T00:27:26.248397+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:27:26.248397+0000","last_undegraded":"2024-09-16T00:27:26.248397+0000","last_fullsized":"2024-09-16T00:27:26.248397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332074+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:28:24.332074+0000","last_peered":"2024-09-16T00:28:24.332074+0000","last_clean":"2024-09-16T00:28:24.332074+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:28:24.332074+0000","last_undegraded":"2024-09-16T00:28:24.332074+0000","last_fullsized":"2024-09-16T00:28:24.332074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:26.231276+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:28:26.231276+0000","last_peered":"2024-09-16T00:28:26.231276+0000","last_clean":"2024-09-16T00:28:26.231276+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:28:26.231276+0000","last_undegraded":"2024-09-16T00:28:26.231276+0000","last_fullsized":"2024-09-16T00:28:26.231276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247207+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:27:26.247207+0000","last_peered":"2024-09-16T00:27:26.247207+0000","last_clean":"2024-09-16T00:27:26.247207+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:27:26.247207+0000","last_undegraded":"2024-09-16T00:27:26.247207+0000","last_fullsized":"2024-09-16T00:27:26.247207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.325854+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:28:24.325854+0000","last_peered":"2024-09-16T00:28:24.325854+0000","last_clean":"2024-09-16T00:28:24.325854+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:28:24.325854+0000","last_undegraded":"2024-09-16T00:28:24.325854+0000","last_fullsized":"2024-09-16T00:28:24.325854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357960+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:26.357960+0000","last_peered":"2024-09-16T00:27:26.357960+0000","last_clean":"2024-09-16T00:27:26.357960+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:26.357960+0000","last_undegraded":"2024-09-16T00:27:26.357960+0000","last_fullsized":"2024-09-16T00:27:26.357960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251785+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:27:26.251785+0000","last_peered":"2024-09-16T00:27:26.251785+0000","last_clean":"2024-09-16T00:27:26.251785+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:27:26.251785+0000","last_undegraded":"2024-09-16T00:27:26.251785+0000","last_fullsized":"2024-09-16T00:27:26.251785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329144+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:28:24.329144+0000","last_peered":"2024-09-16T00:28:24.329144+0000","last_clean":"2024-09-16T00:28:24.329144+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:28:24.329144+0000","last_undegraded":"2024-09-16T00:28:24.329144+0000","last_fullsized":"2024-09-16T00:28:24.329144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252455+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:27:26.252455+0000","last_peered":"2024-09-16T00:27:26.252455+0000","last_clean":"2024-09-16T00:27:26.252455+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:27:26.252455+0000","last_undegraded":"2024-09-16T00:27:26.252455+0000","last_fullsized":"2024-09-16T00:27:26.252455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357048+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:27:26.357048+0000","last_peered":"2024-09-16T00:27:26.357048+0000","last_clean":"2024-09-16T00:27:26.357048+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:27:26.357048+0000","last_undegraded":"2024-09-16T00:27:26.357048+0000","last_fullsized":"2024-09-16T00:27:26.357048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321720+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:28:24.321720+0000","last_peered":"2024-09-16T00:28:24.321720+0000","last_clean":"2024-09-16T00:28:24.321720+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:28:24.321720+0000","last_undegraded":"2024-09-16T00:28:24.321720+0000","last_fullsized":"2024-09-16T00:28:24.321720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357297+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:27:26.357297+0000","last_peered":"2024-09-16T00:27:26.357297+0000","last_clean":"2024-09-16T00:27:26.357297+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:27:26.357297+0000","last_undegraded":"2024-09-16T00:27:26.357297+0000","last_fullsized":"2024-09-16T00:27:26.357297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250069+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:27:26.250069+0000","last_peered":"2024-09-16T00:27:26.250069+0000","last_clean":"2024-09-16T00:27:26.250069+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:27:26.250069+0000","last_undegraded":"2024-09-16T00:27:26.250069+0000","last_fullsized":"2024-09-16T00:27:26.250069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.305108+0000","last_change":"2024-09-16T00:27:26.357501+0000","last_active":"2024-09-16T00:28:24.305108+0000","last_peered":"2024-09-16T00:28:24.305108+0000","last_clean":"2024-09-16T00:28:24.305108+0000","last_became_active":"2024-09-16T00:27:26.357108+0000","last_became_peered":"2024-09-16T00:27:26.357108+0000","last_unstale":"2024-09-16T00:28:24.305108+0000","last_undegraded":"2024-09-16T00:28:24.305108+0000","last_fullsized":"2024-09-16T00:28:24.305108+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:25.264059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:49.675018+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:28:49.675018+0000","last_peered":"2024-09-16T00:28:49.675018+0000","last_clean":"2024-09-16T00:28:49.675018+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:28:49.675018+0000","last_undegraded":"2024-09-16T00:28:49.675018+0000","last_fullsized":"2024-09-16T00:28:49.675018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250857+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:27:26.250857+0000","last_peered":"2024-09-16T00:27:26.250857+0000","last_clean":"2024-09-16T00:27:26.250857+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:27:26.250857+0000","last_undegraded":"2024-09-16T00:27:26.250857+0000","last_fullsized":"2024-09-16T00:27:26.250857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326885+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:28:24.326885+0000","last_peered":"2024-09-16T00:28:24.326885+0000","last_clean":"2024-09-16T00:28:24.326885+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:28:24.326885+0000","last_undegraded":"2024-09-16T00:28:24.326885+0000","last_fullsized":"2024-09-16T00:28:24.326885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357096+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:26.357096+0000","last_peered":"2024-09-16T00:27:26.357096+0000","last_clean":"2024-09-16T00:27:26.357096+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:26.357096+0000","last_undegraded":"2024-09-16T00:27:26.357096+0000","last_fullsized":"2024-09-16T00:27:26.357096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356120+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:27:26.356120+0000","last_peered":"2024-09-16T00:27:26.356120+0000","last_clean":"2024-09-16T00:27:26.356120+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:27:26.356120+0000","last_undegraded":"2024-09-16T00:27:26.356120+0000","last_fullsized":"2024-09-16T00:27:26.356120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336576+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:28:24.336576+0000","last_peered":"2024-09-16T00:28:24.336576+0000","last_clean":"2024-09-16T00:28:24.336576+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:28:24.336576+0000","last_undegraded":"2024-09-16T00:28:24.336576+0000","last_fullsized":"2024-09-16T00:28:24.336576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357054+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:26.357054+0000","last_peered":"2024-09-16T00:27:26.357054+0000","last_clean":"2024-09-16T00:27:26.357054+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:26.357054+0000","last_undegraded":"2024-09-16T00:27:26.357054+0000","last_fullsized":"2024-09-16T00:27:26.357054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357365+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:27:26.357365+0000","last_peered":"2024-09-16T00:27:26.357365+0000","last_clean":"2024-09-16T00:27:26.357365+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:27:26.357365+0000","last_undegraded":"2024-09-16T00:27:26.357365+0000","last_fullsized":"2024-09-16T00:27:26.357365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355730+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:27:26.355730+0000","last_peered":"2024-09-16T00:27:26.355730+0000","last_clean":"2024-09-16T00:27:26.355730+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:27:26.355730+0000","last_undegraded":"2024-09-16T00:27:26.355730+0000","last_fullsized":"2024-09-16T00:27:26.355730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338444+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:28:24.338444+0000","last_peered":"2024-09-16T00:28:24.338444+0000","last_clean":"2024-09-16T00:28:24.338444+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:28:24.338444+0000","last_undegraded":"2024-09-16T00:28:24.338444+0000","last_fullsized":"2024-09-16T00:28:24.338444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248838+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:26.248838+0000","last_peered":"2024-09-16T00:27:26.248838+0000","last_clean":"2024-09-16T00:27:26.248838+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:26.248838+0000","last_undegraded":"2024-09-16T00:27:26.248838+0000","last_fullsized":"2024-09-16T00:27:26.248838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355174+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:27:26.355174+0000","last_peered":"2024-09-16T00:27:26.355174+0000","last_clean":"2024-09-16T00:27:26.355174+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:27:26.355174+0000","last_undegraded":"2024-09-16T00:27:26.355174+0000","last_fullsized":"2024-09-16T00:27:26.355174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331484+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:28:24.331484+0000","last_peered":"2024-09-16T00:28:24.331484+0000","last_clean":"2024-09-16T00:28:24.331484+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:28:24.331484+0000","last_undegraded":"2024-09-16T00:28:24.331484+0000","last_fullsized":"2024-09-16T00:28:24.331484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356422+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:27:26.356422+0000","last_peered":"2024-09-16T00:27:26.356422+0000","last_clean":"2024-09-16T00:27:26.356422+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:27:26.356422+0000","last_undegraded":"2024-09-16T00:27:26.356422+0000","last_fullsized":"2024-09-16T00:27:26.356422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249539+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:27:26.249539+0000","last_peered":"2024-09-16T00:27:26.249539+0000","last_clean":"2024-09-16T00:27:26.249539+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:27:26.249539+0000","last_undegraded":"2024-09-16T00:27:26.249539+0000","last_fullsized":"2024-09-16T00:27:26.249539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.333565+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:28:24.333565+0000","last_peered":"2024-09-16T00:28:24.333565+0000","last_clean":"2024-09-16T00:28:24.333565+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:28:24.333565+0000","last_undegraded":"2024-09-16T00:28:24.333565+0000","last_fullsized":"2024-09-16T00:28:24.333565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250975+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:27:26.250975+0000","last_peered":"2024-09-16T00:27:26.250975+0000","last_clean":"2024-09-16T00:27:26.250975+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:27:26.250975+0000","last_undegraded":"2024-09-16T00:27:26.250975+0000","last_fullsized":"2024-09-16T00:27:26.250975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354985+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:27:26.354985+0000","last_peered":"2024-09-16T00:27:26.354985+0000","last_clean":"2024-09-16T00:27:26.354985+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:27:26.354985+0000","last_undegraded":"2024-09-16T00:27:26.354985+0000","last_fullsized":"2024-09-16T00:27:26.354985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.319032+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:28:24.319032+0000","last_peered":"2024-09-16T00:28:24.319032+0000","last_clean":"2024-09-16T00:28:24.319032+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:28:24.319032+0000","last_undegraded":"2024-09-16T00:28:24.319032+0000","last_fullsized":"2024-09-16T00:28:24.319032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357916+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:26.357916+0000","last_peered":"2024-09-16T00:27:26.357916+0000","last_clean":"2024-09-16T00:27:26.357916+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:26.357916+0000","last_undegraded":"2024-09-16T00:27:26.357916+0000","last_fullsized":"2024-09-16T00:27:26.357916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355183+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:27:26.355183+0000","last_peered":"2024-09-16T00:27:26.355183+0000","last_clean":"2024-09-16T00:27:26.355183+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:27:26.355183+0000","last_undegraded":"2024-09-16T00:27:26.355183+0000","last_fullsized":"2024-09-16T00:27:26.355183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336143+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:28:24.336143+0000","last_peered":"2024-09-16T00:28:24.336143+0000","last_clean":"2024-09-16T00:28:24.336143+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:28:24.336143+0000","last_undegraded":"2024-09-16T00:28:24.336143+0000","last_fullsized":"2024-09-16T00:28:24.336143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356998+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:26.356998+0000","last_peered":"2024-09-16T00:27:26.356998+0000","last_clean":"2024-09-16T00:27:26.356998+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:26.356998+0000","last_undegraded":"2024-09-16T00:27:26.356998+0000","last_fullsized":"2024-09-16T00:27:26.356998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.254250+0000","last_change":"2024-09-16T00:27:26.253824+0000","last_active":"2024-09-16T00:27:26.254250+0000","last_peered":"2024-09-16T00:27:26.254250+0000","last_clean":"2024-09-16T00:27:26.254250+0000","last_became_active":"2024-09-16T00:27:26.253206+0000","last_became_peered":"2024-09-16T00:27:26.253206+0000","last_unstale":"2024-09-16T00:27:26.254250+0000","last_undegraded":"2024-09-16T00:27:26.254250+0000","last_fullsized":"2024-09-16T00:27:26.254250+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:22.543369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332839+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:28:24.332839+0000","last_peered":"2024-09-16T00:28:24.332839+0000","last_clean":"2024-09-16T00:28:24.332839+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:28:24.332839+0000","last_undegraded":"2024-09-16T00:28:24.332839+0000","last_fullsized":"2024-09-16T00:28:24.332839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:48.674171+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:28:48.674171+0000","last_peered":"2024-09-16T00:28:48.674171+0000","last_clean":"2024-09-16T00:28:48.674171+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:28:48.674171+0000","last_undegraded":"2024-09-16T00:28:48.674171+0000","last_fullsized":"2024-09-16T00:28:48.674171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252606+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:27:26.252606+0000","last_peered":"2024-09-16T00:27:26.252606+0000","last_clean":"2024-09-16T00:27:26.252606+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:27:26.252606+0000","last_undegraded":"2024-09-16T00:27:26.252606+0000","last_fullsized":"2024-09-16T00:27:26.252606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.335224+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:28:24.335224+0000","last_peered":"2024-09-16T00:28:24.335224+0000","last_clean":"2024-09-16T00:28:24.335224+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:28:24.335224+0000","last_undegraded":"2024-09-16T00:28:24.335224+0000","last_fullsized":"2024-09-16T00:28:24.335224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356320+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:26.356320+0000","last_peered":"2024-09-16T00:27:26.356320+0000","last_clean":"2024-09-16T00:27:26.356320+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:26.356320+0000","last_undegraded":"2024-09-16T00:27:26.356320+0000","last_fullsized":"2024-09-16T00:27:26.356320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252594+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:27:26.252594+0000","last_peered":"2024-09-16T00:27:26.252594+0000","last_clean":"2024-09-16T00:27:26.252594+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:27:26.252594+0000","last_undegraded":"2024-09-16T00:27:26.252594+0000","last_fullsized":"2024-09-16T00:27:26.252594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330126+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:28:24.330126+0000","last_peered":"2024-09-16T00:28:24.330126+0000","last_clean":"2024-09-16T00:28:24.330126+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:28:24.330126+0000","last_undegraded":"2024-09-16T00:28:24.330126+0000","last_fullsized":"2024-09-16T00:28:24.330126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248474+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:26.248474+0000","last_peered":"2024-09-16T00:27:26.248474+0000","last_clean":"2024-09-16T00:27:26.248474+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:26.248474+0000","last_undegraded":"2024-09-16T00:27:26.248474+0000","last_fullsized":"2024-09-16T00:27:26.248474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355990+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:27:26.355990+0000","last_peered":"2024-09-16T00:27:26.355990+0000","last_clean":"2024-09-16T00:27:26.355990+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:27:26.355990+0000","last_undegraded":"2024-09-16T00:27:26.355990+0000","last_fullsized":"2024-09-16T00:27:26.355990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.311816+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:28:24.311816+0000","last_peered":"2024-09-16T00:28:24.311816+0000","last_clean":"2024-09-16T00:28:24.311816+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:28:24.311816+0000","last_undegraded":"2024-09-16T00:28:24.311816+0000","last_fullsized":"2024-09-16T00:28:24.311816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":324,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:51.682287+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:28:51.682287+0000","last_peered":"2024-09-16T00:28:51.682287+0000","last_clean":"2024-09-16T00:28:51.682287+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:28:51.682287+0000","last_undegraded":"2024-09-16T00:28:51.682287+0000","last_fullsized":"2024-09-16T00:28:51.682287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68006,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":233,"num_read_kb":184,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359395+0000","last_change":"2024-09-16T00:27:26.358804+0000","last_active":"2024-09-16T00:27:26.359395+0000","last_peered":"2024-09-16T00:27:26.359395+0000","last_clean":"2024-09-16T00:27:26.359395+0000","last_became_active":"2024-09-16T00:27:26.358317+0000","last_became_peered":"2024-09-16T00:27:26.358317+0000","last_unstale":"2024-09-16T00:27:26.359395+0000","last_undegraded":"2024-09-16T00:27:26.359395+0000","last_fullsized":"2024-09-16T00:27:26.359395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:38.642392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336469+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:28:24.336469+0000","last_peered":"2024-09-16T00:28:24.336469+0000","last_clean":"2024-09-16T00:28:24.336469+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:28:24.336469+0000","last_undegraded":"2024-09-16T00:28:24.336469+0000","last_fullsized":"2024-09-16T00:28:24.336469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356307+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:26.356307+0000","last_peered":"2024-09-16T00:27:26.356307+0000","last_clean":"2024-09-16T00:27:26.356307+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:26.356307+0000","last_undegraded":"2024-09-16T00:27:26.356307+0000","last_fullsized":"2024-09-16T00:27:26.356307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356802+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:27:26.356802+0000","last_peered":"2024-09-16T00:27:26.356802+0000","last_clean":"2024-09-16T00:27:26.356802+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:27:26.356802+0000","last_undegraded":"2024-09-16T00:27:26.356802+0000","last_fullsized":"2024-09-16T00:27:26.356802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320577+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:28:24.320577+0000","last_peered":"2024-09-16T00:28:24.320577+0000","last_clean":"2024-09-16T00:28:24.320577+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:28:24.320577+0000","last_undegraded":"2024-09-16T00:28:24.320577+0000","last_fullsized":"2024-09-16T00:28:24.320577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357240+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:26.357240+0000","last_peered":"2024-09-16T00:27:26.357240+0000","last_clean":"2024-09-16T00:27:26.357240+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:26.357240+0000","last_undegraded":"2024-09-16T00:27:26.357240+0000","last_fullsized":"2024-09-16T00:27:26.357240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252178+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:27:26.252178+0000","last_peered":"2024-09-16T00:27:26.252178+0000","last_clean":"2024-09-16T00:27:26.252178+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:27:26.252178+0000","last_undegraded":"2024-09-16T00:27:26.252178+0000","last_fullsized":"2024-09-16T00:27:26.252178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336948+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:28:24.336948+0000","last_peered":"2024-09-16T00:28:24.336948+0000","last_clean":"2024-09-16T00:28:24.336948+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:28:24.336948+0000","last_undegraded":"2024-09-16T00:28:24.336948+0000","last_fullsized":"2024-09-16T00:28:24.336948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:50.680734+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:28:50.680734+0000","last_peered":"2024-09-16T00:28:50.680734+0000","last_clean":"2024-09-16T00:28:50.680734+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:28:50.680734+0000","last_undegraded":"2024-09-16T00:28:50.680734+0000","last_fullsized":"2024-09-16T00:28:50.680734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247326+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:27:26.247326+0000","last_peered":"2024-09-16T00:27:26.247326+0000","last_clean":"2024-09-16T00:27:26.247326+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:27:26.247326+0000","last_undegraded":"2024-09-16T00:27:26.247326+0000","last_fullsized":"2024-09-16T00:27:26.247326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250057+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:27:26.250057+0000","last_peered":"2024-09-16T00:27:26.250057+0000","last_clean":"2024-09-16T00:27:26.250057+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:27:26.250057+0000","last_undegraded":"2024-09-16T00:27:26.250057+0000","last_fullsized":"2024-09-16T00:27:26.250057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331405+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:28:24.331405+0000","last_peered":"2024-09-16T00:28:24.331405+0000","last_clean":"2024-09-16T00:28:24.331405+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:28:24.331405+0000","last_undegraded":"2024-09-16T00:28:24.331405+0000","last_fullsized":"2024-09-16T00:28:24.331405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355413+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:27:26.355413+0000","last_peered":"2024-09-16T00:27:26.355413+0000","last_clean":"2024-09-16T00:27:26.355413+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:27:26.355413+0000","last_undegraded":"2024-09-16T00:27:26.355413+0000","last_fullsized":"2024-09-16T00:27:26.355413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.360047+0000","last_change":"2024-09-16T00:27:26.359230+0000","last_active":"2024-09-16T00:27:26.360047+0000","last_peered":"2024-09-16T00:27:26.360047+0000","last_clean":"2024-09-16T00:27:26.360047+0000","last_became_active":"2024-09-16T00:27:26.358662+0000","last_became_peered":"2024-09-16T00:27:26.358662+0000","last_unstale":"2024-09-16T00:27:26.360047+0000","last_undegraded":"2024-09-16T00:27:26.360047+0000","last_fullsized":"2024-09-16T00:27:26.360047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:52.111910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331664+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:28:24.331664+0000","last_peered":"2024-09-16T00:28:24.331664+0000","last_clean":"2024-09-16T00:28:24.331664+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:28:24.331664+0000","last_undegraded":"2024-09-16T00:28:24.331664+0000","last_fullsized":"2024-09-16T00:28:24.331664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250512+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:26.250512+0000","last_peered":"2024-09-16T00:27:26.250512+0000","last_clean":"2024-09-16T00:27:26.250512+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:26.250512+0000","last_undegraded":"2024-09-16T00:27:26.250512+0000","last_fullsized":"2024-09-16T00:27:26.250512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250964+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:26.250964+0000","last_peered":"2024-09-16T00:27:26.250964+0000","last_clean":"2024-09-16T00:27:26.250964+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:26.250964+0000","last_undegraded":"2024-09-16T00:27:26.250964+0000","last_fullsized":"2024-09-16T00:27:26.250964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334137+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:28:24.334137+0000","last_peered":"2024-09-16T00:28:24.334137+0000","last_clean":"2024-09-16T00:28:24.334137+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:28:24.334137+0000","last_undegraded":"2024-09-16T00:28:24.334137+0000","last_fullsized":"2024-09-16T00:28:24.334137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359360+0000","last_change":"2024-09-16T00:27:26.359360+0000","last_active":"2024-09-16T00:27:26.359360+0000","last_peered":"2024-09-16T00:27:26.359360+0000","last_clean":"2024-09-16T00:27:26.359360+0000","last_became_active":"2024-09-16T00:27:26.358462+0000","last_became_peered":"2024-09-16T00:27:26.358462+0000","last_unstale":"2024-09-16T00:27:26.359360+0000","last_undegraded":"2024-09-16T00:27:26.359360+0000","last_fullsized":"2024-09-16T00:27:26.359360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:38.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:49.680040+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:28:49.680040+0000","last_peered":"2024-09-16T00:28:49.680040+0000","last_clean":"2024-09-16T00:28:49.680040+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:28:49.680040+0000","last_undegraded":"2024-09-16T00:28:49.680040+0000","last_fullsized":"2024-09-16T00:28:49.680040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320920+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:28:24.320920+0000","last_peered":"2024-09-16T00:28:24.320920+0000","last_clean":"2024-09-16T00:28:24.320920+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:28:24.320920+0000","last_undegraded":"2024-09-16T00:28:24.320920+0000","last_fullsized":"2024-09-16T00:28:24.320920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248757+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:27:26.248757+0000","last_peered":"2024-09-16T00:27:26.248757+0000","last_clean":"2024-09-16T00:27:26.248757+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:27:26.248757+0000","last_undegraded":"2024-09-16T00:27:26.248757+0000","last_fullsized":"2024-09-16T00:27:26.248757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321717+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:28:24.321717+0000","last_peered":"2024-09-16T00:28:24.321717+0000","last_clean":"2024-09-16T00:28:24.321717+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:28:24.321717+0000","last_undegraded":"2024-09-16T00:28:24.321717+0000","last_fullsized":"2024-09-16T00:28:24.321717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321331+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:28:24.321331+0000","last_peered":"2024-09-16T00:28:24.321331+0000","last_clean":"2024-09-16T00:28:24.321331+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:28:24.321331+0000","last_undegraded":"2024-09-16T00:28:24.321331+0000","last_fullsized":"2024-09-16T00:28:24.321331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329213+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:28:24.329213+0000","last_peered":"2024-09-16T00:28:24.329213+0000","last_clean":"2024-09-16T00:28:24.329213+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:28:24.329213+0000","last_undegraded":"2024-09-16T00:28:24.329213+0000","last_fullsized":"2024-09-16T00:28:24.329213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326265+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:28:24.326265+0000","last_peered":"2024-09-16T00:28:24.326265+0000","last_clean":"2024-09-16T00:28:24.326265+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:28:24.326265+0000","last_undegraded":"2024-09-16T00:28:24.326265+0000","last_fullsized":"2024-09-16T00:28:24.326265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327220+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:28:24.327220+0000","last_peered":"2024-09-16T00:28:24.327220+0000","last_clean":"2024-09-16T00:28:24.327220+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:28:24.327220+0000","last_undegraded":"2024-09-16T00:28:24.327220+0000","last_fullsized":"2024-09-16T00:28:24.327220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.316594+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:28:24.316594+0000","last_peered":"2024-09-16T00:28:24.316594+0000","last_clean":"2024-09-16T00:28:24.316594+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:28:24.316594+0000","last_undegraded":"2024-09-16T00:28:24.316594+0000","last_fullsized":"2024-09-16T00:28:24.316594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337588+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:28:24.337588+0000","last_peered":"2024-09-16T00:28:24.337588+0000","last_clean":"2024-09-16T00:28:24.337588+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:28:24.337588+0000","last_undegraded":"2024-09-16T00:28:24.337588+0000","last_fullsized":"2024-09-16T00:28:24.337588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322996+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:28:24.322996+0000","last_peered":"2024-09-16T00:28:24.322996+0000","last_clean":"2024-09-16T00:28:24.322996+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:28:24.322996+0000","last_undegraded":"2024-09-16T00:28:24.322996+0000","last_fullsized":"2024-09-16T00:28:24.322996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322343+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:28:24.322343+0000","last_peered":"2024-09-16T00:28:24.322343+0000","last_clean":"2024-09-16T00:28:24.322343+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:28:24.322343+0000","last_undegraded":"2024-09-16T00:28:24.322343+0000","last_fullsized":"2024-09-16T00:28:24.322343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330356+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:28:24.330356+0000","last_peered":"2024-09-16T00:28:24.330356+0000","last_clean":"2024-09-16T00:28:24.330356+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:28:24.330356+0000","last_undegraded":"2024-09-16T00:28:24.330356+0000","last_fullsized":"2024-09-16T00:28:24.330356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337238+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:28:24.337238+0000","last_peered":"2024-09-16T00:28:24.337238+0000","last_clean":"2024-09-16T00:28:24.337238+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:28:24.337238+0000","last_undegraded":"2024-09-16T00:28:24.337238+0000","last_fullsized":"2024-09-16T00:28:24.337238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330609+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:28:24.330609+0000","last_peered":"2024-09-16T00:28:24.330609+0000","last_clean":"2024-09-16T00:28:24.330609+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:28:24.330609+0000","last_undegraded":"2024-09-16T00:28:24.330609+0000","last_fullsized":"2024-09-16T00:28:24.330609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.317884+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:28:24.317884+0000","last_peered":"2024-09-16T00:28:24.317884+0000","last_clean":"2024-09-16T00:28:24.317884+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:28:24.317884+0000","last_undegraded":"2024-09-16T00:28:24.317884+0000","last_fullsized":"2024-09-16T00:28:24.317884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329300+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:28:24.329300+0000","last_peered":"2024-09-16T00:28:24.329300+0000","last_clean":"2024-09-16T00:28:24.329300+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:28:24.329300+0000","last_undegraded":"2024-09-16T00:28:24.329300+0000","last_fullsized":"2024-09-16T00:28:24.329300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337159+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:28:24.337159+0000","last_peered":"2024-09-16T00:28:24.337159+0000","last_clean":"2024-09-16T00:28:24.337159+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:28:24.337159+0000","last_undegraded":"2024-09-16T00:28:24.337159+0000","last_fullsized":"2024-09-16T00:28:24.337159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336933+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:28:24.336933+0000","last_peered":"2024-09-16T00:28:24.336933+0000","last_clean":"2024-09-16T00:28:24.336933+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:28:24.336933+0000","last_undegraded":"2024-09-16T00:28:24.336933+0000","last_fullsized":"2024-09-16T00:28:24.336933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337483+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:28:24.337483+0000","last_peered":"2024-09-16T00:28:24.337483+0000","last_clean":"2024-09-16T00:28:24.337483+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:28:24.337483+0000","last_undegraded":"2024-09-16T00:28:24.337483+0000","last_fullsized":"2024-09-16T00:28:24.337483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326582+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:28:24.326582+0000","last_peered":"2024-09-16T00:28:24.326582+0000","last_clean":"2024-09-16T00:28:24.326582+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:28:24.326582+0000","last_undegraded":"2024-09-16T00:28:24.326582+0000","last_fullsized":"2024-09-16T00:28:24.326582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326496+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:28:24.326496+0000","last_peered":"2024-09-16T00:28:24.326496+0000","last_clean":"2024-09-16T00:28:24.326496+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:28:24.326496+0000","last_undegraded":"2024-09-16T00:28:24.326496+0000","last_fullsized":"2024-09-16T00:28:24.326496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326067+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:28:24.326067+0000","last_peered":"2024-09-16T00:28:24.326067+0000","last_clean":"2024-09-16T00:28:24.326067+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:28:24.326067+0000","last_undegraded":"2024-09-16T00:28:24.326067+0000","last_fullsized":"2024-09-16T00:28:24.326067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320083+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:28:24.320083+0000","last_peered":"2024-09-16T00:28:24.320083+0000","last_clean":"2024-09-16T00:28:24.320083+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:28:24.320083+0000","last_undegraded":"2024-09-16T00:28:24.320083+0000","last_fullsized":"2024-09-16T00:28:24.320083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338288+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:28:24.338288+0000","last_peered":"2024-09-16T00:28:24.338288+0000","last_clean":"2024-09-16T00:28:24.338288+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:28:24.338288+0000","last_undegraded":"2024-09-16T00:28:24.338288+0000","last_fullsized":"2024-09-16T00:28:24.338288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.328897+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:28:24.328897+0000","last_peered":"2024-09-16T00:28:24.328897+0000","last_clean":"2024-09-16T00:28:24.328897+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:28:24.328897+0000","last_undegraded":"2024-09-16T00:28:24.328897+0000","last_fullsized":"2024-09-16T00:28:24.328897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334467+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:28:24.334467+0000","last_peered":"2024-09-16T00:28:24.334467+0000","last_clean":"2024-09-16T00:28:24.334467+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:28:24.334467+0000","last_undegraded":"2024-09-16T00:28:24.334467+0000","last_fullsized":"2024-09-16T00:28:24.334467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324496+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:28:24.324496+0000","last_peered":"2024-09-16T00:28:24.324496+0000","last_clean":"2024-09-16T00:28:24.324496+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:28:24.324496+0000","last_undegraded":"2024-09-16T00:28:24.324496+0000","last_fullsized":"2024-09-16T00:28:24.324496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334306+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:28:24.334306+0000","last_peered":"2024-09-16T00:28:24.334306+0000","last_clean":"2024-09-16T00:28:24.334306+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:28:24.334306+0000","last_undegraded":"2024-09-16T00:28:24.334306+0000","last_fullsized":"2024-09-16T00:28:24.334306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.328201+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:28:24.328201+0000","last_peered":"2024-09-16T00:28:24.328201+0000","last_clean":"2024-09-16T00:28:24.328201+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:28:24.328201+0000","last_undegraded":"2024-09-16T00:28:24.328201+0000","last_fullsized":"2024-09-16T00:28:24.328201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322132+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:28:24.322132+0000","last_peered":"2024-09-16T00:28:24.322132+0000","last_clean":"2024-09-16T00:28:24.322132+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:28:24.322132+0000","last_undegraded":"2024-09-16T00:28:24.322132+0000","last_fullsized":"2024-09-16T00:28:24.322132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320484+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:28:24.320484+0000","last_peered":"2024-09-16T00:28:24.320484+0000","last_clean":"2024-09-16T00:28:24.320484+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:28:24.320484+0000","last_undegraded":"2024-09-16T00:28:24.320484+0000","last_fullsized":"2024-09-16T00:28:24.320484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.318969+0000","last_change":"2024-09-16T00:27:26.358604+0000","last_active":"2024-09-16T00:28:24.318969+0000","last_peered":"2024-09-16T00:28:24.318969+0000","last_clean":"2024-09-16T00:28:24.318969+0000","last_became_active":"2024-09-16T00:27:26.358225+0000","last_became_peered":"2024-09-16T00:27:26.358225+0000","last_unstale":"2024-09-16T00:28:24.318969+0000","last_undegraded":"2024-09-16T00:28:24.318969+0000","last_fullsized":"2024-09-16T00:28:24.318969+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:21.806500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326711+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:28:24.326711+0000","last_peered":"2024-09-16T00:28:24.326711+0000","last_clean":"2024-09-16T00:28:24.326711+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:28:24.326711+0000","last_undegraded":"2024-09-16T00:28:24.326711+0000","last_fullsized":"2024-09-16T00:28:24.326711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336940+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:28:24.336940+0000","last_peered":"2024-09-16T00:28:24.336940+0000","last_clean":"2024-09-16T00:28:24.336940+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:28:24.336940+0000","last_undegraded":"2024-09-16T00:28:24.336940+0000","last_fullsized":"2024-09-16T00:28:24.336940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72458,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":689,"num_read_kb":424,"num_write":180,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":217374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514541,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":515002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743474,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1732608,"data_stored":1170276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906999,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":519334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":515158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364864,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1646592,"data_stored":1111518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528390,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1107106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":583070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":510608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":512516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":578330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":511802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:28:55.385 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:28:55.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:28:55.647 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:55.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:55 smithi153 ceph-mon[35538]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-16T00:28:55.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:55 smithi153 ceph-mon[35538]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:56.083 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:55 smithi001 ceph-mon[30934]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-16T00:28:56.083 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:55 smithi001 ceph-mon[30934]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:56.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:55 smithi019 ceph-mon[35186]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-16T00:28:56.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:55 smithi019 ceph-mon[35186]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:56.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 -- 172.21.15.1:0/1871734269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 msgr2=0x7f90ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:56.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 --2- 172.21.15.1:0/1871734269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 0x7f90ac1067e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009900 tx=0x7f90a002a410 comp rx=0 tx=0).stop 2024-09-16T00:28:56.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 -- 172.21.15.1:0/1871734269 shutdown_connections 2024-09-16T00:28:56.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 --2- 172.21.15.1:0/1871734269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90ac106f10 0x7f90ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:56.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 --2- 172.21.15.1:0/1871734269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 0x7f90ac1067e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 --2- 172.21.15.1:0/1871734269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ac101de0 0x7f90ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 -- 172.21.15.1:0/1871734269 >> 172.21.15.1:0/1871734269 conn(0x7f90ac0fd8c0 msgr2=0x7f90ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 -- 172.21.15.1:0/1871734269 shutdown_connections 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 -- 172.21.15.1:0/1871734269 wait complete. 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 Processor -- start 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.166+0000 7f90b283f640 1 -- start start 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90b283f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ac101de0 0x7f90ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90b283f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 0x7f90ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90b283f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90ac106f10 0x7f90ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90b283f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac111f80 con 0x7f90ac102790 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90b283f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac111e00 con 0x7f90ac106f10 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90b283f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac112100 con 0x7f90ac101de0 2024-09-16T00:28:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 0x7f90ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 0x7f90ac074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38598/0 (socket says 172.21.15.1:38598) 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90ab7fe640 1 -- 172.21.15.1:0/3892862761 learned_addr learned my addr 172.21.15.1:0/3892862761 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90b0ddc640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90ac106f10 0x7f90ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90abfff640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ac101de0 0x7f90ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90ab7fe640 1 -- 172.21.15.1:0/3892862761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ac101de0 msgr2=0x7f90ac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90ab7fe640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ac101de0 0x7f90ac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90ab7fe640 1 -- 172.21.15.1:0/3892862761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90ac106f10 msgr2=0x7f90ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90ab7fe640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90ac106f10 0x7f90ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90ab7fe640 1 -- 172.21.15.1:0/3892862761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac1a68e0 con 0x7f90ac102790 2024-09-16T00:28:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90abfff640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ac101de0 0x7f90ac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.167+0000 7f90ab7fe640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 0x7f90ac074a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004770 tx=0x7f90a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.168+0000 7f90a97fa640 1 -- 172.21.15.1:0/3892862761 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a003f070 con 0x7f90ac102790 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.168+0000 7f90a97fa640 1 -- 172.21.15.1:0/3892862761 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a0031ca0 con 0x7f90ac102790 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.168+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac1a6b70 con 0x7f90ac102790 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.168+0000 7f90a97fa640 1 -- 172.21.15.1:0/3892862761 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0035030 con 0x7f90ac102790 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.168+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac1a70a0 con 0x7f90ac102790 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.169+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ac1021c0 con 0x7f90ac102790 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.170+0000 7f90a97fa640 1 -- 172.21.15.1:0/3892862761 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f90a0042070 con 0x7f90ac102790 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.171+0000 7f90a97fa640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f90940a32c0 0x7f90940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.171+0000 7f90abfff640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f90940a32c0 0x7f90940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.171+0000 7f90a97fa640 1 -- 172.21.15.1:0/3892862761 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f90a00f03c0 con 0x7f90ac102790 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.171+0000 7f90abfff640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f90940a32c0 0x7f90940a5780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f909c0058c0 tx=0x7f909c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:56.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.175+0000 7f90a97fa640 1 -- 172.21.15.1:0/3892862761 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90a00bcce0 con 0x7f90ac102790 2024-09-16T00:28:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.351+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 --> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90ac10a160 con 0x7f90940a32c0 2024-09-16T00:28:56.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.359+0000 7f90a97fa640 1 -- 172.21.15.1:0/3892862761 <== mgr.14150 v2:172.21.15.1:6800/3146869121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340486 (secure 0 0 0) 0x7f909c00bca0 con 0x7f90940a32c0 2024-09-16T00:28:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:56.364 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-16T00:28:56.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f90940a32c0 msgr2=0x7f90940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:56.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f90940a32c0 0x7f90940a5780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f909c0058c0 tx=0x7f909c005850 comp rx=0 tx=0).stop 2024-09-16T00:28:56.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 msgr2=0x7f90ac074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 0x7f90ac074a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004770 tx=0x7f90a0003c10 comp rx=0 tx=0).stop 2024-09-16T00:28:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 shutdown_connections 2024-09-16T00:28:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90ac106f10 0x7f90ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90ac102790 0x7f90ac074a30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f90940a32c0 0x7f90940a5780 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 --2- 172.21.15.1:0/3892862761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ac101de0 0x7f90ac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 >> 172.21.15.1:0/3892862761 conn(0x7f90ac0fd8c0 msgr2=0x7f90ac0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 shutdown_connections 2024-09-16T00:28:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:56.363+0000 7f90b283f640 1 -- 172.21.15.1:0/3892862761 wait complete. 2024-09-16T00:28:57.120 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-16T00:28:56.349055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663060,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":831,"num_read_kb":539,"num_write":363,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380140,"kb_used_data":14252,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561684,"statfs":{"total":1151940427776,"available":1151551164416,"internally_reserved":0,"allocated":14594048,"data_stored":8145134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":44,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002971"},"pg_stats":[{"pgid":"2.1f","version":"89'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:21.230796+0000","last_change":"2024-09-16T00:26:16.143602+0000","last_active":"2024-09-16T00:28:21.230796+0000","last_peered":"2024-09-16T00:28:21.230796+0000","last_clean":"2024-09-16T00:28:21.230796+0000","last_became_active":"2024-09-16T00:26:16.142546+0000","last_became_peered":"2024-09-16T00:26:16.142546+0000","last_unstale":"2024-09-16T00:28:21.230796+0000","last_undegraded":"2024-09-16T00:28:21.230796+0000","last_fullsized":"2024-09-16T00:28:21.230796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:21.051434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322537+0000","last_change":"2024-09-16T00:26:18.268470+0000","last_active":"2024-09-16T00:28:24.322537+0000","last_peered":"2024-09-16T00:28:24.322537+0000","last_clean":"2024-09-16T00:28:24.322537+0000","last_became_active":"2024-09-16T00:26:18.266049+0000","last_became_peered":"2024-09-16T00:26:18.266049+0000","last_unstale":"2024-09-16T00:28:24.322537+0000","last_undegraded":"2024-09-16T00:28:24.322537+0000","last_fullsized":"2024-09-16T00:28:24.322537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:32.616414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248176+0000","last_change":"2024-09-16T00:26:25.304161+0000","last_active":"2024-09-16T00:27:26.248176+0000","last_peered":"2024-09-16T00:27:26.248176+0000","last_clean":"2024-09-16T00:27:26.248176+0000","last_became_active":"2024-09-16T00:26:25.303646+0000","last_became_peered":"2024-09-16T00:26:25.303646+0000","last_unstale":"2024-09-16T00:27:26.248176+0000","last_undegraded":"2024-09-16T00:27:26.248176+0000","last_fullsized":"2024-09-16T00:27:26.248176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:19.705446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247830+0000","last_change":"2024-09-16T00:26:16.207136+0000","last_active":"2024-09-16T00:27:26.247830+0000","last_peered":"2024-09-16T00:27:26.247830+0000","last_clean":"2024-09-16T00:27:26.247830+0000","last_became_active":"2024-09-16T00:26:16.205401+0000","last_became_peered":"2024-09-16T00:26:16.205401+0000","last_unstale":"2024-09-16T00:27:26.247830+0000","last_undegraded":"2024-09-16T00:27:26.247830+0000","last_fullsized":"2024-09-16T00:27:26.247830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:38.963263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336305+0000","last_change":"2024-09-16T00:26:18.274312+0000","last_active":"2024-09-16T00:28:24.336305+0000","last_peered":"2024-09-16T00:28:24.336305+0000","last_clean":"2024-09-16T00:28:24.336305+0000","last_became_active":"2024-09-16T00:26:18.274041+0000","last_became_peered":"2024-09-16T00:26:18.274041+0000","last_unstale":"2024-09-16T00:28:24.336305+0000","last_undegraded":"2024-09-16T00:28:24.336305+0000","last_fullsized":"2024-09-16T00:28:24.336305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:03.810341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355341+0000","last_change":"2024-09-16T00:26:25.303503+0000","last_active":"2024-09-16T00:27:26.355341+0000","last_peered":"2024-09-16T00:27:26.355341+0000","last_clean":"2024-09-16T00:27:26.355341+0000","last_became_active":"2024-09-16T00:26:25.302208+0000","last_became_peered":"2024-09-16T00:26:25.302208+0000","last_unstale":"2024-09-16T00:27:26.355341+0000","last_undegraded":"2024-09-16T00:27:26.355341+0000","last_fullsized":"2024-09-16T00:27:26.355341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:21.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357885+0000","last_change":"2024-09-16T00:26:16.145661+0000","last_active":"2024-09-16T00:27:26.357885+0000","last_peered":"2024-09-16T00:27:26.357885+0000","last_clean":"2024-09-16T00:27:26.357885+0000","last_became_active":"2024-09-16T00:26:16.145277+0000","last_became_peered":"2024-09-16T00:26:16.145277+0000","last_unstale":"2024-09-16T00:27:26.357885+0000","last_undegraded":"2024-09-16T00:27:26.357885+0000","last_fullsized":"2024-09-16T00:27:26.357885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:55.986753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337468+0000","last_change":"2024-09-16T00:26:18.277019+0000","last_active":"2024-09-16T00:28:24.337468+0000","last_peered":"2024-09-16T00:28:24.337468+0000","last_clean":"2024-09-16T00:28:24.337468+0000","last_became_active":"2024-09-16T00:26:18.276615+0000","last_became_peered":"2024-09-16T00:26:18.276615+0000","last_unstale":"2024-09-16T00:28:24.337468+0000","last_undegraded":"2024-09-16T00:28:24.337468+0000","last_fullsized":"2024-09-16T00:28:24.337468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:29.403945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356550+0000","last_change":"2024-09-16T00:26:25.303969+0000","last_active":"2024-09-16T00:27:26.356550+0000","last_peered":"2024-09-16T00:27:26.356550+0000","last_clean":"2024-09-16T00:27:26.356550+0000","last_became_active":"2024-09-16T00:26:25.302652+0000","last_became_peered":"2024-09-16T00:26:25.302652+0000","last_unstale":"2024-09-16T00:27:26.356550+0000","last_undegraded":"2024-09-16T00:27:26.356550+0000","last_fullsized":"2024-09-16T00:27:26.356550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:06.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252350+0000","last_change":"2024-09-16T00:26:25.303749+0000","last_active":"2024-09-16T00:27:26.252350+0000","last_peered":"2024-09-16T00:27:26.252350+0000","last_clean":"2024-09-16T00:27:26.252350+0000","last_became_active":"2024-09-16T00:26:25.295993+0000","last_became_peered":"2024-09-16T00:26:25.295993+0000","last_unstale":"2024-09-16T00:27:26.252350+0000","last_undegraded":"2024-09-16T00:27:26.252350+0000","last_fullsized":"2024-09-16T00:27:26.252350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:12.480025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336557+0000","last_change":"2024-09-16T00:26:18.273520+0000","last_active":"2024-09-16T00:28:24.336557+0000","last_peered":"2024-09-16T00:28:24.336557+0000","last_clean":"2024-09-16T00:28:24.336557+0000","last_became_active":"2024-09-16T00:26:18.272594+0000","last_became_peered":"2024-09-16T00:26:18.272594+0000","last_unstale":"2024-09-16T00:28:24.336557+0000","last_undegraded":"2024-09-16T00:28:24.336557+0000","last_fullsized":"2024-09-16T00:28:24.336557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.054768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355907+0000","last_change":"2024-09-16T00:26:25.310461+0000","last_active":"2024-09-16T00:27:26.355907+0000","last_peered":"2024-09-16T00:27:26.355907+0000","last_clean":"2024-09-16T00:27:26.355907+0000","last_became_active":"2024-09-16T00:26:25.309214+0000","last_became_peered":"2024-09-16T00:26:25.309214+0000","last_unstale":"2024-09-16T00:27:26.355907+0000","last_undegraded":"2024-09-16T00:27:26.355907+0000","last_fullsized":"2024-09-16T00:27:26.355907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:32.357355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338941+0000","last_change":"2024-09-16T00:26:18.277126+0000","last_active":"2024-09-16T00:28:24.338941+0000","last_peered":"2024-09-16T00:28:24.338941+0000","last_clean":"2024-09-16T00:28:24.338941+0000","last_became_active":"2024-09-16T00:26:18.276803+0000","last_became_peered":"2024-09-16T00:26:18.276803+0000","last_unstale":"2024-09-16T00:28:24.338941+0000","last_undegraded":"2024-09-16T00:28:24.338941+0000","last_fullsized":"2024-09-16T00:28:24.338941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:58.544674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357104+0000","last_change":"2024-09-16T00:26:16.146183+0000","last_active":"2024-09-16T00:27:26.357104+0000","last_peered":"2024-09-16T00:27:26.357104+0000","last_clean":"2024-09-16T00:27:26.357104+0000","last_became_active":"2024-09-16T00:26:16.145320+0000","last_became_peered":"2024-09-16T00:26:16.145320+0000","last_unstale":"2024-09-16T00:27:26.357104+0000","last_undegraded":"2024-09-16T00:27:26.357104+0000","last_fullsized":"2024-09-16T00:27:26.357104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:44.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356176+0000","last_change":"2024-09-16T00:26:25.309773+0000","last_active":"2024-09-16T00:27:26.356176+0000","last_peered":"2024-09-16T00:27:26.356176+0000","last_clean":"2024-09-16T00:27:26.356176+0000","last_became_active":"2024-09-16T00:26:25.309258+0000","last_became_peered":"2024-09-16T00:26:25.309258+0000","last_unstale":"2024-09-16T00:27:26.356176+0000","last_undegraded":"2024-09-16T00:27:26.356176+0000","last_fullsized":"2024-09-16T00:27:26.356176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:11.922713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327188+0000","last_change":"2024-09-16T00:26:18.262431+0000","last_active":"2024-09-16T00:28:24.327188+0000","last_peered":"2024-09-16T00:28:24.327188+0000","last_clean":"2024-09-16T00:28:24.327188+0000","last_became_active":"2024-09-16T00:26:18.262136+0000","last_became_peered":"2024-09-16T00:26:18.262136+0000","last_unstale":"2024-09-16T00:28:24.327188+0000","last_undegraded":"2024-09-16T00:28:24.327188+0000","last_fullsized":"2024-09-16T00:28:24.327188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:56.703666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248306+0000","last_change":"2024-09-16T00:26:16.142386+0000","last_active":"2024-09-16T00:27:26.248306+0000","last_peered":"2024-09-16T00:27:26.248306+0000","last_clean":"2024-09-16T00:27:26.248306+0000","last_became_active":"2024-09-16T00:26:16.142036+0000","last_became_peered":"2024-09-16T00:26:16.142036+0000","last_unstale":"2024-09-16T00:27:26.248306+0000","last_undegraded":"2024-09-16T00:27:26.248306+0000","last_fullsized":"2024-09-16T00:27:26.248306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.424572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356287+0000","last_change":"2024-09-16T00:26:25.309909+0000","last_active":"2024-09-16T00:27:26.356287+0000","last_peered":"2024-09-16T00:27:26.356287+0000","last_clean":"2024-09-16T00:27:26.356287+0000","last_became_active":"2024-09-16T00:26:25.309482+0000","last_became_peered":"2024-09-16T00:26:25.309482+0000","last_unstale":"2024-09-16T00:27:26.356287+0000","last_undegraded":"2024-09-16T00:27:26.356287+0000","last_fullsized":"2024-09-16T00:27:26.356287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:53.030207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.324477+0000","last_change":"2024-09-16T00:26:18.265233+0000","last_active":"2024-09-16T00:28:24.324477+0000","last_peered":"2024-09-16T00:28:24.324477+0000","last_clean":"2024-09-16T00:28:24.324477+0000","last_became_active":"2024-09-16T00:26:18.264362+0000","last_became_peered":"2024-09-16T00:26:18.264362+0000","last_unstale":"2024-09-16T00:28:24.324477+0000","last_undegraded":"2024-09-16T00:28:24.324477+0000","last_fullsized":"2024-09-16T00:28:24.324477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:28.118211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247518+0000","last_change":"2024-09-16T00:26:16.145268+0000","last_active":"2024-09-16T00:27:26.247518+0000","last_peered":"2024-09-16T00:27:26.247518+0000","last_clean":"2024-09-16T00:27:26.247518+0000","last_became_active":"2024-09-16T00:26:16.144882+0000","last_became_peered":"2024-09-16T00:26:16.144882+0000","last_unstale":"2024-09-16T00:27:26.247518+0000","last_undegraded":"2024-09-16T00:27:26.247518+0000","last_fullsized":"2024-09-16T00:27:26.247518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:25.871287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252497+0000","last_change":"2024-09-16T00:26:25.296176+0000","last_active":"2024-09-16T00:27:26.252497+0000","last_peered":"2024-09-16T00:27:26.252497+0000","last_clean":"2024-09-16T00:27:26.252497+0000","last_became_active":"2024-09-16T00:26:25.295745+0000","last_became_peered":"2024-09-16T00:26:25.295745+0000","last_unstale":"2024-09-16T00:27:26.252497+0000","last_undegraded":"2024-09-16T00:27:26.252497+0000","last_fullsized":"2024-09-16T00:27:26.252497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:19.323632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.312122+0000","last_change":"2024-09-16T00:26:18.268241+0000","last_active":"2024-09-16T00:28:24.312122+0000","last_peered":"2024-09-16T00:28:24.312122+0000","last_clean":"2024-09-16T00:28:24.312122+0000","last_became_active":"2024-09-16T00:26:18.266967+0000","last_became_peered":"2024-09-16T00:26:18.266967+0000","last_unstale":"2024-09-16T00:28:24.312122+0000","last_undegraded":"2024-09-16T00:28:24.312122+0000","last_fullsized":"2024-09-16T00:28:24.312122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:24.422041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355055+0000","last_change":"2024-09-16T00:26:16.205867+0000","last_active":"2024-09-16T00:27:26.355055+0000","last_peered":"2024-09-16T00:27:26.355055+0000","last_clean":"2024-09-16T00:27:26.355055+0000","last_became_active":"2024-09-16T00:26:16.205515+0000","last_became_peered":"2024-09-16T00:26:16.205515+0000","last_unstale":"2024-09-16T00:27:26.355055+0000","last_undegraded":"2024-09-16T00:27:26.355055+0000","last_fullsized":"2024-09-16T00:27:26.355055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:06.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357331+0000","last_change":"2024-09-16T00:26:25.306580+0000","last_active":"2024-09-16T00:27:26.357331+0000","last_peered":"2024-09-16T00:27:26.357331+0000","last_clean":"2024-09-16T00:27:26.357331+0000","last_became_active":"2024-09-16T00:26:25.306154+0000","last_became_peered":"2024-09-16T00:26:25.306154+0000","last_unstale":"2024-09-16T00:27:26.357331+0000","last_undegraded":"2024-09-16T00:27:26.357331+0000","last_fullsized":"2024-09-16T00:27:26.357331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:57.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334895+0000","last_change":"2024-09-16T00:27:26.358639+0000","last_active":"2024-09-16T00:28:24.334895+0000","last_peered":"2024-09-16T00:28:24.334895+0000","last_clean":"2024-09-16T00:28:24.334895+0000","last_became_active":"2024-09-16T00:27:26.358163+0000","last_became_peered":"2024-09-16T00:27:26.358163+0000","last_unstale":"2024-09-16T00:28:24.334895+0000","last_undegraded":"2024-09-16T00:28:24.334895+0000","last_fullsized":"2024-09-16T00:28:24.334895+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:11.811436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356716+0000","last_change":"2024-09-16T00:26:16.138308+0000","last_active":"2024-09-16T00:27:26.356716+0000","last_peered":"2024-09-16T00:27:26.356716+0000","last_clean":"2024-09-16T00:27:26.356716+0000","last_became_active":"2024-09-16T00:26:16.137955+0000","last_became_peered":"2024-09-16T00:26:16.137955+0000","last_unstale":"2024-09-16T00:27:26.356716+0000","last_undegraded":"2024-09-16T00:27:26.356716+0000","last_fullsized":"2024-09-16T00:27:26.356716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:31.677418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248397+0000","last_change":"2024-09-16T00:26:25.303386+0000","last_active":"2024-09-16T00:27:26.248397+0000","last_peered":"2024-09-16T00:27:26.248397+0000","last_clean":"2024-09-16T00:27:26.248397+0000","last_became_active":"2024-09-16T00:26:25.302979+0000","last_became_peered":"2024-09-16T00:26:25.302979+0000","last_unstale":"2024-09-16T00:27:26.248397+0000","last_undegraded":"2024-09-16T00:27:26.248397+0000","last_fullsized":"2024-09-16T00:27:26.248397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.384435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332074+0000","last_change":"2024-09-16T00:26:18.264668+0000","last_active":"2024-09-16T00:28:24.332074+0000","last_peered":"2024-09-16T00:28:24.332074+0000","last_clean":"2024-09-16T00:28:24.332074+0000","last_became_active":"2024-09-16T00:26:18.264371+0000","last_became_peered":"2024-09-16T00:26:18.264371+0000","last_unstale":"2024-09-16T00:28:24.332074+0000","last_undegraded":"2024-09-16T00:28:24.332074+0000","last_fullsized":"2024-09-16T00:28:24.332074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:10.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:26.231276+0000","last_change":"2024-09-16T00:26:16.141566+0000","last_active":"2024-09-16T00:28:26.231276+0000","last_peered":"2024-09-16T00:28:26.231276+0000","last_clean":"2024-09-16T00:28:26.231276+0000","last_became_active":"2024-09-16T00:26:16.141239+0000","last_became_peered":"2024-09-16T00:26:16.141239+0000","last_unstale":"2024-09-16T00:28:26.231276+0000","last_undegraded":"2024-09-16T00:28:26.231276+0000","last_fullsized":"2024-09-16T00:28:26.231276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:27.815488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247207+0000","last_change":"2024-09-16T00:26:25.303013+0000","last_active":"2024-09-16T00:27:26.247207+0000","last_peered":"2024-09-16T00:27:26.247207+0000","last_clean":"2024-09-16T00:27:26.247207+0000","last_became_active":"2024-09-16T00:26:25.301732+0000","last_became_peered":"2024-09-16T00:26:25.301732+0000","last_unstale":"2024-09-16T00:27:26.247207+0000","last_undegraded":"2024-09-16T00:27:26.247207+0000","last_fullsized":"2024-09-16T00:27:26.247207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:27.513343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.325854+0000","last_change":"2024-09-16T00:26:18.268648+0000","last_active":"2024-09-16T00:28:24.325854+0000","last_peered":"2024-09-16T00:28:24.325854+0000","last_clean":"2024-09-16T00:28:24.325854+0000","last_became_active":"2024-09-16T00:26:18.268302+0000","last_became_peered":"2024-09-16T00:26:18.268302+0000","last_unstale":"2024-09-16T00:28:24.325854+0000","last_undegraded":"2024-09-16T00:28:24.325854+0000","last_fullsized":"2024-09-16T00:28:24.325854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:48.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357960+0000","last_change":"2024-09-16T00:26:16.207224+0000","last_active":"2024-09-16T00:27:26.357960+0000","last_peered":"2024-09-16T00:27:26.357960+0000","last_clean":"2024-09-16T00:27:26.357960+0000","last_became_active":"2024-09-16T00:26:16.206847+0000","last_became_peered":"2024-09-16T00:26:16.206847+0000","last_unstale":"2024-09-16T00:27:26.357960+0000","last_undegraded":"2024-09-16T00:27:26.357960+0000","last_fullsized":"2024-09-16T00:27:26.357960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:44.159104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.251785+0000","last_change":"2024-09-16T00:26:25.299989+0000","last_active":"2024-09-16T00:27:26.251785+0000","last_peered":"2024-09-16T00:27:26.251785+0000","last_clean":"2024-09-16T00:27:26.251785+0000","last_became_active":"2024-09-16T00:26:25.299547+0000","last_became_peered":"2024-09-16T00:26:25.299547+0000","last_unstale":"2024-09-16T00:27:26.251785+0000","last_undegraded":"2024-09-16T00:27:26.251785+0000","last_fullsized":"2024-09-16T00:27:26.251785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:14.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329144+0000","last_change":"2024-09-16T00:26:18.255989+0000","last_active":"2024-09-16T00:28:24.329144+0000","last_peered":"2024-09-16T00:28:24.329144+0000","last_clean":"2024-09-16T00:28:24.329144+0000","last_became_active":"2024-09-16T00:26:18.255636+0000","last_became_peered":"2024-09-16T00:26:18.255636+0000","last_unstale":"2024-09-16T00:28:24.329144+0000","last_undegraded":"2024-09-16T00:28:24.329144+0000","last_fullsized":"2024-09-16T00:28:24.329144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:17.222032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252455+0000","last_change":"2024-09-16T00:26:25.299402+0000","last_active":"2024-09-16T00:27:26.252455+0000","last_peered":"2024-09-16T00:27:26.252455+0000","last_clean":"2024-09-16T00:27:26.252455+0000","last_became_active":"2024-09-16T00:26:25.298106+0000","last_became_peered":"2024-09-16T00:26:25.298106+0000","last_unstale":"2024-09-16T00:27:26.252455+0000","last_undegraded":"2024-09-16T00:27:26.252455+0000","last_fullsized":"2024-09-16T00:27:26.252455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:28.105049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357048+0000","last_change":"2024-09-16T00:26:25.307003+0000","last_active":"2024-09-16T00:27:26.357048+0000","last_peered":"2024-09-16T00:27:26.357048+0000","last_clean":"2024-09-16T00:27:26.357048+0000","last_became_active":"2024-09-16T00:26:25.306574+0000","last_became_peered":"2024-09-16T00:26:25.306574+0000","last_unstale":"2024-09-16T00:27:26.357048+0000","last_undegraded":"2024-09-16T00:27:26.357048+0000","last_fullsized":"2024-09-16T00:27:26.357048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:49.679743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321720+0000","last_change":"2024-09-16T00:26:18.264280+0000","last_active":"2024-09-16T00:28:24.321720+0000","last_peered":"2024-09-16T00:28:24.321720+0000","last_clean":"2024-09-16T00:28:24.321720+0000","last_became_active":"2024-09-16T00:26:18.263881+0000","last_became_peered":"2024-09-16T00:26:18.263881+0000","last_unstale":"2024-09-16T00:28:24.321720+0000","last_undegraded":"2024-09-16T00:28:24.321720+0000","last_fullsized":"2024-09-16T00:28:24.321720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:00.448672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357297+0000","last_change":"2024-09-16T00:26:16.147547+0000","last_active":"2024-09-16T00:27:26.357297+0000","last_peered":"2024-09-16T00:27:26.357297+0000","last_clean":"2024-09-16T00:27:26.357297+0000","last_became_active":"2024-09-16T00:26:16.147187+0000","last_became_peered":"2024-09-16T00:26:16.147187+0000","last_unstale":"2024-09-16T00:27:26.357297+0000","last_undegraded":"2024-09-16T00:27:26.357297+0000","last_fullsized":"2024-09-16T00:27:26.357297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:31.437524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250069+0000","last_change":"2024-09-16T00:26:25.309013+0000","last_active":"2024-09-16T00:27:26.250069+0000","last_peered":"2024-09-16T00:27:26.250069+0000","last_clean":"2024-09-16T00:27:26.250069+0000","last_became_active":"2024-09-16T00:26:25.307982+0000","last_became_peered":"2024-09-16T00:26:25.307982+0000","last_unstale":"2024-09-16T00:27:26.250069+0000","last_undegraded":"2024-09-16T00:27:26.250069+0000","last_fullsized":"2024-09-16T00:27:26.250069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:37.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.305108+0000","last_change":"2024-09-16T00:27:26.357501+0000","last_active":"2024-09-16T00:28:24.305108+0000","last_peered":"2024-09-16T00:28:24.305108+0000","last_clean":"2024-09-16T00:28:24.305108+0000","last_became_active":"2024-09-16T00:27:26.357108+0000","last_became_peered":"2024-09-16T00:27:26.357108+0000","last_unstale":"2024-09-16T00:28:24.305108+0000","last_undegraded":"2024-09-16T00:28:24.305108+0000","last_fullsized":"2024-09-16T00:28:24.305108+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:25.264059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:54.682498+0000","last_change":"2024-09-16T00:26:16.205741+0000","last_active":"2024-09-16T00:28:54.682498+0000","last_peered":"2024-09-16T00:28:54.682498+0000","last_clean":"2024-09-16T00:28:54.682498+0000","last_became_active":"2024-09-16T00:26:16.205370+0000","last_became_peered":"2024-09-16T00:26:16.205370+0000","last_unstale":"2024-09-16T00:28:54.682498+0000","last_undegraded":"2024-09-16T00:28:54.682498+0000","last_fullsized":"2024-09-16T00:28:54.682498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:28.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250857+0000","last_change":"2024-09-16T00:26:25.310155+0000","last_active":"2024-09-16T00:27:26.250857+0000","last_peered":"2024-09-16T00:27:26.250857+0000","last_clean":"2024-09-16T00:27:26.250857+0000","last_became_active":"2024-09-16T00:26:25.309344+0000","last_became_peered":"2024-09-16T00:26:25.309344+0000","last_unstale":"2024-09-16T00:27:26.250857+0000","last_undegraded":"2024-09-16T00:27:26.250857+0000","last_fullsized":"2024-09-16T00:27:26.250857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:54.274026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326885+0000","last_change":"2024-09-16T00:26:18.266375+0000","last_active":"2024-09-16T00:28:24.326885+0000","last_peered":"2024-09-16T00:28:24.326885+0000","last_clean":"2024-09-16T00:28:24.326885+0000","last_became_active":"2024-09-16T00:26:18.266081+0000","last_became_peered":"2024-09-16T00:26:18.266081+0000","last_unstale":"2024-09-16T00:28:24.326885+0000","last_undegraded":"2024-09-16T00:28:24.326885+0000","last_fullsized":"2024-09-16T00:28:24.326885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:25.127904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357096+0000","last_change":"2024-09-16T00:26:16.146291+0000","last_active":"2024-09-16T00:27:26.357096+0000","last_peered":"2024-09-16T00:27:26.357096+0000","last_clean":"2024-09-16T00:27:26.357096+0000","last_became_active":"2024-09-16T00:26:16.145932+0000","last_became_peered":"2024-09-16T00:26:16.145932+0000","last_unstale":"2024-09-16T00:27:26.357096+0000","last_undegraded":"2024-09-16T00:27:26.357096+0000","last_fullsized":"2024-09-16T00:27:26.357096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:26.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356120+0000","last_change":"2024-09-16T00:26:25.303080+0000","last_active":"2024-09-16T00:27:26.356120+0000","last_peered":"2024-09-16T00:27:26.356120+0000","last_clean":"2024-09-16T00:27:26.356120+0000","last_became_active":"2024-09-16T00:26:25.302074+0000","last_became_peered":"2024-09-16T00:26:25.302074+0000","last_unstale":"2024-09-16T00:27:26.356120+0000","last_undegraded":"2024-09-16T00:27:26.356120+0000","last_fullsized":"2024-09-16T00:27:26.356120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:04.578201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336576+0000","last_change":"2024-09-16T00:26:18.274263+0000","last_active":"2024-09-16T00:28:24.336576+0000","last_peered":"2024-09-16T00:28:24.336576+0000","last_clean":"2024-09-16T00:28:24.336576+0000","last_became_active":"2024-09-16T00:26:18.273925+0000","last_became_peered":"2024-09-16T00:26:18.273925+0000","last_unstale":"2024-09-16T00:28:24.336576+0000","last_undegraded":"2024-09-16T00:28:24.336576+0000","last_fullsized":"2024-09-16T00:28:24.336576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:30.123857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357054+0000","last_change":"2024-09-16T00:26:16.145417+0000","last_active":"2024-09-16T00:27:26.357054+0000","last_peered":"2024-09-16T00:27:26.357054+0000","last_clean":"2024-09-16T00:27:26.357054+0000","last_became_active":"2024-09-16T00:26:16.144888+0000","last_became_peered":"2024-09-16T00:26:16.144888+0000","last_unstale":"2024-09-16T00:27:26.357054+0000","last_undegraded":"2024-09-16T00:27:26.357054+0000","last_fullsized":"2024-09-16T00:27:26.357054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:27.578611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357365+0000","last_change":"2024-09-16T00:25:05.002525+0000","last_active":"2024-09-16T00:27:26.357365+0000","last_peered":"2024-09-16T00:27:26.357365+0000","last_clean":"2024-09-16T00:27:26.357365+0000","last_became_active":"2024-09-16T00:25:05.002136+0000","last_became_peered":"2024-09-16T00:25:05.002136+0000","last_unstale":"2024-09-16T00:27:26.357365+0000","last_undegraded":"2024-09-16T00:27:26.357365+0000","last_fullsized":"2024-09-16T00:27:26.357365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:21:40.721937+0000","last_clean_scrub_stamp":"2024-09-16T00:21:40.721937+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:46.018426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355730+0000","last_change":"2024-09-16T00:26:25.295466+0000","last_active":"2024-09-16T00:27:26.355730+0000","last_peered":"2024-09-16T00:27:26.355730+0000","last_clean":"2024-09-16T00:27:26.355730+0000","last_became_active":"2024-09-16T00:26:25.295007+0000","last_became_peered":"2024-09-16T00:26:25.295007+0000","last_unstale":"2024-09-16T00:27:26.355730+0000","last_undegraded":"2024-09-16T00:27:26.355730+0000","last_fullsized":"2024-09-16T00:27:26.355730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:32.912369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338444+0000","last_change":"2024-09-16T00:26:18.275493+0000","last_active":"2024-09-16T00:28:24.338444+0000","last_peered":"2024-09-16T00:28:24.338444+0000","last_clean":"2024-09-16T00:28:24.338444+0000","last_became_active":"2024-09-16T00:26:18.275139+0000","last_became_peered":"2024-09-16T00:26:18.275139+0000","last_unstale":"2024-09-16T00:28:24.338444+0000","last_undegraded":"2024-09-16T00:28:24.338444+0000","last_fullsized":"2024-09-16T00:28:24.338444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:53.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248838+0000","last_change":"2024-09-16T00:26:16.205954+0000","last_active":"2024-09-16T00:27:26.248838+0000","last_peered":"2024-09-16T00:27:26.248838+0000","last_clean":"2024-09-16T00:27:26.248838+0000","last_became_active":"2024-09-16T00:26:16.205535+0000","last_became_peered":"2024-09-16T00:26:16.205535+0000","last_unstale":"2024-09-16T00:27:26.248838+0000","last_undegraded":"2024-09-16T00:27:26.248838+0000","last_fullsized":"2024-09-16T00:27:26.248838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:48.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355174+0000","last_change":"2024-09-16T00:26:25.302523+0000","last_active":"2024-09-16T00:27:26.355174+0000","last_peered":"2024-09-16T00:27:26.355174+0000","last_clean":"2024-09-16T00:27:26.355174+0000","last_became_active":"2024-09-16T00:26:25.302024+0000","last_became_peered":"2024-09-16T00:26:25.302024+0000","last_unstale":"2024-09-16T00:27:26.355174+0000","last_undegraded":"2024-09-16T00:27:26.355174+0000","last_fullsized":"2024-09-16T00:27:26.355174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.547341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331484+0000","last_change":"2024-09-16T00:26:18.268985+0000","last_active":"2024-09-16T00:28:24.331484+0000","last_peered":"2024-09-16T00:28:24.331484+0000","last_clean":"2024-09-16T00:28:24.331484+0000","last_became_active":"2024-09-16T00:26:18.268666+0000","last_became_peered":"2024-09-16T00:26:18.268666+0000","last_unstale":"2024-09-16T00:28:24.331484+0000","last_undegraded":"2024-09-16T00:28:24.331484+0000","last_fullsized":"2024-09-16T00:28:24.331484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:47.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356422+0000","last_change":"2024-09-16T00:26:16.145918+0000","last_active":"2024-09-16T00:27:26.356422+0000","last_peered":"2024-09-16T00:27:26.356422+0000","last_clean":"2024-09-16T00:27:26.356422+0000","last_became_active":"2024-09-16T00:26:16.145469+0000","last_became_peered":"2024-09-16T00:26:16.145469+0000","last_unstale":"2024-09-16T00:27:26.356422+0000","last_undegraded":"2024-09-16T00:27:26.356422+0000","last_fullsized":"2024-09-16T00:27:26.356422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:24.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.249539+0000","last_change":"2024-09-16T00:26:25.303353+0000","last_active":"2024-09-16T00:27:26.249539+0000","last_peered":"2024-09-16T00:27:26.249539+0000","last_clean":"2024-09-16T00:27:26.249539+0000","last_became_active":"2024-09-16T00:26:25.302956+0000","last_became_peered":"2024-09-16T00:26:25.302956+0000","last_unstale":"2024-09-16T00:27:26.249539+0000","last_undegraded":"2024-09-16T00:27:26.249539+0000","last_fullsized":"2024-09-16T00:27:26.249539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:30.250440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.333565+0000","last_change":"2024-09-16T00:26:18.266766+0000","last_active":"2024-09-16T00:28:24.333565+0000","last_peered":"2024-09-16T00:28:24.333565+0000","last_clean":"2024-09-16T00:28:24.333565+0000","last_became_active":"2024-09-16T00:26:18.266388+0000","last_became_peered":"2024-09-16T00:26:18.266388+0000","last_unstale":"2024-09-16T00:28:24.333565+0000","last_undegraded":"2024-09-16T00:28:24.333565+0000","last_fullsized":"2024-09-16T00:28:24.333565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:33.783928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250975+0000","last_change":"2024-09-16T00:26:16.138522+0000","last_active":"2024-09-16T00:27:26.250975+0000","last_peered":"2024-09-16T00:27:26.250975+0000","last_clean":"2024-09-16T00:27:26.250975+0000","last_became_active":"2024-09-16T00:26:16.138199+0000","last_became_peered":"2024-09-16T00:26:16.138199+0000","last_unstale":"2024-09-16T00:27:26.250975+0000","last_undegraded":"2024-09-16T00:27:26.250975+0000","last_fullsized":"2024-09-16T00:27:26.250975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:56.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.354985+0000","last_change":"2024-09-16T00:26:25.296448+0000","last_active":"2024-09-16T00:27:26.354985+0000","last_peered":"2024-09-16T00:27:26.354985+0000","last_clean":"2024-09-16T00:27:26.354985+0000","last_became_active":"2024-09-16T00:26:25.295405+0000","last_became_peered":"2024-09-16T00:26:25.295405+0000","last_unstale":"2024-09-16T00:27:26.354985+0000","last_undegraded":"2024-09-16T00:27:26.354985+0000","last_fullsized":"2024-09-16T00:27:26.354985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:24.460033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.319032+0000","last_change":"2024-09-16T00:26:18.262437+0000","last_active":"2024-09-16T00:28:24.319032+0000","last_peered":"2024-09-16T00:28:24.319032+0000","last_clean":"2024-09-16T00:28:24.319032+0000","last_became_active":"2024-09-16T00:26:18.262141+0000","last_became_peered":"2024-09-16T00:26:18.262141+0000","last_unstale":"2024-09-16T00:28:24.319032+0000","last_undegraded":"2024-09-16T00:28:24.319032+0000","last_fullsized":"2024-09-16T00:28:24.319032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:58.912457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357916+0000","last_change":"2024-09-16T00:26:16.147957+0000","last_active":"2024-09-16T00:27:26.357916+0000","last_peered":"2024-09-16T00:27:26.357916+0000","last_clean":"2024-09-16T00:27:26.357916+0000","last_became_active":"2024-09-16T00:26:16.145672+0000","last_became_peered":"2024-09-16T00:26:16.145672+0000","last_unstale":"2024-09-16T00:27:26.357916+0000","last_undegraded":"2024-09-16T00:27:26.357916+0000","last_fullsized":"2024-09-16T00:27:26.357916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:41.150678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355183+0000","last_change":"2024-09-16T00:26:25.298975+0000","last_active":"2024-09-16T00:27:26.355183+0000","last_peered":"2024-09-16T00:27:26.355183+0000","last_clean":"2024-09-16T00:27:26.355183+0000","last_became_active":"2024-09-16T00:26:25.298330+0000","last_became_peered":"2024-09-16T00:26:25.298330+0000","last_unstale":"2024-09-16T00:27:26.355183+0000","last_undegraded":"2024-09-16T00:27:26.355183+0000","last_fullsized":"2024-09-16T00:27:26.355183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:42.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336143+0000","last_change":"2024-09-16T00:26:18.272888+0000","last_active":"2024-09-16T00:28:24.336143+0000","last_peered":"2024-09-16T00:28:24.336143+0000","last_clean":"2024-09-16T00:28:24.336143+0000","last_became_active":"2024-09-16T00:26:18.272553+0000","last_became_peered":"2024-09-16T00:26:18.272553+0000","last_unstale":"2024-09-16T00:28:24.336143+0000","last_undegraded":"2024-09-16T00:28:24.336143+0000","last_fullsized":"2024-09-16T00:28:24.336143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:17.194675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356998+0000","last_change":"2024-09-16T00:26:16.206139+0000","last_active":"2024-09-16T00:27:26.356998+0000","last_peered":"2024-09-16T00:27:26.356998+0000","last_clean":"2024-09-16T00:27:26.356998+0000","last_became_active":"2024-09-16T00:26:16.205700+0000","last_became_peered":"2024-09-16T00:26:16.205700+0000","last_unstale":"2024-09-16T00:27:26.356998+0000","last_undegraded":"2024-09-16T00:27:26.356998+0000","last_fullsized":"2024-09-16T00:27:26.356998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:57.637739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.254250+0000","last_change":"2024-09-16T00:27:26.253824+0000","last_active":"2024-09-16T00:27:26.254250+0000","last_peered":"2024-09-16T00:27:26.254250+0000","last_clean":"2024-09-16T00:27:26.254250+0000","last_became_active":"2024-09-16T00:27:26.253206+0000","last_became_peered":"2024-09-16T00:27:26.253206+0000","last_unstale":"2024-09-16T00:27:26.254250+0000","last_undegraded":"2024-09-16T00:27:26.254250+0000","last_fullsized":"2024-09-16T00:27:26.254250+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:22.543369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.332839+0000","last_change":"2024-09-16T00:26:18.271970+0000","last_active":"2024-09-16T00:28:24.332839+0000","last_peered":"2024-09-16T00:28:24.332839+0000","last_clean":"2024-09-16T00:28:24.332839+0000","last_became_active":"2024-09-16T00:26:18.271627+0000","last_became_peered":"2024-09-16T00:26:18.271627+0000","last_unstale":"2024-09-16T00:28:24.332839+0000","last_undegraded":"2024-09-16T00:28:24.332839+0000","last_fullsized":"2024-09-16T00:28:24.332839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:38.303208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:53.681669+0000","last_change":"2024-09-16T00:26:16.146024+0000","last_active":"2024-09-16T00:28:53.681669+0000","last_peered":"2024-09-16T00:28:53.681669+0000","last_clean":"2024-09-16T00:28:53.681669+0000","last_became_active":"2024-09-16T00:26:16.145168+0000","last_became_peered":"2024-09-16T00:26:16.145168+0000","last_unstale":"2024-09-16T00:28:53.681669+0000","last_undegraded":"2024-09-16T00:28:53.681669+0000","last_fullsized":"2024-09-16T00:28:53.681669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:13.566722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252606+0000","last_change":"2024-09-16T00:26:25.297458+0000","last_active":"2024-09-16T00:27:26.252606+0000","last_peered":"2024-09-16T00:27:26.252606+0000","last_clean":"2024-09-16T00:27:26.252606+0000","last_became_active":"2024-09-16T00:26:25.296221+0000","last_became_peered":"2024-09-16T00:26:25.296221+0000","last_unstale":"2024-09-16T00:27:26.252606+0000","last_undegraded":"2024-09-16T00:27:26.252606+0000","last_fullsized":"2024-09-16T00:27:26.252606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:31.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.335224+0000","last_change":"2024-09-16T00:26:18.264430+0000","last_active":"2024-09-16T00:28:24.335224+0000","last_peered":"2024-09-16T00:28:24.335224+0000","last_clean":"2024-09-16T00:28:24.335224+0000","last_became_active":"2024-09-16T00:26:18.264122+0000","last_became_peered":"2024-09-16T00:26:18.264122+0000","last_unstale":"2024-09-16T00:28:24.335224+0000","last_undegraded":"2024-09-16T00:28:24.335224+0000","last_fullsized":"2024-09-16T00:28:24.335224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:52.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356320+0000","last_change":"2024-09-16T00:26:16.144613+0000","last_active":"2024-09-16T00:27:26.356320+0000","last_peered":"2024-09-16T00:27:26.356320+0000","last_clean":"2024-09-16T00:27:26.356320+0000","last_became_active":"2024-09-16T00:26:16.143177+0000","last_became_peered":"2024-09-16T00:26:16.143177+0000","last_unstale":"2024-09-16T00:27:26.356320+0000","last_undegraded":"2024-09-16T00:27:26.356320+0000","last_fullsized":"2024-09-16T00:27:26.356320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:30.614996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252594+0000","last_change":"2024-09-16T00:26:25.306849+0000","last_active":"2024-09-16T00:27:26.252594+0000","last_peered":"2024-09-16T00:27:26.252594+0000","last_clean":"2024-09-16T00:27:26.252594+0000","last_became_active":"2024-09-16T00:26:25.306257+0000","last_became_peered":"2024-09-16T00:26:25.306257+0000","last_unstale":"2024-09-16T00:27:26.252594+0000","last_undegraded":"2024-09-16T00:27:26.252594+0000","last_fullsized":"2024-09-16T00:27:26.252594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:43.598508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330126+0000","last_change":"2024-09-16T00:26:18.263342+0000","last_active":"2024-09-16T00:28:24.330126+0000","last_peered":"2024-09-16T00:28:24.330126+0000","last_clean":"2024-09-16T00:28:24.330126+0000","last_became_active":"2024-09-16T00:26:18.263013+0000","last_became_peered":"2024-09-16T00:26:18.263013+0000","last_unstale":"2024-09-16T00:28:24.330126+0000","last_undegraded":"2024-09-16T00:28:24.330126+0000","last_fullsized":"2024-09-16T00:28:24.330126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:27.186988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248474+0000","last_change":"2024-09-16T00:26:16.144324+0000","last_active":"2024-09-16T00:27:26.248474+0000","last_peered":"2024-09-16T00:27:26.248474+0000","last_clean":"2024-09-16T00:27:26.248474+0000","last_became_active":"2024-09-16T00:26:16.143967+0000","last_became_peered":"2024-09-16T00:26:16.143967+0000","last_unstale":"2024-09-16T00:27:26.248474+0000","last_undegraded":"2024-09-16T00:27:26.248474+0000","last_fullsized":"2024-09-16T00:27:26.248474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:44.878262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355990+0000","last_change":"2024-09-16T00:26:25.300749+0000","last_active":"2024-09-16T00:27:26.355990+0000","last_peered":"2024-09-16T00:27:26.355990+0000","last_clean":"2024-09-16T00:27:26.355990+0000","last_became_active":"2024-09-16T00:26:25.300370+0000","last_became_peered":"2024-09-16T00:26:25.300370+0000","last_unstale":"2024-09-16T00:27:26.355990+0000","last_undegraded":"2024-09-16T00:27:26.355990+0000","last_fullsized":"2024-09-16T00:27:26.355990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:09.894241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.311816+0000","last_change":"2024-09-16T00:26:18.267777+0000","last_active":"2024-09-16T00:28:24.311816+0000","last_peered":"2024-09-16T00:28:24.311816+0000","last_clean":"2024-09-16T00:28:24.311816+0000","last_became_active":"2024-09-16T00:26:18.267442+0000","last_became_peered":"2024-09-16T00:26:18.267442+0000","last_unstale":"2024-09-16T00:28:24.311816+0000","last_undegraded":"2024-09-16T00:28:24.311816+0000","last_fullsized":"2024-09-16T00:28:24.311816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:19.421762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":324,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:51.682287+0000","last_change":"2024-09-16T00:26:25.315345+0000","last_active":"2024-09-16T00:28:51.682287+0000","last_peered":"2024-09-16T00:28:51.682287+0000","last_clean":"2024-09-16T00:28:51.682287+0000","last_became_active":"2024-09-16T00:26:25.300934+0000","last_became_peered":"2024-09-16T00:26:25.300934+0000","last_unstale":"2024-09-16T00:28:51.682287+0000","last_undegraded":"2024-09-16T00:28:51.682287+0000","last_fullsized":"2024-09-16T00:28:51.682287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:20.846309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68006,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":233,"num_read_kb":184,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359395+0000","last_change":"2024-09-16T00:27:26.358804+0000","last_active":"2024-09-16T00:27:26.359395+0000","last_peered":"2024-09-16T00:27:26.359395+0000","last_clean":"2024-09-16T00:27:26.359395+0000","last_became_active":"2024-09-16T00:27:26.358317+0000","last_became_peered":"2024-09-16T00:27:26.358317+0000","last_unstale":"2024-09-16T00:27:26.359395+0000","last_undegraded":"2024-09-16T00:27:26.359395+0000","last_fullsized":"2024-09-16T00:27:26.359395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:38.642392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336469+0000","last_change":"2024-09-16T00:26:18.278618+0000","last_active":"2024-09-16T00:28:24.336469+0000","last_peered":"2024-09-16T00:28:24.336469+0000","last_clean":"2024-09-16T00:28:24.336469+0000","last_became_active":"2024-09-16T00:26:18.278313+0000","last_became_peered":"2024-09-16T00:26:18.278313+0000","last_unstale":"2024-09-16T00:28:24.336469+0000","last_undegraded":"2024-09-16T00:28:24.336469+0000","last_fullsized":"2024-09-16T00:28:24.336469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:49.726218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356307+0000","last_change":"2024-09-16T00:26:16.205999+0000","last_active":"2024-09-16T00:27:26.356307+0000","last_peered":"2024-09-16T00:27:26.356307+0000","last_clean":"2024-09-16T00:27:26.356307+0000","last_became_active":"2024-09-16T00:26:16.205548+0000","last_became_peered":"2024-09-16T00:26:16.205548+0000","last_unstale":"2024-09-16T00:27:26.356307+0000","last_undegraded":"2024-09-16T00:27:26.356307+0000","last_fullsized":"2024-09-16T00:27:26.356307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:29.445656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.356802+0000","last_change":"2024-09-16T00:26:25.307923+0000","last_active":"2024-09-16T00:27:26.356802+0000","last_peered":"2024-09-16T00:27:26.356802+0000","last_clean":"2024-09-16T00:27:26.356802+0000","last_became_active":"2024-09-16T00:26:25.307504+0000","last_became_peered":"2024-09-16T00:26:25.307504+0000","last_unstale":"2024-09-16T00:27:26.356802+0000","last_undegraded":"2024-09-16T00:27:26.356802+0000","last_fullsized":"2024-09-16T00:27:26.356802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:44.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320577+0000","last_change":"2024-09-16T00:26:25.295744+0000","last_active":"2024-09-16T00:28:24.320577+0000","last_peered":"2024-09-16T00:28:24.320577+0000","last_clean":"2024-09-16T00:28:24.320577+0000","last_became_active":"2024-09-16T00:26:25.295380+0000","last_became_peered":"2024-09-16T00:26:25.295380+0000","last_unstale":"2024-09-16T00:28:24.320577+0000","last_undegraded":"2024-09-16T00:28:24.320577+0000","last_fullsized":"2024-09-16T00:28:24.320577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:14.777086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.357240+0000","last_change":"2024-09-16T00:26:16.146959+0000","last_active":"2024-09-16T00:27:26.357240+0000","last_peered":"2024-09-16T00:27:26.357240+0000","last_clean":"2024-09-16T00:27:26.357240+0000","last_became_active":"2024-09-16T00:26:16.146430+0000","last_became_peered":"2024-09-16T00:26:16.146430+0000","last_unstale":"2024-09-16T00:27:26.357240+0000","last_undegraded":"2024-09-16T00:27:26.357240+0000","last_fullsized":"2024-09-16T00:27:26.357240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:29.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.252178+0000","last_change":"2024-09-16T00:26:25.302127+0000","last_active":"2024-09-16T00:27:26.252178+0000","last_peered":"2024-09-16T00:27:26.252178+0000","last_clean":"2024-09-16T00:27:26.252178+0000","last_became_active":"2024-09-16T00:26:25.301734+0000","last_became_peered":"2024-09-16T00:26:25.301734+0000","last_unstale":"2024-09-16T00:27:26.252178+0000","last_undegraded":"2024-09-16T00:27:26.252178+0000","last_fullsized":"2024-09-16T00:27:26.252178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:38.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336948+0000","last_change":"2024-09-16T00:26:18.273272+0000","last_active":"2024-09-16T00:28:24.336948+0000","last_peered":"2024-09-16T00:28:24.336948+0000","last_clean":"2024-09-16T00:28:24.336948+0000","last_became_active":"2024-09-16T00:26:18.272910+0000","last_became_peered":"2024-09-16T00:26:18.272910+0000","last_unstale":"2024-09-16T00:28:24.336948+0000","last_undegraded":"2024-09-16T00:28:24.336948+0000","last_fullsized":"2024-09-16T00:28:24.336948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:18.512776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:50.680734+0000","last_change":"2024-09-16T00:26:16.147255+0000","last_active":"2024-09-16T00:28:50.680734+0000","last_peered":"2024-09-16T00:28:50.680734+0000","last_clean":"2024-09-16T00:28:50.680734+0000","last_became_active":"2024-09-16T00:26:16.146951+0000","last_became_peered":"2024-09-16T00:26:16.146951+0000","last_unstale":"2024-09-16T00:28:50.680734+0000","last_undegraded":"2024-09-16T00:28:50.680734+0000","last_fullsized":"2024-09-16T00:28:50.680734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:42.204525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.247326+0000","last_change":"2024-09-16T00:26:25.293051+0000","last_active":"2024-09-16T00:27:26.247326+0000","last_peered":"2024-09-16T00:27:26.247326+0000","last_clean":"2024-09-16T00:27:26.247326+0000","last_became_active":"2024-09-16T00:26:25.292661+0000","last_became_peered":"2024-09-16T00:26:25.292661+0000","last_unstale":"2024-09-16T00:27:26.247326+0000","last_undegraded":"2024-09-16T00:27:26.247326+0000","last_fullsized":"2024-09-16T00:27:26.247326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:51.752836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250057+0000","last_change":"2024-09-16T00:26:25.311632+0000","last_active":"2024-09-16T00:27:26.250057+0000","last_peered":"2024-09-16T00:27:26.250057+0000","last_clean":"2024-09-16T00:27:26.250057+0000","last_became_active":"2024-09-16T00:26:25.311259+0000","last_became_peered":"2024-09-16T00:26:25.311259+0000","last_unstale":"2024-09-16T00:27:26.250057+0000","last_undegraded":"2024-09-16T00:27:26.250057+0000","last_fullsized":"2024-09-16T00:27:26.250057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.965058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331405+0000","last_change":"2024-09-16T00:26:18.258970+0000","last_active":"2024-09-16T00:28:24.331405+0000","last_peered":"2024-09-16T00:28:24.331405+0000","last_clean":"2024-09-16T00:28:24.331405+0000","last_became_active":"2024-09-16T00:26:18.258365+0000","last_became_peered":"2024-09-16T00:26:18.258365+0000","last_unstale":"2024-09-16T00:28:24.331405+0000","last_undegraded":"2024-09-16T00:28:24.331405+0000","last_fullsized":"2024-09-16T00:28:24.331405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:03.487254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.355413+0000","last_change":"2024-09-16T00:26:16.147322+0000","last_active":"2024-09-16T00:27:26.355413+0000","last_peered":"2024-09-16T00:27:26.355413+0000","last_clean":"2024-09-16T00:27:26.355413+0000","last_became_active":"2024-09-16T00:26:16.146997+0000","last_became_peered":"2024-09-16T00:26:16.146997+0000","last_unstale":"2024-09-16T00:27:26.355413+0000","last_undegraded":"2024-09-16T00:27:26.355413+0000","last_fullsized":"2024-09-16T00:27:26.355413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:58.120550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.360047+0000","last_change":"2024-09-16T00:27:26.359230+0000","last_active":"2024-09-16T00:27:26.360047+0000","last_peered":"2024-09-16T00:27:26.360047+0000","last_clean":"2024-09-16T00:27:26.360047+0000","last_became_active":"2024-09-16T00:27:26.358662+0000","last_became_peered":"2024-09-16T00:27:26.358662+0000","last_unstale":"2024-09-16T00:27:26.360047+0000","last_undegraded":"2024-09-16T00:27:26.360047+0000","last_fullsized":"2024-09-16T00:27:26.360047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:52.111910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.331664+0000","last_change":"2024-09-16T00:26:18.262571+0000","last_active":"2024-09-16T00:28:24.331664+0000","last_peered":"2024-09-16T00:28:24.331664+0000","last_clean":"2024-09-16T00:28:24.331664+0000","last_became_active":"2024-09-16T00:26:18.262275+0000","last_became_peered":"2024-09-16T00:26:18.262275+0000","last_unstale":"2024-09-16T00:28:24.331664+0000","last_undegraded":"2024-09-16T00:28:24.331664+0000","last_fullsized":"2024-09-16T00:28:24.331664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:12.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250512+0000","last_change":"2024-09-16T00:26:16.137660+0000","last_active":"2024-09-16T00:27:26.250512+0000","last_peered":"2024-09-16T00:27:26.250512+0000","last_clean":"2024-09-16T00:27:26.250512+0000","last_became_active":"2024-09-16T00:26:16.137298+0000","last_became_peered":"2024-09-16T00:26:16.137298+0000","last_unstale":"2024-09-16T00:27:26.250512+0000","last_undegraded":"2024-09-16T00:27:26.250512+0000","last_fullsized":"2024-09-16T00:27:26.250512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.795772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.250964+0000","last_change":"2024-09-16T00:26:16.149088+0000","last_active":"2024-09-16T00:27:26.250964+0000","last_peered":"2024-09-16T00:27:26.250964+0000","last_clean":"2024-09-16T00:27:26.250964+0000","last_became_active":"2024-09-16T00:26:16.148673+0000","last_became_peered":"2024-09-16T00:26:16.148673+0000","last_unstale":"2024-09-16T00:27:26.250964+0000","last_undegraded":"2024-09-16T00:27:26.250964+0000","last_fullsized":"2024-09-16T00:27:26.250964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:09.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334137+0000","last_change":"2024-09-16T00:26:18.261300+0000","last_active":"2024-09-16T00:28:24.334137+0000","last_peered":"2024-09-16T00:28:24.334137+0000","last_clean":"2024-09-16T00:28:24.334137+0000","last_became_active":"2024-09-16T00:26:18.260949+0000","last_became_peered":"2024-09-16T00:26:18.260949+0000","last_unstale":"2024-09-16T00:28:24.334137+0000","last_undegraded":"2024-09-16T00:28:24.334137+0000","last_fullsized":"2024-09-16T00:28:24.334137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:30.204087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.359360+0000","last_change":"2024-09-16T00:27:26.359360+0000","last_active":"2024-09-16T00:27:26.359360+0000","last_peered":"2024-09-16T00:27:26.359360+0000","last_clean":"2024-09-16T00:27:26.359360+0000","last_became_active":"2024-09-16T00:27:26.358462+0000","last_became_peered":"2024-09-16T00:27:26.358462+0000","last_unstale":"2024-09-16T00:27:26.359360+0000","last_undegraded":"2024-09-16T00:27:26.359360+0000","last_fullsized":"2024-09-16T00:27:26.359360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:38.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":163,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:53.685880+0000","last_change":"2024-09-16T00:26:16.146366+0000","last_active":"2024-09-16T00:28:53.685880+0000","last_peered":"2024-09-16T00:28:53.685880+0000","last_clean":"2024-09-16T00:28:53.685880+0000","last_became_active":"2024-09-16T00:26:16.146017+0000","last_became_peered":"2024-09-16T00:26:16.146017+0000","last_unstale":"2024-09-16T00:28:53.685880+0000","last_undegraded":"2024-09-16T00:28:53.685880+0000","last_fullsized":"2024-09-16T00:28:53.685880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:15.123167+0000","last_clean_scrub_stamp":"2024-09-16T00:26:15.123167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:17.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320920+0000","last_change":"2024-09-16T00:26:18.263575+0000","last_active":"2024-09-16T00:28:24.320920+0000","last_peered":"2024-09-16T00:28:24.320920+0000","last_clean":"2024-09-16T00:28:24.320920+0000","last_became_active":"2024-09-16T00:26:18.263242+0000","last_became_peered":"2024-09-16T00:26:18.263242+0000","last_unstale":"2024-09-16T00:28:24.320920+0000","last_undegraded":"2024-09-16T00:28:24.320920+0000","last_fullsized":"2024-09-16T00:28:24.320920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:41.316049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:27:26.248757+0000","last_change":"2024-09-16T00:26:25.300823+0000","last_active":"2024-09-16T00:27:26.248757+0000","last_peered":"2024-09-16T00:27:26.248757+0000","last_clean":"2024-09-16T00:27:26.248757+0000","last_became_active":"2024-09-16T00:26:25.300463+0000","last_became_peered":"2024-09-16T00:26:25.300463+0000","last_unstale":"2024-09-16T00:27:26.248757+0000","last_undegraded":"2024-09-16T00:27:26.248757+0000","last_fullsized":"2024-09-16T00:27:26.248757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:24.277907+0000","last_clean_scrub_stamp":"2024-09-16T00:26:24.277907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:29.482857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321717+0000","last_change":"2024-09-16T00:26:18.266255+0000","last_active":"2024-09-16T00:28:24.321717+0000","last_peered":"2024-09-16T00:28:24.321717+0000","last_clean":"2024-09-16T00:28:24.321717+0000","last_became_active":"2024-09-16T00:26:18.265010+0000","last_became_peered":"2024-09-16T00:26:18.265010+0000","last_unstale":"2024-09-16T00:28:24.321717+0000","last_undegraded":"2024-09-16T00:28:24.321717+0000","last_fullsized":"2024-09-16T00:28:24.321717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:29.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.321331+0000","last_change":"2024-09-16T00:26:25.299325+0000","last_active":"2024-09-16T00:28:24.321331+0000","last_peered":"2024-09-16T00:28:24.321331+0000","last_clean":"2024-09-16T00:28:24.321331+0000","last_became_active":"2024-09-16T00:26:25.298934+0000","last_became_peered":"2024-09-16T00:26:25.298934+0000","last_unstale":"2024-09-16T00:28:24.321331+0000","last_undegraded":"2024-09-16T00:28:24.321331+0000","last_fullsized":"2024-09-16T00:28:24.321331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:07.985334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329213+0000","last_change":"2024-09-16T00:26:18.272763+0000","last_active":"2024-09-16T00:28:24.329213+0000","last_peered":"2024-09-16T00:28:24.329213+0000","last_clean":"2024-09-16T00:28:24.329213+0000","last_became_active":"2024-09-16T00:26:18.272446+0000","last_became_peered":"2024-09-16T00:26:18.272446+0000","last_unstale":"2024-09-16T00:28:24.329213+0000","last_undegraded":"2024-09-16T00:28:24.329213+0000","last_fullsized":"2024-09-16T00:28:24.329213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:17.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326265+0000","last_change":"2024-09-16T00:26:18.264369+0000","last_active":"2024-09-16T00:28:24.326265+0000","last_peered":"2024-09-16T00:28:24.326265+0000","last_clean":"2024-09-16T00:28:24.326265+0000","last_became_active":"2024-09-16T00:26:18.263897+0000","last_became_peered":"2024-09-16T00:26:18.263897+0000","last_unstale":"2024-09-16T00:28:24.326265+0000","last_undegraded":"2024-09-16T00:28:24.326265+0000","last_fullsized":"2024-09-16T00:28:24.326265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.776868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.327220+0000","last_change":"2024-09-16T00:26:18.261641+0000","last_active":"2024-09-16T00:28:24.327220+0000","last_peered":"2024-09-16T00:28:24.327220+0000","last_clean":"2024-09-16T00:28:24.327220+0000","last_became_active":"2024-09-16T00:26:18.261336+0000","last_became_peered":"2024-09-16T00:26:18.261336+0000","last_unstale":"2024-09-16T00:28:24.327220+0000","last_undegraded":"2024-09-16T00:28:24.327220+0000","last_fullsized":"2024-09-16T00:28:24.327220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:16.772395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.316594+0000","last_change":"2024-09-16T00:26:18.269266+0000","last_active":"2024-09-16T00:28:24.316594+0000","last_peered":"2024-09-16T00:28:24.316594+0000","last_clean":"2024-09-16T00:28:24.316594+0000","last_became_active":"2024-09-16T00:26:18.268937+0000","last_became_peered":"2024-09-16T00:26:18.268937+0000","last_unstale":"2024-09-16T00:28:24.316594+0000","last_undegraded":"2024-09-16T00:28:24.316594+0000","last_fullsized":"2024-09-16T00:28:24.316594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:50.612615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337588+0000","last_change":"2024-09-16T00:26:18.274276+0000","last_active":"2024-09-16T00:28:24.337588+0000","last_peered":"2024-09-16T00:28:24.337588+0000","last_clean":"2024-09-16T00:28:24.337588+0000","last_became_active":"2024-09-16T00:26:18.273851+0000","last_became_peered":"2024-09-16T00:26:18.273851+0000","last_unstale":"2024-09-16T00:28:24.337588+0000","last_undegraded":"2024-09-16T00:28:24.337588+0000","last_fullsized":"2024-09-16T00:28:24.337588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:48.465995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322996+0000","last_change":"2024-09-16T00:26:18.266270+0000","last_active":"2024-09-16T00:28:24.322996+0000","last_peered":"2024-09-16T00:28:24.322996+0000","last_clean":"2024-09-16T00:28:24.322996+0000","last_became_active":"2024-09-16T00:26:18.265952+0000","last_became_peered":"2024-09-16T00:26:18.265952+0000","last_unstale":"2024-09-16T00:28:24.322996+0000","last_undegraded":"2024-09-16T00:28:24.322996+0000","last_fullsized":"2024-09-16T00:28:24.322996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:25.735415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322343+0000","last_change":"2024-09-16T00:26:18.260048+0000","last_active":"2024-09-16T00:28:24.322343+0000","last_peered":"2024-09-16T00:28:24.322343+0000","last_clean":"2024-09-16T00:28:24.322343+0000","last_became_active":"2024-09-16T00:26:18.259610+0000","last_became_peered":"2024-09-16T00:26:18.259610+0000","last_unstale":"2024-09-16T00:28:24.322343+0000","last_undegraded":"2024-09-16T00:28:24.322343+0000","last_fullsized":"2024-09-16T00:28:24.322343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:57.360912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330356+0000","last_change":"2024-09-16T00:26:18.269375+0000","last_active":"2024-09-16T00:28:24.330356+0000","last_peered":"2024-09-16T00:28:24.330356+0000","last_clean":"2024-09-16T00:28:24.330356+0000","last_became_active":"2024-09-16T00:26:18.269069+0000","last_became_peered":"2024-09-16T00:26:18.269069+0000","last_unstale":"2024-09-16T00:28:24.330356+0000","last_undegraded":"2024-09-16T00:28:24.330356+0000","last_fullsized":"2024-09-16T00:28:24.330356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.284605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337238+0000","last_change":"2024-09-16T00:26:18.270671+0000","last_active":"2024-09-16T00:28:24.337238+0000","last_peered":"2024-09-16T00:28:24.337238+0000","last_clean":"2024-09-16T00:28:24.337238+0000","last_became_active":"2024-09-16T00:26:18.270304+0000","last_became_peered":"2024-09-16T00:26:18.270304+0000","last_unstale":"2024-09-16T00:28:24.337238+0000","last_undegraded":"2024-09-16T00:28:24.337238+0000","last_fullsized":"2024-09-16T00:28:24.337238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:27.503802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.330609+0000","last_change":"2024-09-16T00:26:18.265107+0000","last_active":"2024-09-16T00:28:24.330609+0000","last_peered":"2024-09-16T00:28:24.330609+0000","last_clean":"2024-09-16T00:28:24.330609+0000","last_became_active":"2024-09-16T00:26:18.264753+0000","last_became_peered":"2024-09-16T00:26:18.264753+0000","last_unstale":"2024-09-16T00:28:24.330609+0000","last_undegraded":"2024-09-16T00:28:24.330609+0000","last_fullsized":"2024-09-16T00:28:24.330609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:44.010768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.317884+0000","last_change":"2024-09-16T00:26:18.269023+0000","last_active":"2024-09-16T00:28:24.317884+0000","last_peered":"2024-09-16T00:28:24.317884+0000","last_clean":"2024-09-16T00:28:24.317884+0000","last_became_active":"2024-09-16T00:26:18.267839+0000","last_became_peered":"2024-09-16T00:26:18.267839+0000","last_unstale":"2024-09-16T00:28:24.317884+0000","last_undegraded":"2024-09-16T00:28:24.317884+0000","last_fullsized":"2024-09-16T00:28:24.317884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:59.946280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.329300+0000","last_change":"2024-09-16T00:26:18.273746+0000","last_active":"2024-09-16T00:28:24.329300+0000","last_peered":"2024-09-16T00:28:24.329300+0000","last_clean":"2024-09-16T00:28:24.329300+0000","last_became_active":"2024-09-16T00:26:18.273358+0000","last_became_peered":"2024-09-16T00:26:18.273358+0000","last_unstale":"2024-09-16T00:28:24.329300+0000","last_undegraded":"2024-09-16T00:28:24.329300+0000","last_fullsized":"2024-09-16T00:28:24.329300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:29.515520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337159+0000","last_change":"2024-09-16T00:26:18.274161+0000","last_active":"2024-09-16T00:28:24.337159+0000","last_peered":"2024-09-16T00:28:24.337159+0000","last_clean":"2024-09-16T00:28:24.337159+0000","last_became_active":"2024-09-16T00:26:18.273838+0000","last_became_peered":"2024-09-16T00:26:18.273838+0000","last_unstale":"2024-09-16T00:28:24.337159+0000","last_undegraded":"2024-09-16T00:28:24.337159+0000","last_fullsized":"2024-09-16T00:28:24.337159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:10.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336933+0000","last_change":"2024-09-16T00:26:18.279673+0000","last_active":"2024-09-16T00:28:24.336933+0000","last_peered":"2024-09-16T00:28:24.336933+0000","last_clean":"2024-09-16T00:28:24.336933+0000","last_became_active":"2024-09-16T00:26:18.279293+0000","last_became_peered":"2024-09-16T00:26:18.279293+0000","last_unstale":"2024-09-16T00:28:24.336933+0000","last_undegraded":"2024-09-16T00:28:24.336933+0000","last_fullsized":"2024-09-16T00:28:24.336933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:56.172762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.337483+0000","last_change":"2024-09-16T00:26:18.277068+0000","last_active":"2024-09-16T00:28:24.337483+0000","last_peered":"2024-09-16T00:28:24.337483+0000","last_clean":"2024-09-16T00:28:24.337483+0000","last_became_active":"2024-09-16T00:26:18.276678+0000","last_became_peered":"2024-09-16T00:26:18.276678+0000","last_unstale":"2024-09-16T00:28:24.337483+0000","last_undegraded":"2024-09-16T00:28:24.337483+0000","last_fullsized":"2024-09-16T00:28:24.337483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:30.818573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326582+0000","last_change":"2024-09-16T00:26:18.270496+0000","last_active":"2024-09-16T00:28:24.326582+0000","last_peered":"2024-09-16T00:28:24.326582+0000","last_clean":"2024-09-16T00:28:24.326582+0000","last_became_active":"2024-09-16T00:26:18.270170+0000","last_became_peered":"2024-09-16T00:26:18.270170+0000","last_unstale":"2024-09-16T00:28:24.326582+0000","last_undegraded":"2024-09-16T00:28:24.326582+0000","last_fullsized":"2024-09-16T00:28:24.326582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:22.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326496+0000","last_change":"2024-09-16T00:26:18.270352+0000","last_active":"2024-09-16T00:28:24.326496+0000","last_peered":"2024-09-16T00:28:24.326496+0000","last_clean":"2024-09-16T00:28:24.326496+0000","last_became_active":"2024-09-16T00:26:18.270003+0000","last_became_peered":"2024-09-16T00:26:18.270003+0000","last_unstale":"2024-09-16T00:28:24.326496+0000","last_undegraded":"2024-09-16T00:28:24.326496+0000","last_fullsized":"2024-09-16T00:28:24.326496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:42.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326067+0000","last_change":"2024-09-16T00:26:18.258426+0000","last_active":"2024-09-16T00:28:24.326067+0000","last_peered":"2024-09-16T00:28:24.326067+0000","last_clean":"2024-09-16T00:28:24.326067+0000","last_became_active":"2024-09-16T00:26:18.256221+0000","last_became_peered":"2024-09-16T00:26:18.256221+0000","last_unstale":"2024-09-16T00:28:24.326067+0000","last_undegraded":"2024-09-16T00:28:24.326067+0000","last_fullsized":"2024-09-16T00:28:24.326067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:46.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320083+0000","last_change":"2024-09-16T00:26:18.267206+0000","last_active":"2024-09-16T00:28:24.320083+0000","last_peered":"2024-09-16T00:28:24.320083+0000","last_clean":"2024-09-16T00:28:24.320083+0000","last_became_active":"2024-09-16T00:26:18.266251+0000","last_became_peered":"2024-09-16T00:26:18.266251+0000","last_unstale":"2024-09-16T00:28:24.320083+0000","last_undegraded":"2024-09-16T00:28:24.320083+0000","last_fullsized":"2024-09-16T00:28:24.320083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:41.941279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.338288+0000","last_change":"2024-09-16T00:26:18.275671+0000","last_active":"2024-09-16T00:28:24.338288+0000","last_peered":"2024-09-16T00:28:24.338288+0000","last_clean":"2024-09-16T00:28:24.338288+0000","last_became_active":"2024-09-16T00:26:18.275327+0000","last_became_peered":"2024-09-16T00:26:18.275327+0000","last_unstale":"2024-09-16T00:28:24.338288+0000","last_undegraded":"2024-09-16T00:28:24.338288+0000","last_fullsized":"2024-09-16T00:28:24.338288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:55.613715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.328897+0000","last_change":"2024-09-16T00:26:25.297426+0000","last_active":"2024-09-16T00:28:24.328897+0000","last_peered":"2024-09-16T00:28:24.328897+0000","last_clean":"2024-09-16T00:28:24.328897+0000","last_became_active":"2024-09-16T00:26:25.296910+0000","last_became_peered":"2024-09-16T00:26:25.296910+0000","last_unstale":"2024-09-16T00:28:24.328897+0000","last_undegraded":"2024-09-16T00:28:24.328897+0000","last_fullsized":"2024-09-16T00:28:24.328897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:18.529603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334467+0000","last_change":"2024-09-16T00:26:25.310414+0000","last_active":"2024-09-16T00:28:24.334467+0000","last_peered":"2024-09-16T00:28:24.334467+0000","last_clean":"2024-09-16T00:28:24.334467+0000","last_became_active":"2024-09-16T00:26:25.309994+0000","last_became_peered":"2024-09-16T00:26:25.309994+0000","last_unstale":"2024-09-16T00:28:24.334467+0000","last_undegraded":"2024-09-16T00:28:24.334467+0000","last_fullsized":"2024-09-16T00:28:24.334467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:31.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.325443+0000","last_change":"2024-09-16T00:26:25.309369+0000","last_active":"2024-09-16T00:28:24.325443+0000","last_peered":"2024-09-16T00:28:24.325443+0000","last_clean":"2024-09-16T00:28:24.325443+0000","last_became_active":"2024-09-16T00:26:25.308734+0000","last_became_peered":"2024-09-16T00:26:25.308734+0000","last_unstale":"2024-09-16T00:28:24.325443+0000","last_undegraded":"2024-09-16T00:28:24.325443+0000","last_fullsized":"2024-09-16T00:28:24.325443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:12.562052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.334306+0000","last_change":"2024-09-16T00:26:18.267860+0000","last_active":"2024-09-16T00:28:24.334306+0000","last_peered":"2024-09-16T00:28:24.334306+0000","last_clean":"2024-09-16T00:28:24.334306+0000","last_became_active":"2024-09-16T00:26:18.267516+0000","last_became_peered":"2024-09-16T00:26:18.267516+0000","last_unstale":"2024-09-16T00:28:24.334306+0000","last_undegraded":"2024-09-16T00:28:24.334306+0000","last_fullsized":"2024-09-16T00:28:24.334306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:10.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.328201+0000","last_change":"2024-09-16T00:26:18.265002+0000","last_active":"2024-09-16T00:28:24.328201+0000","last_peered":"2024-09-16T00:28:24.328201+0000","last_clean":"2024-09-16T00:28:24.328201+0000","last_became_active":"2024-09-16T00:26:18.264665+0000","last_became_peered":"2024-09-16T00:26:18.264665+0000","last_unstale":"2024-09-16T00:28:24.328201+0000","last_undegraded":"2024-09-16T00:28:24.328201+0000","last_fullsized":"2024-09-16T00:28:24.328201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:26.693034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.322132+0000","last_change":"2024-09-16T00:26:18.265583+0000","last_active":"2024-09-16T00:28:24.322132+0000","last_peered":"2024-09-16T00:28:24.322132+0000","last_clean":"2024-09-16T00:28:24.322132+0000","last_became_active":"2024-09-16T00:26:18.265271+0000","last_became_peered":"2024-09-16T00:26:18.265271+0000","last_unstale":"2024-09-16T00:28:24.322132+0000","last_undegraded":"2024-09-16T00:28:24.322132+0000","last_fullsized":"2024-09-16T00:28:24.322132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.173843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.320484+0000","last_change":"2024-09-16T00:26:18.267451+0000","last_active":"2024-09-16T00:28:24.320484+0000","last_peered":"2024-09-16T00:28:24.320484+0000","last_clean":"2024-09-16T00:28:24.320484+0000","last_became_active":"2024-09-16T00:26:18.266934+0000","last_became_peered":"2024-09-16T00:26:18.266934+0000","last_unstale":"2024-09-16T00:28:24.320484+0000","last_undegraded":"2024-09-16T00:28:24.320484+0000","last_fullsized":"2024-09-16T00:28:24.320484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:07.465601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.318969+0000","last_change":"2024-09-16T00:27:26.358604+0000","last_active":"2024-09-16T00:28:24.318969+0000","last_peered":"2024-09-16T00:28:24.318969+0000","last_clean":"2024-09-16T00:28:24.318969+0000","last_became_active":"2024-09-16T00:27:26.358225+0000","last_became_peered":"2024-09-16T00:27:26.358225+0000","last_unstale":"2024-09-16T00:28:24.318969+0000","last_undegraded":"2024-09-16T00:28:24.318969+0000","last_fullsized":"2024-09-16T00:28:24.318969+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:21.806500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.326711+0000","last_change":"2024-09-16T00:26:18.269543+0000","last_active":"2024-09-16T00:28:24.326711+0000","last_peered":"2024-09-16T00:28:24.326711+0000","last_clean":"2024-09-16T00:28:24.326711+0000","last_became_active":"2024-09-16T00:26:18.269257+0000","last_became_peered":"2024-09-16T00:26:18.269257+0000","last_unstale":"2024-09-16T00:28:24.326711+0000","last_undegraded":"2024-09-16T00:28:24.326711+0000","last_fullsized":"2024-09-16T00:28:24.326711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:22.166303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:28:24.336940+0000","last_change":"2024-09-16T00:26:18.279632+0000","last_active":"2024-09-16T00:28:24.336940+0000","last_peered":"2024-09-16T00:28:24.336940+0000","last_clean":"2024-09-16T00:28:24.336940+0000","last_became_active":"2024-09-16T00:26:18.279343+0000","last_became_peered":"2024-09-16T00:26:18.279343+0000","last_unstale":"2024-09-16T00:28:24.336940+0000","last_undegraded":"2024-09-16T00:28:24.336940+0000","last_fullsized":"2024-09-16T00:28:24.336940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:26:17.242724+0000","last_clean_scrub_stamp":"2024-09-16T00:26:17.242724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:21.188265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72458,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":705,"num_read_kb":430,"num_write":180,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":217374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514541,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":515002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743474,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1732608,"data_stored":1170276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907000,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":519334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":515158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364865,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1646592,"data_stored":1111518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528391,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1107106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953553,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":583070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":510608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":512516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":578330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":511802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:28:57.126 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:57.814 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:57 smithi001 ceph-mon[30934]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:28:57.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:57 smithi001 ceph-mon[30934]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 -- 172.21.15.1:0/2435577799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc100e20 msgr2=0x7f71cc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 --2- 172.21.15.1:0/2435577799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc100e20 0x7f71cc101200 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009900 tx=0x7f71bc02a480 comp rx=0 tx=0).stop 2024-09-16T00:28:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 -- 172.21.15.1:0/2435577799 shutdown_connections 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 --2- 172.21.15.1:0/2435577799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71cc102ef0 0x7f71cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 --2- 172.21.15.1:0/2435577799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71cc102570 0x7f71cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 --2- 172.21.15.1:0/2435577799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc100e20 0x7f71cc101200 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 -- 172.21.15.1:0/2435577799 >> 172.21.15.1:0/2435577799 conn(0x7f71cc0fc7d0 msgr2=0x7f71cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 -- 172.21.15.1:0/2435577799 shutdown_connections 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 -- 172.21.15.1:0/2435577799 wait complete. 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.918+0000 7f71d2118640 1 Processor -- start 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71d2118640 1 -- start start 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71d2118640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71cc100e20 0x7f71cc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71d2118640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc102570 0x7f71cc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71d2118640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71cc102ef0 0x7f71cc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71d2118640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc110fd0 con 0x7f71cc102570 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71d2118640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc110e50 con 0x7f71cc100e20 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71d2118640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc111150 con 0x7f71cc102ef0 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71caffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc102570 0x7f71cc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71caffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc102570 0x7f71cc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38624/0 (socket says 172.21.15.1:38624) 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71caffd640 1 -- 172.21.15.1:0/548097152 learned_addr learned my addr 172.21.15.1:0/548097152 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71cbfff640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71cc102ef0 0x7f71cc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71cb7fe640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71cc100e20 0x7f71cc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71caffd640 1 -- 172.21.15.1:0/548097152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71cc102ef0 msgr2=0x7f71cc19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71caffd640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71cc102ef0 0x7f71cc19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71caffd640 1 -- 172.21.15.1:0/548097152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71cc100e20 msgr2=0x7f71cc198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71caffd640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71cc100e20 0x7f71cc198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71caffd640 1 -- 172.21.15.1:0/548097152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc19d400 con 0x7f71cc102570 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71cbfff640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71cc102ef0 0x7f71cc19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.919+0000 7f71cb7fe640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71cc100e20 0x7f71cc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.920+0000 7f71caffd640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc102570 0x7f71cc198990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f71c000c920 tx=0x7f71c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.920+0000 7f71c8ff9640 1 -- 172.21.15.1:0/548097152 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0007c70 con 0x7f71cc102570 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.920+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc19d6f0 con 0x7f71cc102570 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.920+0000 7f71c8ff9640 1 -- 172.21.15.1:0/548097152 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c0007e10 con 0x7f71cc102570 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.920+0000 7f71c8ff9640 1 -- 172.21.15.1:0/548097152 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0018890 con 0x7f71cc102570 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.920+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1a4fd0 con 0x7f71cc102570 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.921+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc101200 con 0x7f71cc102570 2024-09-16T00:28:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.922+0000 7f71c8ff9640 1 -- 172.21.15.1:0/548097152 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f71c0018a70 con 0x7f71cc102570 2024-09-16T00:28:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.923+0000 7f71c8ff9640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71b40a3240 0x7f71b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.923+0000 7f71cb7fe640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71b40a3240 0x7f71b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.923+0000 7f71c8ff9640 1 -- 172.21.15.1:0/548097152 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f71c00cff10 con 0x7f71cc102570 2024-09-16T00:28:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.923+0000 7f71cb7fe640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71b40a3240 0x7f71b40a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:57.927+0000 7f71c8ff9640 1 -- 172.21.15.1:0/548097152 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71c0014030 con 0x7f71cc102570 2024-09-16T00:28:57.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:57 smithi153 ceph-mon[35538]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:28:57.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:57 smithi153 ceph-mon[35538]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:58.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:57 smithi019 ceph-mon[35186]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:28:58.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:57 smithi019 ceph-mon[35186]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:28:58.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.137+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71cc10a0a0 con 0x7f71cc102570 2024-09-16T00:28:58.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.138+0000 7f71c8ff9640 1 -- 172.21.15.1:0/548097152 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f71c000de20 con 0x7f71cc102570 2024-09-16T00:28:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:58.139 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:28:58.139 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:28:58.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71b40a3240 msgr2=0x7f71b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:58.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71b40a3240 0x7f71b40a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc005a40 comp rx=0 tx=0).stop 2024-09-16T00:28:58.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc102570 msgr2=0x7f71cc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:58.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc102570 0x7f71cc198990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f71c000c920 tx=0x7f71c000cde0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 shutdown_connections 2024-09-16T00:28:58.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f71b40a3240 0x7f71b40a5700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71cc102ef0 0x7f71cc19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71cc102570 0x7f71cc198990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 --2- 172.21.15.1:0/548097152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71cc100e20 0x7f71cc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 >> 172.21.15.1:0/548097152 conn(0x7f71cc0fc7d0 msgr2=0x7f71cc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:58.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 shutdown_connections 2024-09-16T00:28:58.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:58.141+0000 7f71d2118640 1 -- 172.21.15.1:0/548097152 wait complete. 2024-09-16T00:28:58.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:58 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/548097152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:58.846 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:59.104 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:59.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:58 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/548097152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:59.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:58 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/548097152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:59.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.610+0000 7f8a2fd34640 1 -- 172.21.15.1:0/3411647941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 msgr2=0x7f8a2810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:59.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.610+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/3411647941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 0x7f8a2810d940 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009cd0 tx=0x7f8a2002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.610+0000 7f8a2fd34640 1 -- 172.21.15.1:0/3411647941 shutdown_connections 2024-09-16T00:28:59.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.610+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/3411647941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 0x7f8a2810d940 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.610+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/3411647941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a28100a70 0x7f8a28100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.610+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/3411647941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28106a70 0x7f8a28106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.610+0000 7f8a2fd34640 1 -- 172.21.15.1:0/3411647941 >> 172.21.15.1:0/3411647941 conn(0x7f8a280fc790 msgr2=0x7f8a280febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.610+0000 7f8a2fd34640 1 -- 172.21.15.1:0/3411647941 shutdown_connections 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.610+0000 7f8a2fd34640 1 -- 172.21.15.1:0/3411647941 wait complete. 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2fd34640 1 Processor -- start 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2fd34640 1 -- start start 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2fd34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28100a70 0x7f8a281992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2fd34640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 0x7f8a281997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2fd34640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a28106a70 0x7f8a281a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2fd34640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28110e70 con 0x7f8a28101410 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2fd34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28110cf0 con 0x7f8a28100a70 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2fd34640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28110ff0 con 0x7f8a28106a70 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2d2cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 0x7f8a281997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2d2cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 0x7f8a281997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38644/0 (socket says 172.21.15.1:38644) 2024-09-16T00:28:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.611+0000 7f8a2d2cf640 1 -- 172.21.15.1:0/829120797 learned_addr learned my addr 172.21.15.1:0/829120797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2dad0640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28100a70 0x7f8a281992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2d2cf640 1 -- 172.21.15.1:0/829120797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a28106a70 msgr2=0x7f8a281a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2d2cf640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a28106a70 0x7f8a281a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2d2cf640 1 -- 172.21.15.1:0/829120797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28100a70 msgr2=0x7f8a281992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2d2cf640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28100a70 0x7f8a281992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2d2cf640 1 -- 172.21.15.1:0/829120797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a281a4870 con 0x7f8a28101410 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2dad0640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28100a70 0x7f8a281992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2d2cf640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 0x7f8a281997f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800d770 tx=0x7f8a1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a1effd640 1 -- 172.21.15.1:0/829120797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18017070 con 0x7f8a28101410 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a1effd640 1 -- 172.21.15.1:0/829120797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a18005500 con 0x7f8a28101410 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a281a4a40 con 0x7f8a28101410 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a1effd640 1 -- 172.21.15.1:0/829120797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18004e00 con 0x7f8a28101410 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.612+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a281a4f00 con 0x7f8a28101410 2024-09-16T00:28:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.613+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a281021b0 con 0x7f8a28101410 2024-09-16T00:28:59.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.614+0000 7f8a1effd640 1 -- 172.21.15.1:0/829120797 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8a18004fa0 con 0x7f8a28101410 2024-09-16T00:28:59.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.615+0000 7f8a1effd640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a140a3240 0x7f8a140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:59.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.615+0000 7f8a2dad0640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a140a3240 0x7f8a140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:59.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.615+0000 7f8a1effd640 1 -- 172.21.15.1:0/829120797 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8a180cfc70 con 0x7f8a28101410 2024-09-16T00:28:59.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.615+0000 7f8a2dad0640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a140a3240 0x7f8a140a5700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a24005fd0 tx=0x7f8a240094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:59.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.618+0000 7f8a1effd640 1 -- 172.21.15.1:0/829120797 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a1809c590 con 0x7f8a28101410 2024-09-16T00:28:59.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.831+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a28108fe0 con 0x7f8a28101410 2024-09-16T00:28:59.832 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:28:59 smithi001 ceph-mon[30934]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:28:59.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.832+0000 7f8a1effd640 1 -- 172.21.15.1:0/829120797 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f8a1809be90 con 0x7f8a28101410 2024-09-16T00:28:59.832 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:28:59.833 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:28:59.833 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a140a3240 msgr2=0x7f8a140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a140a3240 0x7f8a140a5700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a24005fd0 tx=0x7f8a240094d0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 msgr2=0x7f8a281997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 0x7f8a281997f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800d770 tx=0x7f8a1800dc30 comp rx=0 tx=0).stop 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 shutdown_connections 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a140a3240 0x7f8a140a5700 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a28106a70 0x7f8a281a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a28101410 0x7f8a281997f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 --2- 172.21.15.1:0/829120797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28100a70 0x7f8a281992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 >> 172.21.15.1:0/829120797 conn(0x7f8a280fc790 msgr2=0x7f8a280fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 shutdown_connections 2024-09-16T00:28:59.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:28:59.834+0000 7f8a2fd34640 1 -- 172.21.15.1:0/829120797 wait complete. 2024-09-16T00:29:00.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:28:59 smithi019 ceph-mon[35186]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:29:00.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:28:59 smithi153 ceph-mon[35538]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:29:00.538 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:29:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:00.917 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:00 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/829120797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:01.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:00 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/829120797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:01.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:00 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/829120797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:01.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.334+0000 7f22b3f2c640 1 -- 172.21.15.1:0/2449096824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 msgr2=0x7f22ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:01.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.334+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/2449096824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 0x7f22ac10ea90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009900 tx=0x7f22a402a410 comp rx=0 tx=0).stop 2024-09-16T00:29:01.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.334+0000 7f22b3f2c640 1 -- 172.21.15.1:0/2449096824 shutdown_connections 2024-09-16T00:29:01.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.334+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/2449096824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 0x7f22ac10ea90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.334+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/2449096824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22ac102790 0x7f22ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.334+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/2449096824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22ac101de0 0x7f22ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.334+0000 7f22b3f2c640 1 -- 172.21.15.1:0/2449096824 >> 172.21.15.1:0/2449096824 conn(0x7f22ac0fd8c0 msgr2=0x7f22ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:01.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.334+0000 7f22b3f2c640 1 -- 172.21.15.1:0/2449096824 shutdown_connections 2024-09-16T00:29:01.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.334+0000 7f22b3f2c640 1 -- 172.21.15.1:0/2449096824 wait complete. 2024-09-16T00:29:01.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b3f2c640 1 Processor -- start 2024-09-16T00:29:01.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b3f2c640 1 -- start start 2024-09-16T00:29:01.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b3f2c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22ac101de0 0x7f22ac1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:01.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b3f2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22ac102790 0x7f22ac198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:01.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b3f2c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 0x7f22ac19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:01.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b3f2c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac111f70 con 0x7f22ac106f10 2024-09-16T00:29:01.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b3f2c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac111df0 con 0x7f22ac102790 2024-09-16T00:29:01.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b3f2c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1120f0 con 0x7f22ac101de0 2024-09-16T00:29:01.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b24c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 0x7f22ac19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:01.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b24c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 0x7f22ac19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54252/0 (socket says 172.21.15.1:54252) 2024-09-16T00:29:01.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b24c9640 1 -- 172.21.15.1:0/1017250218 learned_addr learned my addr 172.21.15.1:0/1017250218 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:01.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b14c7640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22ac102790 0x7f22ac198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:01.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b1cc8640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22ac101de0 0x7f22ac1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:01.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b24c9640 1 -- 172.21.15.1:0/1017250218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22ac101de0 msgr2=0x7f22ac1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:01.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b24c9640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22ac101de0 0x7f22ac1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b24c9640 1 -- 172.21.15.1:0/1017250218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22ac102790 msgr2=0x7f22ac198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:01.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b24c9640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22ac102790 0x7f22ac198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.335+0000 7f22b24c9640 1 -- 172.21.15.1:0/1017250218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac19d380 con 0x7f22ac106f10 2024-09-16T00:29:01.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.336+0000 7f22b24c9640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 0x7f22ac19cca0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:01.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.336+0000 7f22aaffd640 1 -- 172.21.15.1:0/1017250218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a403f030 con 0x7f22ac106f10 2024-09-16T00:29:01.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.336+0000 7f22aaffd640 1 -- 172.21.15.1:0/1017250218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4003ad0 con 0x7f22ac106f10 2024-09-16T00:29:01.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.336+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ac19d610 con 0x7f22ac106f10 2024-09-16T00:29:01.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.336+0000 7f22aaffd640 1 -- 172.21.15.1:0/1017250218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4035070 con 0x7f22ac106f10 2024-09-16T00:29:01.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.336+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ac1a4f50 con 0x7f22ac106f10 2024-09-16T00:29:01.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.337+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ac1021c0 con 0x7f22ac106f10 2024-09-16T00:29:01.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.338+0000 7f22aaffd640 1 -- 172.21.15.1:0/1017250218 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f22a4042030 con 0x7f22ac106f10 2024-09-16T00:29:01.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.339+0000 7f22aaffd640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22940a3240 0x7f22940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:01.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.339+0000 7f22b1cc8640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22940a3240 0x7f22940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:01.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.339+0000 7f22aaffd640 1 -- 172.21.15.1:0/1017250218 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f22a40f06a0 con 0x7f22ac106f10 2024-09-16T00:29:01.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.340+0000 7f22b1cc8640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22940a3240 0x7f22940a5700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f229c005fd0 tx=0x7f229c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:01.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.343+0000 7f22aaffd640 1 -- 172.21.15.1:0/1017250218 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22a40bcfc0 con 0x7f22ac106f10 2024-09-16T00:29:01.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.520+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22ac10a150 con 0x7f22ac106f10 2024-09-16T00:29:01.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.521+0000 7f22aaffd640 1 -- 172.21.15.1:0/1017250218 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24902 (secure 0 0 0) 0x7f22a40bc8c0 con 0x7f22ac106f10 2024-09-16T00:29:01.523 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:29:01.523 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:27:26.243912+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:29:01.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22940a3240 msgr2=0x7f22940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:01.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22940a3240 0x7f22940a5700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f229c005fd0 tx=0x7f229c009500 comp rx=0 tx=0).stop 2024-09-16T00:29:01.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 msgr2=0x7f22ac19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:01.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 0x7f22ac19cca0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a4004270 comp rx=0 tx=0).stop 2024-09-16T00:29:01.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 shutdown_connections 2024-09-16T00:29:01.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ac106f10 0x7f22ac19cca0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22ac102790 0x7f22ac198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22940a3240 0x7f22940a5700 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 --2- 172.21.15.1:0/1017250218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22ac101de0 0x7f22ac1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 >> 172.21.15.1:0/1017250218 conn(0x7f22ac0fd8c0 msgr2=0x7f22ac0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:01.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 shutdown_connections 2024-09-16T00:29:01.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:01.524+0000 7f22b3f2c640 1 -- 172.21.15.1:0/1017250218 wait complete. 2024-09-16T00:29:01.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:01 smithi001 ceph-mon[30934]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:29:01.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:01 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1017250218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:29:02.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:01 smithi019 ceph-mon[35186]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:29:02.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:01 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1017250218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:29:02.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:01 smithi153 ceph-mon[35538]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:29:02.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:01 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1017250218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:29:02.265 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-16T00:29:02.525 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:03.765 INFO:teuthology.orchestra.run.smithi001.stdout:{"run_dir":"/var/run/ceph"} 2024-09-16T00:29:03.766 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-16T00:29:03.766 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3393 2024-09-16T00:29:03.766 DEBUG:tasks.quiescer:Yielding 2024-09-16T00:29:03.766 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T00:29:03.780 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:29:03.781 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T00:29:03.781 INFO:tasks.workunit:timeout=3h 2024-09-16T00:29:03.781 INFO:tasks.workunit:cleanup=True 2024-09-16T00:29:03.781 DEBUG:teuthology.orchestra.run.smithi001:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T00:29:03.782 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T00:29:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-16T00:29:03.801 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-16T00:29:03.801 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:29:03.801 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-16T00:29:03.801 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-16 00:27:11.456895051 +0000 2024-09-16T00:29:03.801 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-16 00:27:11.456895051 +0000 2024-09-16T00:29:03.801 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-16 00:28:40.623916674 +0000 2024-09-16T00:29:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-16 00:27:11.456895051 +0000 2024-09-16T00:29:03.802 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T00:29:03.802 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T00:29:03.873 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:29:03.931 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T00:29:04.049 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:03 smithi019 ceph-mon[35186]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-09-16T00:29:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:03 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:29:04.172 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:03 smithi001 ceph-mon[30934]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-09-16T00:29:04.173 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:03 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:29:04.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:03 smithi153 ceph-mon[35538]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-09-16T00:29:04.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:03 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:29:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.605+0000 7f2aa2da2640 1 -- 172.21.15.1:0/4063478027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 msgr2=0x7f2a9c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.605+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/4063478027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 0x7f2a9c101200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009900 tx=0x7f2a8402a480 comp rx=0 tx=0).stop 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.605+0000 7f2aa2da2640 1 -- 172.21.15.1:0/4063478027 shutdown_connections 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.605+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/4063478027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c102ef0 0x7f2a9c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.605+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/4063478027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a9c102570 0x7f2a9c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.605+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/4063478027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 0x7f2a9c101200 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.605+0000 7f2aa2da2640 1 -- 172.21.15.1:0/4063478027 >> 172.21.15.1:0/4063478027 conn(0x7f2a9c0fc7d0 msgr2=0x7f2a9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.605+0000 7f2aa2da2640 1 -- 172.21.15.1:0/4063478027 shutdown_connections 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.605+0000 7f2aa2da2640 1 -- 172.21.15.1:0/4063478027 wait complete. 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.606+0000 7f2aa2da2640 1 Processor -- start 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.606+0000 7f2aa2da2640 1 -- start start 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.606+0000 7f2aa2da2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 0x7f2a9c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.606+0000 7f2aa2da2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a9c102570 0x7f2a9c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.606+0000 7f2aa2da2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c102ef0 0x7f2a9c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.606+0000 7f2aa2da2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c110f80 con 0x7f2a9c100e20 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.606+0000 7f2aa2da2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c110e00 con 0x7f2a9c102ef0 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.606+0000 7f2aa2da2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c111100 con 0x7f2a9c102570 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.606+0000 7f2aa0b3e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 0x7f2a9c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa0b3e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 0x7f2a9c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54266/0 (socket says 172.21.15.1:54266) 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa0b3e640 1 -- 172.21.15.1:0/1147906628 learned_addr learned my addr 172.21.15.1:0/1147906628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2a9bfff640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a9c102570 0x7f2a9c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa133f640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c102ef0 0x7f2a9c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa0b3e640 1 -- 172.21.15.1:0/1147906628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a9c102570 msgr2=0x7f2a9c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa0b3e640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a9c102570 0x7f2a9c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa0b3e640 1 -- 172.21.15.1:0/1147906628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c102ef0 msgr2=0x7f2a9c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa0b3e640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c102ef0 0x7f2a9c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa0b3e640 1 -- 172.21.15.1:0/1147906628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c19d4b0 con 0x7f2a9c100e20 2024-09-16T00:29:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa0b3e640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 0x7f2a9c198500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2a84006010 tx=0x7f2a84003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2a99ffb640 1 -- 172.21.15.1:0/1147906628 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8403f070 con 0x7f2a9c100e20 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2a99ffb640 1 -- 172.21.15.1:0/1147906628 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a84031c30 con 0x7f2a9c100e20 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c19d740 con 0x7f2a9c100e20 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.607+0000 7f2a99ffb640 1 -- 172.21.15.1:0/1147906628 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84035030 con 0x7f2a9c100e20 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.608+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c1a5080 con 0x7f2a9c100e20 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.609+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9c101200 con 0x7f2a9c100e20 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.610+0000 7f2a99ffb640 1 -- 172.21.15.1:0/1147906628 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2a84042070 con 0x7f2a9c100e20 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.611+0000 7f2a99ffb640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2a880a32c0 0x7f2a880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.611+0000 7f2a9bfff640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2a880a32c0 0x7f2a880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.611+0000 7f2a99ffb640 1 -- 172.21.15.1:0/1147906628 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2a840f04f0 con 0x7f2a9c100e20 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.611+0000 7f2a9bfff640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2a880a32c0 0x7f2a880a5780 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0ffcc0 tx=0x7f2a8c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:04.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.620+0000 7f2a99ffb640 1 -- 172.21.15.1:0/1147906628 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a84031de0 con 0x7f2a9c100e20 2024-09-16T00:29:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.850+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a9c1a5250 con 0x7f2a9c100e20 2024-09-16T00:29:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.850+0000 7f2a99ffb640 1 -- 172.21.15.1:0/1147906628 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f2a840bce10 con 0x7f2a9c100e20 2024-09-16T00:29:04.851 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:29:04.851 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:29:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:29:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2a880a32c0 msgr2=0x7f2a880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2a880a32c0 0x7f2a880a5780 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0ffcc0 tx=0x7f2a8c009680 comp rx=0 tx=0).stop 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 msgr2=0x7f2a9c198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 0x7f2a9c198500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2a84006010 tx=0x7f2a84003c10 comp rx=0 tx=0).stop 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 shutdown_connections 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c102ef0 0x7f2a9c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2a880a32c0 0x7f2a880a5780 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a9c102570 0x7f2a9c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 --2- 172.21.15.1:0/1147906628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a9c100e20 0x7f2a9c198500 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 >> 172.21.15.1:0/1147906628 conn(0x7f2a9c0fc7d0 msgr2=0x7f2a9c06a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 shutdown_connections 2024-09-16T00:29:04.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:04.853+0000 7f2aa2da2640 1 -- 172.21.15.1:0/1147906628 wait complete. 2024-09-16T00:29:05.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:26:18.890037+0000', 'modified': '2024-09-16T00:28:41.306996+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24470, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/389176790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 389176790}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 389176790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6835/3078906320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 3078906320}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 3078906320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6837/3490951208', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 3490951208}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 3490951208}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.153:6839/1065022363', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 1065022363}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 1065022363}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.19:6837/928170641', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6836', 'nonce': 928170641}, {'type': 'v1', 'addr': '172.21.15.19:6837', 'nonce': 928170641}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6839/3056769092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 3056769092}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 3056769092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24469, 24470, 24506]} 2024-09-16T00:29:05.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-16T00:29:05.687 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T00:29:05.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:05 smithi001 ceph-mon[30934]: pgmap v371: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-16T00:29:05.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:05 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1147906628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:05.989 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:06.014 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:05 smithi153 ceph-mon[35538]: pgmap v371: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-16T00:29:06.014 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:05 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1147906628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:06.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:05 smithi019 ceph-mon[35186]: pgmap v371: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-16T00:29:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:05 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1147906628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:06.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.541+0000 7eff551f5640 1 -- 172.21.15.1:0/2160680640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff50103660 msgr2=0x7eff5010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.541+0000 7eff551f5640 1 --2- 172.21.15.1:0/2160680640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff50103660 0x7eff5010bb50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7eff38009900 tx=0x7eff3802a480 comp rx=0 tx=0).stop 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.541+0000 7eff551f5640 1 -- 172.21.15.1:0/2160680640 shutdown_connections 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.541+0000 7eff551f5640 1 --2- 172.21.15.1:0/2160680640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff5010c170 0x7eff5010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.541+0000 7eff551f5640 1 --2- 172.21.15.1:0/2160680640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff50103660 0x7eff5010bb50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.541+0000 7eff551f5640 1 --2- 172.21.15.1:0/2160680640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff50102d40 0x7eff50103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.541+0000 7eff551f5640 1 -- 172.21.15.1:0/2160680640 >> 172.21.15.1:0/2160680640 conn(0x7eff500fc7a0 msgr2=0x7eff500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.541+0000 7eff551f5640 1 -- 172.21.15.1:0/2160680640 shutdown_connections 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.541+0000 7eff551f5640 1 -- 172.21.15.1:0/2160680640 wait complete. 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff551f5640 1 Processor -- start 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff551f5640 1 -- start start 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff551f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff50102d40 0x7eff501009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff551f5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff50103660 0x7eff50100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff551f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff5010c170 0x7eff50101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff551f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff50111fb0 con 0x7eff5010c170 2024-09-16T00:29:06.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff551f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff50111e30 con 0x7eff50102d40 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff551f5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff50112130 con 0x7eff50103660 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff4f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff5010c170 0x7eff50101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff4f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff5010c170 0x7eff50101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54274/0 (socket says 172.21.15.1:54274) 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff4f59e640 1 -- 172.21.15.1:0/1270703403 learned_addr learned my addr 172.21.15.1:0/1270703403 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff50102d40 0x7eff501009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.542+0000 7eff4e59c640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff50103660 0x7eff50100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff4f59e640 1 -- 172.21.15.1:0/1270703403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff50103660 msgr2=0x7eff50100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff4f59e640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff50103660 0x7eff50100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff4f59e640 1 -- 172.21.15.1:0/1270703403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff50102d40 msgr2=0x7eff501009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff4f59e640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff50102d40 0x7eff501009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff4f59e640 1 -- 172.21.15.1:0/1270703403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff501a6930 con 0x7eff5010c170 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff4f59e640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff5010c170 0x7eff50101420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7eff4800b970 tx=0x7eff4800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff47fff640 1 -- 172.21.15.1:0/1270703403 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48019070 con 0x7eff5010c170 2024-09-16T00:29:06.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff551f5640 1 -- 172.21.15.1:0/1270703403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff501a6c20 con 0x7eff5010c170 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff47fff640 1 -- 172.21.15.1:0/1270703403 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4800cb40 con 0x7eff5010c170 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff47fff640 1 -- 172.21.15.1:0/1270703403 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48007650 con 0x7eff5010c170 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.543+0000 7eff551f5640 1 -- 172.21.15.1:0/1270703403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501a7160 con 0x7eff5010c170 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.545+0000 7eff47fff640 1 -- 172.21.15.1:0/1270703403 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7eff4801c070 con 0x7eff5010c170 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.545+0000 7eff575ce740 1 Processor -- start 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff575ce740 1 -- start start 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff575ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24bd9d0 0x558ca24bddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff575ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff575ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 0x558ca24cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff575ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ca2424090 con 0x558ca24be2f0 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff575ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ca23cba70 con 0x558ca24c8bb0 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff575ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ca2420050 con 0x558ca24bd9d0 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54282/0 (socket says 172.21.15.1:54282) 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4ed9d640 1 -- 172.21.15.1:0/1844983133 learned_addr learned my addr 172.21.15.1:0/1844983133 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff47fff640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7eff3c0a3240 0x7eff3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4e59c640 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24bd9d0 0x558ca24bddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4e59c640 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 0x558ca24cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4ed9d640 1 -- 172.21.15.1:0/1844983133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24bd9d0 msgr2=0x558ca24bddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24bd9d0 0x558ca24bddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4ed9d640 1 -- 172.21.15.1:0/1844983133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 msgr2=0x558ca24cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 0x558ca24cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4ed9d640 1 -- 172.21.15.1:0/1844983133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca23d8130 con 0x558ca24be2f0 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.546+0000 7eff4e59c640 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 0x558ca24cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.547+0000 7eff4f59e640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7eff3c0a3240 0x7eff3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.547+0000 7eff47fff640 1 -- 172.21.15.1:0/1270703403 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7eff480cf2b0 con 0x7eff5010c170 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.547+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24c8670 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7eff4000bd10 tx=0x7eff400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da690c25e3986582 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.547+0000 7eff457fa640 1 -- 172.21.15.1:0/1844983133 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40004030 con 0x558ca24be2f0 2024-09-16T00:29:06.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.547+0000 7eff4f59e640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7eff3c0a3240 0x7eff3c0a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7eff4800b940 tx=0x7eff48021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.547+0000 7eff457fa640 1 -- 172.21.15.1:0/1844983133 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff400041d0 con 0x558ca24be2f0 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.547+0000 7eff457fa640 1 -- 172.21.15.1:0/1844983133 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40011820 con 0x558ca24be2f0 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 -- 172.21.15.1:0/1844983133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 msgr2=0x558ca24c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24c8670 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7eff4000bd10 tx=0x7eff400107a0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 -- 172.21.15.1:0/1844983133 shutdown_connections 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 0x558ca24cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24c8670 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 --2- 172.21.15.1:0/1844983133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24bd9d0 0x558ca24bddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 -- 172.21.15.1:0/1844983133 >> 172.21.15.1:0/1844983133 conn(0x558ca24b9f60 msgr2=0x558ca24ba340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 -- 172.21.15.1:0/1844983133 shutdown_connections 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 -- 172.21.15.1:0/1844983133 wait complete. 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 -- start start 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.548+0000 7eff575ce740 1 Processor -- start 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff575ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24bd9d0 0x558ca24c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff575ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 0x558ca24c5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff575ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 0x558ca24c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff575ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ca24cb780 con 0x558ca24bd9d0 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff575ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ca23def20 con 0x558ca24c8bb0 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff575ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ca24cb900 con 0x558ca24be2f0 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff575ce740 10 client.0 _ll_register_callbacks cb 0x558ca23a8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24bd9d0 0x558ca24c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24bd9d0 0x558ca24c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54298/0 (socket says 172.21.15.1:54298) 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 -- 172.21.15.1:0/688884868 learned_addr learned my addr 172.21.15.1:0/688884868 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 0x558ca24c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 0x558ca24c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 -- 172.21.15.1:0/688884868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 msgr2=0x558ca24c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 0x558ca24c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 -- 172.21.15.1:0/688884868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 msgr2=0x558ca24c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:06.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 0x558ca24c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 -- 172.21.15.1:0/688884868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca21952b0 con 0x558ca24bd9d0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff4e59c640 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24bd9d0 0x558ca24c8430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7eff38005940 tx=0x7eff38005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f0d9e87923bb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.549+0000 7eff367fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff367fc640 1 -- 172.21.15.1:0/688884868 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38035040 con 0x558ca24bd9d0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff367fc640 1 -- 172.21.15.1:0/688884868 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7eff380037f0 con 0x558ca24bd9d0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff575ce740 1 -- 172.21.15.1:0/688884868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff400041d0 con 0x558ca24bd9d0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff367fc640 1 -- 172.21.15.1:0/688884868 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7eff38051050 con 0x558ca24bd9d0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff575ce740 10 client.15390 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff575ce740 1 -- 172.21.15.1:0/688884868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558ca24d7000 con 0x558ca24bd9d0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff367fc640 1 -- 172.21.15.1:0/688884868 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff400041d0 con 0x558ca24bd9d0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff367fc640 1 -- 172.21.15.1:0/688884868 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7eff3803a040 con 0x558ca24bd9d0 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff575ce740 10 client.15390 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:06.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff575ce740 10 client.15390 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff575ce740 20 client.15390 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff575ce740 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x558ca24d9560 0x558ca24da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff575ce740 4 client.15390 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.550+0000 7eff575ce740 1 -- 172.21.15.1:0/688884868 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ca2431600 con 0x558ca24d9560 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.551+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x558ca24d9560 0x558ca24da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.551+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x558ca24d9560 0x558ca24da9d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e63fad1d2414db6 server_cookie=6f0e4287e62c8d79 in_seq=0 out_seq=0 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.551+0000 7eff367fc640 10 client.15390 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff367fc640 1 -- 172.21.15.1:0/688884868 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ca2431600 con 0x558ca24d9560 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff367fc640 10 client.15390 handle_command_reply: tid=0 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 client.15390 shutdown 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 -- 172.21.15.1:0/688884868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24bd9d0 msgr2=0x558ca24c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24bd9d0 0x558ca24c8430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7eff38005940 tx=0x7eff38005a40 comp rx=0 tx=0).stop 2024-09-16T00:29:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 -- 172.21.15.1:0/688884868 shutdown_connections 2024-09-16T00:29:06.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x558ca24d9560 0x558ca24da9d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24c8bb0 0x558ca24c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 0x558ca24c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff367fc640 0 client.15390 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:29:06.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 --2- 172.21.15.1:0/688884868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24bd9d0 0x558ca24c8430 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 -- 172.21.15.1:0/688884868 >> 172.21.15.1:0/688884868 conn(0x558ca24b9f60 msgr2=0x558ca24bfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 -- 172.21.15.1:0/688884868 shutdown_connections 2024-09-16T00:29:06.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 1 -- 172.21.15.1:0/688884868 wait complete. 2024-09-16T00:29:06.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.552+0000 7eff575ce740 20 client.15390 trim_cache size 0 max 16384 2024-09-16T00:29:06.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.570+0000 7eff575ce740 1 Processor -- start 2024-09-16T00:29:06.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff575ce740 1 -- start start 2024-09-16T00:29:06.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff575ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 0x558ca24d9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff575ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff575ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24c8bb0 0x558ca24e31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff575ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ca24d8d10 con 0x558ca24be2f0 2024-09-16T00:29:06.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff575ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ca24d93e0 con 0x558ca24bd9d0 2024-09-16T00:29:06.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff575ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ca24d9ab0 con 0x558ca24c8bb0 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24e2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54306/0 (socket says 172.21.15.1:54306) 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4ed9d640 1 -- 172.21.15.1:0/657866813 learned_addr learned my addr 172.21.15.1:0/657866813 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4e59c640 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 0x558ca24d9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4e59c640 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24c8bb0 0x558ca24e31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4ed9d640 1 -- 172.21.15.1:0/657866813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24c8bb0 msgr2=0x558ca24e31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24c8bb0 0x558ca24e31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4ed9d640 1 -- 172.21.15.1:0/657866813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 msgr2=0x558ca24d9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 0x558ca24d9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4ed9d640 1 -- 172.21.15.1:0/657866813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca24cb310 con 0x558ca24be2f0 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.571+0000 7eff4e59c640 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24c8bb0 0x558ca24e31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.572+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24e2cb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x558ca24badc0 tx=0x7eff40011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d68f2ae748675f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.572+0000 7eff44ff9640 1 -- 172.21.15.1:0/657866813 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40005240 con 0x558ca24be2f0 2024-09-16T00:29:06.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.572+0000 7eff44ff9640 1 -- 172.21.15.1:0/657866813 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4000be80 con 0x558ca24be2f0 2024-09-16T00:29:06.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.572+0000 7eff44ff9640 1 -- 172.21.15.1:0/657866813 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40011710 con 0x558ca24be2f0 2024-09-16T00:29:06.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.572+0000 7eff575ce740 1 -- 172.21.15.1:0/657866813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 msgr2=0x558ca24e2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.572+0000 7eff575ce740 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24e2cb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x558ca24badc0 tx=0x7eff40011440 comp rx=0 tx=0).stop 2024-09-16T00:29:06.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 -- 172.21.15.1:0/657866813 shutdown_connections 2024-09-16T00:29:06.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24c8bb0 0x558ca24e31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24be2f0 0x558ca24e2cb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 --2- 172.21.15.1:0/657866813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 0x558ca24d9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 -- 172.21.15.1:0/657866813 >> 172.21.15.1:0/657866813 conn(0x558ca24b9f60 msgr2=0x558ca24ba340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 -- 172.21.15.1:0/657866813 shutdown_connections 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 -- 172.21.15.1:0/657866813 wait complete. 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 -- start start 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 Processor -- start 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 0x558ca24e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 0x558ca24de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24c8bb0 0x558ca24dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ca24da850 con 0x558ca24c8bb0 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ca24da180 con 0x558ca24bd9d0 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ca24daf20 con 0x558ca24be2f0 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff575ce740 10 client.0 _ll_register_callbacks cb 0x7eff5006b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:06.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff4e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24c8bb0 0x558ca24dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff4e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24c8bb0 0x558ca24dea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54308/0 (socket says 172.21.15.1:54308) 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.573+0000 7eff4e59c640 1 -- 172.21.15.1:0/1904015116 learned_addr learned my addr 172.21.15.1:0/1904015116 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff4e59c640 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 0x558ca24e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff4e59c640 1 -- 172.21.15.1:0/1904015116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 msgr2=0x558ca24de4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 0x558ca24de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff4e59c640 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 0x558ca24de4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff4e59c640 1 -- 172.21.15.1:0/1904015116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 msgr2=0x558ca24e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff4e59c640 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 0x558ca24e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff4e59c640 1 -- 172.21.15.1:0/1904015116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca24def50 con 0x558ca24c8bb0 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 0x558ca24de4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff4e59c640 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24c8bb0 0x558ca24dea10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff3803c840 tx=0x7eff38031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ba78fe91d2137 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff367fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:06.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff367fc640 1 -- 172.21.15.1:0/1904015116 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38035040 con 0x558ca24c8bb0 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff367fc640 1 -- 172.21.15.1:0/1904015116 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7eff3803c8c0 con 0x558ca24c8bb0 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff575ce740 1 -- 172.21.15.1:0/1904015116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ca242bb10 con 0x558ca24c8bb0 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff367fc640 1 -- 172.21.15.1:0/1904015116 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7eff38003820 con 0x558ca24c8bb0 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff575ce740 10 client.15396 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff575ce740 1 -- 172.21.15.1:0/1904015116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558ca25438f0 con 0x558ca24c8bb0 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.574+0000 7eff367fc640 1 -- 172.21.15.1:0/1904015116 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558ca242bb10 con 0x558ca24c8bb0 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff367fc640 1 -- 172.21.15.1:0/1904015116 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7eff3803d7f0 con 0x558ca24c8bb0 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff575ce740 10 client.15396 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff575ce740 10 client.15396 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff575ce740 20 client.15396 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff575ce740 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x558ca25484c0 0x558ca25471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff575ce740 4 client.15396 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:29:06.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff575ce740 1 -- 172.21.15.1:0/1904015116 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "status"}) -- 0x558ca23a80e0 con 0x558ca25484c0 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x558ca25484c0 0x558ca25471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff367fc640 10 client.15396 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.575+0000 7eff4ed9d640 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x558ca25484c0 0x558ca25471d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63b3acbfc3579c5e server_cookie=8cb703a4d3c6d894 in_seq=0 out_seq=0 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff367fc640 1 -- 172.21.15.1:0/1904015116 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558ca23a80e0 con 0x558ca25484c0 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff367fc640 10 client.15396 handle_command_reply: tid=0 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 client.15396 shutdown 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 -- 172.21.15.1:0/1904015116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24c8bb0 msgr2=0x558ca24dea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24c8bb0 0x558ca24dea10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff3803c840 tx=0x7eff38031970 comp rx=0 tx=0).stop 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 -- 172.21.15.1:0/1904015116 shutdown_connections 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca24c8bb0 0x558ca24dea10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ca24be2f0 0x558ca24de4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x558ca25484c0 0x558ca25471d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 --2- 172.21.15.1:0/1904015116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ca24bd9d0 0x558ca24e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 -- 172.21.15.1:0/1904015116 >> 172.21.15.1:0/1904015116 conn(0x558ca24b9f60 msgr2=0x558ca24b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff367fc640 0 client.15396 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 -- 172.21.15.1:0/1904015116 shutdown_connections 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 1 -- 172.21.15.1:0/1904015116 wait complete. 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.576+0000 7eff575ce740 20 client.15396 trim_cache size 0 max 16384 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24469, 2024-09-16T00:29:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:29:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:29:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:29:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 124.976359675, 2024-09-16T00:29:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-09-16T00:29:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-09-16T00:29:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 91, 2024-09-16T00:29:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 187.69692655899999 2024-09-16T00:29:06.583 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 -- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7eff3c0a3240 msgr2=0x7eff3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7eff3c0a3240 0x7eff3c0a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7eff4800b940 tx=0x7eff48021000 comp rx=0 tx=0).stop 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 -- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff5010c170 msgr2=0x7eff50101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff5010c170 0x7eff50101420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7eff4800b970 tx=0x7eff4800be30 comp rx=0 tx=0).stop 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 -- 172.21.15.1:0/1270703403 shutdown_connections 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff5010c170 0x7eff50101420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7eff3c0a3240 0x7eff3c0a5700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff50103660 0x7eff50100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 --2- 172.21.15.1:0/1270703403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff50102d40 0x7eff501009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 -- 172.21.15.1:0/1270703403 >> 172.21.15.1:0/1270703403 conn(0x7eff500fc7a0 msgr2=0x7eff500fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 -- 172.21.15.1:0/1270703403 shutdown_connections 2024-09-16T00:29:06.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:06.577+0000 7eff551f5640 1 -- 172.21.15.1:0/1270703403 wait complete. 2024-09-16T00:29:07.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T00:29:07.743 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:08.020 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:07 smithi001 ceph-mon[30934]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:08.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:07 smithi019 ceph-mon[35186]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:08.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:07 smithi153 ceph-mon[35538]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:08.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 -- 172.21.15.1:0/2751733582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c101de0 msgr2=0x7f934c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:08.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 --2- 172.21.15.1:0/2751733582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c101de0 0x7f934c1021c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f933c009900 tx=0x7f933c02a410 comp rx=0 tx=0).stop 2024-09-16T00:29:08.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 -- 172.21.15.1:0/2751733582 shutdown_connections 2024-09-16T00:29:08.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 --2- 172.21.15.1:0/2751733582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f934c106f10 0x7f934c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 --2- 172.21.15.1:0/2751733582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f934c102790 0x7f934c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 --2- 172.21.15.1:0/2751733582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c101de0 0x7f934c1021c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 -- 172.21.15.1:0/2751733582 >> 172.21.15.1:0/2751733582 conn(0x7f934c0fd8c0 msgr2=0x7f934c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:08.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 -- 172.21.15.1:0/2751733582 shutdown_connections 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 -- 172.21.15.1:0/2751733582 wait complete. 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.268+0000 7f9351b6d640 1 Processor -- start 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f9351b6d640 1 -- start start 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f9351b6d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f934c101de0 0x7f934c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f9351b6d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f934c102790 0x7f934c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f9351b6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c106f10 0x7f934c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f9351b6d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f934c111f70 con 0x7f934c106f10 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f9351b6d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f934c111df0 con 0x7f934c101de0 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f9351b6d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f934c1120f0 con 0x7f934c102790 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c106f10 0x7f934c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c106f10 0x7f934c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54336/0 (socket says 172.21.15.1:54336) 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934bfff640 1 -- 172.21.15.1:0/1047806781 learned_addr learned my addr 172.21.15.1:0/1047806781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:08.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934b7fe640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f934c101de0 0x7f934c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934affd640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f934c102790 0x7f934c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934bfff640 1 -- 172.21.15.1:0/1047806781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f934c102790 msgr2=0x7f934c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934bfff640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f934c102790 0x7f934c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934bfff640 1 -- 172.21.15.1:0/1047806781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f934c101de0 msgr2=0x7f934c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934bfff640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f934c101de0 0x7f934c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.269+0000 7f934bfff640 1 -- 172.21.15.1:0/1047806781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c1a6870 con 0x7f934c106f10 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.270+0000 7f934bfff640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c106f10 0x7f934c074f70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f934400cce0 tx=0x7f934400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.270+0000 7f9348ff9640 1 -- 172.21.15.1:0/1047806781 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934400f450 con 0x7f934c106f10 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.270+0000 7f9348ff9640 1 -- 172.21.15.1:0/1047806781 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934400fad0 con 0x7f934c106f10 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.270+0000 7f9348ff9640 1 -- 172.21.15.1:0/1047806781 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344018c10 con 0x7f934c106f10 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.270+0000 7f9351b6d640 1 -- 172.21.15.1:0/1047806781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c1a6b60 con 0x7f934c106f10 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.270+0000 7f9351b6d640 1 -- 172.21.15.1:0/1047806781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c1a7050 con 0x7f934c106f10 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.272+0000 7f9353f46740 1 Processor -- start 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f9348ff9640 1 -- 172.21.15.1:0/1047806781 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f934400f5f0 con 0x7f934c106f10 2024-09-16T00:29:08.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f9353f46740 1 -- start start 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f9353f46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439b9d0 0x55faa439bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f9353f46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439c2f0 0x55faa43a6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f9353f46740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa43a6bb0 0x55faa43a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f9353f46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55faa4302090 con 0x55faa439c2f0 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f9353f46740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faa42a9a70 con 0x55faa43a6bb0 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f9353f46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55faa42fe050 con 0x55faa439b9d0 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439c2f0 0x55faa43a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439c2f0 0x55faa43a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54340/0 (socket says 172.21.15.1:54340) 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f934b7fe640 1 -- 172.21.15.1:0/1891313846 learned_addr learned my addr 172.21.15.1:0/1891313846 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.273+0000 7f934affd640 1 --2- 172.21.15.1:0/1891313846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa43a6bb0 0x55faa43a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f934b7fe640 1 -- 172.21.15.1:0/1891313846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439b9d0 msgr2=0x55faa439bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f934b7fe640 1 --2- 172.21.15.1:0/1891313846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439b9d0 0x55faa439bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f934b7fe640 1 -- 172.21.15.1:0/1891313846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa43a6bb0 msgr2=0x55faa43a8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:08.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f934b7fe640 1 --2- 172.21.15.1:0/1891313846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa43a6bb0 0x55faa43a8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f934b7fe640 1 -- 172.21.15.1:0/1891313846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa42b6130 con 0x55faa439c2f0 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f9348ff9640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93340a3240 0x7f93340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f934affd640 1 --2- 172.21.15.1:0/1891313846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa43a6bb0 0x55faa43a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f9348ff9640 1 -- 172.21.15.1:0/1047806781 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f93440d0390 con 0x7f934c106f10 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f934b7fe640 1 --2- 172.21.15.1:0/1891313846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439c2f0 0x55faa43a6670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f933c009900 tx=0x7f933c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436987ea7d1160a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f934affd640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93340a3240 0x7f93340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f9331ffb640 1 -- 172.21.15.1:0/1891313846 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c040070 con 0x55faa439c2f0 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f9331ffb640 1 -- 172.21.15.1:0/1891313846 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f933c005020 con 0x55faa439c2f0 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.274+0000 7f9331ffb640 1 -- 172.21.15.1:0/1891313846 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c035040 con 0x55faa439c2f0 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 -- 172.21.15.1:0/1891313846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439c2f0 msgr2=0x55faa43a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f934affd640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93340a3240 0x7f93340a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9340005d90 tx=0x7f934000e490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 --2- 172.21.15.1:0/1891313846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439c2f0 0x55faa43a6670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f933c009900 tx=0x7f933c0316e0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 -- 172.21.15.1:0/1891313846 shutdown_connections 2024-09-16T00:29:08.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 --2- 172.21.15.1:0/1891313846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa43a6bb0 0x55faa43a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 --2- 172.21.15.1:0/1891313846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439c2f0 0x55faa43a6670 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 --2- 172.21.15.1:0/1891313846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439b9d0 0x55faa439bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 -- 172.21.15.1:0/1891313846 >> 172.21.15.1:0/1891313846 conn(0x55faa4397f60 msgr2=0x55faa4398340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 -- 172.21.15.1:0/1891313846 shutdown_connections 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 -- 172.21.15.1:0/1891313846 wait complete. 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 -- start start 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.275+0000 7f9353f46740 1 Processor -- start 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f9353f46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439b9d0 0x55faa43a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f9353f46740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 0x55faa43a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f9353f46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa43a6bb0 0x55faa43a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f9353f46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55faa43a9780 con 0x55faa439b9d0 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f9353f46740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faa42bcf20 con 0x55faa439c2f0 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f9353f46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55faa43a9900 con 0x55faa43a6bb0 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f9353f46740 10 client.0 _ll_register_callbacks cb 0x55faa4286030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:08.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439b9d0 0x55faa43a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439b9d0 0x55faa43a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54356/0 (socket says 172.21.15.1:54356) 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934b7fe640 1 -- 172.21.15.1:0/3007084739 learned_addr learned my addr 172.21.15.1:0/3007084739 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934bfff640 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa43a6bb0 0x55faa43a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934affd640 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 0x55faa43a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934b7fe640 1 -- 172.21.15.1:0/3007084739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa43a6bb0 msgr2=0x55faa43a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934b7fe640 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa43a6bb0 0x55faa43a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934b7fe640 1 -- 172.21.15.1:0/3007084739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 msgr2=0x55faa43a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934b7fe640 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 0x55faa43a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934b7fe640 1 -- 172.21.15.1:0/3007084739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa40732b0 con 0x55faa439b9d0 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f934b7fe640 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439b9d0 0x55faa43a6350 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f933c005940 tx=0x7f933c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406974541503e4f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.276+0000 7f932affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f932affd640 1 -- 172.21.15.1:0/3007084739 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c040070 con 0x55faa439b9d0 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f932affd640 1 -- 172.21.15.1:0/3007084739 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f933c0053e0 con 0x55faa439b9d0 2024-09-16T00:29:08.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f9353f46740 1 -- 172.21.15.1:0/3007084739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f933c005020 con 0x55faa439b9d0 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f932affd640 1 -- 172.21.15.1:0/3007084739 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f933c035050 con 0x55faa439b9d0 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f9353f46740 10 client.15408 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f9353f46740 1 -- 172.21.15.1:0/3007084739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faa43b4fc0 con 0x55faa439b9d0 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f932affd640 1 -- 172.21.15.1:0/3007084739 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f933c005020 con 0x55faa439b9d0 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f932affd640 1 -- 172.21.15.1:0/3007084739 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f933c03cbd0 con 0x55faa439b9d0 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f9353f46740 10 client.15408 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f9353f46740 10 client.15408 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f9353f46740 20 client.15408 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f9353f46740 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55faa43b70c0 0x55faa43b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f9353f46740 4 client.15408 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.277+0000 7f9353f46740 1 -- 172.21.15.1:0/3007084739 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa430f600 con 0x55faa43b70c0 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.278+0000 7f934affd640 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55faa43b70c0 0x55faa43b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.278+0000 7f932affd640 10 client.15408 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.278+0000 7f934affd640 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55faa43b70c0 0x55faa43b8530 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34a6170535cb3165 server_cookie=65033e64dbdf2668 in_seq=0 out_seq=0 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f932affd640 1 -- 172.21.15.1:0/3007084739 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55faa430f600 con 0x55faa43b70c0 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f932affd640 10 client.15408 handle_command_reply: tid=0 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 client.15408 shutdown 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 -- 172.21.15.1:0/3007084739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439b9d0 msgr2=0x55faa43a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439b9d0 0x55faa43a6350 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f933c005940 tx=0x7f933c009300 comp rx=0 tx=0).stop 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 -- 172.21.15.1:0/3007084739 shutdown_connections 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa43a6bb0 0x55faa43a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 0x55faa43a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55faa43b70c0 0x55faa43b8530 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 --2- 172.21.15.1:0/3007084739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa439b9d0 0x55faa43a6350 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 -- 172.21.15.1:0/3007084739 >> 172.21.15.1:0/3007084739 conn(0x55faa4397f60 msgr2=0x55faa439dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f932affd640 0 client.15408 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 -- 172.21.15.1:0/3007084739 shutdown_connections 2024-09-16T00:29:08.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 1 -- 172.21.15.1:0/3007084739 wait complete. 2024-09-16T00:29:08.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.279+0000 7f9353f46740 20 client.15408 trim_cache size 0 max 16384 2024-09-16T00:29:08.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f9353f46740 1 Processor -- start 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f9353f46740 1 -- start start 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f9353f46740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439b9d0 0x55faa43b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f9353f46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439c2f0 0x55faa43a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f9353f46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43c0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f9353f46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55faa43b6870 con 0x55faa43a6bb0 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f9353f46740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faa43b6f40 con 0x55faa439b9d0 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f9353f46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55faa43b7610 con 0x55faa439c2f0 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f934bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f934bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43c0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54358/0 (socket says 172.21.15.1:54358) 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f934bfff640 1 -- 172.21.15.1:0/1601555875 learned_addr learned my addr 172.21.15.1:0/1601555875 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f934b7fe640 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439b9d0 0x55faa43b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f934affd640 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439c2f0 0x55faa43a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f934bfff640 1 -- 172.21.15.1:0/1601555875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439c2f0 msgr2=0x55faa43a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f934bfff640 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439c2f0 0x55faa43a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f934bfff640 1 -- 172.21.15.1:0/1601555875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439b9d0 msgr2=0x55faa43b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.297+0000 7f934bfff640 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439b9d0 0x55faa43b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.298+0000 7f934bfff640 1 -- 172.21.15.1:0/1601555875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa43a66c0 con 0x55faa43a6bb0 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.298+0000 7f934b7fe640 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439b9d0 0x55faa43b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.298+0000 7f934affd640 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439c2f0 0x55faa43a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.298+0000 7f934bfff640 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43c0810 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f934400ccb0 tx=0x7f934409bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e2dcfa98aac221 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.298+0000 7f93317fa640 1 -- 172.21.15.1:0/1601555875 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93440b4020 con 0x55faa43a6bb0 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.298+0000 7f93317fa640 1 -- 172.21.15.1:0/1601555875 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93440a0960 con 0x55faa43a6bb0 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.298+0000 7f93317fa640 1 -- 172.21.15.1:0/1601555875 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344098c20 con 0x55faa43a6bb0 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 -- 172.21.15.1:0/1601555875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 msgr2=0x55faa43c0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43c0810 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f934400ccb0 tx=0x7f934409bf10 comp rx=0 tx=0).stop 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 -- 172.21.15.1:0/1601555875 shutdown_connections 2024-09-16T00:29:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43c0810 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439c2f0 0x55faa43a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 --2- 172.21.15.1:0/1601555875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439b9d0 0x55faa43b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 -- 172.21.15.1:0/1601555875 >> 172.21.15.1:0/1601555875 conn(0x55faa4397f60 msgr2=0x55faa43a1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 -- 172.21.15.1:0/1601555875 shutdown_connections 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 -- 172.21.15.1:0/1601555875 wait complete. 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 -- start start 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 Processor -- start 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439b9d0 0x55faa43c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 0x55faa43ba060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43ba5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.299+0000 7f9353f46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55faa43b83b0 con 0x55faa43a6bb0 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f9353f46740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faa43b7ce0 con 0x55faa439c2f0 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f9353f46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55faa43b8a80 con 0x55faa439b9d0 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f9353f46740 10 client.0 _ll_register_callbacks cb 0x7f933c0315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:08.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43ba5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43ba5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54362/0 (socket says 172.21.15.1:54362) 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934bfff640 1 -- 172.21.15.1:0/2977851454 learned_addr learned my addr 172.21.15.1:0/2977851454 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934b7fe640 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439b9d0 0x55faa43c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934affd640 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 0x55faa43ba060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934bfff640 1 -- 172.21.15.1:0/2977851454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439b9d0 msgr2=0x55faa43c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934bfff640 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439b9d0 0x55faa43c0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934bfff640 1 -- 172.21.15.1:0/2977851454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 msgr2=0x55faa43ba060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934bfff640 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 0x55faa43ba060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934bfff640 1 -- 172.21.15.1:0/2977851454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa43babe0 con 0x55faa43a6bb0 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934affd640 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 0x55faa43ba060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f934bfff640 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43ba5a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f934400ab20 tx=0x7f934409a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889e9dc7a89909d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f932affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:08.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f932affd640 1 -- 172.21.15.1:0/2977851454 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93440b4070 con 0x55faa43a6bb0 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f932affd640 1 -- 172.21.15.1:0/2977851454 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f93440a4d90 con 0x55faa43a6bb0 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.300+0000 7f9353f46740 1 -- 172.21.15.1:0/2977851454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93440a0960 con 0x55faa43a6bb0 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f932affd640 1 -- 172.21.15.1:0/2977851454 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f93440a4f90 con 0x55faa43a6bb0 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f9353f46740 10 client.15414 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f9353f46740 1 -- 172.21.15.1:0/2977851454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faa43bf630 con 0x55faa43a6bb0 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f932affd640 1 -- 172.21.15.1:0/2977851454 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93440a0960 con 0x55faa43a6bb0 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f932affd640 1 -- 172.21.15.1:0/2977851454 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f93440b0ad0 con 0x55faa43a6bb0 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f9353f46740 10 client.15414 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f9353f46740 10 client.15414 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:29:08.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f9353f46740 20 client.15414 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f9353f46740 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55faa4425ba0 0x55faa44248b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f9353f46740 4 client.15414 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f9353f46740 1 -- 172.21.15.1:0/2977851454 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "status"}) -- 0x55faa4309b10 con 0x55faa4425ba0 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.301+0000 7f934b7fe640 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55faa4425ba0 0x55faa44248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f934b7fe640 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55faa4425ba0 0x55faa44248b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f92a3aa0a24a55c server_cookie=a0c9467f79440fd3 in_seq=0 out_seq=0 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f932affd640 10 client.15414 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f932affd640 1 -- 172.21.15.1:0/2977851454 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55faa4309b10 con 0x55faa4425ba0 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f932affd640 10 client.15414 handle_command_reply: tid=0 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 client.15414 shutdown 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 -- 172.21.15.1:0/2977851454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 msgr2=0x55faa43ba5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43ba5a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f934400ab20 tx=0x7f934409a8b0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 -- 172.21.15.1:0/2977851454 shutdown_connections 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55faa4425ba0 0x55faa44248b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55faa43a6bb0 0x55faa43ba5a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa439c2f0 0x55faa43ba060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f932affd640 0 client.15414 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 --2- 172.21.15.1:0/2977851454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa439b9d0 0x55faa43c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 -- 172.21.15.1:0/2977851454 >> 172.21.15.1:0/2977851454 conn(0x55faa4397f60 msgr2=0x55faa43b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 -- 172.21.15.1:0/2977851454 shutdown_connections 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 1 -- 172.21.15.1:0/2977851454 wait complete. 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.302+0000 7f9353f46740 20 client.15414 trim_cache size 0 max 16384 2024-09-16T00:29:08.310 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24470, 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 127.708238244, 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 91, 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 192.16411001899999 2024-09-16T00:29:08.311 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:29:08.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 -- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93340a3240 msgr2=0x7f93340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:08.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93340a3240 0x7f93340a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9340005d90 tx=0x7f934000e490 comp rx=0 tx=0).stop 2024-09-16T00:29:08.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 -- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c106f10 msgr2=0x7f934c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:08.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c106f10 0x7f934c074f70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f934400cce0 tx=0x7f934400a620 comp rx=0 tx=0).stop 2024-09-16T00:29:08.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 -- 172.21.15.1:0/1047806781 shutdown_connections 2024-09-16T00:29:08.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f934c106f10 0x7f934c074f70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93340a3240 0x7f93340a5700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f934c102790 0x7f934c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 --2- 172.21.15.1:0/1047806781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f934c101de0 0x7f934c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:08.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 -- 172.21.15.1:0/1047806781 >> 172.21.15.1:0/1047806781 conn(0x7f934c0fd8c0 msgr2=0x7f934c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:08.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 -- 172.21.15.1:0/1047806781 shutdown_connections 2024-09-16T00:29:08.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:08.303+0000 7f9351b6d640 1 -- 172.21.15.1:0/1047806781 wait complete. 2024-09-16T00:29:08.999 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:29:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.815+0000 7fc757422640 1 -- 172.21.15.1:0/3893269586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750102790 msgr2=0x7fc7501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.815+0000 7fc757422640 1 --2- 172.21.15.1:0/3893269586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750102790 0x7fc7501067e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc740009900 tx=0x7fc74002a410 comp rx=0 tx=0).stop 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.815+0000 7fc757422640 1 -- 172.21.15.1:0/3893269586 shutdown_connections 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.815+0000 7fc757422640 1 --2- 172.21.15.1:0/3893269586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc750106f10 0x7fc75010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.815+0000 7fc757422640 1 --2- 172.21.15.1:0/3893269586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750102790 0x7fc7501067e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.815+0000 7fc757422640 1 --2- 172.21.15.1:0/3893269586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc750101de0 0x7fc7501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.815+0000 7fc757422640 1 -- 172.21.15.1:0/3893269586 >> 172.21.15.1:0/3893269586 conn(0x7fc7500fd8c0 msgr2=0x7fc7500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.815+0000 7fc757422640 1 -- 172.21.15.1:0/3893269586 shutdown_connections 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.815+0000 7fc757422640 1 -- 172.21.15.1:0/3893269586 wait complete. 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc757422640 1 Processor -- start 2024-09-16T00:29:09.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc757422640 1 -- start start 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc757422640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750101de0 0x7fc7501984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc757422640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc750102790 0x7fc7501989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc757422640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc750106f10 0x7fc75019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc757422640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc750111f70 con 0x7fc750101de0 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc757422640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc750111df0 con 0x7fc750106f10 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc757422640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501120f0 con 0x7fc750102790 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc7551be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750101de0 0x7fc7501984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc7551be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750101de0 0x7fc7501984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54378/0 (socket says 172.21.15.1:54378) 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc7551be640 1 -- 172.21.15.1:0/1309954471 learned_addr learned my addr 172.21.15.1:0/1309954471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc7559bf640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc750106f10 0x7fc75019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.816+0000 7fc7549bd640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc750102790 0x7fc7501989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7551be640 1 -- 172.21.15.1:0/1309954471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc750102790 msgr2=0x7fc7501989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:09.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7551be640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc750102790 0x7fc7501989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7551be640 1 -- 172.21.15.1:0/1309954471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc750106f10 msgr2=0x7fc75019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7551be640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc750106f10 0x7fc75019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7551be640 1 -- 172.21.15.1:0/1309954471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75019d460 con 0x7fc750101de0 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7549bd640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc750102790 0x7fc7501989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7551be640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750101de0 0x7fc7501984b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00d770 tx=0x7fc74c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7467fc640 1 -- 172.21.15.1:0/1309954471 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c017070 con 0x7fc750101de0 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7467fc640 1 -- 172.21.15.1:0/1309954471 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74c005500 con 0x7fc750101de0 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc757422640 1 -- 172.21.15.1:0/1309954471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75019d750 con 0x7fc750101de0 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc7467fc640 1 -- 172.21.15.1:0/1309954471 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c004e00 con 0x7fc750101de0 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.817+0000 7fc757422640 1 -- 172.21.15.1:0/1309954471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7501a5030 con 0x7fc750101de0 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.819+0000 7fc7467fc640 1 -- 172.21.15.1:0/1309954471 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc74c004fa0 con 0x7fc750101de0 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7597fb740 1 Processor -- start 2024-09-16T00:29:09.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7597fb740 1 -- start start 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 0x55ad0218fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad0219a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad0219cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7597fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad020f6090 con 0x55ad021902f0 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7597fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad0209da70 con 0x55ad0218f9d0 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7597fb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad020f2050 con 0x55ad0219abb0 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7551be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad0219a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7551be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad0219a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54382/0 (socket says 172.21.15.1:54382) 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7551be640 1 -- 172.21.15.1:0/1448254540 learned_addr learned my addr 172.21.15.1:0/1448254540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7467fc640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc73c0a3240 0x7fc73c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.820+0000 7fc7559bf640 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 0x55ad0218fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7549bd640 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad0219cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7559bf640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc73c0a3240 0x7fc73c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7467fc640 1 -- 172.21.15.1:0/1309954471 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc74c0cf790 con 0x7fc750101de0 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7559bf640 1 -- 172.21.15.1:0/1448254540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 msgr2=0x55ad0219cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7559bf640 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad0219cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7559bf640 1 -- 172.21.15.1:0/1448254540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 msgr2=0x55ad0219a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7559bf640 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad0219a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7559bf640 1 -- 172.21.15.1:0/1448254540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad020aa130 con 0x55ad0218f9d0 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7551be640 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad0219a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7559bf640 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 0x55ad0218fdb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7480149a0 tx=0x7fc748014e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3788a876fbb653a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc73b7fe640 1 -- 172.21.15.1:0/1448254540 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748014470 con 0x55ad0218f9d0 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc7559bf640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc73c0a3240 0x7fc73c0a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc748022000 tx=0x7fc74801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc73b7fe640 1 -- 172.21.15.1:0/1448254540 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748023070 con 0x55ad0218f9d0 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.821+0000 7fc73b7fe640 1 -- 172.21.15.1:0/1448254540 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74800fd00 con 0x55ad0218f9d0 2024-09-16T00:29:09.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 -- 172.21.15.1:0/1448254540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 msgr2=0x55ad0218fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 0x55ad0218fdb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7480149a0 tx=0x7fc748014e60 comp rx=0 tx=0).stop 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 -- 172.21.15.1:0/1448254540 shutdown_connections 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad0219cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad0219a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 --2- 172.21.15.1:0/1448254540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 0x55ad0218fdb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 -- 172.21.15.1:0/1448254540 >> 172.21.15.1:0/1448254540 conn(0x55ad0218bf60 msgr2=0x55ad0218c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 -- 172.21.15.1:0/1448254540 shutdown_connections 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 -- 172.21.15.1:0/1448254540 wait complete. 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 -- start start 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.822+0000 7fc7597fb740 1 Processor -- start 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0218f9d0 0x55ad0219a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad02197820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad02197d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7597fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad0219d780 con 0x55ad0218f9d0 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7597fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad020b0f20 con 0x55ad021902f0 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7597fb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad0219d900 con 0x55ad0219abb0 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7597fb740 10 client.0 _ll_register_callbacks cb 0x55ad0207a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7551be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0218f9d0 0x55ad0219a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7551be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0218f9d0 0x55ad0219a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54388/0 (socket says 172.21.15.1:54388) 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7551be640 1 -- 172.21.15.1:0/3329730938 learned_addr learned my addr 172.21.15.1:0/3329730938 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7549bd640 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad02197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7559bf640 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad02197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7551be640 1 -- 172.21.15.1:0/3329730938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 msgr2=0x55ad02197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7551be640 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad02197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7551be640 1 -- 172.21.15.1:0/3329730938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 msgr2=0x55ad02197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7551be640 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad02197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7551be640 1 -- 172.21.15.1:0/3329730938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad01e672b0 con 0x55ad0218f9d0 2024-09-16T00:29:09.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7549bd640 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad02197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:09.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc7551be640 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0218f9d0 0x55ad0219a4d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0ccee0 tx=0x7fc74c0d3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c444f06e03134bc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:09.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.823+0000 7fc738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc738ff9640 1 -- 172.21.15.1:0/3329730938 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c020070 con 0x55ad0218f9d0 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc738ff9640 1 -- 172.21.15.1:0/3329730938 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc74c0860b0 con 0x55ad0218f9d0 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc7597fb740 1 -- 172.21.15.1:0/3329730938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc748023070 con 0x55ad0218f9d0 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc738ff9640 1 -- 172.21.15.1:0/3329730938 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc74c09c0b0 con 0x55ad0218f9d0 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc7597fb740 10 client.15426 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc7597fb740 1 -- 172.21.15.1:0/3329730938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad021a9100 con 0x55ad0218f9d0 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc738ff9640 1 -- 172.21.15.1:0/3329730938 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748023070 con 0x55ad0218f9d0 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc738ff9640 1 -- 172.21.15.1:0/3329730938 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc74c09cce0 con 0x55ad0218f9d0 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc7597fb740 10 client.15426 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc7597fb740 10 client.15426 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc7597fb740 20 client.15426 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc7597fb740 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad021ab700 0x55ad021acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc7597fb740 4 client.15426 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.824+0000 7fc7597fb740 1 -- 172.21.15.1:0/3329730938 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad02103600 con 0x55ad021ab700 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.825+0000 7fc7549bd640 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad021ab700 0x55ad021acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.825+0000 7fc738ff9640 10 client.15426 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.825+0000 7fc7549bd640 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad021ab700 0x55ad021acb70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=656b4f423a955f5a server_cookie=fa33f53ee987f3fe in_seq=0 out_seq=0 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc738ff9640 1 -- 172.21.15.1:0/3329730938 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad02103600 con 0x55ad021ab700 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc738ff9640 10 client.15426 handle_command_reply: tid=0 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 client.15426 shutdown 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 -- 172.21.15.1:0/3329730938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0218f9d0 msgr2=0x55ad0219a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0218f9d0 0x55ad0219a4d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0ccee0 tx=0x7fc74c0d3040 comp rx=0 tx=0).stop 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 -- 172.21.15.1:0/3329730938 shutdown_connections 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad021ab700 0x55ad021acb70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad02197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad02197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 --2- 172.21.15.1:0/3329730938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0218f9d0 0x55ad0219a4d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 -- 172.21.15.1:0/3329730938 >> 172.21.15.1:0/3329730938 conn(0x55ad0218bf60 msgr2=0x55ad02191c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:09.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc738ff9640 0 client.15426 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:09.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 -- 172.21.15.1:0/3329730938 shutdown_connections 2024-09-16T00:29:09.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 1 -- 172.21.15.1:0/3329730938 wait complete. 2024-09-16T00:29:09.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.826+0000 7fc7597fb740 20 client.15426 trim_cache size 0 max 16384 2024-09-16T00:29:09.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.844+0000 7fc7597fb740 1 Processor -- start 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.844+0000 7fc7597fb740 1 -- start start 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.844+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 0x55ad021abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.844+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad02197100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.844+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad021b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.844+0000 7fc7597fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad021aaeb0 con 0x55ad021902f0 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.844+0000 7fc7597fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad021ab580 con 0x55ad0218f9d0 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.844+0000 7fc7597fb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad021abc50 con 0x55ad0219abb0 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.844+0000 7fc7549bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad02197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc7549bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad02197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54394/0 (socket says 172.21.15.1:54394) 2024-09-16T00:29:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc7549bd640 1 -- 172.21.15.1:0/301882639 learned_addr learned my addr 172.21.15.1:0/301882639 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc7549bd640 1 -- 172.21.15.1:0/301882639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 msgr2=0x55ad021b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc7549bd640 1 --2- 172.21.15.1:0/301882639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad021b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc7551be640 1 --2- 172.21.15.1:0/301882639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 0x55ad021abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc7549bd640 1 -- 172.21.15.1:0/301882639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 msgr2=0x55ad021abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc7549bd640 1 --2- 172.21.15.1:0/301882639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 0x55ad021abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc7549bd640 1 -- 172.21.15.1:0/301882639 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad0219a0f0 con 0x55ad021902f0 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc7549bd640 1 --2- 172.21.15.1:0/301882639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad02197100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc740009850 tx=0x7fc740005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d262a3b54a4f6d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc73affd640 1 -- 172.21.15.1:0/301882639 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740048070 con 0x55ad021902f0 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc73affd640 1 -- 172.21.15.1:0/301882639 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc740005520 con 0x55ad021902f0 2024-09-16T00:29:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.845+0000 7fc73affd640 1 -- 172.21.15.1:0/301882639 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740035040 con 0x55ad021902f0 2024-09-16T00:29:09.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 -- 172.21.15.1:0/301882639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 msgr2=0x55ad02197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 --2- 172.21.15.1:0/301882639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad02197100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc740009850 tx=0x7fc740005b50 comp rx=0 tx=0).stop 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 -- 172.21.15.1:0/301882639 shutdown_connections 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 --2- 172.21.15.1:0/301882639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0219abb0 0x55ad021b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 --2- 172.21.15.1:0/301882639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad021902f0 0x55ad02197100 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 --2- 172.21.15.1:0/301882639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad0218f9d0 0x55ad021abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 -- 172.21.15.1:0/301882639 >> 172.21.15.1:0/301882639 conn(0x55ad0218bf60 msgr2=0x55ad0218c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 -- 172.21.15.1:0/301882639 shutdown_connections 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 -- 172.21.15.1:0/301882639 wait complete. 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 -- start start 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.846+0000 7fc7597fb740 1 Processor -- start 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0218f9d0 0x55ad021afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad021b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7597fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0219abb0 0x55ad021b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7597fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad021ac9f0 con 0x55ad0219abb0 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7597fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad021ac320 con 0x55ad021902f0 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7597fb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad021ad0c0 con 0x55ad0218f9d0 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7597fb740 10 client.0 _ll_register_callbacks cb 0x7fc74c00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7559bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0219abb0 0x55ad021b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7559bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0219abb0 0x55ad021b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54406/0 (socket says 172.21.15.1:54406) 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7559bf640 1 -- 172.21.15.1:0/800156868 learned_addr learned my addr 172.21.15.1:0/800156868 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7549bd640 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad021b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7551be640 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0218f9d0 0x55ad021afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7559bf640 1 -- 172.21.15.1:0/800156868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0218f9d0 msgr2=0x55ad021afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7559bf640 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0218f9d0 0x55ad021afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7559bf640 1 -- 172.21.15.1:0/800156868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 msgr2=0x55ad021b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:09.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7559bf640 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad021b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7559bf640 1 -- 172.21.15.1:0/800156868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad02213f30 con 0x55ad0219abb0 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7551be640 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0218f9d0 0x55ad021afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7549bd640 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad021b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc7559bf640 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0219abb0 0x55ad021b0720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc748009870 tx=0x7fc748009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a132abe886a05d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.847+0000 7fc738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc738ff9640 1 -- 172.21.15.1:0/800156868 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748012ea0 con 0x55ad0219abb0 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc738ff9640 1 -- 172.21.15.1:0/800156868 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc74800f950 con 0x55ad0219abb0 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc7597fb740 1 -- 172.21.15.1:0/800156868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc740005520 con 0x55ad0219abb0 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc738ff9640 1 -- 172.21.15.1:0/800156868 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc74803a050 con 0x55ad0219abb0 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc7597fb740 10 client.15432 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:09.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc7597fb740 1 -- 172.21.15.1:0/800156868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad022151f0 con 0x55ad0219abb0 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc738ff9640 1 -- 172.21.15.1:0/800156868 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc740005520 con 0x55ad0219abb0 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc738ff9640 1 -- 172.21.15.1:0/800156868 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc74800f040 con 0x55ad0219abb0 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc7597fb740 10 client.15432 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc7597fb740 10 client.15432 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc7597fb740 20 client.15432 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc7597fb740 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad02219aa0 0x55ad022187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc7597fb740 4 client.15432 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.848+0000 7fc7597fb740 1 -- 172.21.15.1:0/800156868 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "status"}) -- 0x55ad020fdb10 con 0x55ad02219aa0 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7551be640 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad02219aa0 0x55ad022187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc738ff9640 10 client.15432 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:09.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7551be640 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad02219aa0 0x55ad022187b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b16dc58e798c69 server_cookie=3fda6d2a3b011295 in_seq=0 out_seq=0 2024-09-16T00:29:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc738ff9640 1 -- 172.21.15.1:0/800156868 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ad020fdb10 con 0x55ad02219aa0 2024-09-16T00:29:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc738ff9640 10 client.15432 handle_command_reply: tid=0 2024-09-16T00:29:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7597fb740 1 client.15432 shutdown 2024-09-16T00:29:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7597fb740 1 -- 172.21.15.1:0/800156868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0219abb0 msgr2=0x55ad021b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7597fb740 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0219abb0 0x55ad021b0720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc748009870 tx=0x7fc748009a00 comp rx=0 tx=0).stop 2024-09-16T00:29:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7597fb740 1 -- 172.21.15.1:0/800156868 shutdown_connections 2024-09-16T00:29:09.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7597fb740 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad02219aa0 0x55ad022187b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc738ff9640 0 client.15432 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7597fb740 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad0219abb0 0x55ad021b0720 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7597fb740 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad021902f0 0x55ad021b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7597fb740 1 --2- 172.21.15.1:0/800156868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad0218f9d0 0x55ad021afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.849+0000 7fc7597fb740 1 -- 172.21.15.1:0/800156868 >> 172.21.15.1:0/800156868 conn(0x55ad0218bf60 msgr2=0x55ad02198e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc7597fb740 1 -- 172.21.15.1:0/800156868 shutdown_connections 2024-09-16T00:29:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc7597fb740 1 -- 172.21.15.1:0/800156868 wait complete. 2024-09-16T00:29:09.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc7597fb740 20 client.15432 trim_cache size 0 max 16384 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "1226ed86-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24506, 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 170.578228575, 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-09-16T00:29:09.859 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 91, 2024-09-16T00:29:09.860 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 181.92258323999999 2024-09-16T00:29:09.860 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 -- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc73c0a3240 msgr2=0x7fc73c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc73c0a3240 0x7fc73c0a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc748022000 tx=0x7fc74801e000 comp rx=0 tx=0).stop 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 -- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750101de0 msgr2=0x7fc7501984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750101de0 0x7fc7501984b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00d770 tx=0x7fc74c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 -- 172.21.15.1:0/1309954471 shutdown_connections 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc750106f10 0x7fc75019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc73c0a3240 0x7fc73c0a5700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc750102790 0x7fc7501989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 --2- 172.21.15.1:0/1309954471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc750101de0 0x7fc7501984b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.850+0000 7fc757422640 1 -- 172.21.15.1:0/1309954471 >> 172.21.15.1:0/1309954471 conn(0x7fc7500fd8c0 msgr2=0x7fc7500fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.851+0000 7fc757422640 1 -- 172.21.15.1:0/1309954471 shutdown_connections 2024-09-16T00:29:09.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:09.851+0000 7fc757422640 1 -- 172.21.15.1:0/1309954471 wait complete. 2024-09-16T00:29:10.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:09 smithi019 ceph-mon[35186]: pgmap v373: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:10.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:09 smithi001 ceph-mon[30934]: pgmap v373: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:10.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:09 smithi153 ceph-mon[35538]: pgmap v373: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:10.803 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-16T00:29:12.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:11 smithi019 ceph-mon[35186]: pgmap v374: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:12.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:11 smithi001 ceph-mon[30934]: pgmap v374: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:12.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:11 smithi153 ceph-mon[35538]: pgmap v374: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:14.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:13 smithi019 ceph-mon[35186]: pgmap v375: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:13 smithi001 ceph-mon[30934]: pgmap v375: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:14.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:13 smithi153 ceph-mon[35538]: pgmap v375: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:16.004 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:15 smithi001 ceph-mon[30934]: pgmap v376: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-16T00:29:16.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:15 smithi019 ceph-mon[35186]: pgmap v376: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-16T00:29:16.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:15 smithi153 ceph-mon[35538]: pgmap v376: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-16T00:29:18.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:17 smithi019 ceph-mon[35186]: pgmap v377: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:18.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:17 smithi001 ceph-mon[30934]: pgmap v377: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:18.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:17 smithi153 ceph-mon[35538]: pgmap v377: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:20.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:19 smithi019 ceph-mon[35186]: pgmap v378: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:29:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:19 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:20.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:19 smithi001 ceph-mon[30934]: pgmap v378: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:29:20.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:19 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:20.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:19 smithi153 ceph-mon[35538]: pgmap v378: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:29:20.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:19 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:21.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:29:21.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:29:21.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:21.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:21.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:20 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:29:21.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:29:21.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:29:21.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:21.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:21.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:20 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:29:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:29:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:29:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:20 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:29:22.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:21 smithi001 ceph-mon[30934]: pgmap v379: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-16T00:29:22.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:21 smithi153 ceph-mon[35538]: pgmap v379: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-16T00:29:22.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:21 smithi019 ceph-mon[35186]: pgmap v379: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-16T00:29:23.951 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:29:24.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:23 smithi001 ceph-mon[30934]: pgmap v380: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:29:24.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:23 smithi153 ceph-mon[35538]: pgmap v380: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:29:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:24.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:23 smithi019 ceph-mon[35186]: pgmap v380: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:29:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 -- 172.21.15.1:0/2176356270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8100e20 msgr2=0x7f32a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 --2- 172.21.15.1:0/2176356270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8100e20 0x7f32a8101200 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f329c009900 tx=0x7f329c02a480 comp rx=0 tx=0).stop 2024-09-16T00:29:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 -- 172.21.15.1:0/2176356270 shutdown_connections 2024-09-16T00:29:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 --2- 172.21.15.1:0/2176356270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32a8102ef0 0x7f32a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 --2- 172.21.15.1:0/2176356270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32a8102570 0x7f32a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 --2- 172.21.15.1:0/2176356270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8100e20 0x7f32a8101200 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 -- 172.21.15.1:0/2176356270 >> 172.21.15.1:0/2176356270 conn(0x7f32a80fc7d0 msgr2=0x7f32a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 -- 172.21.15.1:0/2176356270 shutdown_connections 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 -- 172.21.15.1:0/2176356270 wait complete. 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.765+0000 7f32af065640 1 Processor -- start 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32af065640 1 -- start start 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32af065640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32a8100e20 0x7f32a8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32af065640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8102570 0x7f32a8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32af065640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32a8102ef0 0x7f32a819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32af065640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8110fd0 con 0x7f32a8102570 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32af065640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8110e50 con 0x7f32a8100e20 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32af065640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8111150 con 0x7f32a8102ef0 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32a7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8102570 0x7f32a8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32a7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8102570 0x7f32a8198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47698/0 (socket says 172.21.15.1:47698) 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32a7fff640 1 -- 172.21.15.1:0/2251588310 learned_addr learned my addr 172.21.15.1:0/2251588310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32ace01640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32a8100e20 0x7f32a8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32ad602640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32a8102ef0 0x7f32a819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32a7fff640 1 -- 172.21.15.1:0/2251588310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32a8102ef0 msgr2=0x7f32a819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32a7fff640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32a8102ef0 0x7f32a819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32a7fff640 1 -- 172.21.15.1:0/2251588310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32a8100e20 msgr2=0x7f32a8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32a7fff640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32a8100e20 0x7f32a8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.766+0000 7f32a7fff640 1 -- 172.21.15.1:0/2251588310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a819d4d0 con 0x7f32a8102570 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.767+0000 7f32a7fff640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8102570 0x7f32a8198a60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f329400e9f0 tx=0x7f329400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.767+0000 7f32a5ffb640 1 -- 172.21.15.1:0/2251588310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400cd80 con 0x7f32a8102570 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.767+0000 7f32a5ffb640 1 -- 172.21.15.1:0/2251588310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3294010910 con 0x7f32a8102570 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.767+0000 7f32af065640 1 -- 172.21.15.1:0/2251588310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a819d7c0 con 0x7f32a8102570 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.767+0000 7f32a5ffb640 1 -- 172.21.15.1:0/2251588310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294018990 con 0x7f32a8102570 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.767+0000 7f32af065640 1 -- 172.21.15.1:0/2251588310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81a50a0 con 0x7f32a8102570 2024-09-16T00:29:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32a5ffb640 1 -- 172.21.15.1:0/2251588310 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3294010ac0 con 0x7f32a8102570 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32b143e740 1 Processor -- start 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32b143e740 1 -- start start 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32b143e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b844a0 0x55d5d2b84880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32b143e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32b143e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 0x55d5d2b91a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32b143e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d5d2aea2e0 con 0x55d5d2b84dc0 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32b143e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d5d2a91ce0 con 0x55d5d2b8f680 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32b143e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5d2ae62a0 con 0x55d5d2b844a0 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32ace01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32ace01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47700/0 (socket says 172.21.15.1:47700) 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32ace01640 1 -- 172.21.15.1:0/1393145990 learned_addr learned my addr 172.21.15.1:0/1393145990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32ad602640 1 --2- 172.21.15.1:0/1393145990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b844a0 0x55d5d2b84880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.770+0000 7f32a7fff640 1 --2- 172.21.15.1:0/1393145990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 0x55d5d2b91a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32a5ffb640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32980a3240 0x7f32980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32ad602640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32980a3240 0x7f32980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32ace01640 1 -- 172.21.15.1:0/1393145990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b844a0 msgr2=0x55d5d2b84880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32ace01640 1 --2- 172.21.15.1:0/1393145990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b844a0 0x55d5d2b84880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32ace01640 1 -- 172.21.15.1:0/1393145990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 msgr2=0x55d5d2b91a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32ace01640 1 --2- 172.21.15.1:0/1393145990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 0x55d5d2b91a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32ace01640 1 -- 172.21.15.1:0/1393145990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d2a76440 con 0x55d5d2b84dc0 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32a5ffb640 1 -- 172.21.15.1:0/2251588310 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f32940d0130 con 0x7f32a8102570 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32ace01640 1 --2- 172.21.15.1:0/1393145990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8f140 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f329c009230 tx=0x7f329c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1477f02b4b1d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f32ad602640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32980a3240 0x7f32980a5700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32a0005fd0 tx=0x7f32a0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f3292ffd640 1 -- 172.21.15.1:0/1393145990 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c035040 con 0x55d5d2b84dc0 2024-09-16T00:29:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f3292ffd640 1 -- 172.21.15.1:0/1393145990 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c005520 con 0x55d5d2b84dc0 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.771+0000 7f3292ffd640 1 -- 172.21.15.1:0/1393145990 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0035f0 con 0x55d5d2b84dc0 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.772+0000 7f32b143e740 1 -- 172.21.15.1:0/1393145990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 msgr2=0x55d5d2b8f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.772+0000 7f32b143e740 1 --2- 172.21.15.1:0/1393145990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8f140 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f329c009230 tx=0x7f329c004040 comp rx=0 tx=0).stop 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.772+0000 7f32b143e740 1 -- 172.21.15.1:0/1393145990 shutdown_connections 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.772+0000 7f32b143e740 1 --2- 172.21.15.1:0/1393145990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 0x55d5d2b91a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.772+0000 7f32b143e740 1 --2- 172.21.15.1:0/1393145990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8f140 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.772+0000 7f32b143e740 1 --2- 172.21.15.1:0/1393145990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b844a0 0x55d5d2b84880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.772+0000 7f32b143e740 1 -- 172.21.15.1:0/1393145990 >> 172.21.15.1:0/1393145990 conn(0x55d5d2b80a30 msgr2=0x55d5d2b80e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.772+0000 7f32b143e740 1 -- 172.21.15.1:0/1393145990 shutdown_connections 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.772+0000 7f32b143e740 1 -- 172.21.15.1:0/1393145990 wait complete. 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.773+0000 7f32b143e740 1 -- start start 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.773+0000 7f32b143e740 1 Processor -- start 2024-09-16T00:29:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.773+0000 7f32b143e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b8ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.773+0000 7f32b143e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.773+0000 7f32b143e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 0x55d5d2b8c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.773+0000 7f32b143e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d5d2b92250 con 0x55d5d2b844a0 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.773+0000 7f32b143e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d5d2aa5050 con 0x55d5d2b84dc0 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.773+0000 7f32b143e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5d2b923d0 con 0x55d5d2b8f680 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32b143e740 10 client.0 _ll_register_callbacks cb 0x55d5d2ae5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ace01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b8ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ace01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b8ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47710/0 (socket says 172.21.15.1:47710) 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ace01640 1 -- 172.21.15.1:0/2946030978 learned_addr learned my addr 172.21.15.1:0/2946030978 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32a7fff640 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ad602640 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 0x55d5d2b8c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ace01640 1 -- 172.21.15.1:0/2946030978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 msgr2=0x55d5d2b8c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ace01640 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 0x55d5d2b8c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ace01640 1 -- 172.21.15.1:0/2946030978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b84dc0 msgr2=0x55d5d2b8c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ace01640 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ace01640 1 -- 172.21.15.1:0/2946030978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d285b3c0 con 0x55d5d2b844a0 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32ace01640 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b8ef60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f329c006010 tx=0x7f329c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05d04dd74643728 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f3287fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f3287fff640 1 -- 172.21.15.1:0/2946030978 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c053060 con 0x55d5d2b844a0 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f32b143e740 1 -- 172.21.15.1:0/2946030978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c005520 con 0x55d5d2b844a0 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.774+0000 7f3287fff640 1 -- 172.21.15.1:0/2946030978 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f329c004fe0 con 0x55d5d2b844a0 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f3287fff640 1 -- 172.21.15.1:0/2946030978 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f329c05a050 con 0x55d5d2b844a0 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f32b143e740 10 client.15444 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f32b143e740 1 -- 172.21.15.1:0/2946030978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5d2ba1fd0 con 0x55d5d2b844a0 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f3287fff640 1 -- 172.21.15.1:0/2946030978 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c005520 con 0x55d5d2b844a0 2024-09-16T00:29:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f3287fff640 1 -- 172.21.15.1:0/2946030978 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f329c031540 con 0x55d5d2b844a0 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f32b143e740 10 client.15444 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f32b143e740 10 client.15444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f32b143e740 20 client.15444 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f32b143e740 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5d2ba3320 0x55d5d2ba5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f32b143e740 4 client.15444 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f32b143e740 1 -- 172.21.15.1:0/2946030978 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5d2af7850 con 0x55d5d2ba3320 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.775+0000 7f32a7fff640 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5d2ba3320 0x55d5d2ba5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.776+0000 7f32a7fff640 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5d2ba3320 0x55d5d2ba5710 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe30ac94e265f8f1 server_cookie=abd7b63d0e9ea217 in_seq=0 out_seq=0 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.776+0000 7f3287fff640 10 client.15444 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f3287fff640 1 -- 172.21.15.1:0/2946030978 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5d2af7850 con 0x55d5d2ba3320 2024-09-16T00:29:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f3287fff640 10 client.15444 handle_command_reply: tid=0 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 client.15444 shutdown 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 -- 172.21.15.1:0/2946030978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 msgr2=0x55d5d2b8ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b8ef60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f329c006010 tx=0x7f329c009260 comp rx=0 tx=0).stop 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 -- 172.21.15.1:0/2946030978 shutdown_connections 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5d2ba3320 0x55d5d2ba5710 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f3287fff640 0 client.15444 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 0x55d5d2b8c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b8c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 --2- 172.21.15.1:0/2946030978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b8ef60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 -- 172.21.15.1:0/2946030978 >> 172.21.15.1:0/2946030978 conn(0x55d5d2b80a30 msgr2=0x55d5d2b86710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 -- 172.21.15.1:0/2946030978 shutdown_connections 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 1 -- 172.21.15.1:0/2946030978 wait complete. 2024-09-16T00:29:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.777+0000 7f32b143e740 20 client.15444 trim_cache size 0 max 16384 2024-09-16T00:29:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.798+0000 7f32b143e740 1 Processor -- start 2024-09-16T00:29:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.798+0000 7f32b143e740 1 -- start start 2024-09-16T00:29:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.798+0000 7f32b143e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.798+0000 7f32b143e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b87900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.798+0000 7f32b143e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 0x55d5d2b7fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.798+0000 7f32b143e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81af430 con 0x55d5d2b844a0 2024-09-16T00:29:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.798+0000 7f32b143e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81acd00 con 0x55d5d2b8f680 2024-09-16T00:29:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.798+0000 7f32b143e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81aab50 con 0x55d5d2b84dc0 2024-09-16T00:29:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ace01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ace01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b9b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47712/0 (socket says 172.21.15.1:47712) 2024-09-16T00:29:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ace01640 1 -- 172.21.15.1:0/3757590301 learned_addr learned my addr 172.21.15.1:0/3757590301 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ad602640 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 0x55d5d2b7fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32a7fff640 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b87900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ace01640 1 -- 172.21.15.1:0/3757590301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b84dc0 msgr2=0x55d5d2b87900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ace01640 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b87900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ace01640 1 -- 172.21.15.1:0/3757590301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 msgr2=0x55d5d2b7fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ace01640 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 0x55d5d2b7fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ace01640 1 -- 172.21.15.1:0/3757590301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d2b83c70 con 0x55d5d2b844a0 2024-09-16T00:29:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ad602640 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 0x55d5d2b7fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32a7fff640 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b87900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32ace01640 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b9b4b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f329c005b90 tx=0x7f329c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f63466f7fbf253b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32927fc640 1 -- 172.21.15.1:0/3757590301 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c053070 con 0x55d5d2b844a0 2024-09-16T00:29:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32927fc640 1 -- 172.21.15.1:0/3757590301 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c04a070 con 0x55d5d2b844a0 2024-09-16T00:29:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.799+0000 7f32927fc640 1 -- 172.21.15.1:0/3757590301 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c035050 con 0x55d5d2b844a0 2024-09-16T00:29:24.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.800+0000 7f32b143e740 1 -- 172.21.15.1:0/3757590301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 msgr2=0x55d5d2b9b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.800+0000 7f32b143e740 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b9b4b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f329c005b90 tx=0x7f329c005e80 comp rx=0 tx=0).stop 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.800+0000 7f32b143e740 1 -- 172.21.15.1:0/3757590301 shutdown_connections 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.800+0000 7f32b143e740 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b8f680 0x55d5d2b7fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.800+0000 7f32b143e740 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b87900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.800+0000 7f32b143e740 1 --2- 172.21.15.1:0/3757590301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b844a0 0x55d5d2b9b4b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.800+0000 7f32b143e740 1 -- 172.21.15.1:0/3757590301 >> 172.21.15.1:0/3757590301 conn(0x55d5d2b80a30 msgr2=0x55d5d2b7f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.800+0000 7f32b143e740 1 -- 172.21.15.1:0/3757590301 shutdown_connections 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.800+0000 7f32b143e740 1 -- 172.21.15.1:0/3757590301 wait complete. 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32b143e740 1 -- start start 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32b143e740 1 Processor -- start 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32b143e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b844a0 0x55d5d2b92ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32b143e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32b143e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 0x55d5d2c0bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32b143e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81ab1c0 con 0x55d5d2b84dc0 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32b143e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81ac630 con 0x55d5d2b844a0 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32b143e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81ab890 con 0x55d5d2b8f680 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32b143e740 10 client.0 _ll_register_callbacks cb 0x7f32940983c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32a7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32a7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b930e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47716/0 (socket says 172.21.15.1:47716) 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32a7fff640 1 -- 172.21.15.1:0/3811353196 learned_addr learned my addr 172.21.15.1:0/3811353196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32ace01640 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b844a0 0x55d5d2b92ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.801+0000 7f32ad602640 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 0x55d5d2c0bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32a7fff640 1 -- 172.21.15.1:0/3811353196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 msgr2=0x55d5d2c0bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32a7fff640 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 0x55d5d2c0bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32a7fff640 1 -- 172.21.15.1:0/3811353196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b844a0 msgr2=0x55d5d2b92ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32a7fff640 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b844a0 0x55d5d2b92ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32a7fff640 1 -- 172.21.15.1:0/3811353196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d2c0c290 con 0x55d5d2b84dc0 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32ace01640 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b844a0 0x55d5d2b92ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32ad602640 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 0x55d5d2c0bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32a7fff640 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b930e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3294098cc0 tx=0x7f3294098470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2420baafb0fe6d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f3287fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f3287fff640 1 -- 172.21.15.1:0/3811353196 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294096e60 con 0x55d5d2b84dc0 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f3287fff640 1 -- 172.21.15.1:0/3811353196 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3294098960 con 0x55d5d2b84dc0 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32b143e740 1 -- 172.21.15.1:0/3811353196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c04a070 con 0x55d5d2b84dc0 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f3287fff640 1 -- 172.21.15.1:0/3811353196 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f329409c2f0 con 0x55d5d2b84dc0 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32b143e740 10 client.15450 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.802+0000 7f32b143e740 1 -- 172.21.15.1:0/3811353196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5d2c0d4f0 con 0x55d5d2b84dc0 2024-09-16T00:29:24.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f3287fff640 1 -- 172.21.15.1:0/3811353196 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c04a070 con 0x55d5d2b84dc0 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f3287fff640 1 -- 172.21.15.1:0/3811353196 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f32940d20b0 con 0x55d5d2b84dc0 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f32b143e740 10 client.15450 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f32b143e740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f32b143e740 20 client.15450 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f32b143e740 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5d2ba3320 0x55d5d2c0fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f32b143e740 4 client.15450 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f32b143e740 1 -- 172.21.15.1:0/3811353196 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d5d2b8cae0 con 0x55d5d2ba3320 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f32ace01640 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5d2ba3320 0x55d5d2c0fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f32ace01640 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5d2ba3320 0x55d5d2c0fc90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2399ca4e0077f9 server_cookie=829c883a5af76048 in_seq=0 out_seq=0 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.803+0000 7f3287fff640 10 client.15450 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f3287fff640 1 -- 172.21.15.1:0/3811353196 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d5d2b8cae0 con 0x55d5d2ba3320 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f3287fff640 10 client.15450 handle_command_reply: tid=0 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 client.15450 shutdown 2024-09-16T00:29:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 -- 172.21.15.1:0/3811353196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 msgr2=0x55d5d2b930e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b930e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3294098cc0 tx=0x7f3294098470 comp rx=0 tx=0).stop 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 -- 172.21.15.1:0/3811353196 shutdown_connections 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5d2ba3320 0x55d5d2c0fc90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5d2b8f680 0x55d5d2c0bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f3287fff640 0 client.15450 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5d2b84dc0 0x55d5d2b930e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 --2- 172.21.15.1:0/3811353196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5d2b844a0 0x55d5d2b92ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 -- 172.21.15.1:0/3811353196 >> 172.21.15.1:0/3811353196 conn(0x55d5d2b80a30 msgr2=0x55d5d2b86f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 -- 172.21.15.1:0/3811353196 shutdown_connections 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 1 -- 172.21.15.1:0/3811353196 wait complete. 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.804+0000 7f32b143e740 20 client.15450 trim_cache size 0 max 16384 2024-09-16T00:29:24.812 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:29:24.813 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:29:24.813 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:29:24.813 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:29:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 -- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32980a3240 msgr2=0x7f32980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32980a3240 0x7f32980a5700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32a0005fd0 tx=0x7f32a0004ab0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 -- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8102570 msgr2=0x7f32a8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8102570 0x7f32a8198a60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f329400e9f0 tx=0x7f329400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 -- 172.21.15.1:0/2251588310 shutdown_connections 2024-09-16T00:29:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f32980a3240 0x7f32980a5700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32a8102ef0 0x7f32a819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32a8102570 0x7f32a8198a60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 --2- 172.21.15.1:0/2251588310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32a8100e20 0x7f32a8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 -- 172.21.15.1:0/2251588310 >> 172.21.15.1:0/2251588310 conn(0x7f32a80fc7d0 msgr2=0x7f32a806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 -- 172.21.15.1:0/2251588310 shutdown_connections 2024-09-16T00:29:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:24.805+0000 7f32af065640 1 -- 172.21.15.1:0/2251588310 wait complete. 2024-09-16T00:29:25.636 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:29:25.636 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:25.805 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.2 2024-09-16T00:29:25.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:25.918 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:25.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:25 smithi001 ceph-mon[30934]: pgmap v381: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:26.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:25 smithi153 ceph-mon[35538]: pgmap v381: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:25 smithi019 ceph-mon[35186]: pgmap v381: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:26.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.429+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/1628736563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41036e0 msgr2=0x7fa2f4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:26.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.429+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/1628736563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41036e0 0x7fa2f4107b90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009930 tx=0x7fa2e802a420 comp rx=0 tx=0).stop 2024-09-16T00:29:26.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.429+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/1628736563 shutdown_connections 2024-09-16T00:29:26.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.429+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/1628736563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f41082a0 0x7fa2f410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:26.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.429+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/1628736563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41036e0 0x7fa2f4107b90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.429+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/1628736563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2f4102d30 0x7fa2f4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.429+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/1628736563 >> 172.21.15.1:0/1628736563 conn(0x7fa2f40fc7d0 msgr2=0x7fa2f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.429+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/1628736563 shutdown_connections 2024-09-16T00:29:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.429+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/1628736563 wait complete. 2024-09-16T00:29:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f9fa0640 1 Processor -- start 2024-09-16T00:29:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f9fa0640 1 -- start start 2024-09-16T00:29:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f9fa0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f4102d30 0x7fa2f4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f9fa0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2f41036e0 0x7fa2f4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f9fa0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41082a0 0x7fa2f419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f9fa0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41120a0 con 0x7fa2f41082a0 2024-09-16T00:29:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f9fa0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4111f20 con 0x7fa2f41036e0 2024-09-16T00:29:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f9fa0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4112220 con 0x7fa2f4102d30 2024-09-16T00:29:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41082a0 0x7fa2f419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41082a0 0x7fa2f419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47740/0 (socket says 172.21.15.1:47740) 2024-09-16T00:29:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.430+0000 7fa2f3fff640 1 -- 172.21.15.1:0/2554711127 learned_addr learned my addr 172.21.15.1:0/2554711127 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f37fe640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f4102d30 0x7fa2f4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f2ffd640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2f41036e0 0x7fa2f4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f3fff640 1 -- 172.21.15.1:0/2554711127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f4102d30 msgr2=0x7fa2f4198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f3fff640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f4102d30 0x7fa2f4198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f3fff640 1 -- 172.21.15.1:0/2554711127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2f41036e0 msgr2=0x7fa2f4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f3fff640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2f41036e0 0x7fa2f4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f3fff640 1 -- 172.21.15.1:0/2554711127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f419d530 con 0x7fa2f41082a0 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f2ffd640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2f41036e0 0x7fa2f4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f3fff640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41082a0 0x7fa2f419ce50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00eaa0 tx=0x7fa2ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f0ff9640 1 -- 172.21.15.1:0/2554711127 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00cdb0 con 0x7fa2f41082a0 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f0ff9640 1 -- 172.21.15.1:0/2554711127 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2ec005020 con 0x7fa2f41082a0 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f419d820 con 0x7fa2f41082a0 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.431+0000 7fa2f0ff9640 1 -- 172.21.15.1:0/2554711127 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0116b0 con 0x7fa2f41082a0 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.432+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f41a5100 con 0x7fa2f41082a0 2024-09-16T00:29:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.433+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f4103110 con 0x7fa2f41082a0 2024-09-16T00:29:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.434+0000 7fa2f0ff9640 1 -- 172.21.15.1:0/2554711127 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa2ec005560 con 0x7fa2f41082a0 2024-09-16T00:29:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.434+0000 7fa2f0ff9640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa2dc0a32c0 0x7fa2dc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.435+0000 7fa2f37fe640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa2dc0a32c0 0x7fa2dc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.435+0000 7fa2f0ff9640 1 -- 172.21.15.1:0/2554711127 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa2ec0cff40 con 0x7fa2f41082a0 2024-09-16T00:29:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.435+0000 7fa2f37fe640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa2dc0a32c0 0x7fa2dc0a5780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005fd0 tx=0x7fa2e4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.439+0000 7fa2f0ff9640 1 -- 172.21.15.1:0/2554711127 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2ec016030 con 0x7fa2f41082a0 2024-09-16T00:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:26.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.670+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2f410a2b0 con 0x7fa2f41082a0 2024-09-16T00:29:26.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.671+0000 7fa2f0ff9640 1 -- 172.21.15.1:0/2554711127 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fa2ec09c860 con 0x7fa2f41082a0 2024-09-16T00:29:26.672 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:29:26.672 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:29:26.672 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:29:26.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa2dc0a32c0 msgr2=0x7fa2dc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa2dc0a32c0 0x7fa2dc0a5780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005fd0 tx=0x7fa2e4005850 comp rx=0 tx=0).stop 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41082a0 msgr2=0x7fa2f419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41082a0 0x7fa2f419ce50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00eaa0 tx=0x7fa2ec00ef60 comp rx=0 tx=0).stop 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 shutdown_connections 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f41082a0 0x7fa2f419ce50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2f41036e0 0x7fa2f4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa2dc0a32c0 0x7fa2dc0a5780 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 --2- 172.21.15.1:0/2554711127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f4102d30 0x7fa2f4198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 >> 172.21.15.1:0/2554711127 conn(0x7fa2f40fc7d0 msgr2=0x7fa2f4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 shutdown_connections 2024-09-16T00:29:26.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:26.674+0000 7fa2f9fa0640 1 -- 172.21.15.1:0/2554711127 wait complete. 2024-09-16T00:29:27.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:26 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2554711127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:27.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:26 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2554711127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:27.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:26 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2554711127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:28.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:27 smithi001 ceph-mon[30934]: pgmap v382: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:29:28.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:27 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3164318661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:28.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:27 smithi153 ceph-mon[35538]: pgmap v382: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:29:28.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:27 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3164318661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:28.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:27 smithi019 ceph-mon[35186]: pgmap v382: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:29:28.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:27 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3164318661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.584+0000 7f575e6f2640 1 -- 172.21.15.1:0/635329805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758100a80 msgr2=0x7f5758100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.584+0000 7f575e6f2640 1 --2- 172.21.15.1:0/635329805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758100a80 0x7f5758100ee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f574c009930 tx=0x7f574c02a420 comp rx=0 tx=0).stop 2024-09-16T00:29:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.584+0000 7f575e6f2640 1 -- 172.21.15.1:0/635329805 shutdown_connections 2024-09-16T00:29:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.584+0000 7f575e6f2640 1 --2- 172.21.15.1:0/635329805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5758101420 0x7f575810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.584+0000 7f575e6f2640 1 --2- 172.21.15.1:0/635329805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758100a80 0x7f5758100ee0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.584+0000 7f575e6f2640 1 --2- 172.21.15.1:0/635329805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5758106a80 0x7f5758106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.584+0000 7f575e6f2640 1 -- 172.21.15.1:0/635329805 >> 172.21.15.1:0/635329805 conn(0x7f57580fc7a0 msgr2=0x7f57580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.584+0000 7f575e6f2640 1 -- 172.21.15.1:0/635329805 shutdown_connections 2024-09-16T00:29:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.584+0000 7f575e6f2640 1 -- 172.21.15.1:0/635329805 wait complete. 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.585+0000 7f575e6f2640 1 Processor -- start 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.585+0000 7f575e6f2640 1 -- start start 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.585+0000 7f575e6f2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5758100a80 0x7f5758198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.585+0000 7f575e6f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758101420 0x7f5758198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f575e6f2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5758106a80 0x7f575819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f575e6f2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5758110ef0 con 0x7f5758101420 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f575e6f2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5758110d70 con 0x7f5758106a80 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f575e6f2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5758111070 con 0x7f5758100a80 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758101420 0x7f5758198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758101420 0x7f5758198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47766/0 (socket says 172.21.15.1:47766) 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57577fe640 1 -- 172.21.15.1:0/3164318661 learned_addr learned my addr 172.21.15.1:0/3164318661 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f575cc8f640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5758106a80 0x7f575819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f5757fff640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5758100a80 0x7f5758198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57577fe640 1 -- 172.21.15.1:0/3164318661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5758100a80 msgr2=0x7f5758198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57577fe640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5758100a80 0x7f5758198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57577fe640 1 -- 172.21.15.1:0/3164318661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5758106a80 msgr2=0x7f575819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57577fe640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5758106a80 0x7f575819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57577fe640 1 -- 172.21.15.1:0/3164318661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575819d3f0 con 0x7f5758101420 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f5757fff640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5758100a80 0x7f5758198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57577fe640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758101420 0x7f5758198980 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f574c02a940 tx=0x7f574c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.586+0000 7f57557fa640 1 -- 172.21.15.1:0/3164318661 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c03f070 con 0x7f5758101420 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.587+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575819d680 con 0x7f5758101420 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.587+0000 7f57557fa640 1 -- 172.21.15.1:0/3164318661 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f574c03b460 con 0x7f5758101420 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.587+0000 7f57557fa640 1 -- 172.21.15.1:0/3164318661 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c035030 con 0x7f5758101420 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.587+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57581a4fc0 con 0x7f5758101420 2024-09-16T00:29:28.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.588+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5758100ee0 con 0x7f5758101420 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.589+0000 7f57557fa640 1 -- 172.21.15.1:0/3164318661 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f574c042070 con 0x7f5758101420 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.590+0000 7f57557fa640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f57400a3240 0x7f57400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.590+0000 7f57557fa640 1 -- 172.21.15.1:0/3164318661 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f574c0f09c0 con 0x7f5758101420 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.590+0000 7f5757fff640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f57400a3240 0x7f57400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.590+0000 7f5757fff640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f57400a3240 0x7f57400a5700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f574800abe0 tx=0x7f574800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.593+0000 7f57557fa640 1 -- 172.21.15.1:0/3164318661 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f574c0bd2e0 con 0x7f5758101420 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.824+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f575810a050 con 0x7f5758101420 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.825+0000 7f57557fa640 1 -- 172.21.15.1:0/3164318661 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f574c0bcbe0 con 0x7f5758101420 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:29:28.590 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f57400a3240 msgr2=0x7f57400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f57400a3240 0x7f57400a5700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f574800abe0 tx=0x7f574800a5c0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758101420 msgr2=0x7f5758198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758101420 0x7f5758198980 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f574c02a940 tx=0x7f574c003c00 comp rx=0 tx=0).stop 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 shutdown_connections 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5758106a80 0x7f575819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5758101420 0x7f5758198980 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f57400a3240 0x7f57400a5700 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 --2- 172.21.15.1:0/3164318661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5758100a80 0x7f5758198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.827+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 >> 172.21.15.1:0/3164318661 conn(0x7f57580fc7a0 msgr2=0x7f575806a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:28.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.828+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 shutdown_connections 2024-09-16T00:29:28.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:27.828+0000 7f575e6f2640 1 -- 172.21.15.1:0/3164318661 wait complete. 2024-09-16T00:29:30.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:29 smithi153 ceph-mon[35538]: pgmap v383: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:29:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:29 smithi019 ceph-mon[35186]: pgmap v383: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:29:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:29 smithi001 ceph-mon[30934]: pgmap v383: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:29:31.117 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0' 2024-09-16T00:29:31.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0 2024-09-16T00:29:31.619 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:29:31.733 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:32.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:31 smithi001 ceph-mon[30934]: pgmap v384: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:29:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:31 smithi153 ceph-mon[35538]: pgmap v384: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:29:32.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.272+0000 7fabe0b6a640 1 -- 172.21.15.1:0/2310322082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102570 msgr2=0x7fabdc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.272+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/2310322082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102570 0x7fabdc1029b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009ca0 tx=0x7fabd002a950 comp rx=0 tx=0).stop 2024-09-16T00:29:32.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.272+0000 7fabe0b6a640 1 -- 172.21.15.1:0/2310322082 shutdown_connections 2024-09-16T00:29:32.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.272+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/2310322082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabdc102ef0 0x7fabdc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.272+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/2310322082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102570 0x7fabdc1029b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.272+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/2310322082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabdc100e20 0x7fabdc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.272+0000 7fabe0b6a640 1 -- 172.21.15.1:0/2310322082 >> 172.21.15.1:0/2310322082 conn(0x7fabdc0fc7d0 msgr2=0x7fabdc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.272+0000 7fabe0b6a640 1 -- 172.21.15.1:0/2310322082 shutdown_connections 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.272+0000 7fabe0b6a640 1 -- 172.21.15.1:0/2310322082 wait complete. 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabe0b6a640 1 Processor -- start 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabe0b6a640 1 -- start start 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabe0b6a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabdc100e20 0x7fabdc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabe0b6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabdc102570 0x7fabdc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabe0b6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102ef0 0x7fabdc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabe0b6a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc110fd0 con 0x7fabdc102ef0 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabe0b6a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc110e50 con 0x7fabdc102570 2024-09-16T00:29:32.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabe0b6a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc111150 con 0x7fabdc100e20 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102ef0 0x7fabdc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabd9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabdc102570 0x7fabdc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102ef0 0x7fabdc19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44074/0 (socket says 172.21.15.1:44074) 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabdad9d640 1 -- 172.21.15.1:0/1835395701 learned_addr learned my addr 172.21.15.1:0/1835395701 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.273+0000 7fabda59c640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabdc100e20 0x7fabdc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabdad9d640 1 -- 172.21.15.1:0/1835395701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabdc100e20 msgr2=0x7fabdc19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabdad9d640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabdc100e20 0x7fabdc19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabdad9d640 1 -- 172.21.15.1:0/1835395701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabdc102570 msgr2=0x7fabdc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabdad9d640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabdc102570 0x7fabdc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabdad9d640 1 -- 172.21.15.1:0/1835395701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc19f660 con 0x7fabdc102ef0 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabdc102570 0x7fabdc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabdad9d640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102ef0 0x7fabdc19ef80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d750 tx=0x7fabd400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabcb7fe640 1 -- 172.21.15.1:0/1835395701 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4017070 con 0x7fabdc102ef0 2024-09-16T00:29:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabe0b6a640 1 -- 172.21.15.1:0/1835395701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc19f950 con 0x7fabdc102ef0 2024-09-16T00:29:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabcb7fe640 1 -- 172.21.15.1:0/1835395701 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd4005500 con 0x7fabdc102ef0 2024-09-16T00:29:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabcb7fe640 1 -- 172.21.15.1:0/1835395701 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4004d90 con 0x7fabdc102ef0 2024-09-16T00:29:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.274+0000 7fabe0b6a640 1 -- 172.21.15.1:0/1835395701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc1a7230 con 0x7fabdc102ef0 2024-09-16T00:29:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.277+0000 7fabcb7fe640 1 -- 172.21.15.1:0/1835395701 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fabd4004f30 con 0x7fabdc102ef0 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.277+0000 7fabe2f43740 1 Processor -- start 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.278+0000 7fabcb7fe640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabc40a32c0 0x7fabc40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.278+0000 7fabcb7fe640 1 -- 172.21.15.1:0/1835395701 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fabd40cf670 con 0x7fabdc102ef0 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.278+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabc40a32c0 0x7fabc40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.278+0000 7fabe2f43740 1 -- start start 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.278+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a0ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.278+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a0b080 0x561d68a15400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a15940 0x561d68a17d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabe2f43740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561d689702a0 con 0x561d68a0b080 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabe2f43740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d68917b10 con 0x561d68a15940 2024-09-16T00:29:32.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabe2f43740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d6896c260 con 0x561d68a0a760 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a0b080 0x561d68a15400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a0b080 0x561d68a15400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44088/0 (socket says 172.21.15.1:44088) 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 -- 172.21.15.1:0/2265721854 learned_addr learned my addr 172.21.15.1:0/2265721854 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabdad9d640 1 --2- 172.21.15.1:0/2265721854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a15940 0x561d68a17d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabda59c640 1 --2- 172.21.15.1:0/2265721854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a0ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabc40a32c0 0x7fabc40a5780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fabd002af40 tx=0x7fabd0009940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 -- 172.21.15.1:0/2265721854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 msgr2=0x561d68a0ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/2265721854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a0ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 -- 172.21.15.1:0/2265721854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a15940 msgr2=0x561d68a17d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/2265721854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a15940 0x561d68a17d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 -- 172.21.15.1:0/2265721854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d686e1b90 con 0x561d68a0b080 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.279+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/2265721854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a0b080 0x561d68a15400 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fabd002f8d0 tx=0x7fabd0047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8e3fe955d355f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabc8ff9640 1 -- 172.21.15.1:0/2265721854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0034040 con 0x561d68a0b080 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabc8ff9640 1 -- 172.21.15.1:0/2265721854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd002fec0 con 0x561d68a0b080 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabc8ff9640 1 -- 172.21.15.1:0/2265721854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0048bb0 con 0x561d68a0b080 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabe2f43740 1 -- 172.21.15.1:0/2265721854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a0b080 msgr2=0x561d68a15400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabe2f43740 1 --2- 172.21.15.1:0/2265721854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a0b080 0x561d68a15400 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fabd002f8d0 tx=0x7fabd0047000 comp rx=0 tx=0).stop 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabe2f43740 1 -- 172.21.15.1:0/2265721854 shutdown_connections 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabe2f43740 1 --2- 172.21.15.1:0/2265721854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a15940 0x561d68a17d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabe2f43740 1 --2- 172.21.15.1:0/2265721854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a0b080 0x561d68a15400 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabe2f43740 1 --2- 172.21.15.1:0/2265721854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a0ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.280+0000 7fabe2f43740 1 -- 172.21.15.1:0/2265721854 >> 172.21.15.1:0/2265721854 conn(0x561d68a06cf0 msgr2=0x561d68a070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 -- 172.21.15.1:0/2265721854 shutdown_connections 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 -- 172.21.15.1:0/2265721854 wait complete. 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 -- start start 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 Processor -- start 2024-09-16T00:29:32.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0a760 0x561d68a12870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0b080 0x561d68a12db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561d68a18390 con 0x561d68a15940 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d688f86e0 con 0x561d68a0a760 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d68a18510 con 0x561d68a0b080 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.281+0000 7fabe2f43740 10 client.0 _ll_register_callbacks cb 0x561d6891ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabda59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0a760 0x561d68a12870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a132f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44104/0 (socket says 172.21.15.1:44104) 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabda59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0a760 0x561d68a12870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:55066/0 (socket says 172.21.15.1:55066) 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabdad9d640 1 -- 172.21.15.1:0/492164746 learned_addr learned my addr 172.21.15.1:0/492164746 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0b080 0x561d68a12db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabdad9d640 1 -- 172.21.15.1:0/492164746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0b080 msgr2=0x561d68a12db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabdad9d640 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0b080 0x561d68a12db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabdad9d640 1 -- 172.21.15.1:0/492164746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0a760 msgr2=0x561d68a12870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabdad9d640 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0a760 0x561d68a12870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabdad9d640 1 -- 172.21.15.1:0/492164746 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d68a218d0 con 0x561d68a15940 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabda59c640 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0a760 0x561d68a12870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabdad9d640 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a132f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fabd40ccd10 tx=0x7fabd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fedac4e99e2074e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabc1ffb640 1 -- 172.21.15.1:0/492164746 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4020040 con 0x561d68a15940 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabc1ffb640 1 -- 172.21.15.1:0/492164746 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fabd4009c60 con 0x561d68a15940 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabe2f43740 1 -- 172.21.15.1:0/492164746 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd002fec0 con 0x561d68a15940 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabc1ffb640 1 -- 172.21.15.1:0/492164746 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fabd409fbe0 con 0x561d68a15940 2024-09-16T00:29:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabe2f43740 10 client.15474 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.282+0000 7fabe2f43740 1 -- 172.21.15.1:0/492164746 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d68a242b0 con 0x561d68a15940 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.283+0000 7fabc1ffb640 1 -- 172.21.15.1:0/492164746 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd002fec0 con 0x561d68a15940 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.283+0000 7fabc1ffb640 1 -- 172.21.15.1:0/492164746 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fabd4097f50 con 0x561d68a15940 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.283+0000 7fabe2f43740 10 client.15474 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.283+0000 7fabe2f43740 10 client.15474 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.283+0000 7fabe2f43740 20 client.15474 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.283+0000 7fabe2f43740 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d68a25770 0x561d68a27bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.283+0000 7fabe2f43740 4 client.15474 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.283+0000 7fabe2f43740 1 -- 172.21.15.1:0/492164746 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d6897d9d0 con 0x561d68a25770 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.283+0000 7fabda59c640 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d68a25770 0x561d68a27bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.284+0000 7fabc1ffb640 10 client.15474 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.284+0000 7fabda59c640 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d68a25770 0x561d68a27bf0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c8a02683f2517f server_cookie=283c89479bbff1bd in_seq=0 out_seq=0 2024-09-16T00:29:32.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.284+0000 7fabc1ffb640 1 -- 172.21.15.1:0/492164746 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d6897d9d0 con 0x561d68a25770 2024-09-16T00:29:32.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.284+0000 7fabc1ffb640 10 client.15474 handle_command_reply: tid=0 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 client.15474 shutdown 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 -- 172.21.15.1:0/492164746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 msgr2=0x561d68a132f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a132f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fabd40ccd10 tx=0x7fabd4009bd0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 -- 172.21.15.1:0/492164746 shutdown_connections 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d68a25770 0x561d68a27bf0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a132f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0b080 0x561d68a12db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 --2- 172.21.15.1:0/492164746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0a760 0x561d68a12870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 -- 172.21.15.1:0/492164746 >> 172.21.15.1:0/492164746 conn(0x561d68a06cf0 msgr2=0x561d68a05e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabc1ffb640 0 client.15474 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 -- 172.21.15.1:0/492164746 shutdown_connections 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 1 -- 172.21.15.1:0/492164746 wait complete. 2024-09-16T00:29:32.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.285+0000 7fabe2f43740 20 client.15474 trim_cache size 0 max 16384 2024-09-16T00:29:32.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.303+0000 7fabe2f43740 1 Processor -- start 2024-09-16T00:29:32.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.303+0000 7fabe2f43740 1 -- start start 2024-09-16T00:29:32.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a0c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a2d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a2da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabe2f43740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1b38c0 con 0x561d68a15940 2024-09-16T00:29:32.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabe2f43740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1b1190 con 0x561d68a0b080 2024-09-16T00:29:32.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabe2f43740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1aefe0 con 0x561d68a0a760 2024-09-16T00:29:32.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a2da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a2da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44106/0 (socket says 172.21.15.1:44106) 2024-09-16T00:29:32.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabdad9d640 1 -- 172.21.15.1:0/4212312188 learned_addr learned my addr 172.21.15.1:0/4212312188 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:32.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabda59c640 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a0c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a2d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabdad9d640 1 -- 172.21.15.1:0/4212312188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 msgr2=0x561d68a0c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabdad9d640 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a0c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabdad9d640 1 -- 172.21.15.1:0/4212312188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 msgr2=0x561d68a2d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabdad9d640 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a2d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabdad9d640 1 -- 172.21.15.1:0/4212312188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d68a02260 con 0x561d68a15940 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.304+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a2d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.305+0000 7fabdad9d640 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a2da00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fabd409b5e0 tx=0x7fabd409b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659fef5437b4dac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.305+0000 7fabc3fff640 1 -- 172.21.15.1:0/4212312188 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4020050 con 0x561d68a15940 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.305+0000 7fabc3fff640 1 -- 172.21.15.1:0/4212312188 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd4009c60 con 0x561d68a15940 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.305+0000 7fabc3fff640 1 -- 172.21.15.1:0/4212312188 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4005220 con 0x561d68a15940 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.305+0000 7fabe2f43740 1 -- 172.21.15.1:0/4212312188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 msgr2=0x561d68a2da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.305+0000 7fabe2f43740 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a2da00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fabd409b5e0 tx=0x7fabd409b7e0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.306+0000 7fabe2f43740 1 -- 172.21.15.1:0/4212312188 shutdown_connections 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.306+0000 7fabe2f43740 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a2da00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.306+0000 7fabe2f43740 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a2d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.306+0000 7fabe2f43740 1 --2- 172.21.15.1:0/4212312188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a0c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.306+0000 7fabe2f43740 1 -- 172.21.15.1:0/4212312188 >> 172.21.15.1:0/4212312188 conn(0x561d68a06cf0 msgr2=0x561d68a0c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.306+0000 7fabe2f43740 1 -- 172.21.15.1:0/4212312188 shutdown_connections 2024-09-16T00:29:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.306+0000 7fabe2f43740 1 -- 172.21.15.1:0/4212312188 wait complete. 2024-09-16T00:29:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.306+0000 7fabe2f43740 1 -- start start 2024-09-16T00:29:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.306+0000 7fabe2f43740 1 Processor -- start 2024-09-16T00:29:32.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a19790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a19cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabe2f43740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a1a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabe2f43740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1af650 con 0x561d68a15940 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabe2f43740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1b0ac0 con 0x561d68a0b080 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabe2f43740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1afd20 con 0x561d68a0a760 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabe2f43740 10 client.0 _ll_register_callbacks cb 0x7fabd400ded0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a1a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabda59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a19790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a1a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44114/0 (socket says 172.21.15.1:44114) 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabdad9d640 1 -- 172.21.15.1:0/1525933421 learned_addr learned my addr 172.21.15.1:0/1525933421 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a19cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabdad9d640 1 -- 172.21.15.1:0/1525933421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 msgr2=0x561d68a19790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabdad9d640 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a19790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabdad9d640 1 -- 172.21.15.1:0/1525933421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 msgr2=0x561d68a19cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabdad9d640 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a19cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabdad9d640 1 -- 172.21.15.1:0/1525933421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d68a8f780 con 0x561d68a15940 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.307+0000 7fabd9d9b640 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a19cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.308+0000 7fabdad9d640 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a1a210 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fabd4012040 tx=0x7fabd4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0a1219e0306142 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.308+0000 7fabc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.308+0000 7fabc1ffb640 1 -- 172.21.15.1:0/1525933421 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4020070 con 0x561d68a15940 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.308+0000 7fabe2f43740 1 -- 172.21.15.1:0/1525933421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd4009c60 con 0x561d68a15940 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.308+0000 7fabc1ffb640 1 -- 172.21.15.1:0/1525933421 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fabd4005220 con 0x561d68a15940 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.308+0000 7fabc1ffb640 1 -- 172.21.15.1:0/1525933421 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fabd40a44a0 con 0x561d68a15940 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.308+0000 7fabe2f43740 10 client.15480 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:32.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.308+0000 7fabe2f43740 1 -- 172.21.15.1:0/1525933421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d68a8f780 con 0x561d68a15940 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.308+0000 7fabc1ffb640 1 -- 172.21.15.1:0/1525933421 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd4009c60 con 0x561d68a15940 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.309+0000 7fabc1ffb640 1 -- 172.21.15.1:0/1525933421 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fabd409ad30 con 0x561d68a15940 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.309+0000 7fabe2f43740 10 client.15480 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.309+0000 7fabe2f43740 10 client.15480 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.309+0000 7fabe2f43740 20 client.15480 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.309+0000 7fabe2f43740 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d68a90d00 0x561d68a930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.309+0000 7fabe2f43740 4 client.15480 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}] 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.309+0000 7fabe2f43740 1 -- 172.21.15.1:0/1525933421 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}) -- 0x561d68977d20 con 0x561d68a90d00 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.309+0000 7fabda59c640 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d68a90d00 0x561d68a930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.310+0000 7fabc1ffb640 10 client.15480 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:32.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.310+0000 7fabda59c640 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d68a90d00 0x561d68a930f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b17ac2611efe47c server_cookie=f8c7b8451d0aa5ea in_seq=0 out_seq=0 2024-09-16T00:29:32.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabc1ffb640 1 -- 172.21.15.1:0/1525933421 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561d68977d20 con 0x561d68a90d00 2024-09-16T00:29:32.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabc1ffb640 10 client.15480 handle_command_reply: tid=0 2024-09-16T00:29:32.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabe2f43740 1 client.15480 shutdown 2024-09-16T00:29:32.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabe2f43740 1 -- 172.21.15.1:0/1525933421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 msgr2=0x561d68a1a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabe2f43740 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a1a210 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fabd4012040 tx=0x7fabd4005bb0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabe2f43740 1 -- 172.21.15.1:0/1525933421 shutdown_connections 2024-09-16T00:29:32.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabe2f43740 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d68a90d00 0x561d68a930f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabe2f43740 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d68a15940 0x561d68a1a210 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabe2f43740 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d68a0b080 0x561d68a19cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabe2f43740 1 --2- 172.21.15.1:0/1525933421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d68a0a760 0x561d68a19790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabc1ffb640 0 client.15480 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:32.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.320+0000 7fabe2f43740 1 -- 172.21.15.1:0/1525933421 >> 172.21.15.1:0/1525933421 conn(0x561d68a06cf0 msgr2=0x561d68a0c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.321+0000 7fabe2f43740 1 -- 172.21.15.1:0/1525933421 shutdown_connections 2024-09-16T00:29:32.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.321+0000 7fabe2f43740 1 -- 172.21.15.1:0/1525933421 wait complete. 2024-09-16T00:29:32.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.321+0000 7fabe2f43740 20 client.15480 trim_cache size 0 max 16384 2024-09-16T00:29:32.323 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:29:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:29:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:29:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 2, 2024-09-16T00:29:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:29:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "8193ff70": { 2024-09-16T00:29:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 2, 2024-09-16T00:29:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:29:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 150.2, 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:32.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:32.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:32.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:32.325 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:29:32.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.321+0000 7fabe0b6a640 1 -- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabc40a32c0 msgr2=0x7fabc40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.321+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabc40a32c0 0x7fabc40a5780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fabd002af40 tx=0x7fabd0009940 comp rx=0 tx=0).stop 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.321+0000 7fabe0b6a640 1 -- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102ef0 msgr2=0x7fabdc19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.321+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102ef0 0x7fabdc19ef80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d750 tx=0x7fabd400dc10 comp rx=0 tx=0).stop 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.322+0000 7fabe0b6a640 1 -- 172.21.15.1:0/1835395701 shutdown_connections 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.322+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc102ef0 0x7fabdc19ef80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.322+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabdc102570 0x7fabdc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.322+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabc40a32c0 0x7fabc40a5780 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.322+0000 7fabe0b6a640 1 --2- 172.21.15.1:0/1835395701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabdc100e20 0x7fabdc19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.322+0000 7fabe0b6a640 1 -- 172.21.15.1:0/1835395701 >> 172.21.15.1:0/1835395701 conn(0x7fabdc0fc7d0 msgr2=0x7fabdc06a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.322+0000 7fabe0b6a640 1 -- 172.21.15.1:0/1835395701 shutdown_connections 2024-09-16T00:29:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:32.322+0000 7fabe0b6a640 1 -- 172.21.15.1:0/1835395701 wait complete. 2024-09-16T00:29:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:31 smithi019 ceph-mon[35186]: pgmap v384: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:29:33.280 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8193ff70', quiesce took 7.5 seconds. Will release after: 22.7 2024-09-16T00:29:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.938+0000 7ff45a237640 1 -- 172.21.15.1:0/2033185963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454102d90 msgr2=0x7ff454103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.938+0000 7ff45a237640 1 --2- 172.21.15.1:0/2033185963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454102d90 0x7ff454103170 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff444009930 tx=0x7ff44402a420 comp rx=0 tx=0).stop 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.938+0000 7ff45a237640 1 -- 172.21.15.1:0/2033185963 shutdown_connections 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.938+0000 7ff45a237640 1 --2- 172.21.15.1:0/2033185963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454108290 0x7ff45410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.938+0000 7ff45a237640 1 --2- 172.21.15.1:0/2033185963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff454103740 0x7ff454107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.938+0000 7ff45a237640 1 --2- 172.21.15.1:0/2033185963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454102d90 0x7ff454103170 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.938+0000 7ff45a237640 1 -- 172.21.15.1:0/2033185963 >> 172.21.15.1:0/2033185963 conn(0x7ff4540fc7d0 msgr2=0x7ff4540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.938+0000 7ff45a237640 1 -- 172.21.15.1:0/2033185963 shutdown_connections 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.938+0000 7ff45a237640 1 -- 172.21.15.1:0/2033185963 wait complete. 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.939+0000 7ff45a237640 1 Processor -- start 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.939+0000 7ff45a237640 1 -- start start 2024-09-16T00:29:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.939+0000 7ff45a237640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102d90 0x7ff4541985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.939+0000 7ff45a237640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff454103740 0x7ff454198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.939+0000 7ff45a237640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454108290 0x7ff45419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.939+0000 7ff45a237640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff454112090 con 0x7ff454108290 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.939+0000 7ff45a237640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff454111f10 con 0x7ff454102d90 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.939+0000 7ff45a237640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff454112210 con 0x7ff454103740 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff4537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102d90 0x7ff4541985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff453fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454108290 0x7ff45419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff453fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454108290 0x7ff45419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44130/0 (socket says 172.21.15.1:44130) 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff453fff640 1 -- 172.21.15.1:0/212559409 learned_addr learned my addr 172.21.15.1:0/212559409 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:33.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff452ffd640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff454103740 0x7ff454198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff453fff640 1 -- 172.21.15.1:0/212559409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff454103740 msgr2=0x7ff454198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff453fff640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff454103740 0x7ff454198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff453fff640 1 -- 172.21.15.1:0/212559409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102d90 msgr2=0x7ff4541985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff453fff640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102d90 0x7ff4541985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff453fff640 1 -- 172.21.15.1:0/212559409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45419d5a0 con 0x7ff454108290 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.940+0000 7ff453fff640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454108290 0x7ff45419cec0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009fd0 tx=0x7ff44c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.941+0000 7ff450ff9640 1 -- 172.21.15.1:0/212559409 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c017070 con 0x7ff454108290 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.941+0000 7ff45a237640 1 -- 172.21.15.1:0/212559409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45419d890 con 0x7ff454108290 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.941+0000 7ff45a237640 1 -- 172.21.15.1:0/212559409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff454100920 con 0x7ff454108290 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.941+0000 7ff450ff9640 1 -- 172.21.15.1:0/212559409 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff44c003e50 con 0x7ff454108290 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.941+0000 7ff450ff9640 1 -- 172.21.15.1:0/212559409 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0106c0 con 0x7ff454108290 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.943+0000 7ff450ff9640 1 -- 172.21.15.1:0/212559409 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff44c003970 con 0x7ff454108290 2024-09-16T00:29:33.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.944+0000 7ff450ff9640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff43c0a32c0 0x7ff43c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.944+0000 7ff45c610740 1 Processor -- start 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.944+0000 7ff45c610740 1 -- start start 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.944+0000 7ff4537fe640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff43c0a32c0 0x7ff43c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.944+0000 7ff450ff9640 1 -- 172.21.15.1:0/212559409 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff44c0cf8a0 con 0x7ff454108290 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.944+0000 7ff45c610740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 0x557b0a854090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.944+0000 7ff45c610740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a8545d0 0x557b0a854a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.944+0000 7ff45c610740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a867b80 0x557b0a869f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff45c610740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557b0a7ba2f0 con 0x557b0a867b80 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff45c610740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557b0a761cf0 con 0x557b0a855b40 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff45c610740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557b0a7b62b0 con 0x557b0a8545d0 2024-09-16T00:29:33.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a8545d0 0x557b0a854a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a8545d0 0x557b0a854a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:55148/0 (socket says 172.21.15.1:55148) 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff452ffd640 1 -- 172.21.15.1:0/3731476163 learned_addr learned my addr 172.21.15.1:0/3731476163 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff453fff640 1 --2- 172.21.15.1:0/3731476163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a867b80 0x557b0a869f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff453fff640 1 -- 172.21.15.1:0/3731476163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a8545d0 msgr2=0x557b0a854a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff453fff640 1 --2- 172.21.15.1:0/3731476163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a8545d0 0x557b0a854a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff453fff640 1 -- 172.21.15.1:0/3731476163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 msgr2=0x557b0a854090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff453fff640 1 --2- 172.21.15.1:0/3731476163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 0x557b0a854090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff453fff640 1 -- 172.21.15.1:0/3731476163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0a7463a0 con 0x557b0a867b80 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff4537fe640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff43c0a32c0 0x7ff43c0a5780 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff444009900 tx=0x7ff4440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.945+0000 7ff453fff640 1 --2- 172.21.15.1:0/3731476163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a867b80 0x557b0a869f70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff44c005c40 tx=0x7ff44c09b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca9de8af5fe43e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.946+0000 7ff439ffb640 1 -- 172.21.15.1:0/3731476163 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c020040 con 0x557b0a867b80 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.946+0000 7ff439ffb640 1 -- 172.21.15.1:0/3731476163 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff44c09bb70 con 0x557b0a867b80 2024-09-16T00:29:33.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.946+0000 7ff439ffb640 1 -- 172.21.15.1:0/3731476163 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c086110 con 0x557b0a867b80 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.946+0000 7ff45c610740 1 -- 172.21.15.1:0/3731476163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a867b80 msgr2=0x557b0a869f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.946+0000 7ff45c610740 1 --2- 172.21.15.1:0/3731476163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a867b80 0x557b0a869f70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff44c005c40 tx=0x7ff44c09b590 comp rx=0 tx=0).stop 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.946+0000 7ff45c610740 1 -- 172.21.15.1:0/3731476163 shutdown_connections 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.946+0000 7ff45c610740 1 --2- 172.21.15.1:0/3731476163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a867b80 0x557b0a869f70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.946+0000 7ff45c610740 1 --2- 172.21.15.1:0/3731476163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a8545d0 0x557b0a854a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.947+0000 7ff45c610740 1 --2- 172.21.15.1:0/3731476163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 0x557b0a854090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.947+0000 7ff45c610740 1 -- 172.21.15.1:0/3731476163 >> 172.21.15.1:0/3731476163 conn(0x557b0a81d310 msgr2=0x557b0a850c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.947+0000 7ff45c610740 1 -- 172.21.15.1:0/3731476163 shutdown_connections 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.947+0000 7ff45c610740 1 -- 172.21.15.1:0/3731476163 wait complete. 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.948+0000 7ff45c610740 1 -- start start 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.948+0000 7ff45c610740 1 Processor -- start 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.948+0000 7ff45c610740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a8545d0 0x557b0a867580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.948+0000 7ff45c610740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 0x557b0a864830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.948+0000 7ff45c610740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a864e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.948+0000 7ff45c610740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557b0a86a820 con 0x557b0a8545d0 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.948+0000 7ff45c610740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557b0a775060 con 0x557b0a855b40 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.948+0000 7ff45c610740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557b0a86a9a0 con 0x557b0a867b80 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.948+0000 7ff4537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a8545d0 0x557b0a867580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff45c610740 10 client.0 _ll_register_callbacks cb 0x557b0a4c88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff4537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a8545d0 0x557b0a867580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44150/0 (socket says 172.21.15.1:44150) 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff4537fe640 1 -- 172.21.15.1:0/155953466 learned_addr learned my addr 172.21.15.1:0/155953466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff453fff640 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a864e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff452ffd640 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 0x557b0a864830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff4537fe640 1 -- 172.21.15.1:0/155953466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 msgr2=0x557b0a864e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff4537fe640 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a864e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff4537fe640 1 -- 172.21.15.1:0/155953466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 msgr2=0x557b0a864830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff4537fe640 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 0x557b0a864830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff4537fe640 1 -- 172.21.15.1:0/155953466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b0a52b3c0 con 0x557b0a8545d0 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff452ffd640 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 0x557b0a864830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff4537fe640 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a8545d0 0x557b0a867580 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff444005ec0 tx=0x7ff444009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c2fd0ee5dfa39d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.949+0000 7ff432ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff432ffd640 1 -- 172.21.15.1:0/155953466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44404d020 con 0x557b0a8545d0 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff432ffd640 1 -- 172.21.15.1:0/155953466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff444003a00 con 0x557b0a8545d0 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff45c610740 1 -- 172.21.15.1:0/155953466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff44c09bb70 con 0x557b0a8545d0 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff432ffd640 1 -- 172.21.15.1:0/155953466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff444058050 con 0x557b0a8545d0 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff45c610740 10 client.15492 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff45c610740 1 -- 172.21.15.1:0/155953466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b0a876320 con 0x557b0a8545d0 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff432ffd640 1 -- 172.21.15.1:0/155953466 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4440319f0 con 0x557b0a8545d0 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff432ffd640 1 -- 172.21.15.1:0/155953466 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff4440544b0 con 0x557b0a8545d0 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff45c610740 10 client.15492 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff45c610740 10 client.15492 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.950+0000 7ff45c610740 20 client.15492 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.951+0000 7ff45c610740 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x557b0a8778a0 0x557b0a879cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.951+0000 7ff45c610740 4 client.15492 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.951+0000 7ff45c610740 1 -- 172.21.15.1:0/155953466 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff44c09bb70 con 0x557b0a8778a0 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.951+0000 7ff452ffd640 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x557b0a8778a0 0x557b0a879cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.951+0000 7ff452ffd640 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x557b0a8778a0 0x557b0a879cc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69e5793e09617aca server_cookie=755dfe3c0645f1fa in_seq=0 out_seq=0 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.951+0000 7ff432ffd640 10 client.15492 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:29:33.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.952+0000 7ff432ffd640 1 -- 172.21.15.1:0/155953466 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff44c09bb70 con 0x557b0a8778a0 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.952+0000 7ff432ffd640 10 client.15492 handle_command_reply: tid=0 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.952+0000 7ff45c610740 1 client.15492 shutdown 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.952+0000 7ff45c610740 1 -- 172.21.15.1:0/155953466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a8545d0 msgr2=0x557b0a867580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a8545d0 0x557b0a867580 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff444005ec0 tx=0x7ff444009260 comp rx=0 tx=0).stop 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 1 -- 172.21.15.1:0/155953466 shutdown_connections 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a864e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x557b0a8778a0 0x557b0a879cc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a855b40 0x557b0a864830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff432ffd640 0 client.15492 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 1 --2- 172.21.15.1:0/155953466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a8545d0 0x557b0a867580 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 1 -- 172.21.15.1:0/155953466 >> 172.21.15.1:0/155953466 conn(0x557b0a81d310 msgr2=0x557b0a856990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 1 -- 172.21.15.1:0/155953466 shutdown_connections 2024-09-16T00:29:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 1 -- 172.21.15.1:0/155953466 wait complete. 2024-09-16T00:29:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.953+0000 7ff45c610740 20 client.15492 trim_cache size 0 max 16384 2024-09-16T00:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff45c610740 1 Processor -- start 2024-09-16T00:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff45c610740 1 -- start start 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff45c610740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a8545d0 0x557b0a855300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff45c610740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a873af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff45c610740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a850700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff45c610740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff4400125e0 con 0x557b0a855b40 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff45c610740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff44000feb0 con 0x557b0a8545d0 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff45c610740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff44000dd00 con 0x557b0a867b80 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a873af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a873af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44166/0 (socket says 172.21.15.1:44166) 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff452ffd640 1 -- 172.21.15.1:0/362402550 learned_addr learned my addr 172.21.15.1:0/362402550 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff453fff640 1 --2- 172.21.15.1:0/362402550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a850700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff452ffd640 1 -- 172.21.15.1:0/362402550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 msgr2=0x557b0a850700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:33.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff452ffd640 1 --2- 172.21.15.1:0/362402550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a850700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff452ffd640 1 -- 172.21.15.1:0/362402550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a8545d0 msgr2=0x557b0a855300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:29:33.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff452ffd640 1 --2- 172.21.15.1:0/362402550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a8545d0 0x557b0a855300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.980+0000 7ff452ffd640 1 -- 172.21.15.1:0/362402550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0a86a2e0 con 0x557b0a855b40 2024-09-16T00:29:33.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.981+0000 7ff453fff640 1 --2- 172.21.15.1:0/362402550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a850700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:33.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.981+0000 7ff452ffd640 1 --2- 172.21.15.1:0/362402550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a873af0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x557b0a865e70 tx=0x7ff448010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13a85a70d51c9d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:33.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.981+0000 7ff4397fa640 1 -- 172.21.15.1:0/362402550 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448022030 con 0x557b0a855b40 2024-09-16T00:29:33.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.981+0000 7ff4397fa640 1 -- 172.21.15.1:0/362402550 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff448005500 con 0x557b0a855b40 2024-09-16T00:29:33.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.981+0000 7ff4397fa640 1 -- 172.21.15.1:0/362402550 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44801d750 con 0x557b0a855b40 2024-09-16T00:29:33.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.982+0000 7ff45c610740 1 -- 172.21.15.1:0/362402550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 msgr2=0x557b0a873af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.982+0000 7ff45c610740 1 --2- 172.21.15.1:0/362402550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a873af0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x557b0a865e70 tx=0x7ff448010800 comp rx=0 tx=0).stop 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.982+0000 7ff45c610740 1 -- 172.21.15.1:0/362402550 shutdown_connections 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.982+0000 7ff45c610740 1 --2- 172.21.15.1:0/362402550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a850700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.982+0000 7ff45c610740 1 --2- 172.21.15.1:0/362402550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a873af0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.982+0000 7ff45c610740 1 --2- 172.21.15.1:0/362402550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a8545d0 0x557b0a855300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.982+0000 7ff45c610740 1 -- 172.21.15.1:0/362402550 >> 172.21.15.1:0/362402550 conn(0x557b0a81d310 msgr2=0x557b0a84ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.982+0000 7ff45c610740 1 -- 172.21.15.1:0/362402550 shutdown_connections 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.982+0000 7ff45c610740 1 -- 172.21.15.1:0/362402550 wait complete. 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff45c610740 1 -- start start 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff45c610740 1 Processor -- start 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff45c610740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a8545d0 0x557b0a870150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff45c610740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a86b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff45c610740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a86b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff45c610740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff44000e370 con 0x557b0a855b40 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff45c610740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff44000f7e0 con 0x557b0a8545d0 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff45c610740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff44000ea40 con 0x557b0a867b80 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff45c610740 10 client.0 _ll_register_callbacks cb 0x7ff444040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a86b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a86b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44180/0 (socket says 172.21.15.1:44180) 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.983+0000 7ff452ffd640 1 -- 172.21.15.1:0/2151191773 learned_addr learned my addr 172.21.15.1:0/2151191773 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff4537fe640 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a8545d0 0x557b0a870150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff452ffd640 1 -- 172.21.15.1:0/2151191773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 msgr2=0x557b0a86b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff452ffd640 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a86b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff452ffd640 1 -- 172.21.15.1:0/2151191773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a8545d0 msgr2=0x557b0a870150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff452ffd640 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a8545d0 0x557b0a870150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff452ffd640 1 -- 172.21.15.1:0/2151191773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b0a86bf80 con 0x557b0a855b40 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff452ffd640 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a86b480 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff4480058c0 tx=0x7ff448004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44db7b253776d4ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff432ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff432ffd640 1 -- 172.21.15.1:0/2151191773 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44801d620 con 0x557b0a855b40 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.984+0000 7ff432ffd640 1 -- 172.21.15.1:0/2151191773 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff44801d7c0 con 0x557b0a855b40 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff45c610740 1 -- 172.21.15.1:0/2151191773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff448005500 con 0x557b0a855b40 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff432ffd640 1 -- 172.21.15.1:0/2151191773 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff448004ce0 con 0x557b0a855b40 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff45c610740 10 client.15498 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff45c610740 1 -- 172.21.15.1:0/2151191773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b0a8e2260 con 0x557b0a855b40 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff432ffd640 1 -- 172.21.15.1:0/2151191773 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff448005500 con 0x557b0a855b40 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff432ffd640 1 -- 172.21.15.1:0/2151191773 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff448027440 con 0x557b0a855b40 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff45c610740 10 client.15498 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff45c610740 10 client.15498 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:29:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff45c610740 20 client.15498 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff45c610740 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x557b0a8778a0 0x557b0a8e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff45c610740 4 client.15498 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.985+0000 7ff45c610740 1 -- 172.21.15.1:0/2151191773 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b0a7c7860 con 0x557b0a8778a0 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.986+0000 7ff4537fe640 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x557b0a8778a0 0x557b0a8e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.986+0000 7ff4537fe640 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x557b0a8778a0 0x557b0a8e48d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be9418fa5f9a3896 server_cookie=2adaea87694bd0ed in_seq=0 out_seq=0 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.986+0000 7ff432ffd640 10 client.15498 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.986+0000 7ff432ffd640 1 -- 172.21.15.1:0/2151191773 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b0a7c7860 con 0x557b0a8778a0 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.986+0000 7ff432ffd640 10 client.15498 handle_command_reply: tid=0 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.986+0000 7ff45c610740 1 client.15498 shutdown 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 -- 172.21.15.1:0/2151191773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 msgr2=0x557b0a86b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a86b480 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff4480058c0 tx=0x7ff448004930 comp rx=0 tx=0).stop 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 -- 172.21.15.1:0/2151191773 shutdown_connections 2024-09-16T00:29:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b0a867b80 0x557b0a86b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557b0a855b40 0x557b0a86b480 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x557b0a8778a0 0x557b0a8e48d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 --2- 172.21.15.1:0/2151191773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b0a8545d0 0x557b0a870150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 -- 172.21.15.1:0/2151191773 >> 172.21.15.1:0/2151191773 conn(0x557b0a81d310 msgr2=0x557b0a856110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff432ffd640 0 client.15498 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:29:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 -- 172.21.15.1:0/2151191773 shutdown_connections 2024-09-16T00:29:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 1 -- 172.21.15.1:0/2151191773 wait complete. 2024-09-16T00:29:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.987+0000 7ff45c610740 20 client.15498 trim_cache size 0 max 16384 2024-09-16T00:29:33.993 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:29:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 -- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff43c0a32c0 msgr2=0x7ff43c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff43c0a32c0 0x7ff43c0a5780 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff444009900 tx=0x7ff4440058d0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 -- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454108290 msgr2=0x7ff45419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454108290 0x7ff45419cec0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009fd0 tx=0x7ff44c00b260 comp rx=0 tx=0).stop 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 -- 172.21.15.1:0/212559409 shutdown_connections 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff454108290 0x7ff45419cec0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff43c0a32c0 0x7ff43c0a5780 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff454103740 0x7ff454198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 --2- 172.21.15.1:0/212559409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102d90 0x7ff4541985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.988+0000 7ff45a237640 1 -- 172.21.15.1:0/212559409 >> 172.21.15.1:0/212559409 conn(0x7ff4540fc7d0 msgr2=0x7ff454105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.989+0000 7ff45a237640 1 -- 172.21.15.1:0/212559409 shutdown_connections 2024-09-16T00:29:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:33.989+0000 7ff45a237640 1 -- 172.21.15.1:0/212559409 wait complete. 2024-09-16T00:29:34.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:33 smithi001 ceph-mon[30934]: pgmap v385: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:29:34.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:33 smithi153 ceph-mon[35538]: pgmap v385: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:29:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:33 smithi019 ceph-mon[35186]: pgmap v385: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:29:34.815 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:29:35.129 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:35.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.694+0000 7fd20ec63640 1 -- 172.21.15.1:0/13083892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2081009b0 msgr2=0x7fd208100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.694+0000 7fd20ec63640 1 --2- 172.21.15.1:0/13083892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2081009b0 0x7fd208100d90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009900 tx=0x7fd1fc02a410 comp rx=0 tx=0).stop 2024-09-16T00:29:35.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.694+0000 7fd20ec63640 1 -- 172.21.15.1:0/13083892 shutdown_connections 2024-09-16T00:29:35.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.694+0000 7fd20ec63640 1 --2- 172.21.15.1:0/13083892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20810e4d0 0x7fd2081108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.694+0000 7fd20ec63640 1 --2- 172.21.15.1:0/13083892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd208101360 0x7fd20810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.694+0000 7fd20ec63640 1 --2- 172.21.15.1:0/13083892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2081009b0 0x7fd208100d90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.694+0000 7fd20ec63640 1 -- 172.21.15.1:0/13083892 >> 172.21.15.1:0/13083892 conn(0x7fd2080fc7d0 msgr2=0x7fd2080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.694+0000 7fd20ec63640 1 -- 172.21.15.1:0/13083892 shutdown_connections 2024-09-16T00:29:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.694+0000 7fd20ec63640 1 -- 172.21.15.1:0/13083892 wait complete. 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd20ec63640 1 Processor -- start 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd20ec63640 1 -- start start 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd20ec63640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2081009b0 0x7fd2081a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd20ec63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd208101360 0x7fd2081a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd20ec63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20810e4d0 0x7fd20819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd20ec63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd208114230 con 0x7fd208101360 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd20ec63640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2081140b0 con 0x7fd20810e4d0 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd20ec63640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2081143b0 con 0x7fd2081009b0 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd207fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd208101360 0x7fd2081a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.695+0000 7fd207fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd208101360 0x7fd2081a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44200/0 (socket says 172.21.15.1:44200) 2024-09-16T00:29:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd207fff640 1 -- 172.21.15.1:0/3499919724 learned_addr learned my addr 172.21.15.1:0/3499919724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2081009b0 0x7fd2081a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd20d200640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20810e4d0 0x7fd20819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd207fff640 1 -- 172.21.15.1:0/3499919724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2081009b0 msgr2=0x7fd2081a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd207fff640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2081009b0 0x7fd2081a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd207fff640 1 -- 172.21.15.1:0/3499919724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20810e4d0 msgr2=0x7fd20819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd207fff640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20810e4d0 0x7fd20819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd207fff640 1 -- 172.21.15.1:0/3499919724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20819acc0 con 0x7fd208101360 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd20d200640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20810e4d0 0x7fd20819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd207fff640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd208101360 0x7fd2081a0910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400d770 tx=0x7fd1f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd205ffb640 1 -- 172.21.15.1:0/3499919724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4017070 con 0x7fd208101360 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd205ffb640 1 -- 172.21.15.1:0/3499919724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f4005500 con 0x7fd208101360 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd205ffb640 1 -- 172.21.15.1:0/3499919724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4004e00 con 0x7fd208101360 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.696+0000 7fd20ec63640 1 -- 172.21.15.1:0/3499919724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20819afb0 con 0x7fd208101360 2024-09-16T00:29:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.697+0000 7fd20ec63640 1 -- 172.21.15.1:0/3499919724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2081a72a0 con 0x7fd208101360 2024-09-16T00:29:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd21103c740 1 Processor -- start 2024-09-16T00:29:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd21103c740 1 -- start start 2024-09-16T00:29:35.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd21103c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37448090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd205ffb640 1 -- 172.21.15.1:0/3499919724 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd1f4004fa0 con 0x7fd208101360 2024-09-16T00:29:35.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd21103c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 0x560a37448a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd21103c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37453370 0x560a37455760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd20d200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37448090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd21103c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560a373ae2f0 con 0x560a37449b40 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd21103c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a37355cf0 con 0x560a37453370 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd21103c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560a373aa2b0 con 0x560a374485d0 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd20d200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37448090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44210/0 (socket says 172.21.15.1:44210) 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.700+0000 7fd20d200640 1 -- 172.21.15.1:0/3989899078 learned_addr learned my addr 172.21.15.1:0/3989899078 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 0x560a37448a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd20d200640 1 -- 172.21.15.1:0/3989899078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 msgr2=0x560a37448a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd20d200640 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 0x560a37448a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd20d200640 1 -- 172.21.15.1:0/3989899078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37453370 msgr2=0x560a37455760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd207fff640 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37453370 0x560a37455760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd20d200640 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37453370 0x560a37455760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd20d200640 1 -- 172.21.15.1:0/3989899078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3733a3a0 con 0x560a37449b40 2024-09-16T00:29:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd207fff640 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37453370 0x560a37455760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:35.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd205ffb640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1f80a32c0 0x7fd1f80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.701+0000 7fd20d200640 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37448090 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd200009fd0 tx=0x7fd20000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d70da354248d3d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.702+0000 7fd205ffb640 1 -- 172.21.15.1:0/3499919724 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd1f40cf6e0 con 0x7fd208101360 2024-09-16T00:29:35.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.702+0000 7fd1f2ffd640 1 -- 172.21.15.1:0/3989899078 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200015070 con 0x560a37449b40 2024-09-16T00:29:35.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.702+0000 7fd1f2ffd640 1 -- 172.21.15.1:0/3989899078 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd200011be0 con 0x560a37449b40 2024-09-16T00:29:35.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.702+0000 7fd1f2ffd640 1 -- 172.21.15.1:0/3989899078 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200010ca0 con 0x560a37449b40 2024-09-16T00:29:35.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.702+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1f80a32c0 0x7fd1f80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 -- 172.21.15.1:0/3989899078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 msgr2=0x560a37448090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1f80a32c0 0x7fd1f80a5780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009230 tx=0x7fd1fc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37448090 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd200009fd0 tx=0x7fd20000f7f0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 -- 172.21.15.1:0/3989899078 shutdown_connections 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37453370 0x560a37455760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 0x560a37448a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 --2- 172.21.15.1:0/3989899078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37448090 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 -- 172.21.15.1:0/3989899078 >> 172.21.15.1:0/3989899078 conn(0x560a37411310 msgr2=0x560a37444c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 -- 172.21.15.1:0/3989899078 shutdown_connections 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 -- 172.21.15.1:0/3989899078 wait complete. 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.703+0000 7fd21103c740 1 -- start start 2024-09-16T00:29:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd21103c740 1 Processor -- start 2024-09-16T00:29:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd21103c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 0x560a37452d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd21103c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 0x560a37450040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd21103c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37453370 0x560a37450650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd21103c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560a37456070 con 0x560a37453370 2024-09-16T00:29:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd21103c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a37369060 con 0x560a37449b40 2024-09-16T00:29:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd21103c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560a374561f0 con 0x560a374485d0 2024-09-16T00:29:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd21103c740 10 client.0 _ll_register_callbacks cb 0x560a370bc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd20d200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37453370 0x560a37450650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd20c9ff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 0x560a37452d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd20d200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37453370 0x560a37450650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44226/0 (socket says 172.21.15.1:44226) 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd20d200640 1 -- 172.21.15.1:0/99208302 learned_addr learned my addr 172.21.15.1:0/99208302 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.704+0000 7fd207fff640 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 0x560a37450040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.705+0000 7fd20d200640 1 -- 172.21.15.1:0/99208302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 msgr2=0x560a37452d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.705+0000 7fd20d200640 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 0x560a37452d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.705+0000 7fd20d200640 1 -- 172.21.15.1:0/99208302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 msgr2=0x560a37450040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.705+0000 7fd20d200640 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 0x560a37450040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.705+0000 7fd20d200640 1 -- 172.21.15.1:0/99208302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a3711f3c0 con 0x560a37453370 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.705+0000 7fd207fff640 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 0x560a37450040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.705+0000 7fd20d200640 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37453370 0x560a37450650 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd200009fa0 tx=0x7fd200010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b2b9875a9a5ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.705+0000 7fd1e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd1e7fff640 1 -- 172.21.15.1:0/99208302 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200029060 con 0x560a37453370 2024-09-16T00:29:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd1e7fff640 1 -- 172.21.15.1:0/99208302 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd200004c50 con 0x560a37453370 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd21103c740 1 -- 172.21.15.1:0/99208302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd200011be0 con 0x560a37453370 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd1e7fff640 1 -- 172.21.15.1:0/99208302 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd200030050 con 0x560a37453370 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd21103c740 10 client.15510 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd21103c740 1 -- 172.21.15.1:0/99208302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a37461ab0 con 0x560a37453370 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd1e7fff640 1 -- 172.21.15.1:0/99208302 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd200011be0 con 0x560a37453370 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd1e7fff640 1 -- 172.21.15.1:0/99208302 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd200005040 con 0x560a37453370 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd21103c740 10 client.15510 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.706+0000 7fd21103c740 10 client.15510 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.707+0000 7fd21103c740 20 client.15510 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.707+0000 7fd21103c740 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560a37462eb0 0x560a374652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.707+0000 7fd21103c740 4 client.15510 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.707+0000 7fd21103c740 1 -- 172.21.15.1:0/99208302 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a373bb860 con 0x560a37462eb0 2024-09-16T00:29:35.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.707+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560a37462eb0 0x560a374652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.708+0000 7fd1e7fff640 10 client.15510 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.708+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560a37462eb0 0x560a374652a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e81a2476785492d server_cookie=c71f76c9a8e4cf7c in_seq=0 out_seq=0 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.708+0000 7fd1e7fff640 1 -- 172.21.15.1:0/99208302 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a373bb860 con 0x560a37462eb0 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.708+0000 7fd1e7fff640 10 client.15510 handle_command_reply: tid=0 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 client.15510 shutdown 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 -- 172.21.15.1:0/99208302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37453370 msgr2=0x560a37450650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37453370 0x560a37450650 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd200009fa0 tx=0x7fd200010660 comp rx=0 tx=0).stop 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 -- 172.21.15.1:0/99208302 shutdown_connections 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560a37462eb0 0x560a374652a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37453370 0x560a37450650 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 0x560a37450040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 --2- 172.21.15.1:0/99208302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a374485d0 0x560a37452d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd1e7fff640 0 client.15510 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 -- 172.21.15.1:0/99208302 >> 172.21.15.1:0/99208302 conn(0x560a37411310 msgr2=0x560a3744a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 -- 172.21.15.1:0/99208302 shutdown_connections 2024-09-16T00:29:35.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 1 -- 172.21.15.1:0/99208302 wait complete. 2024-09-16T00:29:35.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.709+0000 7fd21103c740 20 client.15510 trim_cache size 0 max 16384 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd21103c740 1 Processor -- start 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd21103c740 1 -- start start 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd21103c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a374485d0 0x560a3745f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd21103c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37467ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd21103c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 0x560a37468420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd21103c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2081af540 con 0x560a37449b40 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd21103c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2081ace10 con 0x560a374485d0 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd21103c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2081aac60 con 0x560a37453370 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd207fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37467ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd207fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37467ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44234/0 (socket says 172.21.15.1:44234) 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd207fff640 1 -- 172.21.15.1:0/518674943 learned_addr learned my addr 172.21.15.1:0/518674943 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.728+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a374485d0 0x560a3745f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd20d200640 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 0x560a37468420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd207fff640 1 -- 172.21.15.1:0/518674943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 msgr2=0x560a37468420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd207fff640 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 0x560a37468420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd207fff640 1 -- 172.21.15.1:0/518674943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a374485d0 msgr2=0x560a3745f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd207fff640 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a374485d0 0x560a3745f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd207fff640 1 -- 172.21.15.1:0/518674943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a37450fe0 con 0x560a37449b40 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a374485d0 0x560a3745f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd207fff640 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37467ee0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd1f409b890 tx=0x7fd1f40048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd64589c5861826 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd1f27fc640 1 -- 172.21.15.1:0/518674943 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4020040 con 0x560a37449b40 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd1f27fc640 1 -- 172.21.15.1:0/518674943 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f409c000 con 0x560a37449b40 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.729+0000 7fd1f27fc640 1 -- 172.21.15.1:0/518674943 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4099c00 con 0x560a37449b40 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.730+0000 7fd21103c740 1 -- 172.21.15.1:0/518674943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 msgr2=0x560a37467ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.730+0000 7fd21103c740 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37467ee0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd1f409b890 tx=0x7fd1f40048a0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.730+0000 7fd21103c740 1 -- 172.21.15.1:0/518674943 shutdown_connections 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.730+0000 7fd21103c740 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 0x560a37468420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.730+0000 7fd21103c740 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a37449b40 0x560a37467ee0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.730+0000 7fd21103c740 1 --2- 172.21.15.1:0/518674943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a374485d0 0x560a3745f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.730+0000 7fd21103c740 1 -- 172.21.15.1:0/518674943 >> 172.21.15.1:0/518674943 conn(0x560a37411310 msgr2=0x560a37450000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.730+0000 7fd21103c740 1 -- 172.21.15.1:0/518674943 shutdown_connections 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.730+0000 7fd21103c740 1 -- 172.21.15.1:0/518674943 wait complete. 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd21103c740 1 -- start start 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd21103c740 1 Processor -- start 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd21103c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a374485d0 0x560a37464ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd21103c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 0x560a37465410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd21103c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 0x560a37465950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd21103c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2081ab2d0 con 0x560a374485d0 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd21103c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2081ac740 con 0x560a37449b40 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd21103c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2081ab9a0 con 0x560a37453370 2024-09-16T00:29:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd21103c740 10 client.0 _ll_register_callbacks cb 0x7fd200009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd20c9ff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a374485d0 0x560a37464ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd20c9ff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a374485d0 0x560a37464ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44244/0 (socket says 172.21.15.1:44244) 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.731+0000 7fd20c9ff640 1 -- 172.21.15.1:0/363680575 learned_addr learned my addr 172.21.15.1:0/363680575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd207fff640 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 0x560a37465410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd20d200640 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 0x560a37465950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd20c9ff640 1 -- 172.21.15.1:0/363680575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 msgr2=0x560a37465950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 0x560a37465950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd20c9ff640 1 -- 172.21.15.1:0/363680575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 msgr2=0x560a37465410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 0x560a37465410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd20c9ff640 1 -- 172.21.15.1:0/363680575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a374cba30 con 0x560a374485d0 2024-09-16T00:29:35.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd20c9ff640 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a374485d0 0x560a37464ed0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc02a930 tx=0x7fd1fc03e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e6813132c12a25 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd1e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd1e7fff640 1 -- 172.21.15.1:0/363680575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc035040 con 0x560a374485d0 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd1e7fff640 1 -- 172.21.15.1:0/363680575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd1fc003a00 con 0x560a374485d0 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.732+0000 7fd21103c740 1 -- 172.21.15.1:0/363680575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1f409c000 con 0x560a374485d0 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd1e7fff640 1 -- 172.21.15.1:0/363680575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd1fc062050 con 0x560a374485d0 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd21103c740 10 client.15516 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd21103c740 1 -- 172.21.15.1:0/363680575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a374ccd50 con 0x560a374485d0 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd1e7fff640 1 -- 172.21.15.1:0/363680575 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f409c000 con 0x560a374485d0 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd1e7fff640 1 -- 172.21.15.1:0/363680575 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd1fc04a430 con 0x560a374485d0 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd21103c740 10 client.15516 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd21103c740 10 client.15516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd21103c740 20 client.15516 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:35.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd21103c740 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560a374cd470 0x560a374cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd21103c740 4 client.15516 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd21103c740 1 -- 172.21.15.1:0/363680575 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a373b5d70 con 0x560a374cd470 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.733+0000 7fd207fff640 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560a374cd470 0x560a374cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd1e7fff640 10 client.15516 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd207fff640 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560a374cd470 0x560a374cf860 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b330b84b3a66ad52 server_cookie=e19147ff0bda0639 in_seq=0 out_seq=0 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd1e7fff640 1 -- 172.21.15.1:0/363680575 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a373b5d70 con 0x560a374cd470 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd1e7fff640 10 client.15516 handle_command_reply: tid=0 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd21103c740 1 client.15516 shutdown 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd21103c740 1 -- 172.21.15.1:0/363680575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a374485d0 msgr2=0x560a37464ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd21103c740 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a374485d0 0x560a37464ed0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc02a930 tx=0x7fd1fc03e070 comp rx=0 tx=0).stop 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd21103c740 1 -- 172.21.15.1:0/363680575 shutdown_connections 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd21103c740 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a37453370 0x560a37465950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd21103c740 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a37449b40 0x560a37465410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd21103c740 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560a374cd470 0x560a374cf860 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd21103c740 1 --2- 172.21.15.1:0/363680575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a374485d0 0x560a37464ed0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.734+0000 7fd21103c740 1 -- 172.21.15.1:0/363680575 >> 172.21.15.1:0/363680575 conn(0x560a37411310 msgr2=0x560a3744f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.735+0000 7fd1e7fff640 0 client.15516 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:29:35.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.735+0000 7fd21103c740 1 -- 172.21.15.1:0/363680575 shutdown_connections 2024-09-16T00:29:35.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.735+0000 7fd21103c740 1 -- 172.21.15.1:0/363680575 wait complete. 2024-09-16T00:29:35.739 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:29:35.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.735+0000 7fd21103c740 20 client.15516 trim_cache size 0 max 16384 2024-09-16T00:29:35.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 -- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1f80a32c0 msgr2=0x7fd1f80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1f80a32c0 0x7fd1f80a5780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009230 tx=0x7fd1fc0058d0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 -- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd208101360 msgr2=0x7fd2081a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd208101360 0x7fd2081a0910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400d770 tx=0x7fd1f400dc30 comp rx=0 tx=0).stop 2024-09-16T00:29:35.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 -- 172.21.15.1:0/3499919724 shutdown_connections 2024-09-16T00:29:35.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20810e4d0 0x7fd20819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd208101360 0x7fd2081a0910 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1f80a32c0 0x7fd1f80a5780 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 --2- 172.21.15.1:0/3499919724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2081009b0 0x7fd2081a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 -- 172.21.15.1:0/3499919724 >> 172.21.15.1:0/3499919724 conn(0x7fd2080fc7d0 msgr2=0x7fd2080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 -- 172.21.15.1:0/3499919724 shutdown_connections 2024-09-16T00:29:35.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:35.736+0000 7fd20ec63640 1 -- 172.21.15.1:0/3499919724 wait complete. 2024-09-16T00:29:36.282 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:35 smithi001 ceph-mon[30934]: pgmap v386: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:35 smithi019 ceph-mon[35186]: pgmap v386: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:35 smithi153 ceph-mon[35538]: pgmap v386: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:36.467 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:29:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.330+0000 7f26379b5640 1 -- 172.21.15.1:0/4282671644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630100e20 msgr2=0x7f2630101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.330+0000 7f26379b5640 1 --2- 172.21.15.1:0/4282671644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630100e20 0x7f2630101200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2628009cc0 tx=0x7f262802cb60 comp rx=0 tx=0).stop 2024-09-16T00:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.330+0000 7f26379b5640 1 -- 172.21.15.1:0/4282671644 shutdown_connections 2024-09-16T00:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.330+0000 7f26379b5640 1 --2- 172.21.15.1:0/4282671644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630102ef0 0x7f263010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.330+0000 7f26379b5640 1 --2- 172.21.15.1:0/4282671644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2630102570 0x7f26301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.330+0000 7f26379b5640 1 --2- 172.21.15.1:0/4282671644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630100e20 0x7f2630101200 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.330+0000 7f26379b5640 1 -- 172.21.15.1:0/4282671644 >> 172.21.15.1:0/4282671644 conn(0x7f26300fc7d0 msgr2=0x7f26300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.330+0000 7f26379b5640 1 -- 172.21.15.1:0/4282671644 shutdown_connections 2024-09-16T00:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.330+0000 7f26379b5640 1 -- 172.21.15.1:0/4282671644 wait complete. 2024-09-16T00:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.331+0000 7f26379b5640 1 Processor -- start 2024-09-16T00:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.331+0000 7f26379b5640 1 -- start start 2024-09-16T00:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.331+0000 7f26379b5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2630100e20 0x7f26301a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.331+0000 7f26379b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630102570 0x7f26301a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.331+0000 7f26379b5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630102ef0 0x7f263019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.331+0000 7f26379b5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f263010fe80 con 0x7f2630102ef0 2024-09-16T00:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.331+0000 7f26379b5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f263010fd00 con 0x7f2630102570 2024-09-16T00:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.331+0000 7f26379b5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2630110000 con 0x7f2630100e20 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635f52640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630102ef0 0x7f263019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635f52640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630102ef0 0x7f263019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44260/0 (socket says 172.21.15.1:44260) 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635f52640 1 -- 172.21.15.1:0/2428622297 learned_addr learned my addr 172.21.15.1:0/2428622297 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635751640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2630100e20 0x7f26301a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2634f50640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630102570 0x7f26301a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635f52640 1 -- 172.21.15.1:0/2428622297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2630100e20 msgr2=0x7f26301a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635f52640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2630100e20 0x7f26301a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635f52640 1 -- 172.21.15.1:0/2428622297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630102570 msgr2=0x7f26301a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635f52640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630102570 0x7f26301a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635f52640 1 -- 172.21.15.1:0/2428622297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263019f010 con 0x7f2630102ef0 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2634f50640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630102570 0x7f26301a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.332+0000 7f2635f52640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630102ef0 0x7f263019e8e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f262400c970 tx=0x7f262400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.333+0000 7f26227fc640 1 -- 172.21.15.1:0/2428622297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624007bb0 con 0x7f2630102ef0 2024-09-16T00:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.333+0000 7f26227fc640 1 -- 172.21.15.1:0/2428622297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2624007d50 con 0x7f2630102ef0 2024-09-16T00:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.333+0000 7f26227fc640 1 -- 172.21.15.1:0/2428622297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624018800 con 0x7f2630102ef0 2024-09-16T00:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.333+0000 7f26379b5640 1 -- 172.21.15.1:0/2428622297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263019f300 con 0x7f2630102ef0 2024-09-16T00:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.333+0000 7f26379b5640 1 -- 172.21.15.1:0/2428622297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26301ab5f0 con 0x7f2630102ef0 2024-09-16T00:29:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.335+0000 7f26227fc640 1 -- 172.21.15.1:0/2428622297 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f262400fa50 con 0x7f2630102ef0 2024-09-16T00:29:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.336+0000 7f2639d8e740 1 Processor -- start 2024-09-16T00:29:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.336+0000 7f2639d8e740 1 -- start start 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.336+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7eceb40 0x55fea7ecd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.336+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7ecd5d0 0x55fea7ecda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.336+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eda760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.336+0000 7f2639d8e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fea7e332f0 con 0x55fea7ed8370 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f26227fc640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f261c0a32c0 0x7f261c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2639d8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fea7ddacf0 con 0x55fea7eceb40 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2634f50640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eda760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2634f50640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eda760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44268/0 (socket says 172.21.15.1:44268) 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2634f50640 1 -- 172.21.15.1:0/46610920 learned_addr learned my addr 172.21.15.1:0/46610920 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2639d8e740 1 -- 172.21.15.1:0/46610920 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fea7e2f2b0 con 0x55fea7ecd5d0 2024-09-16T00:29:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f26227fc640 1 -- 172.21.15.1:0/2428622297 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f26240cfe30 con 0x7f2630102ef0 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2634f50640 1 --2- 172.21.15.1:0/46610920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7eceb40 0x55fea7ecd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2635f52640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f261c0a32c0 0x7f261c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2635751640 1 --2- 172.21.15.1:0/46610920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7ecd5d0 0x55fea7ecda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2634f50640 1 -- 172.21.15.1:0/46610920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7ecd5d0 msgr2=0x55fea7ecda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2634f50640 1 --2- 172.21.15.1:0/46610920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7ecd5d0 0x55fea7ecda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2634f50640 1 -- 172.21.15.1:0/46610920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7eceb40 msgr2=0x55fea7ecd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2634f50640 1 --2- 172.21.15.1:0/46610920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7eceb40 0x55fea7ecd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.337+0000 7f2634f50640 1 -- 172.21.15.1:0/46610920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea7dbf3a0 con 0x55fea7ed8370 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.338+0000 7f2634f50640 1 --2- 172.21.15.1:0/46610920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eda760 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f262c00bfd0 tx=0x7f262c00b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92622153076d7cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.338+0000 7f2635f52640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f261c0a32c0 0x7f261c0a5780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f26240984b0 tx=0x7f2624020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.338+0000 7f261b7fe640 1 -- 172.21.15.1:0/46610920 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c010690 con 0x55fea7ed8370 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.338+0000 7f261b7fe640 1 -- 172.21.15.1:0/46610920 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c010d10 con 0x55fea7ed8370 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.338+0000 7f261b7fe640 1 -- 172.21.15.1:0/46610920 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c011d10 con 0x55fea7ed8370 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.339+0000 7f2639d8e740 1 -- 172.21.15.1:0/46610920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 msgr2=0x55fea7eda760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.339+0000 7f2639d8e740 1 --2- 172.21.15.1:0/46610920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eda760 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f262c00bfd0 tx=0x7f262c00b400 comp rx=0 tx=0).stop 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.339+0000 7f2639d8e740 1 -- 172.21.15.1:0/46610920 shutdown_connections 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.339+0000 7f2639d8e740 1 --2- 172.21.15.1:0/46610920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eda760 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.339+0000 7f2639d8e740 1 --2- 172.21.15.1:0/46610920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7ecd5d0 0x55fea7ecda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.339+0000 7f2639d8e740 1 --2- 172.21.15.1:0/46610920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7eceb40 0x55fea7ecd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.339+0000 7f2639d8e740 1 -- 172.21.15.1:0/46610920 >> 172.21.15.1:0/46610920 conn(0x55fea7e96310 msgr2=0x55fea7ec9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.339+0000 7f2639d8e740 1 -- 172.21.15.1:0/46610920 shutdown_connections 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.339+0000 7f2639d8e740 1 -- 172.21.15.1:0/46610920 wait complete. 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.340+0000 7f2639d8e740 1 -- start start 2024-09-16T00:29:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.340+0000 7f2639d8e740 1 Processor -- start 2024-09-16T00:29:37.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.340+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2634f50640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2634f50640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44276/0 (socket says 172.21.15.1:44276) 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2634f50640 1 -- 172.21.15.1:0/1531577349 learned_addr learned my addr 172.21.15.1:0/1531577349 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2639d8e740 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7eceb40 0x55fea7ed5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2639d8e740 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7ed8370 0x55fea7ed5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fea7edb070 con 0x55fea7ecd5d0 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fea7dee060 con 0x55fea7eceb40 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fea7edb1f0 con 0x55fea7ed8370 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2639d8e740 10 client.0 _ll_register_callbacks cb 0x55fea7b418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2634f50640 1 -- 172.21.15.1:0/1531577349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7ed8370 msgr2=0x55fea7ed5810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:29:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2634f50640 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7ed8370 0x55fea7ed5810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2634f50640 1 -- 172.21.15.1:0/1531577349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7eceb40 msgr2=0x55fea7ed5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2634f50640 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7eceb40 0x55fea7ed5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.341+0000 7f2634f50640 1 -- 172.21.15.1:0/1531577349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea7ba43c0 con 0x55fea7ecd5d0 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.342+0000 7f2634f50640 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed7dd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f262c00b990 tx=0x7f262c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2202a81f3316d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.342+0000 7f2618ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.342+0000 7f2618ff9640 1 -- 172.21.15.1:0/1531577349 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c005240 con 0x55fea7ecd5d0 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.342+0000 7f2618ff9640 1 -- 172.21.15.1:0/1531577349 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f262c0053e0 con 0x55fea7ecd5d0 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.342+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f262c010d10 con 0x55fea7ecd5d0 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.342+0000 7f2618ff9640 1 -- 172.21.15.1:0/1531577349 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f262c005620 con 0x55fea7ecd5d0 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.342+0000 7f2639d8e740 10 client.15528 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.342+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fea7ee6b90 con 0x55fea7ecd5d0 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.343+0000 7f2618ff9640 1 -- 172.21.15.1:0/1531577349 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c010970 con 0x55fea7ecd5d0 2024-09-16T00:29:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.343+0000 7f2618ff9640 1 -- 172.21.15.1:0/1531577349 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f262c024070 con 0x55fea7ecd5d0 2024-09-16T00:29:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.343+0000 7f2639d8e740 10 client.15528 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.343+0000 7f2639d8e740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.343+0000 7f2639d8e740 20 client.15528 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.343+0000 7f2639d8e740 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fea7ee7f90 0x55fea7eea380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.343+0000 7f2639d8e740 4 client.15528 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.343+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f262c010d10 con 0x55fea7ee7f90 2024-09-16T00:29:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.344+0000 7f2635751640 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fea7ee7f90 0x55fea7eea380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.344+0000 7f2635751640 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fea7ee7f90 0x55fea7eea380 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce4620cf86c7b server_cookie=6ab563452a3cb6a5 in_seq=0 out_seq=0 2024-09-16T00:29:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.344+0000 7f2618ff9640 10 client.15528 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2618ff9640 1 -- 172.21.15.1:0/1531577349 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f262c010d10 con 0x55fea7ee7f90 2024-09-16T00:29:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2618ff9640 10 client.15528 handle_command_reply: tid=0 2024-09-16T00:29:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 client.15528 shutdown 2024-09-16T00:29:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ecd5d0 msgr2=0x55fea7ed7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed7dd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f262c00b990 tx=0x7f262c00bf50 comp rx=0 tx=0).stop 2024-09-16T00:29:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 shutdown_connections 2024-09-16T00:29:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fea7ee7f90 0x55fea7eea380 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7ed8370 0x55fea7ed5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7eceb40 0x55fea7ed5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2618ff9640 0 client.15528 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 --2- 172.21.15.1:0/1531577349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed7dd0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 >> 172.21.15.1:0/1531577349 conn(0x55fea7e96310 msgr2=0x55fea7ecf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 shutdown_connections 2024-09-16T00:29:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 1 -- 172.21.15.1:0/1531577349 wait complete. 2024-09-16T00:29:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.345+0000 7f2639d8e740 20 client.15528 trim_cache size 0 max 16384 2024-09-16T00:29:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2639d8e740 1 Processor -- start 2024-09-16T00:29:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2639d8e740 1 -- start start 2024-09-16T00:29:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7ed0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eefac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2639d8e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b7d50 con 0x55fea7ed8370 2024-09-16T00:29:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2639d8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b5620 con 0x55fea7ecd5d0 2024-09-16T00:29:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2639d8e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b3470 con 0x55fea7eceb40 2024-09-16T00:29:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2634f50640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eefac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2634f50640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eefac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44290/0 (socket says 172.21.15.1:44290) 2024-09-16T00:29:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2634f50640 1 -- 172.21.15.1:0/3606237819 learned_addr learned my addr 172.21.15.1:0/3606237819 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2635751640 1 --2- 172.21.15.1:0/3606237819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7ed0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.371+0000 7f2634f50640 1 --2- 172.21.15.1:0/3606237819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.372+0000 7f2634f50640 1 -- 172.21.15.1:0/3606237819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 msgr2=0x55fea7ed0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.372+0000 7f2634f50640 1 --2- 172.21.15.1:0/3606237819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7ed0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.372+0000 7f2634f50640 1 -- 172.21.15.1:0/3606237819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7ecd5d0 msgr2=0x55fea7ed0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.372+0000 7f2634f50640 1 --2- 172.21.15.1:0/3606237819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.372+0000 7f2634f50640 1 -- 172.21.15.1:0/3606237819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea7ed35d0 con 0x55fea7ed8370 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.372+0000 7f2634f50640 1 --2- 172.21.15.1:0/3606237819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eefac0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f262c004e00 tx=0x7f262c010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb3fd256693c99f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.372+0000 7f261affd640 1 -- 172.21.15.1:0/3606237819 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c005620 con 0x55fea7ed8370 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.372+0000 7f261affd640 1 -- 172.21.15.1:0/3606237819 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c005e50 con 0x55fea7ed8370 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.372+0000 7f261affd640 1 -- 172.21.15.1:0/3606237819 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c0047e0 con 0x55fea7ed8370 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.373+0000 7f2639d8e740 1 -- 172.21.15.1:0/3606237819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 msgr2=0x55fea7eefac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.373+0000 7f2639d8e740 1 --2- 172.21.15.1:0/3606237819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eefac0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f262c004e00 tx=0x7f262c010f70 comp rx=0 tx=0).stop 2024-09-16T00:29:37.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.373+0000 7f2639d8e740 1 -- 172.21.15.1:0/3606237819 shutdown_connections 2024-09-16T00:29:37.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.373+0000 7f2639d8e740 1 --2- 172.21.15.1:0/3606237819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7ed8370 0x55fea7eefac0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.373+0000 7f2639d8e740 1 --2- 172.21.15.1:0/3606237819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7ed0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.373+0000 7f2639d8e740 1 --2- 172.21.15.1:0/3606237819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7ecd5d0 0x55fea7ed0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.373+0000 7f2639d8e740 1 -- 172.21.15.1:0/3606237819 >> 172.21.15.1:0/3606237819 conn(0x55fea7e96310 msgr2=0x55fea7ed4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.373+0000 7f2639d8e740 1 -- 172.21.15.1:0/3606237819 shutdown_connections 2024-09-16T00:29:37.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.373+0000 7f2639d8e740 1 -- 172.21.15.1:0/3606237819 wait complete. 2024-09-16T00:29:37.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.374+0000 7f2639d8e740 1 -- start start 2024-09-16T00:29:37.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.374+0000 7f2639d8e740 1 Processor -- start 2024-09-16T00:29:37.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.374+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7ecd5d0 0x55fea7ee0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.374+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7edc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.374+0000 7f2639d8e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7edc730 0x55fea7f51a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.374+0000 7f2639d8e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b3ae0 con 0x55fea7edc730 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2639d8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b4f50 con 0x55fea7ecd5d0 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2639d8e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b41b0 con 0x55fea7eceb40 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2635751640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7edc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2635751640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7edc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:55282/0 (socket says 172.21.15.1:55282) 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2635751640 1 -- 172.21.15.1:0/150635365 learned_addr learned my addr 172.21.15.1:0/150635365 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2639d8e740 10 client.0 _ll_register_callbacks cb 0x7f262c013490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2634f50640 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7edc730 0x55fea7f51a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2635751640 1 -- 172.21.15.1:0/150635365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7ecd5d0 msgr2=0x55fea7ee0db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2635751640 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7ecd5d0 0x55fea7ee0db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2635751640 1 -- 172.21.15.1:0/150635365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7edc730 msgr2=0x55fea7f51a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2635751640 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7edc730 0x55fea7f51a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2635751640 1 -- 172.21.15.1:0/150635365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea7f51fd0 con 0x55fea7eceb40 2024-09-16T00:29:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2634f50640 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7edc730 0x55fea7f51a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2635751640 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7edc040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2628038040 tx=0x7f2628003990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f723200269927f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.375+0000 7f2618ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2618ff9640 1 -- 172.21.15.1:0/150635365 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262802e6c0 con 0x55fea7eceb40 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2639d8e740 1 -- 172.21.15.1:0/150635365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f262c005e50 con 0x55fea7eceb40 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2618ff9640 1 -- 172.21.15.1:0/150635365 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f262802e860 con 0x55fea7eceb40 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2618ff9640 1 -- 172.21.15.1:0/150635365 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2628059050 con 0x55fea7eceb40 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2639d8e740 10 client.25090 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2639d8e740 1 -- 172.21.15.1:0/150635365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fea7f53070 con 0x55fea7eceb40 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2618ff9640 1 -- 172.21.15.1:0/150635365 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c005e50 con 0x55fea7eceb40 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2618ff9640 1 -- 172.21.15.1:0/150635365 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2628036420 con 0x55fea7eceb40 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2639d8e740 10 client.25090 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2639d8e740 10 client.25090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2639d8e740 20 client.25090 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2639d8e740 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fea7f53820 0x55fea7f55c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2639d8e740 4 client.25090 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:29:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.376+0000 7f2639d8e740 1 -- 172.21.15.1:0/150635365 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fea7e40860 con 0x55fea7f53820 2024-09-16T00:29:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.377+0000 7f2634f50640 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fea7f53820 0x55fea7f55c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.377+0000 7f2618ff9640 10 client.25090 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.377+0000 7f2634f50640 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fea7f53820 0x55fea7f55c60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515ffe8da5035778 server_cookie=6b32db7b030518c2 in_seq=0 out_seq=0 2024-09-16T00:29:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.377+0000 7f2618ff9640 1 -- 172.21.15.1:0/150635365 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fea7e40860 con 0x55fea7f53820 2024-09-16T00:29:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.377+0000 7f2618ff9640 10 client.25090 handle_command_reply: tid=0 2024-09-16T00:29:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.377+0000 7f2639d8e740 1 client.25090 shutdown 2024-09-16T00:29:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 -- 172.21.15.1:0/150635365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 msgr2=0x55fea7edc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7edc040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2628038040 tx=0x7f2628003990 comp rx=0 tx=0).stop 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 -- 172.21.15.1:0/150635365 shutdown_connections 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fea7f53820 0x55fea7f55c60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fea7edc730 0x55fea7f51a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea7eceb40 0x55fea7edc040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2618ff9640 0 client.25090 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 --2- 172.21.15.1:0/150635365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fea7ecd5d0 0x55fea7ee0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 -- 172.21.15.1:0/150635365 >> 172.21.15.1:0/150635365 conn(0x55fea7e96310 msgr2=0x55fea7ec84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 -- 172.21.15.1:0/150635365 shutdown_connections 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 1 -- 172.21.15.1:0/150635365 wait complete. 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.378+0000 7f2639d8e740 20 client.25090 trim_cache size 0 max 16384 2024-09-16T00:29:37.382 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:29:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 -- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f261c0a32c0 msgr2=0x7f261c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f261c0a32c0 0x7f261c0a5780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f26240984b0 tx=0x7f2624020000 comp rx=0 tx=0).stop 2024-09-16T00:29:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 -- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630102ef0 msgr2=0x7f263019e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630102ef0 0x7f263019e8e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f262400c970 tx=0x7f262400ce30 comp rx=0 tx=0).stop 2024-09-16T00:29:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 -- 172.21.15.1:0/2428622297 shutdown_connections 2024-09-16T00:29:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2630102ef0 0x7f263019e8e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630102570 0x7f26301a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f261c0a32c0 0x7f261c0a5780 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 --2- 172.21.15.1:0/2428622297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2630100e20 0x7f26301a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.379+0000 7f26379b5640 1 -- 172.21.15.1:0/2428622297 >> 172.21.15.1:0/2428622297 conn(0x7f26300fc7d0 msgr2=0x7f26300fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.380+0000 7f26379b5640 1 -- 172.21.15.1:0/2428622297 shutdown_connections 2024-09-16T00:29:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:37.380+0000 7f26379b5640 1 -- 172.21.15.1:0/2428622297 wait complete. 2024-09-16T00:29:38.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:37 smithi001 ceph-mon[30934]: pgmap v387: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:29:38.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:37 smithi019 ceph-mon[35186]: pgmap v387: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:29:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:37 smithi153 ceph-mon[35538]: pgmap v387: 129 pgs: 129 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:29:39.237 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:29:39.238 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:29:39.584 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:39.966 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:39 smithi001 ceph-mon[30934]: pgmap v388: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:29:40.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.077+0000 7f82d9fba640 1 -- 172.21.15.1:0/3670824574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 msgr2=0x7f82d410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.077+0000 7f82d9fba640 1 --2- 172.21.15.1:0/3670824574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 0x7f82d410ec40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009930 tx=0x7f82cc02a420 comp rx=0 tx=0).stop 2024-09-16T00:29:40.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.077+0000 7f82d9fba640 1 -- 172.21.15.1:0/3670824574 shutdown_connections 2024-09-16T00:29:40.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.077+0000 7f82d9fba640 1 --2- 172.21.15.1:0/3670824574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 0x7f82d410ec40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.077+0000 7f82d9fba640 1 --2- 172.21.15.1:0/3670824574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82d41036e0 0x7f82d4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.077+0000 7f82d9fba640 1 --2- 172.21.15.1:0/3670824574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82d4102d30 0x7f82d4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.077+0000 7f82d9fba640 1 -- 172.21.15.1:0/3670824574 >> 172.21.15.1:0/3670824574 conn(0x7f82d40fc7d0 msgr2=0x7f82d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.077+0000 7f82d9fba640 1 -- 172.21.15.1:0/3670824574 shutdown_connections 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.077+0000 7f82d9fba640 1 -- 172.21.15.1:0/3670824574 wait complete. 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.078+0000 7f82d9fba640 1 Processor -- start 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.078+0000 7f82d9fba640 1 -- start start 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.078+0000 7f82d9fba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82d4102d30 0x7f82d4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.078+0000 7f82d9fba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82d41036e0 0x7f82d4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.078+0000 7f82d9fba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 0x7f82d419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d9fba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41120a0 con 0x7f82d41082a0 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d9fba640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4111f20 con 0x7f82d41036e0 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d9fba640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4112220 con 0x7f82d4102d30 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 0x7f82d419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 0x7f82d419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44324/0 (socket says 172.21.15.1:44324) 2024-09-16T00:29:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d3fff640 1 -- 172.21.15.1:0/1866228660 learned_addr learned my addr 172.21.15.1:0/1866228660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d37fe640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82d4102d30 0x7f82d4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82d41036e0 0x7f82d4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d3fff640 1 -- 172.21.15.1:0/1866228660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82d4102d30 msgr2=0x7f82d4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d3fff640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82d4102d30 0x7f82d4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d3fff640 1 -- 172.21.15.1:0/1866228660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82d41036e0 msgr2=0x7f82d4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d3fff640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82d41036e0 0x7f82d4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d3fff640 1 -- 172.21.15.1:0/1866228660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d419d3b0 con 0x7f82d41082a0 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.079+0000 7f82d3fff640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 0x7f82d419ccd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f82cc005ab0 tx=0x7f82cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.080+0000 7f82d0ff9640 1 -- 172.21.15.1:0/1866228660 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc03f030 con 0x7f82d41082a0 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.080+0000 7f82d0ff9640 1 -- 172.21.15.1:0/1866228660 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc003ce0 con 0x7f82d41082a0 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.080+0000 7f82d0ff9640 1 -- 172.21.15.1:0/1866228660 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc035070 con 0x7f82d41082a0 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.080+0000 7f82d9fba640 1 -- 172.21.15.1:0/1866228660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d419d640 con 0x7f82d41082a0 2024-09-16T00:29:40.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.080+0000 7f82d9fba640 1 -- 172.21.15.1:0/1866228660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d419dad0 con 0x7f82d41082a0 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.083+0000 7f82d0ff9640 1 -- 172.21.15.1:0/1866228660 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f82cc042030 con 0x7f82d41082a0 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.083+0000 7f82dc393740 1 Processor -- start 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.083+0000 7f82dc393740 1 -- start start 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.084+0000 7f82dc393740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f875ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.084+0000 7f82dc393740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f8760480 0x5592f876a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.084+0000 7f82dc393740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f8771540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.084+0000 7f82dc393740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5592f86c5280 con 0x5592f8760480 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.084+0000 7f82dc393740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592f866cb10 con 0x5592f875fb60 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.084+0000 7f82dc393740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5592f86c1240 con 0x5592f876f150 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.084+0000 7f82d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f8760480 0x5592f876a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.084+0000 7f82d0ff9640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82c00a3340 0x7f82c00a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f875ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f875ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:55272/0 (socket says 172.21.15.1:55272) 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 -- 172.21.15.1:0/2201994504 learned_addr learned my addr 172.21.15.1:0/2201994504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/2201994504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f8771540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d0ff9640 1 -- 172.21.15.1:0/1866228660 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f82cc0eff00 con 0x7f82d41082a0 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d3fff640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82c00a3340 0x7f82c00a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 -- 172.21.15.1:0/2201994504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 msgr2=0x5592f8771540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.084+0000 7f82d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f8760480 0x5592f876a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44336/0 (socket says 172.21.15.1:44336) 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/2201994504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f8771540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 -- 172.21.15.1:0/2201994504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f8760480 msgr2=0x5592f876a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/2201994504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f8760480 0x5592f876a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 -- 172.21.15.1:0/2201994504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592f86791f0 con 0x5592f875fb60 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d37fe640 1 --2- 172.21.15.1:0/2201994504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f8760480 0x5592f876a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.085+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/2201994504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f875ff40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82bc013920 tx=0x7f82bc00e430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=908b1a70f6bfd779 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.086+0000 7f82d3fff640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82c00a3340 0x7f82c00a5800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0bc050 tx=0x7f82cc02a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:40.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.086+0000 7f82c9ffb640 1 -- 172.21.15.1:0/2201994504 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc004020 con 0x5592f875fb60 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.086+0000 7f82c9ffb640 1 -- 172.21.15.1:0/2201994504 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc00bdb0 con 0x5592f875fb60 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.086+0000 7f82c9ffb640 1 -- 172.21.15.1:0/2201994504 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc003850 con 0x5592f875fb60 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.086+0000 7f82dc393740 1 -- 172.21.15.1:0/2201994504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 msgr2=0x5592f875ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.086+0000 7f82dc393740 1 --2- 172.21.15.1:0/2201994504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f875ff40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82bc013920 tx=0x7f82bc00e430 comp rx=0 tx=0).stop 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.087+0000 7f82dc393740 1 -- 172.21.15.1:0/2201994504 shutdown_connections 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.087+0000 7f82dc393740 1 --2- 172.21.15.1:0/2201994504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f8771540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.087+0000 7f82dc393740 1 --2- 172.21.15.1:0/2201994504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f8760480 0x5592f876a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.087+0000 7f82dc393740 1 --2- 172.21.15.1:0/2201994504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f875ff40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.087+0000 7f82dc393740 1 -- 172.21.15.1:0/2201994504 >> 172.21.15.1:0/2201994504 conn(0x5592f875c0f0 msgr2=0x5592f875c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.087+0000 7f82dc393740 1 -- 172.21.15.1:0/2201994504 shutdown_connections 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.087+0000 7f82dc393740 1 -- 172.21.15.1:0/2201994504 wait complete. 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.087+0000 7f82dc393740 1 -- start start 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.087+0000 7f82dc393740 1 Processor -- start 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82dc393740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f8763ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82dc393740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f8760480 0x5592f8764400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82dc393740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f876f150 0x5592f87686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82dc393740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5592f876afe0 con 0x5592f876f150 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82dc393740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592f867ff10 con 0x5592f875fb60 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82dc393740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5592f876b160 con 0x5592f8760480 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82dc393740 10 client.0 _ll_register_callbacks cb 0x5592f8648a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f876f150 0x5592f87686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f876f150 0x5592f87686b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44348/0 (socket says 172.21.15.1:44348) 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82d2ffd640 1 -- 172.21.15.1:0/2336758296 learned_addr learned my addr 172.21.15.1:0/2336758296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82d37fe640 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f8760480 0x5592f8764400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f8763ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.088+0000 7f82d2ffd640 1 -- 172.21.15.1:0/2336758296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f8760480 msgr2=0x5592f8764400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f8760480 0x5592f8764400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82d2ffd640 1 -- 172.21.15.1:0/2336758296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 msgr2=0x5592f8763ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f8763ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82d2ffd640 1 -- 172.21.15.1:0/2336758296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592f84366a0 con 0x5592f876f150 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f876f150 0x5592f87686b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0057a0 tx=0x7f82bc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d950b5cd1e84d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82b2ffd640 1 -- 172.21.15.1:0/2336758296 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc011440 con 0x5592f876f150 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82b2ffd640 1 -- 172.21.15.1:0/2336758296 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f82bc0115e0 con 0x5592f876f150 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82dc393740 1 -- 172.21.15.1:0/2336758296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82bc00bdb0 con 0x5592f876f150 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82b2ffd640 1 -- 172.21.15.1:0/2336758296 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f82bc0117e0 con 0x5592f876f150 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82dc393740 10 client.15543 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:40.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.089+0000 7f82dc393740 1 -- 172.21.15.1:0/2336758296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592f8781d40 con 0x5592f876f150 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.090+0000 7f82b2ffd640 1 -- 172.21.15.1:0/2336758296 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc00bdb0 con 0x5592f876f150 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.090+0000 7f82b2ffd640 1 -- 172.21.15.1:0/2336758296 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f82bc010ba0 con 0x5592f876f150 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.090+0000 7f82dc393740 10 client.15543 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.090+0000 7f82dc393740 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.090+0000 7f82dc393740 20 client.15543 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.090+0000 7f82dc393740 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5592f8783090 0x5592f8785480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.090+0000 7f82dc393740 4 client.15543 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.090+0000 7f82dc393740 1 -- 172.21.15.1:0/2336758296 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592f86d27f0 con 0x5592f8783090 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.090+0000 7f82d37fe640 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5592f8783090 0x5592f8785480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.091+0000 7f82d37fe640 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5592f8783090 0x5592f8785480 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2183b93e18c013 server_cookie=514ad7754be4c273 in_seq=0 out_seq=0 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.091+0000 7f82b2ffd640 10 client.15543 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82b2ffd640 1 -- 172.21.15.1:0/2336758296 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592f86d27f0 con 0x5592f8783090 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82b2ffd640 10 client.15543 handle_command_reply: tid=0 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 client.15543 shutdown 2024-09-16T00:29:40.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 -- 172.21.15.1:0/2336758296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f876f150 msgr2=0x5592f87686b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f876f150 0x5592f87686b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0057a0 tx=0x7f82bc005850 comp rx=0 tx=0).stop 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 -- 172.21.15.1:0/2336758296 shutdown_connections 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5592f8783090 0x5592f8785480 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82b2ffd640 0 client.15543 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f876f150 0x5592f87686b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f8760480 0x5592f8764400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 --2- 172.21.15.1:0/2336758296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f8763ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 -- 172.21.15.1:0/2336758296 >> 172.21.15.1:0/2336758296 conn(0x5592f875c0f0 msgr2=0x5592f875b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 -- 172.21.15.1:0/2336758296 shutdown_connections 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 1 -- 172.21.15.1:0/2336758296 wait complete. 2024-09-16T00:29:40.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.092+0000 7f82dc393740 20 client.15543 trim_cache size 0 max 16384 2024-09-16T00:29:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.110+0000 7f82dc393740 1 Processor -- start 2024-09-16T00:29:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.110+0000 7f82dc393740 1 -- start start 2024-09-16T00:29:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.110+0000 7f82dc393740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f875fb60 0x5592f8762e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.110+0000 7f82dc393740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f8760480 0x5592f87685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.110+0000 7f82dc393740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f878aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.110+0000 7f82dc393740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41af410 con 0x5592f875fb60 2024-09-16T00:29:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.110+0000 7f82dc393740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41acce0 con 0x5592f8760480 2024-09-16T00:29:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.110+0000 7f82dc393740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41aab30 con 0x5592f876f150 2024-09-16T00:29:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f875fb60 0x5592f8762e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f875fb60 0x5592f8762e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44352/0 (socket says 172.21.15.1:44352) 2024-09-16T00:29:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d2ffd640 1 -- 172.21.15.1:0/3928474056 learned_addr learned my addr 172.21.15.1:0/3928474056 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d37fe640 1 --2- 172.21.15.1:0/3928474056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f8760480 0x5592f87685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d37fe640 1 -- 172.21.15.1:0/3928474056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 msgr2=0x5592f878aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d37fe640 1 --2- 172.21.15.1:0/3928474056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f878aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d37fe640 1 -- 172.21.15.1:0/3928474056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f875fb60 msgr2=0x5592f8762e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d37fe640 1 --2- 172.21.15.1:0/3928474056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f875fb60 0x5592f8762e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d37fe640 1 -- 172.21.15.1:0/3928474056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592f876b120 con 0x5592f8760480 2024-09-16T00:29:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.111+0000 7f82d37fe640 1 --2- 172.21.15.1:0/3928474056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f8760480 0x5592f87685c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5592f87859c0 tx=0x7f82c40075b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ad3a2448d58f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.112+0000 7f82c97fa640 1 -- 172.21.15.1:0/3928474056 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4005240 con 0x5592f8760480 2024-09-16T00:29:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.112+0000 7f82c97fa640 1 -- 172.21.15.1:0/3928474056 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c401e030 con 0x5592f8760480 2024-09-16T00:29:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.112+0000 7f82c97fa640 1 -- 172.21.15.1:0/3928474056 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c401aa40 con 0x5592f8760480 2024-09-16T00:29:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.112+0000 7f82dc393740 1 -- 172.21.15.1:0/3928474056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f8760480 msgr2=0x5592f87685c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.112+0000 7f82dc393740 1 --2- 172.21.15.1:0/3928474056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f8760480 0x5592f87685c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5592f87859c0 tx=0x7f82c40075b0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.113+0000 7f82dc393740 1 -- 172.21.15.1:0/3928474056 shutdown_connections 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.113+0000 7f82dc393740 1 --2- 172.21.15.1:0/3928474056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f878aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.113+0000 7f82dc393740 1 --2- 172.21.15.1:0/3928474056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f8760480 0x5592f87685c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.113+0000 7f82dc393740 1 --2- 172.21.15.1:0/3928474056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f875fb60 0x5592f8762e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.113+0000 7f82dc393740 1 -- 172.21.15.1:0/3928474056 >> 172.21.15.1:0/3928474056 conn(0x5592f875c0f0 msgr2=0x5592f8761dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.113+0000 7f82dc393740 1 -- 172.21.15.1:0/3928474056 shutdown_connections 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.113+0000 7f82dc393740 1 -- 172.21.15.1:0/3928474056 wait complete. 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.113+0000 7f82dc393740 1 -- start start 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.113+0000 7f82dc393740 1 Processor -- start 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82dc393740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f877f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82dc393740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f877f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f877f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f877f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:55288/0 (socket says 172.21.15.1:55288) 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82d2ffd640 1 -- 172.21.15.1:0/1565566798 learned_addr learned my addr 172.21.15.1:0/1565566798 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82dc393740 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f877fd50 0x5592f877c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41ab1a0 con 0x5592f877fd50 2024-09-16T00:29:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41ac610 con 0x5592f875fb60 2024-09-16T00:29:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41ab870 con 0x5592f876f150 2024-09-16T00:29:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.114+0000 7f82d37fe640 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f877f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82dc393740 10 client.0 _ll_register_callbacks cb 0x7f82bc00ee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:40.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f877fd50 0x5592f877c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82d37fe640 1 -- 172.21.15.1:0/1565566798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 msgr2=0x5592f877f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:40.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82d37fe640 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f877f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82d37fe640 1 -- 172.21.15.1:0/1565566798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f877fd50 msgr2=0x5592f877c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82d37fe640 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f877fd50 0x5592f877c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82d37fe640 1 -- 172.21.15.1:0/1565566798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592f877c8b0 con 0x5592f876f150 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f877fd50 0x5592f877c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82d37fe640 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f877f810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5592f86c5320 tx=0x7f82c4004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c3725f6408002f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.115+0000 7f82b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.116+0000 7f82b2ffd640 1 -- 172.21.15.1:0/1565566798 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c401a8d0 con 0x5592f876f150 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.116+0000 7f82b2ffd640 1 -- 172.21.15.1:0/1565566798 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f82c401aa70 con 0x5592f876f150 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.116+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c401e030 con 0x5592f876f150 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.116+0000 7f82b2ffd640 1 -- 172.21.15.1:0/1565566798 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f82c401ac70 con 0x5592f876f150 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.116+0000 7f82dc393740 10 client.25102 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.116+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592f87ee330 con 0x5592f876f150 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.116+0000 7f82b2ffd640 1 -- 172.21.15.1:0/1565566798 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c401e030 con 0x5592f876f150 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.117+0000 7f82b2ffd640 1 -- 172.21.15.1:0/1565566798 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f82c4023420 con 0x5592f876f150 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.117+0000 7f82dc393740 10 client.25102 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.117+0000 7f82dc393740 10 client.25102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.117+0000 7f82dc393740 20 client.25102 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.117+0000 7f82dc393740 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5592f87ee9e0 0x5592f87f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.117+0000 7f82dc393740 4 client.25102 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:29:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.117+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5592f86ccd00 con 0x5592f87ee9e0 2024-09-16T00:29:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.117+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5592f87ee9e0 0x5592f87f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.117+0000 7f82b2ffd640 10 client.25102 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.118+0000 7f82d2ffd640 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5592f87ee9e0 0x5592f87f0e00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe79ed19f396aeae server_cookie=850e36a7bc15d6f1 in_seq=0 out_seq=0 2024-09-16T00:29:40.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.119+0000 7f82b2ffd640 1 -- 172.21.15.1:0/1565566798 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5592f86ccd00 con 0x5592f87ee9e0 2024-09-16T00:29:40.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.119+0000 7f82b2ffd640 10 client.25102 handle_command_reply: tid=0 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.119+0000 7f82dc393740 1 client.25102 shutdown 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.119+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 msgr2=0x5592f877f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.119+0000 7f82dc393740 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f877f810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5592f86c5320 tx=0x7f82c4004ba0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.119+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 shutdown_connections 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.119+0000 7f82dc393740 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5592f87ee9e0 0x5592f87f0e00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.120+0000 7f82dc393740 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592f877fd50 0x5592f877c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.120+0000 7f82b2ffd640 0 client.25102 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.120+0000 7f82dc393740 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592f876f150 0x5592f877f810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.120+0000 7f82dc393740 1 --2- 172.21.15.1:0/1565566798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592f875fb60 0x5592f877f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.120+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 >> 172.21.15.1:0/1565566798 conn(0x5592f875c0f0 msgr2=0x5592f8761490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.120+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 shutdown_connections 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.120+0000 7f82dc393740 1 -- 172.21.15.1:0/1565566798 wait complete. 2024-09-16T00:29:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.120+0000 7f82dc393740 20 client.25102 trim_cache size 0 max 16384 2024-09-16T00:29:40.123 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:29:40.123 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:29:40.123 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "c07af028-e983-4555-814b-78d1def5bdd3", 2024-09-16T00:29:40.123 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:29:40.123 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 -- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82c00a3340 msgr2=0x7f82c00a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82c00a3340 0x7f82c00a5800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0bc050 tx=0x7f82cc02a420 comp rx=0 tx=0).stop 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 -- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 msgr2=0x7f82d419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 0x7f82d419ccd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f82cc005ab0 tx=0x7f82cc0042f0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 -- 172.21.15.1:0/1866228660 shutdown_connections 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82d41082a0 0x7f82d419ccd0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82d41036e0 0x7f82d4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82c00a3340 0x7f82c00a5800 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 --2- 172.21.15.1:0/1866228660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82d4102d30 0x7f82d4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 -- 172.21.15.1:0/1866228660 >> 172.21.15.1:0/1866228660 conn(0x7f82d40fc7d0 msgr2=0x7f82d40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 -- 172.21.15.1:0/1866228660 shutdown_connections 2024-09-16T00:29:40.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:40.121+0000 7f82d9fba640 1 -- 172.21.15.1:0/1866228660 wait complete. 2024-09-16T00:29:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:39 smithi019 ceph-mon[35186]: pgmap v388: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:29:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:39 smithi153 ceph-mon[35538]: pgmap v388: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:29:40.926 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:29:40.975 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:40 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:29:40.975 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:40 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:41.319 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:41.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:40 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:29:41.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:40 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:41.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:40 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:29:41.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:40 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:41.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:40 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:29:41.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:40 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:41.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:40 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:29:41.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:40 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:41.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:40 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:29:41.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:40 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.920+0000 7f9ee89ed640 1 -- 172.21.15.1:0/1261010555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4102570 msgr2=0x7f9ee41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.920+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/1261010555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4102570 0x7f9ee41029b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009900 tx=0x7f9ed802a480 comp rx=0 tx=0).stop 2024-09-16T00:29:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.921+0000 7f9ee89ed640 1 -- 172.21.15.1:0/1261010555 shutdown_connections 2024-09-16T00:29:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.921+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/1261010555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee4102ef0 0x7f9ee410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.921+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/1261010555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4102570 0x7f9ee41029b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.921+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/1261010555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee4100e20 0x7f9ee4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.921+0000 7f9ee89ed640 1 -- 172.21.15.1:0/1261010555 >> 172.21.15.1:0/1261010555 conn(0x7f9ee40fc7d0 msgr2=0x7f9ee40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.921+0000 7f9ee89ed640 1 -- 172.21.15.1:0/1261010555 shutdown_connections 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.921+0000 7f9ee89ed640 1 -- 172.21.15.1:0/1261010555 wait complete. 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.921+0000 7f9ee89ed640 1 Processor -- start 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.921+0000 7f9ee89ed640 1 -- start start 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee89ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4100e20 0x7f9ee419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee89ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee4102570 0x7f9ee419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee89ed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee4102ef0 0x7f9ee419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee89ed640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4110fd0 con 0x7f9ee4100e20 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee89ed640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4110e50 con 0x7f9ee4102570 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee89ed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4111150 con 0x7f9ee4102ef0 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4100e20 0x7f9ee419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4100e20 0x7f9ee419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59156/0 (socket says 172.21.15.1:59156) 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee259c640 1 -- 172.21.15.1:0/934792559 learned_addr learned my addr 172.21.15.1:0/934792559 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:41.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee1d9b640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee4102570 0x7f9ee419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee4102ef0 0x7f9ee419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.922+0000 7f9ee259c640 1 -- 172.21.15.1:0/934792559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee4102ef0 msgr2=0x7f9ee419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:41.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ee259c640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee4102ef0 0x7f9ee419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ee259c640 1 -- 172.21.15.1:0/934792559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee4102570 msgr2=0x7f9ee419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:41.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ee259c640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee4102570 0x7f9ee419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ee259c640 1 -- 172.21.15.1:0/934792559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee419f4b0 con 0x7f9ee4100e20 2024-09-16T00:29:41.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee4102ef0 0x7f9ee419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:41.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ee259c640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4100e20 0x7f9ee419a500 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400d950 tx=0x7f9ed400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:41.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ed37fe640 1 -- 172.21.15.1:0/934792559 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4017070 con 0x7f9ee4100e20 2024-09-16T00:29:41.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ed37fe640 1 -- 172.21.15.1:0/934792559 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4009c30 con 0x7f9ee4100e20 2024-09-16T00:29:41.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ee89ed640 1 -- 172.21.15.1:0/934792559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee419f7a0 con 0x7f9ee4100e20 2024-09-16T00:29:41.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ed37fe640 1 -- 172.21.15.1:0/934792559 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400f6b0 con 0x7f9ee4100e20 2024-09-16T00:29:41.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.923+0000 7f9ee89ed640 1 -- 172.21.15.1:0/934792559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a7080 con 0x7f9ee4100e20 2024-09-16T00:29:41.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.926+0000 7f9ed37fe640 1 -- 172.21.15.1:0/934792559 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9ed40109d0 con 0x7f9ee4100e20 2024-09-16T00:29:41.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9ed37fe640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9eeadc6740 1 Processor -- start 2024-09-16T00:29:41.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9ed37fe640 1 -- 172.21.15.1:0/934792559 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9ed40cfcb0 con 0x7f9ee4100e20 2024-09-16T00:29:41.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9ee1d9b640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9eeadc6740 1 -- start start 2024-09-16T00:29:41.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257a74a0 0x555e257a7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a7dc0 0x555e257b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555e2570d2e0 con 0x555e257b2680 2024-09-16T00:29:41.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555e256b4ce0 con 0x555e257a74a0 2024-09-16T00:29:41.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555e257092a0 con 0x555e257a7dc0 2024-09-16T00:29:41.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.927+0000 7f9ee2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59162/0 (socket says 172.21.15.1:59162) 2024-09-16T00:29:41.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee2d9d640 1 -- 172.21.15.1:0/672402990 learned_addr learned my addr 172.21.15.1:0/672402990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee1d9b640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ee41049a0 tx=0x7f9ed800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee1d9b640 1 --2- 172.21.15.1:0/672402990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a7dc0 0x555e257b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee259c640 1 --2- 172.21.15.1:0/672402990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257a74a0 0x555e257a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee2d9d640 1 -- 172.21.15.1:0/672402990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a7dc0 msgr2=0x555e257b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/672402990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a7dc0 0x555e257b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee2d9d640 1 -- 172.21.15.1:0/672402990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257a74a0 msgr2=0x555e257a7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/672402990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257a74a0 0x555e257a7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee2d9d640 1 -- 172.21.15.1:0/672402990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e25699440 con 0x555e257b2680 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/672402990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b4a70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009fb0 tx=0x7f9edc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f812cd8d54bbc59 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ed0ff9640 1 -- 172.21.15.1:0/672402990 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc016070 con 0x555e257b2680 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ed0ff9640 1 -- 172.21.15.1:0/672402990 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9edc012bf0 con 0x555e257b2680 2024-09-16T00:29:41.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.928+0000 7f9ed0ff9640 1 -- 172.21.15.1:0/672402990 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc01bd20 con 0x555e257b2680 2024-09-16T00:29:41.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.929+0000 7f9eeadc6740 1 -- 172.21.15.1:0/672402990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 msgr2=0x555e257b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:41.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.929+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/672402990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b4a70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009fb0 tx=0x7f9edc010800 comp rx=0 tx=0).stop 2024-09-16T00:29:41.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.929+0000 7f9eeadc6740 1 -- 172.21.15.1:0/672402990 shutdown_connections 2024-09-16T00:29:41.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.929+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/672402990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b4a70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.929+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/672402990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a7dc0 0x555e257b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.929+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/672402990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257a74a0 0x555e257a7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.929+0000 7f9eeadc6740 1 -- 172.21.15.1:0/672402990 >> 172.21.15.1:0/672402990 conn(0x555e257a3a30 msgr2=0x555e257a3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.929+0000 7f9eeadc6740 1 -- 172.21.15.1:0/672402990 shutdown_connections 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.929+0000 7f9eeadc6740 1 -- 172.21.15.1:0/672402990 wait complete. 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.930+0000 7f9eeadc6740 1 -- start start 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.930+0000 7f9eeadc6740 1 Processor -- start 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.930+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a74a0 0x555e257b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.930+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a7dc0 0x555e257af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 0x555e257af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555e257b5250 con 0x555e257a7dc0 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555e256c8050 con 0x555e257b2680 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555e257b53d0 con 0x555e257a74a0 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9eeadc6740 10 client.0 _ll_register_callbacks cb 0x555e25708160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:41.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a7dc0 0x555e257af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a7dc0 0x555e257af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59174/0 (socket says 172.21.15.1:59174) 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a74a0 0x555e257b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee1d9b640 1 -- 172.21.15.1:0/176759547 learned_addr learned my addr 172.21.15.1:0/176759547 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 0x555e257af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee1d9b640 1 -- 172.21.15.1:0/176759547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a74a0 msgr2=0x555e257b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee1d9b640 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a74a0 0x555e257b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee1d9b640 1 -- 172.21.15.1:0/176759547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 msgr2=0x555e257af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee1d9b640 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 0x555e257af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee1d9b640 1 -- 172.21.15.1:0/176759547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e2547e3c0 con 0x555e257a7dc0 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ee1d9b640 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a7dc0 0x555e257af250 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9ed802aa70 tx=0x7f9ed8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784b16e4b26d90d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.931+0000 7f9ec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.932+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/176759547 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8035040 con 0x555e257a7dc0 2024-09-16T00:29:41.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.932+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/176759547 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9ed8052040 con 0x555e257a7dc0 2024-09-16T00:29:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.932+0000 7f9eeadc6740 1 -- 172.21.15.1:0/176759547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9edc012bf0 con 0x555e257a7dc0 2024-09-16T00:29:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.932+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/176759547 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ed8062050 con 0x555e257a7dc0 2024-09-16T00:29:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.932+0000 7f9eeadc6740 10 client.15561 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.932+0000 7f9eeadc6740 1 -- 172.21.15.1:0/176759547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e257c0bd0 con 0x555e257a7dc0 2024-09-16T00:29:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.932+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/176759547 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed8005500 con 0x555e257a7dc0 2024-09-16T00:29:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/176759547 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9ed803c440 con 0x555e257a7dc0 2024-09-16T00:29:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9eeadc6740 10 client.15561 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9eeadc6740 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9eeadc6740 20 client.15561 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555e257c20e0 0x555e257c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9eeadc6740 4 client.15561 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9eeadc6740 1 -- 172.21.15.1:0/176759547 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9edc012bf0 con 0x555e257c20e0 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9ee259c640 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555e257c20e0 0x555e257c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9ee259c640 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555e257c20e0 0x555e257c4540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6bd2f18e1ccd4fc server_cookie=ee8dd1eb876df768 in_seq=0 out_seq=0 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.933+0000 7f9ec9ffb640 10 client.15561 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/176759547 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9edc012bf0 con 0x555e257c20e0 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9ec9ffb640 10 client.15561 handle_command_reply: tid=0 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9eeadc6740 1 client.15561 shutdown 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9eeadc6740 1 -- 172.21.15.1:0/176759547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a7dc0 msgr2=0x555e257af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a7dc0 0x555e257af250 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9ed802aa70 tx=0x7f9ed8005fd0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9eeadc6740 1 -- 172.21.15.1:0/176759547 shutdown_connections 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555e257c20e0 0x555e257c4540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 0x555e257af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a7dc0 0x555e257af250 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/176759547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a74a0 0x555e257b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9ec9ffb640 0 client.15561 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.934+0000 7f9eeadc6740 1 -- 172.21.15.1:0/176759547 >> 172.21.15.1:0/176759547 conn(0x555e257a3a30 msgr2=0x555e257a9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.935+0000 7f9eeadc6740 1 -- 172.21.15.1:0/176759547 shutdown_connections 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.935+0000 7f9eeadc6740 1 -- 172.21.15.1:0/176759547 wait complete. 2024-09-16T00:29:41.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.935+0000 7f9eeadc6740 20 client.15561 trim_cache size 0 max 16384 2024-09-16T00:29:41.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.952+0000 7f9eeadc6740 1 Processor -- start 2024-09-16T00:29:41.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.952+0000 7f9eeadc6740 1 -- start start 2024-09-16T00:29:41.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.952+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a74a0 0x555e257be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.952+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 0x555e257c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.952+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 0x555e257ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41af4b0 con 0x555e257a74a0 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41acd80 con 0x555e257b2680 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aac10 con 0x555e257a4480 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a74a0 0x555e257be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a74a0 0x555e257be4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59180/0 (socket says 172.21.15.1:59180) 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee259c640 1 -- 172.21.15.1:0/3519970376 learned_addr learned my addr 172.21.15.1:0/3519970376 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee1d9b640 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 0x555e257c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 0x555e257ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee259c640 1 -- 172.21.15.1:0/3519970376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 msgr2=0x555e257ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee259c640 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 0x555e257ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee259c640 1 -- 172.21.15.1:0/3519970376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 msgr2=0x555e257c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee259c640 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 0x555e257c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee259c640 1 -- 172.21.15.1:0/3519970376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e257b1bc0 con 0x555e257a74a0 2024-09-16T00:29:41.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 0x555e257ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:41.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.953+0000 7f9ee259c640 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a74a0 0x555e257be4b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4000c00 tx=0x7f9ed40bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa28a6951d374f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:41.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.954+0000 7f9ecbfff640 1 -- 172.21.15.1:0/3519970376 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40b7070 con 0x555e257a74a0 2024-09-16T00:29:41.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.954+0000 7f9ecbfff640 1 -- 172.21.15.1:0/3519970376 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4099f70 con 0x555e257a74a0 2024-09-16T00:29:41.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.954+0000 7f9ecbfff640 1 -- 172.21.15.1:0/3519970376 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4020040 con 0x555e257a74a0 2024-09-16T00:29:41.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3519970376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a74a0 msgr2=0x555e257be4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:41.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a74a0 0x555e257be4b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4000c00 tx=0x7f9ed40bc040 comp rx=0 tx=0).stop 2024-09-16T00:29:41.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3519970376 shutdown_connections 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 0x555e257ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b2680 0x555e257c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3519970376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257a74a0 0x555e257be4b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3519970376 >> 172.21.15.1:0/3519970376 conn(0x555e257a3a30 msgr2=0x555e257ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3519970376 shutdown_connections 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3519970376 wait complete. 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 -- start start 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.955+0000 7f9eeadc6740 1 Processor -- start 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 0x555e257baf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9eeadc6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b8850 0x555e2582bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ab260 con 0x555e257b2680 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ac6b0 con 0x555e257b8850 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9eeadc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ab910 con 0x555e257a4480 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9eeadc6740 10 client.0 _ll_register_callbacks cb 0x7f9ed8049ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9ee259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 0x555e257baf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9ee2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b8850 0x555e2582bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9ee2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b8850 0x555e2582bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:58290/0 (socket says 172.21.15.1:58290) 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9ee2d9d640 1 -- 172.21.15.1:0/3737421325 learned_addr learned my addr 172.21.15.1:0/3737421325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.956+0000 7f9ee1d9b640 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ee2d9d640 1 -- 172.21.15.1:0/3737421325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 msgr2=0x555e257baf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 0x555e257baf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ee2d9d640 1 -- 172.21.15.1:0/3737421325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 msgr2=0x555e257b81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ee2d9d640 1 -- 172.21.15.1:0/3737421325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e257b8fc0 con 0x555e257b8850 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ee2d9d640 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b8850 0x555e2582bdf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x555e257a4db0 tx=0x7f9edc009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43242c7b6b391805 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/3737421325 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc016040 con 0x555e257b8850 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3737421325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed4099f70 con 0x555e257b8850 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/3737421325 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9edc004c00 con 0x555e257b8850 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/3737421325 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9edc034050 con 0x555e257b8850 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9eeadc6740 10 client.25214 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.957+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3737421325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e2582d3c0 con 0x555e257b8850 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.958+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/3737421325 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4099f70 con 0x555e257b8850 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.958+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/3737421325 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9edc00f040 con 0x555e257b8850 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.958+0000 7f9eeadc6740 10 client.25214 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.958+0000 7f9eeadc6740 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.958+0000 7f9eeadc6740 20 client.25214 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:41.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.958+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555e2582d810 0x555e2582fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.958+0000 7f9eeadc6740 4 client.25214 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.958+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3737421325 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e257afb20 con 0x555e2582d810 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.958+0000 7f9ee259c640 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555e2582d810 0x555e2582fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.959+0000 7f9ec9ffb640 10 client.25214 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.959+0000 7f9ee259c640 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555e2582d810 0x555e2582fc30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5cd1b0de24fc84 server_cookie=a6cc8f9f64a2993e in_seq=0 out_seq=0 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.959+0000 7f9ec9ffb640 1 -- 172.21.15.1:0/3737421325 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555e257afb20 con 0x555e2582d810 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.959+0000 7f9ec9ffb640 10 client.25214 handle_command_reply: tid=0 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.959+0000 7f9eeadc6740 1 client.25214 shutdown 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3737421325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b8850 msgr2=0x555e2582bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b8850 0x555e2582bdf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x555e257a4db0 tx=0x7f9edc009cd0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3737421325 shutdown_connections 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555e2582d810 0x555e2582fc30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e257b8850 0x555e2582bdf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e257b2680 0x555e257b81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 --2- 172.21.15.1:0/3737421325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e257a4480 0x555e257baf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3737421325 >> 172.21.15.1:0/3737421325 conn(0x555e257a3a30 msgr2=0x555e257b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:41.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9ec9ffb640 0 client.25214 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:41.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3737421325 shutdown_connections 2024-09-16T00:29:41.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 1 -- 172.21.15.1:0/3737421325 wait complete. 2024-09-16T00:29:41.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.960+0000 7f9eeadc6740 20 client.25214 trim_cache size 0 max 16384 2024-09-16T00:29:41.965 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:29:41.965 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:29:41.966 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:29:41.966 INFO:teuthology.orchestra.run.smithi001.stdout: "c07af028-e983-4555-814b-78d1def5bdd3": { 2024-09-16T00:29:41.966 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:29:41.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "c07af028-e983-4555-814b-78d1def5bdd3", 2024-09-16T00:29:41.966 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:29:41.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:41.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:41.966 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:29:41.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 -- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ecc0a3240 msgr2=0x7f9ecc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:41.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ee41049a0 tx=0x7f9ed800b040 comp rx=0 tx=0).stop 2024-09-16T00:29:41.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 -- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4100e20 msgr2=0x7f9ee419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:41.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4100e20 0x7f9ee419a500 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400d950 tx=0x7f9ed400de10 comp rx=0 tx=0).stop 2024-09-16T00:29:41.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 -- 172.21.15.1:0/934792559 shutdown_connections 2024-09-16T00:29:41.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee4102ef0 0x7f9ee419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee4102570 0x7f9ee419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 --2- 172.21.15.1:0/934792559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ee4100e20 0x7f9ee419a500 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:41.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 -- 172.21.15.1:0/934792559 >> 172.21.15.1:0/934792559 conn(0x7f9ee40fc7d0 msgr2=0x7f9ee406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:41.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 -- 172.21.15.1:0/934792559 shutdown_connections 2024-09-16T00:29:41.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:41.961+0000 7f9ee89ed640 1 -- 172.21.15.1:0/934792559 wait complete. 2024-09-16T00:29:42.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:41 smithi001 ceph-mon[30934]: pgmap v389: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:29:42.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:41 smithi019 ceph-mon[35186]: pgmap v389: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:29:42.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:41 smithi153 ceph-mon[35538]: pgmap v389: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:29:42.673 INFO:tasks.cephfs.filesystem:scrub status for tag:c07af028-e983-4555-814b-78d1def5bdd3 - {'path': '/', 'tag': 'c07af028-e983-4555-814b-78d1def5bdd3', 'options': 'recursive,force'} 2024-09-16T00:29:43.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:42 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:29:43.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:42 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:29:43.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:42 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:29:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:43 smithi001 ceph-mon[30934]: pgmap v390: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T00:29:44.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:43 smithi019 ceph-mon[35186]: pgmap v390: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T00:29:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:43 smithi153 ceph-mon[35538]: pgmap v390: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T00:29:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:45 smithi019 ceph-mon[35186]: pgmap v391: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:29:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:45 smithi001 ceph-mon[30934]: pgmap v391: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:29:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:45 smithi153 ceph-mon[35538]: pgmap v391: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:47 smithi019 ceph-mon[35186]: pgmap v392: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:47 smithi001 ceph-mon[30934]: pgmap v392: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:47 smithi153 ceph-mon[35538]: pgmap v392: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:50.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:49 smithi019 ceph-mon[35186]: pgmap v393: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:50.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:49 smithi001 ceph-mon[30934]: pgmap v393: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:49 smithi153 ceph-mon[35538]: pgmap v393: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:29:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:51 smithi019 ceph-mon[35186]: pgmap v394: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:52.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:51 smithi001 ceph-mon[30934]: pgmap v394: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:51 smithi153 ceph-mon[35538]: pgmap v394: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:53 smithi019 ceph-mon[35186]: pgmap v395: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:53 smithi001 ceph-mon[30934]: pgmap v395: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:53 smithi153 ceph-mon[35538]: pgmap v395: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:55.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:55 smithi019 ceph-mon[35186]: pgmap v396: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:29:56.361 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:55 smithi001 ceph-mon[30934]: pgmap v396: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:29:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:55 smithi153 ceph-mon[35538]: pgmap v396: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:29:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.950+0000 7f6425149640 1 -- 172.21.15.1:0/1261487745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 msgr2=0x7f642010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.950+0000 7f6425149640 1 --2- 172.21.15.1:0/1261487745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 0x7f642010da40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6418009ca0 tx=0x7f641802a950 comp rx=0 tx=0).stop 2024-09-16T00:29:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.951+0000 7f6425149640 1 -- 172.21.15.1:0/1261487745 shutdown_connections 2024-09-16T00:29:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.951+0000 7f6425149640 1 --2- 172.21.15.1:0/1261487745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 0x7f642010da40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.951+0000 7f6425149640 1 --2- 172.21.15.1:0/1261487745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6420102570 0x7f64201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.951+0000 7f6425149640 1 --2- 172.21.15.1:0/1261487745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6420100e20 0x7f6420101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.951+0000 7f6425149640 1 -- 172.21.15.1:0/1261487745 >> 172.21.15.1:0/1261487745 conn(0x7f64200fc7d0 msgr2=0x7f64200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.951+0000 7f6425149640 1 -- 172.21.15.1:0/1261487745 shutdown_connections 2024-09-16T00:29:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.951+0000 7f6425149640 1 -- 172.21.15.1:0/1261487745 wait complete. 2024-09-16T00:29:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.952+0000 7f6425149640 1 Processor -- start 2024-09-16T00:29:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.952+0000 7f6425149640 1 -- start start 2024-09-16T00:29:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.952+0000 7f6425149640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6420100e20 0x7f6420199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.952+0000 7f6425149640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6420102570 0x7f64201998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.952+0000 7f6425149640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 0x7f64201a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.952+0000 7f6425149640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6420110f80 con 0x7f6420102ef0 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.952+0000 7f6425149640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6420110e00 con 0x7f6420100e20 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.952+0000 7f6425149640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6420111100 con 0x7f6420102570 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 0x7f64201a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 0x7f64201a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59620/0 (socket says 172.21.15.1:59620) 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641f59e640 1 -- 172.21.15.1:0/3161480293 learned_addr learned my addr 172.21.15.1:0/3161480293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641ed9d640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6420100e20 0x7f6420199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641f59e640 1 -- 172.21.15.1:0/3161480293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6420102570 msgr2=0x7f64201998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641f59e640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6420102570 0x7f64201998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641f59e640 1 -- 172.21.15.1:0/3161480293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6420100e20 msgr2=0x7f6420199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641f59e640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6420100e20 0x7f6420199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641f59e640 1 -- 172.21.15.1:0/3161480293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64201a4930 con 0x7f6420102ef0 2024-09-16T00:29:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.953+0000 7f641ed9d640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6420100e20 0x7f6420199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.954+0000 7f641f59e640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 0x7f64201a43f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6418005e30 tx=0x7f6418003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.954+0000 7f6407fff640 1 -- 172.21.15.1:0/3161480293 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418034040 con 0x7f6420102ef0 2024-09-16T00:29:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.954+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6420197160 con 0x7f6420102ef0 2024-09-16T00:29:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.954+0000 7f6407fff640 1 -- 172.21.15.1:0/3161480293 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f641802f930 con 0x7f6420102ef0 2024-09-16T00:29:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.954+0000 7f6407fff640 1 -- 172.21.15.1:0/3161480293 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64180045e0 con 0x7f6420102ef0 2024-09-16T00:29:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.954+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6420197690 con 0x7f6420102ef0 2024-09-16T00:29:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.956+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6420101200 con 0x7f6420102ef0 2024-09-16T00:29:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.957+0000 7f6407fff640 1 -- 172.21.15.1:0/3161480293 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6418037020 con 0x7f6420102ef0 2024-09-16T00:29:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.958+0000 7f6407fff640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f64080a3280 0x7f64080a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.959+0000 7f6407fff640 1 -- 172.21.15.1:0/3161480293 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f64180f0b70 con 0x7f6420102ef0 2024-09-16T00:29:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.959+0000 7f641ed9d640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f64080a3280 0x7f64080a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.959+0000 7f641ed9d640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f64080a3280 0x7f64080a5740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6410009ab0 tx=0x7f6410016040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:56.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:56.967+0000 7f6407fff640 1 -- 172.21.15.1:0/3161480293 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64180bd490 con 0x7f6420102ef0 2024-09-16T00:29:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.393+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6420197860 con 0x7f6420102ef0 2024-09-16T00:29:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.394+0000 7f6407fff640 1 -- 172.21.15.1:0/3161480293 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f64180bcd90 con 0x7f6420102ef0 2024-09-16T00:29:57.395 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:29:57.395 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:29:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:29:57.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f64080a3280 msgr2=0x7f64080a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:57.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f64080a3280 0x7f64080a5740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6410009ab0 tx=0x7f6410016040 comp rx=0 tx=0).stop 2024-09-16T00:29:57.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 msgr2=0x7f64201a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:57.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 0x7f64201a43f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6418005e30 tx=0x7f6418003c10 comp rx=0 tx=0).stop 2024-09-16T00:29:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 shutdown_connections 2024-09-16T00:29:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6420102ef0 0x7f64201a43f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f64080a3280 0x7f64080a5740 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6420102570 0x7f64201998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 --2- 172.21.15.1:0/3161480293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6420100e20 0x7f6420199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 >> 172.21.15.1:0/3161480293 conn(0x7f64200fc7d0 msgr2=0x7f6420106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.396+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 shutdown_connections 2024-09-16T00:29:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:57.397+0000 7f6425149640 1 -- 172.21.15.1:0/3161480293 wait complete. 2024-09-16T00:29:58.097 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 8193ff70 --release --await' 2024-09-16T00:29:58.097 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 8193ff70 --release --await 2024-09-16T00:29:58.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:58 smithi019 ceph-mon[35186]: pgmap v397: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:58.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:29:58 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3161480293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:58.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:58 smithi001 ceph-mon[30934]: pgmap v397: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:58.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:29:58 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3161480293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:58 smithi153 ceph-mon[35538]: pgmap v397: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:29:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:29:58 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3161480293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.008+0000 7f212fca9640 1 -- 172.21.15.1:0/1891285876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2128100430 msgr2=0x7f2128100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.008+0000 7f212fca9640 1 --2- 172.21.15.1:0/1891285876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2128100430 0x7f2128100810 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f212400aa10 tx=0x7f212402b720 comp rx=0 tx=0).stop 2024-09-16T00:29:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.008+0000 7f212fca9640 1 -- 172.21.15.1:0/1891285876 shutdown_connections 2024-09-16T00:29:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.008+0000 7f212fca9640 1 --2- 172.21.15.1:0/1891285876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128105560 0x7f212810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.008+0000 7f212fca9640 1 --2- 172.21.15.1:0/1891285876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2128100de0 0x7f2128104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.008+0000 7f212fca9640 1 --2- 172.21.15.1:0/1891285876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2128100430 0x7f2128100810 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.008+0000 7f212fca9640 1 -- 172.21.15.1:0/1891285876 >> 172.21.15.1:0/1891285876 conn(0x7f2128077230 msgr2=0x7f21280fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.009+0000 7f212fca9640 1 -- 172.21.15.1:0/1891285876 shutdown_connections 2024-09-16T00:29:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.009+0000 7f212fca9640 1 -- 172.21.15.1:0/1891285876 wait complete. 2024-09-16T00:29:59.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.009+0000 7f212fca9640 1 Processor -- start 2024-09-16T00:29:59.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.010+0000 7f212fca9640 1 -- start start 2024-09-16T00:29:59.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.010+0000 7f212fca9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2128100de0 0x7f2128198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212fca9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128105560 0x7f2128198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212fca9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f212819cf00 0x7f212819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212da45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2128100de0 0x7f2128198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212da45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2128100de0 0x7f2128198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:39454/0 (socket says 172.21.15.1:39454) 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212e246640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f212819cf00 0x7f212819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212e246640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f212819cf00 0x7f212819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59646/0 (socket says 172.21.15.1:59646) 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212d244640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128105560 0x7f2128198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212d244640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128105560 0x7f2128198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:36738/0 (socket says 172.21.15.1:36738) 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212fca9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f212810fd40 con 0x7f212819cf00 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212da45640 1 -- 172.21.15.1:0/304604329 learned_addr learned my addr 172.21.15.1:0/304604329 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f212810fbc0 con 0x7f2128100de0 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.011+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f212810fec0 con 0x7f2128105560 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f212da45640 1 -- 172.21.15.1:0/304604329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128105560 msgr2=0x7f2128198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f212da45640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128105560 0x7f2128198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f212da45640 1 -- 172.21.15.1:0/304604329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f212819cf00 msgr2=0x7f212819d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f212da45640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f212819cf00 0x7f212819d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f212da45640 1 -- 172.21.15.1:0/304604329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212819d9c0 con 0x7f2128100de0 2024-09-16T00:29:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f212da45640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2128100de0 0x7f2128198630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2124005940 tx=0x7f2124005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:59.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f211effd640 1 -- 172.21.15.1:0/304604329 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124036040 con 0x7f2128100de0 2024-09-16T00:29:59.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f211effd640 1 -- 172.21.15.1:0/304604329 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2124041030 con 0x7f2128100de0 2024-09-16T00:29:59.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212819dc50 con 0x7f2128100de0 2024-09-16T00:29:59.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f211effd640 1 -- 172.21.15.1:0/304604329 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124003e50 con 0x7f2128100de0 2024-09-16T00:29:59.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.012+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2128196350 con 0x7f2128100de0 2024-09-16T00:29:59.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.015+0000 7f211effd640 1 -- 172.21.15.1:0/304604329 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f21240035f0 con 0x7f2128100de0 2024-09-16T00:29:59.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.016+0000 7f2132082740 1 Processor -- start 2024-09-16T00:29:59.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.016+0000 7f2132082740 1 -- start start 2024-09-16T00:29:59.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.016+0000 7f2132082740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 0x556ea37c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.016+0000 7f2132082740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.016+0000 7f2132082740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.016+0000 7f2132082740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556ea371f620 con 0x556ea37c3190 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.016+0000 7f2132082740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556ea36c6e50 con 0x556ea37cda50 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.016+0000 7f2132082740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556ea371b5e0 con 0x556ea37c2870 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.017+0000 7f212e246640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 0x556ea37c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.017+0000 7f212d244640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.017+0000 7f212da45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.017+0000 7f212d244640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:39464/0 (socket says 172.21.15.1:39464) 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.017+0000 7f212d244640 1 -- 172.21.15.1:0/2131961525 learned_addr learned my addr 172.21.15.1:0/2131961525 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.017+0000 7f211effd640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f21140a3460 0x7f21140a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.017+0000 7f212d244640 1 -- 172.21.15.1:0/2131961525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 msgr2=0x556ea37c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.017+0000 7f212d244640 1 --2- 172.21.15.1:0/2131961525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 0x556ea37c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.017+0000 7f212d244640 1 -- 172.21.15.1:0/2131961525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 msgr2=0x556ea37cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:59.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f212d244640 1 --2- 172.21.15.1:0/2131961525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f212d244640 1 -- 172.21.15.1:0/2131961525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea36b7d40 con 0x556ea37cda50 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f212e246640 1 --2- 172.21.15.1:0/2131961525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 0x556ea37c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f212da45640 1 --2- 172.21.15.1:0/2131961525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f211effd640 1 -- 172.21.15.1:0/304604329 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f21240f41c0 con 0x7f2128100de0 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f212d244640 1 --2- 172.21.15.1:0/2131961525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cfe40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2118009c90 tx=0x7f2118011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d36db5de41cd261 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f212e246640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f21140a3460 0x7f21140a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f2113fff640 1 -- 172.21.15.1:0/2131961525 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118005220 con 0x556ea37cda50 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f2113fff640 1 -- 172.21.15.1:0/2131961525 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2118004c80 con 0x556ea37cda50 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.018+0000 7f2113fff640 1 -- 172.21.15.1:0/2131961525 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118019920 con 0x556ea37cda50 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f2132082740 1 -- 172.21.15.1:0/2131961525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 msgr2=0x556ea37cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f2132082740 1 --2- 172.21.15.1:0/2131961525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cfe40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2118009c90 tx=0x7f2118011810 comp rx=0 tx=0).stop 2024-09-16T00:29:59.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f212e246640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f21140a3460 0x7f21140a5920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2120013b40 tx=0x7f2120004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f2132082740 1 -- 172.21.15.1:0/2131961525 shutdown_connections 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f2132082740 1 --2- 172.21.15.1:0/2131961525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cfe40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f2132082740 1 --2- 172.21.15.1:0/2131961525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f2132082740 1 --2- 172.21.15.1:0/2131961525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 0x556ea37c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f2132082740 1 -- 172.21.15.1:0/2131961525 >> 172.21.15.1:0/2131961525 conn(0x556ea37bee00 msgr2=0x556ea37bf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f2132082740 1 -- 172.21.15.1:0/2131961525 shutdown_connections 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.019+0000 7f2132082740 1 -- 172.21.15.1:0/2131961525 wait complete. 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f2132082740 1 -- start start 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f2132082740 1 Processor -- start 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f2132082740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c2870 0x556ea37ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f2132082740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 0x556ea37cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f2132082740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f2132082740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556ea37d0620 con 0x556ea37c2870 2024-09-16T00:29:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f2132082740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556ea36a7f10 con 0x556ea37cda50 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f2132082740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556ea37d07a0 con 0x556ea37c3190 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f2132082740 10 client.0 _ll_register_callbacks cb 0x556ea36cbdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f212da45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c2870 0x556ea37ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.020+0000 7f212da45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c2870 0x556ea37ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59674/0 (socket says 172.21.15.1:59674) 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212da45640 1 -- 172.21.15.1:0/3588358322 learned_addr learned my addr 172.21.15.1:0/3588358322 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212e246640 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212da45640 1 -- 172.21.15.1:0/3588358322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 msgr2=0x556ea37cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212d244640 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 0x556ea37cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212da45640 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 0x556ea37cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212da45640 1 -- 172.21.15.1:0/3588358322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 msgr2=0x556ea37cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212da45640 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212da45640 1 -- 172.21.15.1:0/3588358322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ea342e580 con 0x556ea37c2870 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212d244640 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 0x556ea37cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212e246640 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f212da45640 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c2870 0x556ea37ca910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f21240f1860 tx=0x7f212403df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573b1eb0f7ccd23d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f21117fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f21117fa640 1 -- 172.21.15.1:0/3588358322 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212404b070 con 0x556ea37c2870 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f21117fa640 1 -- 172.21.15.1:0/3588358322 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f212403b110 con 0x556ea37c2870 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.021+0000 7f2132082740 1 -- 172.21.15.1:0/3588358322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2118004c80 con 0x556ea37c2870 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f21117fa640 1 -- 172.21.15.1:0/3588358322 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f21240bff70 con 0x556ea37c2870 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f2132082740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f2132082740 1 -- 172.21.15.1:0/3588358322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ea34f8960 con 0x556ea37c2870 2024-09-16T00:29:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f21117fa640 1 -- 172.21.15.1:0/3588358322 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2118004c80 con 0x556ea37c2870 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f21117fa640 1 -- 172.21.15.1:0/3588358322 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f212400b040 con 0x556ea37c2870 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f2132082740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f2132082740 10 client.15582 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f2132082740 20 client.15582 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f2132082740 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556ea37ee590 0x556ea37f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f2132082740 4 client.15582 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f2132082740 1 -- 172.21.15.1:0/3588358322 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ea372cb90 con 0x556ea37ee590 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.022+0000 7f212d244640 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556ea37ee590 0x556ea37f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.023+0000 7f21117fa640 10 client.15582 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.023+0000 7f212d244640 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556ea37ee590 0x556ea37f0a50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96d0be0269cbfe9 server_cookie=5c4d7062cee68ae5 in_seq=0 out_seq=0 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f21117fa640 1 -- 172.21.15.1:0/3588358322 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ea372cb90 con 0x556ea37ee590 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f21117fa640 10 client.15582 handle_command_reply: tid=0 2024-09-16T00:29:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 client.15582 shutdown 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 -- 172.21.15.1:0/3588358322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c2870 msgr2=0x556ea37ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c2870 0x556ea37ca910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f21240f1860 tx=0x7f212403df60 comp rx=0 tx=0).stop 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 -- 172.21.15.1:0/3588358322 shutdown_connections 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556ea37ee590 0x556ea37f0a50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f21117fa640 0 client.15582 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 0x556ea37cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 --2- 172.21.15.1:0/3588358322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c2870 0x556ea37ca910 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 -- 172.21.15.1:0/3588358322 >> 172.21.15.1:0/3588358322 conn(0x556ea37bee00 msgr2=0x556ea37b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 -- 172.21.15.1:0/3588358322 shutdown_connections 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 1 -- 172.21.15.1:0/3588358322 wait complete. 2024-09-16T00:29:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.024+0000 7f2132082740 20 client.15582 trim_cache size 0 max 16384 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f2132082740 1 Processor -- start 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f2132082740 1 -- start start 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f2132082740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37c2870 0x556ea37c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f2132082740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 0x556ea37f6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f2132082740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37cda50 0x556ea37f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f212e246640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37cda50 0x556ea37f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f212e246640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37cda50 0x556ea37f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59688/0 (socket says 172.21.15.1:59688) 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f212e246640 1 -- 172.21.15.1:0/1100385350 learned_addr learned my addr 172.21.15.1:0/1100385350 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f2132082740 1 -- 172.21.15.1:0/1100385350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f21281a7300 con 0x556ea37cda50 2024-09-16T00:29:59.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f2132082740 1 -- 172.21.15.1:0/1100385350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2128073e30 con 0x556ea37c2870 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f212d244640 1 --2- 172.21.15.1:0/1100385350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 0x556ea37f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f2132082740 1 -- 172.21.15.1:0/1100385350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2128071c80 con 0x556ea37c3190 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f212e246640 1 -- 172.21.15.1:0/1100385350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 msgr2=0x556ea37f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f212e246640 1 --2- 172.21.15.1:0/1100385350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 0x556ea37f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f212e246640 1 -- 172.21.15.1:0/1100385350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37c2870 msgr2=0x556ea37c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f212e246640 1 --2- 172.21.15.1:0/1100385350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37c2870 0x556ea37c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.042+0000 7f212e246640 1 -- 172.21.15.1:0/1100385350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea37c98c0 con 0x556ea37cda50 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.043+0000 7f212e246640 1 --2- 172.21.15.1:0/1100385350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37cda50 0x556ea37f68a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2120013530 tx=0x7f2120007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0fc3ffc1dd15ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.043+0000 7f21137fe640 1 -- 172.21.15.1:0/1100385350 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120019030 con 0x556ea37cda50 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.043+0000 7f21137fe640 1 -- 172.21.15.1:0/1100385350 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2120027070 con 0x556ea37cda50 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.043+0000 7f21137fe640 1 -- 172.21.15.1:0/1100385350 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120022a60 con 0x556ea37cda50 2024-09-16T00:29:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.044+0000 7f2132082740 1 -- 172.21.15.1:0/1100385350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37cda50 msgr2=0x556ea37f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.044+0000 7f2132082740 1 --2- 172.21.15.1:0/1100385350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37cda50 0x556ea37f68a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2120013530 tx=0x7f2120007bd0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.044+0000 7f2132082740 1 -- 172.21.15.1:0/1100385350 shutdown_connections 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.044+0000 7f2132082740 1 --2- 172.21.15.1:0/1100385350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37cda50 0x556ea37f68a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.044+0000 7f2132082740 1 --2- 172.21.15.1:0/1100385350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c3190 0x556ea37f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.044+0000 7f2132082740 1 --2- 172.21.15.1:0/1100385350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37c2870 0x556ea37c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.044+0000 7f2132082740 1 -- 172.21.15.1:0/1100385350 >> 172.21.15.1:0/1100385350 conn(0x556ea37bee00 msgr2=0x556ea37cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.045+0000 7f2132082740 1 -- 172.21.15.1:0/1100385350 shutdown_connections 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.045+0000 7f2132082740 1 -- 172.21.15.1:0/1100385350 wait complete. 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.045+0000 7f2132082740 1 -- start start 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.045+0000 7f2132082740 1 Processor -- start 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f2132082740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 0x556ea37e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f2132082740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37e3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f2132082740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f2132082740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f21280722f0 con 0x556ea37c3190 2024-09-16T00:29:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f2132082740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2128073760 con 0x556ea37cda50 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f2132082740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f21280729c0 con 0x556ea37c2870 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f2132082740 10 client.0 _ll_register_callbacks cb 0x7f2124030510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212d244640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37e3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212d244640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37e3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59700/0 (socket says 172.21.15.1:59700) 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212d244640 1 -- 172.21.15.1:0/2415911611 learned_addr learned my addr 172.21.15.1:0/2415911611 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212da45640 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 0x556ea37e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212e246640 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212d244640 1 -- 172.21.15.1:0/2415911611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 msgr2=0x556ea37e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212d244640 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 0x556ea37e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212d244640 1 -- 172.21.15.1:0/2415911611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 msgr2=0x556ea37e4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212d244640 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37e4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.046+0000 7f212d244640 1 -- 172.21.15.1:0/2415911611 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ea37e4840 con 0x556ea37c3190 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.047+0000 7f212d244640 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37e3a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2118014040 tx=0x7f21180055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4f39a20565e1a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.047+0000 7f21117fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.047+0000 7f21117fa640 1 -- 172.21.15.1:0/2415911611 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118032060 con 0x556ea37c3190 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.047+0000 7f21117fa640 1 -- 172.21.15.1:0/2415911611 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f21180035f0 con 0x556ea37c3190 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.047+0000 7f2132082740 1 -- 172.21.15.1:0/2415911611 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2120027070 con 0x556ea37c3190 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.047+0000 7f21117fa640 1 -- 172.21.15.1:0/2415911611 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f21180098c0 con 0x556ea37c3190 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.047+0000 7f2132082740 10 client.15588 fetch_fsmap learned FSMap version 25 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.047+0000 7f2132082740 1 -- 172.21.15.1:0/2415911611 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ea385a460 con 0x556ea37c3190 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.047+0000 7f21117fa640 1 -- 172.21.15.1:0/2415911611 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2120027070 con 0x556ea37c3190 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f21117fa640 1 -- 172.21.15.1:0/2415911611 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2118019990 con 0x556ea37c3190 2024-09-16T00:29:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f2132082740 10 client.15588 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:29:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f2132082740 10 client.15588 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:29:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f2132082740 20 client.15588 populate_metadata read hostname 'smithi001' 2024-09-16T00:29:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f2132082740 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556ea37ee590 0x556ea385c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f2132082740 4 client.15588 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8193ff70", "release": true, "await": true}] 2024-09-16T00:29:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f2132082740 1 -- 172.21.15.1:0/2415911611 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8193ff70", "release": true, "await": true}) -- 0x556ea37270a0 con 0x556ea37ee590 2024-09-16T00:29:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f212da45640 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556ea37ee590 0x556ea385c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f21117fa640 10 client.15588 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.048+0000 7f212da45640 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556ea37ee590 0x556ea385c990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda00466f2856e98 server_cookie=319a9440708fe7d in_seq=0 out_seq=0 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f21117fa640 1 -- 172.21.15.1:0/2415911611 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556ea37270a0 con 0x556ea37ee590 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f21117fa640 10 client.15588 handle_command_reply: tid=0 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 client.15588 shutdown 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 -- 172.21.15.1:0/2415911611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 msgr2=0x556ea37e3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37e3a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2118014040 tx=0x7f21180055c0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 -- 172.21.15.1:0/2415911611 shutdown_connections 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556ea37ee590 0x556ea385c990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f21117fa640 0 client.15588 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ea37cda50 0x556ea37e4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ea37c3190 0x556ea37e3a50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 --2- 172.21.15.1:0/2415911611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ea37c2870 0x556ea37e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 -- 172.21.15.1:0/2415911611 >> 172.21.15.1:0/2415911611 conn(0x556ea37bee00 msgr2=0x556ea37e9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:59.058 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:29:59.058 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:29:59.058 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:29:59.058 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 4, 2024-09-16T00:29:59.058 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:29:59.058 INFO:teuthology.orchestra.run.smithi001.stdout: "8193ff70": { 2024-09-16T00:29:59.058 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 4, 2024-09-16T00:29:59.058 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 150.2, 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:59.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 -- 172.21.15.1:0/2415911611 shutdown_connections 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 1 -- 172.21.15.1:0/2415911611 wait complete. 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.051+0000 7f2132082740 20 client.15588 trim_cache size 0 max 16384 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f21140a3460 msgr2=0x7f21140a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f21140a3460 0x7f21140a5920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2120013b40 tx=0x7f2120004ab0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2128100de0 msgr2=0x7f2128198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2128100de0 0x7f2128198630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2124005940 tx=0x7f2124005dc0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 shutdown_connections 2024-09-16T00:29:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f212819cf00 0x7f212819d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f21140a3460 0x7f21140a5920 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128105560 0x7f2128198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 --2- 172.21.15.1:0/304604329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2128100de0 0x7f2128198630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 >> 172.21.15.1:0/304604329 conn(0x7f2128077230 msgr2=0x7f21281038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 shutdown_connections 2024-09-16T00:29:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:29:59.052+0000 7f212fca9640 1 -- 172.21.15.1:0/304604329 wait complete. 2024-09-16T00:29:59.782 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8193ff70', total seconds elapsed: 34.0 2024-09-16T00:30:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:00 smithi019 ceph-mon[35186]: pgmap v398: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:30:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:00 smithi019 ceph-mon[35186]: overall HEALTH_OK 2024-09-16T00:30:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:00 smithi001 ceph-mon[30934]: pgmap v398: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:30:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:00 smithi001 ceph-mon[30934]: overall HEALTH_OK 2024-09-16T00:30:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:00 smithi153 ceph-mon[35538]: pgmap v398: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:30:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:00 smithi153 ceph-mon[35538]: overall HEALTH_OK 2024-09-16T00:30:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:02 smithi019 ceph-mon[35186]: pgmap v399: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:30:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:02 smithi001 ceph-mon[30934]: pgmap v399: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:30:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:02 smithi153 ceph-mon[35538]: pgmap v399: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:30:03.915 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 92% (11921/12889) Updating files: 93% (11987/12889) Updating files: 94% (12116/12889) Updating files: 95% (12245/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-16T00:30:04.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:04 smithi019 ceph-mon[35186]: pgmap v400: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:30:04.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:04 smithi001 ceph-mon[30934]: pgmap v400: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:30:04.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:04 smithi153 ceph-mon[35538]: pgmap v400: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:30:04.852 INFO:tasks.workunit.client.0.smithi001.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr: git switch -c 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr:Or undo this operation with: 2024-09-16T00:30:04.853 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-16T00:30:04.854 INFO:tasks.workunit.client.0.smithi001.stderr: git switch - 2024-09-16T00:30:04.854 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-16T00:30:04.854 INFO:tasks.workunit.client.0.smithi001.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T00:30:04.854 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-16T00:30:04.854 INFO:tasks.workunit.client.0.smithi001.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T00:30:04.858 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T00:30:04.885 INFO:tasks.workunit.client.0.smithi001.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T00:30:04.887 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:30:04.887 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T00:30:05.580 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T00:30:05.654 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T00:30:05.696 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:30:05.697 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:30:05.697 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T00:30:05.738 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:30:05.742 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:30:05.742 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T00:30:05.801 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-16T00:30:05.802 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-16T00:30:05.802 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-16T00:30:05.863 INFO:tasks.workunit.client.0.smithi001.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-16T00:30:05.864 INFO:tasks.workunit.client.0.smithi001.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-16T00:30:05.864 INFO:tasks.workunit.client.0.smithi001.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-16T00:30:06.264 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:06 smithi153 ceph-mon[35538]: pgmap v401: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:06.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:06 smithi019 ceph-mon[35186]: pgmap v401: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:06.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:06 smithi001 ceph-mon[30934]: pgmap v401: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:08.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:08 smithi019 ceph-mon[35186]: pgmap v402: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:30:08.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:08 smithi001 ceph-mon[30934]: pgmap v402: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:30:08.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:08 smithi153 ceph-mon[35538]: pgmap v402: 129 pgs: 129 active+clean; 658 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:30:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:10 smithi019 ceph-mon[35186]: pgmap v403: 129 pgs: 129 active+clean; 677 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-16T00:30:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:10 smithi001 ceph-mon[30934]: pgmap v403: 129 pgs: 129 active+clean; 677 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-16T00:30:10.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:10 smithi153 ceph-mon[35538]: pgmap v403: 129 pgs: 129 active+clean; 677 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-16T00:30:12.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:12 smithi019 ceph-mon[35186]: pgmap v404: 129 pgs: 129 active+clean; 677 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-16T00:30:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:12 smithi001 ceph-mon[30934]: pgmap v404: 129 pgs: 129 active+clean; 677 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-16T00:30:12.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:12 smithi153 ceph-mon[35538]: pgmap v404: 129 pgs: 129 active+clean; 677 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-16T00:30:12.673 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:30:12.948 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 -- 172.21.15.1:0/704355475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8102570 msgr2=0x7f75b81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/704355475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8102570 0x7f75b81029b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009900 tx=0x7f75a802a480 comp rx=0 tx=0).stop 2024-09-16T00:30:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 -- 172.21.15.1:0/704355475 shutdown_connections 2024-09-16T00:30:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/704355475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b8102ef0 0x7f75b810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/704355475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8102570 0x7f75b81029b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/704355475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b8100e20 0x7f75b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 -- 172.21.15.1:0/704355475 >> 172.21.15.1:0/704355475 conn(0x7f75b80fc7d0 msgr2=0x7f75b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 -- 172.21.15.1:0/704355475 shutdown_connections 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 -- 172.21.15.1:0/704355475 wait complete. 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.440+0000 7f75bfaa7640 1 Processor -- start 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bfaa7640 1 -- start start 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bfaa7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8100e20 0x7f75b8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bfaa7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b8102570 0x7f75b8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bfaa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b8102ef0 0x7f75b81a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bfaa7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8110f80 con 0x7f75b8100e20 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bfaa7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8110e00 con 0x7f75b8102ef0 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bfaa7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8111100 con 0x7f75b8102570 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bd843640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8100e20 0x7f75b8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bd843640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8100e20 0x7f75b8199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53466/0 (socket says 172.21.15.1:53466) 2024-09-16T00:30:13.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bd843640 1 -- 172.21.15.1:0/1871758241 learned_addr learned my addr 172.21.15.1:0/1871758241 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bd042640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b8102570 0x7f75b8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75be044640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b8102ef0 0x7f75b81a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bd843640 1 -- 172.21.15.1:0/1871758241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b8102570 msgr2=0x7f75b8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bd843640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b8102570 0x7f75b8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bd843640 1 -- 172.21.15.1:0/1871758241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b8102ef0 msgr2=0x7f75b81a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bd843640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b8102ef0 0x7f75b81a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75bd843640 1 -- 172.21.15.1:0/1871758241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b81a48d0 con 0x7f75b8100e20 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.441+0000 7f75be044640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b8102ef0 0x7f75b81a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.442+0000 7f75bd843640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8100e20 0x7f75b8199310 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f75b400d950 tx=0x7f75b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.442+0000 7f75aeffd640 1 -- 172.21.15.1:0/1871758241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4017070 con 0x7f75b8100e20 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.442+0000 7f75bfaa7640 1 -- 172.21.15.1:0/1871758241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b81a4aa0 con 0x7f75b8100e20 2024-09-16T00:30:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.442+0000 7f75aeffd640 1 -- 172.21.15.1:0/1871758241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b4009c30 con 0x7f75b8100e20 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.442+0000 7f75aeffd640 1 -- 172.21.15.1:0/1871758241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b400f6b0 con 0x7f75b8100e20 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.442+0000 7f75bfaa7640 1 -- 172.21.15.1:0/1871758241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b81a5030 con 0x7f75b8100e20 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.444+0000 7f75c1e80740 1 Processor -- start 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75c1e80740 1 -- start start 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4764380 0x5598a4764760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a4773550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4773a90 0x5598a4775e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75c1e80740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5598a46ca2e0 con 0x5598a4773a90 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75c1e80740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598a4671ce0 con 0x5598a4764380 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75c1e80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5598a46c62a0 con 0x5598a4764ca0 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75aeffd640 1 -- 172.21.15.1:0/1871758241 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f75b40109d0 con 0x7f75b8100e20 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75bd042640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4773a90 0x5598a4775e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75bd042640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4773a90 0x5598a4775e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53474/0 (socket says 172.21.15.1:53474) 2024-09-16T00:30:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75bd042640 1 -- 172.21.15.1:0/24515054 learned_addr learned my addr 172.21.15.1:0/24515054 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.445+0000 7f75be044640 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4764380 0x5598a4764760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75bd843640 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a4773550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75bd042640 1 -- 172.21.15.1:0/24515054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 msgr2=0x5598a4773550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75bd042640 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a4773550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75bd042640 1 -- 172.21.15.1:0/24515054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4764380 msgr2=0x5598a4764760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75bd042640 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4764380 0x5598a4764760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75bd042640 1 -- 172.21.15.1:0/24515054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a4656440 con 0x5598a4773a90 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75bd843640 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a4773550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75be044640 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4764380 0x5598a4764760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75bd042640 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4773a90 0x5598a4775e80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75a80314e0 tx=0x7f75a8004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858c3ac6dc9ee568 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75a3fff640 1 -- 172.21.15.1:0/24515054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8035030 con 0x5598a4773a90 2024-09-16T00:30:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75aeffd640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f75a40a3240 0x7f75a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75a3fff640 1 -- 172.21.15.1:0/24515054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a8005520 con 0x5598a4773a90 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75a3fff640 1 -- 172.21.15.1:0/24515054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a80037f0 con 0x5598a4773a90 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75be044640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f75a40a3240 0x7f75a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.446+0000 7f75aeffd640 1 -- 172.21.15.1:0/1871758241 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f75b40cf9d0 con 0x7f75b8100e20 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75be044640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f75a40a3240 0x7f75a40a5700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005fd0 tx=0x7f75b00085b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75c1e80740 1 -- 172.21.15.1:0/24515054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4773a90 msgr2=0x5598a4775e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75c1e80740 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4773a90 0x5598a4775e80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75a80314e0 tx=0x7f75a8004170 comp rx=0 tx=0).stop 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75c1e80740 1 -- 172.21.15.1:0/24515054 shutdown_connections 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75c1e80740 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4773a90 0x5598a4775e80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75c1e80740 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a4773550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75c1e80740 1 --2- 172.21.15.1:0/24515054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4764380 0x5598a4764760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75c1e80740 1 -- 172.21.15.1:0/24515054 >> 172.21.15.1:0/24515054 conn(0x5598a4760a30 msgr2=0x5598a4760e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75c1e80740 1 -- 172.21.15.1:0/24515054 shutdown_connections 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.447+0000 7f75c1e80740 1 -- 172.21.15.1:0/24515054 wait complete. 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75c1e80740 1 -- start start 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75c1e80740 1 Processor -- start 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 0x5598a4773310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a4770610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4770b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75c1e80740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5598a4776660 con 0x5598a4764ca0 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75c1e80740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598a4685050 con 0x5598a4773a90 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75c1e80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5598a47767e0 con 0x5598a4764380 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75c1e80740 10 client.0 _ll_register_callbacks cb 0x5598a46c5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75bd042640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a4770610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75bd042640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a4770610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53478/0 (socket says 172.21.15.1:53478) 2024-09-16T00:30:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75bd042640 1 -- 172.21.15.1:0/3646049557 learned_addr learned my addr 172.21.15.1:0/3646049557 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.448+0000 7f75bd843640 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 0x5598a4773310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75be044640 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4770b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75bd042640 1 -- 172.21.15.1:0/3646049557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 msgr2=0x5598a4773310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75bd042640 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 0x5598a4773310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75bd042640 1 -- 172.21.15.1:0/3646049557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 msgr2=0x5598a4770b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75bd042640 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4770b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75bd042640 1 -- 172.21.15.1:0/3646049557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a443b3c0 con 0x5598a4764ca0 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75bd042640 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a4770610 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75a802a4c0 tx=0x7f75a8004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8e9d34fbae5eec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75a17fa640 1 -- 172.21.15.1:0/3646049557 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8035030 con 0x5598a4764ca0 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75a17fa640 1 -- 172.21.15.1:0/3646049557 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f75a80050b0 con 0x5598a4764ca0 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75c1e80740 1 -- 172.21.15.1:0/3646049557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a8005520 con 0x5598a4764ca0 2024-09-16T00:30:13.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75a17fa640 1 -- 172.21.15.1:0/3646049557 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75a8056050 con 0x5598a4764ca0 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75c1e80740 10 client.15600 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.449+0000 7f75c1e80740 1 -- 172.21.15.1:0/3646049557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598a4781f90 con 0x5598a4764ca0 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.450+0000 7f75a17fa640 1 -- 172.21.15.1:0/3646049557 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a8005520 con 0x5598a4764ca0 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.450+0000 7f75a17fa640 1 -- 172.21.15.1:0/3646049557 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f75a8031770 con 0x5598a4764ca0 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.450+0000 7f75c1e80740 10 client.15600 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.450+0000 7f75c1e80740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.450+0000 7f75c1e80740 20 client.15600 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.450+0000 7f75c1e80740 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5598a47832e0 0x5598a47856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.450+0000 7f75c1e80740 4 client.15600 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.450+0000 7f75c1e80740 1 -- 172.21.15.1:0/3646049557 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598a46d7850 con 0x5598a47832e0 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.450+0000 7f75bd843640 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5598a47832e0 0x5598a47856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.451+0000 7f75bd843640 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5598a47832e0 0x5598a47856d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6183cec0a322ec9 server_cookie=10c9b2e310d8317d in_seq=0 out_seq=0 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.451+0000 7f75a17fa640 10 client.15600 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.451+0000 7f75a17fa640 1 -- 172.21.15.1:0/3646049557 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598a46d7850 con 0x5598a47832e0 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.451+0000 7f75a17fa640 10 client.15600 handle_command_reply: tid=0 2024-09-16T00:30:13.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.451+0000 7f75c1e80740 1 client.15600 shutdown 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.451+0000 7f75c1e80740 1 -- 172.21.15.1:0/3646049557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 msgr2=0x5598a4770610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a4770610 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75a802a4c0 tx=0x7f75a8004960 comp rx=0 tx=0).stop 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 1 -- 172.21.15.1:0/3646049557 shutdown_connections 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5598a47832e0 0x5598a47856d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4770b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a4770610 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 1 --2- 172.21.15.1:0/3646049557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 0x5598a4773310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75a17fa640 0 client.15600 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 1 -- 172.21.15.1:0/3646049557 >> 172.21.15.1:0/3646049557 conn(0x5598a4760a30 msgr2=0x5598a4766790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 1 -- 172.21.15.1:0/3646049557 shutdown_connections 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 1 -- 172.21.15.1:0/3646049557 wait complete. 2024-09-16T00:30:13.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.452+0000 7f75c1e80740 20 client.15600 trim_cache size 0 max 16384 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.469+0000 7f75c1e80740 1 Processor -- start 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.469+0000 7f75c1e80740 1 -- start start 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.469+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764380 0x5598a477f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a476bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4772580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75c1e80740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81af4f0 con 0x5598a4764380 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75c1e80740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81acdc0 con 0x5598a4773a90 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75c1e80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81aac10 con 0x5598a4764ca0 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd843640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764380 0x5598a477f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd843640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764380 0x5598a477f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53492/0 (socket says 172.21.15.1:53492) 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd843640 1 -- 172.21.15.1:0/453549811 learned_addr learned my addr 172.21.15.1:0/453549811 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75be044640 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4772580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd042640 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a476bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd843640 1 -- 172.21.15.1:0/453549811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 msgr2=0x5598a476bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd843640 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a476bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd843640 1 -- 172.21.15.1:0/453549811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 msgr2=0x5598a4772580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd843640 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4772580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd843640 1 -- 172.21.15.1:0/453549811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a47767c0 con 0x5598a4764380 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd042640 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a476bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.470+0000 7f75bd843640 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764380 0x5598a477f930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f75b40985c0 tx=0x7f75b4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b460bec0a5966b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.471+0000 7f75a37fe640 1 -- 172.21.15.1:0/453549811 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4020040 con 0x5598a4764380 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.471+0000 7f75a37fe640 1 -- 172.21.15.1:0/453549811 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b40a4860 con 0x5598a4764380 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.471+0000 7f75a37fe640 1 -- 172.21.15.1:0/453549811 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b409b2a0 con 0x5598a4764380 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.471+0000 7f75c1e80740 1 -- 172.21.15.1:0/453549811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764380 msgr2=0x5598a477f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.471+0000 7f75c1e80740 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764380 0x5598a477f930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f75b40985c0 tx=0x7f75b4010d70 comp rx=0 tx=0).stop 2024-09-16T00:30:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.472+0000 7f75c1e80740 1 -- 172.21.15.1:0/453549811 shutdown_connections 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.472+0000 7f75c1e80740 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4772580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.472+0000 7f75c1e80740 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764ca0 0x5598a476bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.472+0000 7f75c1e80740 1 --2- 172.21.15.1:0/453549811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764380 0x5598a477f930 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.472+0000 7f75c1e80740 1 -- 172.21.15.1:0/453549811 >> 172.21.15.1:0/453549811 conn(0x5598a4760a30 msgr2=0x5598a4771e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.472+0000 7f75c1e80740 1 -- 172.21.15.1:0/453549811 shutdown_connections 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.472+0000 7f75c1e80740 1 -- 172.21.15.1:0/453549811 wait complete. 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.472+0000 7f75c1e80740 1 -- start start 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.472+0000 7f75c1e80740 1 Processor -- start 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 0x5598a477baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a477bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75c1e80740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4778a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75c1e80740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ab280 con 0x5598a4764ca0 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75c1e80740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ac6f0 con 0x5598a4773a90 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75c1e80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ab950 con 0x5598a4764380 2024-09-16T00:30:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75c1e80740 10 client.0 _ll_register_callbacks cb 0x7f75b0008450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd042640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a477bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd042640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a477bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53500/0 (socket says 172.21.15.1:53500) 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd042640 1 -- 172.21.15.1:0/1204611157 learned_addr learned my addr 172.21.15.1:0/1204611157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75be044640 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4778a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd843640 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 0x5598a477baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd042640 1 -- 172.21.15.1:0/1204611157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 msgr2=0x5598a477baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd042640 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 0x5598a477baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd042640 1 -- 172.21.15.1:0/1204611157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 msgr2=0x5598a4778a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd042640 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4778a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd042640 1 -- 172.21.15.1:0/1204611157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a477c650 con 0x5598a4764ca0 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75be044640 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4778a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75bd042640 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a477bfe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5598a4771bc0 tx=0x7f75a80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55eca9e2f159a35a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.473+0000 7f75a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75a17fa640 1 -- 172.21.15.1:0/1204611157 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a805f060 con 0x5598a4764ca0 2024-09-16T00:30:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75a17fa640 1 -- 172.21.15.1:0/1204611157 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f75a8053070 con 0x5598a4764ca0 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75c1e80740 1 -- 172.21.15.1:0/1204611157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75b40a4860 con 0x5598a4764ca0 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75a17fa640 1 -- 172.21.15.1:0/1204611157 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75a8066050 con 0x5598a4764ca0 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75c1e80740 10 client.15606 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75c1e80740 1 -- 172.21.15.1:0/1204611157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598a47ee260 con 0x5598a4764ca0 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75a17fa640 1 -- 172.21.15.1:0/1204611157 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b40a4860 con 0x5598a4764ca0 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75a17fa640 1 -- 172.21.15.1:0/1204611157 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f75a8031ce0 con 0x5598a4764ca0 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75c1e80740 10 client.15606 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75c1e80740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75c1e80740 20 client.15606 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75c1e80740 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5598a47eea10 0x5598a47f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75c1e80740 4 client.15606 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.474+0000 7f75c1e80740 1 -- 172.21.15.1:0/1204611157 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5598a46d1d60 con 0x5598a47eea10 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.475+0000 7f75bd843640 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5598a47eea10 0x5598a47f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.475+0000 7f75a17fa640 10 client.15606 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.475+0000 7f75bd843640 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5598a47eea10 0x5598a47f0e30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c223113c100e14c server_cookie=3c795a099427e639 in_seq=0 out_seq=0 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.475+0000 7f75a17fa640 1 -- 172.21.15.1:0/1204611157 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5598a46d1d60 con 0x5598a47eea10 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.475+0000 7f75a17fa640 10 client.15606 handle_command_reply: tid=0 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.475+0000 7f75c1e80740 1 client.15606 shutdown 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.475+0000 7f75c1e80740 1 -- 172.21.15.1:0/1204611157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 msgr2=0x5598a477bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.475+0000 7f75c1e80740 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a477bfe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5598a4771bc0 tx=0x7f75a80049d0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75c1e80740 1 -- 172.21.15.1:0/1204611157 shutdown_connections 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75c1e80740 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5598a47eea10 0x5598a47f0e30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75c1e80740 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598a4773a90 0x5598a4778a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75c1e80740 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598a4764ca0 0x5598a477bfe0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75c1e80740 1 --2- 172.21.15.1:0/1204611157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5598a4764380 0x5598a477baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75c1e80740 1 -- 172.21.15.1:0/1204611157 >> 172.21.15.1:0/1204611157 conn(0x5598a4760a30 msgr2=0x5598a4772690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75a17fa640 0 client.15606 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75c1e80740 1 -- 172.21.15.1:0/1204611157 shutdown_connections 2024-09-16T00:30:13.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75c1e80740 1 -- 172.21.15.1:0/1204611157 wait complete. 2024-09-16T00:30:13.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75c1e80740 20 client.15606 trim_cache size 0 max 16384 2024-09-16T00:30:13.483 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:30:13.483 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:30:13.483 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:30:13.483 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:30:13.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 -- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f75a40a3240 msgr2=0x7f75a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f75a40a3240 0x7f75a40a5700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005fd0 tx=0x7f75b00085b0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 -- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8100e20 msgr2=0x7f75b8199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8100e20 0x7f75b8199310 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f75b400d950 tx=0x7f75b400de10 comp rx=0 tx=0).stop 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 -- 172.21.15.1:0/1871758241 shutdown_connections 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b8102ef0 0x7f75b81a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f75a40a3240 0x7f75a40a5700 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75b8102570 0x7f75b8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 --2- 172.21.15.1:0/1871758241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75b8100e20 0x7f75b8199310 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.476+0000 7f75bfaa7640 1 -- 172.21.15.1:0/1871758241 >> 172.21.15.1:0/1871758241 conn(0x7f75b80fc7d0 msgr2=0x7f75b8106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.477+0000 7f75bfaa7640 1 -- 172.21.15.1:0/1871758241 shutdown_connections 2024-09-16T00:30:13.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:13.477+0000 7f75bfaa7640 1 -- 172.21.15.1:0/1871758241 wait complete. 2024-09-16T00:30:14.186 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:30:14.186 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:14 smithi019 ceph-mon[35186]: pgmap v405: 129 pgs: 129 active+clean; 735 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-16T00:30:14.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:14 smithi001 ceph-mon[30934]: pgmap v405: 129 pgs: 129 active+clean; 735 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-16T00:30:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:14.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:14 smithi153 ceph-mon[35538]: pgmap v405: 129 pgs: 129 active+clean; 735 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-16T00:30:14.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.929+0000 7fcb2069d640 1 -- 172.21.15.1:0/582508787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 msgr2=0x7fcb18100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.929+0000 7fcb2069d640 1 --2- 172.21.15.1:0/582508787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 0x7fcb18100ee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009930 tx=0x7fcb0802a420 comp rx=0 tx=0).stop 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.929+0000 7fcb2069d640 1 -- 172.21.15.1:0/582508787 shutdown_connections 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.929+0000 7fcb2069d640 1 --2- 172.21.15.1:0/582508787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb18101420 0x7fcb1810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.929+0000 7fcb2069d640 1 --2- 172.21.15.1:0/582508787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 0x7fcb18100ee0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.929+0000 7fcb2069d640 1 --2- 172.21.15.1:0/582508787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb18106a80 0x7fcb18106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.929+0000 7fcb2069d640 1 -- 172.21.15.1:0/582508787 >> 172.21.15.1:0/582508787 conn(0x7fcb180fc7a0 msgr2=0x7fcb180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.929+0000 7fcb2069d640 1 -- 172.21.15.1:0/582508787 shutdown_connections 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.929+0000 7fcb2069d640 1 -- 172.21.15.1:0/582508787 wait complete. 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb2069d640 1 Processor -- start 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb2069d640 1 -- start start 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb2069d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 0x7fcb180fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb2069d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb18101420 0x7fcb180fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb2069d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb18106a80 0x7fcb180fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb2069d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18110e40 con 0x7fcb18100a80 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb2069d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18110cc0 con 0x7fcb18106a80 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb2069d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18110fc0 con 0x7fcb18101420 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb1e439640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 0x7fcb180fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb1e439640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 0x7fcb180fcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53514/0 (socket says 172.21.15.1:53514) 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb1e439640 1 -- 172.21.15.1:0/1846199845 learned_addr learned my addr 172.21.15.1:0/1846199845 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.930+0000 7fcb1ec3a640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb18106a80 0x7fcb180fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb1dc38640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb18101420 0x7fcb180fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb1e439640 1 -- 172.21.15.1:0/1846199845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb18101420 msgr2=0x7fcb180fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb1e439640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb18101420 0x7fcb180fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb1e439640 1 -- 172.21.15.1:0/1846199845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb18106a80 msgr2=0x7fcb180fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb1e439640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb18106a80 0x7fcb180fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb1e439640 1 -- 172.21.15.1:0/1846199845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb181aadc0 con 0x7fcb18100a80 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb1ec3a640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb18106a80 0x7fcb180fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb1e439640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 0x7fcb180fcdc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400c920 tx=0x7fcb1400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb0f7fe640 1 -- 172.21.15.1:0/1846199845 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14007c70 con 0x7fcb18100a80 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb0f7fe640 1 -- 172.21.15.1:0/1846199845 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14007e10 con 0x7fcb18100a80 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb181aaff0 con 0x7fcb18100a80 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb0f7fe640 1 -- 172.21.15.1:0/1846199845 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14018890 con 0x7fcb18100a80 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.931+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb181ab3f0 con 0x7fcb18100a80 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.932+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb181021c0 con 0x7fcb18100a80 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.933+0000 7fcb0f7fe640 1 -- 172.21.15.1:0/1846199845 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fcb14018a70 con 0x7fcb18100a80 2024-09-16T00:30:14.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.934+0000 7fcb0f7fe640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb040a3240 0x7fcb040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.934+0000 7fcb1dc38640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb040a3240 0x7fcb040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.934+0000 7fcb0f7fe640 1 -- 172.21.15.1:0/1846199845 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fcb140cff00 con 0x7fcb18100a80 2024-09-16T00:30:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.935+0000 7fcb1dc38640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb040a3240 0x7fcb040a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcb08006010 tx=0x7fcb08005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:14.939+0000 7fcb0f7fe640 1 -- 172.21.15.1:0/1846199845 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb14014030 con 0x7fcb18100a80 2024-09-16T00:30:15.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.152+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb18109a90 con 0x7fcb18100a80 2024-09-16T00:30:15.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.153+0000 7fcb0f7fe640 1 -- 172.21.15.1:0/1846199845 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fcb1400de20 con 0x7fcb18100a80 2024-09-16T00:30:15.153 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:30:15.153 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:30:15.154 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb040a3240 msgr2=0x7fcb040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb040a3240 0x7fcb040a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcb08006010 tx=0x7fcb08005a20 comp rx=0 tx=0).stop 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 msgr2=0x7fcb180fcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 0x7fcb180fcdc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400c920 tx=0x7fcb1400cde0 comp rx=0 tx=0).stop 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 shutdown_connections 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb18106a80 0x7fcb180fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb040a3240 0x7fcb040a5700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb18101420 0x7fcb180fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 --2- 172.21.15.1:0/1846199845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb18100a80 0x7fcb180fcdc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 >> 172.21.15.1:0/1846199845 conn(0x7fcb180fc7a0 msgr2=0x7fcb1806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 shutdown_connections 2024-09-16T00:30:15.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:15.155+0000 7fcb2069d640 1 -- 172.21.15.1:0/1846199845 wait complete. 2024-09-16T00:30:15.877 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:30:16.140 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:16.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:16 smithi001 ceph-mon[30934]: pgmap v406: 129 pgs: 129 active+clean; 735 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-09-16T00:30:16.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:16 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1846199845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:16 smithi019 ceph-mon[35186]: pgmap v406: 129 pgs: 129 active+clean; 735 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-09-16T00:30:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:16 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1846199845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:16.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:16 smithi153 ceph-mon[35538]: pgmap v406: 129 pgs: 129 active+clean; 735 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-09-16T00:30:16.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:16 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1846199845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:16.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.657+0000 7f37e94bf640 1 -- 172.21.15.1:0/2528694940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e410e480 msgr2=0x7f37e4110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.657+0000 7f37e94bf640 1 --2- 172.21.15.1:0/2528694940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e410e480 0x7f37e4110870 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009ca0 tx=0x7f37dc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.658+0000 7f37e94bf640 1 -- 172.21.15.1:0/2528694940 shutdown_connections 2024-09-16T00:30:16.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.658+0000 7f37e94bf640 1 --2- 172.21.15.1:0/2528694940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e410e480 0x7f37e4110870 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.658+0000 7f37e94bf640 1 --2- 172.21.15.1:0/2528694940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e4101280 0x7f37e410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.658+0000 7f37e94bf640 1 --2- 172.21.15.1:0/2528694940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e41008d0 0x7f37e4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.658+0000 7f37e94bf640 1 -- 172.21.15.1:0/2528694940 >> 172.21.15.1:0/2528694940 conn(0x7f37e40fc7a0 msgr2=0x7f37e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.658+0000 7f37e94bf640 1 -- 172.21.15.1:0/2528694940 shutdown_connections 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.658+0000 7f37e94bf640 1 -- 172.21.15.1:0/2528694940 wait complete. 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.658+0000 7f37e94bf640 1 Processor -- start 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.658+0000 7f37e94bf640 1 -- start start 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e94bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e41008d0 0x7f37e419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e94bf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e4101280 0x7f37e419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e94bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e410e480 0x7f37e419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e94bf640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41141d0 con 0x7f37e41008d0 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e94bf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37e4114050 con 0x7f37e410e480 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e94bf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37e4114350 con 0x7f37e4101280 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e41008d0 0x7f37e419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e41008d0 0x7f37e419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53532/0 (socket says 172.21.15.1:53532) 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e2ffd640 1 -- 172.21.15.1:0/1298871663 learned_addr learned my addr 172.21.15.1:0/1298871663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:16.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e27fc640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e4101280 0x7f37e419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e37fe640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e410e480 0x7f37e419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e2ffd640 1 -- 172.21.15.1:0/1298871663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e4101280 msgr2=0x7f37e419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e4101280 0x7f37e419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e2ffd640 1 -- 172.21.15.1:0/1298871663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e410e480 msgr2=0x7f37e419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e410e480 0x7f37e419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e2ffd640 1 -- 172.21.15.1:0/1298871663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e419f5c0 con 0x7f37e41008d0 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e37fe640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e410e480 0x7f37e419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e41008d0 0x7f37e419a610 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f37d400e9f0 tx=0x7f37d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37cbfff640 1 -- 172.21.15.1:0/1298871663 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d400cd80 con 0x7f37e41008d0 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37e94bf640 1 -- 172.21.15.1:0/1298871663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e419f8b0 con 0x7f37e41008d0 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.659+0000 7f37cbfff640 1 -- 172.21.15.1:0/1298871663 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d4010910 con 0x7f37e41008d0 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.660+0000 7f37cbfff640 1 -- 172.21.15.1:0/1298871663 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d40188f0 con 0x7f37e41008d0 2024-09-16T00:30:16.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.660+0000 7f37e94bf640 1 -- 172.21.15.1:0/1298871663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e41a7190 con 0x7f37e41008d0 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.662+0000 7f37cbfff640 1 -- 172.21.15.1:0/1298871663 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f37d4010ac0 con 0x7f37e41008d0 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.662+0000 7f37eb898740 1 Processor -- start 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.662+0000 7f37eb898740 1 -- start start 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37eb898740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37eb898740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 0x56526f5a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37cbfff640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37cc0a3240 0x7f37cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37eb898740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 0x56526f5b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37eb898740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56526f50d2f0 con 0x56526f5a8b40 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37eb898740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56526f4b4cf0 con 0x56526f5a75d0 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37eb898740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56526f5092b0 con 0x56526f5b2370 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53544/0 (socket says 172.21.15.1:53544) 2024-09-16T00:30:16.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 -- 172.21.15.1:0/2211223983 learned_addr learned my addr 172.21.15.1:0/2211223983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e27fc640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37cc0a3240 0x7f37cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37cbfff640 1 -- 172.21.15.1:0/1298871663 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f37d40d01b0 con 0x7f37e41008d0 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/2211223983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 0x56526f5a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 --2- 172.21.15.1:0/2211223983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 0x56526f5b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 -- 172.21.15.1:0/2211223983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 msgr2=0x56526f5b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 --2- 172.21.15.1:0/2211223983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 0x56526f5b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 -- 172.21.15.1:0/2211223983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 msgr2=0x56526f5a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 --2- 172.21.15.1:0/2211223983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 0x56526f5a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 -- 172.21.15.1:0/2211223983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526f4993a0 con 0x56526f5a8b40 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e27fc640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37cc0a3240 0x7f37cc0a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f37e4102280 tx=0x7f37d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.663+0000 7f37e37fe640 1 --2- 172.21.15.1:0/2211223983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5a7090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f37dc02f540 tx=0x7f37dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79987560aa245603 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37c97fa640 1 -- 172.21.15.1:0/2211223983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc003b70 con 0x56526f5a8b40 2024-09-16T00:30:16.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37c97fa640 1 -- 172.21.15.1:0/2211223983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc046070 con 0x56526f5a8b40 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37c97fa640 1 -- 172.21.15.1:0/2211223983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0043f0 con 0x56526f5a8b40 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37eb898740 1 -- 172.21.15.1:0/2211223983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 msgr2=0x56526f5a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37eb898740 1 --2- 172.21.15.1:0/2211223983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5a7090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f37dc02f540 tx=0x7f37dc004ab0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37eb898740 1 -- 172.21.15.1:0/2211223983 shutdown_connections 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37eb898740 1 --2- 172.21.15.1:0/2211223983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 0x56526f5b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37eb898740 1 --2- 172.21.15.1:0/2211223983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 0x56526f5a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37eb898740 1 --2- 172.21.15.1:0/2211223983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5a7090 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.664+0000 7f37eb898740 1 -- 172.21.15.1:0/2211223983 >> 172.21.15.1:0/2211223983 conn(0x56526f570310 msgr2=0x56526f5a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 -- 172.21.15.1:0/2211223983 shutdown_connections 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 -- 172.21.15.1:0/2211223983 wait complete. 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 -- start start 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 Processor -- start 2024-09-16T00:30:16.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5a75d0 0x56526f5b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5af570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56526f5b5010 con 0x56526f5b2370 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56526f4c8060 con 0x56526f5a8b40 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56526f5b5190 con 0x56526f5a75d0 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37eb898740 10 client.0 _ll_register_callbacks cb 0x56526f21b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37e27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53556/0 (socket says 172.21.15.1:53556) 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.665+0000 7f37e37fe640 1 -- 172.21.15.1:0/3931747114 learned_addr learned my addr 172.21.15.1:0/3931747114 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5a75d0 0x56526f5b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37e37fe640 1 -- 172.21.15.1:0/3931747114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5a75d0 msgr2=0x56526f5b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37e37fe640 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5a75d0 0x56526f5b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37e37fe640 1 -- 172.21.15.1:0/3931747114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 msgr2=0x56526f5aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37e37fe640 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37e37fe640 1 -- 172.21.15.1:0/3931747114 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526f27e3c0 con 0x56526f5b2370 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37e27fc640 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37e37fe640 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5af570 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f37dc004a80 tx=0x7f37dc031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90074d605eb71ead server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37c6ff5640 1 -- 172.21.15.1:0/3931747114 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc04e070 con 0x56526f5b2370 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37c6ff5640 1 -- 172.21.15.1:0/3931747114 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f37dc0043d0 con 0x56526f5b2370 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37eb898740 1 -- 172.21.15.1:0/3931747114 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37dc046070 con 0x56526f5b2370 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37c6ff5640 1 -- 172.21.15.1:0/3931747114 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37dc0045d0 con 0x56526f5b2370 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37eb898740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.666+0000 7f37eb898740 1 -- 172.21.15.1:0/3931747114 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56526f5b5010 con 0x56526f5b2370 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.667+0000 7f37c6ff5640 1 -- 172.21.15.1:0/3931747114 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc046070 con 0x56526f5b2370 2024-09-16T00:30:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.667+0000 7f37c6ff5640 1 -- 172.21.15.1:0/3931747114 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f37dc02fbf0 con 0x56526f5b2370 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.667+0000 7f37eb898740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.667+0000 7f37eb898740 10 client.15624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.667+0000 7f37eb898740 20 client.15624 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.667+0000 7f37eb898740 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56526f5c1be0 0x56526f5c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.667+0000 7f37eb898740 4 client.15624 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.667+0000 7f37eb898740 1 -- 172.21.15.1:0/3931747114 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56526f51a860 con 0x56526f5c1be0 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.667+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56526f5c1be0 0x56526f5c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56526f5c1be0 0x56526f5c3fd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4e34e1b9ab073c7 server_cookie=6119b27237bf96cc in_seq=0 out_seq=0 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37c6ff5640 10 client.15624 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37c6ff5640 1 -- 172.21.15.1:0/3931747114 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56526f51a860 con 0x56526f5c1be0 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37c6ff5640 10 client.15624 handle_command_reply: tid=0 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37eb898740 1 client.15624 shutdown 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37eb898740 1 -- 172.21.15.1:0/3931747114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 msgr2=0x56526f5af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37eb898740 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5af570 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f37dc004a80 tx=0x7f37dc031680 comp rx=0 tx=0).stop 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37eb898740 1 -- 172.21.15.1:0/3931747114 shutdown_connections 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37eb898740 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5af570 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37eb898740 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56526f5c1be0 0x56526f5c3fd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37eb898740 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37eb898740 1 --2- 172.21.15.1:0/3931747114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5a75d0 0x56526f5b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37eb898740 1 -- 172.21.15.1:0/3931747114 >> 172.21.15.1:0/3931747114 conn(0x56526f570310 msgr2=0x56526f5a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.668+0000 7f37c6ff5640 0 client.15624 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.669+0000 7f37eb898740 1 -- 172.21.15.1:0/3931747114 shutdown_connections 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.669+0000 7f37eb898740 1 -- 172.21.15.1:0/3931747114 wait complete. 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.669+0000 7f37eb898740 20 client.15624 trim_cache size 0 max 16384 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.681+0000 7f37eb898740 1 Processor -- start 2024-09-16T00:30:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.681+0000 7f37eb898740 1 -- start start 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37eb898740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5a75d0 0x56526f5be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37eb898740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37eb898740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37eb898740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41b3830 con 0x56526f5b2370 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37eb898740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41b1100 con 0x56526f5a8b40 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37eb898740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41aef50 con 0x56526f5a75d0 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5aa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53568/0 (socket says 172.21.15.1:53568) 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e37fe640 1 -- 172.21.15.1:0/1232923592 learned_addr learned my addr 172.21.15.1:0/1232923592 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e27fc640 1 --2- 172.21.15.1:0/1232923592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e37fe640 1 -- 172.21.15.1:0/1232923592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5a75d0 msgr2=0x56526f5be2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e37fe640 1 --2- 172.21.15.1:0/1232923592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5a75d0 0x56526f5be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e37fe640 1 -- 172.21.15.1:0/1232923592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 msgr2=0x56526f5c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e37fe640 1 --2- 172.21.15.1:0/1232923592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.709 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:30:16.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e37fe640 1 -- 172.21.15.1:0/1232923592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526f5a34f0 con 0x56526f5b2370 2024-09-16T00:30:16.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e27fc640 1 --2- 172.21.15.1:0/1232923592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:16.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37e37fe640 1 --2- 172.21.15.1:0/1232923592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5aa120 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f37dc02a900 tx=0x7f37dc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7ced9273af03bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37c8ff9640 1 -- 172.21.15.1:0/1232923592 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc04e070 con 0x56526f5b2370 2024-09-16T00:30:16.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37c8ff9640 1 -- 172.21.15.1:0/1232923592 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc046040 con 0x56526f5b2370 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.682+0000 7f37c8ff9640 1 -- 172.21.15.1:0/1232923592 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc003c80 con 0x56526f5b2370 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 -- 172.21.15.1:0/1232923592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 msgr2=0x56526f5aa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 --2- 172.21.15.1:0/1232923592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5aa120 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f37dc02a900 tx=0x7f37dc009b60 comp rx=0 tx=0).stop 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 -- 172.21.15.1:0/1232923592 shutdown_connections 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 --2- 172.21.15.1:0/1232923592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5b2370 0x56526f5aa120 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 --2- 172.21.15.1:0/1232923592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a8b40 0x56526f5c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 --2- 172.21.15.1:0/1232923592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5a75d0 0x56526f5be2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 -- 172.21.15.1:0/1232923592 >> 172.21.15.1:0/1232923592 conn(0x56526f570310 msgr2=0x56526f5b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 -- 172.21.15.1:0/1232923592 shutdown_connections 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 -- 172.21.15.1:0/1232923592 wait complete. 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 -- start start 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.683+0000 7f37eb898740 1 Processor -- start 2024-09-16T00:30:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37eb898740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 0x56526f5ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37eb898740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37eb898740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 0x56526f5abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37eb898740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41af5c0 con 0x56526f5a8b40 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37eb898740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41b0a30 con 0x56526f5a75d0 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37eb898740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41afc90 con 0x56526f5b2370 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37eb898740 10 client.0 _ll_register_callbacks cb 0x7f37d4097ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53584/0 (socket says 172.21.15.1:53584) 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e27fc640 1 -- 172.21.15.1:0/2999053657 learned_addr learned my addr 172.21.15.1:0/2999053657 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e37fe640 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 0x56526f5abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 0x56526f5ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e27fc640 1 -- 172.21.15.1:0/2999053657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 msgr2=0x56526f5abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e27fc640 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 0x56526f5abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e27fc640 1 -- 172.21.15.1:0/2999053657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 msgr2=0x56526f5ba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e27fc640 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 0x56526f5ba8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e27fc640 1 -- 172.21.15.1:0/2999053657 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526f5ac4c0 con 0x56526f5a8b40 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37e27fc640 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5aba40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f37d8004650 tx=0x7f37d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc5a1ccf2ef189c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.684+0000 7f37c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37c6ff5640 1 -- 172.21.15.1:0/2999053657 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8005220 con 0x56526f5a8b40 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37c6ff5640 1 -- 172.21.15.1:0/2999053657 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f37d80053c0 con 0x56526f5a8b40 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37eb898740 1 -- 172.21.15.1:0/2999053657 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37dc046040 con 0x56526f5a8b40 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37c6ff5640 1 -- 172.21.15.1:0/2999053657 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37d8033050 con 0x56526f5a8b40 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37eb898740 10 client.15630 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37eb898740 1 -- 172.21.15.1:0/2999053657 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56526f62c430 con 0x56526f5a8b40 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37c6ff5640 1 -- 172.21.15.1:0/2999053657 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc046040 con 0x56526f5a8b40 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37c6ff5640 1 -- 172.21.15.1:0/2999053657 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f37d8027030 con 0x56526f5a8b40 2024-09-16T00:30:16.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37eb898740 10 client.15630 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37eb898740 10 client.15630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37eb898740 20 client.15630 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37eb898740 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56526f62cbd0 0x56526f62eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37eb898740 4 client.15630 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37eb898740 1 -- 172.21.15.1:0/2999053657 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56526f514d70 con 0x56526f62cbd0 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.685+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56526f62cbd0 0x56526f62eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37e2ffd640 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56526f62cbd0 0x56526f62eff0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4881fa16788d1efa server_cookie=bba5c47d1179d273 in_seq=0 out_seq=0 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37c6ff5640 10 client.15630 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37c6ff5640 1 -- 172.21.15.1:0/2999053657 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56526f514d70 con 0x56526f62cbd0 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37c6ff5640 10 client.15630 handle_command_reply: tid=0 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 client.15630 shutdown 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 -- 172.21.15.1:0/2999053657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 msgr2=0x56526f5aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5aba40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f37d8004650 tx=0x7f37d8004730 comp rx=0 tx=0).stop 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 -- 172.21.15.1:0/2999053657 shutdown_connections 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56526f5b2370 0x56526f5abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526f5a8b40 0x56526f5aba40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56526f62cbd0 0x56526f62eff0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 --2- 172.21.15.1:0/2999053657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526f5a75d0 0x56526f5ba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37c6ff5640 0 client.15630 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 -- 172.21.15.1:0/2999053657 >> 172.21.15.1:0/2999053657 conn(0x56526f570310 msgr2=0x56526f5a24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 -- 172.21.15.1:0/2999053657 shutdown_connections 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 1 -- 172.21.15.1:0/2999053657 wait complete. 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.686+0000 7f37eb898740 20 client.15630 trim_cache size 0 max 16384 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 -- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37cc0a3240 msgr2=0x7f37cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37cc0a3240 0x7f37cc0a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f37e4102280 tx=0x7f37d800b390 comp rx=0 tx=0).stop 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 -- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e41008d0 msgr2=0x7f37e419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e41008d0 0x7f37e419a610 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f37d400e9f0 tx=0x7f37d400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 -- 172.21.15.1:0/1298871663 shutdown_connections 2024-09-16T00:30:16.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e410e480 0x7f37e419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37cc0a3240 0x7f37cc0a5700 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e4101280 0x7f37e419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 --2- 172.21.15.1:0/1298871663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37e41008d0 0x7f37e419a610 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 -- 172.21.15.1:0/1298871663 >> 172.21.15.1:0/1298871663 conn(0x7f37e40fc7a0 msgr2=0x7f37e40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 -- 172.21.15.1:0/1298871663 shutdown_connections 2024-09-16T00:30:16.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:16.687+0000 7f37e94bf640 1 -- 172.21.15.1:0/1298871663 wait complete. 2024-09-16T00:30:17.427 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:30:17.686 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:18.124 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:18 smithi001 ceph-mon[30934]: pgmap v407: 129 pgs: 129 active+clean; 735 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-16T00:30:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.207+0000 7fb491fe2640 1 -- 172.21.15.1:0/839937616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 msgr2=0x7fb48c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.207+0000 7fb491fe2640 1 --2- 172.21.15.1:0/839937616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 0x7fb48c10d950 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb484009cd0 tx=0x7fb48402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.208+0000 7fb491fe2640 1 -- 172.21.15.1:0/839937616 shutdown_connections 2024-09-16T00:30:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.208+0000 7fb491fe2640 1 --2- 172.21.15.1:0/839937616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 0x7fb48c10d950 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.208+0000 7fb491fe2640 1 --2- 172.21.15.1:0/839937616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c100a80 0x7fb48c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.208+0000 7fb491fe2640 1 --2- 172.21.15.1:0/839937616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48c106a80 0x7fb48c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.208+0000 7fb491fe2640 1 -- 172.21.15.1:0/839937616 >> 172.21.15.1:0/839937616 conn(0x7fb48c0fc7a0 msgr2=0x7fb48c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.208+0000 7fb491fe2640 1 -- 172.21.15.1:0/839937616 shutdown_connections 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.208+0000 7fb491fe2640 1 -- 172.21.15.1:0/839937616 wait complete. 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.208+0000 7fb491fe2640 1 Processor -- start 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb491fe2640 1 -- start start 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb491fe2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48c100a80 0x7fb48c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb491fe2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 0x7fb48c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb491fe2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c106a80 0x7fb48c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb491fe2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c110ea0 con 0x7fb48c101420 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb491fe2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c110d20 con 0x7fb48c106a80 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb491fe2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c111020 con 0x7fb48c100a80 2024-09-16T00:30:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 0x7fb48c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 0x7fb48c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53604/0 (socket says 172.21.15.1:53604) 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48affd640 1 -- 172.21.15.1:0/1443475890 learned_addr learned my addr 172.21.15.1:0/1443475890 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48bfff640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c106a80 0x7fb48c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48c100a80 0x7fb48c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48affd640 1 -- 172.21.15.1:0/1443475890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48c100a80 msgr2=0x7fb48c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48affd640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48c100a80 0x7fb48c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48affd640 1 -- 172.21.15.1:0/1443475890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c106a80 msgr2=0x7fb48c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48affd640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c106a80 0x7fb48c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48affd640 1 -- 172.21.15.1:0/1443475890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c19d3a0 con 0x7fb48c101420 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48bfff640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c106a80 0x7fb48c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48c100a80 0x7fb48c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.209+0000 7fb48affd640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 0x7fb48c198930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c920 tx=0x7fb48000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.210+0000 7fb488ff9640 1 -- 172.21.15.1:0/1443475890 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480007c70 con 0x7fb48c101420 2024-09-16T00:30:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.210+0000 7fb488ff9640 1 -- 172.21.15.1:0/1443475890 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb480007e10 con 0x7fb48c101420 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.210+0000 7fb491fe2640 1 -- 172.21.15.1:0/1443475890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c19d690 con 0x7fb48c101420 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.210+0000 7fb488ff9640 1 -- 172.21.15.1:0/1443475890 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480018890 con 0x7fb48c101420 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.210+0000 7fb491fe2640 1 -- 172.21.15.1:0/1443475890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c1a4f70 con 0x7fb48c101420 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.212+0000 7fb488ff9640 1 -- 172.21.15.1:0/1443475890 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fb480018a70 con 0x7fb48c101420 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.212+0000 7fb4943bb740 1 Processor -- start 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.212+0000 7fb4943bb740 1 -- start start 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.212+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.212+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 0x55b86d7a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.212+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.212+0000 7fb4943bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b86d7072f0 con 0x55b86d7b0780 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.212+0000 7fb4943bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b86d6aecf0 con 0x55b86d7a15d0 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.212+0000 7fb4943bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b86d7032b0 con 0x55b86d7a2b40 2024-09-16T00:30:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53616/0 (socket says 172.21.15.1:53616) 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48affd640 1 -- 172.21.15.1:0/3138767841 learned_addr learned my addr 172.21.15.1:0/3138767841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb488ff9640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb4740a3240 0x7fb4740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 0x55b86d7a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48bfff640 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48affd640 1 -- 172.21.15.1:0/3138767841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 msgr2=0x55b86d7a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48bfff640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb4740a3240 0x7fb4740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48affd640 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48affd640 1 -- 172.21.15.1:0/3138767841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 msgr2=0x55b86d7a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48affd640 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 0x55b86d7a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb488ff9640 1 -- 172.21.15.1:0/1443475890 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb4800cff10 con 0x7fb48c101420 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48affd640 1 -- 172.21.15.1:0/3138767841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b86d6933a0 con 0x55b86d7b0780 2024-09-16T00:30:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 0x55b86d7a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb48affd640 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b2b70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb4800cd610 tx=0x7fb48000ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4a31fc430979e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.213+0000 7fb471ffb640 1 -- 172.21.15.1:0/3138767841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48009bc70 con 0x55b86d7b0780 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb471ffb640 1 -- 172.21.15.1:0/3138767841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48000de20 con 0x55b86d7b0780 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb471ffb640 1 -- 172.21.15.1:0/3138767841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48009ba90 con 0x55b86d7b0780 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb48bfff640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb4740a3240 0x7fb4740a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb48402a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb4943bb740 1 -- 172.21.15.1:0/3138767841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 msgr2=0x55b86d7b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b2b70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb4800cd610 tx=0x7fb48000ff30 comp rx=0 tx=0).stop 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb4943bb740 1 -- 172.21.15.1:0/3138767841 shutdown_connections 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b2b70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 0x55b86d7a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3138767841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb4943bb740 1 -- 172.21.15.1:0/3138767841 >> 172.21.15.1:0/3138767841 conn(0x55b86d76a310 msgr2=0x55b86d79dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb4943bb740 1 -- 172.21.15.1:0/3138767841 shutdown_connections 2024-09-16T00:30:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.214+0000 7fb4943bb740 1 -- 172.21.15.1:0/3138767841 wait complete. 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb4943bb740 1 -- start start 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb4943bb740 1 Processor -- start 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d7b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a2b40 0x55b86d7ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7b0780 0x55b86d7ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb4943bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b86d7b3420 con 0x55b86d7a15d0 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb4943bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b86d6c2060 con 0x55b86d7a2b40 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb4943bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b86d7b35a0 con 0x55b86d7b0780 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb4943bb740 10 client.0 _ll_register_callbacks cb 0x55b86d4158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d7b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d7b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53620/0 (socket says 172.21.15.1:53620) 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.215+0000 7fb48b7fe640 1 -- 172.21.15.1:0/3280061924 learned_addr learned my addr 172.21.15.1:0/3280061924 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48bfff640 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7b0780 0x55b86d7ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48b7fe640 1 -- 172.21.15.1:0/3280061924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7b0780 msgr2=0x55b86d7ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7b0780 0x55b86d7ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48b7fe640 1 -- 172.21.15.1:0/3280061924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a2b40 msgr2=0x55b86d7ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48affd640 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a2b40 0x55b86d7ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a2b40 0x55b86d7ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48b7fe640 1 -- 172.21.15.1:0/3280061924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b86d4783c0 con 0x55b86d7a15d0 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48bfff640 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7b0780 0x55b86d7ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48affd640 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a2b40 0x55b86d7ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d7b0180 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00e2d0 tx=0x7fb47c00e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e251f4415f9f9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb46affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb46affd640 1 -- 172.21.15.1:0/3280061924 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c018020 con 0x55b86d7a15d0 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb4943bb740 1 -- 172.21.15.1:0/3280061924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48000de20 con 0x55b86d7a15d0 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb46affd640 1 -- 172.21.15.1:0/3280061924 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb47c013660 con 0x55b86d7a15d0 2024-09-16T00:30:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb46affd640 1 -- 172.21.15.1:0/3280061924 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb47c023050 con 0x55b86d7a15d0 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb4943bb740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb4943bb740 1 -- 172.21.15.1:0/3280061924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b86d7bef20 con 0x55b86d7a15d0 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.216+0000 7fb46affd640 1 -- 172.21.15.1:0/3280061924 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48000de20 con 0x55b86d7a15d0 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb46affd640 1 -- 172.21.15.1:0/3280061924 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb47c014b60 con 0x55b86d7a15d0 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb4943bb740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb4943bb740 10 client.15642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb4943bb740 20 client.15642 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b86d7c0430 0x55b86d7c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb4943bb740 4 client.15642 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb4943bb740 1 -- 172.21.15.1:0/3280061924 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b86d714860 con 0x55b86d7c0430 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb48affd640 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b86d7c0430 0x55b86d7c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb46affd640 10 client.15642 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.217+0000 7fb48affd640 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b86d7c0430 0x55b86d7c28b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e625c4dddac626e server_cookie=a631361eb8edf7fa in_seq=0 out_seq=0 2024-09-16T00:30:18.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb46affd640 1 -- 172.21.15.1:0/3280061924 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b86d714860 con 0x55b86d7c0430 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb46affd640 10 client.15642 handle_command_reply: tid=0 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 client.15642 shutdown 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 -- 172.21.15.1:0/3280061924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 msgr2=0x55b86d7b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d7b0180 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00e2d0 tx=0x7fb47c00e790 comp rx=0 tx=0).stop 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 -- 172.21.15.1:0/3280061924 shutdown_connections 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7b0780 0x55b86d7ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a2b40 0x55b86d7ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b86d7c0430 0x55b86d7c28b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3280061924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d7b0180 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 -- 172.21.15.1:0/3280061924 >> 172.21.15.1:0/3280061924 conn(0x55b86d76a310 msgr2=0x55b86d7a3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb46affd640 0 client.15642 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 -- 172.21.15.1:0/3280061924 shutdown_connections 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.218+0000 7fb4943bb740 1 -- 172.21.15.1:0/3280061924 wait complete. 2024-09-16T00:30:18.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.219+0000 7fb4943bb740 20 client.15642 trim_cache size 0 max 16384 2024-09-16T00:30:18.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.231+0000 7fb4943bb740 1 Processor -- start 2024-09-16T00:30:18.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb4943bb740 1 -- start start 2024-09-16T00:30:18.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d79de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7bc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7b0780 0x55b86d7accf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb4943bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c1af490 con 0x55b86d7a15d0 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb4943bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c1acd60 con 0x55b86d7b0780 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb4943bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c1aabb0 con 0x55b86d7a2b40 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d79de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d79de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53636/0 (socket says 172.21.15.1:53636) 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48b7fe640 1 -- 172.21.15.1:0/3563224293 learned_addr learned my addr 172.21.15.1:0/3563224293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48affd640 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7bc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48bfff640 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7b0780 0x55b86d7accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48b7fe640 1 -- 172.21.15.1:0/3563224293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 msgr2=0x55b86d7bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7bc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48b7fe640 1 -- 172.21.15.1:0/3563224293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7b0780 msgr2=0x55b86d7accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7b0780 0x55b86d7accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48b7fe640 1 -- 172.21.15.1:0/3563224293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b86d7ac3e0 con 0x55b86d7a15d0 2024-09-16T00:30:18.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48bfff640 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7b0780 0x55b86d7accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.232+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d79de10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb47c004700 tx=0x7fb47c012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f9c54000236dba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4717fa640 1 -- 172.21.15.1:0/3563224293 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c02c070 con 0x55b86d7a15d0 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4717fa640 1 -- 172.21.15.1:0/3563224293 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb47c012710 con 0x55b86d7a15d0 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4717fa640 1 -- 172.21.15.1:0/3563224293 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c018070 con 0x55b86d7a15d0 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4943bb740 1 -- 172.21.15.1:0/3563224293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 msgr2=0x55b86d79de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d79de10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb47c004700 tx=0x7fb47c012520 comp rx=0 tx=0).stop 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4943bb740 1 -- 172.21.15.1:0/3563224293 shutdown_connections 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7b0780 0x55b86d7accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7bc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4943bb740 1 --2- 172.21.15.1:0/3563224293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7a15d0 0x55b86d79de10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4943bb740 1 -- 172.21.15.1:0/3563224293 >> 172.21.15.1:0/3563224293 conn(0x55b86d76a310 msgr2=0x55b86d79d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4943bb740 1 -- 172.21.15.1:0/3563224293 shutdown_connections 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.233+0000 7fb4943bb740 1 -- 172.21.15.1:0/3563224293 wait complete. 2024-09-16T00:30:18.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb4943bb740 1 -- start start 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb4943bb740 1 Processor -- start 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 0x55b86d7b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb4943bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb4943bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c1ab220 con 0x55b86d7b0780 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb4943bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c1ac690 con 0x55b86d7a15d0 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb4943bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c1ab8f0 con 0x55b86d7a2b40 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb4943bb740 10 client.0 _ll_register_callbacks cb 0x7fb47c00c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb48bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb48bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53644/0 (socket says 172.21.15.1:53644) 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb48bfff640 1 -- 172.21.15.1:0/1355823780 learned_addr learned my addr 172.21.15.1:0/1355823780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb48affd640 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.234+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 0x55b86d7b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb48bfff640 1 -- 172.21.15.1:0/1355823780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 msgr2=0x55b86d7b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb48bfff640 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb48bfff640 1 -- 172.21.15.1:0/1355823780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 msgr2=0x55b86d7b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb48bfff640 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 0x55b86d7b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb48bfff640 1 -- 172.21.15.1:0/1355823780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b86d7b49e0 con 0x55b86d7b0780 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb48bfff640 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b44a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb484003ba0 tx=0x7fb4840095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755ef6c305bade01 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb46affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb46affd640 1 -- 172.21.15.1:0/1355823780 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4840047c0 con 0x55b86d7b0780 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb46affd640 1 -- 172.21.15.1:0/1355823780 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb4840305a0 con 0x55b86d7b0780 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb4943bb740 1 -- 172.21.15.1:0/1355823780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b86d70ed70 con 0x55b86d7b0780 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb46affd640 1 -- 172.21.15.1:0/1355823780 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb4840307a0 con 0x55b86d7b0780 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb4943bb740 10 client.15648 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb4943bb740 1 -- 172.21.15.1:0/1355823780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b86d82ad70 con 0x55b86d7b0780 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.235+0000 7fb46affd640 1 -- 172.21.15.1:0/1355823780 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b86d70ed70 con 0x55b86d7b0780 2024-09-16T00:30:18.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb46affd640 1 -- 172.21.15.1:0/1355823780 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb48404f070 con 0x55b86d7b0780 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb4943bb740 10 client.15648 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb4943bb740 10 client.15648 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb4943bb740 20 client.15648 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb4943bb740 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b86d82b4c0 0x55b86d82d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb4943bb740 4 client.15648 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb4943bb740 1 -- 172.21.15.1:0/1355823780 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b86d68b170 con 0x55b86d82b4c0 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b86d82b4c0 0x55b86d82d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb46affd640 10 client.15648 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb48b7fe640 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b86d82b4c0 0x55b86d82d8e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5a7430af9aab94a server_cookie=5052f49e2b28e8a9 in_seq=0 out_seq=0 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb46affd640 1 -- 172.21.15.1:0/1355823780 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b86d68b170 con 0x55b86d82b4c0 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb46affd640 10 client.15648 handle_command_reply: tid=0 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.236+0000 7fb4943bb740 1 client.15648 shutdown 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 -- 172.21.15.1:0/1355823780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 msgr2=0x55b86d7b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:18.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b44a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb484003ba0 tx=0x7fb4840095c0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 -- 172.21.15.1:0/1355823780 shutdown_connections 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b86d82b4c0 0x55b86d82d8e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b86d7b0780 0x55b86d7b44a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b86d7a2b40 0x55b86d7b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 --2- 172.21.15.1:0/1355823780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b86d7a15d0 0x55b86d7b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 -- 172.21.15.1:0/1355823780 >> 172.21.15.1:0/1355823780 conn(0x55b86d76a310 msgr2=0x55b86d7a3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb46affd640 0 client.15648 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 -- 172.21.15.1:0/1355823780 shutdown_connections 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 1 -- 172.21.15.1:0/1355823780 wait complete. 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.237+0000 7fb4943bb740 20 client.15648 trim_cache size 0 max 16384 2024-09-16T00:30:18.244 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:30:18.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 -- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb4740a3240 msgr2=0x7fb4740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:18.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb4740a3240 0x7fb4740a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb48402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 -- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 msgr2=0x7fb48c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:18.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 0x7fb48c198930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c920 tx=0x7fb48000cde0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 -- 172.21.15.1:0/1443475890 shutdown_connections 2024-09-16T00:30:18.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c106a80 0x7fb48c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb48c101420 0x7fb48c198930 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb4740a3240 0x7fb4740a5700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 --2- 172.21.15.1:0/1443475890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48c100a80 0x7fb48c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:18.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 -- 172.21.15.1:0/1443475890 >> 172.21.15.1:0/1443475890 conn(0x7fb48c0fc7a0 msgr2=0x7fb48c06a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:18.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 -- 172.21.15.1:0/1443475890 shutdown_connections 2024-09-16T00:30:18.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:18.238+0000 7fb491fe2640 1 -- 172.21.15.1:0/1443475890 wait complete. 2024-09-16T00:30:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:18 smithi019 ceph-mon[35186]: pgmap v407: 129 pgs: 129 active+clean; 735 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-16T00:30:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:18 smithi153 ceph-mon[35538]: pgmap v407: 129 pgs: 129 active+clean; 735 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-16T00:30:18.977 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:30:19.237 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.744+0000 7f940cedc640 1 -- 172.21.15.1:0/2219545939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408101de0 msgr2=0x7f94081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.744+0000 7f940cedc640 1 --2- 172.21.15.1:0/2219545939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408101de0 0x7f94081021c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009900 tx=0x7f93f802a410 comp rx=0 tx=0).stop 2024-09-16T00:30:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.744+0000 7f940cedc640 1 -- 172.21.15.1:0/2219545939 shutdown_connections 2024-09-16T00:30:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.744+0000 7f940cedc640 1 --2- 172.21.15.1:0/2219545939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9408106f10 0x7f940810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.744+0000 7f940cedc640 1 --2- 172.21.15.1:0/2219545939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9408102790 0x7f94081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.744+0000 7f940cedc640 1 --2- 172.21.15.1:0/2219545939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408101de0 0x7f94081021c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.744+0000 7f940cedc640 1 -- 172.21.15.1:0/2219545939 >> 172.21.15.1:0/2219545939 conn(0x7f94080fd8c0 msgr2=0x7f94080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.744+0000 7f940cedc640 1 -- 172.21.15.1:0/2219545939 shutdown_connections 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.744+0000 7f940cedc640 1 -- 172.21.15.1:0/2219545939 wait complete. 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f940cedc640 1 Processor -- start 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f940cedc640 1 -- start start 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f940cedc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9408101de0 0x7f94080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f940cedc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9408102790 0x7f9408074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f940cedc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408106f10 0x7f9408074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f940cedc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9408111f30 con 0x7f9408106f10 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f940cedc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9408111db0 con 0x7f9408102790 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f940cedc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94081120b0 con 0x7f9408101de0 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f9406d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408106f10 0x7f9408074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f9406d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408106f10 0x7f9408074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53652/0 (socket says 172.21.15.1:53652) 2024-09-16T00:30:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.745+0000 7f9406d9d640 1 -- 172.21.15.1:0/162210848 learned_addr learned my addr 172.21.15.1:0/162210848 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f93fffff640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9408102790 0x7f9408074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f940659c640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9408101de0 0x7f94080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f9406d9d640 1 -- 172.21.15.1:0/162210848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9408101de0 msgr2=0x7f94080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f9406d9d640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9408101de0 0x7f94080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f9406d9d640 1 -- 172.21.15.1:0/162210848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9408102790 msgr2=0x7f9408074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f9406d9d640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9408102790 0x7f9408074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f9406d9d640 1 -- 172.21.15.1:0/162210848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94081aad40 con 0x7f9408106f10 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f9406d9d640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408106f10 0x7f9408074f70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f940000eaa0 tx=0x7f940000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f93ff7fe640 1 -- 172.21.15.1:0/162210848 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400004d20 con 0x7f9408106f10 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f93ff7fe640 1 -- 172.21.15.1:0/162210848 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9400005020 con 0x7f9408106f10 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f940cedc640 1 -- 172.21.15.1:0/162210848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94081aaf70 con 0x7f9408106f10 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f93ff7fe640 1 -- 172.21.15.1:0/162210848 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94000116d0 con 0x7f9408106f10 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.746+0000 7f940cedc640 1 -- 172.21.15.1:0/162210848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94081ab3d0 con 0x7f9408106f10 2024-09-16T00:30:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f93ff7fe640 1 -- 172.21.15.1:0/162210848 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f94000051d0 con 0x7f9408106f10 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f940f2b5740 1 Processor -- start 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f940f2b5740 1 -- start start 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c447090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c447a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c454760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f940f2b5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55803c3ad2f0 con 0x55803c452370 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f940f2b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55803c354cf0 con 0x55803c4475d0 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f940f2b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55803c3a92b0 con 0x55803c448b40 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.749+0000 7f93ff7fe640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f40a3240 0x7f93f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c454760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c454760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53658/0 (socket says 172.21.15.1:53658) 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 -- 172.21.15.1:0/1111844438 learned_addr learned my addr 172.21.15.1:0/1111844438 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f940659c640 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c447a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f9406d9d640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f40a3240 0x7f93f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c447090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93ff7fe640 1 -- 172.21.15.1:0/162210848 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f94000cfef0 con 0x7f9408106f10 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 -- 172.21.15.1:0/1111844438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 msgr2=0x55803c447090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c447090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 -- 172.21.15.1:0/1111844438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 msgr2=0x55803c447a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c447a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 -- 172.21.15.1:0/1111844438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55803c3393a0 con 0x55803c452370 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f940659c640 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c447a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fffff640 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c454760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f93f0011560 tx=0x7f93f000a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745d3dafd986a47b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f9406d9d640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f40a3240 0x7f93f40a5700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f940006ef60 tx=0x7f94000c6ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fcff9640 1 -- 172.21.15.1:0/1111844438 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0005220 con 0x55803c452370 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fcff9640 1 -- 172.21.15.1:0/1111844438 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f0004c80 con 0x55803c452370 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.750+0000 7f93fcff9640 1 -- 172.21.15.1:0/1111844438 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f000fb30 con 0x55803c452370 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.751+0000 7f940f2b5740 1 -- 172.21.15.1:0/1111844438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 msgr2=0x55803c454760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.751+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c454760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f93f0011560 tx=0x7f93f000a700 comp rx=0 tx=0).stop 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.751+0000 7f940f2b5740 1 -- 172.21.15.1:0/1111844438 shutdown_connections 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.751+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c454760 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.751+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c447a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.751+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1111844438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c447090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.751+0000 7f940f2b5740 1 -- 172.21.15.1:0/1111844438 >> 172.21.15.1:0/1111844438 conn(0x55803c410310 msgr2=0x55803c443c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.751+0000 7f940f2b5740 1 -- 172.21.15.1:0/1111844438 shutdown_connections 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.751+0000 7f940f2b5740 1 -- 172.21.15.1:0/1111844438 wait complete. 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940f2b5740 1 -- start start 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940f2b5740 1 Processor -- start 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c451d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c448b40 0x55803c44f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c452370 0x55803c44f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940f2b5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55803c455070 con 0x55803c448b40 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940f2b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55803c368060 con 0x55803c4475d0 2024-09-16T00:30:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940f2b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55803c4551f0 con 0x55803c452370 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940f2b5740 10 client.0 _ll_register_callbacks cb 0x55803c0bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c448b40 0x55803c44f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c448b40 0x55803c44f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53666/0 (socket says 172.21.15.1:53666) 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940659c640 1 -- 172.21.15.1:0/1061043972 learned_addr learned my addr 172.21.15.1:0/1061043972 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f93fffff640 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c452370 0x55803c44f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f93fffff640 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c451d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940659c640 1 -- 172.21.15.1:0/1061043972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c452370 msgr2=0x55803c44f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940659c640 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c452370 0x55803c44f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940659c640 1 -- 172.21.15.1:0/1061043972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 msgr2=0x55803c451d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940659c640 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c451d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940659c640 1 -- 172.21.15.1:0/1061043972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55803c11e3c0 con 0x55803c448b40 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f93fffff640 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c452370 0x55803c44f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:19.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f940659c640 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c448b40 0x55803c44f020 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009900 tx=0x7f93f8031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c374a416afb24afb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.752+0000 7f93edffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f93edffb640 1 -- 172.21.15.1:0/1061043972 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8035040 con 0x55803c448b40 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f93edffb640 1 -- 172.21.15.1:0/1061043972 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f93f80042c0 con 0x55803c448b40 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f940f2b5740 1 -- 172.21.15.1:0/1061043972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f0004c80 con 0x55803c448b40 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f93edffb640 1 -- 172.21.15.1:0/1061043972 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f93f80044c0 con 0x55803c448b40 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f940f2b5740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f940f2b5740 1 -- 172.21.15.1:0/1061043972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55803c460b10 con 0x55803c448b40 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f93edffb640 1 -- 172.21.15.1:0/1061043972 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f0004c80 con 0x55803c448b40 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f93edffb640 1 -- 172.21.15.1:0/1061043972 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f93f80035f0 con 0x55803c448b40 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f940f2b5740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f940f2b5740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f940f2b5740 20 client.15660 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55803c462020 0x55803c464430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f940f2b5740 4 client.15660 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.753+0000 7f940f2b5740 1 -- 172.21.15.1:0/1061043972 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55803c3ba860 con 0x55803c462020 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.754+0000 7f93fffff640 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55803c462020 0x55803c464430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.754+0000 7f93edffb640 10 client.15660 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.754+0000 7f93fffff640 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55803c462020 0x55803c464430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fdafdfc2547bdf2 server_cookie=f6eb81fc639b97e7 in_seq=0 out_seq=0 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f93edffb640 1 -- 172.21.15.1:0/1061043972 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55803c3ba860 con 0x55803c462020 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f93edffb640 10 client.15660 handle_command_reply: tid=0 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 client.15660 shutdown 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 -- 172.21.15.1:0/1061043972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c448b40 msgr2=0x55803c44f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c448b40 0x55803c44f020 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009900 tx=0x7f93f8031ad0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 -- 172.21.15.1:0/1061043972 shutdown_connections 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55803c462020 0x55803c464430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c452370 0x55803c44f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c448b40 0x55803c44f020 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 --2- 172.21.15.1:0/1061043972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c451d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 -- 172.21.15.1:0/1061043972 >> 172.21.15.1:0/1061043972 conn(0x55803c410310 msgr2=0x55803c449960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f93edffb640 0 client.15660 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 -- 172.21.15.1:0/1061043972 shutdown_connections 2024-09-16T00:30:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 1 -- 172.21.15.1:0/1061043972 wait complete. 2024-09-16T00:30:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.755+0000 7f940f2b5740 20 client.15660 trim_cache size 0 max 16384 2024-09-16T00:30:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.768+0000 7f940f2b5740 1 Processor -- start 2024-09-16T00:30:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.768+0000 7f940f2b5740 1 -- start start 2024-09-16T00:30:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c443e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c44e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c45e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f940f2b5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94081b7c80 con 0x55803c452370 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f940f2b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94081b5550 con 0x55803c4475d0 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f940f2b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94081b33a0 con 0x55803c448b40 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c45e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c45e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53670/0 (socket says 172.21.15.1:53670) 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 -- 172.21.15.1:0/2274919296 learned_addr learned my addr 172.21.15.1:0/2274919296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 --2- 172.21.15.1:0/2274919296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c443e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f940659c640 1 --2- 172.21.15.1:0/2274919296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c44e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 -- 172.21.15.1:0/2274919296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 msgr2=0x55803c44e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 --2- 172.21.15.1:0/2274919296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c44e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 -- 172.21.15.1:0/2274919296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 msgr2=0x55803c443e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 --2- 172.21.15.1:0/2274919296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c443e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 -- 172.21.15.1:0/2274919296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55803c44dff0 con 0x55803c452370 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93fffff640 1 --2- 172.21.15.1:0/2274919296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c45e310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f93f000f700 tx=0x7f93f000a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4de9e4c935d9deb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.769+0000 7f93effff640 1 -- 172.21.15.1:0/2274919296 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0004660 con 0x55803c452370 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f93effff640 1 -- 172.21.15.1:0/2274919296 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f0005000 con 0x55803c452370 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f93effff640 1 -- 172.21.15.1:0/2274919296 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f000faa0 con 0x55803c452370 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f940f2b5740 1 -- 172.21.15.1:0/2274919296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 msgr2=0x55803c45e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2274919296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c45e310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f93f000f700 tx=0x7f93f000a8a0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f940f2b5740 1 -- 172.21.15.1:0/2274919296 shutdown_connections 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2274919296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c452370 0x55803c45e310 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2274919296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c44e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2274919296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c4475d0 0x55803c443e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f940f2b5740 1 -- 172.21.15.1:0/2274919296 >> 172.21.15.1:0/2274919296 conn(0x55803c410310 msgr2=0x55803c443700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f940f2b5740 1 -- 172.21.15.1:0/2274919296 shutdown_connections 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.770+0000 7f940f2b5740 1 -- 172.21.15.1:0/2274919296 wait complete. 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940f2b5740 1 -- start start 2024-09-16T00:30:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940f2b5740 1 Processor -- start 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c4475d0 0x55803c45a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c44bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940f2b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c452370 0x55803c44c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940f2b5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94081b3a10 con 0x55803c4475d0 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940f2b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94081b4e80 con 0x55803c452370 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940f2b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94081b40e0 con 0x55803c448b40 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940f2b5740 10 client.0 _ll_register_callbacks cb 0x7f93f8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f93fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c4475d0 0x55803c45a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f93fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c4475d0 0x55803c45a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53680/0 (socket says 172.21.15.1:53680) 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f93fffff640 1 -- 172.21.15.1:0/2384925589 learned_addr learned my addr 172.21.15.1:0/2384925589 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f940659c640 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c44bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f93fffff640 1 -- 172.21.15.1:0/2384925589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 msgr2=0x55803c44bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f93fffff640 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c44bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f93fffff640 1 -- 172.21.15.1:0/2384925589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c452370 msgr2=0x55803c44c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f93fffff640 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c452370 0x55803c44c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.771+0000 7f93fffff640 1 -- 172.21.15.1:0/2384925589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55803c44c5f0 con 0x55803c4475d0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f93fffff640 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c4475d0 0x55803c45a900 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f93f000ab10 tx=0x7f93f0005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b8c2bd91119b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f93edffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f93edffb640 1 -- 172.21.15.1:0/2384925589 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0035070 con 0x55803c4475d0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f93edffb640 1 -- 172.21.15.1:0/2384925589 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f93f0004660 con 0x55803c4475d0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f940f2b5740 1 -- 172.21.15.1:0/2384925589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f0005000 con 0x55803c4475d0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f93edffb640 1 -- 172.21.15.1:0/2384925589 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f93f0041050 con 0x55803c4475d0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f940f2b5740 10 client.15666 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f940f2b5740 1 -- 172.21.15.1:0/2384925589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55803c4cca70 con 0x55803c4475d0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f93edffb640 1 -- 172.21.15.1:0/2384925589 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f0005000 con 0x55803c4475d0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f93edffb640 1 -- 172.21.15.1:0/2384925589 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f93f001b070 con 0x55803c4475d0 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f940f2b5740 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:19.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f940f2b5740 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f940f2b5740 20 client.15666 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55803c4cce20 0x55803c4cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f940f2b5740 4 client.15666 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.772+0000 7f940f2b5740 1 -- 172.21.15.1:0/2384925589 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55803c3b4d70 con 0x55803c4cce20 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.773+0000 7f940659c640 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55803c4cce20 0x55803c4cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.773+0000 7f93edffb640 10 client.15666 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.773+0000 7f940659c640 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55803c4cce20 0x55803c4cf240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7a09104bede876 server_cookie=ab7678382fc62541 in_seq=0 out_seq=0 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.773+0000 7f93edffb640 1 -- 172.21.15.1:0/2384925589 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55803c3b4d70 con 0x55803c4cce20 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.773+0000 7f93edffb640 10 client.15666 handle_command_reply: tid=0 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.773+0000 7f940f2b5740 1 client.15666 shutdown 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.773+0000 7f940f2b5740 1 -- 172.21.15.1:0/2384925589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c4475d0 msgr2=0x55803c45a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.773+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c4475d0 0x55803c45a900 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f93f000ab10 tx=0x7f93f0005240 comp rx=0 tx=0).stop 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.773+0000 7f940f2b5740 1 -- 172.21.15.1:0/2384925589 shutdown_connections 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55803c4cce20 0x55803c4cf240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803c452370 0x55803c44c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803c448b40 0x55803c44bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940f2b5740 1 --2- 172.21.15.1:0/2384925589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55803c4475d0 0x55803c45a900 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f93edffb640 0 client.15666 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940f2b5740 1 -- 172.21.15.1:0/2384925589 >> 172.21.15.1:0/2384925589 conn(0x55803c410310 msgr2=0x55803c450880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940f2b5740 1 -- 172.21.15.1:0/2384925589 shutdown_connections 2024-09-16T00:30:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940f2b5740 1 -- 172.21.15.1:0/2384925589 wait complete. 2024-09-16T00:30:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940f2b5740 20 client.15666 trim_cache size 0 max 16384 2024-09-16T00:30:19.787 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:30:19.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 -- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f40a3240 msgr2=0x7f93f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f40a3240 0x7f93f40a5700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f940006ef60 tx=0x7f94000c6ee0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 -- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408106f10 msgr2=0x7f9408074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408106f10 0x7f9408074f70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f940000eaa0 tx=0x7f940000eea0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 -- 172.21.15.1:0/162210848 shutdown_connections 2024-09-16T00:30:19.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9408106f10 0x7f9408074f70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9408102790 0x7f9408074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f93f40a3240 0x7f93f40a5700 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 --2- 172.21.15.1:0/162210848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9408101de0 0x7f94080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 -- 172.21.15.1:0/162210848 >> 172.21.15.1:0/162210848 conn(0x7f94080fd8c0 msgr2=0x7f94080fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 -- 172.21.15.1:0/162210848 shutdown_connections 2024-09-16T00:30:19.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:19.774+0000 7f940cedc640 1 -- 172.21.15.1:0/162210848 wait complete. 2024-09-16T00:30:20.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:20 smithi019 ceph-mon[35186]: pgmap v408: 129 pgs: 129 active+clean; 788 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:30:20.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:20 smithi001 ceph-mon[30934]: pgmap v408: 129 pgs: 129 active+clean; 788 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:30:20.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:20 smithi153 ceph-mon[35538]: pgmap v408: 129 pgs: 129 active+clean; 788 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:30:21.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:21.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:21 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:21 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:21.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:21 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:21.494 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:30:21.494 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:22.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:22 smithi001 ceph-mon[30934]: pgmap v409: 129 pgs: 129 active+clean; 788 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:22.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:22 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:22.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:22 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:22.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:22 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.278+0000 7fdcbff59640 1 -- 172.21.15.1:0/495040265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb81037a0 msgr2=0x7fdcb810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.278+0000 7fdcbff59640 1 --2- 172.21.15.1:0/495040265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb81037a0 0x7fdcb810bc00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009900 tx=0x7fdca802a410 comp rx=0 tx=0).stop 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.278+0000 7fdcbff59640 1 -- 172.21.15.1:0/495040265 shutdown_connections 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.278+0000 7fdcbff59640 1 --2- 172.21.15.1:0/495040265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb810c2f0 0x7fdcb810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.278+0000 7fdcbff59640 1 --2- 172.21.15.1:0/495040265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb81037a0 0x7fdcb810bc00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.278+0000 7fdcbff59640 1 --2- 172.21.15.1:0/495040265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcb8102df0 0x7fdcb81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.278+0000 7fdcbff59640 1 -- 172.21.15.1:0/495040265 >> 172.21.15.1:0/495040265 conn(0x7fdcb80fc7d0 msgr2=0x7fdcb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.278+0000 7fdcbff59640 1 -- 172.21.15.1:0/495040265 shutdown_connections 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.278+0000 7fdcbff59640 1 -- 172.21.15.1:0/495040265 wait complete. 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.279+0000 7fdcbff59640 1 Processor -- start 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.279+0000 7fdcbff59640 1 -- start start 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.279+0000 7fdcbff59640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcb8102df0 0x7fdcb81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.279+0000 7fdcbff59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb81037a0 0x7fdcb8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbff59640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb810c2f0 0x7fdcb819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbff59640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb81120b0 con 0x7fdcb810c2f0 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbff59640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8111f30 con 0x7fdcb81037a0 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbff59640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8112230 con 0x7fdcb8102df0 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbe4f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb810c2f0 0x7fdcb819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbe4f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb810c2f0 0x7fdcb819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60228/0 (socket says 172.21.15.1:60228) 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbe4f6640 1 -- 172.21.15.1:0/4151293862 learned_addr learned my addr 172.21.15.1:0/4151293862 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb81037a0 0x7fdcb8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcb8102df0 0x7fdcb81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbe4f6640 1 -- 172.21.15.1:0/4151293862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcb8102df0 msgr2=0x7fdcb81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbe4f6640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcb8102df0 0x7fdcb81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbe4f6640 1 -- 172.21.15.1:0/4151293862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb81037a0 msgr2=0x7fdcb8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbe4f6640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb81037a0 0x7fdcb8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbe4f6640 1 -- 172.21.15.1:0/4151293862 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb819d560 con 0x7fdcb810c2f0 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb81037a0 0x7fdcb8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbe4f6640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb810c2f0 0x7fdcb819ce80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009fd0 tx=0x7fdcb000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcaeffd640 1 -- 172.21.15.1:0/4151293862 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0017070 con 0x7fdcb810c2f0 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbff59640 1 -- 172.21.15.1:0/4151293862 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb819d850 con 0x7fdcb810c2f0 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcaeffd640 1 -- 172.21.15.1:0/4151293862 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb0003e50 con 0x7fdcb810c2f0 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcaeffd640 1 -- 172.21.15.1:0/4151293862 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00106c0 con 0x7fdcb810c2f0 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.280+0000 7fdcbff59640 1 -- 172.21.15.1:0/4151293862 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb81a5130 con 0x7fdcb810c2f0 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.283+0000 7fdcaeffd640 1 -- 172.21.15.1:0/4151293862 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fdcb0003970 con 0x7fdcb810c2f0 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.283+0000 7fdcc2332740 1 Processor -- start 2024-09-16T00:30:22.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.283+0000 7fdcc2332740 1 -- start start 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.283+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b6126f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.283+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 0x5559b6131800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.283+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b6134130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.283+0000 7fdcc2332740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5559b608c280 con 0x5559b6131d40 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.283+0000 7fdcc2332740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5559b6033b10 con 0x5559b6126b60 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.283+0000 7fdcc2332740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5559b6088240 con 0x5559b6127480 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b6134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b6134130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60234/0 (socket says 172.21.15.1:60234) 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/374798968 learned_addr learned my addr 172.21.15.1:0/374798968 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcaeffd640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdca40a3240 0x7fdca40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/374798968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b6126f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/374798968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 0x5559b6131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/374798968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 msgr2=0x5559b6131800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/374798968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 0x5559b6131800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/374798968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 msgr2=0x5559b6126f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/374798968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b6126f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/374798968 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b60401f0 con 0x5559b6131d40 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbe4f6640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdca40a3240 0x7fdca40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcaeffd640 1 -- 172.21.15.1:0/4151293862 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdcb00cfb30 con 0x7fdcb810c2f0 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/374798968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b6134130 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80697d0 tx=0x7fdca8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd2174976a95e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdca3fff640 1 -- 172.21.15.1:0/374798968 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8035040 con 0x5559b6131d40 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdca3fff640 1 -- 172.21.15.1:0/374798968 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca8004030 con 0x5559b6131d40 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdca3fff640 1 -- 172.21.15.1:0/374798968 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8003c60 con 0x5559b6131d40 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.284+0000 7fdcbe4f6640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdca40a3240 0x7fdca40a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0005c40 tx=0x7fdcb00044b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.285+0000 7fdcc2332740 1 -- 172.21.15.1:0/374798968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 msgr2=0x5559b6134130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.285+0000 7fdcc2332740 1 --2- 172.21.15.1:0/374798968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b6134130 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80697d0 tx=0x7fdca8005750 comp rx=0 tx=0).stop 2024-09-16T00:30:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.285+0000 7fdcc2332740 1 -- 172.21.15.1:0/374798968 shutdown_connections 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.285+0000 7fdcc2332740 1 --2- 172.21.15.1:0/374798968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b6134130 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.285+0000 7fdcc2332740 1 --2- 172.21.15.1:0/374798968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 0x5559b6131800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.285+0000 7fdcc2332740 1 --2- 172.21.15.1:0/374798968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b6126f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.285+0000 7fdcc2332740 1 -- 172.21.15.1:0/374798968 >> 172.21.15.1:0/374798968 conn(0x5559b61230f0 msgr2=0x5559b61234d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.285+0000 7fdcc2332740 1 -- 172.21.15.1:0/374798968 shutdown_connections 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.285+0000 7fdcc2332740 1 -- 172.21.15.1:0/374798968 wait complete. 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcc2332740 1 -- start start 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcc2332740 1 Processor -- start 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6126b60 0x5559b612ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6127480 0x5559b612f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6131d40 0x5559b612f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcc2332740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5559b6134910 con 0x5559b6126b60 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcc2332740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5559b6046f10 con 0x5559b6127480 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcc2332740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5559b6134a90 con 0x5559b6131d40 2024-09-16T00:30:22.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcc2332740 10 client.0 _ll_register_callbacks cb 0x5559b600fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcbd4f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6126b60 0x5559b612ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcbd4f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6126b60 0x5559b612ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60244/0 (socket says 172.21.15.1:60244) 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/3093179654 learned_addr learned my addr 172.21.15.1:0/3093179654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.286+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6127480 0x5559b612f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/3093179654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6131d40 msgr2=0x5559b612f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6131d40 0x5559b612f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/3093179654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6127480 msgr2=0x5559b612f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6127480 0x5559b612f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/3093179654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b5dfd6a0 con 0x5559b6126b60 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6127480 0x5559b612f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6126b60 0x5559b612ec40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdca80057c0 tx=0x7fdca80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5db38c7bf6f898b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdca17fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdca17fa640 1 -- 172.21.15.1:0/3093179654 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8035070 con 0x5559b6126b60 2024-09-16T00:30:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdca17fa640 1 -- 172.21.15.1:0/3093179654 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdca80319c0 con 0x5559b6126b60 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcc2332740 1 -- 172.21.15.1:0/3093179654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca8004030 con 0x5559b6126b60 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdca17fa640 1 -- 172.21.15.1:0/3093179654 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdca805b050 con 0x5559b6126b60 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcc2332740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdcc2332740 1 -- 172.21.15.1:0/3093179654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559b6140540 con 0x5559b6126b60 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdca17fa640 1 -- 172.21.15.1:0/3093179654 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca8004030 con 0x5559b6126b60 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.287+0000 7fdca17fa640 1 -- 172.21.15.1:0/3093179654 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdca803a040 con 0x5559b6126b60 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.288+0000 7fdcc2332740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.288+0000 7fdcc2332740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.288+0000 7fdcc2332740 20 client.15678 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.288+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5559b6141890 0x5559b6143c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.288+0000 7fdcc2332740 4 client.15678 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.288+0000 7fdcc2332740 1 -- 172.21.15.1:0/3093179654 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559b60997f0 con 0x5559b6141890 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.288+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5559b6141890 0x5559b6143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.288+0000 7fdca17fa640 10 client.15678 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.288+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5559b6141890 0x5559b6143c80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e407c3f7e3d467 server_cookie=a4a71fee9da8fc86 in_seq=0 out_seq=0 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdca17fa640 1 -- 172.21.15.1:0/3093179654 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559b60997f0 con 0x5559b6141890 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdca17fa640 10 client.15678 handle_command_reply: tid=0 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 client.15678 shutdown 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 -- 172.21.15.1:0/3093179654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6126b60 msgr2=0x5559b612ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6126b60 0x5559b612ec40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdca80057c0 tx=0x7fdca80045e0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 -- 172.21.15.1:0/3093179654 shutdown_connections 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5559b6141890 0x5559b6143c80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6131d40 0x5559b612f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6127480 0x5559b612f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3093179654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6126b60 0x5559b612ec40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdca17fa640 0 client.15678 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 -- 172.21.15.1:0/3093179654 >> 172.21.15.1:0/3093179654 conn(0x5559b61230f0 msgr2=0x5559b6128dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 -- 172.21.15.1:0/3093179654 shutdown_connections 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.289+0000 7fdcc2332740 1 -- 172.21.15.1:0/3093179654 wait complete. 2024-09-16T00:30:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.290+0000 7fdcc2332740 20 client.15678 trim_cache size 0 max 16384 2024-09-16T00:30:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcc2332740 1 Processor -- start 2024-09-16T00:30:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcc2332740 1 -- start start 2024-09-16T00:30:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b612daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6127480 0x5559b6149220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6131d40 0x5559b6149760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcc2332740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb81b15e0 con 0x5559b6127480 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcc2332740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb81aeeb0 con 0x5559b6126b60 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcc2332740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb81acd00 con 0x5559b6131d40 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcbdcf5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6127480 0x5559b6149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcbdcf5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6127480 0x5559b6149220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60252/0 (socket says 172.21.15.1:60252) 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.307+0000 7fdcbdcf5640 1 -- 172.21.15.1:0/3771266598 learned_addr learned my addr 172.21.15.1:0/3771266598 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b612daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6131d40 0x5559b6149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdcbdcf5640 1 -- 172.21.15.1:0/3771266598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6131d40 msgr2=0x5559b6149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6131d40 0x5559b6149760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdcbdcf5640 1 -- 172.21.15.1:0/3771266598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 msgr2=0x5559b612daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b612daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdcbdcf5640 1 -- 172.21.15.1:0/3771266598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b6131220 con 0x5559b6127480 2024-09-16T00:30:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b612daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6127480 0x5559b6149220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x5559b6130280 tx=0x7fdcb400a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a7713bdd5bd8f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdca37fe640 1 -- 172.21.15.1:0/3771266598 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4004110 con 0x5559b6127480 2024-09-16T00:30:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdca37fe640 1 -- 172.21.15.1:0/3771266598 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb4005500 con 0x5559b6127480 2024-09-16T00:30:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.308+0000 7fdca37fe640 1 -- 172.21.15.1:0/3771266598 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40035f0 con 0x5559b6127480 2024-09-16T00:30:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 -- 172.21.15.1:0/3771266598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6127480 msgr2=0x5559b6149220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6127480 0x5559b6149220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x5559b6130280 tx=0x7fdcb400a310 comp rx=0 tx=0).stop 2024-09-16T00:30:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 -- 172.21.15.1:0/3771266598 shutdown_connections 2024-09-16T00:30:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6131d40 0x5559b6149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6127480 0x5559b6149220 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 --2- 172.21.15.1:0/3771266598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b612daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 -- 172.21.15.1:0/3771266598 >> 172.21.15.1:0/3771266598 conn(0x5559b61230f0 msgr2=0x5559b612d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 -- 172.21.15.1:0/3771266598 shutdown_connections 2024-09-16T00:30:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 -- 172.21.15.1:0/3771266598 wait complete. 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 -- start start 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.309+0000 7fdcc2332740 1 Processor -- start 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b61361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 0x5559b6136730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcc2332740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b61ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcc2332740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb81ad370 con 0x5559b6131d40 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcc2332740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb81ae7e0 con 0x5559b6126b60 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcc2332740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb81ada40 con 0x5559b6127480 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcc2332740 10 client.0 _ll_register_callbacks cb 0x7fdca8032eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b61ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b61ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60256/0 (socket says 172.21.15.1:60256) 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/1713462147 learned_addr learned my addr 172.21.15.1:0/1713462147 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b61361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 0x5559b6136730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/1713462147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 msgr2=0x5559b6136730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 0x5559b6136730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/1713462147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 msgr2=0x5559b61361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b61361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 -- 172.21.15.1:0/1713462147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b61ab7e0 con 0x5559b6131d40 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdcbd4f4640 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b61ab1f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdca8033000 tx=0x7fdca804b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385e91b6afe78105 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.310+0000 7fdca17fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdca17fa640 1 -- 172.21.15.1:0/1713462147 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8035070 con 0x5559b6131d40 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdca17fa640 1 -- 172.21.15.1:0/1713462147 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdca8044040 con 0x5559b6131d40 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdcc2332740 1 -- 172.21.15.1:0/1713462147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb4005500 con 0x5559b6131d40 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdca17fa640 1 -- 172.21.15.1:0/1713462147 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdca80315b0 con 0x5559b6131d40 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdcc2332740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdcc2332740 1 -- 172.21.15.1:0/1713462147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559b61aca50 con 0x5559b6131d40 2024-09-16T00:30:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdca17fa640 1 -- 172.21.15.1:0/1713462147 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb4005500 con 0x5559b6131d40 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdca17fa640 1 -- 172.21.15.1:0/1713462147 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdca8005030 con 0x5559b6131d40 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdcc2332740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdcc2332740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdcc2332740 20 client.15684 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdcc2332740 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5559b61ad0b0 0x5559b61af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdcc2332740 4 client.15684 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.311+0000 7fdcc2332740 1 -- 172.21.15.1:0/1713462147 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5559b6093d00 con 0x5559b61ad0b0 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.312+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5559b61ad0b0 0x5559b61af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.312+0000 7fdcbdcf5640 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5559b61ad0b0 0x5559b61af4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631fcb28ec282120 server_cookie=5de41fc9e036f54c in_seq=0 out_seq=0 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.312+0000 7fdca17fa640 10 client.15684 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdca17fa640 1 -- 172.21.15.1:0/1713462147 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5559b6093d00 con 0x5559b61ad0b0 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdca17fa640 10 client.15684 handle_command_reply: tid=0 2024-09-16T00:30:22.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 client.15684 shutdown 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 -- 172.21.15.1:0/1713462147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 msgr2=0x5559b61ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b61ab1f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdca8033000 tx=0x7fdca804b040 comp rx=0 tx=0).stop 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 -- 172.21.15.1:0/1713462147 shutdown_connections 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5559b61ad0b0 0x5559b61af4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559b6131d40 0x5559b61ab1f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559b6127480 0x5559b6136730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 --2- 172.21.15.1:0/1713462147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559b6126b60 0x5559b61361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdca17fa640 0 client.15684 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 -- 172.21.15.1:0/1713462147 >> 172.21.15.1:0/1713462147 conn(0x5559b61230f0 msgr2=0x5559b612f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 -- 172.21.15.1:0/1713462147 shutdown_connections 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.313+0000 7fdcc2332740 1 -- 172.21.15.1:0/1713462147 wait complete. 2024-09-16T00:30:22.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcc2332740 20 client.15684 trim_cache size 0 max 16384 2024-09-16T00:30:22.319 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:30:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:30:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "75f88b83-5617-44b0-9bc9-43a960bbc4d0", 2024-09-16T00:30:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:30:22.319 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 -- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdca40a3240 msgr2=0x7fdca40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdca40a3240 0x7fdca40a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0005c40 tx=0x7fdcb00044b0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 -- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb810c2f0 msgr2=0x7fdcb819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb810c2f0 0x7fdcb819ce80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009fd0 tx=0x7fdcb000b260 comp rx=0 tx=0).stop 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 -- 172.21.15.1:0/4151293862 shutdown_connections 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcb810c2f0 0x7fdcb819ce80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb81037a0 0x7fdcb8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdca40a3240 0x7fdca40a5700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 --2- 172.21.15.1:0/4151293862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcb8102df0 0x7fdcb81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 -- 172.21.15.1:0/4151293862 >> 172.21.15.1:0/4151293862 conn(0x7fdcb80fc7d0 msgr2=0x7fdcb80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:22.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 -- 172.21.15.1:0/4151293862 shutdown_connections 2024-09-16T00:30:22.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:22.314+0000 7fdcbff59640 1 -- 172.21.15.1:0/4151293862 wait complete. 2024-09-16T00:30:22.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:22 smithi019 ceph-mon[35186]: pgmap v409: 129 pgs: 129 active+clean; 788 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:22.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:22 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:22.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:22 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:22.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:22 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:22 smithi153 ceph-mon[35538]: pgmap v409: 129 pgs: 129 active+clean; 788 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:22 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:22 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:22 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:30:23.020 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:30:23.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:23 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:30:23.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:23 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:23.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:23 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:30:23.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:23 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:23.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:23 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:30:23.279 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:23.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:23 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:30:23.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:23 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:23.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:23 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:30:23.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:23 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:23.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:23 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:30:23.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:23 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:30:23.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:23 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:23.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:23 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:30:23.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:23 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:23.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:23 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:30:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.796+0000 7fe6a997f640 1 -- 172.21.15.1:0/2924138272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4101de0 msgr2=0x7fe6a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.796+0000 7fe6a997f640 1 --2- 172.21.15.1:0/2924138272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4101de0 0x7fe6a41021c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe694009900 tx=0x7fe69402a410 comp rx=0 tx=0).stop 2024-09-16T00:30:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.796+0000 7fe6a997f640 1 -- 172.21.15.1:0/2924138272 shutdown_connections 2024-09-16T00:30:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.796+0000 7fe6a997f640 1 --2- 172.21.15.1:0/2924138272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4106f10 0x7fe6a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.796+0000 7fe6a997f640 1 --2- 172.21.15.1:0/2924138272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a4102790 0x7fe6a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.796+0000 7fe6a997f640 1 --2- 172.21.15.1:0/2924138272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4101de0 0x7fe6a41021c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.796+0000 7fe6a997f640 1 -- 172.21.15.1:0/2924138272 >> 172.21.15.1:0/2924138272 conn(0x7fe6a40fd8c0 msgr2=0x7fe6a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.796+0000 7fe6a997f640 1 -- 172.21.15.1:0/2924138272 shutdown_connections 2024-09-16T00:30:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.796+0000 7fe6a997f640 1 -- 172.21.15.1:0/2924138272 wait complete. 2024-09-16T00:30:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a997f640 1 Processor -- start 2024-09-16T00:30:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a997f640 1 -- start start 2024-09-16T00:30:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a997f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a4101de0 0x7fe6a40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a997f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4102790 0x7fe6a4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a997f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4106f10 0x7fe6a4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a997f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4111f70 con 0x7fe6a4102790 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a997f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4111df0 con 0x7fe6a4101de0 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a997f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41120f0 con 0x7fe6a4106f10 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4102790 0x7fe6a4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4102790 0x7fe6a4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60270/0 (socket says 172.21.15.1:60270) 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.797+0000 7fe6a27fc640 1 -- 172.21.15.1:0/1243206629 learned_addr learned my addr 172.21.15.1:0/1243206629 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a2ffd640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a4101de0 0x7fe6a40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4106f10 0x7fe6a4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a27fc640 1 -- 172.21.15.1:0/1243206629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4106f10 msgr2=0x7fe6a4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a27fc640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4106f10 0x7fe6a4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a27fc640 1 -- 172.21.15.1:0/1243206629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a4101de0 msgr2=0x7fe6a40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a27fc640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a4101de0 0x7fe6a40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a27fc640 1 -- 172.21.15.1:0/1243206629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a41a6890 con 0x7fe6a4102790 2024-09-16T00:30:23.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a27fc640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4102790 0x7fe6a4074a30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c890 tx=0x7fe69800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:23.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a897d640 1 -- 172.21.15.1:0/1243206629 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698004220 con 0x7fe6a4102790 2024-09-16T00:30:23.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a997f640 1 -- 172.21.15.1:0/1243206629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a41a6b80 con 0x7fe6a4102790 2024-09-16T00:30:23.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a897d640 1 -- 172.21.15.1:0/1243206629 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6980043c0 con 0x7fe6a4102790 2024-09-16T00:30:23.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a897d640 1 -- 172.21.15.1:0/1243206629 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698004a00 con 0x7fe6a4102790 2024-09-16T00:30:23.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.798+0000 7fe6a997f640 1 -- 172.21.15.1:0/1243206629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a41a70c0 con 0x7fe6a4102790 2024-09-16T00:30:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6a897d640 1 -- 172.21.15.1:0/1243206629 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe6980079b0 con 0x7fe6a4102790 2024-09-16T00:30:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6abd58740 1 Processor -- start 2024-09-16T00:30:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6abd58740 1 -- start start 2024-09-16T00:30:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7d4a0 0x55e284c7d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7ddc0 0x55e284c88140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c8aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6abd58740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e284be32e0 con 0x55e284c88680 2024-09-16T00:30:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6abd58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e284b8ace0 con 0x55e284c7ddc0 2024-09-16T00:30:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6abd58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e284bdf2a0 con 0x55e284c7d4a0 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.801+0000 7fe6a897d640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe68c0a3240 0x7fe68c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c8aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe68c0a3240 0x7fe68c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c8aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60286/0 (socket says 172.21.15.1:60286) 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a27fc640 1 -- 172.21.15.1:0/1298800752 learned_addr learned my addr 172.21.15.1:0/1298800752 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/1298800752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7d4a0 0x55e284c7d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a2ffd640 1 --2- 172.21.15.1:0/1298800752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7ddc0 0x55e284c88140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a897d640 1 -- 172.21.15.1:0/1243206629 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe6980cfa70 con 0x7fe6a4102790 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a27fc640 1 -- 172.21.15.1:0/1298800752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7d4a0 msgr2=0x55e284c7d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a27fc640 1 --2- 172.21.15.1:0/1298800752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7d4a0 0x55e284c7d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a27fc640 1 -- 172.21.15.1:0/1298800752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7ddc0 msgr2=0x55e284c88140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a27fc640 1 --2- 172.21.15.1:0/1298800752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7ddc0 0x55e284c88140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a27fc640 1 -- 172.21.15.1:0/1298800752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e284b6f440 con 0x55e284c88680 2024-09-16T00:30:23.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe68c0a3240 0x7fe68c0a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe69c007fd0 tx=0x7fe69c007c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:23.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.802+0000 7fe6a27fc640 1 --2- 172.21.15.1:0/1298800752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c8aa70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe6980058c0 tx=0x7fe698005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c85dfc3e034aba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:23.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe689ffb640 1 -- 172.21.15.1:0/1298800752 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69801d040 con 0x55e284c88680 2024-09-16T00:30:23.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe689ffb640 1 -- 172.21.15.1:0/1298800752 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69809c330 con 0x55e284c88680 2024-09-16T00:30:23.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe689ffb640 1 -- 172.21.15.1:0/1298800752 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698099280 con 0x55e284c88680 2024-09-16T00:30:23.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe6abd58740 1 -- 172.21.15.1:0/1298800752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 msgr2=0x55e284c8aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1298800752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c8aa70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe6980058c0 tx=0x7fe698005300 comp rx=0 tx=0).stop 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe6abd58740 1 -- 172.21.15.1:0/1298800752 shutdown_connections 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1298800752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c8aa70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1298800752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7ddc0 0x55e284c88140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1298800752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7d4a0 0x55e284c7d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe6abd58740 1 -- 172.21.15.1:0/1298800752 >> 172.21.15.1:0/1298800752 conn(0x55e284c79a30 msgr2=0x55e284c79e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe6abd58740 1 -- 172.21.15.1:0/1298800752 shutdown_connections 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.803+0000 7fe6abd58740 1 -- 172.21.15.1:0/1298800752 wait complete. 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6abd58740 1 -- start start 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6abd58740 1 Processor -- start 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 0x55e284c87f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7ddc0 0x55e284c85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c85750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6abd58740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e284c8b250 con 0x55e284c88680 2024-09-16T00:30:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6abd58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e284b9e050 con 0x55e284c7d4a0 2024-09-16T00:30:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6abd58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e284c8b3d0 con 0x55e284c7ddc0 2024-09-16T00:30:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6abd58740 10 client.0 _ll_register_callbacks cb 0x55e284bde160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c85750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c85750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60294/0 (socket says 172.21.15.1:60294) 2024-09-16T00:30:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6a37fe640 1 -- 172.21.15.1:0/367006690 learned_addr learned my addr 172.21.15.1:0/367006690 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.804+0000 7fe6a2ffd640 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 0x55e284c87f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6a27fc640 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7ddc0 0x55e284c85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6a37fe640 1 -- 172.21.15.1:0/367006690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7ddc0 msgr2=0x55e284c85210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7ddc0 0x55e284c85210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6a37fe640 1 -- 172.21.15.1:0/367006690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 msgr2=0x55e284c87f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 0x55e284c87f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6a37fe640 1 -- 172.21.15.1:0/367006690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2849543c0 con 0x55e284c88680 2024-09-16T00:30:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6a2ffd640 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 0x55e284c87f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c85750 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe69c007f20 tx=0x7fe69c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b444935d34a78b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:23.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:23.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe682ffd640 1 -- 172.21.15.1:0/367006690 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c024030 con 0x55e284c88680 2024-09-16T00:30:23.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe682ffd640 1 -- 172.21.15.1:0/367006690 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe69c010cc0 con 0x55e284c88680 2024-09-16T00:30:23.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6abd58740 1 -- 172.21.15.1:0/367006690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe69809c330 con 0x55e284c88680 2024-09-16T00:30:23.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe682ffd640 1 -- 172.21.15.1:0/367006690 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe69c031050 con 0x55e284c88680 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6abd58740 10 client.15696 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.805+0000 7fe6abd58740 1 -- 172.21.15.1:0/367006690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e284c96bc0 con 0x55e284c88680 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.806+0000 7fe682ffd640 1 -- 172.21.15.1:0/367006690 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69809c330 con 0x55e284c88680 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.806+0000 7fe682ffd640 1 -- 172.21.15.1:0/367006690 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe69c02b4d0 con 0x55e284c88680 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.806+0000 7fe6abd58740 10 client.15696 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.806+0000 7fe6abd58740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.806+0000 7fe6abd58740 20 client.15696 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.806+0000 7fe6abd58740 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e284c97fd0 0x55e284c9a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.806+0000 7fe6abd58740 4 client.15696 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.806+0000 7fe6abd58740 1 -- 172.21.15.1:0/367006690 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e284bf0850 con 0x55e284c97fd0 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.806+0000 7fe6a2ffd640 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e284c97fd0 0x55e284c9a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.807+0000 7fe6a2ffd640 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e284c97fd0 0x55e284c9a450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae033808f29d8d49 server_cookie=2d84c14d174af93f in_seq=0 out_seq=0 2024-09-16T00:30:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.807+0000 7fe682ffd640 10 client.15696 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.807+0000 7fe682ffd640 1 -- 172.21.15.1:0/367006690 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e284bf0850 con 0x55e284c97fd0 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.807+0000 7fe682ffd640 10 client.15696 handle_command_reply: tid=0 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.807+0000 7fe6abd58740 1 client.15696 shutdown 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.807+0000 7fe6abd58740 1 -- 172.21.15.1:0/367006690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 msgr2=0x55e284c85750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.807+0000 7fe6abd58740 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c85750 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe69c007f20 tx=0x7fe69c01a000 comp rx=0 tx=0).stop 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe6abd58740 1 -- 172.21.15.1:0/367006690 shutdown_connections 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe6abd58740 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e284c97fd0 0x55e284c9a450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe6abd58740 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c88680 0x55e284c85750 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe6abd58740 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7ddc0 0x55e284c85210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe682ffd640 0 client.15696 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe6abd58740 1 --2- 172.21.15.1:0/367006690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 0x55e284c87f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe6abd58740 1 -- 172.21.15.1:0/367006690 >> 172.21.15.1:0/367006690 conn(0x55e284c79a30 msgr2=0x55e284c7f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe6abd58740 1 -- 172.21.15.1:0/367006690 shutdown_connections 2024-09-16T00:30:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe6abd58740 1 -- 172.21.15.1:0/367006690 wait complete. 2024-09-16T00:30:23.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.808+0000 7fe6abd58740 20 client.15696 trim_cache size 0 max 16384 2024-09-16T00:30:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6abd58740 1 Processor -- start 2024-09-16T00:30:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6abd58740 1 -- start start 2024-09-16T00:30:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c79a30 0x55e284c944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7d4a0 0x55e284c80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c9fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6abd58740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41b3930 con 0x55e284c7ddc0 2024-09-16T00:30:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6abd58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41b1200 con 0x55e284c79a30 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6abd58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41af050 con 0x55e284c7d4a0 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c9fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c9fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60306/0 (socket says 172.21.15.1:60306) 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.825+0000 7fe6a37fe640 1 -- 172.21.15.1:0/906911318 learned_addr learned my addr 172.21.15.1:0/906911318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.826+0000 7fe6a37fe640 1 -- 172.21.15.1:0/906911318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7d4a0 msgr2=0x55e284c80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.826+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/906911318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7d4a0 0x55e284c80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.826+0000 7fe6a37fe640 1 -- 172.21.15.1:0/906911318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c79a30 msgr2=0x55e284c944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.826+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/906911318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c79a30 0x55e284c944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.826+0000 7fe6a37fe640 1 -- 172.21.15.1:0/906911318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e284c8adb0 con 0x55e284c7ddc0 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.826+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/906911318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c9fd20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe69c014010 tx=0x7fe69c013500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31781ecbc47c0103 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.826+0000 7fe6897fa640 1 -- 172.21.15.1:0/906911318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c024070 con 0x55e284c7ddc0 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.826+0000 7fe6897fa640 1 -- 172.21.15.1:0/906911318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69c003970 con 0x55e284c7ddc0 2024-09-16T00:30:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.826+0000 7fe6897fa640 1 -- 172.21.15.1:0/906911318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c02b340 con 0x55e284c7ddc0 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 -- 172.21.15.1:0/906911318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 msgr2=0x55e284c9fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 --2- 172.21.15.1:0/906911318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c9fd20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe69c014010 tx=0x7fe69c013500 comp rx=0 tx=0).stop 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 -- 172.21.15.1:0/906911318 shutdown_connections 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 --2- 172.21.15.1:0/906911318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c9fd20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 --2- 172.21.15.1:0/906911318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c7d4a0 0x55e284c80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 --2- 172.21.15.1:0/906911318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c79a30 0x55e284c944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 -- 172.21.15.1:0/906911318 >> 172.21.15.1:0/906911318 conn(0x55e284c858b0 msgr2=0x55e284c80050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 -- 172.21.15.1:0/906911318 shutdown_connections 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 -- 172.21.15.1:0/906911318 wait complete. 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 -- start start 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.827+0000 7fe6abd58740 1 Processor -- start 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c79a30 0x55e284c90b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 0x55e284c8be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6abd58740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c8c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6abd58740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41af6c0 con 0x55e284c7ddc0 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6abd58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41b0b30 con 0x55e284c7d4a0 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6abd58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41afd90 con 0x55e284c79a30 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6abd58740 10 client.0 _ll_register_callbacks cb 0x7fe69c0137e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c8c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c8c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60312/0 (socket says 172.21.15.1:60312) 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a37fe640 1 -- 172.21.15.1:0/1869143107 learned_addr learned my addr 172.21.15.1:0/1869143107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a27fc640 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 0x55e284c8be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a2ffd640 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c79a30 0x55e284c90b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a37fe640 1 -- 172.21.15.1:0/1869143107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c79a30 msgr2=0x55e284c90b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c79a30 0x55e284c90b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a37fe640 1 -- 172.21.15.1:0/1869143107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 msgr2=0x55e284c8be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 0x55e284c8be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a37fe640 1 -- 172.21.15.1:0/1869143107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e284c8c9c0 con 0x55e284c7ddc0 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6a37fe640 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c8c3d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe69c007f60 tx=0x7fe69c010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb58f0d12b2b85d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe682ffd640 1 -- 172.21.15.1:0/1869143107 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c024040 con 0x55e284c7ddc0 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe682ffd640 1 -- 172.21.15.1:0/1869143107 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe69c02ba90 con 0x55e284c7ddc0 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.828+0000 7fe6abd58740 1 -- 172.21.15.1:0/1869143107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe69c003970 con 0x55e284c7ddc0 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe682ffd640 1 -- 172.21.15.1:0/1869143107 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe69c048050 con 0x55e284c7ddc0 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe6abd58740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe6abd58740 1 -- 172.21.15.1:0/1869143107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e284d02b00 con 0x55e284c7ddc0 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe682ffd640 1 -- 172.21.15.1:0/1869143107 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69c003970 con 0x55e284c7ddc0 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe682ffd640 1 -- 172.21.15.1:0/1869143107 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe69c005310 con 0x55e284c7ddc0 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe6abd58740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe6abd58740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe6abd58740 20 client.15702 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e284c88680 0x55e284d05170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe6abd58740 4 client.15702 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe6abd58740 1 -- 172.21.15.1:0/1869143107 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e284bead60 con 0x55e284c88680 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.829+0000 7fe6a2ffd640 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e284c88680 0x55e284d05170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe682ffd640 10 client.15702 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6a2ffd640 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e284c88680 0x55e284d05170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2a5ca22fa756b0 server_cookie=d88f2555d97f3b7c in_seq=0 out_seq=0 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe682ffd640 1 -- 172.21.15.1:0/1869143107 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e284bead60 con 0x55e284c88680 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe682ffd640 10 client.15702 handle_command_reply: tid=0 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 client.15702 shutdown 2024-09-16T00:30:23.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 -- 172.21.15.1:0/1869143107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 msgr2=0x55e284c8c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c8c3d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe69c007f60 tx=0x7fe69c010ed0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 -- 172.21.15.1:0/1869143107 shutdown_connections 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e284c88680 0x55e284d05170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e284c7ddc0 0x55e284c8c3d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e284c7d4a0 0x55e284c8be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe682ffd640 0 client.15702 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 --2- 172.21.15.1:0/1869143107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e284c79a30 0x55e284c90b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 -- 172.21.15.1:0/1869143107 >> 172.21.15.1:0/1869143107 conn(0x55e284c858b0 msgr2=0x55e284c80870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 -- 172.21.15.1:0/1869143107 shutdown_connections 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 1 -- 172.21.15.1:0/1869143107 wait complete. 2024-09-16T00:30:23.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.830+0000 7fe6abd58740 20 client.15702 trim_cache size 0 max 16384 2024-09-16T00:30:23.836 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:30:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:30:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:30:23.836 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 -- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe68c0a3240 msgr2=0x7fe68c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe68c0a3240 0x7fe68c0a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe69c007fd0 tx=0x7fe69c007c60 comp rx=0 tx=0).stop 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 -- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4102790 msgr2=0x7fe6a4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4102790 0x7fe6a4074a30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c890 tx=0x7fe69800cd50 comp rx=0 tx=0).stop 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 -- 172.21.15.1:0/1243206629 shutdown_connections 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe68c0a3240 0x7fe68c0a5700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4106f10 0x7fe6a4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a4102790 0x7fe6a4074a30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 --2- 172.21.15.1:0/1243206629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a4101de0 0x7fe6a40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 -- 172.21.15.1:0/1243206629 >> 172.21.15.1:0/1243206629 conn(0x7fe6a40fd8c0 msgr2=0x7fe6a40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 -- 172.21.15.1:0/1243206629 shutdown_connections 2024-09-16T00:30:23.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:23.831+0000 7fe6a997f640 1 -- 172.21.15.1:0/1243206629 wait complete. 2024-09-16T00:30:24.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:24 smithi019 ceph-mon[35186]: pgmap v410: 129 pgs: 129 active+clean; 845 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:24.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:24 smithi001 ceph-mon[30934]: pgmap v410: 129 pgs: 129 active+clean; 845 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:24.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:24 smithi153 ceph-mon[35538]: pgmap v410: 129 pgs: 129 active+clean; 845 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:24.673 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:30:24.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:24.934 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:25.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.422+0000 7f6ae519b640 1 -- 172.21.15.1:0/4232673256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 msgr2=0x7f6ae0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:25.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.422+0000 7f6ae519b640 1 --2- 172.21.15.1:0/4232673256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 0x7f6ae0103120 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009900 tx=0x7f6ad002a480 comp rx=0 tx=0).stop 2024-09-16T00:30:25.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.423+0000 7f6ae519b640 1 -- 172.21.15.1:0/4232673256 shutdown_connections 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.423+0000 7f6ae519b640 1 --2- 172.21.15.1:0/4232673256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ae010c170 0x7f6ae010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.423+0000 7f6ae519b640 1 --2- 172.21.15.1:0/4232673256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ae0103660 0x7f6ae010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.423+0000 7f6ae519b640 1 --2- 172.21.15.1:0/4232673256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 0x7f6ae0103120 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.423+0000 7f6ae519b640 1 -- 172.21.15.1:0/4232673256 >> 172.21.15.1:0/4232673256 conn(0x7f6ae00fc7a0 msgr2=0x7f6ae00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.423+0000 7f6ae519b640 1 -- 172.21.15.1:0/4232673256 shutdown_connections 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.423+0000 7f6ae519b640 1 -- 172.21.15.1:0/4232673256 wait complete. 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.423+0000 7f6ae519b640 1 Processor -- start 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.423+0000 7f6ae519b640 1 -- start start 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6ae519b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 0x7f6ae0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6ae519b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ae0103660 0x7f6ae0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6ae519b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ae010c170 0x7f6ae019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:25.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6ae519b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae0111fc0 con 0x7f6ae0102d40 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6ae519b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae0111e40 con 0x7f6ae010c170 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6ae519b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae0112140 con 0x7f6ae0103660 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6aded9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 0x7f6ae0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6aded9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 0x7f6ae0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60338/0 (socket says 172.21.15.1:60338) 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6aded9d640 1 -- 172.21.15.1:0/159524673 learned_addr learned my addr 172.21.15.1:0/159524673 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6adf59e640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ae010c170 0x7f6ae019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6ad7fff640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ae0103660 0x7f6ae0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6aded9d640 1 -- 172.21.15.1:0/159524673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ae0103660 msgr2=0x7f6ae0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6aded9d640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ae0103660 0x7f6ae0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6aded9d640 1 -- 172.21.15.1:0/159524673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ae010c170 msgr2=0x7f6ae019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6aded9d640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ae010c170 0x7f6ae019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6aded9d640 1 -- 172.21.15.1:0/159524673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae019d3e0 con 0x7f6ae0102d40 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6aded9d640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 0x7f6ae0198430 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0004770 tx=0x7f6ad0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:25.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.424+0000 7f6adcd99640 1 -- 172.21.15.1:0/159524673 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad003f070 con 0x7f6ae0102d40 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.425+0000 7f6adcd99640 1 -- 172.21.15.1:0/159524673 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad0031ca0 con 0x7f6ae0102d40 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.425+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae019d670 con 0x7f6ae0102d40 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.425+0000 7f6adcd99640 1 -- 172.21.15.1:0/159524673 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0035030 con 0x7f6ae0102d40 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.425+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae01a4fb0 con 0x7f6ae0102d40 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.426+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae006b070 con 0x7f6ae0102d40 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.427+0000 7f6adcd99640 1 -- 172.21.15.1:0/159524673 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6ad0042070 con 0x7f6ae0102d40 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.428+0000 7f6adcd99640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6acc0a3240 0x7f6acc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.428+0000 7f6adcd99640 1 -- 172.21.15.1:0/159524673 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6ad00f0940 con 0x7f6ae0102d40 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.428+0000 7f6ad7fff640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6acc0a3240 0x7f6acc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.428+0000 7f6ad7fff640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6acc0a3240 0x7f6acc0a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6ae010d5b0 tx=0x7f6ac800b360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:25.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.431+0000 7f6adcd99640 1 -- 172.21.15.1:0/159524673 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ad00bcbb0 con 0x7f6ae0102d40 2024-09-16T00:30:25.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.644+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ae0103120 con 0x7f6ae0102d40 2024-09-16T00:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.645+0000 7f6adcd99640 1 -- 172.21.15.1:0/159524673 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f6ae0103120 con 0x7f6ae0102d40 2024-09-16T00:30:25.645 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:30:25.646 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6acc0a3240 msgr2=0x7f6acc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6acc0a3240 0x7f6acc0a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6ae010d5b0 tx=0x7f6ac800b360 comp rx=0 tx=0).stop 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 msgr2=0x7f6ae0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 0x7f6ae0198430 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0004770 tx=0x7f6ad0003c10 comp rx=0 tx=0).stop 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 shutdown_connections 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ae010c170 0x7f6ae019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6acc0a3240 0x7f6acc0a5700 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ae0103660 0x7f6ae0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 --2- 172.21.15.1:0/159524673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ae0102d40 0x7f6ae0198430 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 >> 172.21.15.1:0/159524673 conn(0x7f6ae00fc7a0 msgr2=0x7f6ae00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 shutdown_connections 2024-09-16T00:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:25.647+0000 7f6ae519b640 1 -- 172.21.15.1:0/159524673 wait complete. 2024-09-16T00:30:26.347 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:26 smithi001 ceph-mon[30934]: pgmap v411: 129 pgs: 129 active+clean; 845 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:26.347 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:26 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/159524673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:26.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:26 smithi019 ceph-mon[35186]: pgmap v411: 129 pgs: 129 active+clean; 845 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:26 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/159524673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:26.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:30:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:26 smithi153 ceph-mon[35538]: pgmap v411: 129 pgs: 129 active+clean; 845 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:26 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/159524673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:26.635 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:27.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.158+0000 7f8e91925640 1 -- 172.21.15.1:0/38863453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c103660 msgr2=0x7f8e8c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.158+0000 7f8e91925640 1 --2- 172.21.15.1:0/38863453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c103660 0x7f8e8c10bb50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009900 tx=0x7f8e8002a480 comp rx=0 tx=0).stop 2024-09-16T00:30:27.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.158+0000 7f8e91925640 1 -- 172.21.15.1:0/38863453 shutdown_connections 2024-09-16T00:30:27.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.158+0000 7f8e91925640 1 --2- 172.21.15.1:0/38863453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8c10c170 0x7f8e8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.158+0000 7f8e91925640 1 --2- 172.21.15.1:0/38863453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c103660 0x7f8e8c10bb50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.158+0000 7f8e91925640 1 --2- 172.21.15.1:0/38863453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c102d40 0x7f8e8c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.158+0000 7f8e91925640 1 -- 172.21.15.1:0/38863453 >> 172.21.15.1:0/38863453 conn(0x7f8e8c0fc7a0 msgr2=0x7f8e8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:27.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.158+0000 7f8e91925640 1 -- 172.21.15.1:0/38863453 shutdown_connections 2024-09-16T00:30:27.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.158+0000 7f8e91925640 1 -- 172.21.15.1:0/38863453 wait complete. 2024-09-16T00:30:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.159+0000 7f8e91925640 1 Processor -- start 2024-09-16T00:30:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.159+0000 7f8e91925640 1 -- start start 2024-09-16T00:30:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.159+0000 7f8e91925640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c102d40 0x7f8e8c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.159+0000 7f8e91925640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8c103660 0x7f8e8c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.159+0000 7f8e91925640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c10c170 0x7f8e8c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.159+0000 7f8e91925640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c111f70 con 0x7f8e8c102d40 2024-09-16T00:30:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.159+0000 7f8e91925640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c111df0 con 0x7f8e8c103660 2024-09-16T00:30:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.159+0000 7f8e91925640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1120f0 con 0x7f8e8c10c170 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c102d40 0x7f8e8c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c102d40 0x7f8e8c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60348/0 (socket says 172.21.15.1:60348) 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8affd640 1 -- 172.21.15.1:0/3037542376 learned_addr learned my addr 172.21.15.1:0/3037542376 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c10c170 0x7f8e8c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8c103660 0x7f8e8c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8affd640 1 -- 172.21.15.1:0/3037542376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c10c170 msgr2=0x7f8e8c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8affd640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c10c170 0x7f8e8c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8affd640 1 -- 172.21.15.1:0/3037542376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8c103660 msgr2=0x7f8e8c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8affd640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8c103660 0x7f8e8c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8affd640 1 -- 172.21.15.1:0/3037542376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c1a68f0 con 0x7f8e8c102d40 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c10c170 0x7f8e8c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e8affd640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c102d40 0x7f8e8c1009a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00c890 tx=0x7f8e7c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e90923640 1 -- 172.21.15.1:0/3037542376 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004220 con 0x7f8e8c102d40 2024-09-16T00:30:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e91925640 1 -- 172.21.15.1:0/3037542376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c1a6be0 con 0x7f8e8c102d40 2024-09-16T00:30:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e90923640 1 -- 172.21.15.1:0/3037542376 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e7c0043c0 con 0x7f8e8c102d40 2024-09-16T00:30:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.160+0000 7f8e90923640 1 -- 172.21.15.1:0/3037542376 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004a00 con 0x7f8e8c102d40 2024-09-16T00:30:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.161+0000 7f8e91925640 1 -- 172.21.15.1:0/3037542376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a7170 con 0x7f8e8c102d40 2024-09-16T00:30:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e90923640 1 -- 172.21.15.1:0/3037542376 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8e7c0079b0 con 0x7f8e8c102d40 2024-09-16T00:30:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e93cfe740 1 Processor -- start 2024-09-16T00:30:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e93cfe740 1 -- start start 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e31090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e31a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5629d0d972f0 con 0x5629d0e32b40 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5629d0d3ecf0 con 0x5629d0e315d0 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5629d0d932b0 con 0x5629d0e3c370 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e8b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e31090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e8b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e31090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60356/0 (socket says 172.21.15.1:60356) 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.163+0000 7f8e8b7fe640 1 -- 172.21.15.1:0/3369315229 learned_addr learned my addr 172.21.15.1:0/3369315229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e90923640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8e740a3240 0x7f8e740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8affd640 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e31a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8b7fe640 1 -- 172.21.15.1:0/3369315229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 msgr2=0x5629d0e3e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8b7fe640 1 -- 172.21.15.1:0/3369315229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 msgr2=0x5629d0e31a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e31a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8b7fe640 1 -- 172.21.15.1:0/3369315229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d0d233a0 con 0x5629d0e32b40 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8e740a3240 0x7f8e740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e90923640 1 -- 172.21.15.1:0/3037542376 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8e7c0cfb20 con 0x7f8e8c102d40 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e31090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8e84010420 tx=0x7f8e840108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ca89cdbb13ac5c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e71ffb640 1 -- 172.21.15.1:0/3369315229 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84023070 con 0x5629d0e32b40 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e71ffb640 1 -- 172.21.15.1:0/3369315229 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8401c470 con 0x5629d0e32b40 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.164+0000 7f8e71ffb640 1 -- 172.21.15.1:0/3369315229 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8401d6a0 con 0x5629d0e32b40 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3369315229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 msgr2=0x5629d0e31090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8e740a3240 0x7f8e740a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8e84020000 tx=0x7f8e8401a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e31090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8e84010420 tx=0x7f8e840108e0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3369315229 shutdown_connections 2024-09-16T00:30:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e31a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/3369315229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e31090 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3369315229 >> 172.21.15.1:0/3369315229 conn(0x5629d0dfa310 msgr2=0x5629d0e2dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3369315229 shutdown_connections 2024-09-16T00:30:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.165+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3369315229 wait complete. 2024-09-16T00:30:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e93cfe740 1 -- start start 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e93cfe740 1 Processor -- start 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e315d0 0x5629d0e3bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 0x5629d0e39070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e3c370 0x5629d0e39800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5629d0e3f070 con 0x5629d0e315d0 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5629d0d52060 con 0x5629d0e3c370 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5629d0e3f1f0 con 0x5629d0e32b40 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e93cfe740 10 client.0 _ll_register_callbacks cb 0x5629d0aa58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e315d0 0x5629d0e3bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e315d0 0x5629d0e3bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60362/0 (socket says 172.21.15.1:60362) 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e8affd640 1 -- 172.21.15.1:0/1827717604 learned_addr learned my addr 172.21.15.1:0/1827717604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.166+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e3c370 0x5629d0e39800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 0x5629d0e39070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e8affd640 1 -- 172.21.15.1:0/1827717604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 msgr2=0x5629d0e39070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e8affd640 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 0x5629d0e39070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e8affd640 1 -- 172.21.15.1:0/1827717604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e3c370 msgr2=0x5629d0e39800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e8affd640 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e3c370 0x5629d0e39800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e8affd640 1 -- 172.21.15.1:0/1827717604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d0b083c0 con 0x5629d0e315d0 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e3c370 0x5629d0e39800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 0x5629d0e39070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e8affd640 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e315d0 0x5629d0e3bdc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0cd1c0 tx=0x7f8e7c0980b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646bc39b2c408b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e6affd640 1 -- 172.21.15.1:0/1827717604 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c01d030 con 0x5629d0e315d0 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e6affd640 1 -- 172.21.15.1:0/1827717604 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8e7c007ca0 con 0x5629d0e315d0 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e93cfe740 1 -- 172.21.15.1:0/1827717604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e8401c470 con 0x5629d0e315d0 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e6affd640 1 -- 172.21.15.1:0/1827717604 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8e7c0a0090 con 0x5629d0e315d0 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e93cfe740 10 client.15720 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e93cfe740 1 -- 172.21.15.1:0/1827717604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629d0e4abe0 con 0x5629d0e315d0 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.167+0000 7f8e6affd640 1 -- 172.21.15.1:0/1827717604 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8401c470 con 0x5629d0e315d0 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.168+0000 7f8e6affd640 1 -- 172.21.15.1:0/1827717604 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8e7c09b130 con 0x5629d0e315d0 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.168+0000 7f8e93cfe740 10 client.15720 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.168+0000 7f8e93cfe740 10 client.15720 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.168+0000 7f8e93cfe740 20 client.15720 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.168+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5629d0e4c520 0x5629d0e4e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.168+0000 7f8e93cfe740 4 client.15720 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.168+0000 7f8e93cfe740 1 -- 172.21.15.1:0/1827717604 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629d0da4860 con 0x5629d0e4c520 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.168+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5629d0e4c520 0x5629d0e4e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.169+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5629d0e4c520 0x5629d0e4e9b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76cd9a0d0d44f0dd server_cookie=c5e6ab853e75feee in_seq=0 out_seq=0 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.169+0000 7f8e6affd640 10 client.15720 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.169+0000 7f8e6affd640 1 -- 172.21.15.1:0/1827717604 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629d0da4860 con 0x5629d0e4c520 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.169+0000 7f8e6affd640 10 client.15720 handle_command_reply: tid=0 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.169+0000 7f8e93cfe740 1 client.15720 shutdown 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.169+0000 7f8e93cfe740 1 -- 172.21.15.1:0/1827717604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e315d0 msgr2=0x5629d0e3bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.169+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e315d0 0x5629d0e3bdc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0cd1c0 tx=0x7f8e7c0980b0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e93cfe740 1 -- 172.21.15.1:0/1827717604 shutdown_connections 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5629d0e4c520 0x5629d0e4e9b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e3c370 0x5629d0e39800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 0x5629d0e39070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/1827717604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e315d0 0x5629d0e3bdc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e93cfe740 1 -- 172.21.15.1:0/1827717604 >> 172.21.15.1:0/1827717604 conn(0x5629d0dfa310 msgr2=0x5629d0e33930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e6affd640 0 client.15720 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:30:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e93cfe740 1 -- 172.21.15.1:0/1827717604 shutdown_connections 2024-09-16T00:30:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e93cfe740 1 -- 172.21.15.1:0/1827717604 wait complete. 2024-09-16T00:30:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.170+0000 7f8e93cfe740 20 client.15720 trim_cache size 0 max 16384 2024-09-16T00:30:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e93cfe740 1 Processor -- start 2024-09-16T00:30:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e93cfe740 1 -- start start 2024-09-16T00:30:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e38930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 0x5629d0e28ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e3c370 0x5629d0e34120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1b3940 con 0x5629d0e3c370 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1b1210 con 0x5629d0e315d0 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1af060 con 0x5629d0e32b40 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e3c370 0x5629d0e34120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e3c370 0x5629d0e34120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60368/0 (socket says 172.21.15.1:60368) 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8b7fe640 1 -- 172.21.15.1:0/3737256370 learned_addr learned my addr 172.21.15.1:0/3737256370 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8affd640 1 --2- 172.21.15.1:0/3737256370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e38930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8b7fe640 1 -- 172.21.15.1:0/3737256370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 msgr2=0x5629d0e28ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3737256370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 0x5629d0e28ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8b7fe640 1 -- 172.21.15.1:0/3737256370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 msgr2=0x5629d0e38930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3737256370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e38930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8b7fe640 1 -- 172.21.15.1:0/3737256370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d0e3ead0 con 0x5629d0e3c370 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.183+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/3737256370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e3c370 0x5629d0e34120 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009730 tx=0x7f8e84027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395c9612758f96fc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e717fa640 1 -- 172.21.15.1:0/3737256370 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84023070 con 0x5629d0e3c370 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e717fa640 1 -- 172.21.15.1:0/3737256370 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e840136c0 con 0x5629d0e3c370 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e717fa640 1 -- 172.21.15.1:0/3737256370 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84027ad0 con 0x5629d0e3c370 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3737256370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e3c370 msgr2=0x5629d0e34120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/3737256370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e3c370 0x5629d0e34120 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009730 tx=0x7f8e84027000 comp rx=0 tx=0).stop 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3737256370 shutdown_connections 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/3737256370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e3c370 0x5629d0e34120 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/3737256370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e32b40 0x5629d0e28ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/3737256370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e38930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3737256370 >> 172.21.15.1:0/3737256370 conn(0x5629d0dfa310 msgr2=0x5629d0e4e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3737256370 shutdown_connections 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.184+0000 7f8e93cfe740 1 -- 172.21.15.1:0/3737256370 wait complete. 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e93cfe740 1 -- start start 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e93cfe740 1 Processor -- start 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e3f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e3f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e93cfe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1af6d0 con 0x5629d0e32b40 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1b0b40 con 0x5629d0e315d0 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e93cfe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1afda0 con 0x5629d0e3c370 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e93cfe740 10 client.0 _ll_register_callbacks cb 0x7f8e7c004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e8a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e3f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e8a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e3f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60376/0 (socket says 172.21.15.1:60376) 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.185+0000 7f8e8a7fc640 1 -- 172.21.15.1:0/2149235578 learned_addr learned my addr 172.21.15.1:0/2149235578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8affd640 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e3f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8a7fc640 1 -- 172.21.15.1:0/2149235578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 msgr2=0x5629d0e3fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8a7fc640 1 -- 172.21.15.1:0/2149235578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 msgr2=0x5629d0e3f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e3f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8a7fc640 1 -- 172.21.15.1:0/2149235578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d0eb5240 con 0x5629d0e32b40 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8affd640 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e3f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8b7fe640 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e8a7fc640 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e3f980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8e80032fd0 tx=0x7f8e80031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efed9f78734e8a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e6affd640 1 -- 172.21.15.1:0/2149235578 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80035040 con 0x5629d0e32b40 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e6affd640 1 -- 172.21.15.1:0/2149235578 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8e80004390 con 0x5629d0e32b40 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e93cfe740 1 -- 172.21.15.1:0/2149235578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629d0d9ed70 con 0x5629d0e32b40 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e6affd640 1 -- 172.21.15.1:0/2149235578 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8e80050050 con 0x5629d0e32b40 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e93cfe740 10 client.15726 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.186+0000 7f8e93cfe740 1 -- 172.21.15.1:0/2149235578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629d0eb6480 con 0x5629d0e32b40 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.187+0000 7f8e6affd640 1 -- 172.21.15.1:0/2149235578 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5629d0d9ed70 con 0x5629d0e32b40 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.187+0000 7f8e6affd640 1 -- 172.21.15.1:0/2149235578 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8e80004cc0 con 0x5629d0e32b40 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.187+0000 7f8e93cfe740 10 client.15726 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.187+0000 7f8e93cfe740 10 client.15726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.187+0000 7f8e93cfe740 20 client.15726 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.187+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5629d0eb6c30 0x5629d0eb9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.187+0000 7f8e93cfe740 4 client.15726 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.187+0000 7f8e93cfe740 1 -- 172.21.15.1:0/2149235578 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629d0d1b170 con 0x5629d0eb6c30 2024-09-16T00:30:27.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.187+0000 7f8e8affd640 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5629d0eb6c30 0x5629d0eb9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e6affd640 10 client.15726 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:30:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e8affd640 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5629d0eb6c30 0x5629d0eb9050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e72e58718ab108e server_cookie=f0f71ba21a95ac22 in_seq=0 out_seq=0 2024-09-16T00:30:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e6affd640 1 -- 172.21.15.1:0/2149235578 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629d0d1b170 con 0x5629d0eb6c30 2024-09-16T00:30:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e6affd640 10 client.15726 handle_command_reply: tid=0 2024-09-16T00:30:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 client.15726 shutdown 2024-09-16T00:30:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 -- 172.21.15.1:0/2149235578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 msgr2=0x5629d0e3f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e3f980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8e80032fd0 tx=0x7f8e80031490 comp rx=0 tx=0).stop 2024-09-16T00:30:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 -- 172.21.15.1:0/2149235578 shutdown_connections 2024-09-16T00:30:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629d0e3c370 0x5629d0e3fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5629d0e32b40 0x5629d0e3f980 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5629d0eb6c30 0x5629d0eb9050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 --2- 172.21.15.1:0/2149235578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629d0e315d0 0x5629d0e3f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 -- 172.21.15.1:0/2149235578 >> 172.21.15.1:0/2149235578 conn(0x5629d0dfa310 msgr2=0x5629d0e4e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:27.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e6affd640 0 client.15726 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:30:27.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 -- 172.21.15.1:0/2149235578 shutdown_connections 2024-09-16T00:30:27.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 1 -- 172.21.15.1:0/2149235578 wait complete. 2024-09-16T00:30:27.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.188+0000 7f8e93cfe740 20 client.15726 trim_cache size 0 max 16384 2024-09-16T00:30:27.195 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:30:27.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 -- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8e740a3240 msgr2=0x7f8e740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8e740a3240 0x7f8e740a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8e84020000 tx=0x7f8e8401a000 comp rx=0 tx=0).stop 2024-09-16T00:30:27.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 -- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c102d40 msgr2=0x7f8e8c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c102d40 0x7f8e8c1009a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00c890 tx=0x7f8e7c00cd50 comp rx=0 tx=0).stop 2024-09-16T00:30:27.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 -- 172.21.15.1:0/3037542376 shutdown_connections 2024-09-16T00:30:27.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8e740a3240 0x7f8e740a5700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c10c170 0x7f8e8c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8c103660 0x7f8e8c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 --2- 172.21.15.1:0/3037542376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e8c102d40 0x7f8e8c1009a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 -- 172.21.15.1:0/3037542376 >> 172.21.15.1:0/3037542376 conn(0x7f8e8c0fc7a0 msgr2=0x7f8e8c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:27.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 -- 172.21.15.1:0/3037542376 shutdown_connections 2024-09-16T00:30:27.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:27.189+0000 7f8e91925640 1 -- 172.21.15.1:0/3037542376 wait complete. 2024-09-16T00:30:28.111 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:30:28.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:28 smithi019 ceph-mon[35186]: pgmap v412: 129 pgs: 129 active+clean; 845 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:28.369 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:28.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:28 smithi001 ceph-mon[30934]: pgmap v412: 129 pgs: 129 active+clean; 845 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:28.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:28 smithi153 ceph-mon[35538]: pgmap v412: 129 pgs: 129 active+clean; 845 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:29.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.195+0000 7f0a3e359640 1 -- 172.21.15.1:0/3373436933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38101de0 msgr2=0x7f0a381021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.195+0000 7f0a3e359640 1 --2- 172.21.15.1:0/3373436933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38101de0 0x7f0a381021c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009900 tx=0x7f0a2802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.195+0000 7f0a3e359640 1 -- 172.21.15.1:0/3373436933 shutdown_connections 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.195+0000 7f0a3e359640 1 --2- 172.21.15.1:0/3373436933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a38106f10 0x7f0a3810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.195+0000 7f0a3e359640 1 --2- 172.21.15.1:0/3373436933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38102790 0x7f0a381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.195+0000 7f0a3e359640 1 --2- 172.21.15.1:0/3373436933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38101de0 0x7f0a381021c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.195+0000 7f0a3e359640 1 -- 172.21.15.1:0/3373436933 >> 172.21.15.1:0/3373436933 conn(0x7f0a380fd8c0 msgr2=0x7f0a380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.195+0000 7f0a3e359640 1 -- 172.21.15.1:0/3373436933 shutdown_connections 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.195+0000 7f0a3e359640 1 -- 172.21.15.1:0/3373436933 wait complete. 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.196+0000 7f0a3e359640 1 Processor -- start 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.196+0000 7f0a3e359640 1 -- start start 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.196+0000 7f0a3e359640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38101de0 0x7f0a38198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.196+0000 7f0a3e359640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38102790 0x7f0a381988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.196+0000 7f0a3e359640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a38106f10 0x7f0a3819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.196+0000 7f0a3e359640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a38111f30 con 0x7f0a38102790 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.196+0000 7f0a3e359640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a38111db0 con 0x7f0a38106f10 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.196+0000 7f0a3e359640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381120b0 con 0x7f0a38101de0 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38102790 0x7f0a381988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38102790 0x7f0a381988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60402/0 (socket says 172.21.15.1:60402) 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a377fe640 1 -- 172.21.15.1:0/1112721624 learned_addr learned my addr 172.21.15.1:0/1112721624 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a38106f10 0x7f0a3819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a37fff640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38101de0 0x7f0a38198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a377fe640 1 -- 172.21.15.1:0/1112721624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38101de0 msgr2=0x7f0a38198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a377fe640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38101de0 0x7f0a38198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a377fe640 1 -- 172.21.15.1:0/1112721624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a38106f10 msgr2=0x7f0a3819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a377fe640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a38106f10 0x7f0a3819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a377fe640 1 -- 172.21.15.1:0/1112721624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3819d340 con 0x7f0a38102790 2024-09-16T00:30:29.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a377fe640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38102790 0x7f0a381988d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c920 tx=0x7f0a2c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a357fa640 1 -- 172.21.15.1:0/1112721624 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c007c70 con 0x7f0a38102790 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a357fa640 1 -- 172.21.15.1:0/1112721624 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c007e10 con 0x7f0a38102790 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a3e359640 1 -- 172.21.15.1:0/1112721624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3819d630 con 0x7f0a38102790 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a357fa640 1 -- 172.21.15.1:0/1112721624 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c018890 con 0x7f0a38102790 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.197+0000 7f0a3e359640 1 -- 172.21.15.1:0/1112721624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a381a4f10 con 0x7f0a38102790 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.200+0000 7f0a357fa640 1 -- 172.21.15.1:0/1112721624 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f0a2c018a70 con 0x7f0a38102790 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.200+0000 7f0a40732740 1 Processor -- start 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.200+0000 7f0a357fa640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a200a3240 0x7f0a200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.200+0000 7f0a40732740 1 -- start start 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.200+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a200a3240 0x7f0a200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.200+0000 7f0a357fa640 1 -- 172.21.15.1:0/1112721624 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0a2c0cfdb0 con 0x7f0a38102790 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.200+0000 7f0a40732740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8aab40 0x55b16a8a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a40732740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8a95d0 0x55b16a8a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a40732740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a40732740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b16a80f2f0 con 0x55b16a8b8780 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a40732740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b16a7b6cf0 con 0x55b16a8aab40 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a40732740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b16a80b2b0 con 0x55b16a8a95d0 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60414/0 (socket says 172.21.15.1:60414) 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 -- 172.21.15.1:0/1278684050 learned_addr learned my addr 172.21.15.1:0/1278684050 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a37fff640 1 --2- 172.21.15.1:0/1278684050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8aab40 0x55b16a8a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a377fe640 1 --2- 172.21.15.1:0/1278684050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8a95d0 0x55b16a8a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a200a3240 0x7f0a200a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a30014820 tx=0x7f0a30014c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 -- 172.21.15.1:0/1278684050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8a95d0 msgr2=0x55b16a8a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/1278684050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8a95d0 0x55b16a8a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 -- 172.21.15.1:0/1278684050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8aab40 msgr2=0x55b16a8a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/1278684050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8aab40 0x55b16a8a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 -- 172.21.15.1:0/1278684050 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b16a79b3a0 con 0x55b16a8b8780 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/1278684050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bab70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0a30012ac0 tx=0x7f0a3001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367d2bd1cd9e6c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.201+0000 7f0a1e7fc640 1 -- 172.21.15.1:0/1278684050 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30003b70 con 0x55b16a8b8780 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a1e7fc640 1 -- 172.21.15.1:0/1278684050 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3001ed90 con 0x55b16a8b8780 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a1e7fc640 1 -- 172.21.15.1:0/1278684050 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30020440 con 0x55b16a8b8780 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a40732740 1 -- 172.21.15.1:0/1278684050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 msgr2=0x55b16a8bab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a40732740 1 --2- 172.21.15.1:0/1278684050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bab70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0a30012ac0 tx=0x7f0a3001e000 comp rx=0 tx=0).stop 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a40732740 1 -- 172.21.15.1:0/1278684050 shutdown_connections 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a40732740 1 --2- 172.21.15.1:0/1278684050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bab70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a40732740 1 --2- 172.21.15.1:0/1278684050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8a95d0 0x55b16a8a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a40732740 1 --2- 172.21.15.1:0/1278684050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8aab40 0x55b16a8a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a40732740 1 -- 172.21.15.1:0/1278684050 >> 172.21.15.1:0/1278684050 conn(0x55b16a872310 msgr2=0x55b16a8a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a40732740 1 -- 172.21.15.1:0/1278684050 shutdown_connections 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.202+0000 7f0a40732740 1 -- 172.21.15.1:0/1278684050 wait complete. 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a40732740 1 -- start start 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a40732740 1 Processor -- start 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a40732740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 0x55b16a8b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a40732740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8aab40 0x55b16a8b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a40732740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8b8780 0x55b16a8b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a40732740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b16a8bb420 con 0x55b16a8aab40 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a40732740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b16a7ca060 con 0x55b16a8a95d0 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a40732740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b16a8bb5a0 con 0x55b16a8b8780 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a40732740 10 client.0 _ll_register_callbacks cb 0x55b16a51d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8aab40 0x55b16a8b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8aab40 0x55b16a8b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60430/0 (socket says 172.21.15.1:60430) 2024-09-16T00:30:29.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a377fe640 1 -- 172.21.15.1:0/2259728430 learned_addr learned my addr 172.21.15.1:0/2259728430 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:29.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.203+0000 7f0a37fff640 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 0x55b16a8b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a377fe640 1 -- 172.21.15.1:0/2259728430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8b8780 msgr2=0x55b16a8b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:29.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a377fe640 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8b8780 0x55b16a8b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a377fe640 1 -- 172.21.15.1:0/2259728430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 msgr2=0x55b16a8b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:29.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a377fe640 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 0x55b16a8b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a377fe640 1 -- 172.21.15.1:0/2259728430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b16a5803c0 con 0x55b16a8aab40 2024-09-16T00:30:29.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a377fe640 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8aab40 0x55b16a8b53c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0cd500 tx=0x7f0a2c09c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c382de6c7ba8650 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:29.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a177fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:29.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a177fe640 1 -- 172.21.15.1:0/2259728430 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0b4070 con 0x55b16a8aab40 2024-09-16T00:30:29.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a40732740 1 -- 172.21.15.1:0/2259728430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3001ed90 con 0x55b16a8aab40 2024-09-16T00:30:29.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a177fe640 1 -- 172.21.15.1:0/2259728430 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0a2c018d60 con 0x55b16a8aab40 2024-09-16T00:30:29.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a177fe640 1 -- 172.21.15.1:0/2259728430 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a2c0a4c40 con 0x55b16a8aab40 2024-09-16T00:30:29.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a40732740 10 client.15738 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:29.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a40732740 1 -- 172.21.15.1:0/2259728430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b16a8c6eb0 con 0x55b16a8aab40 2024-09-16T00:30:29.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a177fe640 1 -- 172.21.15.1:0/2259728430 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3001ed90 con 0x55b16a8aab40 2024-09-16T00:30:29.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a177fe640 1 -- 172.21.15.1:0/2259728430 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0a2c096b60 con 0x55b16a8aab40 2024-09-16T00:30:29.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a40732740 10 client.15738 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:29.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a40732740 10 client.15738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a40732740 20 client.15738 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.204+0000 7f0a40732740 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b16a8c82c0 0x55b16a8ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.205+0000 7f0a40732740 4 client.15738 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.205+0000 7f0a40732740 1 -- 172.21.15.1:0/2259728430 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b16a81c860 con 0x55b16a8c82c0 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.205+0000 7f0a37fff640 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b16a8c82c0 0x55b16a8ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.205+0000 7f0a177fe640 10 client.15738 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.205+0000 7f0a37fff640 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b16a8c82c0 0x55b16a8ca730 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ecd8a21f51bb1da server_cookie=edbe56229f1c5379 in_seq=0 out_seq=0 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a177fe640 1 -- 172.21.15.1:0/2259728430 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b16a81c860 con 0x55b16a8c82c0 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a177fe640 10 client.15738 handle_command_reply: tid=0 2024-09-16T00:30:29.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 client.15738 shutdown 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 -- 172.21.15.1:0/2259728430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8aab40 msgr2=0x55b16a8b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8aab40 0x55b16a8b53c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0cd500 tx=0x7f0a2c09c7a0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 -- 172.21.15.1:0/2259728430 shutdown_connections 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8b8780 0x55b16a8b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b16a8c82c0 0x55b16a8ca730 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8aab40 0x55b16a8b53c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 --2- 172.21.15.1:0/2259728430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 0x55b16a8b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a177fe640 0 client.15738 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 -- 172.21.15.1:0/2259728430 >> 172.21.15.1:0/2259728430 conn(0x55b16a872310 msgr2=0x55b16a8ab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 -- 172.21.15.1:0/2259728430 shutdown_connections 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.206+0000 7f0a40732740 1 -- 172.21.15.1:0/2259728430 wait complete. 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.207+0000 7f0a40732740 20 client.15738 trim_cache size 0 max 16384 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.219+0000 7f0a40732740 1 Processor -- start 2024-09-16T00:30:29.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.219+0000 7f0a40732740 1 -- start start 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a40732740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8a95d0 0x55b16a8c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a40732740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 0x55b16a8a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a40732740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8b8780 0x55b16a8b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a40732740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381af3f0 con 0x55b16a8a95d0 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a40732740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381accc0 con 0x55b16a8b8780 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a40732740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381aab10 con 0x55b16a8aab40 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a37fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8a95d0 0x55b16a8c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a37fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8a95d0 0x55b16a8c46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60432/0 (socket says 172.21.15.1:60432) 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a37fff640 1 -- 172.21.15.1:0/1814520779 learned_addr learned my addr 172.21.15.1:0/1814520779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a377fe640 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 0x55b16a8a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a37fff640 1 -- 172.21.15.1:0/1814520779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 msgr2=0x55b16a8a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a37fff640 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 0x55b16a8a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8b8780 0x55b16a8b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a37fff640 1 -- 172.21.15.1:0/1814520779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8b8780 msgr2=0x55b16a8b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a37fff640 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8b8780 0x55b16a8b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a37fff640 1 -- 172.21.15.1:0/1814520779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b16a8a6960 con 0x55b16a8a95d0 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8b8780 0x55b16a8b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a37fff640 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8a95d0 0x55b16a8c46c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0a2802a910 tx=0x7f0a28031b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5d78377b8b841e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a1dffb640 1 -- 172.21.15.1:0/1814520779 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2804d030 con 0x55b16a8a95d0 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.220+0000 7f0a1dffb640 1 -- 172.21.15.1:0/1814520779 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a28005020 con 0x55b16a8a95d0 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a1dffb640 1 -- 172.21.15.1:0/1814520779 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28035040 con 0x55b16a8a95d0 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 -- 172.21.15.1:0/1814520779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8a95d0 msgr2=0x55b16a8c46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8a95d0 0x55b16a8c46c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0a2802a910 tx=0x7f0a28031b40 comp rx=0 tx=0).stop 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 -- 172.21.15.1:0/1814520779 shutdown_connections 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8b8780 0x55b16a8b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 0x55b16a8a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 --2- 172.21.15.1:0/1814520779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8a95d0 0x55b16a8c46c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 -- 172.21.15.1:0/1814520779 >> 172.21.15.1:0/1814520779 conn(0x55b16a872310 msgr2=0x55b16a8b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:29.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 -- 172.21.15.1:0/1814520779 shutdown_connections 2024-09-16T00:30:29.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 -- 172.21.15.1:0/1814520779 wait complete. 2024-09-16T00:30:29.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.221+0000 7f0a40732740 1 -- start start 2024-09-16T00:30:29.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a40732740 1 Processor -- start 2024-09-16T00:30:29.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a40732740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 0x55b16a8bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a40732740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 0x55b16a8bc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a40732740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a40732740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381ab180 con 0x55b16a8b8780 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a40732740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381ac5f0 con 0x55b16a8a95d0 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a40732740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381ab850 con 0x55b16a8aab40 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a40732740 10 client.0 _ll_register_callbacks cb 0x7f0a2c09bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a3c8f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a3c8f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60436/0 (socket says 172.21.15.1:60436) 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a3c8f6640 1 -- 172.21.15.1:0/702516780 learned_addr learned my addr 172.21.15.1:0/702516780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a377fe640 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 0x55b16a8bc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a37fff640 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 0x55b16a8bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a3c8f6640 1 -- 172.21.15.1:0/702516780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 msgr2=0x55b16a8bc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 0x55b16a8bc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a3c8f6640 1 -- 172.21.15.1:0/702516780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 msgr2=0x55b16a8bc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 0x55b16a8bc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.222+0000 7f0a3c8f6640 1 -- 172.21.15.1:0/702516780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b16a8ac120 con 0x55b16a8b8780 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a3c8f6640 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bcf30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0a30024000 tx=0x7f0a30004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363f1b1df91eeb9d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a177fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a177fe640 1 -- 172.21.15.1:0/702516780 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3001ecc0 con 0x55b16a8b8780 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a177fe640 1 -- 172.21.15.1:0/702516780 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0a30003970 con 0x55b16a8b8780 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a40732740 1 -- 172.21.15.1:0/702516780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a28005020 con 0x55b16a8b8780 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a177fe640 1 -- 172.21.15.1:0/702516780 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a3001ee60 con 0x55b16a8b8780 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a40732740 10 client.15744 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a40732740 1 -- 172.21.15.1:0/702516780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b16a932f70 con 0x55b16a8b8780 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a177fe640 1 -- 172.21.15.1:0/702516780 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a28005020 con 0x55b16a8b8780 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a177fe640 1 -- 172.21.15.1:0/702516780 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0a3000f040 con 0x55b16a8b8780 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a40732740 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.223+0000 7f0a40732740 10 client.15744 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a40732740 20 client.15744 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a40732740 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b16a933490 0x55b16a935880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a40732740 4 client.15744 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a40732740 1 -- 172.21.15.1:0/702516780 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b16a816d70 con 0x55b16a933490 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a37fff640 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b16a933490 0x55b16a935880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a37fff640 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b16a933490 0x55b16a935880 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=978d2eb38db7c162 server_cookie=7275cfaddf55cff3 in_seq=0 out_seq=0 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a177fe640 10 client.15744 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a177fe640 1 -- 172.21.15.1:0/702516780 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b16a816d70 con 0x55b16a933490 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a177fe640 10 client.15744 handle_command_reply: tid=0 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a40732740 1 client.15744 shutdown 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a40732740 1 -- 172.21.15.1:0/702516780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 msgr2=0x55b16a8bcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.224+0000 7f0a40732740 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bcf30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0a30024000 tx=0x7f0a30004e50 comp rx=0 tx=0).stop 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a40732740 1 -- 172.21.15.1:0/702516780 shutdown_connections 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a40732740 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b16a933490 0x55b16a935880 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a40732740 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b16a8b8780 0x55b16a8bcf30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a40732740 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b16a8aab40 0x55b16a8bc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a40732740 1 --2- 172.21.15.1:0/702516780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16a8a95d0 0x55b16a8bc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a177fe640 0 client.15744 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:30:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a40732740 1 -- 172.21.15.1:0/702516780 >> 172.21.15.1:0/702516780 conn(0x55b16a872310 msgr2=0x55b16a8b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a40732740 1 -- 172.21.15.1:0/702516780 shutdown_connections 2024-09-16T00:30:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a40732740 1 -- 172.21.15.1:0/702516780 wait complete. 2024-09-16T00:30:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a40732740 20 client.15744 trim_cache size 0 max 16384 2024-09-16T00:30:29.241 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 -- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a200a3240 msgr2=0x7f0a200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a200a3240 0x7f0a200a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a30014820 tx=0x7f0a30014c20 comp rx=0 tx=0).stop 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 -- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38102790 msgr2=0x7f0a381988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38102790 0x7f0a381988d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c920 tx=0x7f0a2c00cde0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 -- 172.21.15.1:0/1112721624 shutdown_connections 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a38106f10 0x7f0a3819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a38102790 0x7f0a381988d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0a200a3240 0x7f0a200a5700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 --2- 172.21.15.1:0/1112721624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38101de0 0x7f0a38198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.225+0000 7f0a3e359640 1 -- 172.21.15.1:0/1112721624 >> 172.21.15.1:0/1112721624 conn(0x7f0a380fd8c0 msgr2=0x7f0a380fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.226+0000 7f0a3e359640 1 -- 172.21.15.1:0/1112721624 shutdown_connections 2024-09-16T00:30:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:29.226+0000 7f0a3e359640 1 -- 172.21.15.1:0/1112721624 wait complete. 2024-09-16T00:30:30.065 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:30:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:30.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:30 smithi019 ceph-mon[35186]: pgmap v413: 129 pgs: 129 active+clean; 906 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:30:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:30 smithi001 ceph-mon[30934]: pgmap v413: 129 pgs: 129 active+clean; 906 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:30:30.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:30 smithi153 ceph-mon[35538]: pgmap v413: 129 pgs: 129 active+clean; 906 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:30:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.838+0000 7efedcc5c640 1 -- 172.21.15.1:0/412464048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8100a80 msgr2=0x7efed8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.838+0000 7efedcc5c640 1 --2- 172.21.15.1:0/412464048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8100a80 0x7efed8100ee0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efecc009930 tx=0x7efecc02a420 comp rx=0 tx=0).stop 2024-09-16T00:30:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.838+0000 7efedcc5c640 1 -- 172.21.15.1:0/412464048 shutdown_connections 2024-09-16T00:30:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.838+0000 7efedcc5c640 1 --2- 172.21.15.1:0/412464048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed8101420 0x7efed810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.838+0000 7efedcc5c640 1 --2- 172.21.15.1:0/412464048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8100a80 0x7efed8100ee0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.838+0000 7efedcc5c640 1 --2- 172.21.15.1:0/412464048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efed8106a80 0x7efed8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.838+0000 7efedcc5c640 1 -- 172.21.15.1:0/412464048 >> 172.21.15.1:0/412464048 conn(0x7efed80fc7a0 msgr2=0x7efed80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.838+0000 7efedcc5c640 1 -- 172.21.15.1:0/412464048 shutdown_connections 2024-09-16T00:30:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.838+0000 7efedcc5c640 1 -- 172.21.15.1:0/412464048 wait complete. 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.839+0000 7efedcc5c640 1 Processor -- start 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.839+0000 7efedcc5c640 1 -- start start 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.839+0000 7efedcc5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed8100a80 0x7efed8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.839+0000 7efedcc5c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8101420 0x7efed8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.839+0000 7efedcc5c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efed8106a80 0x7efed819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.839+0000 7efedcc5c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efed8110e80 con 0x7efed8101420 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.839+0000 7efedcc5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efed8110d00 con 0x7efed8100a80 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.839+0000 7efedcc5c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efed8111000 con 0x7efed8106a80 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8101420 0x7efed8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8101420 0x7efed8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53656/0 (socket says 172.21.15.1:53656) 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed5d9b640 1 -- 172.21.15.1:0/2420863318 learned_addr learned my addr 172.21.15.1:0/2420863318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed659c640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed8100a80 0x7efed8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed6d9d640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efed8106a80 0x7efed819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed5d9b640 1 -- 172.21.15.1:0/2420863318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efed8106a80 msgr2=0x7efed819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed5d9b640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efed8106a80 0x7efed819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed5d9b640 1 -- 172.21.15.1:0/2420863318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed8100a80 msgr2=0x7efed8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed5d9b640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed8100a80 0x7efed8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed5d9b640 1 -- 172.21.15.1:0/2420863318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed819d3e0 con 0x7efed8101420 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efed5d9b640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8101420 0x7efed8198970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efecc009230 tx=0x7efecc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efec77fe640 1 -- 172.21.15.1:0/2420863318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc03f030 con 0x7efed8101420 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efec77fe640 1 -- 172.21.15.1:0/2420863318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efecc003ad0 con 0x7efed8101420 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efedcc5c640 1 -- 172.21.15.1:0/2420863318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed819d670 con 0x7efed8101420 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.840+0000 7efec77fe640 1 -- 172.21.15.1:0/2420863318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc035070 con 0x7efed8101420 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.841+0000 7efedcc5c640 1 -- 172.21.15.1:0/2420863318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed81a4fb0 con 0x7efed8101420 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efec77fe640 1 -- 172.21.15.1:0/2420863318 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7efecc042030 con 0x7efed8101420 2024-09-16T00:30:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efedf035740 1 Processor -- start 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efedf035740 1 -- start start 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efedf035740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed5063f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efedf035740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed5063fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efedf035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed5064c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efedf035740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ed505a52f0 con 0x55ed5063f5d0 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efedf035740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ed5054ccf0 con 0x55ed50640b40 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efedf035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed505a12b0 con 0x55ed5064a370 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efed659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed5063fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.843+0000 7efed659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed5063fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53672/0 (socket says 172.21.15.1:53672) 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed659c640 1 -- 172.21.15.1:0/3533534081 learned_addr learned my addr 172.21.15.1:0/3533534081 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed6d9d640 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed5063f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed5d9b640 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed5064c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed659c640 1 -- 172.21.15.1:0/3533534081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 msgr2=0x55ed5064c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efec77fe640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efec00a32c0 0x7efec00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed659c640 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed5064c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed659c640 1 -- 172.21.15.1:0/3533534081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 msgr2=0x55ed5063f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed659c640 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed5063f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed659c640 1 -- 172.21.15.1:0/3533534081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed505313a0 con 0x55ed5063f5d0 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed6d9d640 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed5063f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed5d9b640 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed5064c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed6d9d640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efec00a32c0 0x7efec00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efec77fe640 1 -- 172.21.15.1:0/2420863318 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efecc0f00f0 con 0x7efed8101420 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efed659c640 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed5063fa10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efec8010370 tx=0x7efec8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f01f3e1765d4524 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efec4ff9640 1 -- 172.21.15.1:0/3533534081 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8016070 con 0x55ed5063f5d0 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efec4ff9640 1 -- 172.21.15.1:0/3533534081 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec8004d20 con 0x55ed5063f5d0 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.844+0000 7efec4ff9640 1 -- 172.21.15.1:0/3533534081 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80054c0 con 0x55ed5063f5d0 2024-09-16T00:30:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efedf035740 1 -- 172.21.15.1:0/3533534081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 msgr2=0x55ed5063fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efed6d9d640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efec00a32c0 0x7efec00a5780 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efed000b6c0 tx=0x7efed00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efedf035740 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed5063fa10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efec8010370 tx=0x7efec8010830 comp rx=0 tx=0).stop 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efedf035740 1 -- 172.21.15.1:0/3533534081 shutdown_connections 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efedf035740 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed5064c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efedf035740 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed5063fa10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efedf035740 1 --2- 172.21.15.1:0/3533534081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed5063f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efedf035740 1 -- 172.21.15.1:0/3533534081 >> 172.21.15.1:0/3533534081 conn(0x55ed50608310 msgr2=0x55ed5063bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efedf035740 1 -- 172.21.15.1:0/3533534081 shutdown_connections 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.845+0000 7efedf035740 1 -- 172.21.15.1:0/3533534081 wait complete. 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efedf035740 1 -- start start 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efedf035740 1 Processor -- start 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efedf035740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5063f5d0 0x55ed50649c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efedf035740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed50640b40 0x55ed50646f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efedf035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed506474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efedf035740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ed5064d070 con 0x55ed50640b40 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efedf035740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ed50560060 con 0x55ed5063f5d0 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efedf035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed5064d1f0 con 0x55ed5064a370 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efedf035740 10 client.0 _ll_register_callbacks cb 0x55ed502b38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed50640b40 0x55ed50646f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed50640b40 0x55ed50646f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53678/0 (socket says 172.21.15.1:53678) 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed5d9b640 1 -- 172.21.15.1:0/3064579013 learned_addr learned my addr 172.21.15.1:0/3064579013 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed6d9d640 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed506474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed659c640 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5063f5d0 0x55ed50649c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed5d9b640 1 -- 172.21.15.1:0/3064579013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 msgr2=0x55ed506474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed5d9b640 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed506474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed5d9b640 1 -- 172.21.15.1:0/3064579013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5063f5d0 msgr2=0x55ed50649c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed5d9b640 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5063f5d0 0x55ed50649c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed5d9b640 1 -- 172.21.15.1:0/3064579013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed503163c0 con 0x55ed50640b40 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.846+0000 7efed659c640 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5063f5d0 0x55ed50649c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efed5d9b640 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed50640b40 0x55ed50646f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efecc0bc0c0 tx=0x7efecc0bbf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6309c0211e241a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efebdffb640 1 -- 172.21.15.1:0/3064579013 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc049070 con 0x55ed50640b40 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efebdffb640 1 -- 172.21.15.1:0/3064579013 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efecc03b460 con 0x55ed50640b40 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efedf035740 1 -- 172.21.15.1:0/3064579013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec8004d20 con 0x55ed50640b40 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efebdffb640 1 -- 172.21.15.1:0/3064579013 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efecc0c06c0 con 0x55ed50640b40 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efedf035740 10 client.15756 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efedf035740 1 -- 172.21.15.1:0/3064579013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed506589f0 con 0x55ed50640b40 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efebdffb640 1 -- 172.21.15.1:0/3064579013 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec8004d20 con 0x55ed50640b40 2024-09-16T00:30:30.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efebdffb640 1 -- 172.21.15.1:0/3064579013 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efecc031bd0 con 0x55ed50640b40 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efedf035740 10 client.15756 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efedf035740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efedf035740 20 client.15756 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efedf035740 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed50659e50 0x55ed5065c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efedf035740 4 client.15756 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.847+0000 7efedf035740 1 -- 172.21.15.1:0/3064579013 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed505b2860 con 0x55ed50659e50 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.848+0000 7efed659c640 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed50659e50 0x55ed5065c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.848+0000 7efebdffb640 10 client.15756 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.848+0000 7efed659c640 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed50659e50 0x55ed5065c2a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867f88212388920b server_cookie=761eeb3bf75174c0 in_seq=0 out_seq=0 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efebdffb640 1 -- 172.21.15.1:0/3064579013 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed505b2860 con 0x55ed50659e50 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efebdffb640 10 client.15756 handle_command_reply: tid=0 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 client.15756 shutdown 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 -- 172.21.15.1:0/3064579013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed50640b40 msgr2=0x55ed50646f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed50640b40 0x55ed50646f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efecc0bc0c0 tx=0x7efecc0bbf00 comp rx=0 tx=0).stop 2024-09-16T00:30:30.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 -- 172.21.15.1:0/3064579013 shutdown_connections 2024-09-16T00:30:30.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed50659e50 0x55ed5065c2a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed506474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed50640b40 0x55ed50646f90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 --2- 172.21.15.1:0/3064579013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5063f5d0 0x55ed50649c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 -- 172.21.15.1:0/3064579013 >> 172.21.15.1:0/3064579013 conn(0x55ed50608310 msgr2=0x55ed50641930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:30.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efebdffb640 0 client.15756 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:30.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 -- 172.21.15.1:0/3064579013 shutdown_connections 2024-09-16T00:30:30.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 1 -- 172.21.15.1:0/3064579013 wait complete. 2024-09-16T00:30:30.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.849+0000 7efedf035740 20 client.15756 trim_cache size 0 max 16384 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efedf035740 1 Processor -- start 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efedf035740 1 -- start start 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efedf035740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed50656310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efedf035740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed5063ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efedf035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed5064b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efedf035740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b1630 con 0x55ed5063f5d0 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efedf035740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efed81aef00 con 0x55ed50640b40 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efedf035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efed81acd50 con 0x55ed5064a370 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efed659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed50656310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efed659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed50656310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53692/0 (socket says 172.21.15.1:53692) 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.862+0000 7efed659c640 1 -- 172.21.15.1:0/4196072882 learned_addr learned my addr 172.21.15.1:0/4196072882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efed5d9b640 1 --2- 172.21.15.1:0/4196072882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed5063ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efed6d9d640 1 --2- 172.21.15.1:0/4196072882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed5064b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efed659c640 1 -- 172.21.15.1:0/4196072882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 msgr2=0x55ed5064b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efed659c640 1 --2- 172.21.15.1:0/4196072882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed5064b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efed659c640 1 -- 172.21.15.1:0/4196072882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 msgr2=0x55ed5063ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efed659c640 1 --2- 172.21.15.1:0/4196072882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed5063ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efed659c640 1 -- 172.21.15.1:0/4196072882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed506454e0 con 0x55ed5063f5d0 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efed659c640 1 --2- 172.21.15.1:0/4196072882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed50656310 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efec8010370 tx=0x7efec8003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154736d917a73d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:30.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efebffff640 1 -- 172.21.15.1:0/4196072882 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8016040 con 0x55ed5063f5d0 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efebffff640 1 -- 172.21.15.1:0/4196072882 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec8003df0 con 0x55ed5063f5d0 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.863+0000 7efebffff640 1 -- 172.21.15.1:0/4196072882 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8005670 con 0x55ed5063f5d0 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 -- 172.21.15.1:0/4196072882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 msgr2=0x55ed50656310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 --2- 172.21.15.1:0/4196072882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed50656310 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efec8010370 tx=0x7efec8003b00 comp rx=0 tx=0).stop 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 -- 172.21.15.1:0/4196072882 shutdown_connections 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 --2- 172.21.15.1:0/4196072882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5064a370 0x55ed5064b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 --2- 172.21.15.1:0/4196072882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed5063ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 --2- 172.21.15.1:0/4196072882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5063f5d0 0x55ed50656310 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 -- 172.21.15.1:0/4196072882 >> 172.21.15.1:0/4196072882 conn(0x55ed50608310 msgr2=0x55ed50645860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 -- 172.21.15.1:0/4196072882 shutdown_connections 2024-09-16T00:30:30.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 -- 172.21.15.1:0/4196072882 wait complete. 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 -- start start 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 Processor -- start 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5063f5d0 0x55ed50652730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.864+0000 7efedf035740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed50652c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efedf035740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5064a370 0x55ed50644230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efedf035740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efed81ad3c0 con 0x55ed5064a370 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efedf035740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efed81ae830 con 0x55ed50640b40 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efedf035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efed81ada90 con 0x55ed5063f5d0 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efedf035740 10 client.0 _ll_register_callbacks cb 0x7efecc009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5064a370 0x55ed50644230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5064a370 0x55ed50644230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53708/0 (socket says 172.21.15.1:53708) 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed6d9d640 1 -- 172.21.15.1:0/4155446051 learned_addr learned my addr 172.21.15.1:0/4155446051 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed5d9b640 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed50652c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed659c640 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5063f5d0 0x55ed50652730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed6d9d640 1 -- 172.21.15.1:0/4155446051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5063f5d0 msgr2=0x55ed50652730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed6d9d640 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5063f5d0 0x55ed50652730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed6d9d640 1 -- 172.21.15.1:0/4155446051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 msgr2=0x55ed50652c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed6d9d640 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed50652c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed6d9d640 1 -- 172.21.15.1:0/4155446051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed50644a60 con 0x55ed5064a370 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed5d9b640 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed50652c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efed6d9d640 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5064a370 0x55ed50644230 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efed0003e10 tx=0x7efed0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbf20603cd440ba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:30.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efebdffb640 1 -- 172.21.15.1:0/4155446051 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0011850 con 0x55ed5064a370 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efebdffb640 1 -- 172.21.15.1:0/4155446051 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efed00119f0 con 0x55ed5064a370 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.865+0000 7efedf035740 1 -- 172.21.15.1:0/4155446051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec8003df0 con 0x55ed5064a370 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efebdffb640 1 -- 172.21.15.1:0/4155446051 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efed0003e40 con 0x55ed5064a370 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efedf035740 10 client.15762 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efedf035740 1 -- 172.21.15.1:0/4155446051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed50661ce0 con 0x55ed5064a370 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efebdffb640 1 -- 172.21.15.1:0/4155446051 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec8003df0 con 0x55ed5064a370 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efebdffb640 1 -- 172.21.15.1:0/4155446051 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efed0026020 con 0x55ed5064a370 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efedf035740 10 client.15762 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efedf035740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efedf035740 20 client.15762 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efedf035740 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed506c4ec0 0x55ed506c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efedf035740 4 client.15762 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efedf035740 1 -- 172.21.15.1:0/4155446051 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed505acd70 con 0x55ed506c4ec0 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.866+0000 7efed659c640 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed506c4ec0 0x55ed506c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efebdffb640 10 client.15762 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efed659c640 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed506c4ec0 0x55ed506c72e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c19c89fd11e3cba server_cookie=5931961d2827df1e in_seq=0 out_seq=0 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efebdffb640 1 -- 172.21.15.1:0/4155446051 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed505acd70 con 0x55ed506c4ec0 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efebdffb640 10 client.15762 handle_command_reply: tid=0 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 client.15762 shutdown 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 -- 172.21.15.1:0/4155446051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5064a370 msgr2=0x55ed50644230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5064a370 0x55ed50644230 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efed0003e10 tx=0x7efed0004730 comp rx=0 tx=0).stop 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 -- 172.21.15.1:0/4155446051 shutdown_connections 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed506c4ec0 0x55ed506c72e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5064a370 0x55ed50644230 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed50640b40 0x55ed50652c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 --2- 172.21.15.1:0/4155446051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5063f5d0 0x55ed50652730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 -- 172.21.15.1:0/4155446051 >> 172.21.15.1:0/4155446051 conn(0x55ed50608310 msgr2=0x55ed50649010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efebdffb640 0 client.15762 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 -- 172.21.15.1:0/4155446051 shutdown_connections 2024-09-16T00:30:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 1 -- 172.21.15.1:0/4155446051 wait complete. 2024-09-16T00:30:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.867+0000 7efedf035740 20 client.15762 trim_cache size 0 max 16384 2024-09-16T00:30:30.878 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:30:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 -- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efec00a32c0 msgr2=0x7efec00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efec00a32c0 0x7efec00a5780 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efed000b6c0 tx=0x7efed00046c0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 -- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8101420 msgr2=0x7efed8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8101420 0x7efed8198970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efecc009230 tx=0x7efecc004270 comp rx=0 tx=0).stop 2024-09-16T00:30:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 -- 172.21.15.1:0/2420863318 shutdown_connections 2024-09-16T00:30:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efec00a32c0 0x7efec00a5780 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efed8106a80 0x7efed819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed8101420 0x7efed8198970 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 --2- 172.21.15.1:0/2420863318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed8100a80 0x7efed8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 -- 172.21.15.1:0/2420863318 >> 172.21.15.1:0/2420863318 conn(0x7efed80fc7a0 msgr2=0x7efed806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 -- 172.21.15.1:0/2420863318 shutdown_connections 2024-09-16T00:30:30.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:30.868+0000 7efedcc5c640 1 -- 172.21.15.1:0/2420863318 wait complete. 2024-09-16T00:30:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:32 smithi019 ceph-mon[35186]: pgmap v414: 129 pgs: 129 active+clean; 906 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:30:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:32 smithi001 ceph-mon[30934]: pgmap v414: 129 pgs: 129 active+clean; 906 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:30:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:32 smithi153 ceph-mon[35538]: pgmap v414: 129 pgs: 129 active+clean; 906 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:30:32.739 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:30:32.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:30:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.492+0000 7fad678c3640 1 -- 172.21.15.1:0/2680108330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 msgr2=0x7fad60103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.492+0000 7fad678c3640 1 --2- 172.21.15.1:0/2680108330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 0x7fad60103110 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009930 tx=0x7fad5c02a420 comp rx=0 tx=0).stop 2024-09-16T00:30:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.492+0000 7fad678c3640 1 -- 172.21.15.1:0/2680108330 shutdown_connections 2024-09-16T00:30:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.492+0000 7fad678c3640 1 --2- 172.21.15.1:0/2680108330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad601082a0 0x7fad6010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.493+0000 7fad678c3640 1 --2- 172.21.15.1:0/2680108330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad601036e0 0x7fad60107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.493+0000 7fad678c3640 1 --2- 172.21.15.1:0/2680108330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 0x7fad60103110 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.493+0000 7fad678c3640 1 -- 172.21.15.1:0/2680108330 >> 172.21.15.1:0/2680108330 conn(0x7fad600fc7d0 msgr2=0x7fad600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.493+0000 7fad678c3640 1 -- 172.21.15.1:0/2680108330 shutdown_connections 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.493+0000 7fad678c3640 1 -- 172.21.15.1:0/2680108330 wait complete. 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.493+0000 7fad678c3640 1 Processor -- start 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.493+0000 7fad678c3640 1 -- start start 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad678c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 0x7fad60100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad678c3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad601036e0 0x7fad60100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad678c3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad601082a0 0x7fad60101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad678c3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad601120a0 con 0x7fad60102d30 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad678c3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad60111f20 con 0x7fad601082a0 2024-09-16T00:30:33.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad678c3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad60112220 con 0x7fad601036e0 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad6565f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 0x7fad60100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad6565f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 0x7fad60100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53728/0 (socket says 172.21.15.1:53728) 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad6565f640 1 -- 172.21.15.1:0/1923251774 learned_addr learned my addr 172.21.15.1:0/1923251774 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad64e5e640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad601036e0 0x7fad60100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad65e60640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad601082a0 0x7fad60101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad6565f640 1 -- 172.21.15.1:0/1923251774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad601036e0 msgr2=0x7fad60100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad6565f640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad601036e0 0x7fad60100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad6565f640 1 -- 172.21.15.1:0/1923251774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad601082a0 msgr2=0x7fad60101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad6565f640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad601082a0 0x7fad60101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad6565f640 1 -- 172.21.15.1:0/1923251774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad601a6a20 con 0x7fad60102d30 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad6565f640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 0x7fad60100990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009230 tx=0x7fad5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad567fc640 1 -- 172.21.15.1:0/1923251774 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c03f030 con 0x7fad60102d30 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad567fc640 1 -- 172.21.15.1:0/1923251774 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad5c003ad0 con 0x7fad60102d30 2024-09-16T00:30:33.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad678c3640 1 -- 172.21.15.1:0/1923251774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad601a6cb0 con 0x7fad60102d30 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad567fc640 1 -- 172.21.15.1:0/1923251774 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c035070 con 0x7fad60102d30 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.494+0000 7fad678c3640 1 -- 172.21.15.1:0/1923251774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad601a7190 con 0x7fad60102d30 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.496+0000 7fad567fc640 1 -- 172.21.15.1:0/1923251774 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fad5c042030 con 0x7fad60102d30 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.497+0000 7fad69c9c740 1 Processor -- start 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.497+0000 7fad567fc640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad4c0a32c0 0x7fad4c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.497+0000 7fad69c9c740 1 -- start start 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad567fc640 1 -- 172.21.15.1:0/1923251774 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fad5c0f0200 con 0x7fad60102d30 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad65e60640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad4c0a32c0 0x7fad4c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad69c9c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bb4f139280 con 0x55bb4f1ded40 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad69c9c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb4f0e0b10 con 0x55bb4f1d4480 2024-09-16T00:30:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad69c9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb4f135240 con 0x55bb4f1d3b60 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad65e60640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad65e60640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53734/0 (socket says 172.21.15.1:53734) 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad65e60640 1 -- 172.21.15.1:0/109127773 learned_addr learned my addr 172.21.15.1:0/109127773 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad64e5e640 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad6565f640 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad65e60640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad4c0a32c0 0x7fad4c0a5780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fad580146e0 tx=0x7fad58014ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad64e5e640 1 -- 172.21.15.1:0/109127773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 msgr2=0x55bb4f1d3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad64e5e640 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1d3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad64e5e640 1 -- 172.21.15.1:0/109127773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 msgr2=0x55bb4f1e1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad64e5e640 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad64e5e640 1 -- 172.21.15.1:0/109127773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4f0ed1f0 con 0x55bb4f1d4480 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad65e60640 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.498+0000 7fad64e5e640 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1de800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad6006a370 tx=0x7fad5000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c91d6da731a1860e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad4b7fe640 1 -- 172.21.15.1:0/109127773 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50004030 con 0x55bb4f1d4480 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad4b7fe640 1 -- 172.21.15.1:0/109127773 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50005500 con 0x55bb4f1d4480 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad4b7fe640 1 -- 172.21.15.1:0/109127773 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50004ce0 con 0x55bb4f1d4480 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad69c9c740 1 -- 172.21.15.1:0/109127773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 msgr2=0x55bb4f1de800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad69c9c740 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1de800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad6006a370 tx=0x7fad5000e350 comp rx=0 tx=0).stop 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad69c9c740 1 -- 172.21.15.1:0/109127773 shutdown_connections 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad69c9c740 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad69c9c740 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1de800 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad69c9c740 1 --2- 172.21.15.1:0/109127773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1d3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad69c9c740 1 -- 172.21.15.1:0/109127773 >> 172.21.15.1:0/109127773 conn(0x55bb4f1d00f0 msgr2=0x55bb4f1d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad69c9c740 1 -- 172.21.15.1:0/109127773 shutdown_connections 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.499+0000 7fad69c9c740 1 -- 172.21.15.1:0/109127773 wait complete. 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad69c9c740 1 -- start start 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad69c9c740 1 Processor -- start 2024-09-16T00:30:33.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1dc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad69c9c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bb4f1e1910 con 0x55bb4f1ded40 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad69c9c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb4f0f3f10 con 0x55bb4f1d4480 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad69c9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb4f1e1a90 con 0x55bb4f1d3b60 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad69c9c740 10 client.0 _ll_register_callbacks cb 0x55bb4f0bca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad65e60640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1dc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad65e60640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1dc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53744/0 (socket says 172.21.15.1:53744) 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad65e60640 1 -- 172.21.15.1:0/692533694 learned_addr learned my addr 172.21.15.1:0/692533694 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.500+0000 7fad64e5e640 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad6565f640 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad65e60640 1 -- 172.21.15.1:0/692533694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 msgr2=0x55bb4f1de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad65e60640 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad65e60640 1 -- 172.21.15.1:0/692533694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 msgr2=0x55bb4f1dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad65e60640 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1dbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad65e60640 1 -- 172.21.15.1:0/692533694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4eeaa6a0 con 0x55bb4f1ded40 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad6565f640 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad65e60640 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1dc2e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fad580092f0 tx=0x7fad5801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37e89d5231fa278 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad48ff9640 1 -- 172.21.15.1:0/692533694 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58004d20 con 0x55bb4f1ded40 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad48ff9640 1 -- 172.21.15.1:0/692533694 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fad58013900 con 0x55bb4f1ded40 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad69c9c740 1 -- 172.21.15.1:0/692533694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad50005500 con 0x55bb4f1ded40 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad48ff9640 1 -- 172.21.15.1:0/692533694 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad58013b00 con 0x55bb4f1ded40 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad69c9c740 10 client.15774 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.501+0000 7fad69c9c740 1 -- 172.21.15.1:0/692533694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bb4f1ed470 con 0x55bb4f1ded40 2024-09-16T00:30:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad48ff9640 1 -- 172.21.15.1:0/692533694 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50005500 con 0x55bb4f1ded40 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad48ff9640 1 -- 172.21.15.1:0/692533694 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fad5800f040 con 0x55bb4f1ded40 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad69c9c740 10 client.15774 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad69c9c740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad69c9c740 20 client.15774 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad69c9c740 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bb4f1ee930 0x55bb4f1f0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad69c9c740 4 client.15774 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad69c9c740 1 -- 172.21.15.1:0/692533694 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb4f1467f0 con 0x55bb4f1ee930 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad6565f640 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bb4f1ee930 0x55bb4f1f0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad6565f640 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bb4f1ee930 0x55bb4f1f0dd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55354d6ee8341ec server_cookie=870b3d16c9848211 in_seq=0 out_seq=0 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.502+0000 7fad48ff9640 10 client.15774 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad48ff9640 1 -- 172.21.15.1:0/692533694 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb4f1467f0 con 0x55bb4f1ee930 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad48ff9640 10 client.15774 handle_command_reply: tid=0 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad69c9c740 1 client.15774 shutdown 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad69c9c740 1 -- 172.21.15.1:0/692533694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 msgr2=0x55bb4f1dc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad69c9c740 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1dc2e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fad580092f0 tx=0x7fad5801a000 comp rx=0 tx=0).stop 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad69c9c740 1 -- 172.21.15.1:0/692533694 shutdown_connections 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad69c9c740 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bb4f1ee930 0x55bb4f1f0dd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad69c9c740 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1dc2e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad69c9c740 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad69c9c740 1 --2- 172.21.15.1:0/692533694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad69c9c740 1 -- 172.21.15.1:0/692533694 >> 172.21.15.1:0/692533694 conn(0x55bb4f1d00f0 msgr2=0x55bb4f1cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.503+0000 7fad48ff9640 0 client.15774 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.504+0000 7fad69c9c740 1 -- 172.21.15.1:0/692533694 shutdown_connections 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.504+0000 7fad69c9c740 1 -- 172.21.15.1:0/692533694 wait complete. 2024-09-16T00:30:33.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.504+0000 7fad69c9c740 20 client.15774 trim_cache size 0 max 16384 2024-09-16T00:30:33.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad69c9c740 1 Processor -- start 2024-09-16T00:30:33.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad69c9c740 1 -- start start 2024-09-16T00:30:33.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1d65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1d4480 0x55bb4f1f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1ded40 0x55bb4f1f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad69c9c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad601b38e0 con 0x55bb4f1d4480 2024-09-16T00:30:33.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad69c9c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad601b11b0 con 0x55bb4f1d3b60 2024-09-16T00:30:33.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad69c9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad601af000 con 0x55bb4f1ded40 2024-09-16T00:30:33.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad64e5e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1d4480 0x55bb4f1f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad64e5e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1d4480 0x55bb4f1f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53756/0 (socket says 172.21.15.1:53756) 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad64e5e640 1 -- 172.21.15.1:0/2497346031 learned_addr learned my addr 172.21.15.1:0/2497346031 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad6565f640 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1d65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad65e60640 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1ded40 0x55bb4f1f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad64e5e640 1 -- 172.21.15.1:0/2497346031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1ded40 msgr2=0x55bb4f1f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad64e5e640 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1ded40 0x55bb4f1f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad64e5e640 1 -- 172.21.15.1:0/2497346031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d3b60 msgr2=0x55bb4f1d65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad64e5e640 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1d65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad64e5e640 1 -- 172.21.15.1:0/2497346031 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4f1cff20 con 0x55bb4f1d4480 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad65e60640 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1ded40 0x55bb4f1f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.522+0000 7fad64e5e640 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1d4480 0x55bb4f1f66f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fad6006a370 tx=0x7fad500048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888002e876802844 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad4affd640 1 -- 172.21.15.1:0/2497346031 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5000ee40 con 0x55bb4f1d4480 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad4affd640 1 -- 172.21.15.1:0/2497346031 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50004520 con 0x55bb4f1d4480 2024-09-16T00:30:33.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad4affd640 1 -- 172.21.15.1:0/2497346031 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5001a450 con 0x55bb4f1d4480 2024-09-16T00:30:33.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad69c9c740 1 -- 172.21.15.1:0/2497346031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1d4480 msgr2=0x55bb4f1f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1d4480 0x55bb4f1f66f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fad6006a370 tx=0x7fad500048b0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad69c9c740 1 -- 172.21.15.1:0/2497346031 shutdown_connections 2024-09-16T00:30:33.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1ded40 0x55bb4f1f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1d4480 0x55bb4f1f66f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2497346031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1d65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad69c9c740 1 -- 172.21.15.1:0/2497346031 >> 172.21.15.1:0/2497346031 conn(0x55bb4f1d00f0 msgr2=0x55bb4f1da420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad69c9c740 1 -- 172.21.15.1:0/2497346031 shutdown_connections 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.523+0000 7fad69c9c740 1 -- 172.21.15.1:0/2497346031 wait complete. 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad69c9c740 1 -- start start 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad69c9c740 1 Processor -- start 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1e3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad69c9c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad69c9c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad601af670 con 0x55bb4f1ded40 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad69c9c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad601b0ae0 con 0x55bb4f1d4480 2024-09-16T00:30:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad69c9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad601afd40 con 0x55bb4f1d3b60 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad69c9c740 10 client.0 _ll_register_callbacks cb 0x7fad58014040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad65e60640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad65e60640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53772/0 (socket says 172.21.15.1:53772) 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.524+0000 7fad65e60640 1 -- 172.21.15.1:0/2784335574 learned_addr learned my addr 172.21.15.1:0/2784335574 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad6565f640 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad64e5e640 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad65e60640 1 -- 172.21.15.1:0/2784335574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 msgr2=0x55bb4f1e2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad65e60640 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1e2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad65e60640 1 -- 172.21.15.1:0/2784335574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 msgr2=0x55bb4f1e3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad65e60640 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1e3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad65e60640 1 -- 172.21.15.1:0/2784335574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4f258770 con 0x55bb4f1ded40 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad65e60640 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e3580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fad58012390 tx=0x7fad58009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12af7bfa332b1915 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:33.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad48ff9640 1 -- 172.21.15.1:0/2784335574 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58013ea0 con 0x55bb4f1ded40 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad48ff9640 1 -- 172.21.15.1:0/2784335574 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fad58020860 con 0x55bb4f1ded40 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad69c9c740 1 -- 172.21.15.1:0/2784335574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad50004520 con 0x55bb4f1ded40 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad48ff9640 1 -- 172.21.15.1:0/2784335574 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad58047050 con 0x55bb4f1ded40 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad69c9c740 10 client.15780 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.525+0000 7fad69c9c740 1 -- 172.21.15.1:0/2784335574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bb4f2599e0 con 0x55bb4f1ded40 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.526+0000 7fad48ff9640 1 -- 172.21.15.1:0/2784335574 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50004520 con 0x55bb4f1ded40 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.526+0000 7fad48ff9640 1 -- 172.21.15.1:0/2784335574 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fad58028c20 con 0x55bb4f1ded40 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.526+0000 7fad69c9c740 10 client.15780 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.526+0000 7fad69c9c740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.526+0000 7fad69c9c740 20 client.15780 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.526+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bb4f25a010 0x55bb4f25c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.526+0000 7fad69c9c740 4 client.15780 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:30:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.526+0000 7fad69c9c740 1 -- 172.21.15.1:0/2784335574 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bb4f140d00 con 0x55bb4f25a010 2024-09-16T00:30:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.526+0000 7fad6565f640 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bb4f25a010 0x55bb4f25c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.527+0000 7fad48ff9640 10 client.15780 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.527+0000 7fad6565f640 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bb4f25a010 0x55bb4f25c430 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ccb54abbf22bfa server_cookie=d9347944a33ccc3c in_seq=0 out_seq=0 2024-09-16T00:30:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad48ff9640 1 -- 172.21.15.1:0/2784335574 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bb4f140d00 con 0x55bb4f25a010 2024-09-16T00:30:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad48ff9640 10 client.15780 handle_command_reply: tid=0 2024-09-16T00:30:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 client.15780 shutdown 2024-09-16T00:30:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 -- 172.21.15.1:0/2784335574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 msgr2=0x55bb4f1e3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e3580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fad58012390 tx=0x7fad58009e10 comp rx=0 tx=0).stop 2024-09-16T00:30:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 -- 172.21.15.1:0/2784335574 shutdown_connections 2024-09-16T00:30:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bb4f25a010 0x55bb4f25c430 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb4f1ded40 0x55bb4f1e3580 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4f1d4480 0x55bb4f1e3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 --2- 172.21.15.1:0/2784335574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4f1d3b60 0x55bb4f1e2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad48ff9640 0 client.15780 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 -- 172.21.15.1:0/2784335574 >> 172.21.15.1:0/2784335574 conn(0x55bb4f1d00f0 msgr2=0x55bb4f1cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 -- 172.21.15.1:0/2784335574 shutdown_connections 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 1 -- 172.21.15.1:0/2784335574 wait complete. 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.528+0000 7fad69c9c740 20 client.15780 trim_cache size 0 max 16384 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:30:33.533 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "bc5264de-c16c-4438-be73-cd30616fc501", 2024-09-16T00:30:33.534 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:30:33.534 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:30:33.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 -- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad4c0a32c0 msgr2=0x7fad4c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad4c0a32c0 0x7fad4c0a5780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fad580146e0 tx=0x7fad58014ae0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 -- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 msgr2=0x7fad60100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 0x7fad60100990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009230 tx=0x7fad5c004270 comp rx=0 tx=0).stop 2024-09-16T00:30:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 -- 172.21.15.1:0/1923251774 shutdown_connections 2024-09-16T00:30:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad601082a0 0x7fad60101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad4c0a32c0 0x7fad4c0a5780 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad601036e0 0x7fad60100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 --2- 172.21.15.1:0/1923251774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad60102d30 0x7fad60100990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 -- 172.21.15.1:0/1923251774 >> 172.21.15.1:0/1923251774 conn(0x7fad600fc7d0 msgr2=0x7fad60105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 -- 172.21.15.1:0/1923251774 shutdown_connections 2024-09-16T00:30:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:33.529+0000 7fad678c3640 1 -- 172.21.15.1:0/1923251774 wait complete. 2024-09-16T00:30:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:34 smithi019 ceph-mon[35186]: pgmap v415: 129 pgs: 129 active+clean; 952 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:34 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:30:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:34 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:34 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:30:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:34 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:34 smithi001 ceph-mon[30934]: pgmap v415: 129 pgs: 129 active+clean; 952 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:34 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:30:34.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:34 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:34.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:34 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:30:34.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:34 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:34 smithi153 ceph-mon[35538]: pgmap v415: 129 pgs: 129 active+clean; 952 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:34 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:30:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:34 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:34 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:30:34.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:34 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:30:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:36 smithi019 ceph-mon[35186]: pgmap v416: 129 pgs: 129 active+clean; 952 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:36 smithi001 ceph-mon[30934]: pgmap v416: 129 pgs: 129 active+clean; 952 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:36 smithi153 ceph-mon[35538]: pgmap v416: 129 pgs: 129 active+clean; 952 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:36.574 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:30:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.338+0000 7f000d841640 1 -- 172.21.15.1:0/2873584228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0008103700 msgr2=0x7f000810bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.338+0000 7f000d841640 1 --2- 172.21.15.1:0/2873584228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0008103700 0x7f000810bbf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efffc009900 tx=0x7efffc02a480 comp rx=0 tx=0).stop 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.338+0000 7f000d841640 1 -- 172.21.15.1:0/2873584228 shutdown_connections 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.338+0000 7f000d841640 1 --2- 172.21.15.1:0/2873584228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f000810c180 0x7f000810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.338+0000 7f000d841640 1 --2- 172.21.15.1:0/2873584228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0008103700 0x7f000810bbf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.338+0000 7f000d841640 1 --2- 172.21.15.1:0/2873584228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0008102de0 0x7f00081031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.338+0000 7f000d841640 1 -- 172.21.15.1:0/2873584228 >> 172.21.15.1:0/2873584228 conn(0x7f00080fc7a0 msgr2=0x7f00080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.338+0000 7f000d841640 1 -- 172.21.15.1:0/2873584228 shutdown_connections 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.338+0000 7f000d841640 1 -- 172.21.15.1:0/2873584228 wait complete. 2024-09-16T00:30:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.339+0000 7f000d841640 1 Processor -- start 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.339+0000 7f000d841640 1 -- start start 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.339+0000 7f000d841640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0008102de0 0x7f0008100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.339+0000 7f000d841640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0008103700 0x7f0008100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.339+0000 7f000d841640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f000810c180 0x7f00081014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.339+0000 7f000d841640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0008111fb0 con 0x7f000810c180 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.339+0000 7f000d841640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0008111e30 con 0x7f0008102de0 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.339+0000 7f000d841640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0008112130 con 0x7f0008103700 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f000810c180 0x7f00081014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f000810c180 0x7f00081014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53788/0 (socket says 172.21.15.1:53788) 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00077fe640 1 -- 172.21.15.1:0/2546249586 learned_addr learned my addr 172.21.15.1:0/2546249586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00067fc640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0008103700 0x7f0008100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f0006ffd640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0008102de0 0x7f0008100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00077fe640 1 -- 172.21.15.1:0/2546249586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0008103700 msgr2=0x7f0008100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00077fe640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0008103700 0x7f0008100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00077fe640 1 -- 172.21.15.1:0/2546249586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0008102de0 msgr2=0x7f0008100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00077fe640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0008102de0 0x7f0008100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00077fe640 1 -- 172.21.15.1:0/2546249586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00081a68b0 con 0x7f000810c180 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f00077fe640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f000810c180 0x7f00081014c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f000000b970 tx=0x7f000000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f000c83f640 1 -- 172.21.15.1:0/2546249586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000019070 con 0x7f000810c180 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f000c83f640 1 -- 172.21.15.1:0/2546249586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000000cb40 con 0x7f000810c180 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f000d841640 1 -- 172.21.15.1:0/2546249586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00081a6ba0 con 0x7f000810c180 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f000c83f640 1 -- 172.21.15.1:0/2546249586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000007650 con 0x7f000810c180 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.340+0000 7f000d841640 1 -- 172.21.15.1:0/2546249586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00081a70b0 con 0x7f000810c180 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.343+0000 7f000c83f640 1 -- 172.21.15.1:0/2546249586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f000001c070 con 0x7f000810c180 2024-09-16T00:30:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f000fc1a740 1 Processor -- start 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f000fc1a740 1 -- start start 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc40880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f000fc1a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebba62e0 con 0x55a8ebc40dc0 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f000fc1a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebb4dce0 con 0x55a8ebc4b680 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f000fc1a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebba22a0 con 0x55a8ebc404a0 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f0006ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f0006ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53792/0 (socket says 172.21.15.1:53792) 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f0006ffd640 1 -- 172.21.15.1:0/3968846276 learned_addr learned my addr 172.21.15.1:0/3968846276 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f000c83f640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efff00a3240 0x7efff00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f00067fc640 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc40880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f00067fc640 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f0006ffd640 1 -- 172.21.15.1:0/3968846276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 msgr2=0x55a8ebc40880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f0006ffd640 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc40880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f0006ffd640 1 -- 172.21.15.1:0/3968846276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 msgr2=0x55a8ebc4da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f0006ffd640 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f0006ffd640 1 -- 172.21.15.1:0/3968846276 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ebb32440 con 0x55a8ebc40dc0 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f00067fc640 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc40880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.344+0000 7f00077fe640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efff00a3240 0x7efff00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.345+0000 7f000c83f640 1 -- 172.21.15.1:0/2546249586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f00000cf300 con 0x7f000810c180 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.345+0000 7f0006ffd640 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4b140 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efff800cc60 tx=0x7efff800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467817b052d737c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.345+0000 7effedffb640 1 -- 172.21.15.1:0/3968846276 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8014070 con 0x55a8ebc40dc0 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.345+0000 7effedffb640 1 -- 172.21.15.1:0/3968846276 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff80047a0 con 0x55a8ebc40dc0 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.345+0000 7effedffb640 1 -- 172.21.15.1:0/3968846276 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8010790 con 0x55a8ebc40dc0 2024-09-16T00:30:37.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 -- 172.21.15.1:0/3968846276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc40dc0 msgr2=0x55a8ebc4b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f00077fe640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efff00a3240 0x7efff00a5700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0000096010 tx=0x7f000000e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4b140 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efff800cc60 tx=0x7efff800e720 comp rx=0 tx=0).stop 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 -- 172.21.15.1:0/3968846276 shutdown_connections 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4b140 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 --2- 172.21.15.1:0/3968846276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc40880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 -- 172.21.15.1:0/3968846276 >> 172.21.15.1:0/3968846276 conn(0x55a8ebc3ca30 msgr2=0x55a8ebc3ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 -- 172.21.15.1:0/3968846276 shutdown_connections 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 -- 172.21.15.1:0/3968846276 wait complete. 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 -- start start 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 Processor -- start 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc48270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.346+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 0x55a8ebc487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f000fc1a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebc4e250 con 0x55a8ebc404a0 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f000fc1a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebb61050 con 0x55a8ebc4b680 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f000fc1a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebc4e3d0 con 0x55a8ebc40dc0 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f000fc1a740 10 client.0 _ll_register_callbacks cb 0x55a8ebba1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53804/0 (socket says 172.21.15.1:53804) 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 -- 172.21.15.1:0/1210073861 learned_addr learned my addr 172.21.15.1:0/1210073861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f0006ffd640 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc48270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 0x55a8ebc487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 -- 172.21.15.1:0/1210073861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc40dc0 msgr2=0x55a8ebc48270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc48270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 -- 172.21.15.1:0/1210073861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 msgr2=0x55a8ebc487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 0x55a8ebc487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 -- 172.21.15.1:0/1210073861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8eb9173c0 con 0x55a8ebc404a0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f00067fc640 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4afc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efffc009900 tx=0x7efffc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82728a97d209e7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7effe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7effe6ffd640 1 -- 172.21.15.1:0/1210073861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc035050 con 0x55a8ebc404a0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7effe6ffd640 1 -- 172.21.15.1:0/1210073861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efffc005020 con 0x55a8ebc404a0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.347+0000 7f000fc1a740 1 -- 172.21.15.1:0/1210073861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff80047a0 con 0x55a8ebc404a0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7effe6ffd640 1 -- 172.21.15.1:0/1210073861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efffc04f050 con 0x55a8ebc404a0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7f000fc1a740 10 client.15792 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7f000fc1a740 1 -- 172.21.15.1:0/1210073861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a8ebc59b70 con 0x55a8ebc404a0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7effe6ffd640 1 -- 172.21.15.1:0/1210073861 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff80047a0 con 0x55a8ebc404a0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7effe6ffd640 1 -- 172.21.15.1:0/1210073861 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efffc0035f0 con 0x55a8ebc404a0 2024-09-16T00:30:37.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7f000fc1a740 10 client.15792 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7f000fc1a740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7f000fc1a740 20 client.15792 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a8ebc5af80 0x55a8ebc5d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7f000fc1a740 4 client.15792 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7f000fc1a740 1 -- 172.21.15.1:0/1210073861 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8ebbb3850 con 0x55a8ebc5af80 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.348+0000 7f0006ffd640 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a8ebc5af80 0x55a8ebc5d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.349+0000 7effe6ffd640 10 client.15792 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.349+0000 7f0006ffd640 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a8ebc5af80 0x55a8ebc5d370 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679a224fd21e4bfe server_cookie=223bbb7f7636d61f in_seq=0 out_seq=0 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.349+0000 7effe6ffd640 1 -- 172.21.15.1:0/1210073861 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8ebbb3850 con 0x55a8ebc5af80 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.349+0000 7effe6ffd640 10 client.15792 handle_command_reply: tid=0 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.349+0000 7f000fc1a740 1 client.15792 shutdown 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 -- 172.21.15.1:0/1210073861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc404a0 msgr2=0x55a8ebc4afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4afc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efffc009900 tx=0x7efffc009260 comp rx=0 tx=0).stop 2024-09-16T00:30:37.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 -- 172.21.15.1:0/1210073861 shutdown_connections 2024-09-16T00:30:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a8ebc5af80 0x55a8ebc5d370 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc4b680 0x55a8ebc487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc48270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1210073861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4afc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7effe6ffd640 0 client.15792 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 -- 172.21.15.1:0/1210073861 >> 172.21.15.1:0/1210073861 conn(0x55a8ebc3ca30 msgr2=0x55a8ebc42710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 -- 172.21.15.1:0/1210073861 shutdown_connections 2024-09-16T00:30:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 1 -- 172.21.15.1:0/1210073861 wait complete. 2024-09-16T00:30:37.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.350+0000 7f000fc1a740 20 client.15792 trim_cache size 0 max 16384 2024-09-16T00:30:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f000fc1a740 1 Processor -- start 2024-09-16T00:30:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f000fc1a740 1 -- start start 2024-09-16T00:30:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc62bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc63130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f000fc1a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b3950 con 0x55a8ebc4b680 2024-09-16T00:30:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f000fc1a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b1220 con 0x55a8ebc40dc0 2024-09-16T00:30:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f000fc1a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00081af070 con 0x55a8ebc404a0 2024-09-16T00:30:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc63130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc63130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53808/0 (socket says 172.21.15.1:53808) 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 -- 172.21.15.1:0/1421580243 learned_addr learned my addr 172.21.15.1:0/1421580243 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 --2- 172.21.15.1:0/1421580243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f0006ffd640 1 --2- 172.21.15.1:0/1421580243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc62bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 -- 172.21.15.1:0/1421580243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 msgr2=0x55a8ebc574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 --2- 172.21.15.1:0/1421580243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 -- 172.21.15.1:0/1421580243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 msgr2=0x55a8ebc62bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 --2- 172.21.15.1:0/1421580243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc62bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 -- 172.21.15.1:0/1421580243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ebc5d6e0 con 0x55a8ebc4b680 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.368+0000 7f00067fc640 1 --2- 172.21.15.1:0/1421580243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc63130 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efffc005ec0 tx=0x7efffc02ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b651bd5e88a3af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7effed7fa640 1 -- 172.21.15.1:0/1421580243 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc058070 con 0x55a8ebc4b680 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7effed7fa640 1 -- 172.21.15.1:0/1421580243 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efffc005500 con 0x55a8ebc4b680 2024-09-16T00:30:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7effed7fa640 1 -- 172.21.15.1:0/1421580243 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc035030 con 0x55a8ebc4b680 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7f000fc1a740 1 -- 172.21.15.1:0/1421580243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 msgr2=0x55a8ebc63130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1421580243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc63130 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efffc005ec0 tx=0x7efffc02ac00 comp rx=0 tx=0).stop 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7f000fc1a740 1 -- 172.21.15.1:0/1421580243 shutdown_connections 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1421580243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc63130 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1421580243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc62bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7f000fc1a740 1 --2- 172.21.15.1:0/1421580243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.369+0000 7f000fc1a740 1 -- 172.21.15.1:0/1421580243 >> 172.21.15.1:0/1421580243 conn(0x55a8ebc3ca30 msgr2=0x55a8ebc43850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 -- 172.21.15.1:0/1421580243 shutdown_connections 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 -- 172.21.15.1:0/1421580243 wait complete. 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 -- start start 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 Processor -- start 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00081af6e0 con 0x55a8ebc4b680 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b0b50 con 0x55a8ebc40dc0 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00081afdb0 con 0x55a8ebc404a0 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.370+0000 7f000fc1a740 10 client.0 _ll_register_callbacks cb 0x7efffc005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53822/0 (socket says 172.21.15.1:53822) 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 -- 172.21.15.1:0/534447210 learned_addr learned my addr 172.21.15.1:0/534447210 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f0006ffd640 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 -- 172.21.15.1:0/534447210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 msgr2=0x55a8ebc4f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 -- 172.21.15.1:0/534447210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 msgr2=0x55a8ebc4f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 -- 172.21.15.1:0/534447210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ebcc4d20 con 0x55a8ebc4b680 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f00067fc640 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4ff10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efffc009670 tx=0x7efffc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f977ae385ad4880 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7effe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7effe6ffd640 1 -- 172.21.15.1:0/534447210 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc064060 con 0x55a8ebc4b680 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7effe6ffd640 1 -- 172.21.15.1:0/534447210 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efffc05e680 con 0x55a8ebc4b680 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f000fc1a740 1 -- 172.21.15.1:0/534447210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efffc005500 con 0x55a8ebc4b680 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7effe6ffd640 1 -- 172.21.15.1:0/534447210 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efffc035070 con 0x55a8ebc4b680 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f000fc1a740 10 client.15798 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.371+0000 7f000fc1a740 1 -- 172.21.15.1:0/534447210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a8ebcc5fb0 con 0x55a8ebc4b680 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.372+0000 7effe6ffd640 1 -- 172.21.15.1:0/534447210 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efffc005500 con 0x55a8ebc4b680 2024-09-16T00:30:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.372+0000 7effe6ffd640 1 -- 172.21.15.1:0/534447210 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efffc0484d0 con 0x55a8ebc4b680 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.372+0000 7f000fc1a740 10 client.15798 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.372+0000 7f000fc1a740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.372+0000 7f000fc1a740 20 client.15798 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.372+0000 7f000fc1a740 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a8ebcc65c0 0x55a8ebcc89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.372+0000 7f000fc1a740 4 client.15798 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.372+0000 7f000fc1a740 1 -- 172.21.15.1:0/534447210 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a8ebc48b40 con 0x55a8ebcc65c0 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.372+0000 7f0006ffd640 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a8ebcc65c0 0x55a8ebcc89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f0006ffd640 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a8ebcc65c0 0x55a8ebcc89b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3fe7db5220b568 server_cookie=8f9dbbff3ad105c in_seq=0 out_seq=0 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7effe6ffd640 10 client.15798 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7effe6ffd640 1 -- 172.21.15.1:0/534447210 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a8ebc48b40 con 0x55a8ebcc65c0 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7effe6ffd640 10 client.15798 handle_command_reply: tid=0 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 client.15798 shutdown 2024-09-16T00:30:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 -- 172.21.15.1:0/534447210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 msgr2=0x55a8ebc4ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4ff10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efffc009670 tx=0x7efffc031400 comp rx=0 tx=0).stop 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 -- 172.21.15.1:0/534447210 shutdown_connections 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a8ebcc65c0 0x55a8ebcc89b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8ebc4b680 0x55a8ebc4ff10 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ebc40dc0 0x55a8ebc4f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7effe6ffd640 0 client.15798 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 --2- 172.21.15.1:0/534447210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8ebc404a0 0x55a8ebc4f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 -- 172.21.15.1:0/534447210 >> 172.21.15.1:0/534447210 conn(0x55a8ebc3ca30 msgr2=0x55a8ebc49ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 -- 172.21.15.1:0/534447210 shutdown_connections 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 1 -- 172.21.15.1:0/534447210 wait complete. 2024-09-16T00:30:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.373+0000 7f000fc1a740 20 client.15798 trim_cache size 0 max 16384 2024-09-16T00:30:37.385 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:30:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:30:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:30:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "bc5264de-c16c-4438-be73-cd30616fc501": { 2024-09-16T00:30:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:30:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "bc5264de-c16c-4438-be73-cd30616fc501", 2024-09-16T00:30:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:30:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:30:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:30:37.386 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:30:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 -- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efff00a3240 msgr2=0x7efff00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efff00a3240 0x7efff00a5700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0000096010 tx=0x7f000000e040 comp rx=0 tx=0).stop 2024-09-16T00:30:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 -- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f000810c180 msgr2=0x7f00081014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f000810c180 0x7f00081014c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f000000b970 tx=0x7f000000be30 comp rx=0 tx=0).stop 2024-09-16T00:30:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 -- 172.21.15.1:0/2546249586 shutdown_connections 2024-09-16T00:30:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f000810c180 0x7f00081014c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efff00a3240 0x7efff00a5700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0008103700 0x7f0008100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 --2- 172.21.15.1:0/2546249586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0008102de0 0x7f0008100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 -- 172.21.15.1:0/2546249586 >> 172.21.15.1:0/2546249586 conn(0x7f00080fc7a0 msgr2=0x7f00080fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 -- 172.21.15.1:0/2546249586 shutdown_connections 2024-09-16T00:30:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:37.374+0000 7f000d841640 1 -- 172.21.15.1:0/2546249586 wait complete. 2024-09-16T00:30:38.164 INFO:tasks.cephfs.filesystem:scrub status for tag:bc5264de-c16c-4438-be73-cd30616fc501 - {'path': '/', 'tag': 'bc5264de-c16c-4438-be73-cd30616fc501', 'options': 'recursive,force'} 2024-09-16T00:30:38.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:38 smithi019 ceph-mon[35186]: pgmap v417: 129 pgs: 129 active+clean; 952 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:38.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:38 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:30:38.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:38 smithi001 ceph-mon[30934]: pgmap v417: 129 pgs: 129 active+clean; 952 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:38.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:38 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:30:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:38 smithi153 ceph-mon[35538]: pgmap v417: 129 pgs: 129 active+clean; 952 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:38 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:30:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:40 smithi019 ceph-mon[35186]: pgmap v418: 129 pgs: 129 active+clean; 1013 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:40.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:40 smithi001 ceph-mon[30934]: pgmap v418: 129 pgs: 129 active+clean; 1013 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:40 smithi153 ceph-mon[35538]: pgmap v418: 129 pgs: 129 active+clean; 1013 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:42.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:42 smithi019 ceph-mon[35186]: pgmap v419: 129 pgs: 129 active+clean; 1013 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:42.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:42 smithi001 ceph-mon[30934]: pgmap v419: 129 pgs: 129 active+clean; 1013 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:42.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:42 smithi153 ceph-mon[35538]: pgmap v419: 129 pgs: 129 active+clean; 1013 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:44.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:44 smithi019 ceph-mon[35186]: pgmap v420: 129 pgs: 129 active+clean; 1.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-16T00:30:44.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:44 smithi001 ceph-mon[30934]: pgmap v420: 129 pgs: 129 active+clean; 1.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-16T00:30:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:44 smithi153 ceph-mon[35538]: pgmap v420: 129 pgs: 129 active+clean; 1.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-16T00:30:46.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:46 smithi019 ceph-mon[35186]: pgmap v421: 129 pgs: 129 active+clean; 1.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:30:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:46 smithi001 ceph-mon[30934]: pgmap v421: 129 pgs: 129 active+clean; 1.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:30:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:46 smithi153 ceph-mon[35538]: pgmap v421: 129 pgs: 129 active+clean; 1.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:30:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:48 smithi019 ceph-mon[35186]: pgmap v422: 129 pgs: 129 active+clean; 1.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:30:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:48 smithi001 ceph-mon[30934]: pgmap v422: 129 pgs: 129 active+clean; 1.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:30:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:48 smithi153 ceph-mon[35538]: pgmap v422: 129 pgs: 129 active+clean; 1.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:30:50.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:50 smithi019 ceph-mon[35186]: pgmap v423: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:30:50.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:50 smithi001 ceph-mon[30934]: pgmap v423: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:30:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:50 smithi153 ceph-mon[35538]: pgmap v423: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:30:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:52 smithi019 ceph-mon[35186]: pgmap v424: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:30:52.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:52 smithi001 ceph-mon[30934]: pgmap v424: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:30:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:52 smithi153 ceph-mon[35538]: pgmap v424: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:30:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:54 smithi019 ceph-mon[35186]: pgmap v425: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:54 smithi001 ceph-mon[30934]: pgmap v425: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:54 smithi153 ceph-mon[35538]: pgmap v425: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:30:55.869 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:30:55.869 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-16T00:30:55.869 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:56.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:56 smithi001 ceph-mon[30934]: pgmap v426: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:30:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:56 smithi019 ceph-mon[35186]: pgmap v426: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:30:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:56 smithi153 ceph-mon[35538]: pgmap v426: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:30:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.673+0000 7f139ea88640 1 -- 172.21.15.1:0/2844134409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398102570 msgr2=0x7f13981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.673+0000 7f139ea88640 1 --2- 172.21.15.1:0/2844134409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398102570 0x7f13981029b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1384009860 tx=0x7f138402a3d0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.673+0000 7f139ea88640 1 -- 172.21.15.1:0/2844134409 shutdown_connections 2024-09-16T00:30:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.673+0000 7f139ea88640 1 --2- 172.21.15.1:0/2844134409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1398102ef0 0x7f139810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.673+0000 7f139ea88640 1 --2- 172.21.15.1:0/2844134409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398102570 0x7f13981029b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.673+0000 7f139ea88640 1 --2- 172.21.15.1:0/2844134409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1398100e20 0x7f1398101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.673+0000 7f139ea88640 1 -- 172.21.15.1:0/2844134409 >> 172.21.15.1:0/2844134409 conn(0x7f13980fc7d0 msgr2=0x7f13980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.673+0000 7f139ea88640 1 -- 172.21.15.1:0/2844134409 shutdown_connections 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.673+0000 7f139ea88640 1 -- 172.21.15.1:0/2844134409 wait complete. 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139ea88640 1 Processor -- start 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139ea88640 1 -- start start 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139ea88640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398100e20 0x7f13981985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139ea88640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1398102570 0x7f1398198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139ea88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1398102ef0 0x7f139819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139ea88640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1398110f50 con 0x7f1398100e20 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139ea88640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1398110dd0 con 0x7f1398102ef0 2024-09-16T00:30:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139ea88640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f13981110d0 con 0x7f1398102570 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139c824640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398100e20 0x7f13981985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139c824640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398100e20 0x7f13981985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53674/0 (socket says 172.21.15.1:53674) 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139c824640 1 -- 172.21.15.1:0/454427821 learned_addr learned my addr 172.21.15.1:0/454427821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.674+0000 7f139d025640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1398102ef0 0x7f139819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f1397fff640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1398102570 0x7f1398198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f139c824640 1 -- 172.21.15.1:0/454427821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1398102570 msgr2=0x7f1398198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f139c824640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1398102570 0x7f1398198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f139c824640 1 -- 172.21.15.1:0/454427821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1398102ef0 msgr2=0x7f139819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f139c824640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1398102ef0 0x7f139819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f139c824640 1 -- 172.21.15.1:0/454427821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139819d560 con 0x7f1398100e20 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f139c824640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398100e20 0x7f13981985b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f138c00e9c0 tx=0x7f138c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f1395ffb640 1 -- 172.21.15.1:0/454427821 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c00ccc0 con 0x7f1398100e20 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f1395ffb640 1 -- 172.21.15.1:0/454427821 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f138c005500 con 0x7f1398100e20 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139819d850 con 0x7f1398100e20 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f1395ffb640 1 -- 172.21.15.1:0/454427821 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c010430 con 0x7f1398100e20 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.675+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13981a5130 con 0x7f1398100e20 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.676+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1398101200 con 0x7f1398100e20 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.677+0000 7f1395ffb640 1 -- 172.21.15.1:0/454427821 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f138c0046c0 con 0x7f1398100e20 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.678+0000 7f1395ffb640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f13880a3240 0x7f13880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.678+0000 7f1397fff640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f13880a3240 0x7f13880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.678+0000 7f1395ffb640 1 -- 172.21.15.1:0/454427821 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f138c0d00f0 con 0x7f1398100e20 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.679+0000 7f1397fff640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f13880a3240 0x7f13880a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13840095f0 tx=0x7f1384009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.683+0000 7f1395ffb640 1 -- 172.21.15.1:0/454427821 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f138c09c9b0 con 0x7f1398100e20 2024-09-16T00:30:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.893+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f139810a0a0 con 0x7f1398100e20 2024-09-16T00:30:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.894+0000 7f1395ffb640 1 -- 172.21.15.1:0/454427821 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f138c096e20 con 0x7f1398100e20 2024-09-16T00:30:56.894 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:30:56.895 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:30:56.895 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:30:56.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f13880a3240 msgr2=0x7f13880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:56.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f13880a3240 0x7f13880a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13840095f0 tx=0x7f1384009560 comp rx=0 tx=0).stop 2024-09-16T00:30:56.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398100e20 msgr2=0x7f13981985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:56.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398100e20 0x7f13981985b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f138c00e9c0 tx=0x7f138c00ee80 comp rx=0 tx=0).stop 2024-09-16T00:30:56.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 shutdown_connections 2024-09-16T00:30:56.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1398102ef0 0x7f139819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f13880a3240 0x7f13880a5700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1398102570 0x7f1398198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 --2- 172.21.15.1:0/454427821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1398100e20 0x7f13981985b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 >> 172.21.15.1:0/454427821 conn(0x7f13980fc7d0 msgr2=0x7f139806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:56.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 shutdown_connections 2024-09-16T00:30:56.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:56.897+0000 7f139ea88640 1 -- 172.21.15.1:0/454427821 wait complete. 2024-09-16T00:30:57.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:57 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/454427821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:57.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:57 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/454427821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:57.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:57 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/454427821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:57.747 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-16T00:30:57.748 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-16T00:30:58.012 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:58.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:30:58 smithi019 ceph-mon[35186]: pgmap v427: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:30:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:30:58 smithi001 ceph-mon[30934]: pgmap v427: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:30:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:30:58 smithi153 ceph-mon[35538]: pgmap v427: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:30:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.525+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/4149991301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c101df0 msgr2=0x7fc69c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.525+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/4149991301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c101df0 0x7fc69c1021d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009900 tx=0x7fc68c02a410 comp rx=0 tx=0).stop 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.525+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/4149991301 shutdown_connections 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.525+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/4149991301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c106f20 0x7fc69c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.525+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/4149991301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc69c1027a0 0x7fc69c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.525+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/4149991301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c101df0 0x7fc69c1021d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.525+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/4149991301 >> 172.21.15.1:0/4149991301 conn(0x7fc69c0fd8f0 msgr2=0x7fc69c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.525+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/4149991301 shutdown_connections 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.525+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/4149991301 wait complete. 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc6a0eaf640 1 Processor -- start 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc6a0eaf640 1 -- start start 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc6a0eaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c101df0 0x7fc69c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc6a0eaf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c1027a0 0x7fc69c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc6a0eaf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc69c106f20 0x7fc69c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc6a0eaf640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c111f80 con 0x7fc69c1027a0 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc6a0eaf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c111e00 con 0x7fc69c101df0 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc6a0eaf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c112100 con 0x7fc69c106f20 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc693fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c1027a0 0x7fc69c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc693fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c1027a0 0x7fc69c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53700/0 (socket says 172.21.15.1:53700) 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc693fff640 1 -- 172.21.15.1:0/115636719 learned_addr learned my addr 172.21.15.1:0/115636719 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc69a59c640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c101df0 0x7fc69c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.526+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc69c106f20 0x7fc69c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc693fff640 1 -- 172.21.15.1:0/115636719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc69c106f20 msgr2=0x7fc69c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc693fff640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc69c106f20 0x7fc69c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc693fff640 1 -- 172.21.15.1:0/115636719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c101df0 msgr2=0x7fc69c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc693fff640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c101df0 0x7fc69c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc693fff640 1 -- 172.21.15.1:0/115636719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c1a6900 con 0x7fc69c1027a0 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc69c106f20 0x7fc69c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc693fff640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c1027a0 0x7fc69c074a30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc68400e9f0 tx=0x7fc68400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc6937fe640 1 -- 172.21.15.1:0/115636719 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68400cd80 con 0x7fc69c1027a0 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc6937fe640 1 -- 172.21.15.1:0/115636719 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684005500 con 0x7fc69c1027a0 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/115636719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c1a6bf0 con 0x7fc69c1027a0 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc6937fe640 1 -- 172.21.15.1:0/115636719 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684010690 con 0x7fc69c1027a0 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.527+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/115636719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c1a70e0 con 0x7fc69c1027a0 2024-09-16T00:30:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc6a3288740 1 Processor -- start 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc6937fe640 1 -- 172.21.15.1:0/115636719 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc6840046c0 con 0x7fc69c1027a0 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc6a3288740 1 -- start start 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612900850 0x55d612900c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612901170 0x55d61290b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d61290de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc6a3288740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d612865160 con 0x55d61290ba30 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc6a3288740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d61280c720 con 0x55d612901170 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc6a3288740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d612861120 con 0x55d612900850 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc693fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d61290de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc69a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612901170 0x55d61290b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc693fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d61290de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53716/0 (socket says 172.21.15.1:53716) 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc693fff640 1 -- 172.21.15.1:0/2734899800 learned_addr learned my addr 172.21.15.1:0/2734899800 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.530+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/2734899800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612900850 0x55d612900c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc6937fe640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6880a3240 0x7fc6880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc693fff640 1 -- 172.21.15.1:0/2734899800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612900850 msgr2=0x55d612900c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc693fff640 1 --2- 172.21.15.1:0/2734899800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612900850 0x55d612900c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc693fff640 1 -- 172.21.15.1:0/2734899800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612901170 msgr2=0x55d61290b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc693fff640 1 --2- 172.21.15.1:0/2734899800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612901170 0x55d61290b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc693fff640 1 -- 172.21.15.1:0/2734899800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6125d6ba0 con 0x55d61290ba30 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2734899800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612901170 0x55d61290b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6880a3240 0x7fc6880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc693fff640 1 --2- 172.21.15.1:0/2734899800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d61290de20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc6840b8630 tx=0x7fc6840984d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88f180492ab0fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc6937fe640 1 -- 172.21.15.1:0/115636719 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc6840d01e0 con 0x7fc69c1027a0 2024-09-16T00:30:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc690ff9640 1 -- 172.21.15.1:0/2734899800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68409c1d0 con 0x55d61290ba30 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc690ff9640 1 -- 172.21.15.1:0/2734899800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6840b62e0 con 0x55d61290ba30 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.531+0000 7fc690ff9640 1 -- 172.21.15.1:0/2734899800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68409bd60 con 0x55d61290ba30 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 -- 172.21.15.1:0/2734899800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 msgr2=0x55d61290de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6880a3240 0x7fc6880a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc694005fd0 tx=0x7fc694004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2734899800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d61290de20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc6840b8630 tx=0x7fc6840984d0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 -- 172.21.15.1:0/2734899800 shutdown_connections 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2734899800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d61290de20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2734899800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612901170 0x55d61290b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2734899800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612900850 0x55d612900c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 -- 172.21.15.1:0/2734899800 >> 172.21.15.1:0/2734899800 conn(0x55d6128fcde0 msgr2=0x55d6128fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 -- 172.21.15.1:0/2734899800 shutdown_connections 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 -- 172.21.15.1:0/2734899800 wait complete. 2024-09-16T00:30:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 -- start start 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.532+0000 7fc6a3288740 1 Processor -- start 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 0x55d612908970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612908eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d6129093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc6a3288740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d61290e5a0 con 0x55d61290ba30 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc6a3288740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d6127ed900 con 0x55d612900850 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc6a3288740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d61290e720 con 0x55d612901170 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc6a3288740 10 client.0 _ll_register_callbacks cb 0x55d6127eb780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d6129093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d6129093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53726/0 (socket says 172.21.15.1:53726) 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69ad9d640 1 -- 172.21.15.1:0/2262557766 learned_addr learned my addr 172.21.15.1:0/2262557766 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 0x55d612908970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc693fff640 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612908eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69ad9d640 1 -- 172.21.15.1:0/2262557766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 msgr2=0x55d612908eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612908eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69ad9d640 1 -- 172.21.15.1:0/2262557766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 msgr2=0x55d612908970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 0x55d612908970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69ad9d640 1 -- 172.21.15.1:0/2262557766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d61282dc50 con 0x55d61290ba30 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc693fff640 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612908eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d6129093f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc694012790 tx=0x7fc694004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f051578f1777e39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.533+0000 7fc681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc681ffb640 1 -- 172.21.15.1:0/2262557766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69401cb60 con 0x55d61290ba30 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc681ffb640 1 -- 172.21.15.1:0/2262557766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc69401cd00 con 0x55d61290ba30 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc6a3288740 1 -- 172.21.15.1:0/2262557766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d612872890 con 0x55d61290ba30 2024-09-16T00:30:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc681ffb640 1 -- 172.21.15.1:0/2262557766 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc694030050 con 0x55d61290ba30 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc6a3288740 10 client.15816 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc6a3288740 1 -- 172.21.15.1:0/2262557766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d61291a300 con 0x55d61290ba30 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc681ffb640 1 -- 172.21.15.1:0/2262557766 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d612872890 con 0x55d61290ba30 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc681ffb640 1 -- 172.21.15.1:0/2262557766 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc69401a4d0 con 0x55d61290ba30 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc6a3288740 10 client.15816 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc6a3288740 10 client.15816 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc6a3288740 20 client.15816 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d61291b7c0 0x55d61291dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc6a3288740 4 client.15816 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.534+0000 7fc6a3288740 1 -- 172.21.15.1:0/2262557766 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d61286cbe0 con 0x55d61291b7c0 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.535+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d61291b7c0 0x55d61291dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.535+0000 7fc681ffb640 10 client.15816 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.535+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d61291b7c0 0x55d61291dc20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d577ab2df14c925 server_cookie=8d46253bbb57158 in_seq=0 out_seq=0 2024-09-16T00:30:58.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc681ffb640 1 -- 172.21.15.1:0/2262557766 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d61286cbe0 con 0x55d61291b7c0 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc681ffb640 10 client.15816 handle_command_reply: tid=0 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 client.15816 shutdown 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 -- 172.21.15.1:0/2262557766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 msgr2=0x55d6129093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d6129093f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc694012790 tx=0x7fc694004ab0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 -- 172.21.15.1:0/2262557766 shutdown_connections 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d61291b7c0 0x55d61291dc20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61290ba30 0x55d6129093f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612908eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2262557766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 0x55d612908970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc681ffb640 0 client.15816 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 -- 172.21.15.1:0/2262557766 >> 172.21.15.1:0/2262557766 conn(0x55d6128fcde0 msgr2=0x55d612902ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 -- 172.21.15.1:0/2262557766 shutdown_connections 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 1 -- 172.21.15.1:0/2262557766 wait complete. 2024-09-16T00:30:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.536+0000 7fc6a3288740 20 client.15816 trim_cache size 0 max 16384 2024-09-16T00:30:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.553+0000 7fc6a3288740 1 Processor -- start 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.553+0000 7fc6a3288740 1 -- start start 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.553+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612900850 0x55d612923540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.553+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612923a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.553+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d61290ba30 0x55d612905bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.553+0000 7fc6a3288740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c1b3890 con 0x55d612900850 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.553+0000 7fc6a3288740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c1b1160 con 0x55d61290ba30 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.553+0000 7fc6a3288740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c1aefb0 con 0x55d612901170 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612900850 0x55d612923540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612900850 0x55d612923540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53730/0 (socket says 172.21.15.1:53730) 2024-09-16T00:30:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69a59c640 1 -- 172.21.15.1:0/2592552585 learned_addr learned my addr 172.21.15.1:0/2592552585 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d61290ba30 0x55d612905bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc693fff640 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612923a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69a59c640 1 -- 172.21.15.1:0/2592552585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 msgr2=0x55d612923a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612923a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69a59c640 1 -- 172.21.15.1:0/2592552585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d61290ba30 msgr2=0x55d612905bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d61290ba30 0x55d612905bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69a59c640 1 -- 172.21.15.1:0/2592552585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6128fda70 con 0x55d612900850 2024-09-16T00:30:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d61290ba30 0x55d612905bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc693fff640 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612923a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612900850 0x55d612923540 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc68c02a930 tx=0x7fc68c0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2342e6796d81a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc683fff640 1 -- 172.21.15.1:0/2592552585 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c035040 con 0x55d612900850 2024-09-16T00:30:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc683fff640 1 -- 172.21.15.1:0/2592552585 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc68c005500 con 0x55d612900850 2024-09-16T00:30:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.554+0000 7fc683fff640 1 -- 172.21.15.1:0/2592552585 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c004c30 con 0x55d612900850 2024-09-16T00:30:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 -- 172.21.15.1:0/2592552585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612900850 msgr2=0x55d612923540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612900850 0x55d612923540 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc68c02a930 tx=0x7fc68c0319a0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 -- 172.21.15.1:0/2592552585 shutdown_connections 2024-09-16T00:30:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d61290ba30 0x55d612905bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d612901170 0x55d612923a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2592552585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612900850 0x55d612923540 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 -- 172.21.15.1:0/2592552585 >> 172.21.15.1:0/2592552585 conn(0x55d6128fcde0 msgr2=0x55d61290a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 -- 172.21.15.1:0/2592552585 shutdown_connections 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 -- 172.21.15.1:0/2592552585 wait complete. 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 -- start start 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.555+0000 7fc6a3288740 1 Processor -- start 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 0x55d61290f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612901170 0x55d61290fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc6a3288740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d61290ba30 0x55d612910410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc6a3288740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c1af620 con 0x55d612901170 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc6a3288740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c1b0a90 con 0x55d612900850 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc6a3288740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c1afcf0 con 0x55d61290ba30 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc6a3288740 10 client.0 _ll_register_callbacks cb 0x7fc694005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc693fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612901170 0x55d61290fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc693fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612901170 0x55d61290fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53738/0 (socket says 172.21.15.1:53738) 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc693fff640 1 -- 172.21.15.1:0/2736044454 learned_addr learned my addr 172.21.15.1:0/2736044454 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 0x55d61290f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc69ad9d640 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d61290ba30 0x55d612910410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc693fff640 1 -- 172.21.15.1:0/2736044454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d61290ba30 msgr2=0x55d612910410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc693fff640 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d61290ba30 0x55d612910410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc693fff640 1 -- 172.21.15.1:0/2736044454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 msgr2=0x55d61290f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc693fff640 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 0x55d61290f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc693fff640 1 -- 172.21.15.1:0/2736044454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d612909940 con 0x55d612901170 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc693fff640 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612901170 0x55d61290fed0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc68400c6a0 tx=0x7fc6840d3840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ebe89e8c3b0697 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.556+0000 7fc681ffb640 1 -- 172.21.15.1:0/2736044454 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68409bf40 con 0x55d612901170 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc681ffb640 1 -- 172.21.15.1:0/2736044454 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc68409a700 con 0x55d612901170 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc6a3288740 1 -- 172.21.15.1:0/2736044454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68c005500 con 0x55d612901170 2024-09-16T00:30:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc681ffb640 1 -- 172.21.15.1:0/2736044454 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc68409c1d0 con 0x55d612901170 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc6a3288740 10 client.15822 fetch_fsmap learned FSMap version 25 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc6a3288740 1 -- 172.21.15.1:0/2736044454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d6129867e0 con 0x55d612901170 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc681ffb640 1 -- 172.21.15.1:0/2736044454 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc68c005500 con 0x55d612901170 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc681ffb640 1 -- 172.21.15.1:0/2736044454 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc68409c660 con 0x55d612901170 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc6a3288740 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc6a3288740 10 client.15822 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc6a3288740 20 client.15822 populate_metadata read hostname 'smithi001' 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d612986b10 0x55d612988f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc6a3288740 4 client.15822 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc6a3288740 1 -- 172.21.15.1:0/2736044454 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x55d6127e8f40 con 0x55d612986b10 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.557+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d612986b10 0x55d612988f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.558+0000 7fc69a59c640 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d612986b10 0x55d612988f30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4478b75990b91934 server_cookie=becfbe7e768a9cfb in_seq=0 out_seq=0 2024-09-16T00:30:58.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.558+0000 7fc681ffb640 10 client.15822 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc681ffb640 1 -- 172.21.15.1:0/2736044454 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55d6127e8f40 con 0x55d612986b10 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc681ffb640 10 client.15822 handle_command_reply: tid=0 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 client.15822 shutdown 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 -- 172.21.15.1:0/2736044454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612901170 msgr2=0x55d61290fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612901170 0x55d61290fed0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc68400c6a0 tx=0x7fc6840d3840 comp rx=0 tx=0).stop 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 -- 172.21.15.1:0/2736044454 shutdown_connections 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d612986b10 0x55d612988f30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc681ffb640 0 client.15822 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d61290ba30 0x55d612910410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d612901170 0x55d61290fed0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 --2- 172.21.15.1:0/2736044454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d612900850 0x55d61290f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 -- 172.21.15.1:0/2736044454 >> 172.21.15.1:0/2736044454 conn(0x55d6128fcde0 msgr2=0x55d61290a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 -- 172.21.15.1:0/2736044454 shutdown_connections 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 1 -- 172.21.15.1:0/2736044454 wait complete. 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a3288740 20 client.15822 trim_cache size 0 max 16384 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 6, 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "a19d02ed": { 2024-09-16T00:30:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 6, 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 139.6, 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:30:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:30:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:30:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:30:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:30:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:30:58.573 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:30:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6880a3240 msgr2=0x7fc6880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6880a3240 0x7fc6880a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc694005fd0 tx=0x7fc694004a40 comp rx=0 tx=0).stop 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c1027a0 msgr2=0x7fc69c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.569+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c1027a0 0x7fc69c074a30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc68400e9f0 tx=0x7fc68400edf0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.570+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/115636719 shutdown_connections 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.570+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6880a3240 0x7fc6880a5700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.570+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc69c106f20 0x7fc69c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.570+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c1027a0 0x7fc69c074a30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.570+0000 7fc6a0eaf640 1 --2- 172.21.15.1:0/115636719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c101df0 0x7fc69c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.570+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/115636719 >> 172.21.15.1:0/115636719 conn(0x7fc69c0fd8f0 msgr2=0x7fc69c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.570+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/115636719 shutdown_connections 2024-09-16T00:30:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:30:58.570+0000 7fc6a0eaf640 1 -- 172.21.15.1:0/115636719 wait complete. 2024-09-16T00:30:59.497 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a19d02ed', quiesce took 3.6 seconds. Will release after: 15.950000000000001 2024-09-16T00:31:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:00 smithi019 ceph-mon[35186]: pgmap v428: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:31:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:00 smithi001 ceph-mon[30934]: pgmap v428: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:31:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:00 smithi153 ceph-mon[35538]: pgmap v428: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:31:02.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:02 smithi019 ceph-mon[35186]: pgmap v429: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:31:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:02 smithi001 ceph-mon[30934]: pgmap v429: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:31:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:02 smithi153 ceph-mon[35538]: pgmap v429: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:31:04.360 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:04 smithi019 ceph-mon[35186]: pgmap v430: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:31:04.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:04 smithi001 ceph-mon[30934]: pgmap v430: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:31:04.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:04 smithi153 ceph-mon[35538]: pgmap v430: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:31:06.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:06 smithi019 ceph-mon[35186]: pgmap v431: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-09-16T00:31:06.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:06 smithi001 ceph-mon[30934]: pgmap v431: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-09-16T00:31:06.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:06 smithi153 ceph-mon[35538]: pgmap v431: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-09-16T00:31:08.165 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:31:08.436 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:08.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:08 smithi001 ceph-mon[30934]: pgmap v432: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 9 op/s 2024-09-16T00:31:08.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:08 smithi153 ceph-mon[35538]: pgmap v432: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 9 op/s 2024-09-16T00:31:08.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:08 smithi019 ceph-mon[35186]: pgmap v432: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 9 op/s 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.944+0000 7f1189251640 1 -- 172.21.15.1:0/2467294840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 msgr2=0x7f118410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.944+0000 7f1189251640 1 --2- 172.21.15.1:0/2467294840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 0x7f118410da40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1174009ca0 tx=0x7f117402a950 comp rx=0 tx=0).stop 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.944+0000 7f1189251640 1 -- 172.21.15.1:0/2467294840 shutdown_connections 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.944+0000 7f1189251640 1 --2- 172.21.15.1:0/2467294840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 0x7f118410da40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.944+0000 7f1189251640 1 --2- 172.21.15.1:0/2467294840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1184102570 0x7f11841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.944+0000 7f1189251640 1 --2- 172.21.15.1:0/2467294840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1184100e20 0x7f1184101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.944+0000 7f1189251640 1 -- 172.21.15.1:0/2467294840 >> 172.21.15.1:0/2467294840 conn(0x7f11840fc7d0 msgr2=0x7f11840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.944+0000 7f1189251640 1 -- 172.21.15.1:0/2467294840 shutdown_connections 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.944+0000 7f1189251640 1 -- 172.21.15.1:0/2467294840 wait complete. 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f1189251640 1 Processor -- start 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f1189251640 1 -- start start 2024-09-16T00:31:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f1189251640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1184100e20 0x7f1184198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f1189251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1184102570 0x7f1184198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f1189251640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 0x7f118419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f1189251640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1184110f80 con 0x7f1184102ef0 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f1189251640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1184110e00 con 0x7f1184102570 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f1189251640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1184111100 con 0x7f1184100e20 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f118359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 0x7f118419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f118359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 0x7f118419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49680/0 (socket says 172.21.15.1:49680) 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f118359e640 1 -- 172.21.15.1:0/513475586 learned_addr learned my addr 172.21.15.1:0/513475586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f1182d9d640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1184100e20 0x7f1184198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f118259c640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1184102570 0x7f1184198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.945+0000 7f118359e640 1 -- 172.21.15.1:0/513475586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1184100e20 msgr2=0x7f1184198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f118359e640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1184100e20 0x7f1184198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f118359e640 1 -- 172.21.15.1:0/513475586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1184102570 msgr2=0x7f1184198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f118359e640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1184102570 0x7f1184198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f118359e640 1 -- 172.21.15.1:0/513475586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118419d400 con 0x7f1184102ef0 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f118259c640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1184102570 0x7f1184198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f118359e640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 0x7f118419cd20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f117402ae70 tx=0x7f117402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f117bfff640 1 -- 172.21.15.1:0/513475586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174034040 con 0x7f1184102ef0 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f1189251640 1 -- 172.21.15.1:0/513475586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118419d690 con 0x7f1184102ef0 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f117bfff640 1 -- 172.21.15.1:0/513475586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1174030920 con 0x7f1184102ef0 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f117bfff640 1 -- 172.21.15.1:0/513475586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174045480 con 0x7f1184102ef0 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.946+0000 7f1189251640 1 -- 172.21.15.1:0/513475586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11841a4fd0 con 0x7f1184102ef0 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.948+0000 7f117bfff640 1 -- 172.21.15.1:0/513475586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f1174037020 con 0x7f1184102ef0 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.948+0000 7f118b62a740 1 Processor -- start 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118b62a740 1 -- start start 2024-09-16T00:31:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118b62a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd5064a0 0x5597cd506880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118b62a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd506dc0 0x5597cd511140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118b62a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd511680 0x5597cd513a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118b62a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5597cd46c2e0 con 0x5597cd511680 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118b62a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597cd413ce0 con 0x5597cd5064a0 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118b62a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5597cd4682a0 con 0x5597cd506dc0 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd511680 0x5597cd513a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f1182d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd506dc0 0x5597cd511140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd511680 0x5597cd513a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49688/0 (socket says 172.21.15.1:49688) 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f1182d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd506dc0 0x5597cd511140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:38458/0 (socket says 172.21.15.1:38458) 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118259c640 1 -- 172.21.15.1:0/486061193 learned_addr learned my addr 172.21.15.1:0/486061193 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f118259c640 1 --2- 172.21.15.1:0/486061193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd5064a0 0x5597cd506880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.949+0000 7f117bfff640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f11700a3240 0x7f11700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f118359e640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f11700a3240 0x7f11700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f1182d9d640 1 -- 172.21.15.1:0/486061193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd5064a0 msgr2=0x5597cd506880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f1182d9d640 1 --2- 172.21.15.1:0/486061193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd5064a0 0x5597cd506880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f1182d9d640 1 -- 172.21.15.1:0/486061193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd511680 msgr2=0x5597cd513a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f1182d9d640 1 --2- 172.21.15.1:0/486061193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd511680 0x5597cd513a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f1182d9d640 1 -- 172.21.15.1:0/486061193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597cd3f8440 con 0x5597cd506dc0 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f117bfff640 1 -- 172.21.15.1:0/513475586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f11740f0c80 con 0x7f1184102ef0 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f1182d9d640 1 --2- 172.21.15.1:0/486061193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd506dc0 0x5597cd511140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f117c00bbf0 tx=0x7f117c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=267aa0ccdfbbcb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f118359e640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f11700a3240 0x7f11700a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1174006010 tx=0x7f11740095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f11797fa640 1 -- 172.21.15.1:0/486061193 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c004030 con 0x5597cd506dc0 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f11797fa640 1 -- 172.21.15.1:0/486061193 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f117c0041d0 con 0x5597cd506dc0 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.950+0000 7f11797fa640 1 -- 172.21.15.1:0/486061193 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c0116a0 con 0x5597cd506dc0 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 -- 172.21.15.1:0/486061193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd506dc0 msgr2=0x5597cd511140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 --2- 172.21.15.1:0/486061193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd506dc0 0x5597cd511140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f117c00bbf0 tx=0x7f117c0107a0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 -- 172.21.15.1:0/486061193 shutdown_connections 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 --2- 172.21.15.1:0/486061193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd511680 0x5597cd513a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 --2- 172.21.15.1:0/486061193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd506dc0 0x5597cd511140 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 --2- 172.21.15.1:0/486061193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd5064a0 0x5597cd506880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 -- 172.21.15.1:0/486061193 >> 172.21.15.1:0/486061193 conn(0x5597cd502a30 msgr2=0x5597cd502e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 -- 172.21.15.1:0/486061193 shutdown_connections 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 -- 172.21.15.1:0/486061193 wait complete. 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 -- start start 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.951+0000 7f118b62a740 1 Processor -- start 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118b62a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd5064a0 0x5597cd510fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118b62a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd506dc0 0x5597cd50e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118b62a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd511680 0x5597cd50e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118b62a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5597cd514250 con 0x5597cd5064a0 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118b62a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597cd427050 con 0x5597cd506dc0 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118b62a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5597cd5143d0 con 0x5597cd511680 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118b62a740 10 client.0 _ll_register_callbacks cb 0x5597cd467160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd5064a0 0x5597cd510fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd5064a0 0x5597cd510fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49690/0 (socket says 172.21.15.1:49690) 2024-09-16T00:31:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118259c640 1 -- 172.21.15.1:0/3838012856 learned_addr learned my addr 172.21.15.1:0/3838012856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f1182d9d640 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd506dc0 0x5597cd50e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118259c640 1 -- 172.21.15.1:0/3838012856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd511680 msgr2=0x5597cd50e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118259c640 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd511680 0x5597cd50e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118259c640 1 -- 172.21.15.1:0/3838012856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd506dc0 msgr2=0x5597cd50e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118259c640 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd506dc0 0x5597cd50e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118259c640 1 -- 172.21.15.1:0/3838012856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597cd1dd3c0 con 0x5597cd5064a0 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f118259c640 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd5064a0 0x5597cd510fa0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f116c00d810 tx=0x7f116c012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa04182e5dbca9e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f116a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.952+0000 7f116a7fc640 1 -- 172.21.15.1:0/3838012856 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c005020 con 0x5597cd5064a0 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f116a7fc640 1 -- 172.21.15.1:0/3838012856 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f116c0051c0 con 0x5597cd5064a0 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f118b62a740 1 -- 172.21.15.1:0/3838012856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f117c0041d0 con 0x5597cd5064a0 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f116a7fc640 1 -- 172.21.15.1:0/3838012856 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f116c027050 con 0x5597cd5064a0 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f118b62a740 10 client.15834 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f118b62a740 1 -- 172.21.15.1:0/3838012856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597cd51fc20 con 0x5597cd5064a0 2024-09-16T00:31:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f116a7fc640 1 -- 172.21.15.1:0/3838012856 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f117c0041d0 con 0x5597cd5064a0 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f116a7fc640 1 -- 172.21.15.1:0/3838012856 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f116c01c420 con 0x5597cd5064a0 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f118b62a740 10 client.15834 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f118b62a740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f118b62a740 20 client.15834 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f118b62a740 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5597cd5210e0 0x5597cd5234f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f118b62a740 4 client.15834 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f118b62a740 1 -- 172.21.15.1:0/3838012856 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597cd479850 con 0x5597cd5210e0 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.953+0000 7f1182d9d640 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5597cd5210e0 0x5597cd5234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.954+0000 7f1182d9d640 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5597cd5210e0 0x5597cd5234f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83fa8239570f2265 server_cookie=ad51020b8d7938cb in_seq=0 out_seq=0 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.954+0000 7f116a7fc640 10 client.15834 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.954+0000 7f116a7fc640 1 -- 172.21.15.1:0/3838012856 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597cd479850 con 0x5597cd5210e0 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.954+0000 7f116a7fc640 10 client.15834 handle_command_reply: tid=0 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.954+0000 7f118b62a740 1 client.15834 shutdown 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 -- 172.21.15.1:0/3838012856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd5064a0 msgr2=0x5597cd510fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd5064a0 0x5597cd510fa0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f116c00d810 tx=0x7f116c012f80 comp rx=0 tx=0).stop 2024-09-16T00:31:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 -- 172.21.15.1:0/3838012856 shutdown_connections 2024-09-16T00:31:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5597cd5210e0 0x5597cd5234f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd511680 0x5597cd50e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd506dc0 0x5597cd50e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f116a7fc640 0 client.15834 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 --2- 172.21.15.1:0/3838012856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd5064a0 0x5597cd510fa0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 -- 172.21.15.1:0/3838012856 >> 172.21.15.1:0/3838012856 conn(0x5597cd502a30 msgr2=0x5597cd508710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 -- 172.21.15.1:0/3838012856 shutdown_connections 2024-09-16T00:31:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 1 -- 172.21.15.1:0/3838012856 wait complete. 2024-09-16T00:31:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.955+0000 7f118b62a740 20 client.15834 trim_cache size 0 max 16384 2024-09-16T00:31:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.972+0000 7f118b62a740 1 Processor -- start 2024-09-16T00:31:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.972+0000 7f118b62a740 1 -- start start 2024-09-16T00:31:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.972+0000 7f118b62a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 0x5597cd501a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.972+0000 7f118b62a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd528dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.972+0000 7f118b62a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 0x5597cd529300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.972+0000 7f118b62a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f11841b1530 con 0x5597cd506dc0 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.972+0000 7f118b62a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11841aee00 con 0x5597cd511680 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.972+0000 7f118b62a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11841acc50 con 0x5597cd5064a0 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.972+0000 7f1182d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd528dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1182d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd528dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49702/0 (socket says 172.21.15.1:49702) 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1182d9d640 1 -- 172.21.15.1:0/290659819 learned_addr learned my addr 172.21.15.1:0/290659819 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f118259c640 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 0x5597cd529300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f118259c640 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 0x5597cd501a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1182d9d640 1 -- 172.21.15.1:0/290659819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 msgr2=0x5597cd501a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1182d9d640 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 0x5597cd501a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1182d9d640 1 -- 172.21.15.1:0/290659819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 msgr2=0x5597cd529300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1182d9d640 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 0x5597cd529300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1182d9d640 1 -- 172.21.15.1:0/290659819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597cd5062d0 con 0x5597cd506dc0 2024-09-16T00:31:08.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f118259c640 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 0x5597cd529300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1182d9d640 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd528dc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f117c00bb10 tx=0x7f117c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd8508171ba3c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1178ff9640 1 -- 172.21.15.1:0/290659819 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c0052c0 con 0x5597cd506dc0 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1178ff9640 1 -- 172.21.15.1:0/290659819 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f117c0035f0 con 0x5597cd506dc0 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.973+0000 7f1178ff9640 1 -- 172.21.15.1:0/290659819 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c011870 con 0x5597cd506dc0 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 -- 172.21.15.1:0/290659819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 msgr2=0x5597cd528dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd528dc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f117c00bb10 tx=0x7f117c00b390 comp rx=0 tx=0).stop 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 -- 172.21.15.1:0/290659819 shutdown_connections 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 0x5597cd529300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd528dc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 --2- 172.21.15.1:0/290659819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 0x5597cd501a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 -- 172.21.15.1:0/290659819 >> 172.21.15.1:0/290659819 conn(0x5597cd502a30 msgr2=0x5597cd501350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 -- 172.21.15.1:0/290659819 shutdown_connections 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 -- 172.21.15.1:0/290659819 wait complete. 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 -- start start 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.974+0000 7f118b62a740 1 Processor -- start 2024-09-16T00:31:08.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118b62a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 0x5597cd5199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118b62a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd519f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118b62a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 0x5597cd516f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118b62a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f11841ad2c0 con 0x5597cd506dc0 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118b62a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11841ae730 con 0x5597cd511680 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118b62a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11841ad990 con 0x5597cd5064a0 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118b62a740 10 client.0 _ll_register_callbacks cb 0x7f116c005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f1182d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd519f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f1182d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd519f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49704/0 (socket says 172.21.15.1:49704) 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f1182d9d640 1 -- 172.21.15.1:0/3497374301 learned_addr learned my addr 172.21.15.1:0/3497374301 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118259c640 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 0x5597cd516f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118259c640 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 0x5597cd5199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f1182d9d640 1 -- 172.21.15.1:0/3497374301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 msgr2=0x5597cd5199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f1182d9d640 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 0x5597cd5199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f1182d9d640 1 -- 172.21.15.1:0/3497374301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 msgr2=0x5597cd516f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f1182d9d640 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 0x5597cd516f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f1182d9d640 1 -- 172.21.15.1:0/3497374301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597cd517740 con 0x5597cd506dc0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f118259c640 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 0x5597cd5199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f1182d9d640 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd519f00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f117c011010 tx=0x7f117c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a6cd0cbb26b0c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.975+0000 7f116a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f116a7fc640 1 -- 172.21.15.1:0/3497374301 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c003cb0 con 0x5597cd506dc0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f116a7fc640 1 -- 172.21.15.1:0/3497374301 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f117c005050 con 0x5597cd506dc0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f118b62a740 1 -- 172.21.15.1:0/3497374301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f117c0035f0 con 0x5597cd506dc0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f116a7fc640 1 -- 172.21.15.1:0/3497374301 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f117c041050 con 0x5597cd506dc0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f118b62a740 10 client.15840 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f118b62a740 1 -- 172.21.15.1:0/3497374301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597cd58ba10 con 0x5597cd506dc0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f116a7fc640 1 -- 172.21.15.1:0/3497374301 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f117c0035f0 con 0x5597cd506dc0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f116a7fc640 1 -- 172.21.15.1:0/3497374301 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f117c03b4b0 con 0x5597cd506dc0 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f118b62a740 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f118b62a740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f118b62a740 20 client.15840 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f118b62a740 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5597cd58bf30 0x5597cd58e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f118b62a740 4 client.15840 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.976+0000 7f118b62a740 1 -- 172.21.15.1:0/3497374301 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597cd50eb20 con 0x5597cd58bf30 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118259c640 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5597cd58bf30 0x5597cd58e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118259c640 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5597cd58bf30 0x5597cd58e350 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e41f78d20bf01ed6 server_cookie=23171e8a79505ad4 in_seq=0 out_seq=0 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f116a7fc640 10 client.15840 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f116a7fc640 1 -- 172.21.15.1:0/3497374301 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5597cd50eb20 con 0x5597cd58bf30 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f116a7fc640 10 client.15840 handle_command_reply: tid=0 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 client.15840 shutdown 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 -- 172.21.15.1:0/3497374301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 msgr2=0x5597cd519f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd519f00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f117c011010 tx=0x7f117c00bea0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 -- 172.21.15.1:0/3497374301 shutdown_connections 2024-09-16T00:31:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5597cd58bf30 0x5597cd58e350 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597cd511680 0x5597cd516f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597cd506dc0 0x5597cd519f00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f116a7fc640 0 client.15840 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 --2- 172.21.15.1:0/3497374301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597cd5064a0 0x5597cd5199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 -- 172.21.15.1:0/3497374301 >> 172.21.15.1:0/3497374301 conn(0x5597cd502a30 msgr2=0x5597cd501350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 -- 172.21.15.1:0/3497374301 shutdown_connections 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.977+0000 7f118b62a740 1 -- 172.21.15.1:0/3497374301 wait complete. 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f118b62a740 20 client.15840 trim_cache size 0 max 16384 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:31:08.988 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:31:08.989 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:31:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 -- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f11700a3240 msgr2=0x7f11700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f11700a3240 0x7f11700a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1174006010 tx=0x7f11740095c0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 -- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 msgr2=0x7f118419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 0x7f118419cd20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f117402ae70 tx=0x7f117402f5d0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 -- 172.21.15.1:0/513475586 shutdown_connections 2024-09-16T00:31:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1184102ef0 0x7f118419cd20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1184102570 0x7f1184198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f11700a3240 0x7f11700a5700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 --2- 172.21.15.1:0/513475586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1184100e20 0x7f1184198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 -- 172.21.15.1:0/513475586 >> 172.21.15.1:0/513475586 conn(0x7f11840fc7d0 msgr2=0x7f118406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 -- 172.21.15.1:0/513475586 shutdown_connections 2024-09-16T00:31:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:08.978+0000 7f1189251640 1 -- 172.21.15.1:0/513475586 wait complete. 2024-09-16T00:31:09.800 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:31:09.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:31:10.061 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:10.367 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:10 smithi001 ceph-mon[30934]: pgmap v433: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-09-16T00:31:10.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:10 smithi153 ceph-mon[35538]: pgmap v433: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-09-16T00:31:10.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:10 smithi019 ceph-mon[35186]: pgmap v433: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-09-16T00:31:10.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.595+0000 7f1dfad16640 1 -- 172.21.15.1:0/2805146242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4102790 msgr2=0x7f1df41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.595+0000 7f1dfad16640 1 --2- 172.21.15.1:0/2805146242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4102790 0x7f1df41067e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009900 tx=0x7f1de002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.595+0000 7f1dfad16640 1 -- 172.21.15.1:0/2805146242 shutdown_connections 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.595+0000 7f1dfad16640 1 --2- 172.21.15.1:0/2805146242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df4106f10 0x7f1df410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.595+0000 7f1dfad16640 1 --2- 172.21.15.1:0/2805146242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4102790 0x7f1df41067e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.595+0000 7f1dfad16640 1 --2- 172.21.15.1:0/2805146242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4101de0 0x7f1df41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.595+0000 7f1dfad16640 1 -- 172.21.15.1:0/2805146242 >> 172.21.15.1:0/2805146242 conn(0x7f1df40fd8c0 msgr2=0x7f1df40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.595+0000 7f1dfad16640 1 -- 172.21.15.1:0/2805146242 shutdown_connections 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.595+0000 7f1dfad16640 1 -- 172.21.15.1:0/2805146242 wait complete. 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1dfad16640 1 Processor -- start 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1dfad16640 1 -- start start 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1dfad16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4101de0 0x7f1df419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1dfad16640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df4102790 0x7f1df419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1dfad16640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4106f10 0x7f1df419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1dfad16640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4111f70 con 0x7f1df4106f10 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1dfad16640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4111df0 con 0x7f1df4101de0 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1dfad16640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1df41120f0 con 0x7f1df4102790 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1df92b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4106f10 0x7f1df419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1df92b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4106f10 0x7f1df419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52870/0 (socket says 172.21.15.1:52870) 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.596+0000 7f1df92b3640 1 -- 172.21.15.1:0/1765089450 learned_addr learned my addr 172.21.15.1:0/1765089450 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df8ab2640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4101de0 0x7f1df419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df3fff640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df4102790 0x7f1df419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df92b3640 1 -- 172.21.15.1:0/1765089450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df4102790 msgr2=0x7f1df419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df92b3640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df4102790 0x7f1df419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df92b3640 1 -- 172.21.15.1:0/1765089450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4101de0 msgr2=0x7f1df419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df92b3640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4101de0 0x7f1df419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df92b3640 1 -- 172.21.15.1:0/1765089450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df419f5f0 con 0x7f1df4106f10 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df3fff640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df4102790 0x7f1df419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df92b3640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4106f10 0x7f1df419ef10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00cad0 tx=0x7f1dec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df1ffb640 1 -- 172.21.15.1:0/1765089450 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00ae20 con 0x7f1df4106f10 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df419f8e0 con 0x7f1df4106f10 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df1ffb640 1 -- 172.21.15.1:0/1765089450 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dec005020 con 0x7f1df4106f10 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1df1ffb640 1 -- 172.21.15.1:0/1765089450 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00f6b0 con 0x7f1df4106f10 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.597+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df41a71c0 con 0x7f1df4106f10 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.598+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df41021c0 con 0x7f1df4106f10 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.599+0000 7f1df1ffb640 1 -- 172.21.15.1:0/1765089450 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f1dec0051d0 con 0x7f1df4106f10 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.600+0000 7f1df1ffb640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1de40a3240 0x7f1de40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.600+0000 7f1df8ab2640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1de40a3240 0x7f1de40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.600+0000 7f1df1ffb640 1 -- 172.21.15.1:0/1765089450 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1dec0d00a0 con 0x7f1df4106f10 2024-09-16T00:31:10.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.601+0000 7f1df8ab2640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1de40a3240 0x7f1de40a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005e10 tx=0x7f1de8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:10.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.604+0000 7f1df1ffb640 1 -- 172.21.15.1:0/1765089450 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1dec014030 con 0x7f1df4106f10 2024-09-16T00:31:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.814+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1df41a7420 con 0x7f1df4106f10 2024-09-16T00:31:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.815+0000 7f1df1ffb640 1 -- 172.21.15.1:0/1765089450 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f1dec09c9c0 con 0x7f1df4106f10 2024-09-16T00:31:10.816 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:31:10.816 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:31:10.817 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:31:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1de40a3240 msgr2=0x7f1de40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1de40a3240 0x7f1de40a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005e10 tx=0x7f1de8005da0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4106f10 msgr2=0x7f1df419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4106f10 0x7f1df419ef10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00cad0 tx=0x7f1dec00cf90 comp rx=0 tx=0).stop 2024-09-16T00:31:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 shutdown_connections 2024-09-16T00:31:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1df4106f10 0x7f1df419ef10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1de40a3240 0x7f1de40a5700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df4102790 0x7f1df419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 --2- 172.21.15.1:0/1765089450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4101de0 0x7f1df419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 >> 172.21.15.1:0/1765089450 conn(0x7f1df40fd8c0 msgr2=0x7f1df40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 shutdown_connections 2024-09-16T00:31:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:10.817+0000 7f1dfad16640 1 -- 172.21.15.1:0/1765089450 wait complete. 2024-09-16T00:31:11.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:11 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1765089450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:11.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:11 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1765089450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:11.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:11 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1765089450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:11.638 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:31:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:12.317 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:12 smithi001 ceph-mon[30934]: pgmap v434: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.398+0000 7f46784c0640 1 -- 172.21.15.1:0/1585709140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4670102d40 msgr2=0x7f4670103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.398+0000 7f46784c0640 1 --2- 172.21.15.1:0/1585709140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4670102d40 0x7f4670103120 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f466c009900 tx=0x7f466c02a480 comp rx=0 tx=0).stop 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 -- 172.21.15.1:0/1585709140 shutdown_connections 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 --2- 172.21.15.1:0/1585709140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467010c170 0x7f467010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 --2- 172.21.15.1:0/1585709140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4670103660 0x7f467010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 --2- 172.21.15.1:0/1585709140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4670102d40 0x7f4670103120 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 -- 172.21.15.1:0/1585709140 >> 172.21.15.1:0/1585709140 conn(0x7f46700fc7a0 msgr2=0x7f46700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 -- 172.21.15.1:0/1585709140 shutdown_connections 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 -- 172.21.15.1:0/1585709140 wait complete. 2024-09-16T00:31:12.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 Processor -- start 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 -- start start 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.399+0000 7f46784c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670102d40 0x7f4670198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f46784c0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4670103660 0x7f4670198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f46784c0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f467010c170 0x7f467019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f46784c0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4670112030 con 0x7f467010c170 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f46784c0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4670111eb0 con 0x7f4670102d40 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f46784c0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46701121b0 con 0x7f4670103660 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f4676a5d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f467010c170 0x7f467019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f467625c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670102d40 0x7f4670198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f467625c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670102d40 0x7f4670198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:60842/0 (socket says 172.21.15.1:60842) 2024-09-16T00:31:12.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f4676a5d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f467010c170 0x7f467019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52880/0 (socket says 172.21.15.1:52880) 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f467625c640 1 -- 172.21.15.1:0/651179800 learned_addr learned my addr 172.21.15.1:0/651179800 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f4675a5b640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4670103660 0x7f4670198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f4676a5d640 1 -- 172.21.15.1:0/651179800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4670103660 msgr2=0x7f4670198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f4676a5d640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4670103660 0x7f4670198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f4676a5d640 1 -- 172.21.15.1:0/651179800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670102d40 msgr2=0x7f4670198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f4676a5d640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670102d40 0x7f4670198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f4676a5d640 1 -- 172.21.15.1:0/651179800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467019d640 con 0x7f467010c170 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f467625c640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670102d40 0x7f4670198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f4676a5d640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f467010c170 0x7f467019cf60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f466800be20 tx=0x7f466800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f46677fe640 1 -- 172.21.15.1:0/651179800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668019070 con 0x7f467010c170 2024-09-16T00:31:12.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f46677fe640 1 -- 172.21.15.1:0/651179800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4668014920 con 0x7f467010c170 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.400+0000 7f46784c0640 1 -- 172.21.15.1:0/651179800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467019d930 con 0x7f467010c170 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.401+0000 7f46677fe640 1 -- 172.21.15.1:0/651179800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668012920 con 0x7f467010c170 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.401+0000 7f46784c0640 1 -- 172.21.15.1:0/651179800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46701a5210 con 0x7f467010c170 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.403+0000 7f46677fe640 1 -- 172.21.15.1:0/651179800 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f466801c070 con 0x7f467010c170 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.403+0000 7f467a899740 1 Processor -- start 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.403+0000 7f467a899740 1 -- start start 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f467a899740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af8b40 0x55b945af7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f467a899740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af75d0 0x55b945af7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f467a899740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945b02370 0x55b945b04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f467a899740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b945a5d2f0 con 0x55b945af8b40 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f467a899740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b945a04cf0 con 0x55b945af75d0 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f467a899740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b945a592b0 con 0x55b945b02370 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f46677fe640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f465c0a3240 0x7f465c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af8b40 0x55b945af7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af8b40 0x55b945af7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52888/0 (socket says 172.21.15.1:52888) 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 -- 172.21.15.1:0/1502628564 learned_addr learned my addr 172.21.15.1:0/1502628564 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f467625c640 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af75d0 0x55b945af7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4676a5d640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f465c0a3240 0x7f465c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945b02370 0x55b945b04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 -- 172.21.15.1:0/1502628564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945b02370 msgr2=0x55b945b04760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945b02370 0x55b945b04760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 -- 172.21.15.1:0/1502628564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af75d0 msgr2=0x55b945af7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af75d0 0x55b945af7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f46677fe640 1 -- 172.21.15.1:0/651179800 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f46680cf5d0 con 0x7f467010c170 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 -- 172.21.15.1:0/1502628564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9459e93a0 con 0x55b945af8b40 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f467625c640 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af75d0 0x55b945af7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4675a5b640 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af8b40 0x55b945af7090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4660006aa0 tx=0x7f46600095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed1594149321bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.404+0000 7f4676a5d640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f465c0a3240 0x7f465c0a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f46680962e0 tx=0x7f466806e6c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f4664ff9640 1 -- 172.21.15.1:0/1502628564 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660015620 con 0x55b945af8b40 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f4664ff9640 1 -- 172.21.15.1:0/1502628564 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466001a070 con 0x55b945af8b40 2024-09-16T00:31:12.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f4664ff9640 1 -- 172.21.15.1:0/1502628564 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660014880 con 0x55b945af8b40 2024-09-16T00:31:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f467a899740 1 -- 172.21.15.1:0/1502628564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af8b40 msgr2=0x55b945af7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f467a899740 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af8b40 0x55b945af7090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4660006aa0 tx=0x7f46600095c0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f467a899740 1 -- 172.21.15.1:0/1502628564 shutdown_connections 2024-09-16T00:31:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f467a899740 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945b02370 0x55b945b04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f467a899740 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af75d0 0x55b945af7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f467a899740 1 --2- 172.21.15.1:0/1502628564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af8b40 0x55b945af7090 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f467a899740 1 -- 172.21.15.1:0/1502628564 >> 172.21.15.1:0/1502628564 conn(0x55b945ac0310 msgr2=0x55b945af3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f467a899740 1 -- 172.21.15.1:0/1502628564 shutdown_connections 2024-09-16T00:31:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.405+0000 7f467a899740 1 -- 172.21.15.1:0/1502628564 wait complete. 2024-09-16T00:31:12.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f467a899740 1 -- start start 2024-09-16T00:31:12.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f467a899740 1 Processor -- start 2024-09-16T00:31:12.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f467a899740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b01cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f467a899740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945af8b40 0x55b945afef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f467a899740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945b02370 0x55b945aff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f467a899740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b945b05010 con 0x55b945af75d0 2024-09-16T00:31:12.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f467a899740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b945a18060 con 0x55b945b02370 2024-09-16T00:31:12.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f467a899740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b945b05190 con 0x55b945af8b40 2024-09-16T00:31:12.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f467a899740 10 client.0 _ll_register_callbacks cb 0x55b94576b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f4675a5b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b01cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f4675a5b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b01cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52896/0 (socket says 172.21.15.1:52896) 2024-09-16T00:31:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.406+0000 7f4675a5b640 1 -- 172.21.15.1:0/3665858040 learned_addr learned my addr 172.21.15.1:0/3665858040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4675a5b640 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945b02370 0x55b945aff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f467625c640 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945af8b40 0x55b945afef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4675a5b640 1 -- 172.21.15.1:0/3665858040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945af8b40 msgr2=0x55b945afef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4675a5b640 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945af8b40 0x55b945afef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4675a5b640 1 -- 172.21.15.1:0/3665858040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945b02370 msgr2=0x55b945aff4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4675a5b640 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945b02370 0x55b945aff4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4675a5b640 1 -- 172.21.15.1:0/3665858040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9457ce3c0 con 0x55b945af75d0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4675a5b640 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b01cd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4660009280 tx=0x7f46600091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c760aee41b7a3b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4659ffb640 1 -- 172.21.15.1:0/3665858040 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660015b30 con 0x55b945af75d0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f467a899740 1 -- 172.21.15.1:0/3665858040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f466001a070 con 0x55b945af75d0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4659ffb640 1 -- 172.21.15.1:0/3665858040 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f46600145c0 con 0x55b945af75d0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4659ffb640 1 -- 172.21.15.1:0/3665858040 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4660009890 con 0x55b945af75d0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f467a899740 10 client.15858 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f467a899740 1 -- 172.21.15.1:0/3665858040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b945b10a20 con 0x55b945af75d0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.407+0000 7f4659ffb640 1 -- 172.21.15.1:0/3665858040 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466001a070 con 0x55b945af75d0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f4659ffb640 1 -- 172.21.15.1:0/3665858040 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4660022920 con 0x55b945af75d0 2024-09-16T00:31:12.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f467a899740 10 client.15858 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f467a899740 10 client.15858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f467a899740 20 client.15858 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f467a899740 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b945b11e30 0x55b945b14220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f467a899740 4 client.15858 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f467a899740 1 -- 172.21.15.1:0/3665858040 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b945a6a860 con 0x55b945b11e30 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f467625c640 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b945b11e30 0x55b945b14220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f467625c640 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b945b11e30 0x55b945b14220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8370cf5cb8e1933 server_cookie=12da9667ce27070e in_seq=0 out_seq=0 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.408+0000 7f4659ffb640 10 client.15858 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f4659ffb640 1 -- 172.21.15.1:0/3665858040 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b945a6a860 con 0x55b945b11e30 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f4659ffb640 10 client.15858 handle_command_reply: tid=0 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 client.15858 shutdown 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 -- 172.21.15.1:0/3665858040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 msgr2=0x55b945b01cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b01cd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4660009280 tx=0x7f46600091d0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 -- 172.21.15.1:0/3665858040 shutdown_connections 2024-09-16T00:31:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b945b11e30 0x55b945b14220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945b02370 0x55b945aff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945af8b40 0x55b945afef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 --2- 172.21.15.1:0/3665858040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b01cd0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f4659ffb640 0 client.15858 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 -- 172.21.15.1:0/3665858040 >> 172.21.15.1:0/3665858040 conn(0x55b945ac0310 msgr2=0x55b945af9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.409+0000 7f467a899740 1 -- 172.21.15.1:0/3665858040 shutdown_connections 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.410+0000 7f467a899740 1 -- 172.21.15.1:0/3665858040 wait complete. 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.410+0000 7f467a899740 20 client.15858 trim_cache size 0 max 16384 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.422+0000 7f467a899740 1 Processor -- start 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.422+0000 7f467a899740 1 -- start start 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f467a899740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945af2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f4675a5b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945af2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f4675a5b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945af2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52910/0 (socket says 172.21.15.1:52910) 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f4675a5b640 1 -- 172.21.15.1:0/3034462513 learned_addr learned my addr 172.21.15.1:0/3034462513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f467a899740 1 --2- 172.21.15.1:0/3034462513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af8b40 0x55b945afa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f467a899740 1 --2- 172.21.15.1:0/3034462513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945b02370 0x55b945afa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f467a899740 1 -- 172.21.15.1:0/3034462513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b17c0 con 0x55b945af75d0 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f467a899740 1 -- 172.21.15.1:0/3034462513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46701af090 con 0x55b945af8b40 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f467a899740 1 -- 172.21.15.1:0/3034462513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46701acee0 con 0x55b945b02370 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f4675a5b640 1 -- 172.21.15.1:0/3034462513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945b02370 msgr2=0x55b945afa660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f4675a5b640 1 --2- 172.21.15.1:0/3034462513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945b02370 0x55b945afa660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f4675a5b640 1 -- 172.21.15.1:0/3034462513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af8b40 msgr2=0x55b945afa120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f4675a5b640 1 --2- 172.21.15.1:0/3034462513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af8b40 0x55b945afa120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.423+0000 7f4675a5b640 1 -- 172.21.15.1:0/3034462513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b945afd4d0 con 0x55b945af75d0 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.424+0000 7f4675a5b640 1 --2- 172.21.15.1:0/3034462513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945af2500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4660014470 tx=0x7f4660010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737b1e1f6202924e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.424+0000 7f465bfff640 1 -- 172.21.15.1:0/3034462513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660037070 con 0x55b945af75d0 2024-09-16T00:31:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.424+0000 7f465bfff640 1 -- 172.21.15.1:0/3034462513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466001a050 con 0x55b945af75d0 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.424+0000 7f465bfff640 1 -- 172.21.15.1:0/3034462513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660032600 con 0x55b945af75d0 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 -- 172.21.15.1:0/3034462513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 msgr2=0x55b945af2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 --2- 172.21.15.1:0/3034462513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945af2500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4660014470 tx=0x7f4660010e10 comp rx=0 tx=0).stop 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 -- 172.21.15.1:0/3034462513 shutdown_connections 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 --2- 172.21.15.1:0/3034462513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945b02370 0x55b945afa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 --2- 172.21.15.1:0/3034462513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945af8b40 0x55b945afa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 --2- 172.21.15.1:0/3034462513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945af2500 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 -- 172.21.15.1:0/3034462513 >> 172.21.15.1:0/3034462513 conn(0x55b945ac0310 msgr2=0x55b945afdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 -- 172.21.15.1:0/3034462513 shutdown_connections 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 -- 172.21.15.1:0/3034462513 wait complete. 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 -- start start 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 Processor -- start 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b151a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945af8b40 0x55b945b156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945b02370 0x55b945b15c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ad550 con 0x55b945af75d0 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ae9c0 con 0x55b945b02370 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.425+0000 7f467a899740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46701adc20 con 0x55b945af8b40 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f467a899740 10 client.0 _ll_register_callbacks cb 0x7f466c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4675a5b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4675a5b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b151a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52912/0 (socket says 172.21.15.1:52912) 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4675a5b640 1 -- 172.21.15.1:0/1478739594 learned_addr learned my addr 172.21.15.1:0/1478739594 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4675a5b640 1 -- 172.21.15.1:0/1478739594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945af8b40 msgr2=0x55b945b156e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4675a5b640 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945af8b40 0x55b945b156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4675a5b640 1 -- 172.21.15.1:0/1478739594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945b02370 msgr2=0x55b945b15c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4675a5b640 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945b02370 0x55b945b15c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4675a5b640 1 -- 172.21.15.1:0/1478739594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b945b7b910 con 0x55b945af75d0 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4675a5b640 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b151a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4660000c00 tx=0x7f4660006e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9706733e74d8b206 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4659ffb640 1 -- 172.21.15.1:0/1478739594 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660040060 con 0x55b945af75d0 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4659ffb640 1 -- 172.21.15.1:0/1478739594 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4660009430 con 0x55b945af75d0 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f467a899740 1 -- 172.21.15.1:0/1478739594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f466001a050 con 0x55b945af75d0 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f4659ffb640 1 -- 172.21.15.1:0/1478739594 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4660047050 con 0x55b945af75d0 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f467a899740 10 client.15864 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.426+0000 7f467a899740 1 -- 172.21.15.1:0/1478739594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b945b7cbd0 con 0x55b945af75d0 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.427+0000 7f4659ffb640 1 -- 172.21.15.1:0/1478739594 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466001a050 con 0x55b945af75d0 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.427+0000 7f4659ffb640 1 -- 172.21.15.1:0/1478739594 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f466001e4e0 con 0x55b945af75d0 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.427+0000 7f467a899740 10 client.15864 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.427+0000 7f467a899740 10 client.15864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.427+0000 7f467a899740 20 client.15864 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.427+0000 7f467a899740 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b945b7d230 0x55b945b7f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.427+0000 7f467a899740 4 client.15864 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:31:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.427+0000 7f467a899740 1 -- 172.21.15.1:0/1478739594 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b945af33b0 con 0x55b945b7d230 2024-09-16T00:31:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.427+0000 7f467625c640 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b945b7d230 0x55b945b7f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f4659ffb640 10 client.15864 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:31:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467625c640 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b945b7d230 0x55b945b7f620 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8d6cd1039093100 server_cookie=1b4a335adbbac6f3 in_seq=0 out_seq=0 2024-09-16T00:31:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f4659ffb640 1 -- 172.21.15.1:0/1478739594 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b945af33b0 con 0x55b945b7d230 2024-09-16T00:31:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f4659ffb640 10 client.15864 handle_command_reply: tid=0 2024-09-16T00:31:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467a899740 1 client.15864 shutdown 2024-09-16T00:31:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467a899740 1 -- 172.21.15.1:0/1478739594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 msgr2=0x55b945b151a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467a899740 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b151a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4660000c00 tx=0x7f4660006e40 comp rx=0 tx=0).stop 2024-09-16T00:31:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467a899740 1 -- 172.21.15.1:0/1478739594 shutdown_connections 2024-09-16T00:31:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467a899740 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b945b7d230 0x55b945b7f620 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467a899740 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b945b02370 0x55b945b15c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467a899740 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b945af8b40 0x55b945b156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467a899740 1 --2- 172.21.15.1:0/1478739594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b945af75d0 0x55b945b151a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f467a899740 1 -- 172.21.15.1:0/1478739594 >> 172.21.15.1:0/1478739594 conn(0x55b945ac0310 msgr2=0x55b945b007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.428+0000 7f4659ffb640 0 client.15864 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:31:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f467a899740 1 -- 172.21.15.1:0/1478739594 shutdown_connections 2024-09-16T00:31:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f467a899740 1 -- 172.21.15.1:0/1478739594 wait complete. 2024-09-16T00:31:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f467a899740 20 client.15864 trim_cache size 0 max 16384 2024-09-16T00:31:12.447 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:31:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 -- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f465c0a3240 msgr2=0x7f465c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f465c0a3240 0x7f465c0a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f46680962e0 tx=0x7f466806e6c0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 -- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f467010c170 msgr2=0x7f467019cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f467010c170 0x7f467019cf60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f466800be20 tx=0x7f466800bf10 comp rx=0 tx=0).stop 2024-09-16T00:31:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 -- 172.21.15.1:0/651179800 shutdown_connections 2024-09-16T00:31:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f467010c170 0x7f467019cf60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f465c0a3240 0x7f465c0a5700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4670103660 0x7f4670198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 --2- 172.21.15.1:0/651179800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670102d40 0x7f4670198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 -- 172.21.15.1:0/651179800 >> 172.21.15.1:0/651179800 conn(0x7f46700fc7a0 msgr2=0x7f46700fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 -- 172.21.15.1:0/651179800 shutdown_connections 2024-09-16T00:31:12.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:12.429+0000 7f46784c0640 1 -- 172.21.15.1:0/651179800 wait complete. 2024-09-16T00:31:12.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:12 smithi153 ceph-mon[35538]: pgmap v434: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:31:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:12 smithi019 ceph-mon[35186]: pgmap v434: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:31:13.344 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:31:13.608 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:14.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.135+0000 7ff0ace3b640 1 -- 172.21.15.1:0/3247507069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 msgr2=0x7ff0a810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.135+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/3247507069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 0x7ff0a810bb50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff090009900 tx=0x7ff09002a480 comp rx=0 tx=0).stop 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.135+0000 7ff0ace3b640 1 -- 172.21.15.1:0/3247507069 shutdown_connections 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.135+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/3247507069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0a810c170 0x7ff0a810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.135+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/3247507069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 0x7ff0a810bb50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.135+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/3247507069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a8102d40 0x7ff0a8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.135+0000 7ff0ace3b640 1 -- 172.21.15.1:0/3247507069 >> 172.21.15.1:0/3247507069 conn(0x7ff0a80fc7a0 msgr2=0x7ff0a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.135+0000 7ff0ace3b640 1 -- 172.21.15.1:0/3247507069 shutdown_connections 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.135+0000 7ff0ace3b640 1 -- 172.21.15.1:0/3247507069 wait complete. 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0ace3b640 1 Processor -- start 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0ace3b640 1 -- start start 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0ace3b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0a8102d40 0x7ff0a81a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0ace3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 0x7ff0a81a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0ace3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a810c170 0x7ff0a819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0ace3b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8112030 con 0x7ff0a8103660 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0ace3b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8111eb0 con 0x7ff0a810c170 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0ace3b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81121b0 con 0x7ff0a8102d40 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff09dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 0x7ff0a81a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff09dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 0x7ff0a81a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52930/0 (socket says 172.21.15.1:52930) 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff09dd9b640 1 -- 172.21.15.1:0/2437671271 learned_addr learned my addr 172.21.15.1:0/2437671271 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a810c170 0x7ff0a819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff0a659c640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0a8102d40 0x7ff0a81a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.136+0000 7ff09dd9b640 1 -- 172.21.15.1:0/2437671271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0a8102d40 msgr2=0x7ff0a81a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0a8102d40 0x7ff0a81a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff09dd9b640 1 -- 172.21.15.1:0/2437671271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a810c170 msgr2=0x7ff0a819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a810c170 0x7ff0a819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff09dd9b640 1 -- 172.21.15.1:0/2437671271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a819ad50 con 0x7ff0a8103660 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a810c170 0x7ff0a819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 0x7ff0a81a09a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff090005ab0 tx=0x7ff0900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff09ffff640 1 -- 172.21.15.1:0/2437671271 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09003f030 con 0x7ff0a8103660 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff0ace3b640 1 -- 172.21.15.1:0/2437671271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a819afe0 con 0x7ff0a8103660 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff09ffff640 1 -- 172.21.15.1:0/2437671271 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff090003ce0 con 0x7ff0a8103660 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff09ffff640 1 -- 172.21.15.1:0/2437671271 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090035070 con 0x7ff0a8103660 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.137+0000 7ff0ace3b640 1 -- 172.21.15.1:0/2437671271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a81a7330 con 0x7ff0a8103660 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.139+0000 7ff09ffff640 1 -- 172.21.15.1:0/2437671271 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff090042030 con 0x7ff0a8103660 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0af214740 1 Processor -- start 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0af214740 1 -- start start 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0af214740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf8922a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0af214740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 0x55bf8922aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0af214740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 0x55bf89237760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0af214740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bf891902f0 con 0x55bf8922bb40 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0af214740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bf89137cf0 con 0x55bf89235370 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0af214740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf8918c2b0 con 0x55bf8922a5d0 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0a6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf8922a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0a6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf8922a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52934/0 (socket says 172.21.15.1:52934) 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff0a6d9d640 1 -- 172.21.15.1:0/2197817315 learned_addr learned my addr 172.21.15.1:0/2197817315 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.140+0000 7ff09ffff640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0940a3240 0x7ff0940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff0a659c640 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 0x55bf8922aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff0a6d9d640 1 -- 172.21.15.1:0/2197817315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 msgr2=0x55bf8922aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 0x55bf89237760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 0x55bf8922aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff0a6d9d640 1 -- 172.21.15.1:0/2197817315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 msgr2=0x55bf89237760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 0x55bf89237760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff09ffff640 1 -- 172.21.15.1:0/2437671271 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff0900f0620 con 0x7ff0a8103660 2024-09-16T00:31:14.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff0a6d9d640 1 -- 172.21.15.1:0/2197817315 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8911c3a0 con 0x55bf8922bb40 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 0x55bf89237760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0940a3240 0x7ff0940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf8922a090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c550 tx=0x7ff0a000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d329e2a3f918d87f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff09cd99640 1 -- 172.21.15.1:0/2197817315 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000a600 con 0x55bf8922bb40 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff09cd99640 1 -- 172.21.15.1:0/2197817315 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a000ac80 con 0x55bf8922bb40 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.141+0000 7ff09cd99640 1 -- 172.21.15.1:0/2197817315 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00179e0 con 0x55bf8922bb40 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 -- 172.21.15.1:0/2197817315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 msgr2=0x55bf8922a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0940a3240 0x7ff0940a5700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff0a001f000 tx=0x7ff0a0018000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf8922a090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c550 tx=0x7ff0a000ca10 comp rx=0 tx=0).stop 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 -- 172.21.15.1:0/2197817315 shutdown_connections 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 0x55bf89237760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 0x55bf8922aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 --2- 172.21.15.1:0/2197817315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf8922a090 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 -- 172.21.15.1:0/2197817315 >> 172.21.15.1:0/2197817315 conn(0x55bf891f3310 msgr2=0x55bf89226c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 -- 172.21.15.1:0/2197817315 shutdown_connections 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 -- 172.21.15.1:0/2197817315 wait complete. 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 -- start start 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.142+0000 7ff0af214740 1 Processor -- start 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0af214740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 0x55bf89234dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0af214740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89232080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0af214740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89235370 0x55bf89232810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0af214740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bf89238070 con 0x55bf8922bb40 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0af214740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bf8914b060 con 0x55bf8922a5d0 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0af214740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf892381f0 con 0x55bf89235370 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0af214740 10 client.0 _ll_register_callbacks cb 0x55bf88e9e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff09dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89232080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff09dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89232080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52936/0 (socket says 172.21.15.1:52936) 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff09dd9b640 1 -- 172.21.15.1:0/1917200177 learned_addr learned my addr 172.21.15.1:0/1917200177 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89235370 0x55bf89232810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0a659c640 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 0x55bf89234dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff09dd9b640 1 -- 172.21.15.1:0/1917200177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89235370 msgr2=0x55bf89232810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89235370 0x55bf89232810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff09dd9b640 1 -- 172.21.15.1:0/1917200177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 msgr2=0x55bf89234dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 0x55bf89234dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff09dd9b640 1 -- 172.21.15.1:0/1917200177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf88f013c0 con 0x55bf8922bb40 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89232080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff0900bc4e0 tx=0x7ff09002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445357b4e5fa045b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff08dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff08dffb640 1 -- 172.21.15.1:0/1917200177 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090049040 con 0x55bf8922bb40 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff08dffb640 1 -- 172.21.15.1:0/1917200177 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff0900f02c0 con 0x55bf8922bb40 2024-09-16T00:31:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.143+0000 7ff0af214740 1 -- 172.21.15.1:0/1917200177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a000ac80 con 0x55bf8922bb40 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff08dffb640 1 -- 172.21.15.1:0/1917200177 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0900bc320 con 0x55bf8922bb40 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff0af214740 10 client.15876 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff0af214740 1 -- 172.21.15.1:0/1917200177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf89243b00 con 0x55bf8922bb40 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff08dffb640 1 -- 172.21.15.1:0/1917200177 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a000ac80 con 0x55bf8922bb40 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff08dffb640 1 -- 172.21.15.1:0/1917200177 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff0900b8e50 con 0x55bf8922bb40 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff0af214740 10 client.15876 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff0af214740 10 client.15876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff0af214740 20 client.15876 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff0af214740 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bf89244ea0 0x55bf89247330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff0af214740 4 client.15876 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff0af214740 1 -- 172.21.15.1:0/1917200177 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf8919d860 con 0x55bf89244ea0 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.144+0000 7ff0a659c640 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bf89244ea0 0x55bf89247330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.145+0000 7ff0a659c640 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bf89244ea0 0x55bf89247330 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee3d404bb59efed5 server_cookie=eb0fbd5980754b9f in_seq=0 out_seq=0 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.145+0000 7ff08dffb640 10 client.15876 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:31:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff08dffb640 1 -- 172.21.15.1:0/1917200177 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf8919d860 con 0x55bf89244ea0 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff08dffb640 10 client.15876 handle_command_reply: tid=0 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 client.15876 shutdown 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 -- 172.21.15.1:0/1917200177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 msgr2=0x55bf89232080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89232080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff0900bc4e0 tx=0x7ff09002a480 comp rx=0 tx=0).stop 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 -- 172.21.15.1:0/1917200177 shutdown_connections 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89235370 0x55bf89232810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bf89244ea0 0x55bf89247330 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89232080 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 --2- 172.21.15.1:0/1917200177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 0x55bf89234dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff08dffb640 0 client.15876 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 -- 172.21.15.1:0/1917200177 >> 172.21.15.1:0/1917200177 conn(0x55bf891f3310 msgr2=0x55bf8922c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 -- 172.21.15.1:0/1917200177 shutdown_connections 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 1 -- 172.21.15.1:0/1917200177 wait complete. 2024-09-16T00:31:14.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.146+0000 7ff0af214740 20 client.15876 trim_cache size 0 max 16384 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.159+0000 7ff0af214740 1 Processor -- start 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.159+0000 7ff0af214740 1 -- start start 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.159+0000 7ff0af214740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 0x55bf89225c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.159+0000 7ff0af214740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922bb40 0x55bf8922d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.159+0000 7ff0af214740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf89235370 0x55bf8922d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.159+0000 7ff0af214740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b39a0 con 0x55bf89235370 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.159+0000 7ff0af214740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b1270 con 0x55bf8922a5d0 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.159+0000 7ff0af214740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81af0c0 con 0x55bf8922bb40 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.159+0000 7ff0a6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf89235370 0x55bf8922d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff0a6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf89235370 0x55bf8922d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52946/0 (socket says 172.21.15.1:52946) 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff0a6d9d640 1 -- 172.21.15.1:0/1452629074 learned_addr learned my addr 172.21.15.1:0/1452629074 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff0a659c640 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 0x55bf89225c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922bb40 0x55bf8922d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff0a6d9d640 1 -- 172.21.15.1:0/1452629074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922bb40 msgr2=0x55bf8922d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:14.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922bb40 0x55bf8922d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff0a6d9d640 1 -- 172.21.15.1:0/1452629074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 msgr2=0x55bf89225c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:14.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 0x55bf89225c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff0a6d9d640 1 -- 172.21.15.1:0/1452629074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf892479b0 con 0x55bf89235370 2024-09-16T00:31:14.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922bb40 0x55bf8922d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:14.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf89235370 0x55bf8922d660 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c3f0 tx=0x7ff0a0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9def767c30dc6309 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff08ffff640 1 -- 172.21.15.1:0/1452629074 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0034070 con 0x55bf89235370 2024-09-16T00:31:14.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff08ffff640 1 -- 172.21.15.1:0/1452629074 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a00098e0 con 0x55bf89235370 2024-09-16T00:31:14.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.160+0000 7ff08ffff640 1 -- 172.21.15.1:0/1452629074 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000a660 con 0x55bf89235370 2024-09-16T00:31:14.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 -- 172.21.15.1:0/1452629074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf89235370 msgr2=0x55bf8922d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf89235370 0x55bf8922d660 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c3f0 tx=0x7ff0a0019000 comp rx=0 tx=0).stop 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 -- 172.21.15.1:0/1452629074 shutdown_connections 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf89235370 0x55bf8922d660 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922bb40 0x55bf8922d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 --2- 172.21.15.1:0/1452629074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf8922a5d0 0x55bf89225c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 -- 172.21.15.1:0/1452629074 >> 172.21.15.1:0/1452629074 conn(0x55bf891f3310 msgr2=0x55bf89225500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 -- 172.21.15.1:0/1452629074 shutdown_connections 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 -- 172.21.15.1:0/1452629074 wait complete. 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 -- start start 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.161+0000 7ff0af214740 1 Processor -- start 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0af214740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 0x55bf89239420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0af214740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89239960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0af214740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 0x55bf89239ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0af214740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81af730 con 0x55bf8922bb40 2024-09-16T00:31:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0af214740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b0ba0 con 0x55bf89235370 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0af214740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81afe00 con 0x55bf8922a5d0 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0af214740 10 client.0 _ll_register_callbacks cb 0x7ff090005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff09dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89239960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff09dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89239960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52948/0 (socket says 172.21.15.1:52948) 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff09dd9b640 1 -- 172.21.15.1:0/1480034711 learned_addr learned my addr 172.21.15.1:0/1480034711 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0a659c640 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 0x55bf89239420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0a6d9d640 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 0x55bf89239ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff09dd9b640 1 -- 172.21.15.1:0/1480034711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 msgr2=0x55bf89239420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 0x55bf89239420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff09dd9b640 1 -- 172.21.15.1:0/1480034711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 msgr2=0x55bf89239ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 0x55bf89239ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff09dd9b640 1 -- 172.21.15.1:0/1480034711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf892aee10 con 0x55bf8922bb40 2024-09-16T00:31:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff09dd9b640 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89239960 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff090006010 tx=0x7ff090003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6932ed2b7abd8c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff08dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff08dffb640 1 -- 172.21.15.1:0/1480034711 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090049070 con 0x55bf8922bb40 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff08dffb640 1 -- 172.21.15.1:0/1480034711 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff09003bd50 con 0x55bf8922bb40 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.162+0000 7ff0af214740 1 -- 172.21.15.1:0/1480034711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a00098e0 con 0x55bf8922bb40 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff08dffb640 1 -- 172.21.15.1:0/1480034711 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0900bc140 con 0x55bf8922bb40 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff0af214740 10 client.15882 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff0af214740 1 -- 172.21.15.1:0/1480034711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf892b00d0 con 0x55bf8922bb40 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff08dffb640 1 -- 172.21.15.1:0/1480034711 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a00098e0 con 0x55bf8922bb40 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff08dffb640 1 -- 172.21.15.1:0/1480034711 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff0900c4fc0 con 0x55bf8922bb40 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff0af214740 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff0af214740 10 client.15882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff0af214740 20 client.15882 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff0af214740 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bf892b0690 0x55bf892b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff0af214740 4 client.15882 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:31:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff0af214740 1 -- 172.21.15.1:0/1480034711 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf89197d70 con 0x55bf892b0690 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.163+0000 7ff0a659c640 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bf892b0690 0x55bf892b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0a659c640 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bf892b0690 0x55bf892b2ab0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd8f7ba237c1bab3 server_cookie=96c7bcda4b62ccde in_seq=0 out_seq=0 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff08dffb640 10 client.15882 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff08dffb640 1 -- 172.21.15.1:0/1480034711 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf89197d70 con 0x55bf892b0690 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff08dffb640 10 client.15882 handle_command_reply: tid=0 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 client.15882 shutdown 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 -- 172.21.15.1:0/1480034711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 msgr2=0x55bf89239960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89239960 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff090006010 tx=0x7ff090003f50 comp rx=0 tx=0).stop 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 -- 172.21.15.1:0/1480034711 shutdown_connections 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf89235370 0x55bf89239ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bf892b0690 0x55bf892b2ab0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf8922bb40 0x55bf89239960 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 --2- 172.21.15.1:0/1480034711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf8922a5d0 0x55bf89239420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 -- 172.21.15.1:0/1480034711 >> 172.21.15.1:0/1480034711 conn(0x55bf891f3310 msgr2=0x55bf89234100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff08dffb640 0 client.15882 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:31:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 -- 172.21.15.1:0/1480034711 shutdown_connections 2024-09-16T00:31:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 1 -- 172.21.15.1:0/1480034711 wait complete. 2024-09-16T00:31:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.164+0000 7ff0af214740 20 client.15882 trim_cache size 0 max 16384 2024-09-16T00:31:14.180 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 -- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0940a3240 msgr2=0x7ff0940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0940a3240 0x7ff0940a5700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff0a001f000 tx=0x7ff0a0018000 comp rx=0 tx=0).stop 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 -- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 msgr2=0x7ff0a81a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 0x7ff0a81a09a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff090005ab0 tx=0x7ff0900042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 -- 172.21.15.1:0/2437671271 shutdown_connections 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a810c170 0x7ff0a819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0a8103660 0x7ff0a81a09a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0940a3240 0x7ff0940a5700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 --2- 172.21.15.1:0/2437671271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0a8102d40 0x7ff0a81a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 -- 172.21.15.1:0/2437671271 >> 172.21.15.1:0/2437671271 conn(0x7ff0a80fc7a0 msgr2=0x7ff0a81008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 -- 172.21.15.1:0/2437671271 shutdown_connections 2024-09-16T00:31:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:14.165+0000 7ff0ace3b640 1 -- 172.21.15.1:0/2437671271 wait complete. 2024-09-16T00:31:14.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:14 smithi001 ceph-mon[30934]: pgmap v435: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-16T00:31:14.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:14 smithi153 ceph-mon[35538]: pgmap v435: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-16T00:31:14.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:14 smithi019 ceph-mon[35186]: pgmap v435: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-16T00:31:14.991 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:31:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:15.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:31:15.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.761+0000 7fbc3ba68640 1 -- 172.21.15.1:0/908958737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 msgr2=0x7fbc34100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.761+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/908958737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 0x7fbc34100d90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009900 tx=0x7fbc3002a410 comp rx=0 tx=0).stop 2024-09-16T00:31:15.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.761+0000 7fbc3ba68640 1 -- 172.21.15.1:0/908958737 shutdown_connections 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.761+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/908958737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc3410e4d0 0x7fbc341108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.761+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/908958737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc34101360 0x7fbc3410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.761+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/908958737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 0x7fbc34100d90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.761+0000 7fbc3ba68640 1 -- 172.21.15.1:0/908958737 >> 172.21.15.1:0/908958737 conn(0x7fbc340fc7d0 msgr2=0x7fbc340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.761+0000 7fbc3ba68640 1 -- 172.21.15.1:0/908958737 shutdown_connections 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.761+0000 7fbc3ba68640 1 -- 172.21.15.1:0/908958737 wait complete. 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc3ba68640 1 Processor -- start 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc3ba68640 1 -- start start 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc3ba68640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 0x7fbc3419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc3ba68640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc34101360 0x7fbc3419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc3ba68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc3410e4d0 0x7fbc3419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc3ba68640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34114230 con 0x7fbc341009b0 2024-09-16T00:31:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc3ba68640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341140b0 con 0x7fbc3410e4d0 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc3ba68640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341143b0 con 0x7fbc34101360 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39804640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 0x7fbc3419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39804640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 0x7fbc3419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52958/0 (socket says 172.21.15.1:52958) 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39804640 1 -- 172.21.15.1:0/2241196010 learned_addr learned my addr 172.21.15.1:0/2241196010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc3a005640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc3410e4d0 0x7fbc3419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39003640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc34101360 0x7fbc3419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39804640 1 -- 172.21.15.1:0/2241196010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc34101360 msgr2=0x7fbc3419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39804640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc34101360 0x7fbc3419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39804640 1 -- 172.21.15.1:0/2241196010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc3410e4d0 msgr2=0x7fbc3419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39804640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc3410e4d0 0x7fbc3419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39804640 1 -- 172.21.15.1:0/2241196010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3419f6a0 con 0x7fbc341009b0 2024-09-16T00:31:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39003640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc34101360 0x7fbc3419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.762+0000 7fbc39804640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 0x7fbc3419a6f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbc30005ab0 tx=0x7fbc300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.763+0000 7fbc26ffd640 1 -- 172.21.15.1:0/2241196010 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3003f030 con 0x7fbc341009b0 2024-09-16T00:31:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.763+0000 7fbc26ffd640 1 -- 172.21.15.1:0/2241196010 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc30003ce0 con 0x7fbc341009b0 2024-09-16T00:31:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.763+0000 7fbc26ffd640 1 -- 172.21.15.1:0/2241196010 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30035070 con 0x7fbc341009b0 2024-09-16T00:31:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.763+0000 7fbc3ba68640 1 -- 172.21.15.1:0/2241196010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3419f930 con 0x7fbc341009b0 2024-09-16T00:31:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.763+0000 7fbc3ba68640 1 -- 172.21.15.1:0/2241196010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc341a7270 con 0x7fbc341009b0 2024-09-16T00:31:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.765+0000 7fbc26ffd640 1 -- 172.21.15.1:0/2241196010 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fbc30042030 con 0x7fbc341009b0 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.765+0000 7fbc3de41740 1 Processor -- start 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.765+0000 7fbc3de41740 1 -- start start 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef8b40 0x55ae95ef7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef75d0 0x55ae95ef7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3de41740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ae95e5d2f0 con 0x55ae95f02370 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3de41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ae95e04cf0 con 0x55ae95ef8b40 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3de41740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ae95e592b0 con 0x55ae95ef75d0 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc39003640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3a005640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef8b40 0x55ae95ef7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc39003640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f04760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52962/0 (socket says 172.21.15.1:52962) 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc39003640 1 -- 172.21.15.1:0/3217813409 learned_addr learned my addr 172.21.15.1:0/3217813409 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc39804640 1 --2- 172.21.15.1:0/3217813409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef75d0 0x55ae95ef7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc26ffd640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc200a32c0 0x7fbc200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3a005640 1 -- 172.21.15.1:0/3217813409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef75d0 msgr2=0x55ae95ef7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3a005640 1 --2- 172.21.15.1:0/3217813409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef75d0 0x55ae95ef7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3a005640 1 -- 172.21.15.1:0/3217813409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 msgr2=0x55ae95f04760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3a005640 1 --2- 172.21.15.1:0/3217813409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f04760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3a005640 1 -- 172.21.15.1:0/3217813409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae95de93a0 con 0x55ae95ef8b40 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc39003640 1 --2- 172.21.15.1:0/3217813409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc3a005640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc200a32c0 0x7fbc200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.766+0000 7fbc26ffd640 1 -- 172.21.15.1:0/2241196010 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbc300f0650 con 0x7fbc341009b0 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.767+0000 7fbc3a005640 1 --2- 172.21.15.1:0/3217813409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef8b40 0x55ae95ef7090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c007e50 tx=0x7fbc2c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a3586a877638fee server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.767+0000 7fbc1ffff640 1 -- 172.21.15.1:0/3217813409 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c022070 con 0x55ae95ef8b40 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.767+0000 7fbc1ffff640 1 -- 172.21.15.1:0/3217813409 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c01c470 con 0x55ae95ef8b40 2024-09-16T00:31:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.767+0000 7fbc1ffff640 1 -- 172.21.15.1:0/3217813409 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00ee30 con 0x55ae95ef8b40 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.767+0000 7fbc3a005640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc200a32c0 0x7fbc200a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c01f000 tx=0x7fbc2c020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 -- 172.21.15.1:0/3217813409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef8b40 msgr2=0x55ae95ef7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 --2- 172.21.15.1:0/3217813409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef8b40 0x55ae95ef7090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c007e50 tx=0x7fbc2c00e720 comp rx=0 tx=0).stop 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 -- 172.21.15.1:0/3217813409 shutdown_connections 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 --2- 172.21.15.1:0/3217813409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 --2- 172.21.15.1:0/3217813409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef75d0 0x55ae95ef7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 --2- 172.21.15.1:0/3217813409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef8b40 0x55ae95ef7090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 -- 172.21.15.1:0/3217813409 >> 172.21.15.1:0/3217813409 conn(0x55ae95ec0310 msgr2=0x55ae95ef3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 -- 172.21.15.1:0/3217813409 shutdown_connections 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 -- 172.21.15.1:0/3217813409 wait complete. 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 -- start start 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.768+0000 7fbc3de41740 1 Processor -- start 2024-09-16T00:31:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 0x55ae95f01d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef8b40 0x55ae95efeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95f02370 0x55ae95eff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc3de41740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ae95f05070 con 0x55ae95ef8b40 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc3de41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ae95e18060 con 0x55ae95ef75d0 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc3de41740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ae95f051f0 con 0x55ae95f02370 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc3de41740 10 client.0 _ll_register_callbacks cb 0x55ae95b6b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39003640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef8b40 0x55ae95efeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39003640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef8b40 0x55ae95efeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52968/0 (socket says 172.21.15.1:52968) 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39003640 1 -- 172.21.15.1:0/343023133 learned_addr learned my addr 172.21.15.1:0/343023133 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39804640 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 0x55ae95f01d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc3a005640 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95f02370 0x55ae95eff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39003640 1 -- 172.21.15.1:0/343023133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95f02370 msgr2=0x55ae95eff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39003640 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95f02370 0x55ae95eff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39003640 1 -- 172.21.15.1:0/343023133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 msgr2=0x55ae95f01d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39003640 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 0x55ae95f01d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39003640 1 -- 172.21.15.1:0/343023133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae95bce3c0 con 0x55ae95ef8b40 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc39003640 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef8b40 0x55ae95efeff0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fbc280107f0 tx=0x7fbc28010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949d52a276c56e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.769+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/343023133 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2801d060 con 0x55ae95ef8b40 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/343023133 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbc2800eda0 con 0x55ae95ef8b40 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc3de41740 1 -- 172.21.15.1:0/343023133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2c01c470 con 0x55ae95ef8b40 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/343023133 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc28024050 con 0x55ae95ef8b40 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc3de41740 10 client.15894 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc3de41740 1 -- 172.21.15.1:0/343023133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae95f10b40 con 0x55ae95ef8b40 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/343023133 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c01c470 con 0x55ae95ef8b40 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/343023133 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbc28016600 con 0x55ae95ef8b40 2024-09-16T00:31:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc3de41740 10 client.15894 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc3de41740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc3de41740 20 client.15894 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc3de41740 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ae95f12050 0x55ae95f14460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc3de41740 4 client.15894 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc3de41740 1 -- 172.21.15.1:0/343023133 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae95e6a860 con 0x55ae95f12050 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.770+0000 7fbc39804640 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ae95f12050 0x55ae95f14460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.771+0000 7fbc1d7fa640 10 client.15894 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.771+0000 7fbc39804640 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ae95f12050 0x55ae95f14460 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c2556f91e569bc server_cookie=75acf98187b1322 in_seq=0 out_seq=0 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/343023133 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae95e6a860 con 0x55ae95f12050 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc1d7fa640 10 client.15894 handle_command_reply: tid=0 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 client.15894 shutdown 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 -- 172.21.15.1:0/343023133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef8b40 msgr2=0x55ae95efeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef8b40 0x55ae95efeff0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fbc280107f0 tx=0x7fbc28010cb0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 -- 172.21.15.1:0/343023133 shutdown_connections 2024-09-16T00:31:15.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ae95f12050 0x55ae95f14460 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95f02370 0x55ae95eff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef8b40 0x55ae95efeff0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc1d7fa640 0 client.15894 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:15.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 --2- 172.21.15.1:0/343023133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 0x55ae95f01d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 -- 172.21.15.1:0/343023133 >> 172.21.15.1:0/343023133 conn(0x55ae95ec0310 msgr2=0x55ae95ef9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 -- 172.21.15.1:0/343023133 shutdown_connections 2024-09-16T00:31:15.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 1 -- 172.21.15.1:0/343023133 wait complete. 2024-09-16T00:31:15.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.772+0000 7fbc3de41740 20 client.15894 trim_cache size 0 max 16384 2024-09-16T00:31:15.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc3de41740 1 Processor -- start 2024-09-16T00:31:15.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc3de41740 1 -- start start 2024-09-16T00:31:15.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef75d0 0x55ae95ef3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 0x55ae95efa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95f02370 0x55ae95efa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc3de41740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341b3830 con 0x55ae95ef75d0 2024-09-16T00:31:15.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc3de41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341b1100 con 0x55ae95f02370 2024-09-16T00:31:15.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc3de41740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341aef50 con 0x55ae95ef8b40 2024-09-16T00:31:15.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39804640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef75d0 0x55ae95ef3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39804640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef75d0 0x55ae95ef3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52974/0 (socket says 172.21.15.1:52974) 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39804640 1 -- 172.21.15.1:0/2904128519 learned_addr learned my addr 172.21.15.1:0/2904128519 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39003640 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 0x55ae95efa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39804640 1 -- 172.21.15.1:0/2904128519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 msgr2=0x55ae95efa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39804640 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 0x55ae95efa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc3a005640 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95f02370 0x55ae95efa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39804640 1 -- 172.21.15.1:0/2904128519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95f02370 msgr2=0x55ae95efa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39804640 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95f02370 0x55ae95efa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39804640 1 -- 172.21.15.1:0/2904128519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae95ef34f0 con 0x55ae95ef75d0 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.786+0000 7fbc39003640 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 0x55ae95efa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.787+0000 7fbc39804640 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef75d0 0x55ae95ef3d70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fbc300ee0c0 tx=0x7fbc300edc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4407809183470a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.787+0000 7fbc1f7fe640 1 -- 172.21.15.1:0/2904128519 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30049040 con 0x55ae95ef75d0 2024-09-16T00:31:15.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.787+0000 7fbc1f7fe640 1 -- 172.21.15.1:0/2904128519 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc300bc870 con 0x55ae95ef75d0 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.787+0000 7fbc1f7fe640 1 -- 172.21.15.1:0/2904128519 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc300f0350 con 0x55ae95ef75d0 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.787+0000 7fbc3de41740 1 -- 172.21.15.1:0/2904128519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef75d0 msgr2=0x55ae95ef3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.787+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef75d0 0x55ae95ef3d70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fbc300ee0c0 tx=0x7fbc300edc20 comp rx=0 tx=0).stop 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.787+0000 7fbc3de41740 1 -- 172.21.15.1:0/2904128519 shutdown_connections 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.787+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95f02370 0x55ae95efa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 0x55ae95efa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2904128519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95ef75d0 0x55ae95ef3d70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 -- 172.21.15.1:0/2904128519 >> 172.21.15.1:0/2904128519 conn(0x55ae95ec0310 msgr2=0x55ae95f01070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 -- 172.21.15.1:0/2904128519 shutdown_connections 2024-09-16T00:31:15.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 -- 172.21.15.1:0/2904128519 wait complete. 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 -- start start 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 Processor -- start 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 0x55ae95f76040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 0x55ae95f76580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341af5c0 con 0x55ae95f02370 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341b0a30 con 0x55ae95ef75d0 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341afc90 con 0x55ae95ef8b40 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.788+0000 7fbc3de41740 10 client.0 _ll_register_callbacks cb 0x7fbc2800b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc3a005640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc3a005640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52978/0 (socket says 172.21.15.1:52978) 2024-09-16T00:31:15.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc3a005640 1 -- 172.21.15.1:0/2207535257 learned_addr learned my addr 172.21.15.1:0/2207535257 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc39804640 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 0x55ae95f76040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc39003640 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 0x55ae95f76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc3a005640 1 -- 172.21.15.1:0/2207535257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 msgr2=0x55ae95f76580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc3a005640 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 0x55ae95f76580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc3a005640 1 -- 172.21.15.1:0/2207535257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 msgr2=0x55ae95f76040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc3a005640 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 0x55ae95f76040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc3a005640 1 -- 172.21.15.1:0/2207535257 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae95f07220 con 0x55ae95f02370 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc3a005640 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f069f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c01cda0 tx=0x7fbc2c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bf4d146fbf3150 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.789+0000 7fbc1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/2207535257 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c022040 con 0x55ae95f02370 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/2207535257 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbc2c01a980 con 0x55ae95f02370 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc3de41740 1 -- 172.21.15.1:0/2207535257 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc300bc870 con 0x55ae95f02370 2024-09-16T00:31:15.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/2207535257 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc2c007580 con 0x55ae95f02370 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc3de41740 10 client.15900 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc3de41740 1 -- 172.21.15.1:0/2207535257 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae95f7d7c0 con 0x55ae95f02370 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/2207535257 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc300bc870 con 0x55ae95f02370 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/2207535257 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbc2c030070 con 0x55ae95f02370 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc3de41740 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc3de41740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc3de41740 20 client.15900 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ae95f7dd40 0x55ae95f80130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc3de41740 4 client.15900 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc3de41740 1 -- 172.21.15.1:0/2207535257 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae95e64d70 con 0x55ae95f7dd40 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.790+0000 7fbc39804640 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ae95f7dd40 0x55ae95f80130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc1d7fa640 10 client.15900 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc39804640 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ae95f7dd40 0x55ae95f80130 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4debd55a36c5a0 server_cookie=5b0697b72f8c2583 in_seq=0 out_seq=0 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc1d7fa640 1 -- 172.21.15.1:0/2207535257 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae95e64d70 con 0x55ae95f7dd40 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc1d7fa640 10 client.15900 handle_command_reply: tid=0 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 client.15900 shutdown 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 -- 172.21.15.1:0/2207535257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 msgr2=0x55ae95f069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f069f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c01cda0 tx=0x7fbc2c011d10 comp rx=0 tx=0).stop 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 -- 172.21.15.1:0/2207535257 shutdown_connections 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ae95f7dd40 0x55ae95f80130 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae95f02370 0x55ae95f069f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc1d7fa640 0 client.15900 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae95ef8b40 0x55ae95f76580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 --2- 172.21.15.1:0/2207535257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae95ef75d0 0x55ae95f76040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 -- 172.21.15.1:0/2207535257 >> 172.21.15.1:0/2207535257 conn(0x55ae95ec0310 msgr2=0x55ae95ef24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.791+0000 7fbc3de41740 1 -- 172.21.15.1:0/2207535257 shutdown_connections 2024-09-16T00:31:15.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3de41740 1 -- 172.21.15.1:0/2207535257 wait complete. 2024-09-16T00:31:15.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3de41740 20 client.15900 trim_cache size 0 max 16384 2024-09-16T00:31:15.804 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 -- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc200a32c0 msgr2=0x7fbc200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc200a32c0 0x7fbc200a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c01f000 tx=0x7fbc2c020000 comp rx=0 tx=0).stop 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 -- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 msgr2=0x7fbc3419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 0x7fbc3419a6f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbc30005ab0 tx=0x7fbc300042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 -- 172.21.15.1:0/2241196010 shutdown_connections 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc3410e4d0 0x7fbc3419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbc200a32c0 0x7fbc200a5780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc34101360 0x7fbc3419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 --2- 172.21.15.1:0/2241196010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc341009b0 0x7fbc3419a6f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 -- 172.21.15.1:0/2241196010 >> 172.21.15.1:0/2241196010 conn(0x7fbc340fc7d0 msgr2=0x7fbc340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 -- 172.21.15.1:0/2241196010 shutdown_connections 2024-09-16T00:31:15.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:15.792+0000 7fbc3ba68640 1 -- 172.21.15.1:0/2241196010 wait complete. 2024-09-16T00:31:16.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:16 smithi001 ceph-mon[30934]: pgmap v436: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-16T00:31:16.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:16 smithi153 ceph-mon[35538]: pgmap v436: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-16T00:31:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:16 smithi019 ceph-mon[35186]: pgmap v436: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-16T00:31:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.253+0000 7f6ca5654640 1 -- 172.21.15.1:0/3883895402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 msgr2=0x7f6ca010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.253+0000 7f6ca5654640 1 --2- 172.21.15.1:0/3883895402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 0x7f6ca010ea90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009ca0 tx=0x7f6c9802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 -- 172.21.15.1:0/3883895402 shutdown_connections 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 --2- 172.21.15.1:0/3883895402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 0x7f6ca010ea90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 --2- 172.21.15.1:0/3883895402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ca0102790 0x7f6ca01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 --2- 172.21.15.1:0/3883895402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca0101de0 0x7f6ca01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 -- 172.21.15.1:0/3883895402 >> 172.21.15.1:0/3883895402 conn(0x7f6ca00fd8c0 msgr2=0x7f6ca00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 -- 172.21.15.1:0/3883895402 shutdown_connections 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 -- 172.21.15.1:0/3883895402 wait complete. 2024-09-16T00:31:17.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 Processor -- start 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 -- start start 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.254+0000 7f6ca5654640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ca0101de0 0x7f6ca0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6ca5654640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca0102790 0x7f6ca0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6ca5654640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 0x7f6ca019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6ca5654640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0111f70 con 0x7f6ca0106f10 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6ca5654640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0111df0 con 0x7f6ca0102790 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6ca5654640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca01120f0 con 0x7f6ca0101de0 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 0x7f6ca019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 0x7f6ca019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52988/0 (socket says 172.21.15.1:52988) 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9f7fe640 1 -- 172.21.15.1:0/94413867 learned_addr learned my addr 172.21.15.1:0/94413867 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9e7fc640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca0102790 0x7f6ca0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9effd640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ca0101de0 0x7f6ca0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9f7fe640 1 -- 172.21.15.1:0/94413867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ca0101de0 msgr2=0x7f6ca0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9f7fe640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ca0101de0 0x7f6ca0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9f7fe640 1 -- 172.21.15.1:0/94413867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca0102790 msgr2=0x7f6ca0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9f7fe640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca0102790 0x7f6ca0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9f7fe640 1 -- 172.21.15.1:0/94413867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca019d2e0 con 0x7f6ca0106f10 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c9f7fe640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 0x7f6ca019cc00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6c98031a60 tx=0x7f6c98031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c87fff640 1 -- 172.21.15.1:0/94413867 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98003b70 con 0x7f6ca0106f10 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c87fff640 1 -- 172.21.15.1:0/94413867 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c98003d10 con 0x7f6ca0106f10 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca019d570 con 0x7f6ca0106f10 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.255+0000 7f6c87fff640 1 -- 172.21.15.1:0/94413867 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98004770 con 0x7f6ca0106f10 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.256+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca01a4eb0 con 0x7f6ca0106f10 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.257+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca01021c0 con 0x7f6ca0106f10 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.258+0000 7f6c87fff640 1 -- 172.21.15.1:0/94413867 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6c9802f790 con 0x7f6ca0106f10 2024-09-16T00:31:17.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.258+0000 7f6c87fff640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c880a3240 0x7f6c880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:17.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.259+0000 7f6c9effd640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c880a3240 0x7f6c880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:17.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.259+0000 7f6c87fff640 1 -- 172.21.15.1:0/94413867 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6c980f11d0 con 0x7f6ca0106f10 2024-09-16T00:31:17.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.259+0000 7f6c9effd640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c880a3240 0x7f6c880a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000abe0 tx=0x7f6c9000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:17.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.262+0000 7f6c87fff640 1 -- 172.21.15.1:0/94413867 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c980bda90 con 0x7f6ca0106f10 2024-09-16T00:31:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.473+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ca0199690 con 0x7f6ca0106f10 2024-09-16T00:31:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.474+0000 7f6c87fff640 1 -- 172.21.15.1:0/94413867 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f6ca0199690 con 0x7f6ca0106f10 2024-09-16T00:31:17.476 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:31:17.476 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:31:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c880a3240 msgr2=0x7f6c880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c880a3240 0x7f6c880a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000abe0 tx=0x7f6c9000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 msgr2=0x7f6ca019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 0x7f6ca019cc00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6c98031a60 tx=0x7f6c98031a90 comp rx=0 tx=0).stop 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 shutdown_connections 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca0106f10 0x7f6ca019cc00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca0102790 0x7f6ca0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6c880a3240 0x7f6c880a5700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 --2- 172.21.15.1:0/94413867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ca0101de0 0x7f6ca0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.476+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 >> 172.21.15.1:0/94413867 conn(0x7f6ca00fd8c0 msgr2=0x7f6ca00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.477+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 shutdown_connections 2024-09-16T00:31:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:17.477+0000 7f6ca5654640 1 -- 172.21.15.1:0/94413867 wait complete. 2024-09-16T00:31:17.631 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:31:17.631 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:31:18.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:18 smithi001 ceph-mon[30934]: pgmap v437: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:31:18.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:18 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/94413867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:18.233 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id a19d02ed --release --await' 2024-09-16T00:31:18.234 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id a19d02ed --release --await 2024-09-16T00:31:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:18 smithi153 ceph-mon[35538]: pgmap v437: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:31:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:18 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/94413867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:18 smithi019 ceph-mon[35186]: pgmap v437: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:18 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/94413867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:18.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.859+0000 7f260d15c640 1 -- 172.21.15.1:0/494356002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608103740 msgr2=0x7f2608107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.859+0000 7f260d15c640 1 --2- 172.21.15.1:0/494356002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608103740 0x7f2608107b80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009930 tx=0x7f25f002a420 comp rx=0 tx=0).stop 2024-09-16T00:31:18.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.859+0000 7f260d15c640 1 -- 172.21.15.1:0/494356002 shutdown_connections 2024-09-16T00:31:18.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.859+0000 7f260d15c640 1 --2- 172.21.15.1:0/494356002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2608108290 0x7f260810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.859+0000 7f260d15c640 1 --2- 172.21.15.1:0/494356002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608103740 0x7f2608107b80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.859+0000 7f260d15c640 1 --2- 172.21.15.1:0/494356002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2608102d90 0x7f2608103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.859+0000 7f260d15c640 1 -- 172.21.15.1:0/494356002 >> 172.21.15.1:0/494356002 conn(0x7f26080fc7d0 msgr2=0x7f26080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.860+0000 7f260d15c640 1 -- 172.21.15.1:0/494356002 shutdown_connections 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.860+0000 7f260d15c640 1 -- 172.21.15.1:0/494356002 wait complete. 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.860+0000 7f260d15c640 1 Processor -- start 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.860+0000 7f260d15c640 1 -- start start 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f260d15c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608102d90 0x7f2608198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f260d15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2608103740 0x7f2608198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f260d15c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2608108290 0x7f260819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f260d15c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2608112090 con 0x7f2608102d90 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f260d15c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2608111f10 con 0x7f2608103740 2024-09-16T00:31:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f260d15c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2608112210 con 0x7f2608108290 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2606d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608102d90 0x7f2608198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2606d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608102d90 0x7f2608198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53000/0 (socket says 172.21.15.1:53000) 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2606d9d640 1 -- 172.21.15.1:0/3037944547 learned_addr learned my addr 172.21.15.1:0/3037944547 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f260759e640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2608108290 0x7f260819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f25fe59c640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2608103740 0x7f2608198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2606d9d640 1 -- 172.21.15.1:0/3037944547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2608108290 msgr2=0x7f260819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2606d9d640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2608108290 0x7f260819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2606d9d640 1 -- 172.21.15.1:0/3037944547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2608103740 msgr2=0x7f2608198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2606d9d640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2608103740 0x7f2608198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2606d9d640 1 -- 172.21.15.1:0/3037944547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260819d4c0 con 0x7f2608102d90 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f25fe59c640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2608103740 0x7f2608198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2606d9d640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608102d90 0x7f2608198510 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f25f800c890 tx=0x7f25f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.861+0000 7f2604d99640 1 -- 172.21.15.1:0/3037944547 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8004220 con 0x7f2608102d90 2024-09-16T00:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.862+0000 7f260d15c640 1 -- 172.21.15.1:0/3037944547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260819d7b0 con 0x7f2608102d90 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.862+0000 7f2604d99640 1 -- 172.21.15.1:0/3037944547 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f80043c0 con 0x7f2608102d90 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.862+0000 7f2604d99640 1 -- 172.21.15.1:0/3037944547 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8004a00 con 0x7f2608102d90 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.862+0000 7f260d15c640 1 -- 172.21.15.1:0/3037944547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26081a5090 con 0x7f2608102d90 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.864+0000 7f2604d99640 1 -- 172.21.15.1:0/3037944547 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f25f80079b0 con 0x7f2608102d90 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.864+0000 7f260f535740 1 Processor -- start 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f260f535740 1 -- start start 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f2604d99640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a3240 0x7f25f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f260f535740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7259f7b60 0x55d7259f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f260f535740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f8480 0x55d725a02800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f260f535740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d725a02d40 0x55d725a05130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f260f535740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d72595d280 con 0x55d7259f7b60 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f260f535740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d725904b10 con 0x55d7259f8480 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f260f535740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d725959240 con 0x55d725a02d40 2024-09-16T00:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f2606d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7259f7b60 0x55d7259f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f260759e640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a3240 0x7f25f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f2606d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7259f7b60 0x55d7259f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53010/0 (socket says 172.21.15.1:53010) 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f2606d9d640 1 -- 172.21.15.1:0/554575631 learned_addr learned my addr 172.21.15.1:0/554575631 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f25fe59c640 1 --2- 172.21.15.1:0/554575631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f8480 0x55d725a02800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f2604d99640 1 -- 172.21.15.1:0/3037944547 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f25f80cfac0 con 0x7f2608102d90 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.865+0000 7f2606d9d640 1 -- 172.21.15.1:0/554575631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d725a02d40 msgr2=0x55d725a05130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.866+0000 7f2606d9d640 1 --2- 172.21.15.1:0/554575631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d725a02d40 0x55d725a05130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.866+0000 7f2606d9d640 1 -- 172.21.15.1:0/554575631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f8480 msgr2=0x55d725a02800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.866+0000 7f2606d9d640 1 --2- 172.21.15.1:0/554575631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f8480 0x55d725a02800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.866+0000 7f2606d9d640 1 -- 172.21.15.1:0/554575631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7259111f0 con 0x55d7259f7b60 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.866+0000 7f260759e640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a3240 0x7f25f40a5700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f260000acf0 tx=0x7f260000e720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.866+0000 7f2606d9d640 1 --2- 172.21.15.1:0/554575631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7259f7b60 0x55d7259f7f40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f25f8005fd0 tx=0x7f25f8005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818f74f9dc57570 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.866+0000 7f25fd59a640 1 -- 172.21.15.1:0/554575631 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f801d040 con 0x55d7259f7b60 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.866+0000 7f25fd59a640 1 -- 172.21.15.1:0/554575631 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f80a5810 con 0x55d7259f7b60 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.866+0000 7f25fd59a640 1 -- 172.21.15.1:0/554575631 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f809c180 con 0x55d7259f7b60 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 -- 172.21.15.1:0/554575631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7259f7b60 msgr2=0x55d7259f7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 --2- 172.21.15.1:0/554575631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7259f7b60 0x55d7259f7f40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f25f8005fd0 tx=0x7f25f8005300 comp rx=0 tx=0).stop 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 -- 172.21.15.1:0/554575631 shutdown_connections 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 --2- 172.21.15.1:0/554575631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d725a02d40 0x55d725a05130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 --2- 172.21.15.1:0/554575631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f8480 0x55d725a02800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 --2- 172.21.15.1:0/554575631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7259f7b60 0x55d7259f7f40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 -- 172.21.15.1:0/554575631 >> 172.21.15.1:0/554575631 conn(0x55d7259f40f0 msgr2=0x55d7259f44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 -- 172.21.15.1:0/554575631 shutdown_connections 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 -- 172.21.15.1:0/554575631 wait complete. 2024-09-16T00:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 -- start start 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 Processor -- start 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a00700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d725a058b0 con 0x55d725a02d40 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d725917f10 con 0x55d7259f7b60 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.867+0000 7f260f535740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d725a05a30 con 0x55d7259f8480 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260f535740 10 client.0 _ll_register_callbacks cb 0x55d7258e0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a00700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a00700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53022/0 (socket says 172.21.15.1:53022) 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260759e640 1 -- 172.21.15.1:0/1786831419 learned_addr learned my addr 172.21.15.1:0/1786831419 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f2606d9d640 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f25fe59c640 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260759e640 1 -- 172.21.15.1:0/1786831419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 msgr2=0x55d725a001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260759e640 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260759e640 1 -- 172.21.15.1:0/1786831419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 msgr2=0x55d7259ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260759e640 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260759e640 1 -- 172.21.15.1:0/1786831419 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7256ce6a0 con 0x55d725a02d40 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f2606d9d640 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260759e640 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a00700 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2600012e30 tx=0x7f2600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2040aafdea31ca28 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f25ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f25ee7fc640 1 -- 172.21.15.1:0/1786831419 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260001f680 con 0x55d725a02d40 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f260f535740 1 -- 172.21.15.1:0/1786831419 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f80a5810 con 0x55d725a02d40 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.868+0000 7f25ee7fc640 1 -- 172.21.15.1:0/1786831419 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f260001f820 con 0x55d725a02d40 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f25ee7fc640 1 -- 172.21.15.1:0/1786831419 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f260002d050 con 0x55d725a02d40 2024-09-16T00:31:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f260f535740 10 client.15918 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f260f535740 1 -- 172.21.15.1:0/1786831419 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d725a11570 con 0x55d725a02d40 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f25ee7fc640 1 -- 172.21.15.1:0/1786831419 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f80a5810 con 0x55d725a02d40 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f25ee7fc640 1 -- 172.21.15.1:0/1786831419 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f260001e8f0 con 0x55d725a02d40 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f260f535740 10 client.15918 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f260f535740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f260f535740 20 client.15918 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f260f535740 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d725a12980 0x55d725a14dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f260f535740 4 client.15918 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f260f535740 1 -- 172.21.15.1:0/1786831419 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d72596a7f0 con 0x55d725a12980 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.869+0000 7f2606d9d640 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d725a12980 0x55d725a14dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.870+0000 7f2606d9d640 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d725a12980 0x55d725a14dd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c13b99375c33cb server_cookie=e0e94e5a668c6cda in_seq=0 out_seq=0 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.870+0000 7f25ee7fc640 10 client.15918 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.870+0000 7f25ee7fc640 1 -- 172.21.15.1:0/1786831419 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d72596a7f0 con 0x55d725a12980 2024-09-16T00:31:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.870+0000 7f25ee7fc640 10 client.15918 handle_command_reply: tid=0 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.870+0000 7f260f535740 1 client.15918 shutdown 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.870+0000 7f260f535740 1 -- 172.21.15.1:0/1786831419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 msgr2=0x55d725a00700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.870+0000 7f260f535740 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a00700 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2600012e30 tx=0x7f2600004a40 comp rx=0 tx=0).stop 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f260f535740 1 -- 172.21.15.1:0/1786831419 shutdown_connections 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f260f535740 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d725a12980 0x55d725a14dd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f25ee7fc640 0 client.15918 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f260f535740 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a00700 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f260f535740 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f260f535740 1 --2- 172.21.15.1:0/1786831419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f260f535740 1 -- 172.21.15.1:0/1786831419 >> 172.21.15.1:0/1786831419 conn(0x55d7259f40f0 msgr2=0x55d7259f3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f260f535740 1 -- 172.21.15.1:0/1786831419 shutdown_connections 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f260f535740 1 -- 172.21.15.1:0/1786831419 wait complete. 2024-09-16T00:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.871+0000 7f260f535740 20 client.15918 trim_cache size 0 max 16384 2024-09-16T00:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.888+0000 7f260f535740 1 Processor -- start 2024-09-16T00:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.888+0000 7f260f535740 1 -- start start 2024-09-16T00:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260f535740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260f535740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a01ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260f535740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a1a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260f535740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b17b0 con 0x55d725a02d40 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260f535740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26081af080 con 0x55d7259f7b60 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260f535740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26081aced0 con 0x55d7259f8480 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a1a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a1a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53036/0 (socket says 172.21.15.1:53036) 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260759e640 1 -- 172.21.15.1:0/1037726185 learned_addr learned my addr 172.21.15.1:0/1037726185 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f25fe59c640 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a01ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f2606d9d640 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260759e640 1 -- 172.21.15.1:0/1037726185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 msgr2=0x55d725a01ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.889+0000 7f260759e640 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a01ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.890+0000 7f260759e640 1 -- 172.21.15.1:0/1037726185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 msgr2=0x55d7259fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.890+0000 7f260759e640 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.890+0000 7f260759e640 1 -- 172.21.15.1:0/1037726185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d725a02550 con 0x55d725a02d40 2024-09-16T00:31:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.890+0000 7f2606d9d640 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.890+0000 7f260759e640 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a1a6f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f260000ab60 tx=0x7f2600012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23292f633d2802fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.890+0000 7f25fcd99640 1 -- 172.21.15.1:0/1037726185 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600004cb0 con 0x55d725a02d40 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.890+0000 7f25fcd99640 1 -- 172.21.15.1:0/1037726185 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2600024070 con 0x55d725a02d40 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.890+0000 7f25fcd99640 1 -- 172.21.15.1:0/1037726185 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260001f920 con 0x55d725a02d40 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 -- 172.21.15.1:0/1037726185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 msgr2=0x55d725a1a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a1a6f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f260000ab60 tx=0x7f2600012b30 comp rx=0 tx=0).stop 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 -- 172.21.15.1:0/1037726185 shutdown_connections 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a1a6f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a01ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 --2- 172.21.15.1:0/1037726185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d7259fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 -- 172.21.15.1:0/1037726185 >> 172.21.15.1:0/1037726185 conn(0x55d7259f40f0 msgr2=0x55d725a013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 -- 172.21.15.1:0/1037726185 shutdown_connections 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 -- 172.21.15.1:0/1037726185 wait complete. 2024-09-16T00:31:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.891+0000 7f260f535740 1 -- start start 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260f535740 1 Processor -- start 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260f535740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d725a0b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260f535740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a06530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260f535740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a06a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260f535740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f26081ad540 con 0x55d725a02d40 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260f535740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26081ae9b0 con 0x55d7259f7b60 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260f535740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26081adc10 con 0x55d7259f8480 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260f535740 10 client.0 _ll_register_callbacks cb 0x7f2600016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a06a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a06a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53052/0 (socket says 172.21.15.1:53052) 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260759e640 1 -- 172.21.15.1:0/3316312760 learned_addr learned my addr 172.21.15.1:0/3316312760 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f25fe59c640 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a06530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f2606d9d640 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d725a0b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260759e640 1 -- 172.21.15.1:0/3316312760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 msgr2=0x55d725a06530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260759e640 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a06530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260759e640 1 -- 172.21.15.1:0/3316312760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 msgr2=0x55d725a0b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260759e640 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d725a0b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.892+0000 7f260759e640 1 -- 172.21.15.1:0/3316312760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d725a070b0 con 0x55d725a02d40 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f260759e640 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a06a70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f260001f580 tx=0x7f260001ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e93128418fd88d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f25ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f25ee7fc640 1 -- 172.21.15.1:0/3316312760 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260001e3e0 con 0x55d725a02d40 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f25ee7fc640 1 -- 172.21.15.1:0/3316312760 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f260001e580 con 0x55d725a02d40 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f260f535740 1 -- 172.21.15.1:0/3316312760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2600024070 con 0x55d725a02d40 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f25ee7fc640 1 -- 172.21.15.1:0/3316312760 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f260004d050 con 0x55d725a02d40 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f260f535740 10 client.15924 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f260f535740 1 -- 172.21.15.1:0/3316312760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d725a7d530 con 0x55d725a02d40 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f25ee7fc640 1 -- 172.21.15.1:0/3316312760 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2600024070 con 0x55d725a02d40 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.893+0000 7f25ee7fc640 1 -- 172.21.15.1:0/3316312760 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f260003b520 con 0x55d725a02d40 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.894+0000 7f260f535740 10 client.15924 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.894+0000 7f260f535740 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.894+0000 7f260f535740 20 client.15924 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.894+0000 7f260f535740 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d725a12980 0x55d725a7fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.894+0000 7f260f535740 4 client.15924 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.894+0000 7f260f535740 1 -- 172.21.15.1:0/3316312760 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d725964d00 con 0x55d725a12980 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.894+0000 7f2606d9d640 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d725a12980 0x55d725a7fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.894+0000 7f25ee7fc640 10 client.15924 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.894+0000 7f2606d9d640 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d725a12980 0x55d725a7fba0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a60927c0ae0288d server_cookie=e6a1d0f9f8ba14f9 in_seq=0 out_seq=0 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.895+0000 7f25ee7fc640 1 -- 172.21.15.1:0/3316312760 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d725964d00 con 0x55d725a12980 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.895+0000 7f25ee7fc640 10 client.15924 handle_command_reply: tid=0 2024-09-16T00:31:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.895+0000 7f260f535740 1 client.15924 shutdown 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.895+0000 7f260f535740 1 -- 172.21.15.1:0/3316312760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 msgr2=0x55d725a06a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.895+0000 7f260f535740 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a06a70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f260001f580 tx=0x7f260001ea00 comp rx=0 tx=0).stop 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260f535740 1 -- 172.21.15.1:0/3316312760 shutdown_connections 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260f535740 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d725a12980 0x55d725a7fba0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260f535740 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d725a02d40 0x55d725a06a70 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f25ee7fc640 0 client.15924 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260f535740 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7259f8480 0x55d725a06530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260f535740 1 --2- 172.21.15.1:0/3316312760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7259f7b60 0x55d725a0b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260f535740 1 -- 172.21.15.1:0/3316312760 >> 172.21.15.1:0/3316312760 conn(0x55d7259f40f0 msgr2=0x55d7259f3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260f535740 1 -- 172.21.15.1:0/3316312760 shutdown_connections 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260f535740 1 -- 172.21.15.1:0/3316312760 wait complete. 2024-09-16T00:31:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260f535740 20 client.15924 trim_cache size 0 max 16384 2024-09-16T00:31:18.904 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:31:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:31:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "497044e8-af20-4d2b-b9ef-ff35efbfc46e", 2024-09-16T00:31:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:31:18.904 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:31:18.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260d15c640 1 -- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a3240 msgr2=0x7f25f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260d15c640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a3240 0x7f25f40a5700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f260000acf0 tx=0x7f260000e720 comp rx=0 tx=0).stop 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260d15c640 1 -- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608102d90 msgr2=0x7f2608198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.896+0000 7f260d15c640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608102d90 0x7f2608198510 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f25f800c890 tx=0x7f25f800cd50 comp rx=0 tx=0).stop 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.897+0000 7f260d15c640 1 -- 172.21.15.1:0/3037944547 shutdown_connections 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.897+0000 7f260d15c640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f25f40a3240 0x7f25f40a5700 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.897+0000 7f260d15c640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2608108290 0x7f260819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.897+0000 7f260d15c640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2608103740 0x7f2608198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.897+0000 7f260d15c640 1 --2- 172.21.15.1:0/3037944547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2608102d90 0x7f2608198510 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.897+0000 7f260d15c640 1 -- 172.21.15.1:0/3037944547 >> 172.21.15.1:0/3037944547 conn(0x7f26080fc7d0 msgr2=0x7f2608105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.897+0000 7f260d15c640 1 -- 172.21.15.1:0/3037944547 shutdown_connections 2024-09-16T00:31:18.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:18.897+0000 7f260d15c640 1 -- 172.21.15.1:0/3037944547 wait complete. 2024-09-16T00:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:19.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:19 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:31:19.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:19 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:31:19.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:19 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:31:19.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:19 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:31:19.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:19 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:31:19.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:19 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:31:19.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:19 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:31:19.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:19 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:31:19.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:19 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:31:19.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:19 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:31:19.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:19 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:31:19.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:19 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:31:19.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:31:20.324 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:20 smithi001 ceph-mon[30934]: pgmap v438: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-16T00:31:20.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.322+0000 7fc239889640 1 -- 172.21.15.1:0/49207585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 msgr2=0x7fc23410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.322+0000 7fc239889640 1 --2- 172.21.15.1:0/49207585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 0x7fc23410e610 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009ca0 tx=0x7fc22c02a950 comp rx=0 tx=0).stop 2024-09-16T00:31:20.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.322+0000 7fc239889640 1 -- 172.21.15.1:0/49207585 shutdown_connections 2024-09-16T00:31:20.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.322+0000 7fc239889640 1 --2- 172.21.15.1:0/49207585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 0x7fc23410e610 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.322+0000 7fc239889640 1 --2- 172.21.15.1:0/49207585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234103660 0x7fc23410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.322+0000 7fc239889640 1 --2- 172.21.15.1:0/49207585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc234102d40 0x7fc234103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.322+0000 7fc239889640 1 -- 172.21.15.1:0/49207585 >> 172.21.15.1:0/49207585 conn(0x7fc2340fc7a0 msgr2=0x7fc2340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:20.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.322+0000 7fc239889640 1 -- 172.21.15.1:0/49207585 shutdown_connections 2024-09-16T00:31:20.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.322+0000 7fc239889640 1 -- 172.21.15.1:0/49207585 wait complete. 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc239889640 1 Processor -- start 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc239889640 1 -- start start 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc239889640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234102d40 0x7fc234198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc239889640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc234103660 0x7fc2341988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc239889640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 0x7fc23419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc239889640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc234111f70 con 0x7fc23410c170 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc239889640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc234111df0 con 0x7fc234102d40 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc239889640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341120f0 con 0x7fc234103660 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc2337fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 0x7fc23419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.323+0000 7fc2337fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 0x7fc23419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43630/0 (socket says 172.21.15.1:43630) 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc232ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234102d40 0x7fc234198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc2337fe640 1 -- 172.21.15.1:0/779019379 learned_addr learned my addr 172.21.15.1:0/779019379 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:20.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc2327fc640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc234103660 0x7fc2341988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc2337fe640 1 -- 172.21.15.1:0/779019379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc234103660 msgr2=0x7fc2341988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc2337fe640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc234103660 0x7fc2341988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc2337fe640 1 -- 172.21.15.1:0/779019379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234102d40 msgr2=0x7fc234198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc2337fe640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234102d40 0x7fc234198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc2337fe640 1 -- 172.21.15.1:0/779019379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23419d320 con 0x7fc23410c170 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc2337fe640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 0x7fc23419cc40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc22c02ae70 tx=0x7fc22c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc238887640 1 -- 172.21.15.1:0/779019379 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c034040 con 0x7fc23410c170 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc239889640 1 -- 172.21.15.1:0/779019379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23419d5b0 con 0x7fc23410c170 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc238887640 1 -- 172.21.15.1:0/779019379 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc22c030920 con 0x7fc23410c170 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc238887640 1 -- 172.21.15.1:0/779019379 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0047b0 con 0x7fc23410c170 2024-09-16T00:31:20.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.324+0000 7fc239889640 1 -- 172.21.15.1:0/779019379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2341a4ef0 con 0x7fc23410c170 2024-09-16T00:31:20.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.327+0000 7fc238887640 1 -- 172.21.15.1:0/779019379 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc22c037020 con 0x7fc23410c170 2024-09-16T00:31:20.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc23bc62740 1 Processor -- start 2024-09-16T00:31:20.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc23bc62740 1 -- start start 2024-09-16T00:31:20.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26698f60 0x561d26699340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26699880 0x561d266a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc23bc62740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561d265fe620 con 0x561d26698f60 2024-09-16T00:31:20.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc23bc62740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d265a5e50 con 0x561d266a8650 2024-09-16T00:31:20.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc23bc62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d265fa5e0 con 0x561d26699880 2024-09-16T00:31:20.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc2327fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26698f60 0x561d26699340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc2327fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26698f60 0x561d26699340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43640/0 (socket says 172.21.15.1:43640) 2024-09-16T00:31:20.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc2327fc640 1 -- 172.21.15.1:0/1415243929 learned_addr learned my addr 172.21.15.1:0/1415243929 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:20.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc238887640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc21c0a32c0 0x7fc21c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc232ffd640 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26699880 0x561d266a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.328+0000 7fc2327fc640 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc238887640 1 -- 172.21.15.1:0/779019379 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc22c0f0bf0 con 0x7fc23410c170 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc2327fc640 1 -- 172.21.15.1:0/1415243929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26699880 msgr2=0x561d266a8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc2327fc640 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26699880 0x561d266a8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc2337fe640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc21c0a32c0 0x7fc21c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc2327fc640 1 -- 172.21.15.1:0/1415243929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 msgr2=0x561d266aaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc2327fc640 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266aaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc2327fc640 1 -- 172.21.15.1:0/1415243929 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d26596d40 con 0x561d26698f60 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc232ffd640 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26699880 0x561d266a8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc2327fc640 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26698f60 0x561d26699340 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc228003db0 tx=0x7fc228004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b85082d07d646b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc219ffb640 1 -- 172.21.15.1:0/1415243929 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228019020 con 0x561d26698f60 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc219ffb640 1 -- 172.21.15.1:0/1415243929 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc228011d10 con 0x561d26698f60 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.329+0000 7fc219ffb640 1 -- 172.21.15.1:0/1415243929 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22800fe60 con 0x561d26698f60 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc23bc62740 1 -- 172.21.15.1:0/1415243929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26698f60 msgr2=0x561d26699340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc23bc62740 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26698f60 0x561d26699340 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc228003db0 tx=0x7fc228004300 comp rx=0 tx=0).stop 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc2337fe640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc21c0a32c0 0x7fc21c0a5780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc22c004af0 tx=0x7fc22c02a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc23bc62740 1 -- 172.21.15.1:0/1415243929 shutdown_connections 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc23bc62740 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc23bc62740 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26699880 0x561d266a8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc23bc62740 1 --2- 172.21.15.1:0/1415243929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26698f60 0x561d26699340 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc23bc62740 1 -- 172.21.15.1:0/1415243929 >> 172.21.15.1:0/1415243929 conn(0x561d266955f0 msgr2=0x561d266959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc23bc62740 1 -- 172.21.15.1:0/1415243929 shutdown_connections 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.330+0000 7fc23bc62740 1 -- 172.21.15.1:0/1415243929 wait complete. 2024-09-16T00:31:20.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.331+0000 7fc23bc62740 1 -- start start 2024-09-16T00:31:20.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.331+0000 7fc23bc62740 1 Processor -- start 2024-09-16T00:31:20.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.331+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266a5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.331+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.331+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266a6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.331+0000 7fc23bc62740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561d266ab280 con 0x561d26699880 2024-09-16T00:31:20.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.331+0000 7fc23bc62740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d26586f10 con 0x561d266a8650 2024-09-16T00:31:20.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.331+0000 7fc23bc62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d266ab400 con 0x561d26698f60 2024-09-16T00:31:20.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.331+0000 7fc23bc62740 10 client.0 _ll_register_callbacks cb 0x561d265aadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:20.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc232ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc232ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266a5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43650/0 (socket says 172.21.15.1:43650) 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc232ffd640 1 -- 172.21.15.1:0/2688949510 learned_addr learned my addr 172.21.15.1:0/2688949510 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc2327fc640 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc2327fc640 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc232ffd640 1 -- 172.21.15.1:0/2688949510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 msgr2=0x561d266a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc232ffd640 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc232ffd640 1 -- 172.21.15.1:0/2688949510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 msgr2=0x561d266a6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc232ffd640 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266a6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc232ffd640 1 -- 172.21.15.1:0/2688949510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2630d580 con 0x561d26699880 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc232ffd640 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266a5ad0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc224012800 tx=0x7fc224012cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab08c41fece7cde1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:20.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc212ffd640 1 -- 172.21.15.1:0/2688949510 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224010dc0 con 0x561d26699880 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc212ffd640 1 -- 172.21.15.1:0/2688949510 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc2240047a0 con 0x561d26699880 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.332+0000 7fc23bc62740 1 -- 172.21.15.1:0/2688949510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc228011d10 con 0x561d26699880 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc212ffd640 1 -- 172.21.15.1:0/2688949510 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc224025050 con 0x561d26699880 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc23bc62740 10 client.15936 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc23bc62740 1 -- 172.21.15.1:0/2688949510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d263d7960 con 0x561d26699880 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc212ffd640 1 -- 172.21.15.1:0/2688949510 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc228011d10 con 0x561d26699880 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc212ffd640 1 -- 172.21.15.1:0/2688949510 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc22400e610 con 0x561d26699880 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc23bc62740 10 client.15936 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc23bc62740 10 client.15936 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc23bc62740 20 client.15936 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc23bc62740 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d266b83b0 0x561d266ba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc23bc62740 4 client.15936 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.333+0000 7fc23bc62740 1 -- 172.21.15.1:0/2688949510 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d2660bb90 con 0x561d266b83b0 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.334+0000 7fc2327fc640 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d266b83b0 0x561d266ba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.334+0000 7fc212ffd640 10 client.15936 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.334+0000 7fc2327fc640 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d266b83b0 0x561d266ba7d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c55824f32828c server_cookie=5c7ea21b09901247 in_seq=0 out_seq=0 2024-09-16T00:31:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc212ffd640 1 -- 172.21.15.1:0/2688949510 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d2660bb90 con 0x561d266b83b0 2024-09-16T00:31:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc212ffd640 10 client.15936 handle_command_reply: tid=0 2024-09-16T00:31:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 client.15936 shutdown 2024-09-16T00:31:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 -- 172.21.15.1:0/2688949510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 msgr2=0x561d266a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266a5ad0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc224012800 tx=0x7fc224012cc0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 -- 172.21.15.1:0/2688949510 shutdown_connections 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d266b83b0 0x561d266ba7d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266a6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266a5ad0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 --2- 172.21.15.1:0/2688949510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 -- 172.21.15.1:0/2688949510 >> 172.21.15.1:0/2688949510 conn(0x561d266955f0 msgr2=0x561d2669b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc212ffd640 0 client.15936 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 -- 172.21.15.1:0/2688949510 shutdown_connections 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 1 -- 172.21.15.1:0/2688949510 wait complete. 2024-09-16T00:31:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.335+0000 7fc23bc62740 20 client.15936 trim_cache size 0 max 16384 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.358+0000 7fc23bc62740 1 Processor -- start 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc23bc62740 1 -- start start 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266c0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc23bc62740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341af460 con 0x561d26699880 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc23bc62740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341acd30 con 0x561d266a8650 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc23bc62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341aab80 con 0x561d26698f60 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc232ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc2327fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc232ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43658/0 (socket says 172.21.15.1:43658) 2024-09-16T00:31:20.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc2327fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266c0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:48434/0 (socket says 172.21.15.1:48434) 2024-09-16T00:31:20.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc232ffd640 1 -- 172.21.15.1:0/1723275860 learned_addr learned my addr 172.21.15.1:0/1723275860 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:20.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc2327fc640 1 --2- 172.21.15.1:0/1723275860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc2327fc640 1 -- 172.21.15.1:0/1723275860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 msgr2=0x561d266a27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:20.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc2327fc640 1 --2- 172.21.15.1:0/1723275860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266a27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc2327fc640 1 -- 172.21.15.1:0/1723275860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 msgr2=0x561d266c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:20.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc2327fc640 1 --2- 172.21.15.1:0/1723275860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266c05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc2327fc640 1 -- 172.21.15.1:0/1723275860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d26695090 con 0x561d26698f60 2024-09-16T00:31:20.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.359+0000 7fc232ffd640 1 --2- 172.21.15.1:0/1723275860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:31:20.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.360+0000 7fc2327fc640 1 --2- 172.21.15.1:0/1723275860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266c0080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc22800fbe0 tx=0x7fc228005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fb84e011b32ca21 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.360+0000 7fc2197fa640 1 -- 172.21.15.1:0/1723275860 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228019030 con 0x561d26698f60 2024-09-16T00:31:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.360+0000 7fc2197fa640 1 -- 172.21.15.1:0/1723275860 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc228005500 con 0x561d26698f60 2024-09-16T00:31:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.360+0000 7fc2197fa640 1 -- 172.21.15.1:0/1723275860 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22800fe60 con 0x561d26698f60 2024-09-16T00:31:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 -- 172.21.15.1:0/1723275860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 msgr2=0x561d266c0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 --2- 172.21.15.1:0/1723275860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266c0080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc22800fbe0 tx=0x7fc228005c50 comp rx=0 tx=0).stop 2024-09-16T00:31:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 -- 172.21.15.1:0/1723275860 shutdown_connections 2024-09-16T00:31:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 --2- 172.21.15.1:0/1723275860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266a27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 --2- 172.21.15.1:0/1723275860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 --2- 172.21.15.1:0/1723275860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266c0080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 -- 172.21.15.1:0/1723275860 >> 172.21.15.1:0/1723275860 conn(0x561d266955f0 msgr2=0x561d266a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 -- 172.21.15.1:0/1723275860 shutdown_connections 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 -- 172.21.15.1:0/1723275860 wait complete. 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 -- start start 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.361+0000 7fc23bc62740 1 Processor -- start 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc23bc62740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc23bc62740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341ab1f0 con 0x561d26699880 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc23bc62740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341ac660 con 0x561d266a8650 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc23bc62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341ab8c0 con 0x561d26698f60 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc23bc62740 10 client.0 _ll_register_callbacks cb 0x7fc22400af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc232ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc232ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43672/0 (socket says 172.21.15.1:43672) 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc232ffd640 1 -- 172.21.15.1:0/342323951 learned_addr learned my addr 172.21.15.1:0/342323951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc2327fc640 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc2327fc640 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc232ffd640 1 -- 172.21.15.1:0/342323951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 msgr2=0x561d266ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc232ffd640 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc232ffd640 1 -- 172.21.15.1:0/342323951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 msgr2=0x561d266ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc232ffd640 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc232ffd640 1 -- 172.21.15.1:0/342323951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d26722250 con 0x561d26699880 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc2327fc640 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc232ffd640 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266acb10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc22400eab0 tx=0x7fc22400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a8c6ff489ea054 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.362+0000 7fc212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc212ffd640 1 -- 172.21.15.1:0/342323951 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224012020 con 0x561d26699880 2024-09-16T00:31:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc212ffd640 1 -- 172.21.15.1:0/342323951 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc2240121c0 con 0x561d26699880 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc23bc62740 1 -- 172.21.15.1:0/342323951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc228005500 con 0x561d26699880 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc212ffd640 1 -- 172.21.15.1:0/342323951 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc22403a050 con 0x561d26699880 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc23bc62740 10 client.15942 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc23bc62740 1 -- 172.21.15.1:0/342323951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d26723550 con 0x561d26699880 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc212ffd640 1 -- 172.21.15.1:0/342323951 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc228005500 con 0x561d26699880 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc212ffd640 1 -- 172.21.15.1:0/342323951 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc224005550 con 0x561d26699880 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc23bc62740 10 client.15942 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc23bc62740 10 client.15942 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc23bc62740 20 client.15942 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc23bc62740 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d26723c70 0x561d26726090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc23bc62740 4 client.15942 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a19d02ed", "release": true, "await": true}] 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.363+0000 7fc23bc62740 1 -- 172.21.15.1:0/342323951 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a19d02ed", "release": true, "await": true}) -- 0x561d266060a0 con 0x561d26723c70 2024-09-16T00:31:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.364+0000 7fc2327fc640 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d26723c70 0x561d26726090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.364+0000 7fc212ffd640 10 client.15942 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.364+0000 7fc2327fc640 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d26723c70 0x561d26726090 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f73a629a58996a server_cookie=84e19a19db13ebb7 in_seq=0 out_seq=0 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc212ffd640 1 -- 172.21.15.1:0/342323951 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561d266060a0 con 0x561d26723c70 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc212ffd640 10 client.15942 handle_command_reply: tid=0 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 client.15942 shutdown 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 -- 172.21.15.1:0/342323951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 msgr2=0x561d266acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266acb10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc22400eab0 tx=0x7fc22400ef30 comp rx=0 tx=0).stop 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 -- 172.21.15.1:0/342323951 shutdown_connections 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d26723c70 0x561d26726090 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d266a8650 0x561d266ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d26699880 0x561d266acb10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 --2- 172.21.15.1:0/342323951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d26698f60 0x561d266ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 -- 172.21.15.1:0/342323951 >> 172.21.15.1:0/342323951 conn(0x561d266955f0 msgr2=0x561d2669aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc212ffd640 0 client.15942 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 -- 172.21.15.1:0/342323951 shutdown_connections 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 1 -- 172.21.15.1:0/342323951 wait complete. 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.367+0000 7fc23bc62740 20 client.15942 trim_cache size 0 max 16384 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 8, 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "a19d02ed": { 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 8, 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 139.6, 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:31:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:31:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:31:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:31:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:31:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:20.373 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:31:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 -- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc21c0a32c0 msgr2=0x7fc21c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc21c0a32c0 0x7fc21c0a5780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc22c004af0 tx=0x7fc22c02a950 comp rx=0 tx=0).stop 2024-09-16T00:31:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 -- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 msgr2=0x7fc23419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 0x7fc23419cc40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc22c02ae70 tx=0x7fc22c02f5b0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 -- 172.21.15.1:0/779019379 shutdown_connections 2024-09-16T00:31:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23410c170 0x7fc23419cc40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc21c0a32c0 0x7fc21c0a5780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc234103660 0x7fc2341988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 --2- 172.21.15.1:0/779019379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234102d40 0x7fc234198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 -- 172.21.15.1:0/779019379 >> 172.21.15.1:0/779019379 conn(0x7fc2340fc7a0 msgr2=0x7fc2340fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 -- 172.21.15.1:0/779019379 shutdown_connections 2024-09-16T00:31:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:20.368+0000 7fc239889640 1 -- 172.21.15.1:0/779019379 wait complete. 2024-09-16T00:31:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:20.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:20 smithi153 ceph-mon[35538]: pgmap v438: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-16T00:31:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:20 smithi019 ceph-mon[35186]: pgmap v438: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-16T00:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.066+0000 7fcc422e4640 1 -- 172.21.15.1:0/1792378645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c108290 msgr2=0x7fcc3c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.066+0000 7fcc422e4640 1 --2- 172.21.15.1:0/1792378645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c108290 0x7fcc3c10ec30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009cd0 tx=0x7fcc3402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.066+0000 7fcc422e4640 1 -- 172.21.15.1:0/1792378645 shutdown_connections 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.066+0000 7fcc422e4640 1 --2- 172.21.15.1:0/1792378645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c108290 0x7fcc3c10ec30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.066+0000 7fcc422e4640 1 --2- 172.21.15.1:0/1792378645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc3c103740 0x7fcc3c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.066+0000 7fcc422e4640 1 --2- 172.21.15.1:0/1792378645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc3c102d90 0x7fcc3c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.066+0000 7fcc422e4640 1 -- 172.21.15.1:0/1792378645 >> 172.21.15.1:0/1792378645 conn(0x7fcc3c0fc7d0 msgr2=0x7fcc3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.066+0000 7fcc422e4640 1 -- 172.21.15.1:0/1792378645 shutdown_connections 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.066+0000 7fcc422e4640 1 -- 172.21.15.1:0/1792378645 wait complete. 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc422e4640 1 Processor -- start 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc422e4640 1 -- start start 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc422e4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c102d90 0x7fcc3c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc422e4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc3c103740 0x7fcc3c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc422e4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc3c108290 0x7fcc3c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc422e4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c112090 con 0x7fcc3c102d90 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc422e4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c111f10 con 0x7fcc3c108290 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc422e4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c112210 con 0x7fcc3c103740 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c102d90 0x7fcc3c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c102d90 0x7fcc3c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43694/0 (socket says 172.21.15.1:43694) 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc3bfff640 1 -- 172.21.15.1:0/4283518765 learned_addr learned my addr 172.21.15.1:0/4283518765 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.067+0000 7fcc40881640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc3c108290 0x7fcc3c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc3bfff640 1 -- 172.21.15.1:0/4283518765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc3c103740 msgr2=0x7fcc3c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc3c103740 0x7fcc3c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc3bfff640 1 -- 172.21.15.1:0/4283518765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc3c108290 msgr2=0x7fcc3c19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc3c108290 0x7fcc3c19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc3bfff640 1 -- 172.21.15.1:0/4283518765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c19d520 con 0x7fcc3c102d90 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c102d90 0x7fcc3c198570 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00c920 tx=0x7fcc2c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc397fa640 1 -- 172.21.15.1:0/4283518765 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c007c70 con 0x7fcc3c102d90 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc422e4640 1 -- 172.21.15.1:0/4283518765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c19d810 con 0x7fcc3c102d90 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc397fa640 1 -- 172.21.15.1:0/4283518765 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc2c007e10 con 0x7fcc3c102d90 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc397fa640 1 -- 172.21.15.1:0/4283518765 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c018890 con 0x7fcc3c102d90 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.068+0000 7fcc422e4640 1 -- 172.21.15.1:0/4283518765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c1a50f0 con 0x7fcc3c102d90 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.070+0000 7fcc397fa640 1 -- 172.21.15.1:0/4283518765 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fcc2c018a70 con 0x7fcc3c102d90 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc446bd740 1 Processor -- start 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc446bd740 1 -- start start 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 0x55da4951f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da4952a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 0x55da4952ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc446bd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55da494852e0 con 0x55da4951fdc0 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc446bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55da4942cce0 con 0x55da4951f4a0 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc446bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55da494812a0 con 0x55da4952a680 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da4952a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da4952a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43696/0 (socket says 172.21.15.1:43696) 2024-09-16T00:31:21.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc3bfff640 1 -- 172.21.15.1:0/1682237231 learned_addr learned my addr 172.21.15.1:0/1682237231 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc40881640 1 --2- 172.21.15.1:0/1682237231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 0x55da4951f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc397fa640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc240a3240 0x7fcc240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/1682237231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 0x55da4952ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc40881640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc240a3240 0x7fcc240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc397fa640 1 -- 172.21.15.1:0/4283518765 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fcc2c0cff70 con 0x7fcc3c102d90 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.071+0000 7fcc3bfff640 1 -- 172.21.15.1:0/1682237231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 msgr2=0x55da4952ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.072+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/1682237231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 0x55da4952ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.072+0000 7fcc3bfff640 1 -- 172.21.15.1:0/1682237231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 msgr2=0x55da4951f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.072+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/1682237231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 0x55da4951f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.072+0000 7fcc3bfff640 1 -- 172.21.15.1:0/1682237231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da49411440 con 0x55da4951fdc0 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.072+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/1682237231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da4952a140 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0cd610 tx=0x7fcc2c00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29a78546c3d6e71 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.072+0000 7fcc40881640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc240a3240 0x7fcc240a5700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcc34005cc0 tx=0x7fcc3402a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:21.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.072+0000 7fcc227fc640 1 -- 172.21.15.1:0/1682237231 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00de20 con 0x55da4951fdc0 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.072+0000 7fcc227fc640 1 -- 172.21.15.1:0/1682237231 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc2c09c130 con 0x55da4951fdc0 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.072+0000 7fcc227fc640 1 -- 172.21.15.1:0/1682237231 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0987a0 con 0x55da4951fdc0 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 -- 172.21.15.1:0/1682237231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 msgr2=0x55da4952a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 --2- 172.21.15.1:0/1682237231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da4952a140 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0cd610 tx=0x7fcc2c00ff30 comp rx=0 tx=0).stop 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 -- 172.21.15.1:0/1682237231 shutdown_connections 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 --2- 172.21.15.1:0/1682237231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 0x55da4952ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 --2- 172.21.15.1:0/1682237231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da4952a140 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 --2- 172.21.15.1:0/1682237231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 0x55da4951f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 -- 172.21.15.1:0/1682237231 >> 172.21.15.1:0/1682237231 conn(0x55da4951ba30 msgr2=0x55da4951be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 -- 172.21.15.1:0/1682237231 shutdown_connections 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 -- 172.21.15.1:0/1682237231 wait complete. 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 -- start start 2024-09-16T00:31:21.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.073+0000 7fcc446bd740 1 Processor -- start 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 0x55da49529fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951fdc0 0x55da49527250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4952a680 0x55da49527790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc446bd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55da4952d250 con 0x55da4952a680 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc446bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55da49440050 con 0x55da4951f4a0 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc446bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55da4952d3d0 con 0x55da4951fdc0 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc446bd740 10 client.0 _ll_register_callbacks cb 0x55da49480160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc40881640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4952a680 0x55da49527790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc40881640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4952a680 0x55da49527790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43710/0 (socket says 172.21.15.1:43710) 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc40881640 1 -- 172.21.15.1:0/772580926 learned_addr learned my addr 172.21.15.1:0/772580926 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 0x55da49529fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951fdc0 0x55da49527250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc40881640 1 -- 172.21.15.1:0/772580926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951fdc0 msgr2=0x55da49527250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc40881640 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951fdc0 0x55da49527250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc40881640 1 -- 172.21.15.1:0/772580926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 msgr2=0x55da49529fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc40881640 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 0x55da49529fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc40881640 1 -- 172.21.15.1:0/772580926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da491f63c0 con 0x55da4952a680 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951fdc0 0x55da49527250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc40881640 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4952a680 0x55da49527790 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcc34035000 tx=0x7fcc340095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804b434c9933e8c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.074+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/772580926 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3404e070 con 0x55da4952a680 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/772580926 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fcc34004510 con 0x55da4952a680 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc446bd740 1 -- 172.21.15.1:0/772580926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2c09c130 con 0x55da4952a680 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/772580926 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc34004710 con 0x55da4952a680 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc446bd740 10 client.15954 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc446bd740 1 -- 172.21.15.1:0/772580926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55da49538bd0 con 0x55da4952a680 2024-09-16T00:31:21.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/772580926 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc2c09c130 con 0x55da4952a680 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/772580926 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcc34030070 con 0x55da4952a680 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc446bd740 10 client.15954 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc446bd740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc446bd740 20 client.15954 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc446bd740 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55da4953a090 0x55da4953c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc446bd740 4 client.15954 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.075+0000 7fcc446bd740 1 -- 172.21.15.1:0/772580926 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da49492850 con 0x55da4953a090 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.076+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55da4953a090 0x55da4953c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.076+0000 7fcc1b7fe640 10 client.15954 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.076+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55da4953a090 0x55da4953c4c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d604e70c5fceca3 server_cookie=e5ea001f33b7a3ee in_seq=0 out_seq=0 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/772580926 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da49492850 con 0x55da4953a090 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc1b7fe640 10 client.15954 handle_command_reply: tid=0 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 client.15954 shutdown 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 -- 172.21.15.1:0/772580926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4952a680 msgr2=0x55da49527790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4952a680 0x55da49527790 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcc34035000 tx=0x7fcc340095c0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 -- 172.21.15.1:0/772580926 shutdown_connections 2024-09-16T00:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55da4953a090 0x55da4953c4c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4952a680 0x55da49527790 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951fdc0 0x55da49527250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 --2- 172.21.15.1:0/772580926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951f4a0 0x55da49529fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc1b7fe640 0 client.15954 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 -- 172.21.15.1:0/772580926 >> 172.21.15.1:0/772580926 conn(0x55da4951ba30 msgr2=0x55da49521760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 -- 172.21.15.1:0/772580926 shutdown_connections 2024-09-16T00:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 1 -- 172.21.15.1:0/772580926 wait complete. 2024-09-16T00:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.077+0000 7fcc446bd740 20 client.15954 trim_cache size 0 max 16384 2024-09-16T00:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.094+0000 7fcc446bd740 1 Processor -- start 2024-09-16T00:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc446bd740 1 -- start start 2024-09-16T00:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951f4a0 0x55da495291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da49541d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4952a680 0x55da495422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc446bd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b1740 con 0x55da4951fdc0 2024-09-16T00:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc446bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1af010 con 0x55da4952a680 2024-09-16T00:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc446bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1ace60 con 0x55da4951f4a0 2024-09-16T00:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da49541d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da49541d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43722/0 (socket says 172.21.15.1:43722) 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3b7fe640 1 -- 172.21.15.1:0/2673941751 learned_addr learned my addr 172.21.15.1:0/2673941751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951f4a0 0x55da495291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc40881640 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4952a680 0x55da495422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3b7fe640 1 -- 172.21.15.1:0/2673941751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951f4a0 msgr2=0x55da495291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951f4a0 0x55da495291c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3b7fe640 1 -- 172.21.15.1:0/2673941751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4952a680 msgr2=0x55da495422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4952a680 0x55da495422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3b7fe640 1 -- 172.21.15.1:0/2673941751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4951c6c0 con 0x55da4951fdc0 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc40881640 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4952a680 0x55da495422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.095+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da49541d90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0697d0 tx=0x7fcc3000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be3459ddc54651e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc21ffb640 1 -- 172.21.15.1:0/2673941751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300039f0 con 0x55da4951fdc0 2024-09-16T00:31:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc21ffb640 1 -- 172.21.15.1:0/2673941751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc30003b90 con 0x55da4951fdc0 2024-09-16T00:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc21ffb640 1 -- 172.21.15.1:0/2673941751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300106e0 con 0x55da4951fdc0 2024-09-16T00:31:21.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc446bd740 1 -- 172.21.15.1:0/2673941751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 msgr2=0x55da49541d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da49541d90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0697d0 tx=0x7fcc3000b9d0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc446bd740 1 -- 172.21.15.1:0/2673941751 shutdown_connections 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4952a680 0x55da495422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951fdc0 0x55da49541d90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2673941751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4951f4a0 0x55da495291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.096+0000 7fcc446bd740 1 -- 172.21.15.1:0/2673941751 >> 172.21.15.1:0/2673941751 conn(0x55da4951ba30 msgr2=0x55da49528ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 -- 172.21.15.1:0/2673941751 shutdown_connections 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 -- 172.21.15.1:0/2673941751 wait complete. 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 -- start start 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 Processor -- start 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951f4a0 0x55da49532e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951fdc0 0x55da4959fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 0x55da495a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1ad4d0 con 0x55da4951f4a0 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1ae940 con 0x55da4951fdc0 2024-09-16T00:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1adba0 con 0x55da4952a680 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc446bd740 10 client.0 _ll_register_callbacks cb 0x7fcc34031dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.097+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951f4a0 0x55da49532e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951f4a0 0x55da49532e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43724/0 (socket says 172.21.15.1:43724) 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3bfff640 1 -- 172.21.15.1:0/2733070288 learned_addr learned my addr 172.21.15.1:0/2733070288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951fdc0 0x55da4959fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc40881640 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 0x55da495a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3bfff640 1 -- 172.21.15.1:0/2733070288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 msgr2=0x55da495a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 0x55da495a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3bfff640 1 -- 172.21.15.1:0/2733070288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951fdc0 msgr2=0x55da4959fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951fdc0 0x55da4959fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3bfff640 1 -- 172.21.15.1:0/2733070288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da495a0b00 con 0x55da4951f4a0 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951fdc0 0x55da4959fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc3bfff640 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951f4a0 0x55da49532e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c097e10 tx=0x7fcc2c09bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3cfeac49f55a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/2733070288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c09c130 con 0x55da4951f4a0 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/2733070288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fcc2c00fb30 con 0x55da4951f4a0 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc446bd740 1 -- 172.21.15.1:0/2733070288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc30003b90 con 0x55da4951f4a0 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/2733070288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc2c00de20 con 0x55da4951f4a0 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc446bd740 10 client.15960 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.098+0000 7fcc446bd740 1 -- 172.21.15.1:0/2733070288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55da495a91d0 con 0x55da4951f4a0 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/2733070288 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc30003b90 con 0x55da4951f4a0 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/2733070288 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcc2c0a0150 con 0x55da4951f4a0 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc446bd740 10 client.15960 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc446bd740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc446bd740 20 client.15960 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55da495a94c0 0x55da495ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc446bd740 4 client.15960 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc446bd740 1 -- 172.21.15.1:0/2733070288 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da49527b20 con 0x55da495a94c0 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55da495a94c0 0x55da495ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc1b7fe640 10 client.15960 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.099+0000 7fcc3b7fe640 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55da495a94c0 0x55da495ab8b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ab2d899dcb3a4b server_cookie=700bd181efa1d0c1 in_seq=0 out_seq=0 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc1b7fe640 1 -- 172.21.15.1:0/2733070288 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55da49527b20 con 0x55da495a94c0 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc1b7fe640 10 client.15960 handle_command_reply: tid=0 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 client.15960 shutdown 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 -- 172.21.15.1:0/2733070288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951f4a0 msgr2=0x55da49532e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951f4a0 0x55da49532e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c097e10 tx=0x7fcc2c09bb90 comp rx=0 tx=0).stop 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 -- 172.21.15.1:0/2733070288 shutdown_connections 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55da495a94c0 0x55da495ab8b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da4952a680 0x55da495a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55da4951fdc0 0x55da4959fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 --2- 172.21.15.1:0/2733070288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da4951f4a0 0x55da49532e20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 -- 172.21.15.1:0/2733070288 >> 172.21.15.1:0/2733070288 conn(0x55da4951ba30 msgr2=0x55da495292d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc1b7fe640 0 client.15960 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 -- 172.21.15.1:0/2733070288 shutdown_connections 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 1 -- 172.21.15.1:0/2733070288 wait complete. 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.100+0000 7fcc446bd740 20 client.15960 trim_cache size 0 max 16384 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:31:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "497044e8-af20-4d2b-b9ef-ff35efbfc46e": { 2024-09-16T00:31:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:31:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "497044e8-af20-4d2b-b9ef-ff35efbfc46e", 2024-09-16T00:31:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:31:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:21.108 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:31:21.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 -- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc240a3240 msgr2=0x7fcc240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc240a3240 0x7fcc240a5700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcc34005cc0 tx=0x7fcc3402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 -- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c102d90 msgr2=0x7fcc3c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c102d90 0x7fcc3c198570 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00c920 tx=0x7fcc2c00cde0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 -- 172.21.15.1:0/4283518765 shutdown_connections 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc3c108290 0x7fcc3c19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcc240a3240 0x7fcc240a5700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc3c103740 0x7fcc3c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 --2- 172.21.15.1:0/4283518765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc3c102d90 0x7fcc3c198570 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 -- 172.21.15.1:0/4283518765 >> 172.21.15.1:0/4283518765 conn(0x7fcc3c0fc7d0 msgr2=0x7fcc3c106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 -- 172.21.15.1:0/4283518765 shutdown_connections 2024-09-16T00:31:21.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:21.101+0000 7fcc422e4640 1 -- 172.21.15.1:0/4283518765 wait complete. 2024-09-16T00:31:21.935 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a19d02ed', total seconds elapsed: 26.1 2024-09-16T00:31:21.935 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-16T00:31:21.935 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:31:22.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:22 smithi001 ceph-mon[30934]: pgmap v439: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:31:22.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:22 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:22.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:22 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:22.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:22 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:22.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:22 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:31:22.393 INFO:tasks.cephfs.filesystem:scrub status for tag:497044e8-af20-4d2b-b9ef-ff35efbfc46e - {'path': '/', 'tag': '497044e8-af20-4d2b-b9ef-ff35efbfc46e', 'options': 'recursive,force'} 2024-09-16T00:31:22.410 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:22 smithi153 ceph-mon[35538]: pgmap v439: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:31:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:22 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:22 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:22 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:22.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:22 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:31:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:22 smithi019 ceph-mon[35186]: pgmap v439: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:31:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:22 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:22 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:22 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:22 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:31:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.878+0000 7f415d87a640 1 -- 172.21.15.1:0/1862429332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158100a80 msgr2=0x7f4158100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.878+0000 7f415d87a640 1 --2- 172.21.15.1:0/1862429332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158100a80 0x7f4158100ee0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4140009930 tx=0x7f414002a420 comp rx=0 tx=0).stop 2024-09-16T00:31:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.878+0000 7f415d87a640 1 -- 172.21.15.1:0/1862429332 shutdown_connections 2024-09-16T00:31:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.878+0000 7f415d87a640 1 --2- 172.21.15.1:0/1862429332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4158101420 0x7f415810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.878+0000 7f415d87a640 1 --2- 172.21.15.1:0/1862429332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158100a80 0x7f4158100ee0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.878+0000 7f415d87a640 1 --2- 172.21.15.1:0/1862429332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4158106a80 0x7f4158106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.878+0000 7f415d87a640 1 -- 172.21.15.1:0/1862429332 >> 172.21.15.1:0/1862429332 conn(0x7f41580fc7a0 msgr2=0x7f41580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.878+0000 7f415d87a640 1 -- 172.21.15.1:0/1862429332 shutdown_connections 2024-09-16T00:31:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.878+0000 7f415d87a640 1 -- 172.21.15.1:0/1862429332 wait complete. 2024-09-16T00:31:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f415d87a640 1 Processor -- start 2024-09-16T00:31:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f415d87a640 1 -- start start 2024-09-16T00:31:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f415d87a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4158100a80 0x7f4158198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f415d87a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4158101420 0x7f41581989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f415d87a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158106a80 0x7f415819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f415d87a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4158110e80 con 0x7f4158106a80 2024-09-16T00:31:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f415d87a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4158110d00 con 0x7f4158100a80 2024-09-16T00:31:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f415d87a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4158111000 con 0x7f4158101420 2024-09-16T00:31:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f41577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158106a80 0x7f415819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f41577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158106a80 0x7f415819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43744/0 (socket says 172.21.15.1:43744) 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f41577fe640 1 -- 172.21.15.1:0/2538547413 learned_addr learned my addr 172.21.15.1:0/2538547413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.879+0000 7f4156ffd640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4158100a80 0x7f4158198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f414ffff640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4158101420 0x7f41581989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f41577fe640 1 -- 172.21.15.1:0/2538547413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4158101420 msgr2=0x7f41581989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f41577fe640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4158101420 0x7f41581989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f41577fe640 1 -- 172.21.15.1:0/2538547413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4158100a80 msgr2=0x7f4158198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f41577fe640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4158100a80 0x7f4158198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f41577fe640 1 -- 172.21.15.1:0/2538547413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415819d440 con 0x7f4158106a80 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f414ffff640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4158101420 0x7f41581989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f41577fe640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158106a80 0x7f415819cd60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f415000eaa0 tx=0x7f415000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f415c878640 1 -- 172.21.15.1:0/2538547413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415000cdb0 con 0x7f4158106a80 2024-09-16T00:31:22.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f415c878640 1 -- 172.21.15.1:0/2538547413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4150005020 con 0x7f4158106a80 2024-09-16T00:31:22.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415819d730 con 0x7f4158106a80 2024-09-16T00:31:22.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f415c878640 1 -- 172.21.15.1:0/2538547413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41500116b0 con 0x7f4158106a80 2024-09-16T00:31:22.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.880+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41581a5010 con 0x7f4158106a80 2024-09-16T00:31:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.881+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4158100ee0 con 0x7f4158106a80 2024-09-16T00:31:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.882+0000 7f415c878640 1 -- 172.21.15.1:0/2538547413 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4150005560 con 0x7f4158106a80 2024-09-16T00:31:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.884+0000 7f415c878640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f41440a3240 0x7f41440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.884+0000 7f4156ffd640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f41440a3240 0x7f41440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.884+0000 7f415c878640 1 -- 172.21.15.1:0/2538547413 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f41500cfff0 con 0x7f4158106a80 2024-09-16T00:31:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.884+0000 7f4156ffd640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f41440a3240 0x7f41440a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4148009b50 tx=0x7f4148005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:22.888+0000 7f415c878640 1 -- 172.21.15.1:0/2538547413 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4150016030 con 0x7f4158106a80 2024-09-16T00:31:23.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.098+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4158109a70 con 0x7f4158106a80 2024-09-16T00:31:23.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.098+0000 7f415c878640 1 -- 172.21.15.1:0/2538547413 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f415009c8b0 con 0x7f4158106a80 2024-09-16T00:31:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:31:23.100 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:31:23.101 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:31:23.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.100+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f41440a3240 msgr2=0x7f41440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:23.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.100+0000 7f415d87a640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f41440a3240 0x7f41440a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4148009b50 tx=0x7f4148005f90 comp rx=0 tx=0).stop 2024-09-16T00:31:23.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.100+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158106a80 msgr2=0x7f415819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:23.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.100+0000 7f415d87a640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158106a80 0x7f415819cd60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f415000eaa0 tx=0x7f415000ef60 comp rx=0 tx=0).stop 2024-09-16T00:31:23.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.100+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 shutdown_connections 2024-09-16T00:31:23.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.100+0000 7f415d87a640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4158106a80 0x7f415819cd60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.100+0000 7f415d87a640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f41440a3240 0x7f41440a5700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.101+0000 7f415d87a640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4158101420 0x7f41581989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.101+0000 7f415d87a640 1 --2- 172.21.15.1:0/2538547413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4158100a80 0x7f4158198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.101+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 >> 172.21.15.1:0/2538547413 conn(0x7f41580fc7a0 msgr2=0x7f415806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:23.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.101+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 shutdown_connections 2024-09-16T00:31:23.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:23.101+0000 7f415d87a640 1 -- 172.21.15.1:0/2538547413 wait complete. 2024-09-16T00:31:23.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:23 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:31:23.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:23 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2538547413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:23.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:23 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:31:23.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:23 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2538547413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:23 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:23 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2538547413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:23.791 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-16T00:31:23.792 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-16T00:31:24.091 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:24.392 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:24 smithi001 ceph-mon[30934]: pgmap v440: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:31:24.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:24 smithi153 ceph-mon[35538]: pgmap v440: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:31:24.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:24 smithi019 ceph-mon[35186]: pgmap v440: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:31:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.609+0000 7f093cce2640 1 -- 172.21.15.1:0/2853427669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 msgr2=0x7f093810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.609+0000 7f093cce2640 1 --2- 172.21.15.1:0/2853427669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 0x7f093810da40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0930009ca0 tx=0x7f093002a950 comp rx=0 tx=0).stop 2024-09-16T00:31:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.609+0000 7f093cce2640 1 -- 172.21.15.1:0/2853427669 shutdown_connections 2024-09-16T00:31:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.609+0000 7f093cce2640 1 --2- 172.21.15.1:0/2853427669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 0x7f093810da40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.609+0000 7f093cce2640 1 --2- 172.21.15.1:0/2853427669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0938102570 0x7f09381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.609+0000 7f093cce2640 1 --2- 172.21.15.1:0/2853427669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0938100e20 0x7f0938101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.609+0000 7f093cce2640 1 -- 172.21.15.1:0/2853427669 >> 172.21.15.1:0/2853427669 conn(0x7f09380fc7d0 msgr2=0x7f09380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:24.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.609+0000 7f093cce2640 1 -- 172.21.15.1:0/2853427669 shutdown_connections 2024-09-16T00:31:24.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.609+0000 7f093cce2640 1 -- 172.21.15.1:0/2853427669 wait complete. 2024-09-16T00:31:24.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f093cce2640 1 Processor -- start 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f093cce2640 1 -- start start 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f093cce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0938100e20 0x7f0938197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f093cce2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0938102570 0x7f0938197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f093cce2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 0x7f093819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f093cce2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0938110f80 con 0x7f0938102ef0 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f093cce2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0938110e00 con 0x7f0938100e20 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f093cce2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0938111100 con 0x7f0938102570 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f0936d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 0x7f093819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f0936d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 0x7f093819bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43772/0 (socket says 172.21.15.1:43772) 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f0936d9d640 1 -- 172.21.15.1:0/3567691238 learned_addr learned my addr 172.21.15.1:0/3567691238 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.610+0000 7f093659c640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0938100e20 0x7f0938197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f0935d9b640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0938102570 0x7f0938197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f0936d9d640 1 -- 172.21.15.1:0/3567691238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0938102570 msgr2=0x7f0938197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f0936d9d640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0938102570 0x7f0938197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f0936d9d640 1 -- 172.21.15.1:0/3567691238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0938100e20 msgr2=0x7f0938197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f0936d9d640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0938100e20 0x7f0938197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f0936d9d640 1 -- 172.21.15.1:0/3567691238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093819c3b0 con 0x7f0938102ef0 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f093659c640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0938100e20 0x7f0938197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f0936d9d640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 0x7f093819bcd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f093002ae70 tx=0x7f093002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f092f7fe640 1 -- 172.21.15.1:0/3567691238 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930034040 con 0x7f0938102ef0 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f092f7fe640 1 -- 172.21.15.1:0/3567691238 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0930030920 con 0x7f0938102ef0 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f093cce2640 1 -- 172.21.15.1:0/3567691238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093819c640 con 0x7f0938102ef0 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f092f7fe640 1 -- 172.21.15.1:0/3567691238 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093003a5c0 con 0x7f0938102ef0 2024-09-16T00:31:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.611+0000 7f093cce2640 1 -- 172.21.15.1:0/3567691238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093819cac0 con 0x7f0938102ef0 2024-09-16T00:31:24.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.614+0000 7f092f7fe640 1 -- 172.21.15.1:0/3567691238 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f0930037020 con 0x7f0938102ef0 2024-09-16T00:31:24.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.614+0000 7f093f0bb740 1 Processor -- start 2024-09-16T00:31:24.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.614+0000 7f093f0bb740 1 -- start start 2024-09-16T00:31:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.614+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412de850 0x55bd412dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.614+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412df170 0x55bd412e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.614+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412e9a30 0x55bd412ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f093f0bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bd41243160 con 0x55bd412de850 2024-09-16T00:31:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f093f0bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bd411ea720 con 0x55bd412e9a30 2024-09-16T00:31:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f093f0bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bd4123f120 con 0x55bd412df170 2024-09-16T00:31:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f092f7fe640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09240a3240 0x7f09240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412de850 0x55bd412dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412de850 0x55bd412dec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43780/0 (socket says 172.21.15.1:43780) 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 -- 172.21.15.1:0/1250249798 learned_addr learned my addr 172.21.15.1:0/1250249798 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 --2- 172.21.15.1:0/1250249798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412e9a30 0x55bd412ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f093659c640 1 --2- 172.21.15.1:0/1250249798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412df170 0x55bd412e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0936d9d640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09240a3240 0x7f09240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f092f7fe640 1 -- 172.21.15.1:0/3567691238 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f09300f2550 con 0x7f0938102ef0 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 -- 172.21.15.1:0/1250249798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412df170 msgr2=0x55bd412e94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 --2- 172.21.15.1:0/1250249798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412df170 0x55bd412e94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 -- 172.21.15.1:0/1250249798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412e9a30 msgr2=0x55bd412ebe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 --2- 172.21.15.1:0/1250249798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412e9a30 0x55bd412ebe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 -- 172.21.15.1:0/1250249798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd40fb4ba0 con 0x55bd412de850 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0935d9b640 1 --2- 172.21.15.1:0/1250249798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412de850 0x55bd412dec30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f092000bfd0 tx=0x7f092000b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ebd102c222c242 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.615+0000 7f0936d9d640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09240a3240 0x7f09240a5700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f09300be470 tx=0x7f09300b3c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:24.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f092cff9640 1 -- 172.21.15.1:0/1250249798 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092000d6e0 con 0x55bd412de850 2024-09-16T00:31:24.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f092cff9640 1 -- 172.21.15.1:0/1250249798 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f092000dd60 con 0x55bd412de850 2024-09-16T00:31:24.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f092cff9640 1 -- 172.21.15.1:0/1250249798 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920011b50 con 0x55bd412de850 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f093f0bb740 1 -- 172.21.15.1:0/1250249798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412de850 msgr2=0x55bd412dec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f093f0bb740 1 --2- 172.21.15.1:0/1250249798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412de850 0x55bd412dec30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f092000bfd0 tx=0x7f092000b470 comp rx=0 tx=0).stop 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f093f0bb740 1 -- 172.21.15.1:0/1250249798 shutdown_connections 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f093f0bb740 1 --2- 172.21.15.1:0/1250249798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412e9a30 0x55bd412ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f093f0bb740 1 --2- 172.21.15.1:0/1250249798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412df170 0x55bd412e94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f093f0bb740 1 --2- 172.21.15.1:0/1250249798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412de850 0x55bd412dec30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f093f0bb740 1 -- 172.21.15.1:0/1250249798 >> 172.21.15.1:0/1250249798 conn(0x55bd412dade0 msgr2=0x55bd412db1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f093f0bb740 1 -- 172.21.15.1:0/1250249798 shutdown_connections 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.616+0000 7f093f0bb740 1 -- 172.21.15.1:0/1250249798 wait complete. 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093f0bb740 1 -- start start 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093f0bb740 1 Processor -- start 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412de850 0x55bd412e6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412e9a30 0x55bd412e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093f0bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bd412ec600 con 0x55bd412df170 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093f0bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bd411cb900 con 0x55bd412de850 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093f0bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bd412ec780 con 0x55bd412e9a30 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093f0bb740 10 client.0 _ll_register_callbacks cb 0x55bd411c9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43790/0 (socket says 172.21.15.1:43790) 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f093659c640 1 -- 172.21.15.1:0/231896835 learned_addr learned my addr 172.21.15.1:0/231896835 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.617+0000 7f0935d9b640 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412de850 0x55bd412e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f0935d9b640 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412e9a30 0x55bd412e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f093659c640 1 -- 172.21.15.1:0/231896835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412e9a30 msgr2=0x55bd412e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f093659c640 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412e9a30 0x55bd412e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f093659c640 1 -- 172.21.15.1:0/231896835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412de850 msgr2=0x55bd412e6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f093659c640 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412de850 0x55bd412e6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f093659c640 1 -- 172.21.15.1:0/231896835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd4120bc50 con 0x55bd412df170 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f0935d9b640 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412e9a30 0x55bd412e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f093659c640 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e6d90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0928012480 tx=0x7f0928012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8312c7397f62551d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f091dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f091dffb640 1 -- 172.21.15.1:0/231896835 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928004440 con 0x55bd412df170 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f091dffb640 1 -- 172.21.15.1:0/231896835 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f09280035f0 con 0x55bd412df170 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f093f0bb740 1 -- 172.21.15.1:0/231896835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f092000dd60 con 0x55bd412df170 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f091dffb640 1 -- 172.21.15.1:0/231896835 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f09280037f0 con 0x55bd412df170 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f093f0bb740 10 client.15978 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f093f0bb740 1 -- 172.21.15.1:0/231896835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd412f8270 con 0x55bd412df170 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.618+0000 7f091dffb640 1 -- 172.21.15.1:0/231896835 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f092000dd60 con 0x55bd412df170 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f091dffb640 1 -- 172.21.15.1:0/231896835 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0928019070 con 0x55bd412df170 2024-09-16T00:31:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f093f0bb740 10 client.15978 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f093f0bb740 10 client.15978 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f093f0bb740 20 client.15978 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f093f0bb740 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bd412f9730 0x55bd412fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f093f0bb740 4 client.15978 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f093f0bb740 1 -- 172.21.15.1:0/231896835 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd41250890 con 0x55bd412f9730 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f0935d9b640 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bd412f9730 0x55bd412fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f0935d9b640 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bd412f9730 0x55bd412fbbd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88dfae7406f02179 server_cookie=b52ac4a240fec57f in_seq=0 out_seq=0 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.619+0000 7f091dffb640 10 client.15978 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f091dffb640 1 -- 172.21.15.1:0/231896835 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd41250890 con 0x55bd412f9730 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f091dffb640 10 client.15978 handle_command_reply: tid=0 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f093f0bb740 1 client.15978 shutdown 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f093f0bb740 1 -- 172.21.15.1:0/231896835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 msgr2=0x55bd412e6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f093f0bb740 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e6d90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0928012480 tx=0x7f0928012940 comp rx=0 tx=0).stop 2024-09-16T00:31:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f093f0bb740 1 -- 172.21.15.1:0/231896835 shutdown_connections 2024-09-16T00:31:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f093f0bb740 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bd412f9730 0x55bd412fbbd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f093f0bb740 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412e9a30 0x55bd412e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f093f0bb740 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e6d90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f091dffb640 0 client.15978 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f093f0bb740 1 --2- 172.21.15.1:0/231896835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412de850 0x55bd412e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.620+0000 7f093f0bb740 1 -- 172.21.15.1:0/231896835 >> 172.21.15.1:0/231896835 conn(0x55bd412dade0 msgr2=0x55bd412e0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.621+0000 7f093f0bb740 1 -- 172.21.15.1:0/231896835 shutdown_connections 2024-09-16T00:31:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.621+0000 7f093f0bb740 1 -- 172.21.15.1:0/231896835 wait complete. 2024-09-16T00:31:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.621+0000 7f093f0bb740 20 client.15978 trim_cache size 0 max 16384 2024-09-16T00:31:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093f0bb740 1 Processor -- start 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093f0bb740 1 -- start start 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 0x55bd412e12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412e9a30 0x55bd412f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093f0bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f09381af510 con 0x55bd412df170 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093f0bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09381acde0 con 0x55bd412e9a30 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093f0bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09381aac30 con 0x55bd412de850 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43806/0 (socket says 172.21.15.1:43806) 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093659c640 1 -- 172.21.15.1:0/413501252 learned_addr learned my addr 172.21.15.1:0/413501252 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f0935d9b640 1 --2- 172.21.15.1:0/413501252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 0x55bd412e12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093659c640 1 -- 172.21.15.1:0/413501252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 msgr2=0x55bd412e12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093659c640 1 --2- 172.21.15.1:0/413501252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 0x55bd412e12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093659c640 1 -- 172.21.15.1:0/413501252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412e9a30 msgr2=0x55bd412f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:24.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093659c640 1 --2- 172.21.15.1:0/413501252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412e9a30 0x55bd412f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093659c640 1 -- 172.21.15.1:0/413501252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd412fc250 con 0x55bd412df170 2024-09-16T00:31:24.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f093659c640 1 --2- 172.21.15.1:0/413501252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e5790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0928000c00 tx=0x7f0928003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5876ba24d3941f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:24.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f091ffff640 1 -- 172.21.15.1:0/413501252 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928000e60 con 0x55bd412df170 2024-09-16T00:31:24.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f091ffff640 1 -- 172.21.15.1:0/413501252 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0928003e70 con 0x55bd412df170 2024-09-16T00:31:24.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.638+0000 7f091ffff640 1 -- 172.21.15.1:0/413501252 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928005620 con 0x55bd412df170 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 -- 172.21.15.1:0/413501252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 msgr2=0x55bd412e5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 --2- 172.21.15.1:0/413501252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e5790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0928000c00 tx=0x7f0928003780 comp rx=0 tx=0).stop 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 -- 172.21.15.1:0/413501252 shutdown_connections 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 --2- 172.21.15.1:0/413501252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412e9a30 0x55bd412f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 --2- 172.21.15.1:0/413501252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412df170 0x55bd412e5790 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 --2- 172.21.15.1:0/413501252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 0x55bd412e12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 -- 172.21.15.1:0/413501252 >> 172.21.15.1:0/413501252 conn(0x55bd412dade0 msgr2=0x55bd412e88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 -- 172.21.15.1:0/413501252 shutdown_connections 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 -- 172.21.15.1:0/413501252 wait complete. 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.639+0000 7f093f0bb740 1 -- start start 2024-09-16T00:31:24.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093f0bb740 1 Processor -- start 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 0x55bd412e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412df170 0x55bd412ea530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093f0bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412e9a30 0x55bd413614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093f0bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f09381ab2a0 con 0x55bd412e9a30 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093f0bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09381ac710 con 0x55bd412df170 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093f0bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09381ab970 con 0x55bd412de850 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093f0bb740 10 client.0 _ll_register_callbacks cb 0x7f092800b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412e9a30 0x55bd413614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412e9a30 0x55bd413614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43814/0 (socket says 172.21.15.1:43814) 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f0935d9b640 1 -- 172.21.15.1:0/2125803322 learned_addr learned my addr 172.21.15.1:0/2125803322 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093659c640 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412df170 0x55bd412ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f0935d9b640 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 0x55bd412e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f0935d9b640 1 -- 172.21.15.1:0/2125803322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 msgr2=0x55bd412e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f0935d9b640 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 0x55bd412e9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f0935d9b640 1 -- 172.21.15.1:0/2125803322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412df170 msgr2=0x55bd412ea530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f0935d9b640 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412df170 0x55bd412ea530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f0935d9b640 1 -- 172.21.15.1:0/2125803322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd412d63f0 con 0x55bd412e9a30 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.640+0000 7f093659c640 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412df170 0x55bd412ea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f0935d9b640 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412e9a30 0x55bd413614b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0920006010 tx=0x7f0920022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701752f1d6318b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f091dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f091dffb640 1 -- 172.21.15.1:0/2125803322 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092002f060 con 0x55bd412e9a30 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f091dffb640 1 -- 172.21.15.1:0/2125803322 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0920011400 con 0x55bd412e9a30 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f093f0bb740 1 -- 172.21.15.1:0/2125803322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bd4124abe0 con 0x55bd412e9a30 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f091dffb640 1 -- 172.21.15.1:0/2125803322 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f09200140d0 con 0x55bd412e9a30 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f093f0bb740 10 client.15984 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f093f0bb740 1 -- 172.21.15.1:0/2125803322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd412dbe90 con 0x55bd412e9a30 2024-09-16T00:31:24.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f091dffb640 1 -- 172.21.15.1:0/2125803322 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bd4124abe0 con 0x55bd412e9a30 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f091dffb640 1 -- 172.21.15.1:0/2125803322 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f09200053e0 con 0x55bd412e9a30 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f093f0bb740 10 client.15984 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f093f0bb740 10 client.15984 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f093f0bb740 20 client.15984 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f093f0bb740 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bd41369180 0x55bd4136b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f093f0bb740 4 client.15984 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.641+0000 7f093f0bb740 1 -- 172.21.15.1:0/2125803322 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x55bd411c6f40 con 0x55bd41369180 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.642+0000 7f093659c640 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bd41369180 0x55bd4136b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.642+0000 7f093659c640 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bd41369180 0x55bd4136b570 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=366efbdf06f8a698 server_cookie=994a1a653aeffa9f in_seq=0 out_seq=0 2024-09-16T00:31:24.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.642+0000 7f091dffb640 10 client.15984 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f091dffb640 1 -- 172.21.15.1:0/2125803322 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bd411c6f40 con 0x55bd41369180 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f091dffb640 10 client.15984 handle_command_reply: tid=0 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 client.15984 shutdown 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 -- 172.21.15.1:0/2125803322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412e9a30 msgr2=0x55bd413614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412e9a30 0x55bd413614b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0920006010 tx=0x7f0920022040 comp rx=0 tx=0).stop 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 -- 172.21.15.1:0/2125803322 shutdown_connections 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55bd41369180 0x55bd4136b570 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd412e9a30 0x55bd413614b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd412df170 0x55bd412ea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 --2- 172.21.15.1:0/2125803322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd412de850 0x55bd412e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 -- 172.21.15.1:0/2125803322 >> 172.21.15.1:0/2125803322 conn(0x55bd412dade0 msgr2=0x55bd412e66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f091dffb640 0 client.15984 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 -- 172.21.15.1:0/2125803322 shutdown_connections 2024-09-16T00:31:24.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 1 -- 172.21.15.1:0/2125803322 wait complete. 2024-09-16T00:31:24.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.654+0000 7f093f0bb740 20 client.15984 trim_cache size 0 max 16384 2024-09-16T00:31:24.657 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:31:24.657 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:31:24.657 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:31:24.657 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 10, 2024-09-16T00:31:24.657 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:31:24.657 INFO:teuthology.orchestra.run.smithi001.stdout: "9db30367": { 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 10, 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 139.6, 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:31:24.658 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:31:24.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:24.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:24.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:24.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:24.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:24.659 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:31:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 -- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09240a3240 msgr2=0x7f09240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09240a3240 0x7f09240a5700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f09300be470 tx=0x7f09300b3c50 comp rx=0 tx=0).stop 2024-09-16T00:31:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 -- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 msgr2=0x7f093819bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 0x7f093819bcd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f093002ae70 tx=0x7f093002f5d0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 -- 172.21.15.1:0/3567691238 shutdown_connections 2024-09-16T00:31:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0938102ef0 0x7f093819bcd0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09240a3240 0x7f09240a5700 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0938102570 0x7f0938197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 --2- 172.21.15.1:0/3567691238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0938100e20 0x7f0938197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 -- 172.21.15.1:0/3567691238 >> 172.21.15.1:0/3567691238 conn(0x7f09380fc7d0 msgr2=0x7f0938106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 -- 172.21.15.1:0/3567691238 shutdown_connections 2024-09-16T00:31:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:24.655+0000 7f093cce2640 1 -- 172.21.15.1:0/3567691238 wait complete. 2024-09-16T00:31:25.538 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9db30367', quiesce took 3.6 seconds. Will release after: 15.950000000000001 2024-09-16T00:31:26.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:26 smithi001 ceph-mon[30934]: pgmap v441: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-16T00:31:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:26 smithi153 ceph-mon[35538]: pgmap v441: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-16T00:31:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:26 smithi019 ceph-mon[35186]: pgmap v441: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-16T00:31:28.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:28 smithi001 ceph-mon[30934]: pgmap v442: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:31:28.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:28 smithi153 ceph-mon[35538]: pgmap v442: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:31:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:28 smithi019 ceph-mon[35186]: pgmap v442: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:31:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:30 smithi001 ceph-mon[30934]: pgmap v443: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-09-16T00:31:30.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:30 smithi153 ceph-mon[35538]: pgmap v443: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-09-16T00:31:30.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:30 smithi019 ceph-mon[35186]: pgmap v443: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-09-16T00:31:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:32 smithi001 ceph-mon[30934]: pgmap v444: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-09-16T00:31:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:32 smithi153 ceph-mon[35538]: pgmap v444: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-09-16T00:31:32.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:32 smithi019 ceph-mon[35186]: pgmap v444: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-09-16T00:31:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:34 smithi001 ceph-mon[30934]: pgmap v445: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-09-16T00:31:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:34 smithi153 ceph-mon[35538]: pgmap v445: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-09-16T00:31:34.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:34 smithi019 ceph-mon[35186]: pgmap v445: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-09-16T00:31:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:36 smithi001 ceph-mon[30934]: pgmap v446: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-09-16T00:31:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:36 smithi153 ceph-mon[35538]: pgmap v446: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-09-16T00:31:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:36 smithi019 ceph-mon[35186]: pgmap v446: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-09-16T00:31:38.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:38 smithi001 ceph-mon[30934]: pgmap v447: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-16T00:31:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:38 smithi153 ceph-mon[35538]: pgmap v447: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-16T00:31:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:38 smithi019 ceph-mon[35186]: pgmap v447: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-16T00:31:40.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:40 smithi001 ceph-mon[30934]: pgmap v448: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-16T00:31:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:40 smithi153 ceph-mon[35538]: pgmap v448: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-16T00:31:40.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:40 smithi019 ceph-mon[35186]: pgmap v448: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-16T00:31:41.490 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:31:41.762 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 -- 172.21.15.1:0/943198731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 msgr2=0x7fd078103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/943198731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 0x7fd078103120 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd068009900 tx=0x7fd06802a480 comp rx=0 tx=0).stop 2024-09-16T00:31:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 -- 172.21.15.1:0/943198731 shutdown_connections 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/943198731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd07810c170 0x7fd07810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/943198731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd078103660 0x7fd07810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/943198731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 0x7fd078103120 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 -- 172.21.15.1:0/943198731 >> 172.21.15.1:0/943198731 conn(0x7fd0780fc7a0 msgr2=0x7fd0780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 -- 172.21.15.1:0/943198731 shutdown_connections 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 -- 172.21.15.1:0/943198731 wait complete. 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.241+0000 7fd07c8d9640 1 Processor -- start 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07c8d9640 1 -- start start 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07c8d9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 0x7fd0781009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07c8d9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd078103660 0x7fd078100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07c8d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd07810c170 0x7fd078101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07c8d9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd078111f70 con 0x7fd078102d40 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07c8d9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd078111df0 con 0x7fd07810c170 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07c8d9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781120f0 con 0x7fd078103660 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 0x7fd0781009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 0x7fd0781009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48012/0 (socket says 172.21.15.1:48012) 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07659c640 1 -- 172.21.15.1:0/2691172541 learned_addr learned my addr 172.21.15.1:0/2691172541 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd075d9b640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd078103660 0x7fd078100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd076d9d640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd07810c170 0x7fd078101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07659c640 1 -- 172.21.15.1:0/2691172541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd078103660 msgr2=0x7fd078100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07659c640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd078103660 0x7fd078100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07659c640 1 -- 172.21.15.1:0/2691172541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd07810c170 msgr2=0x7fd078101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07659c640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd07810c170 0x7fd078101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07659c640 1 -- 172.21.15.1:0/2691172541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0781a68b0 con 0x7fd078102d40 2024-09-16T00:31:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.242+0000 7fd07659c640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 0x7fd0781009a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd068006010 tx=0x7fd0680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.243+0000 7fd0677fe640 1 -- 172.21.15.1:0/2691172541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06803f030 con 0x7fd078102d40 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.243+0000 7fd0677fe640 1 -- 172.21.15.1:0/2691172541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd068003ce0 con 0x7fd078102d40 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.243+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0781a6b40 con 0x7fd078102d40 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.243+0000 7fd0677fe640 1 -- 172.21.15.1:0/2691172541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068035070 con 0x7fd078102d40 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.243+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0781a7020 con 0x7fd078102d40 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.244+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd078103120 con 0x7fd078102d40 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.245+0000 7fd0677fe640 1 -- 172.21.15.1:0/2691172541 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd068042030 con 0x7fd078102d40 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.245+0000 7fd0677fe640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0600a3240 0x7fd0600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.246+0000 7fd075d9b640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0600a3240 0x7fd0600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.246+0000 7fd0677fe640 1 -- 172.21.15.1:0/2691172541 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd0680efe80 con 0x7fd078102d40 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.246+0000 7fd075d9b640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0600a3240 0x7fd0600a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd078102430 tx=0x7fd06c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.250+0000 7fd0677fe640 1 -- 172.21.15.1:0/2691172541 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0680bc1b0 con 0x7fd078102d40 2024-09-16T00:31:42.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:42 smithi001 ceph-mon[30934]: pgmap v449: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:31:42.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:42 smithi153 ceph-mon[35538]: pgmap v449: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:31:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.462+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd078106c80 con 0x7fd078102d40 2024-09-16T00:31:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.462+0000 7fd0677fe640 1 -- 172.21.15.1:0/2691172541 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fd0680c04b0 con 0x7fd078102d40 2024-09-16T00:31:42.463 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:31:42.463 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:31:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:31:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0600a3240 msgr2=0x7fd0600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0600a3240 0x7fd0600a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd078102430 tx=0x7fd06c00a250 comp rx=0 tx=0).stop 2024-09-16T00:31:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 msgr2=0x7fd0781009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 0x7fd0781009a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd068006010 tx=0x7fd0680042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 shutdown_connections 2024-09-16T00:31:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd07810c170 0x7fd078101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd0600a3240 0x7fd0600a5700 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd078103660 0x7fd078100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 --2- 172.21.15.1:0/2691172541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd078102d40 0x7fd0781009a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.464+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 >> 172.21.15.1:0/2691172541 conn(0x7fd0780fc7a0 msgr2=0x7fd0780fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.465+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 shutdown_connections 2024-09-16T00:31:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:42.465+0000 7fd07c8d9640 1 -- 172.21.15.1:0/2691172541 wait complete. 2024-09-16T00:31:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:42 smithi019 ceph-mon[35186]: pgmap v449: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:31:43.309 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 9db30367 --release --await' 2024-09-16T00:31:43.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 9db30367 --release --await 2024-09-16T00:31:43.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:43 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2691172541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:43.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:43 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2691172541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:43.570 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:43 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2691172541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:44.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.124+0000 7f6b76e9a640 1 -- 172.21.15.1:0/670027658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70102db0 msgr2=0x7f6b70103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.124+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/670027658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70102db0 0x7f6b70103190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009930 tx=0x7f6b6002a420 comp rx=0 tx=0).stop 2024-09-16T00:31:44.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 -- 172.21.15.1:0/670027658 shutdown_connections 2024-09-16T00:31:44.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/670027658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b701082e0 0x7f6b7010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/670027658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b70103760 0x7f6b70107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/670027658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70102db0 0x7f6b70103190 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 -- 172.21.15.1:0/670027658 >> 172.21.15.1:0/670027658 conn(0x7f6b700fc7d0 msgr2=0x7f6b700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:44.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 -- 172.21.15.1:0/670027658 shutdown_connections 2024-09-16T00:31:44.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 -- 172.21.15.1:0/670027658 wait complete. 2024-09-16T00:31:44.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 Processor -- start 2024-09-16T00:31:44.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 -- start start 2024-09-16T00:31:44.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b70102db0 0x7f6b70198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.125+0000 7f6b76e9a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70103760 0x7f6b70198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b76e9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b701082e0 0x7f6b7019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b76e9a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b701120b0 con 0x7f6b70103760 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b76e9a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70111f30 con 0x7f6b701082e0 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b76e9a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70112230 con 0x7f6b70102db0 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70103760 0x7f6b70198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70103760 0x7f6b70198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48030/0 (socket says 172.21.15.1:48030) 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6ffff640 1 -- 172.21.15.1:0/3440753360 learned_addr learned my addr 172.21.15.1:0/3440753360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b74c36640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b70102db0 0x7f6b70198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b75437640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b701082e0 0x7f6b7019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6ffff640 1 -- 172.21.15.1:0/3440753360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b70102db0 msgr2=0x7f6b70198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b70102db0 0x7f6b70198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6ffff640 1 -- 172.21.15.1:0/3440753360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b701082e0 msgr2=0x7f6b7019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b701082e0 0x7f6b7019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6ffff640 1 -- 172.21.15.1:0/3440753360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7019d500 con 0x7f6b70103760 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b74c36640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b70102db0 0x7f6b70198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:44.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b75437640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b701082e0 0x7f6b7019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70103760 0x7f6b70198a90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c8c0 tx=0x7f6b6400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6dffb640 1 -- 172.21.15.1:0/3440753360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64007bb0 con 0x7f6b70103760 2024-09-16T00:31:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6dffb640 1 -- 172.21.15.1:0/3440753360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b64007d50 con 0x7f6b70103760 2024-09-16T00:31:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b76e9a640 1 -- 172.21.15.1:0/3440753360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7019d7f0 con 0x7f6b70103760 2024-09-16T00:31:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.126+0000 7f6b6dffb640 1 -- 172.21.15.1:0/3440753360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64018830 con 0x7f6b70103760 2024-09-16T00:31:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.127+0000 7f6b76e9a640 1 -- 172.21.15.1:0/3440753360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b701a50d0 con 0x7f6b70103760 2024-09-16T00:31:44.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b6dffb640 1 -- 172.21.15.1:0/3440753360 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6b64018a10 con 0x7f6b70103760 2024-09-16T00:31:44.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b79273740 1 Processor -- start 2024-09-16T00:31:44.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b79273740 1 -- start start 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b79273740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 0x55d5a74d6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b79273740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b79273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b79273740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d5a743b620 con 0x55d5a74d6980 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b79273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d5a73e2e50 con 0x55d5a74e1240 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b79273740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5a74375e0 con 0x55d5a74d6060 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b74c36640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b74c36640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74e0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48036/0 (socket says 172.21.15.1:48036) 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b74c36640 1 -- 172.21.15.1:0/3858721049 learned_addr learned my addr 172.21.15.1:0/3858721049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b75437640 1 --2- 172.21.15.1:0/3858721049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 0x55d5a74d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b6dffb640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6b580a32c0 0x7f6b580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/3858721049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b74c36640 1 -- 172.21.15.1:0/3858721049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 msgr2=0x55d5a74d6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b74c36640 1 --2- 172.21.15.1:0/3858721049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 0x55d5a74d6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b74c36640 1 -- 172.21.15.1:0/3858721049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 msgr2=0x55d5a74e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.130+0000 7f6b74c36640 1 --2- 172.21.15.1:0/3858721049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.131+0000 7f6b74c36640 1 -- 172.21.15.1:0/3858721049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a73d3d40 con 0x55d5a74d6980 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.131+0000 7f6b75437640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6b580a32c0 0x7f6b580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.131+0000 7f6b6dffb640 1 -- 172.21.15.1:0/3440753360 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6b640cfe50 con 0x7f6b70103760 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.131+0000 7f6b74c36640 1 --2- 172.21.15.1:0/3858721049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74e0d00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6b600314e0 tx=0x7f6b60004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e932e26a72ab3e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.131+0000 7f6b56ffd640 1 -- 172.21.15.1:0/3858721049 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60035030 con 0x55d5a74d6980 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.131+0000 7f6b56ffd640 1 -- 172.21.15.1:0/3858721049 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b60005520 con 0x55d5a74d6980 2024-09-16T00:31:44.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.131+0000 7f6b75437640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6b580a32c0 0x7f6b580a5780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b68005fd0 tx=0x7f6b68004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.131+0000 7f6b56ffd640 1 -- 172.21.15.1:0/3858721049 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60003850 con 0x55d5a74d6980 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 -- 172.21.15.1:0/3858721049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 msgr2=0x55d5a74e0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 --2- 172.21.15.1:0/3858721049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74e0d00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6b600314e0 tx=0x7f6b60004200 comp rx=0 tx=0).stop 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 -- 172.21.15.1:0/3858721049 shutdown_connections 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 --2- 172.21.15.1:0/3858721049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 --2- 172.21.15.1:0/3858721049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74e0d00 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 --2- 172.21.15.1:0/3858721049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 0x55d5a74d6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 -- 172.21.15.1:0/3858721049 >> 172.21.15.1:0/3858721049 conn(0x55d5a74d25f0 msgr2=0x55d5a74d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 -- 172.21.15.1:0/3858721049 shutdown_connections 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 -- 172.21.15.1:0/3858721049 wait complete. 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 -- start start 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.132+0000 7f6b79273740 1 Processor -- start 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b79273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74d6060 0x55d5a74de130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b79273740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b79273740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74e1240 0x55d5a74debb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b79273740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d5a74e3e10 con 0x55d5a74d6980 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b79273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d5a73c3f10 con 0x55d5a74d6060 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b79273740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5a74e3f90 con 0x55d5a74e1240 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b79273740 10 client.0 _ll_register_callbacks cb 0x55d5a73e7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74de670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48040/0 (socket says 172.21.15.1:48040) 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b6ffff640 1 -- 172.21.15.1:0/1958348519 learned_addr learned my addr 172.21.15.1:0/1958348519 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b74c36640 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74d6060 0x55d5a74de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b75437640 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74e1240 0x55d5a74debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b6ffff640 1 -- 172.21.15.1:0/1958348519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74e1240 msgr2=0x55d5a74debb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74e1240 0x55d5a74debb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b6ffff640 1 -- 172.21.15.1:0/1958348519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74d6060 msgr2=0x55d5a74de130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74d6060 0x55d5a74de130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b6ffff640 1 -- 172.21.15.1:0/1958348519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a714a580 con 0x55d5a74d6980 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b74c36640 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74d6060 0x55d5a74de130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74de670 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6b640984d0 tx=0x7f6b6401f300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f267c3492af098e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.133+0000 7f6b4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b4ffff640 1 -- 172.21.15.1:0/1958348519 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6409c070 con 0x55d5a74d6980 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b79273740 1 -- 172.21.15.1:0/1958348519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b60005520 con 0x55d5a74d6980 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b4ffff640 1 -- 172.21.15.1:0/1958348519 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6b6400fac0 con 0x55d5a74d6980 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b4ffff640 1 -- 172.21.15.1:0/1958348519 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b640a0420 con 0x55d5a74d6980 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b79273740 10 client.16002 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b79273740 1 -- 172.21.15.1:0/1958348519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5a7214960 con 0x55d5a74d6980 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b4ffff640 1 -- 172.21.15.1:0/1958348519 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b60005520 con 0x55d5a74d6980 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b4ffff640 1 -- 172.21.15.1:0/1958348519 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b64096c50 con 0x55d5a74d6980 2024-09-16T00:31:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b79273740 10 client.16002 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b79273740 10 client.16002 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:31:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b79273740 20 client.16002 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b79273740 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5a74f12d0 0x55d5a74f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b79273740 4 client.16002 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.134+0000 7f6b79273740 1 -- 172.21.15.1:0/1958348519 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5a7448b90 con 0x55d5a74f12d0 2024-09-16T00:31:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.135+0000 7f6b74c36640 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5a74f12d0 0x55d5a74f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.135+0000 7f6b74c36640 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5a74f12d0 0x55d5a74f3760 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748ac57eb32ed380 server_cookie=6e7179be7e1b6619 in_seq=0 out_seq=0 2024-09-16T00:31:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.135+0000 7f6b4ffff640 10 client.16002 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b4ffff640 1 -- 172.21.15.1:0/1958348519 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5a7448b90 con 0x55d5a74f12d0 2024-09-16T00:31:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b4ffff640 10 client.16002 handle_command_reply: tid=0 2024-09-16T00:31:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 client.16002 shutdown 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 -- 172.21.15.1:0/1958348519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 msgr2=0x55d5a74de670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74de670 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6b640984d0 tx=0x7f6b6401f300 comp rx=0 tx=0).stop 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 -- 172.21.15.1:0/1958348519 shutdown_connections 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5a74f12d0 0x55d5a74f3760 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74e1240 0x55d5a74debb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74de670 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 --2- 172.21.15.1:0/1958348519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74d6060 0x55d5a74de130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 -- 172.21.15.1:0/1958348519 >> 172.21.15.1:0/1958348519 conn(0x55d5a74d25f0 msgr2=0x55d5a74d82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b4ffff640 0 client.16002 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 -- 172.21.15.1:0/1958348519 shutdown_connections 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 1 -- 172.21.15.1:0/1958348519 wait complete. 2024-09-16T00:31:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.136+0000 7f6b79273740 20 client.16002 trim_cache size 0 max 16384 2024-09-16T00:31:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.154+0000 7f6b79273740 1 Processor -- start 2024-09-16T00:31:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.154+0000 7f6b79273740 1 -- start start 2024-09-16T00:31:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.154+0000 7f6b79273740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 0x55d5a74d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.154+0000 7f6b79273740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.154+0000 7f6b79273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b79273740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b701b1640 con 0x55d5a74d6980 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b79273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b701aef10 con 0x55d5a74e1240 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b79273740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b701acd60 con 0x55d5a74d6060 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74f9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48046/0 (socket says 172.21.15.1:48046) 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b6ffff640 1 -- 172.21.15.1:0/2342231727 learned_addr learned my addr 172.21.15.1:0/2342231727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b74c36640 1 --2- 172.21.15.1:0/2342231727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 0x55d5a74d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b75437640 1 --2- 172.21.15.1:0/2342231727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b6ffff640 1 -- 172.21.15.1:0/2342231727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 msgr2=0x55d5a74d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/2342231727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 0x55d5a74d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b6ffff640 1 -- 172.21.15.1:0/2342231727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 msgr2=0x55d5a74f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:44.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/2342231727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b6ffff640 1 -- 172.21.15.1:0/2342231727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a74dd0e0 con 0x55d5a74d6980 2024-09-16T00:31:44.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/2342231727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74f9040 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b6409c6d0 tx=0x7f6b640d01e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4c1c3d207400f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:44.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b567fc640 1 -- 172.21.15.1:0/2342231727 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6400fac0 con 0x55d5a74d6980 2024-09-16T00:31:44.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b567fc640 1 -- 172.21.15.1:0/2342231727 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b6409c320 con 0x55d5a74d6980 2024-09-16T00:31:44.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.155+0000 7f6b567fc640 1 -- 172.21.15.1:0/2342231727 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64096be0 con 0x55d5a74d6980 2024-09-16T00:31:44.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 -- 172.21.15.1:0/2342231727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 msgr2=0x55d5a74f9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 --2- 172.21.15.1:0/2342231727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74f9040 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b6409c6d0 tx=0x7f6b640d01e0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 -- 172.21.15.1:0/2342231727 shutdown_connections 2024-09-16T00:31:44.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 --2- 172.21.15.1:0/2342231727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 --2- 172.21.15.1:0/2342231727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6980 0x55d5a74f9040 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 --2- 172.21.15.1:0/2342231727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6060 0x55d5a74d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 -- 172.21.15.1:0/2342231727 >> 172.21.15.1:0/2342231727 conn(0x55d5a74d25f0 msgr2=0x55d5a74d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 -- 172.21.15.1:0/2342231727 shutdown_connections 2024-09-16T00:31:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 -- 172.21.15.1:0/2342231727 wait complete. 2024-09-16T00:31:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.156+0000 7f6b79273740 1 -- start start 2024-09-16T00:31:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b79273740 1 Processor -- start 2024-09-16T00:31:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b79273740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6060 0x55d5a74e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b79273740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6980 0x55d5a74e5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b79273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b79273740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b701ad3d0 con 0x55d5a74d6060 2024-09-16T00:31:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b79273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b701ae840 con 0x55d5a74e1240 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b79273740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b701adaa0 con 0x55d5a74d6980 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b79273740 10 client.0 _ll_register_callbacks cb 0x7f6b6409bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b74c36640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6060 0x55d5a74e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b74c36640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6060 0x55d5a74e4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48060/0 (socket says 172.21.15.1:48060) 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b74c36640 1 -- 172.21.15.1:0/433001809 learned_addr learned my addr 172.21.15.1:0/433001809 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6980 0x55d5a74e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b74c36640 1 -- 172.21.15.1:0/433001809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6980 msgr2=0x55d5a74e5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b74c36640 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6980 0x55d5a74e5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b75437640 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b74c36640 1 -- 172.21.15.1:0/433001809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 msgr2=0x55d5a74e5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b74c36640 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74e5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.157+0000 7f6b74c36640 1 -- 172.21.15.1:0/433001809 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a74d67b0 con 0x55d5a74d6060 2024-09-16T00:31:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b74c36640 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6060 0x55d5a74e4fd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55d5a74d3450 tx=0x7f6b6002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fcdb44694dc54f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b4ffff640 1 -- 172.21.15.1:0/433001809 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60035040 con 0x55d5a74d6060 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b4ffff640 1 -- 172.21.15.1:0/433001809 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6b60004d80 con 0x55d5a74d6060 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b79273740 1 -- 172.21.15.1:0/433001809 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b6409c320 con 0x55d5a74d6060 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b4ffff640 1 -- 172.21.15.1:0/433001809 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b6005d050 con 0x55d5a74d6060 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b79273740 10 client.16008 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b79273740 1 -- 172.21.15.1:0/433001809 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5a75604f0 con 0x55d5a74d6060 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b4ffff640 1 -- 172.21.15.1:0/433001809 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b6409c320 con 0x55d5a74d6060 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b4ffff640 1 -- 172.21.15.1:0/433001809 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b60003bb0 con 0x55d5a74d6060 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b79273740 10 client.16008 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b79273740 10 client.16008 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b79273740 20 client.16008 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b79273740 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5a7560890 0x55d5a7562c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b79273740 4 client.16008 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9db30367", "release": true, "await": true}] 2024-09-16T00:31:44.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.158+0000 7f6b79273740 1 -- 172.21.15.1:0/433001809 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9db30367", "release": true, "await": true}) -- 0x55d5a74430a0 con 0x55d5a7560890 2024-09-16T00:31:44.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.159+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5a7560890 0x55d5a7562c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.159+0000 7f6b6ffff640 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5a7560890 0x55d5a7562c80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab2df8d881a374dc server_cookie=58687747833d7273 in_seq=0 out_seq=0 2024-09-16T00:31:44.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.159+0000 7f6b4ffff640 10 client.16008 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.161+0000 7f6b4ffff640 1 -- 172.21.15.1:0/433001809 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d5a74430a0 con 0x55d5a7560890 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.161+0000 7f6b4ffff640 10 client.16008 handle_command_reply: tid=0 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.161+0000 7f6b79273740 1 client.16008 shutdown 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 -- 172.21.15.1:0/433001809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6060 msgr2=0x55d5a74e4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6060 0x55d5a74e4fd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55d5a74d3450 tx=0x7f6b6002a420 comp rx=0 tx=0).stop 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 -- 172.21.15.1:0/433001809 shutdown_connections 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d5a7560890 0x55d5a7562c80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5a74e1240 0x55d5a74e5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5a74d6980 0x55d5a74e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b4ffff640 0 client.16008 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 --2- 172.21.15.1:0/433001809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5a74d6060 0x55d5a74e4fd0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 -- 172.21.15.1:0/433001809 >> 172.21.15.1:0/433001809 conn(0x55d5a74d25f0 msgr2=0x55d5a74e01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 -- 172.21.15.1:0/433001809 shutdown_connections 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 1 -- 172.21.15.1:0/433001809 wait complete. 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b79273740 20 client.16008 trim_cache size 0 max 16384 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 12, 2024-09-16T00:31:44.166 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "9db30367": { 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 12, 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 139.6, 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:31:44.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:31:44.168 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:31:44.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:44.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:44.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:44.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:44.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:31:44.168 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:31:44.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b76e9a640 1 -- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6b580a32c0 msgr2=0x7f6b580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6b580a32c0 0x7f6b580a5780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b68005fd0 tx=0x7f6b68004a40 comp rx=0 tx=0).stop 2024-09-16T00:31:44.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b76e9a640 1 -- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70103760 msgr2=0x7f6b70198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.162+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70103760 0x7f6b70198a90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c8c0 tx=0x7f6b6400cd80 comp rx=0 tx=0).stop 2024-09-16T00:31:44.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.163+0000 7f6b76e9a640 1 -- 172.21.15.1:0/3440753360 shutdown_connections 2024-09-16T00:31:44.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.163+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b701082e0 0x7f6b7019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.163+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70103760 0x7f6b70198a90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.163+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6b580a32c0 0x7f6b580a5780 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.163+0000 7f6b76e9a640 1 --2- 172.21.15.1:0/3440753360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b70102db0 0x7f6b70198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.163+0000 7f6b76e9a640 1 -- 172.21.15.1:0/3440753360 >> 172.21.15.1:0/3440753360 conn(0x7f6b700fc7d0 msgr2=0x7f6b70106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.163+0000 7f6b76e9a640 1 -- 172.21.15.1:0/3440753360 shutdown_connections 2024-09-16T00:31:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:44.163+0000 7f6b76e9a640 1 -- 172.21.15.1:0/3440753360 wait complete. 2024-09-16T00:31:44.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:44 smithi001 ceph-mon[30934]: pgmap v450: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:31:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:44 smithi153 ceph-mon[35538]: pgmap v450: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:31:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:44 smithi019 ceph-mon[35186]: pgmap v450: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:31:45.025 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9db30367', total seconds elapsed: 23.1 2024-09-16T00:31:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:46 smithi001 ceph-mon[30934]: pgmap v451: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:31:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:46 smithi153 ceph-mon[35538]: pgmap v451: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:31:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:46 smithi019 ceph-mon[35186]: pgmap v451: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:31:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:48 smithi001 ceph-mon[30934]: pgmap v452: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T00:31:48.464 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:48 smithi153 ceph-mon[35538]: pgmap v452: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T00:31:48.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:48 smithi019 ceph-mon[35186]: pgmap v452: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T00:31:49.202 INFO:tasks.workunit.client.0.smithi001.stdout:1/600: open fd = 3 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:2/600: open fd = 3 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:3/600: open fd = 3 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.203 INFO:tasks.workunit.client.0.smithi001.stdout:4/600: open fd = 3 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:5/600: open fd = 3 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:6/600: open fd = 3 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:7/600: open fd = 3 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.204 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:8/600: open fd = 3 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:9/600: open fd = 3 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:10/600: open fd = 3 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.205 INFO:tasks.workunit.client.0.smithi001.stdout:11/600: open fd = 3 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:12/600: open fd = 3 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:13/600: open fd = 3 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:14/600: open fd = 3 2024-09-16T00:31:49.206 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:15/600: open fd = 3 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:16/600: open fd = 3 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:17/600: open fd = 3 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.207 INFO:tasks.workunit.client.0.smithi001.stdout:18/600: open fd = 3 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:19/600: open fd = 3 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:20/600: open fd = 3 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:21/600: open fd = 3 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:22/600: open fd = 3 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:23/600: open fd = 3 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:24/600: open fd = 3 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:25/600: open fd = 3 2024-09-16T00:31:49.209 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:26/600: open fd = 3 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:27/600: open fd = 3 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:28/600: open fd = 3 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.210 INFO:tasks.workunit.client.0.smithi001.stdout:29/600: open fd = 3 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:30/600: open fd = 3 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:31/600: open fd = 3 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:32/600: open fd = 3 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.211 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:33/600: open fd = 3 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:34/600: open fd = 3 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:35/600: open fd = 3 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:36/600: open fd = 3 2024-09-16T00:31:49.212 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:37/600: open fd = 3 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:38/600: open fd = 3 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:39/600: open fd = 3 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.213 INFO:tasks.workunit.client.0.smithi001.stdout:40/600: open fd = 3 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:41/600: open fd = 3 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:42/600: open fd = 3 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:43/600: open fd = 3 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.214 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:44/600: open fd = 3 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:45/600: open fd = 3 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:46/600: open fd = 3 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.215 INFO:tasks.workunit.client.0.smithi001.stdout:47/600: open fd = 3 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:48/600: open fd = 3 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:49/600: open fd = 3 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:50/600: open fd = 3 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.216 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:51/600: open fd = 3 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:52/600: open fd = 3 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:53/600: open fd = 3 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.217 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:54/600: open fd = 3 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:55/600: open fd = 3 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:56/600: open fd = 3 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.218 INFO:tasks.workunit.client.0.smithi001.stdout:57/600: open fd = 3 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:58/600: open fd = 3 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:59/600: open fd = 3 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:60/600: open fd = 3 2024-09-16T00:31:49.219 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:61/600: open fd = 3 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:62/600: open fd = 3 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:63/600: open fd = 3 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:31:49.220 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:31:50.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:50 smithi001 ceph-mon[30934]: pgmap v453: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:31:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:50 smithi153 ceph-mon[35538]: pgmap v453: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:31:50.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:50 smithi019 ceph-mon[35186]: pgmap v453: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:31:52.394 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:31:52.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:52 smithi001 ceph-mon[30934]: pgmap v454: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:31:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:52 smithi153 ceph-mon[35538]: pgmap v454: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:31:52.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:52 smithi019 ceph-mon[35186]: pgmap v454: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:31:52.670 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:53.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.186+0000 7fbb70ef2640 1 -- 172.21.15.1:0/712890343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102ef0 msgr2=0x7fbb6c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.186+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/712890343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102ef0 0x7fbb6c10da40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009ca0 tx=0x7fbb6402a950 comp rx=0 tx=0).stop 2024-09-16T00:31:53.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.186+0000 7fbb70ef2640 1 -- 172.21.15.1:0/712890343 shutdown_connections 2024-09-16T00:31:53.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.186+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/712890343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102ef0 0x7fbb6c10da40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.186+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/712890343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb6c102570 0x7fbb6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.186+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/712890343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c100e20 0x7fbb6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.186+0000 7fbb70ef2640 1 -- 172.21.15.1:0/712890343 >> 172.21.15.1:0/712890343 conn(0x7fbb6c0fc7d0 msgr2=0x7fbb6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.186+0000 7fbb70ef2640 1 -- 172.21.15.1:0/712890343 shutdown_connections 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.186+0000 7fbb70ef2640 1 -- 172.21.15.1:0/712890343 wait complete. 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb70ef2640 1 Processor -- start 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb70ef2640 1 -- start start 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb70ef2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb6c100e20 0x7fbb6c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb70ef2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102570 0x7fbb6c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb70ef2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c102ef0 0x7fbb6c1a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb70ef2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c111070 con 0x7fbb6c102570 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb70ef2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c110ef0 con 0x7fbb6c100e20 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb70ef2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1111f0 con 0x7fbb6c102ef0 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb69d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102570 0x7fbb6c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb69d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102570 0x7fbb6c199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38912/0 (socket says 172.21.15.1:38912) 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb69d9b640 1 -- 172.21.15.1:0/3867687011 learned_addr learned my addr 172.21.15.1:0/3867687011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.187+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb6c100e20 0x7fbb6c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.188+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c102ef0 0x7fbb6c1a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.188+0000 7fbb69d9b640 1 -- 172.21.15.1:0/3867687011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c102ef0 msgr2=0x7fbb6c1a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.188+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c102ef0 0x7fbb6c1a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.188+0000 7fbb69d9b640 1 -- 172.21.15.1:0/3867687011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb6c100e20 msgr2=0x7fbb6c199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.188+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb6c100e20 0x7fbb6c199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.188+0000 7fbb69d9b640 1 -- 172.21.15.1:0/3867687011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c1a4ac0 con 0x7fbb6c102570 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.188+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102570 0x7fbb6c199a40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000b4a0 tx=0x7fbb6000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.189+0000 7fbb5b7fe640 1 -- 172.21.15.1:0/3867687011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60015020 con 0x7fbb6c102570 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.189+0000 7fbb70ef2640 1 -- 172.21.15.1:0/3867687011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6c1a4c90 con 0x7fbb6c102570 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.189+0000 7fbb5b7fe640 1 -- 172.21.15.1:0/3867687011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb60011d10 con 0x7fbb6c102570 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.189+0000 7fbb5b7fe640 1 -- 172.21.15.1:0/3867687011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000f2e0 con 0x7fbb6c102570 2024-09-16T00:31:53.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.189+0000 7fbb70ef2640 1 -- 172.21.15.1:0/3867687011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6c1a5180 con 0x7fbb6c102570 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.191+0000 7fbb5b7fe640 1 -- 172.21.15.1:0/3867687011 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fbb60018020 con 0x7fbb6c102570 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.191+0000 7fbb732cb740 1 Processor -- start 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb732cb740 1 -- start start 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd04a0 0x558a05bd0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bdb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bdb680 0x558a05bdda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb732cb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558a05b362e0 con 0x558a05bd0dc0 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb732cb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a05addce0 con 0x558a05bdb680 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb732cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a05b322a0 con 0x558a05bd04a0 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb5b7fe640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb540a3240 0x7fbb540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bdb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bdb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38924/0 (socket says 172.21.15.1:38924) 2024-09-16T00:31:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6a59c640 1 -- 172.21.15.1:0/692068180 learned_addr learned my addr 172.21.15.1:0/692068180 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/692068180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd04a0 0x558a05bd0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb540a3240 0x7fbb540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb5b7fe640 1 -- 172.21.15.1:0/3867687011 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbb600cf5c0 con 0x7fbb6c102570 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/692068180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bdb680 0x558a05bdda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6a59c640 1 -- 172.21.15.1:0/692068180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd04a0 msgr2=0x558a05bd0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/692068180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd04a0 0x558a05bd0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6a59c640 1 -- 172.21.15.1:0/692068180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bdb680 msgr2=0x558a05bdda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/692068180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bdb680 0x558a05bdda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.192+0000 7fbb6a59c640 1 -- 172.21.15.1:0/692068180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a05ac2440 con 0x558a05bd0dc0 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.193+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/692068180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bdb140 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009a70 tx=0x7fbb5c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26024d3f2a323b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.193+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb540a3240 0x7fbb540a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009430 tx=0x7fbb64009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.193+0000 7fbb58ff9640 1 -- 172.21.15.1:0/692068180 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c017070 con 0x558a05bd0dc0 2024-09-16T00:31:53.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.193+0000 7fbb58ff9640 1 -- 172.21.15.1:0/692068180 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb5c013470 con 0x558a05bd0dc0 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.193+0000 7fbb58ff9640 1 -- 172.21.15.1:0/692068180 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c012630 con 0x558a05bd0dc0 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 -- 172.21.15.1:0/692068180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 msgr2=0x558a05bdb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 --2- 172.21.15.1:0/692068180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bdb140 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009a70 tx=0x7fbb5c009f30 comp rx=0 tx=0).stop 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 -- 172.21.15.1:0/692068180 shutdown_connections 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 --2- 172.21.15.1:0/692068180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bdb680 0x558a05bdda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 --2- 172.21.15.1:0/692068180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bdb140 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 --2- 172.21.15.1:0/692068180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd04a0 0x558a05bd0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 -- 172.21.15.1:0/692068180 >> 172.21.15.1:0/692068180 conn(0x558a05bcca30 msgr2=0x558a05bcce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 -- 172.21.15.1:0/692068180 shutdown_connections 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 -- 172.21.15.1:0/692068180 wait complete. 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 -- start start 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.194+0000 7fbb732cb740 1 Processor -- start 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd0dc0 0x558a05bd8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bdb680 0x558a05bd8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb732cb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558a05bde250 con 0x558a05bd04a0 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb732cb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a05af1050 con 0x558a05bd0dc0 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb732cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a05bde3d0 con 0x558a05bdb680 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb732cb740 10 client.0 _ll_register_callbacks cb 0x558a05b31160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38936/0 (socket says 172.21.15.1:38936) 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6a59c640 1 -- 172.21.15.1:0/2951125572 learned_addr learned my addr 172.21.15.1:0/2951125572 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd0dc0 0x558a05bd8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bdb680 0x558a05bd8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6a59c640 1 -- 172.21.15.1:0/2951125572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bdb680 msgr2=0x558a05bd8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bdb680 0x558a05bd8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6a59c640 1 -- 172.21.15.1:0/2951125572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd0dc0 msgr2=0x558a05bd8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd0dc0 0x558a05bd8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6a59c640 1 -- 172.21.15.1:0/2951125572 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a058a73c0 con 0x558a05bd04a0 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bdb680 0x558a05bd8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdaf60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c013f30 tx=0x7fbb5c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6763f53c0b5065 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb51ffb640 1 -- 172.21.15.1:0/2951125572 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c017050 con 0x558a05bd04a0 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb51ffb640 1 -- 172.21.15.1:0/2951125572 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbb5c01c450 con 0x558a05bd04a0 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.195+0000 7fbb732cb740 1 -- 172.21.15.1:0/2951125572 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb5c013470 con 0x558a05bd04a0 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb51ffb640 1 -- 172.21.15.1:0/2951125572 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb5c02f050 con 0x558a05bd04a0 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb732cb740 10 client.16020 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb732cb740 1 -- 172.21.15.1:0/2951125572 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a05be9bc0 con 0x558a05bd04a0 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb51ffb640 1 -- 172.21.15.1:0/2951125572 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb5c013470 con 0x558a05bd04a0 2024-09-16T00:31:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb51ffb640 1 -- 172.21.15.1:0/2951125572 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbb5c026710 con 0x558a05bd04a0 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb732cb740 10 client.16020 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb732cb740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb732cb740 20 client.16020 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a05beb080 0x558a05bed470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb732cb740 4 client.16020 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb732cb740 1 -- 172.21.15.1:0/2951125572 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a05b43850 con 0x558a05beb080 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.196+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a05beb080 0x558a05bed470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.197+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a05beb080 0x558a05bed470 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53a8327440718f96 server_cookie=390a35803654f674 in_seq=0 out_seq=0 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.197+0000 7fbb51ffb640 10 client.16020 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.197+0000 7fbb51ffb640 1 -- 172.21.15.1:0/2951125572 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a05b43850 con 0x558a05beb080 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.197+0000 7fbb51ffb640 10 client.16020 handle_command_reply: tid=0 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.197+0000 7fbb732cb740 1 client.16020 shutdown 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.197+0000 7fbb732cb740 1 -- 172.21.15.1:0/2951125572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 msgr2=0x558a05bdaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.197+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdaf60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c013f30 tx=0x7fbb5c00ef70 comp rx=0 tx=0).stop 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb732cb740 1 -- 172.21.15.1:0/2951125572 shutdown_connections 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a05beb080 0x558a05bed470 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bdb680 0x558a05bd8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd0dc0 0x558a05bd8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2951125572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdaf60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb732cb740 1 -- 172.21.15.1:0/2951125572 >> 172.21.15.1:0/2951125572 conn(0x558a05bcca30 msgr2=0x558a05bd2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb51ffb640 0 client.16020 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb732cb740 1 -- 172.21.15.1:0/2951125572 shutdown_connections 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb732cb740 1 -- 172.21.15.1:0/2951125572 wait complete. 2024-09-16T00:31:53.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.198+0000 7fbb732cb740 20 client.16020 trim_cache size 0 max 16384 2024-09-16T00:31:53.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb732cb740 1 Processor -- start 2024-09-16T00:31:53.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb732cb740 1 -- start start 2024-09-16T00:31:53.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bcca30 0x558a05be74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd04a0 0x558a05bd3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bd71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb732cb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1b17e0 con 0x558a05bd0dc0 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb732cb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1af0b0 con 0x558a05bd04a0 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb732cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1acf00 con 0x558a05bcca30 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb6ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bd71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb6ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bd71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38946/0 (socket says 172.21.15.1:38946) 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb6ad9d640 1 -- 172.21.15.1:0/2915001142 learned_addr learned my addr 172.21.15.1:0/2915001142 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/2915001142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bcca30 0x558a05be74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.215+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/2915001142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd04a0 0x558a05bd3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.216+0000 7fbb6ad9d640 1 -- 172.21.15.1:0/2915001142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bcca30 msgr2=0x558a05be74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.216+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/2915001142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bcca30 0x558a05be74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.216+0000 7fbb6ad9d640 1 -- 172.21.15.1:0/2915001142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd04a0 msgr2=0x558a05bd3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.216+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/2915001142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd04a0 0x558a05bd3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.216+0000 7fbb6ad9d640 1 -- 172.21.15.1:0/2915001142 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a05bcda50 con 0x558a05bd0dc0 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.216+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/2915001142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bd71f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009ca0 tx=0x7fbb64005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4382775549ae6c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.216+0000 7fbb53fff640 1 -- 172.21.15.1:0/2915001142 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64034040 con 0x558a05bd0dc0 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.216+0000 7fbb53fff640 1 -- 172.21.15.1:0/2915001142 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb6402fe80 con 0x558a05bd0dc0 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.216+0000 7fbb53fff640 1 -- 172.21.15.1:0/2915001142 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6403ab30 con 0x558a05bd0dc0 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 -- 172.21.15.1:0/2915001142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 msgr2=0x558a05bd71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2915001142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bd71f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009ca0 tx=0x7fbb64005f30 comp rx=0 tx=0).stop 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 -- 172.21.15.1:0/2915001142 shutdown_connections 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2915001142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd0dc0 0x558a05bd71f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2915001142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bd04a0 0x558a05bd3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 --2- 172.21.15.1:0/2915001142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bcca30 0x558a05be74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 -- 172.21.15.1:0/2915001142 >> 172.21.15.1:0/2915001142 conn(0x558a05bd88b0 msgr2=0x558a05bd6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 -- 172.21.15.1:0/2915001142 shutdown_connections 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 -- 172.21.15.1:0/2915001142 wait complete. 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 -- start start 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.217+0000 7fbb732cb740 1 Processor -- start 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bcca30 0x558a05bded60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb732cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd0dc0 0x558a05c53a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb732cb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1ad570 con 0x558a05bd04a0 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb732cb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1ae9e0 con 0x558a05bcca30 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb69d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb732cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1adc40 con 0x558a05bd0dc0 2024-09-16T00:31:53.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb69d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38948/0 (socket says 172.21.15.1:38948) 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb69d9b640 1 -- 172.21.15.1:0/1263985610 learned_addr learned my addr 172.21.15.1:0/1263985610 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb732cb740 10 client.0 _ll_register_callbacks cb 0x7fbb60097a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bcca30 0x558a05bded60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb6ad9d640 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd0dc0 0x558a05c53a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb69d9b640 1 -- 172.21.15.1:0/1263985610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd0dc0 msgr2=0x558a05c53a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd0dc0 0x558a05c53a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb69d9b640 1 -- 172.21.15.1:0/1263985610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bcca30 msgr2=0x558a05bded60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bcca30 0x558a05bded60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.218+0000 7fbb69d9b640 1 -- 172.21.15.1:0/1263985610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a05c54070 con 0x558a05bd04a0 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb69d9b640 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdf2a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbb6009b480 tx=0x7fbb600047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6736f668f631397 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb51ffb640 1 -- 172.21.15.1:0/1263985610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6009fb30 con 0x558a05bd04a0 2024-09-16T00:31:53.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb51ffb640 1 -- 172.21.15.1:0/1263985610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbb60010510 con 0x558a05bd04a0 2024-09-16T00:31:53.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb732cb740 1 -- 172.21.15.1:0/1263985610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb6402fe80 con 0x558a05bd04a0 2024-09-16T00:31:53.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb51ffb640 1 -- 172.21.15.1:0/1263985610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb600a43f0 con 0x558a05bd04a0 2024-09-16T00:31:53.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb732cb740 10 client.16026 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:53.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb732cb740 1 -- 172.21.15.1:0/1263985610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a05c5db70 con 0x558a05bd04a0 2024-09-16T00:31:53.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb51ffb640 1 -- 172.21.15.1:0/1263985610 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb6402fe80 con 0x558a05bd04a0 2024-09-16T00:31:53.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb51ffb640 1 -- 172.21.15.1:0/1263985610 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbb60097040 con 0x558a05bd04a0 2024-09-16T00:31:53.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb732cb740 10 client.16026 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:53.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb732cb740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:31:53.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb732cb740 20 client.16026 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb732cb740 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a05bdb680 0x558a05c60310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb732cb740 4 client.16026 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.219+0000 7fbb732cb740 1 -- 172.21.15.1:0/1263985610 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a05b3dd60 con 0x558a05bdb680 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.220+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a05bdb680 0x558a05c60310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.220+0000 7fbb6a59c640 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a05bdb680 0x558a05c60310 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6e582143010c3d server_cookie=b0465e8076b5d4a6 in_seq=0 out_seq=0 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.220+0000 7fbb51ffb640 10 client.16026 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.220+0000 7fbb51ffb640 1 -- 172.21.15.1:0/1263985610 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558a05b3dd60 con 0x558a05bdb680 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.220+0000 7fbb51ffb640 10 client.16026 handle_command_reply: tid=0 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.220+0000 7fbb732cb740 1 client.16026 shutdown 2024-09-16T00:31:53.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 -- 172.21.15.1:0/1263985610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 msgr2=0x558a05bdf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdf2a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbb6009b480 tx=0x7fbb600047d0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 -- 172.21.15.1:0/1263985610 shutdown_connections 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a05bdb680 0x558a05c60310 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a05bd0dc0 0x558a05c53a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a05bd04a0 0x558a05bdf2a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 --2- 172.21.15.1:0/1263985610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a05bcca30 0x558a05bded60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 -- 172.21.15.1:0/1263985610 >> 172.21.15.1:0/1263985610 conn(0x558a05bd88b0 msgr2=0x558a05bcb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb51ffb640 0 client.16026 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 -- 172.21.15.1:0/1263985610 shutdown_connections 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 1 -- 172.21.15.1:0/1263985610 wait complete. 2024-09-16T00:31:53.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb732cb740 20 client.16026 trim_cache size 0 max 16384 2024-09-16T00:31:53.230 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:31:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:31:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:31:53.230 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:31:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 -- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb540a3240 msgr2=0x7fbb540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb540a3240 0x7fbb540a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009430 tx=0x7fbb64009800 comp rx=0 tx=0).stop 2024-09-16T00:31:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 -- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102570 msgr2=0x7fbb6c199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102570 0x7fbb6c199a40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000b4a0 tx=0x7fbb6000b8a0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 -- 172.21.15.1:0/3867687011 shutdown_connections 2024-09-16T00:31:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb540a3240 0x7fbb540a5700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb6c102ef0 0x7fbb6c1a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb6c102570 0x7fbb6c199a40 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 --2- 172.21.15.1:0/3867687011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb6c100e20 0x7fbb6c199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.221+0000 7fbb70ef2640 1 -- 172.21.15.1:0/3867687011 >> 172.21.15.1:0/3867687011 conn(0x7fbb6c0fc7d0 msgr2=0x7fbb6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:53.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.222+0000 7fbb70ef2640 1 -- 172.21.15.1:0/3867687011 shutdown_connections 2024-09-16T00:31:53.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:53.222+0000 7fbb70ef2640 1 -- 172.21.15.1:0/3867687011 wait complete. 2024-09-16T00:31:54.322 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:31:54.322 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:31:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:54 smithi001 ceph-mon[30934]: pgmap v455: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-09-16T00:31:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:54 smithi153 ceph-mon[35538]: pgmap v455: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-09-16T00:31:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:54.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:54 smithi019 ceph-mon[35186]: pgmap v455: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-09-16T00:31:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.059+0000 7f18f1f3c640 1 -- 172.21.15.1:0/2480408751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 msgr2=0x7f18ec103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.059+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/2480408751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 0x7f18ec103120 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009900 tx=0x7f18dc02a480 comp rx=0 tx=0).stop 2024-09-16T00:31:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 -- 172.21.15.1:0/2480408751 shutdown_connections 2024-09-16T00:31:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/2480408751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18ec10c170 0x7f18ec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/2480408751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18ec103660 0x7f18ec10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/2480408751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 0x7f18ec103120 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 -- 172.21.15.1:0/2480408751 >> 172.21.15.1:0/2480408751 conn(0x7f18ec0fc7a0 msgr2=0x7f18ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 -- 172.21.15.1:0/2480408751 shutdown_connections 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 -- 172.21.15.1:0/2480408751 wait complete. 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 Processor -- start 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 -- start start 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.060+0000 7f18f1f3c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 0x7f18ec198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18f1f3c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18ec103660 0x7f18ec198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18f1f3c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18ec10c170 0x7f18ec19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18f1f3c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec111f70 con 0x7f18ec102d40 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18f1f3c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec111df0 con 0x7f18ec10c170 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18f1f3c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec1120f0 con 0x7f18ec103660 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18eb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 0x7f18ec198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18eb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 0x7f18ec198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38952/0 (socket says 172.21.15.1:38952) 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18eb7fe640 1 -- 172.21.15.1:0/4120337629 learned_addr learned my addr 172.21.15.1:0/4120337629 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18ebfff640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18ec10c170 0x7f18ec19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18e3fff640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18ec103660 0x7f18ec198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18eb7fe640 1 -- 172.21.15.1:0/4120337629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18ec103660 msgr2=0x7f18ec198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18eb7fe640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18ec103660 0x7f18ec198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18eb7fe640 1 -- 172.21.15.1:0/4120337629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18ec10c170 msgr2=0x7f18ec19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18eb7fe640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18ec10c170 0x7f18ec19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18eb7fe640 1 -- 172.21.15.1:0/4120337629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec19d3e0 con 0x7f18ec102d40 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18e3fff640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18ec103660 0x7f18ec198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18eb7fe640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 0x7f18ec198430 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f18dc006010 tx=0x7f18dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18e97fa640 1 -- 172.21.15.1:0/4120337629 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc03f030 con 0x7f18ec102d40 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18e97fa640 1 -- 172.21.15.1:0/4120337629 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18dc003ce0 con 0x7f18ec102d40 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec19d670 con 0x7f18ec102d40 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18e97fa640 1 -- 172.21.15.1:0/4120337629 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc035070 con 0x7f18ec102d40 2024-09-16T00:31:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.061+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1a4fb0 con 0x7f18ec102d40 2024-09-16T00:31:55.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.062+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec06b070 con 0x7f18ec102d40 2024-09-16T00:31:55.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.064+0000 7f18e97fa640 1 -- 172.21.15.1:0/4120337629 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f18dc042030 con 0x7f18ec102d40 2024-09-16T00:31:55.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.064+0000 7f18e97fa640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f18d80a3240 0x7f18d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:55.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.064+0000 7f18e3fff640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f18d80a3240 0x7f18d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:55.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.064+0000 7f18e97fa640 1 -- 172.21.15.1:0/4120337629 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f18dc0f0670 con 0x7f18ec102d40 2024-09-16T00:31:55.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.065+0000 7f18e3fff640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f18d80a3240 0x7f18d80a5700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f18ec10d5b0 tx=0x7f18d40095d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:55.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.067+0000 7f18e97fa640 1 -- 172.21.15.1:0/4120337629 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18dc0bd050 con 0x7f18ec102d40 2024-09-16T00:31:55.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.279+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18ec103120 con 0x7f18ec102d40 2024-09-16T00:31:55.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.280+0000 7f18e97fa640 1 -- 172.21.15.1:0/4120337629 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f18ec103120 con 0x7f18ec102d40 2024-09-16T00:31:55.281 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:31:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:31:55.281 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:31:55.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f18d80a3240 msgr2=0x7f18d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:55.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f18d80a3240 0x7f18d80a5700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f18ec10d5b0 tx=0x7f18d40095d0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 msgr2=0x7f18ec198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 0x7f18ec198430 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f18dc006010 tx=0x7f18dc0042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 shutdown_connections 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18ec10c170 0x7f18ec19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f18d80a3240 0x7f18d80a5700 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18ec103660 0x7f18ec198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 --2- 172.21.15.1:0/4120337629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18ec102d40 0x7f18ec198430 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 >> 172.21.15.1:0/4120337629 conn(0x7f18ec0fc7a0 msgr2=0x7f18ec0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 shutdown_connections 2024-09-16T00:31:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:55.283+0000 7f18f1f3c640 1 -- 172.21.15.1:0/4120337629 wait complete. 2024-09-16T00:31:56.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:31:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:56 smithi001 ceph-mon[30934]: pgmap v456: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-09-16T00:31:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:56 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4120337629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:56 smithi153 ceph-mon[35538]: pgmap v456: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-09-16T00:31:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:56 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4120337629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:56.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:56 smithi019 ceph-mon[35186]: pgmap v456: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-09-16T00:31:56.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:56 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4120337629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.912+0000 7f69cbbdd640 1 -- 172.21.15.1:0/1846725925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 msgr2=0x7f69c4074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/1846725925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 0x7f69c4074970 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009900 tx=0x7f69c002a480 comp rx=0 tx=0).stop 2024-09-16T00:31:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 -- 172.21.15.1:0/1846725925 shutdown_connections 2024-09-16T00:31:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/1846725925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69c4112b90 0x7f69c4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/1846725925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69c4074eb0 0x7f69c4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/1846725925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 0x7f69c4074970 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 -- 172.21.15.1:0/1846725925 >> 172.21.15.1:0/1846725925 conn(0x7f69c40fd8c0 msgr2=0x7f69c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 -- 172.21.15.1:0/1846725925 shutdown_connections 2024-09-16T00:31:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 -- 172.21.15.1:0/1846725925 wait complete. 2024-09-16T00:31:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 Processor -- start 2024-09-16T00:31:56.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.913+0000 7f69cbbdd640 1 -- start start 2024-09-16T00:31:56.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69cbbdd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 0x7f69c41a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69cbbdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69c4074eb0 0x7f69c41a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69cbbdd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69c4112b90 0x7f69c419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69cbbdd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4118550 con 0x7f69c4074590 2024-09-16T00:31:56.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69cbbdd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41183d0 con 0x7f69c4074eb0 2024-09-16T00:31:56.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69cbbdd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41186d0 con 0x7f69c4112b90 2024-09-16T00:31:56.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9979640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 0x7f69c41a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9979640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 0x7f69c41a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38964/0 (socket says 172.21.15.1:38964) 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9979640 1 -- 172.21.15.1:0/68534449 learned_addr learned my addr 172.21.15.1:0/68534449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69ca17a640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69c4112b90 0x7f69c419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9178640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69c4074eb0 0x7f69c41a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9979640 1 -- 172.21.15.1:0/68534449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69c4112b90 msgr2=0x7f69c419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9979640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69c4112b90 0x7f69c419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9979640 1 -- 172.21.15.1:0/68534449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69c4074eb0 msgr2=0x7f69c41a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9979640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69c4074eb0 0x7f69c41a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9979640 1 -- 172.21.15.1:0/68534449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c419f010 con 0x7f69c4074590 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69c9979640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 0x7f69c41a4720 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f69c0006010 tx=0x7f69c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69baffd640 1 -- 172.21.15.1:0/68534449 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c003f030 con 0x7f69c4074590 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69baffd640 1 -- 172.21.15.1:0/68534449 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c0003ce0 con 0x7f69c4074590 2024-09-16T00:31:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69cbbdd640 1 -- 172.21.15.1:0/68534449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c419f2a0 con 0x7f69c4074590 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69baffd640 1 -- 172.21.15.1:0/68534449 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0035070 con 0x7f69c4074590 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.914+0000 7f69cbbdd640 1 -- 172.21.15.1:0/68534449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c41ab5f0 con 0x7f69c4074590 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.917+0000 7f69baffd640 1 -- 172.21.15.1:0/68534449 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f69c0042030 con 0x7f69c4074590 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.917+0000 7f69cdfb6740 1 Processor -- start 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.917+0000 7f69cdfb6740 1 -- start start 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.917+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbad0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.917+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 0x5620dbad0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.917+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbadd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.917+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5620dba362f0 con 0x5620dbadb370 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.917+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5620db9ddcf0 con 0x5620dbad1b40 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.917+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5620dba322b0 con 0x5620dbad05d0 2024-09-16T00:31:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9178640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbadd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9178640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbadd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38976/0 (socket says 172.21.15.1:38976) 2024-09-16T00:31:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9178640 1 -- 172.21.15.1:0/3895920832 learned_addr learned my addr 172.21.15.1:0/3895920832 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69ca17a640 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbad0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9979640 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 0x5620dbad0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69baffd640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f69b00a32c0 0x7f69b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9178640 1 -- 172.21.15.1:0/3895920832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 msgr2=0x5620dbad0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9178640 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 0x5620dbad0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9178640 1 -- 172.21.15.1:0/3895920832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 msgr2=0x5620dbad0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9178640 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbad0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9178640 1 -- 172.21.15.1:0/3895920832 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620db9c23a0 con 0x5620dbadb370 2024-09-16T00:31:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9979640 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 0x5620dbad0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69ca17a640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f69b00a32c0 0x7f69b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69baffd640 1 -- 172.21.15.1:0/68534449 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f69c00efee0 con 0x7f69c4074590 2024-09-16T00:31:56.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69c9178640 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbadd760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f69b400dc50 tx=0x7f69b400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3185793a1d9e6edc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69affff640 1 -- 172.21.15.1:0/3895920832 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4015070 con 0x5620dbadb370 2024-09-16T00:31:56.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69affff640 1 -- 172.21.15.1:0/3895920832 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b40035f0 con 0x5620dbadb370 2024-09-16T00:31:56.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.918+0000 7f69affff640 1 -- 172.21.15.1:0/3895920832 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40116a0 con 0x5620dbadb370 2024-09-16T00:31:56.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3895920832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 msgr2=0x5620dbadd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69ca17a640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f69b00a32c0 0x7f69b00a5780 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f69c4076600 tx=0x7f69bc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbadd760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f69b400dc50 tx=0x7f69b400f730 comp rx=0 tx=0).stop 2024-09-16T00:31:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3895920832 shutdown_connections 2024-09-16T00:31:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbadd760 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 0x5620dbad0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3895920832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbad0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3895920832 >> 172.21.15.1:0/3895920832 conn(0x5620dba99310 msgr2=0x5620dbaccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3895920832 shutdown_connections 2024-09-16T00:31:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.919+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3895920832 wait complete. 2024-09-16T00:31:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69cdfb6740 1 -- start start 2024-09-16T00:31:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69cdfb6740 1 Processor -- start 2024-09-16T00:31:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad05d0 0x5620dbadad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad1b40 0x5620dbad8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5620dbade070 con 0x5620dbadb370 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5620db9f1060 con 0x5620dbad05d0 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5620dbade1f0 con 0x5620dbad1b40 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69cdfb6740 10 client.0 _ll_register_callbacks cb 0x5620db7448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69ca17a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69ca17a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38986/0 (socket says 172.21.15.1:38986) 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69ca17a640 1 -- 172.21.15.1:0/2669077269 learned_addr learned my addr 172.21.15.1:0/2669077269 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69c9178640 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad1b40 0x5620dbad8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.920+0000 7f69c9979640 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad05d0 0x5620dbadad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ca17a640 1 -- 172.21.15.1:0/2669077269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad1b40 msgr2=0x5620dbad8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ca17a640 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad1b40 0x5620dbad8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ca17a640 1 -- 172.21.15.1:0/2669077269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad05d0 msgr2=0x5620dbadad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ca17a640 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad05d0 0x5620dbadad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ca17a640 1 -- 172.21.15.1:0/2669077269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620db7a73c0 con 0x5620dbadb370 2024-09-16T00:31:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69c9979640 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad05d0 0x5620dbadad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69c9178640 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad1b40 0x5620dbad8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ca17a640 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad85d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f69bc012a30 tx=0x7f69bc009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6897563cace19684 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:56.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ad7fa640 1 -- 172.21.15.1:0/2669077269 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc02b060 con 0x5620dbadb370 2024-09-16T00:31:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ad7fa640 1 -- 172.21.15.1:0/2669077269 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f69bc018070 con 0x5620dbadb370 2024-09-16T00:31:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69cdfb6740 1 -- 172.21.15.1:0/2669077269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b40035f0 con 0x5620dbadb370 2024-09-16T00:31:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ad7fa640 1 -- 172.21.15.1:0/2669077269 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69bc032050 con 0x5620dbadb370 2024-09-16T00:31:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69cdfb6740 10 client.16044 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69cdfb6740 1 -- 172.21.15.1:0/2669077269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620dbae9b50 con 0x5620dbadb370 2024-09-16T00:31:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.921+0000 7f69ad7fa640 1 -- 172.21.15.1:0/2669077269 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b40035f0 con 0x5620dbadb370 2024-09-16T00:31:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69ad7fa640 1 -- 172.21.15.1:0/2669077269 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f69bc00f040 con 0x5620dbadb370 2024-09-16T00:31:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69cdfb6740 10 client.16044 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:56.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69cdfb6740 10 client.16044 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:31:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69cdfb6740 20 client.16044 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5620dbaeb010 0x5620dbaed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69cdfb6740 4 client.16044 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69cdfb6740 1 -- 172.21.15.1:0/2669077269 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620dba43860 con 0x5620dbaeb010 2024-09-16T00:31:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69c9979640 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5620dbaeb010 0x5620dbaed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69ad7fa640 10 client.16044 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:31:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.922+0000 7f69c9979640 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5620dbaeb010 0x5620dbaed460 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=385af94cd7df345d server_cookie=d76e51fcb9db979a in_seq=0 out_seq=0 2024-09-16T00:31:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69ad7fa640 1 -- 172.21.15.1:0/2669077269 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620dba43860 con 0x5620dbaeb010 2024-09-16T00:31:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69ad7fa640 10 client.16044 handle_command_reply: tid=0 2024-09-16T00:31:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 client.16044 shutdown 2024-09-16T00:31:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 -- 172.21.15.1:0/2669077269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 msgr2=0x5620dbad85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad85d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f69bc012a30 tx=0x7f69bc009600 comp rx=0 tx=0).stop 2024-09-16T00:31:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 -- 172.21.15.1:0/2669077269 shutdown_connections 2024-09-16T00:31:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad85d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad1b40 0x5620dbad8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5620dbaeb010 0x5620dbaed460 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/2669077269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad05d0 0x5620dbadad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 -- 172.21.15.1:0/2669077269 >> 172.21.15.1:0/2669077269 conn(0x5620dba99310 msgr2=0x5620dbad2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69ad7fa640 0 client.16044 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:31:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 -- 172.21.15.1:0/2669077269 shutdown_connections 2024-09-16T00:31:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 1 -- 172.21.15.1:0/2669077269 wait complete. 2024-09-16T00:31:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.923+0000 7f69cdfb6740 20 client.16044 trim_cache size 0 max 16384 2024-09-16T00:31:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.936+0000 7f69cdfb6740 1 Processor -- start 2024-09-16T00:31:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.936+0000 7f69cdfb6740 1 -- start start 2024-09-16T00:31:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 0x5620dbacce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbae7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b7db0 con 0x5620dbadb370 2024-09-16T00:31:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b5680 con 0x5620dbad1b40 2024-09-16T00:31:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b34d0 con 0x5620dbad05d0 2024-09-16T00:31:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69ca17a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69ca17a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38990/0 (socket says 172.21.15.1:38990) 2024-09-16T00:31:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69ca17a640 1 -- 172.21.15.1:0/1538670360 learned_addr learned my addr 172.21.15.1:0/1538670360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69c9178640 1 --2- 172.21.15.1:0/1538670360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbae7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69c9979640 1 --2- 172.21.15.1:0/1538670360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 0x5620dbacce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69ca17a640 1 -- 172.21.15.1:0/1538670360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 msgr2=0x5620dbacce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69ca17a640 1 --2- 172.21.15.1:0/1538670360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 0x5620dbacce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69ca17a640 1 -- 172.21.15.1:0/1538670360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 msgr2=0x5620dbae7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69ca17a640 1 --2- 172.21.15.1:0/1538670360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbae7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69ca17a640 1 -- 172.21.15.1:0/1538670360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620dbad6fb0 con 0x5620dbadb370 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69ca17a640 1 --2- 172.21.15.1:0/1538670360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad78c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f69bc012f70 tx=0x7f69bc009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160dbbf26725c3df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69af7fe640 1 -- 172.21.15.1:0/1538670360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc004ab0 con 0x5620dbadb370 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69af7fe640 1 -- 172.21.15.1:0/1538670360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc01c960 con 0x5620dbadb370 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.937+0000 7f69af7fe640 1 -- 172.21.15.1:0/1538670360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc036430 con 0x5620dbadb370 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.938+0000 7f69cdfb6740 1 -- 172.21.15.1:0/1538670360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 msgr2=0x5620dbad78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.938+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/1538670360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad78c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f69bc012f70 tx=0x7f69bc009f40 comp rx=0 tx=0).stop 2024-09-16T00:31:56.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.938+0000 7f69cdfb6740 1 -- 172.21.15.1:0/1538670360 shutdown_connections 2024-09-16T00:31:56.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.938+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/1538670360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbadb370 0x5620dbad78c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.938+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/1538670360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbae7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.938+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/1538670360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbad05d0 0x5620dbacce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.938+0000 7f69cdfb6740 1 -- 172.21.15.1:0/1538670360 >> 172.21.15.1:0/1538670360 conn(0x5620dba99310 msgr2=0x5620dbacc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.938+0000 7f69cdfb6740 1 -- 172.21.15.1:0/1538670360 shutdown_connections 2024-09-16T00:31:56.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.938+0000 7f69cdfb6740 1 -- 172.21.15.1:0/1538670360 wait complete. 2024-09-16T00:31:56.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69cdfb6740 1 -- start start 2024-09-16T00:31:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69cdfb6740 1 Processor -- start 2024-09-16T00:31:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbad05d0 0x5620dbadf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbadf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69cdfb6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbadb370 0x5620dbadfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b3b40 con 0x5620dbad05d0 2024-09-16T00:31:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b4fb0 con 0x5620dbad1b40 2024-09-16T00:31:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69cdfb6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b4210 con 0x5620dbadb370 2024-09-16T00:31:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69cdfb6740 10 client.0 _ll_register_callbacks cb 0x7f69bc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69c9979640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbad05d0 0x5620dbadf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69c9979640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbad05d0 0x5620dbadf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38992/0 (socket says 172.21.15.1:38992) 2024-09-16T00:31:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69c9979640 1 -- 172.21.15.1:0/3358008466 learned_addr learned my addr 172.21.15.1:0/3358008466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69c9178640 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbadf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.939+0000 7f69ca17a640 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbadb370 0x5620dbadfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69c9979640 1 -- 172.21.15.1:0/3358008466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbadb370 msgr2=0x5620dbadfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69c9979640 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbadb370 0x5620dbadfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69c9979640 1 -- 172.21.15.1:0/3358008466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 msgr2=0x5620dbadf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:56.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69c9979640 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbadf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69c9979640 1 -- 172.21.15.1:0/3358008466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620dbb54dd0 con 0x5620dbad05d0 2024-09-16T00:31:56.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69c9979640 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbad05d0 0x5620dbadf2a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f69c00ed580 tx=0x7f69c00bbb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe6feaaa51a3e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:56.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69ad7fa640 1 -- 172.21.15.1:0/3358008466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0049070 con 0x5620dbad05d0 2024-09-16T00:31:56.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69ad7fa640 1 -- 172.21.15.1:0/3358008466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f69c00ed690 con 0x5620dbad05d0 2024-09-16T00:31:56.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3358008466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc01c960 con 0x5620dbad05d0 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69ad7fa640 1 -- 172.21.15.1:0/3358008466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69c00bc860 con 0x5620dbad05d0 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69cdfb6740 10 client.16050 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3358008466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620dbb56110 con 0x5620dbad05d0 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69ad7fa640 1 -- 172.21.15.1:0/3358008466 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc01c960 con 0x5620dbad05d0 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69ad7fa640 1 -- 172.21.15.1:0/3358008466 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f69c00dace0 con 0x5620dbad05d0 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.940+0000 7f69cdfb6740 10 client.16050 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69cdfb6740 10 client.16050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69cdfb6740 20 client.16050 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5620dbb567e0 0x5620dbb58bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69cdfb6740 4 client.16050 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:31:56.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3358008466 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620dba3dd70 con 0x5620dbb567e0 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69c9178640 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5620dbb567e0 0x5620dbb58bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69ad7fa640 10 client.16050 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69c9178640 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5620dbb567e0 0x5620dbb58bd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=304ad96942e489d5 server_cookie=7dd2bbd3b5099097 in_seq=0 out_seq=0 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69ad7fa640 1 -- 172.21.15.1:0/3358008466 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620dba3dd70 con 0x5620dbb567e0 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69ad7fa640 10 client.16050 handle_command_reply: tid=0 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.941+0000 7f69cdfb6740 1 client.16050 shutdown 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3358008466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbad05d0 msgr2=0x5620dbadf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbad05d0 0x5620dbadf2a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f69c00ed580 tx=0x7f69c00bbb80 comp rx=0 tx=0).stop 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3358008466 shutdown_connections 2024-09-16T00:31:56.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620dbadb370 0x5620dbadfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5620dbb567e0 0x5620dbb58bd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620dbad1b40 0x5620dbadf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 --2- 172.21.15.1:0/3358008466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5620dbad05d0 0x5620dbadf2a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69ad7fa640 0 client.16050 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:31:56.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3358008466 >> 172.21.15.1:0/3358008466 conn(0x5620dba99310 msgr2=0x5620dbada080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3358008466 shutdown_connections 2024-09-16T00:31:56.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 1 -- 172.21.15.1:0/3358008466 wait complete. 2024-09-16T00:31:56.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cdfb6740 20 client.16050 trim_cache size 0 max 16384 2024-09-16T00:31:56.969 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:31:56.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 -- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f69b00a32c0 msgr2=0x7f69b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f69b00a32c0 0x7f69b00a5780 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f69c4076600 tx=0x7f69bc004a40 comp rx=0 tx=0).stop 2024-09-16T00:31:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 -- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 msgr2=0x7f69c41a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 0x7f69c41a4720 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f69c0006010 tx=0x7f69c00042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 -- 172.21.15.1:0/68534449 shutdown_connections 2024-09-16T00:31:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f69b00a32c0 0x7f69b00a5780 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69c4112b90 0x7f69c419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69c4074eb0 0x7f69c41a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 --2- 172.21.15.1:0/68534449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69c4074590 0x7f69c41a4720 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.942+0000 7f69cbbdd640 1 -- 172.21.15.1:0/68534449 >> 172.21.15.1:0/68534449 conn(0x7f69c40fd8c0 msgr2=0x7f69c40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.943+0000 7f69cbbdd640 1 -- 172.21.15.1:0/68534449 shutdown_connections 2024-09-16T00:31:56.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:56.943+0000 7f69cbbdd640 1 -- 172.21.15.1:0/68534449 wait complete. 2024-09-16T00:31:57.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:31:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:58.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:31:58 smithi001 ceph-mon[30934]: pgmap v457: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-16T00:31:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:31:58 smithi153 ceph-mon[35538]: pgmap v457: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-16T00:31:58.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.581+0000 7fc84ce73640 1 -- 172.21.15.1:0/483627449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848101420 msgr2=0x7fc84810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.581+0000 7fc84ce73640 1 --2- 172.21.15.1:0/483627449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848101420 0x7fc84810d950 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc840009cd0 tx=0x7fc84002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.581+0000 7fc84ce73640 1 -- 172.21.15.1:0/483627449 shutdown_connections 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.581+0000 7fc84ce73640 1 --2- 172.21.15.1:0/483627449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848101420 0x7fc84810d950 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.581+0000 7fc84ce73640 1 --2- 172.21.15.1:0/483627449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc848100a80 0x7fc848100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.581+0000 7fc84ce73640 1 --2- 172.21.15.1:0/483627449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc848106a80 0x7fc848106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.581+0000 7fc84ce73640 1 -- 172.21.15.1:0/483627449 >> 172.21.15.1:0/483627449 conn(0x7fc8480fc7a0 msgr2=0x7fc8480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.581+0000 7fc84ce73640 1 -- 172.21.15.1:0/483627449 shutdown_connections 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.581+0000 7fc84ce73640 1 -- 172.21.15.1:0/483627449 wait complete. 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84ce73640 1 Processor -- start 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84ce73640 1 -- start start 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84ce73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848100a80 0x7fc848198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84ce73640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc848101420 0x7fc848198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84ce73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc848106a80 0x7fc84819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84ce73640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc848110e80 con 0x7fc848100a80 2024-09-16T00:31:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84ce73640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc848110d00 con 0x7fc848106a80 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84ce73640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111000 con 0x7fc848101420 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848100a80 0x7fc848198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848100a80 0x7fc848198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39012/0 (socket says 172.21.15.1:39012) 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84659c640 1 -- 172.21.15.1:0/1363451821 learned_addr learned my addr 172.21.15.1:0/1363451821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc845d9b640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc848101420 0x7fc848198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc846d9d640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc848106a80 0x7fc84819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84659c640 1 -- 172.21.15.1:0/1363451821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc848101420 msgr2=0x7fc848198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84659c640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc848101420 0x7fc848198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84659c640 1 -- 172.21.15.1:0/1363451821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc848106a80 msgr2=0x7fc84819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84659c640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc848106a80 0x7fc84819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc84659c640 1 -- 172.21.15.1:0/1363451821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84819d3c0 con 0x7fc848100a80 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.582+0000 7fc846d9d640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc848106a80 0x7fc84819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.583+0000 7fc84659c640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848100a80 0x7fc848198410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d950 tx=0x7fc83800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.583+0000 7fc8377fe640 1 -- 172.21.15.1:0/1363451821 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838017070 con 0x7fc848100a80 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.583+0000 7fc8377fe640 1 -- 172.21.15.1:0/1363451821 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc838009c30 con 0x7fc848100a80 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.583+0000 7fc84ce73640 1 -- 172.21.15.1:0/1363451821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84819d6b0 con 0x7fc848100a80 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.583+0000 7fc8377fe640 1 -- 172.21.15.1:0/1363451821 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83800f6b0 con 0x7fc848100a80 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.583+0000 7fc84ce73640 1 -- 172.21.15.1:0/1363451821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a4f90 con 0x7fc848100a80 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.585+0000 7fc8377fe640 1 -- 172.21.15.1:0/1363451821 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc8380109d0 con 0x7fc848100a80 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.585+0000 7fc84f24c740 1 Processor -- start 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc84f24c740 1 -- start start 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d51b40 0x564ee0d50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 0x564ee0d50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d5b370 0x564ee0d5d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc84f24c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ee0cb62f0 con 0x564ee0d51b40 2024-09-16T00:31:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc84f24c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ee0c5dcf0 con 0x564ee0d505d0 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc84f24c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ee0cb22b0 con 0x564ee0d5b370 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc846d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d51b40 0x564ee0d50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc846d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d51b40 0x564ee0d50090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39026/0 (socket says 172.21.15.1:39026) 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc846d9d640 1 -- 172.21.15.1:0/4213663258 learned_addr learned my addr 172.21.15.1:0/4213663258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc84659c640 1 --2- 172.21.15.1:0/4213663258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 0x564ee0d50a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc845d9b640 1 --2- 172.21.15.1:0/4213663258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d5b370 0x564ee0d5d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.586+0000 7fc846d9d640 1 -- 172.21.15.1:0/4213663258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d5b370 msgr2=0x564ee0d5d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc846d9d640 1 --2- 172.21.15.1:0/4213663258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d5b370 0x564ee0d5d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc846d9d640 1 -- 172.21.15.1:0/4213663258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 msgr2=0x564ee0d50a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc846d9d640 1 --2- 172.21.15.1:0/4213663258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 0x564ee0d50a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc846d9d640 1 -- 172.21.15.1:0/4213663258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee0c423a0 con 0x564ee0d51b40 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc8377fe640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc8300a3240 0x7fc8300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc84659c640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc8300a3240 0x7fc8300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc846d9d640 1 --2- 172.21.15.1:0/4213663258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d51b40 0x564ee0d50090 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc840005e30 tx=0x7fc840009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa83a75dc8bfc4b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc8377fe640 1 -- 172.21.15.1:0/1363451821 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc8380cf960 con 0x7fc848100a80 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc834ff9640 1 -- 172.21.15.1:0/4213663258 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400318c0 con 0x564ee0d51b40 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc834ff9640 1 -- 172.21.15.1:0/4213663258 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc840036040 con 0x564ee0d51b40 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.587+0000 7fc834ff9640 1 -- 172.21.15.1:0/4213663258 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840004640 con 0x564ee0d51b40 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 -- 172.21.15.1:0/4213663258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d51b40 msgr2=0x564ee0d50090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 --2- 172.21.15.1:0/4213663258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d51b40 0x564ee0d50090 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc840005e30 tx=0x7fc840009a20 comp rx=0 tx=0).stop 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84659c640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc8300a3240 0x7fc8300a5700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8380cd3d0 tx=0x7fc8380d54a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 -- 172.21.15.1:0/4213663258 shutdown_connections 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 --2- 172.21.15.1:0/4213663258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d5b370 0x564ee0d5d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 --2- 172.21.15.1:0/4213663258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 0x564ee0d50a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 --2- 172.21.15.1:0/4213663258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d51b40 0x564ee0d50090 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 -- 172.21.15.1:0/4213663258 >> 172.21.15.1:0/4213663258 conn(0x564ee0d19310 msgr2=0x564ee0d4cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 -- 172.21.15.1:0/4213663258 shutdown_connections 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 -- 172.21.15.1:0/4213663258 wait complete. 2024-09-16T00:31:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 -- start start 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.588+0000 7fc84f24c740 1 Processor -- start 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 0x564ee0d5ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 0x564ee0d57ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0d585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc84f24c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ee0d5e010 con 0x564ee0d5b370 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc84f24c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ee0c71060 con 0x564ee0d505d0 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc84f24c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ee0d5e190 con 0x564ee0d51b40 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc84f24c740 10 client.0 _ll_register_callbacks cb 0x564ee09c48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0d585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0d585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39028/0 (socket says 172.21.15.1:39028) 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 -- 172.21.15.1:0/3040079761 learned_addr learned my addr 172.21.15.1:0/3040079761 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:58.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc845d9b640 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 0x564ee0d57ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 0x564ee0d5ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 -- 172.21.15.1:0/3040079761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 msgr2=0x564ee0d57ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 0x564ee0d57ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 -- 172.21.15.1:0/3040079761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 msgr2=0x564ee0d5ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 0x564ee0d5ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 -- 172.21.15.1:0/3040079761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee0a273c0 con 0x564ee0d5b370 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc845d9b640 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 0x564ee0d57ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc846d9d640 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0d585e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc840004380 tx=0x7fc840004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b517b85f7c36f10d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.589+0000 7fc82dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc82dffb640 1 -- 172.21.15.1:0/3040079761 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840004860 con 0x564ee0d5b370 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc82dffb640 1 -- 172.21.15.1:0/3040079761 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc840003b70 con 0x564ee0d5b370 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc84f24c740 1 -- 172.21.15.1:0/3040079761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc840036040 con 0x564ee0d5b370 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc82dffb640 1 -- 172.21.15.1:0/3040079761 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc840003d70 con 0x564ee0d5b370 2024-09-16T00:31:58.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc84f24c740 10 client.16062 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc84f24c740 1 -- 172.21.15.1:0/3040079761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ee0d69a80 con 0x564ee0d5b370 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc82dffb640 1 -- 172.21.15.1:0/3040079761 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc840036040 con 0x564ee0d5b370 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc82dffb640 1 -- 172.21.15.1:0/3040079761 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc840030070 con 0x564ee0d5b370 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc84f24c740 10 client.16062 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc84f24c740 10 client.16062 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc84f24c740 20 client.16062 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc84f24c740 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x564ee0d6add0 0x564ee0d6d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc84f24c740 4 client.16062 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.590+0000 7fc84f24c740 1 -- 172.21.15.1:0/3040079761 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ee0cc3860 con 0x564ee0d6add0 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.591+0000 7fc845d9b640 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x564ee0d6add0 0x564ee0d6d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.591+0000 7fc845d9b640 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x564ee0d6add0 0x564ee0d6d1c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7971b037c80b5d01 server_cookie=36d46b000c28a98a in_seq=0 out_seq=0 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.591+0000 7fc82dffb640 10 client.16062 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc82dffb640 1 -- 172.21.15.1:0/3040079761 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ee0cc3860 con 0x564ee0d6add0 2024-09-16T00:31:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc82dffb640 10 client.16062 handle_command_reply: tid=0 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 client.16062 shutdown 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 -- 172.21.15.1:0/3040079761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 msgr2=0x564ee0d585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0d585e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc840004380 tx=0x7fc840004280 comp rx=0 tx=0).stop 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 -- 172.21.15.1:0/3040079761 shutdown_connections 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x564ee0d6add0 0x564ee0d6d1c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0d585e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 0x564ee0d57ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 --2- 172.21.15.1:0/3040079761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d505d0 0x564ee0d5ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 -- 172.21.15.1:0/3040079761 >> 172.21.15.1:0/3040079761 conn(0x564ee0d19310 msgr2=0x564ee0d52930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc82dffb640 0 client.16062 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 -- 172.21.15.1:0/3040079761 shutdown_connections 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 1 -- 172.21.15.1:0/3040079761 wait complete. 2024-09-16T00:31:58.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.592+0000 7fc84f24c740 20 client.16062 trim_cache size 0 max 16384 2024-09-16T00:31:58.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:31:58 smithi019 ceph-mon[35186]: pgmap v457: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.605+0000 7fc84f24c740 1 Processor -- start 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.605+0000 7fc84f24c740 1 -- start start 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d505d0 0x564ee0d56fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 0x564ee0d6fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d5b370 0x564ee0d70340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc84f24c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b16a0 con 0x564ee0d505d0 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc84f24c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481aef70 con 0x564ee0d5b370 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc84f24c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481ace10 con 0x564ee0d51b40 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc846d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d505d0 0x564ee0d56fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc846d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d505d0 0x564ee0d56fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39042/0 (socket says 172.21.15.1:39042) 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc846d9d640 1 -- 172.21.15.1:0/720274628 learned_addr learned my addr 172.21.15.1:0/720274628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc846d9d640 1 -- 172.21.15.1:0/720274628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 msgr2=0x564ee0d6fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc846d9d640 1 --2- 172.21.15.1:0/720274628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 0x564ee0d6fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc846d9d640 1 -- 172.21.15.1:0/720274628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d5b370 msgr2=0x564ee0d70340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc846d9d640 1 --2- 172.21.15.1:0/720274628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d5b370 0x564ee0d70340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc846d9d640 1 -- 172.21.15.1:0/720274628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee0d4c4f0 con 0x564ee0d505d0 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc846d9d640 1 --2- 172.21.15.1:0/720274628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d505d0 0x564ee0d56fd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc840002230 tx=0x7fc84002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988abb5b15fce65 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc82ffff640 1 -- 172.21.15.1:0/720274628 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840064070 con 0x564ee0d505d0 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc82ffff640 1 -- 172.21.15.1:0/720274628 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc840036040 con 0x564ee0d505d0 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.606+0000 7fc82ffff640 1 -- 172.21.15.1:0/720274628 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840003f10 con 0x564ee0d505d0 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 -- 172.21.15.1:0/720274628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d505d0 msgr2=0x564ee0d56fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 --2- 172.21.15.1:0/720274628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d505d0 0x564ee0d56fd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc840002230 tx=0x7fc84002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 -- 172.21.15.1:0/720274628 shutdown_connections 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 --2- 172.21.15.1:0/720274628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d5b370 0x564ee0d70340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 --2- 172.21.15.1:0/720274628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d51b40 0x564ee0d6fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 --2- 172.21.15.1:0/720274628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d505d0 0x564ee0d56fd0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 -- 172.21.15.1:0/720274628 >> 172.21.15.1:0/720274628 conn(0x564ee0d19310 msgr2=0x564ee0d568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 -- 172.21.15.1:0/720274628 shutdown_connections 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 -- 172.21.15.1:0/720274628 wait complete. 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 -- start start 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.607+0000 7fc84f24c740 1 Processor -- start 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d505d0 0x564ee0d6d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d51b40 0x564ee0d6dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc84f24c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0dcc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc84f24c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481ad430 con 0x564ee0d5b370 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc84f24c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481ae8a0 con 0x564ee0d51b40 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc84f24c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481adb00 con 0x564ee0d505d0 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc84f24c740 10 client.0 _ll_register_callbacks cb 0x7fc84003f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0dcc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0dcc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39058/0 (socket says 172.21.15.1:39058) 2024-09-16T00:31:58.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 -- 172.21.15.1:0/197636651 learned_addr learned my addr 172.21.15.1:0/197636651 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d505d0 0x564ee0d6d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc845d9b640 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d51b40 0x564ee0d6dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 -- 172.21.15.1:0/197636651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d505d0 msgr2=0x564ee0d6d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d505d0 0x564ee0d6d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 -- 172.21.15.1:0/197636651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d51b40 msgr2=0x564ee0d6dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d51b40 0x564ee0d6dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 -- 172.21.15.1:0/197636651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee0dccd60 con 0x564ee0d5b370 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc846d9d640 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0dcc6c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc8400317f0 tx=0x7fc840031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e874af9ce7a5db4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc82dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.608+0000 7fc82dffb640 1 -- 172.21.15.1:0/197636651 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840059020 con 0x564ee0d5b370 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc82dffb640 1 -- 172.21.15.1:0/197636651 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc840003970 con 0x564ee0d5b370 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc84f24c740 1 -- 172.21.15.1:0/197636651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc840036040 con 0x564ee0d5b370 2024-09-16T00:31:58.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc82dffb640 1 -- 172.21.15.1:0/197636651 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc840064040 con 0x564ee0d5b370 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc84f24c740 10 client.16068 fetch_fsmap learned FSMap version 25 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc84f24c740 1 -- 172.21.15.1:0/197636651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ee0dd52f0 con 0x564ee0d5b370 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc82dffb640 1 -- 172.21.15.1:0/197636651 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc840036040 con 0x564ee0d5b370 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc82dffb640 1 -- 172.21.15.1:0/197636651 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc84005e030 con 0x564ee0d5b370 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc84f24c740 10 client.16068 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc84f24c740 10 client.16068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc84f24c740 20 client.16068 populate_metadata read hostname 'smithi001' 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc84f24c740 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x564ee0dd57c0 0x564ee0dd7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc84f24c740 4 client.16068 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc84f24c740 1 -- 172.21.15.1:0/197636651 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ee0cbdd70 con 0x564ee0dd57c0 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.609+0000 7fc845d9b640 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x564ee0dd57c0 0x564ee0dd7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc845d9b640 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x564ee0dd57c0 0x564ee0dd7bb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bc519ce63308c84 server_cookie=39185dd3f90f5925 in_seq=0 out_seq=0 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc82dffb640 10 client.16068 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc82dffb640 1 -- 172.21.15.1:0/197636651 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ee0cbdd70 con 0x564ee0dd57c0 2024-09-16T00:31:58.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc82dffb640 10 client.16068 handle_command_reply: tid=0 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 client.16068 shutdown 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 -- 172.21.15.1:0/197636651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 msgr2=0x564ee0dcc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0dcc6c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc8400317f0 tx=0x7fc840031e80 comp rx=0 tx=0).stop 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 -- 172.21.15.1:0/197636651 shutdown_connections 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x564ee0dd57c0 0x564ee0dd7bb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee0d5b370 0x564ee0dcc6c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee0d51b40 0x564ee0d6dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 --2- 172.21.15.1:0/197636651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ee0d505d0 0x564ee0d6d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 -- 172.21.15.1:0/197636651 >> 172.21.15.1:0/197636651 conn(0x564ee0d19310 msgr2=0x564ee0d570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc82dffb640 0 client.16068 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 -- 172.21.15.1:0/197636651 shutdown_connections 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 1 -- 172.21.15.1:0/197636651 wait complete. 2024-09-16T00:31:58.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.610+0000 7fc84f24c740 20 client.16068 trim_cache size 0 max 16384 2024-09-16T00:31:58.619 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:31:58.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 -- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc8300a3240 msgr2=0x7fc8300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc8300a3240 0x7fc8300a5700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8380cd3d0 tx=0x7fc8380d54a0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 -- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848100a80 msgr2=0x7fc848198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848100a80 0x7fc848198410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d950 tx=0x7fc83800de10 comp rx=0 tx=0).stop 2024-09-16T00:31:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 -- 172.21.15.1:0/1363451821 shutdown_connections 2024-09-16T00:31:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc848106a80 0x7fc84819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc8300a3240 0x7fc8300a5700 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc848101420 0x7fc848198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 --2- 172.21.15.1:0/1363451821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc848100a80 0x7fc848198410 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 -- 172.21.15.1:0/1363451821 >> 172.21.15.1:0/1363451821 conn(0x7fc8480fc7a0 msgr2=0x7fc84806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 -- 172.21.15.1:0/1363451821 shutdown_connections 2024-09-16T00:31:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:31:58.611+0000 7fc84ce73640 1 -- 172.21.15.1:0/1363451821 wait complete. 2024-09-16T00:31:59.467 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:31:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:00.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.211+0000 7f4ee5e09640 1 -- 172.21.15.1:0/670542656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee0103660 msgr2=0x7f4ee010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.211+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/670542656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee0103660 0x7f4ee010bb50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009900 tx=0x7f4ed402a480 comp rx=0 tx=0).stop 2024-09-16T00:32:00.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.211+0000 7f4ee5e09640 1 -- 172.21.15.1:0/670542656 shutdown_connections 2024-09-16T00:32:00.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.211+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/670542656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee010c170 0x7f4ee010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.211+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/670542656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee0103660 0x7f4ee010bb50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.211+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/670542656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee0102d40 0x7f4ee0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.211+0000 7f4ee5e09640 1 -- 172.21.15.1:0/670542656 >> 172.21.15.1:0/670542656 conn(0x7f4ee00fc7a0 msgr2=0x7f4ee00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.211+0000 7f4ee5e09640 1 -- 172.21.15.1:0/670542656 shutdown_connections 2024-09-16T00:32:00.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.211+0000 7f4ee5e09640 1 -- 172.21.15.1:0/670542656 wait complete. 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4ee5e09640 1 Processor -- start 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4ee5e09640 1 -- start start 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4ee5e09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee0102d40 0x7f4ee0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4ee5e09640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee0103660 0x7f4ee0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4ee5e09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee010c170 0x7f4ee019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4ee5e09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee0112030 con 0x7f4ee010c170 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4ee5e09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee0111eb0 con 0x7f4ee0102d40 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4ee5e09640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01121b0 con 0x7f4ee0103660 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4edffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee010c170 0x7f4ee019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4edffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee010c170 0x7f4ee019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39072/0 (socket says 172.21.15.1:39072) 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.212+0000 7f4edffff640 1 -- 172.21.15.1:0/2932555068 learned_addr learned my addr 172.21.15.1:0/2932555068 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee0102d40 0x7f4ee0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edeffd640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee0103660 0x7f4ee0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edffff640 1 -- 172.21.15.1:0/2932555068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee0103660 msgr2=0x7f4ee0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edffff640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee0103660 0x7f4ee0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edffff640 1 -- 172.21.15.1:0/2932555068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee0102d40 msgr2=0x7f4ee0198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edffff640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee0102d40 0x7f4ee0198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edffff640 1 -- 172.21.15.1:0/2932555068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee019d530 con 0x7f4ee010c170 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edffff640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee010c170 0x7f4ee019ce50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b970 tx=0x7f4ed800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edcff9640 1 -- 172.21.15.1:0/2932555068 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8019070 con 0x7f4ee010c170 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4ee5e09640 1 -- 172.21.15.1:0/2932555068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee019d820 con 0x7f4ee010c170 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edcff9640 1 -- 172.21.15.1:0/2932555068 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed8014440 con 0x7f4ee010c170 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4edcff9640 1 -- 172.21.15.1:0/2932555068 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80075d0 con 0x7f4ee010c170 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.213+0000 7f4ee5e09640 1 -- 172.21.15.1:0/2932555068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee01a5100 con 0x7f4ee010c170 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.216+0000 7f4edcff9640 1 -- 172.21.15.1:0/2932555068 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4ed801c070 con 0x7f4ee010c170 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.216+0000 7f4ee81e2740 1 Processor -- start 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4ee81e2740 1 -- start start 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9b5b40 0x5599eb9b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9c3780 0x5599eb9c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5599eb91a2f0 con 0x5599eb9c3780 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599eb8c1cf0 con 0x5599eb9b45d0 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5599eb9162b0 con 0x5599eb9b5b40 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9c3780 0x5599eb9c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9b5b40 0x5599eb9b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9b5b40 0x5599eb9b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:42310/0 (socket says 172.21.15.1:42310) 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edeffd640 1 -- 172.21.15.1:0/1297460696 learned_addr learned my addr 172.21.15.1:0/1297460696 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edeffd640 1 -- 172.21.15.1:0/1297460696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 msgr2=0x5599eb9b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edeffd640 1 --2- 172.21.15.1:0/1297460696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edeffd640 1 -- 172.21.15.1:0/1297460696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9c3780 msgr2=0x5599eb9c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:00.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edeffd640 1 --2- 172.21.15.1:0/1297460696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9c3780 0x5599eb9c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edeffd640 1 -- 172.21.15.1:0/1297460696 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599eb8a63a0 con 0x5599eb9b5b40 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edcff9640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4ec80a3240 0x7f4ec80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.217+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/1297460696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.218+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4ec80a3240 0x7f4ec80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.218+0000 7f4edeffd640 1 --2- 172.21.15.1:0/1297460696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9b5b40 0x5599eb9b4090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ed402a9a0 tx=0x7f4ed40058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5060bcd1bc97221a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.218+0000 7f4edcff9640 1 -- 172.21.15.1:0/2932555068 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4ed80cf2b0 con 0x7f4ee010c170 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.218+0000 7f4ec5ffb640 1 -- 172.21.15.1:0/1297460696 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4035050 con 0x5599eb9b5b40 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.218+0000 7f4ec5ffb640 1 -- 172.21.15.1:0/1297460696 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed4005020 con 0x5599eb9b5b40 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.218+0000 7f4ec5ffb640 1 -- 172.21.15.1:0/1297460696 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40049a0 con 0x5599eb9b5b40 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 -- 172.21.15.1:0/1297460696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9b5b40 msgr2=0x5599eb9b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4ec80a3240 0x7f4ec80a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5599eb9b1ac0 tx=0x7f4ed0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/1297460696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9b5b40 0x5599eb9b4090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ed402a9a0 tx=0x7f4ed40058d0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 -- 172.21.15.1:0/1297460696 shutdown_connections 2024-09-16T00:32:00.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/1297460696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9c3780 0x5599eb9c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/1297460696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/1297460696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9b5b40 0x5599eb9b4090 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 -- 172.21.15.1:0/1297460696 >> 172.21.15.1:0/1297460696 conn(0x5599eb97d310 msgr2=0x5599eb9b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 -- 172.21.15.1:0/1297460696 shutdown_connections 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 -- 172.21.15.1:0/1297460696 wait complete. 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 -- start start 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 Processor -- start 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9c0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5599eb9c6420 con 0x5599eb9b5b40 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599eb8d5060 con 0x5599eb9b45d0 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.219+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5599eb9c65a0 con 0x5599eb9c3780 2024-09-16T00:32:00.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4ee81e2740 10 client.0 _ll_register_callbacks cb 0x5599eb6288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9c03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39092/0 (socket says 172.21.15.1:39092) 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edeffd640 1 -- 172.21.15.1:0/3035689063 learned_addr learned my addr 172.21.15.1:0/3035689063 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edffff640 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9c0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edeffd640 1 -- 172.21.15.1:0/3035689063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 msgr2=0x5599eb9c0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edeffd640 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9c0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edeffd640 1 -- 172.21.15.1:0/3035689063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 msgr2=0x5599eb9c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edeffd640 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edeffd640 1 -- 172.21.15.1:0/3035689063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599eb68b3c0 con 0x5599eb9b5b40 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4edeffd640 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9c03e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4ed402a4c0 tx=0x7f4ed402ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93611b8acb2c79e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4ebeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:00.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3035689063 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed404f060 con 0x5599eb9b5b40 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3035689063 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4ed40053d0 con 0x5599eb9b5b40 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3035689063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed4005020 con 0x5599eb9b5b40 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3035689063 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ed4056050 con 0x5599eb9b5b40 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4ee81e2740 10 client.16080 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.220+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3035689063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599eb9d1e70 con 0x5599eb9b5b40 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.221+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3035689063 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed4005020 con 0x5599eb9b5b40 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.221+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3035689063 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4ed40035f0 con 0x5599eb9b5b40 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.221+0000 7f4ee81e2740 10 client.16080 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.221+0000 7f4ee81e2740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.221+0000 7f4ee81e2740 20 client.16080 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.221+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5599eb9d31c0 0x5599eb9d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.221+0000 7f4ee81e2740 4 client.16080 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:00.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.221+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3035689063 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599eb927860 con 0x5599eb9d31c0 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.221+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5599eb9d31c0 0x5599eb9d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.222+0000 7f4ebeffd640 10 client.16080 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.222+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5599eb9d31c0 0x5599eb9d55b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fae91a7bcffd72f server_cookie=39105dd7dec16645 in_seq=0 out_seq=0 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.222+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3035689063 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599eb927860 con 0x5599eb9d31c0 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.222+0000 7f4ebeffd640 10 client.16080 handle_command_reply: tid=0 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.222+0000 7f4ee81e2740 1 client.16080 shutdown 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3035689063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 msgr2=0x5599eb9c03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9c03e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4ed402a4c0 tx=0x7f4ed402ac10 comp rx=0 tx=0).stop 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3035689063 shutdown_connections 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5599eb9d31c0 0x5599eb9d55b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9c0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9c03e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3035689063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ebeffd640 0 client.16080 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3035689063 >> 172.21.15.1:0/3035689063 conn(0x5599eb97d310 msgr2=0x5599eb9b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3035689063 shutdown_connections 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3035689063 wait complete. 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.223+0000 7f4ee81e2740 20 client.16080 trim_cache size 0 max 16384 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4ee81e2740 1 Processor -- start 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4ee81e2740 1 -- start start 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01af590 con 0x5599eb9b5b40 2024-09-16T00:32:00.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01ace60 con 0x5599eb9b45d0 2024-09-16T00:32:00.262 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:32:00.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01aacb0 con 0x5599eb9c3780 2024-09-16T00:32:00.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48634/0 (socket says 172.21.15.1:48634) 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edeffd640 1 -- 172.21.15.1:0/949978834 learned_addr learned my addr 172.21.15.1:0/949978834 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edffff640 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edeffd640 1 -- 172.21.15.1:0/949978834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 msgr2=0x5599eb9d85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edeffd640 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9d85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edeffd640 1 -- 172.21.15.1:0/949978834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 msgr2=0x5599eb9c03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edeffd640 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9c03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edeffd640 1 -- 172.21.15.1:0/949978834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599eb9c34a0 con 0x5599eb9b5b40 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edffff640 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.236+0000 7f4edeffd640 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9abad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40039c0 tx=0x7f4ed4032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f102c8fa1b7262 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ec57fa640 1 -- 172.21.15.1:0/949978834 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed404f070 con 0x5599eb9b5b40 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ec57fa640 1 -- 172.21.15.1:0/949978834 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed4041070 con 0x5599eb9b5b40 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ec57fa640 1 -- 172.21.15.1:0/949978834 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4035040 con 0x5599eb9b5b40 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ee81e2740 1 -- 172.21.15.1:0/949978834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 msgr2=0x5599eb9abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9abad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40039c0 tx=0x7f4ed4032f70 comp rx=0 tx=0).stop 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ee81e2740 1 -- 172.21.15.1:0/949978834 shutdown_connections 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b5b40 0x5599eb9abad0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/949978834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b45d0 0x5599eb9c03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ee81e2740 1 -- 172.21.15.1:0/949978834 >> 172.21.15.1:0/949978834 conn(0x5599eb97d310 msgr2=0x5599eb9bfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ee81e2740 1 -- 172.21.15.1:0/949978834 shutdown_connections 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.237+0000 7f4ee81e2740 1 -- 172.21.15.1:0/949978834 wait complete. 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4ee81e2740 1 -- start start 2024-09-16T00:32:00.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4ee81e2740 1 Processor -- start 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b45d0 0x5599eb9d6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b5b40 0x5599eb9d68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4ee81e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01ab320 con 0x5599eb9b45d0 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01ac790 con 0x5599eb9b5b40 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4ee81e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01ab9f0 con 0x5599eb9c3780 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4ee81e2740 10 client.0 _ll_register_callbacks cb 0x7f4ed4032e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4edf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b45d0 0x5599eb9d6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4edf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b45d0 0x5599eb9d6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48650/0 (socket says 172.21.15.1:48650) 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4edf7fe640 1 -- 172.21.15.1:0/3599646493 learned_addr learned my addr 172.21.15.1:0/3599646493 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.238+0000 7f4edeffd640 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b5b40 0x5599eb9d68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4edf7fe640 1 -- 172.21.15.1:0/3599646493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 msgr2=0x5599eb9d6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:00.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4edf7fe640 1 -- 172.21.15.1:0/3599646493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b5b40 msgr2=0x5599eb9d68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b5b40 0x5599eb9d68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4edf7fe640 1 -- 172.21.15.1:0/3599646493 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599eba3cb50 con 0x5599eb9b45d0 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4edeffd640 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b5b40 0x5599eb9d68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4edf7fe640 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b45d0 0x5599eb9d6390 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0011b20 tx=0x7f4ed00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c534eebd177f2ed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ebeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3599646493 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0024e20 con 0x5599eb9b45d0 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3599646493 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4ed000f040 con 0x5599eb9b45d0 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3599646493 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed4041070 con 0x5599eb9b45d0 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3599646493 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ed003b050 con 0x5599eb9b45d0 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ee81e2740 10 client.16086 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3599646493 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599eba3de10 con 0x5599eb9b45d0 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3599646493 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed4041070 con 0x5599eb9b45d0 2024-09-16T00:32:00.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3599646493 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4ed0020040 con 0x5599eb9b45d0 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ee81e2740 10 client.16086 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.239+0000 7f4ee81e2740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ee81e2740 20 client.16086 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5599eba3e560 0x5599eba409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ee81e2740 4 client.16086 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3599646493 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599eb921d70 con 0x5599eba3e560 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4edeffd640 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5599eba3e560 0x5599eba409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4edeffd640 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5599eba3e560 0x5599eba409e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51da1f1246f4cd68 server_cookie=27c9cbcf43a082f2 in_seq=0 out_seq=0 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ebeffd640 10 client.16086 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ebeffd640 1 -- 172.21.15.1:0/3599646493 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599eb921d70 con 0x5599eba3e560 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ebeffd640 10 client.16086 handle_command_reply: tid=0 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ee81e2740 1 client.16086 shutdown 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3599646493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b45d0 msgr2=0x5599eb9d6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.240+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b45d0 0x5599eb9d6390 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0011b20 tx=0x7f4ed00047d0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3599646493 shutdown_connections 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5599eba3e560 0x5599eba409e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599eb9c3780 0x5599eb9d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599eb9b5b40 0x5599eb9d68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee81e2740 1 --2- 172.21.15.1:0/3599646493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599eb9b45d0 0x5599eb9d6390 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3599646493 >> 172.21.15.1:0/3599646493 conn(0x5599eb97d310 msgr2=0x5599eb9be870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ebeffd640 0 client.16086 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3599646493 shutdown_connections 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee81e2740 1 -- 172.21.15.1:0/3599646493 wait complete. 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee81e2740 20 client.16086 trim_cache size 0 max 16384 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 -- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4ec80a3240 msgr2=0x7f4ec80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4ec80a3240 0x7f4ec80a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5599eb9b1ac0 tx=0x7f4ed0009110 comp rx=0 tx=0).stop 2024-09-16T00:32:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 -- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee010c170 msgr2=0x7f4ee019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee010c170 0x7f4ee019ce50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b970 tx=0x7f4ed800bd70 comp rx=0 tx=0).stop 2024-09-16T00:32:00.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 -- 172.21.15.1:0/2932555068 shutdown_connections 2024-09-16T00:32:00.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee010c170 0x7f4ee019ce50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4ec80a3240 0x7f4ec80a5700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee0103660 0x7f4ee0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 --2- 172.21.15.1:0/2932555068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee0102d40 0x7f4ee0198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 -- 172.21.15.1:0/2932555068 >> 172.21.15.1:0/2932555068 conn(0x7f4ee00fc7a0 msgr2=0x7f4ee00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 -- 172.21.15.1:0/2932555068 shutdown_connections 2024-09-16T00:32:00.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:00.241+0000 7f4ee5e09640 1 -- 172.21.15.1:0/2932555068 wait complete. 2024-09-16T00:32:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:00 smithi001 ceph-mon[30934]: pgmap v458: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-16T00:32:00.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:00 smithi153 ceph-mon[35538]: pgmap v458: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-16T00:32:00.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:00 smithi019 ceph-mon[35186]: pgmap v458: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-16T00:32:02.056 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:32:02.056 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:32:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:02 smithi001 ceph-mon[30934]: pgmap v459: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:32:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:02 smithi153 ceph-mon[35538]: pgmap v459: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:32:02.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:02 smithi019 ceph-mon[35186]: pgmap v459: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:32:02.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.821+0000 7f3648245640 1 -- 172.21.15.1:0/2096355600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640100e20 msgr2=0x7f3640101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.821+0000 7f3648245640 1 --2- 172.21.15.1:0/2096355600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640100e20 0x7f3640101200 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f363c009900 tx=0x7f363c02a480 comp rx=0 tx=0).stop 2024-09-16T00:32:02.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.821+0000 7f3648245640 1 -- 172.21.15.1:0/2096355600 shutdown_connections 2024-09-16T00:32:02.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.821+0000 7f3648245640 1 --2- 172.21.15.1:0/2096355600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3640102ef0 0x7f364010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.821+0000 7f3648245640 1 --2- 172.21.15.1:0/2096355600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3640102570 0x7f36401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.821+0000 7f3648245640 1 --2- 172.21.15.1:0/2096355600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640100e20 0x7f3640101200 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.821+0000 7f3648245640 1 -- 172.21.15.1:0/2096355600 >> 172.21.15.1:0/2096355600 conn(0x7f36400fc7d0 msgr2=0x7f36400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.821+0000 7f3648245640 1 -- 172.21.15.1:0/2096355600 shutdown_connections 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.821+0000 7f3648245640 1 -- 172.21.15.1:0/2096355600 wait complete. 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f3648245640 1 Processor -- start 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f3648245640 1 -- start start 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f3648245640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3640100e20 0x7f36401a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f3648245640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640102570 0x7f36401a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f3648245640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3640102ef0 0x7f364019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f3648245640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3640114290 con 0x7f3640102570 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f3648245640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3640114110 con 0x7f3640102ef0 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f3648245640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3640114410 con 0x7f3640100e20 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f36457e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640102570 0x7f36401a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f36457e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640102570 0x7f36401a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48658/0 (socket says 172.21.15.1:48658) 2024-09-16T00:32:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f36457e0640 1 -- 172.21.15.1:0/513854972 learned_addr learned my addr 172.21.15.1:0/513854972 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f36467e2640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3640102ef0 0x7f364019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.822+0000 7f3645fe1640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3640100e20 0x7f36401a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f36457e0640 1 -- 172.21.15.1:0/513854972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3640100e20 msgr2=0x7f36401a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f36457e0640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3640100e20 0x7f36401a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f36457e0640 1 -- 172.21.15.1:0/513854972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3640102ef0 msgr2=0x7f364019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f36457e0640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3640102ef0 0x7f364019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f36457e0640 1 -- 172.21.15.1:0/513854972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364019ace0 con 0x7f3640102570 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f36467e2640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3640102ef0 0x7f364019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f36457e0640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640102570 0x7f36401a0930 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f363000e9f0 tx=0x7f363000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f3636ffd640 1 -- 172.21.15.1:0/513854972 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363000cd80 con 0x7f3640102570 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f3636ffd640 1 -- 172.21.15.1:0/513854972 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3630005500 con 0x7f3640102570 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f3648245640 1 -- 172.21.15.1:0/513854972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364019afd0 con 0x7f3640102570 2024-09-16T00:32:02.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f3636ffd640 1 -- 172.21.15.1:0/513854972 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630010690 con 0x7f3640102570 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.823+0000 7f3648245640 1 -- 172.21.15.1:0/513854972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36401a72c0 con 0x7f3640102570 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.825+0000 7f3636ffd640 1 -- 172.21.15.1:0/513854972 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f36300046c0 con 0x7f3640102570 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f364a61e740 1 Processor -- start 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f364a61e740 1 -- start start 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f364a61e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f364a61e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c1480 0x5636e47cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f364a61e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 0x5636e47ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f364a61e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5636e4726280 con 0x5636e47c1480 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f364a61e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636e46cdb10 con 0x5636e47c0b60 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f364a61e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5636e4722240 con 0x5636e47cbd40 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f3645fe1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c1480 0x5636e47cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f3645fe1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c1480 0x5636e47cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48674/0 (socket says 172.21.15.1:48674) 2024-09-16T00:32:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f3645fe1640 1 -- 172.21.15.1:0/3748413841 learned_addr learned my addr 172.21.15.1:0/3748413841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f3636ffd640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f362c0a3230 0x7f362c0a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f36457e0640 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 0x5636e47ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.826+0000 7f36467e2640 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f3645fe1640 1 -- 172.21.15.1:0/3748413841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 msgr2=0x5636e47ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f3645fe1640 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 0x5636e47ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f3645fe1640 1 -- 172.21.15.1:0/3748413841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 msgr2=0x5636e47c0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f3645fe1640 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47c0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f3645fe1640 1 -- 172.21.15.1:0/3748413841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e46da1f0 con 0x5636e47c1480 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f36467e2640 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f36457e0640 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 0x5636e47ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f36467e2640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f362c0a3230 0x7f362c0a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f3636ffd640 1 -- 172.21.15.1:0/513854972 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f36300d01a0 con 0x7f3640102570 2024-09-16T00:32:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f3645fe1640 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c1480 0x5636e47cb800 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f363c02a9a0 tx=0x7f363c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4613c61c61b22e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f362bfff640 1 -- 172.21.15.1:0/3748413841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c035030 con 0x5636e47c1480 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f362bfff640 1 -- 172.21.15.1:0/3748413841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363c040070 con 0x5636e47c1480 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.827+0000 7f362bfff640 1 -- 172.21.15.1:0/3748413841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c0318c0 con 0x5636e47c1480 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 -- 172.21.15.1:0/3748413841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c1480 msgr2=0x5636e47cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c1480 0x5636e47cb800 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f363c02a9a0 tx=0x7f363c005dd0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f36467e2640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f362c0a3230 0x7f362c0a56f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3638005c40 tx=0x7f3638004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 -- 172.21.15.1:0/3748413841 shutdown_connections 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 0x5636e47ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c1480 0x5636e47cb800 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 --2- 172.21.15.1:0/3748413841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 -- 172.21.15.1:0/3748413841 >> 172.21.15.1:0/3748413841 conn(0x5636e47bd0f0 msgr2=0x5636e47bd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 -- 172.21.15.1:0/3748413841 shutdown_connections 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 -- 172.21.15.1:0/3748413841 wait complete. 2024-09-16T00:32:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 -- start start 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.828+0000 7f364a61e740 1 Processor -- start 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f364a61e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c0b60 0x5636e47cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f364a61e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f364a61e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f364a61e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5636e47ce910 con 0x5636e47cbd40 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f364a61e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636e46e0f10 con 0x5636e47c1480 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f364a61e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5636e47cea90 con 0x5636e47c0b60 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f364a61e740 10 client.0 _ll_register_callbacks cb 0x5636e46a9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36467e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36467e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48682/0 (socket says 172.21.15.1:48682) 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36467e2640 1 -- 172.21.15.1:0/3779334559 learned_addr learned my addr 172.21.15.1:0/3779334559 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36457e0640 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f3645fe1640 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c0b60 0x5636e47cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36467e2640 1 -- 172.21.15.1:0/3779334559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c0b60 msgr2=0x5636e47cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36467e2640 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c0b60 0x5636e47cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36467e2640 1 -- 172.21.15.1:0/3779334559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 msgr2=0x5636e47c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36467e2640 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36467e2640 1 -- 172.21.15.1:0/3779334559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e44976a0 con 0x5636e47cbd40 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36457e0640 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f3645fe1640 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c0b60 0x5636e47cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.829+0000 7f36467e2640 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c92e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3638009b10 tx=0x7f3638009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1e477c8b580ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f36297fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f36297fa640 1 -- 172.21.15.1:0/3779334559 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363801c920 con 0x5636e47cbd40 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f364a61e740 1 -- 172.21.15.1:0/3779334559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f363c040070 con 0x5636e47cbd40 2024-09-16T00:32:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f36297fa640 1 -- 172.21.15.1:0/3779334559 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f363801cac0 con 0x5636e47cbd40 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f36297fa640 1 -- 172.21.15.1:0/3779334559 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3638031050 con 0x5636e47cbd40 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f364a61e740 10 client.16098 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f364a61e740 1 -- 172.21.15.1:0/3779334559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5636e47da410 con 0x5636e47cbd40 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f36297fa640 1 -- 172.21.15.1:0/3779334559 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363c040070 con 0x5636e47cbd40 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f36297fa640 1 -- 172.21.15.1:0/3779334559 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f363801a5c0 con 0x5636e47cbd40 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f364a61e740 10 client.16098 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f364a61e740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f364a61e740 20 client.16098 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f364a61e740 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5636e47db8d0 0x5636e47ddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f364a61e740 4 client.16098 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.830+0000 7f364a61e740 1 -- 172.21.15.1:0/3779334559 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636e47337f0 con 0x5636e47db8d0 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.831+0000 7f3645fe1640 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5636e47db8d0 0x5636e47ddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.831+0000 7f3645fe1640 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5636e47db8d0 0x5636e47ddcc0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7554b7cbc48c4591 server_cookie=5ebdf46516f9fd9f in_seq=0 out_seq=0 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.831+0000 7f36297fa640 10 client.16098 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f36297fa640 1 -- 172.21.15.1:0/3779334559 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636e47337f0 con 0x5636e47db8d0 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f36297fa640 10 client.16098 handle_command_reply: tid=0 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 client.16098 shutdown 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 -- 172.21.15.1:0/3779334559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 msgr2=0x5636e47c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c92e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3638009b10 tx=0x7f3638009500 comp rx=0 tx=0).stop 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 -- 172.21.15.1:0/3779334559 shutdown_connections 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5636e47db8d0 0x5636e47ddcc0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c92e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f36297fa640 0 client.16098 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 --2- 172.21.15.1:0/3779334559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c0b60 0x5636e47cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 -- 172.21.15.1:0/3779334559 >> 172.21.15.1:0/3779334559 conn(0x5636e47bd0f0 msgr2=0x5636e47c2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 -- 172.21.15.1:0/3779334559 shutdown_connections 2024-09-16T00:32:02.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 1 -- 172.21.15.1:0/3779334559 wait complete. 2024-09-16T00:32:02.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.832+0000 7f364a61e740 20 client.16098 trim_cache size 0 max 16384 2024-09-16T00:32:02.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.849+0000 7f364a61e740 1 Processor -- start 2024-09-16T00:32:02.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f364a61e740 1 -- start start 2024-09-16T00:32:02.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f364a61e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47e3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f364a61e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c1480 0x5636e47e3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f364a61e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f364a61e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b3960 con 0x5636e47cbd40 2024-09-16T00:32:02.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f364a61e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b1230 con 0x5636e47c0b60 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f364a61e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36401af080 con 0x5636e47c1480 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36467e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36467e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48692/0 (socket says 172.21.15.1:48692) 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36467e2640 1 -- 172.21.15.1:0/2391432481 learned_addr learned my addr 172.21.15.1:0/2391432481 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f3645fe1640 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36457e0640 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c1480 0x5636e47e3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36467e2640 1 -- 172.21.15.1:0/2391432481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c1480 msgr2=0x5636e47e3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36467e2640 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c1480 0x5636e47e3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36467e2640 1 -- 172.21.15.1:0/2391432481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 msgr2=0x5636e47e3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36467e2640 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47e3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36467e2640 1 -- 172.21.15.1:0/2391432481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e47c12b0 con 0x5636e47cbd40 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f3645fe1640 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36457e0640 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c1480 0x5636e47e3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.850+0000 7f36467e2640 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c5db0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3638016040 tx=0x7f3638000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d29da54bbc88 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f362b7fe640 1 -- 172.21.15.1:0/2391432481 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800f880 con 0x5636e47cbd40 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f362b7fe640 1 -- 172.21.15.1:0/2391432481 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363801add0 con 0x5636e47cbd40 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f362b7fe640 1 -- 172.21.15.1:0/2391432481 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638028480 con 0x5636e47cbd40 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f364a61e740 1 -- 172.21.15.1:0/2391432481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 msgr2=0x5636e47c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f364a61e740 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c5db0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3638016040 tx=0x7f3638000f40 comp rx=0 tx=0).stop 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f364a61e740 1 -- 172.21.15.1:0/2391432481 shutdown_connections 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f364a61e740 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47cbd40 0x5636e47c5db0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f364a61e740 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47c1480 0x5636e47e3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f364a61e740 1 --2- 172.21.15.1:0/2391432481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c0b60 0x5636e47e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f364a61e740 1 -- 172.21.15.1:0/2391432481 >> 172.21.15.1:0/2391432481 conn(0x5636e47bd0f0 msgr2=0x5636e47c7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f364a61e740 1 -- 172.21.15.1:0/2391432481 shutdown_connections 2024-09-16T00:32:02.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.851+0000 7f364a61e740 1 -- 172.21.15.1:0/2391432481 wait complete. 2024-09-16T00:32:02.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f364a61e740 1 -- start start 2024-09-16T00:32:02.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f364a61e740 1 Processor -- start 2024-09-16T00:32:02.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f364a61e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c0b60 0x5636e47d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f364a61e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f364a61e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 0x5636e483f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f364a61e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36401af6f0 con 0x5636e47c0b60 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f364a61e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b0b60 con 0x5636e47c1480 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f364a61e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36401afdc0 con 0x5636e47cbd40 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f364a61e740 10 client.0 _ll_register_callbacks cb 0x7f3638009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f3645fe1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c0b60 0x5636e47d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f3645fe1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c0b60 0x5636e47d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48702/0 (socket says 172.21.15.1:48702) 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.852+0000 7f3645fe1640 1 -- 172.21.15.1:0/52474676 learned_addr learned my addr 172.21.15.1:0/52474676 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f36457e0640 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f3645fe1640 1 -- 172.21.15.1:0/52474676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 msgr2=0x5636e483f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f36467e2640 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 0x5636e483f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f3645fe1640 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 0x5636e483f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f3645fe1640 1 -- 172.21.15.1:0/52474676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 msgr2=0x5636e47d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:02.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f3645fe1640 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f3645fe1640 1 -- 172.21.15.1:0/52474676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e483fb70 con 0x5636e47c0b60 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f36457e0640 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f3645fe1640 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c0b60 0x5636e47d02a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f363c009900 tx=0x7f363c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3d43072d055809 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f36297fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f36297fa640 1 -- 172.21.15.1:0/52474676 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c035030 con 0x5636e47c0b60 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f36297fa640 1 -- 172.21.15.1:0/52474676 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f363c005460 con 0x5636e47c0b60 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f364a61e740 1 -- 172.21.15.1:0/52474676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f363801add0 con 0x5636e47c0b60 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f36297fa640 1 -- 172.21.15.1:0/52474676 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f363c05d050 con 0x5636e47c0b60 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f364a61e740 10 client.16104 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.853+0000 7f364a61e740 1 -- 172.21.15.1:0/52474676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5636e4846ef0 con 0x5636e47c0b60 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.854+0000 7f36297fa640 1 -- 172.21.15.1:0/52474676 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363801add0 con 0x5636e47c0b60 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.854+0000 7f36297fa640 1 -- 172.21.15.1:0/52474676 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f363c058030 con 0x5636e47c0b60 2024-09-16T00:32:02.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.854+0000 7f364a61e740 10 client.16104 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.854+0000 7f364a61e740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.854+0000 7f364a61e740 20 client.16104 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.854+0000 7f364a61e740 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5636e47db8d0 0x5636e4849240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.854+0000 7f364a61e740 4 client.16104 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.854+0000 7f364a61e740 1 -- 172.21.15.1:0/52474676 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5636e472dd00 con 0x5636e47db8d0 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.854+0000 7f36457e0640 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5636e47db8d0 0x5636e4849240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.855+0000 7f36457e0640 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5636e47db8d0 0x5636e4849240 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3b077331e418cd server_cookie=7ccca089494e8c81 in_seq=0 out_seq=0 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.855+0000 7f36297fa640 10 client.16104 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.855+0000 7f36297fa640 1 -- 172.21.15.1:0/52474676 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5636e472dd00 con 0x5636e47db8d0 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.855+0000 7f36297fa640 10 client.16104 handle_command_reply: tid=0 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.855+0000 7f364a61e740 1 client.16104 shutdown 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 -- 172.21.15.1:0/52474676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c0b60 msgr2=0x5636e47d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c0b60 0x5636e47d02a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f363c009900 tx=0x7f363c005d50 comp rx=0 tx=0).stop 2024-09-16T00:32:02.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 -- 172.21.15.1:0/52474676 shutdown_connections 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5636e47db8d0 0x5636e4849240 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636e47cbd40 0x5636e483f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f36297fa640 0 client.16104 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636e47c1480 0x5636e47d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 --2- 172.21.15.1:0/52474676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636e47c0b60 0x5636e47d02a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 -- 172.21.15.1:0/52474676 >> 172.21.15.1:0/52474676 conn(0x5636e47bd0f0 msgr2=0x5636e47c7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 -- 172.21.15.1:0/52474676 shutdown_connections 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 1 -- 172.21.15.1:0/52474676 wait complete. 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.856+0000 7f364a61e740 20 client.16104 trim_cache size 0 max 16384 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:32:02.862 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:32:02.863 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "835c3866-31ba-4530-9dbe-a5bdf33fe3ae", 2024-09-16T00:32:02.863 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:32:02.863 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:32:02.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 -- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f362c0a3230 msgr2=0x7f362c0a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f362c0a3230 0x7f362c0a56f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3638005c40 tx=0x7f3638004a40 comp rx=0 tx=0).stop 2024-09-16T00:32:02.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 -- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640102570 msgr2=0x7f36401a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640102570 0x7f36401a0930 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f363000e9f0 tx=0x7f363000edf0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 -- 172.21.15.1:0/513854972 shutdown_connections 2024-09-16T00:32:02.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3640102ef0 0x7f364019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3640102570 0x7f36401a0930 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f362c0a3230 0x7f362c0a56f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 --2- 172.21.15.1:0/513854972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3640100e20 0x7f36401a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 -- 172.21.15.1:0/513854972 >> 172.21.15.1:0/513854972 conn(0x7f36400fc7d0 msgr2=0x7f36400fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 -- 172.21.15.1:0/513854972 shutdown_connections 2024-09-16T00:32:02.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:02.857+0000 7f3648245640 1 -- 172.21.15.1:0/513854972 wait complete. 2024-09-16T00:32:03.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:03 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:32:03.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:03 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:03.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:03 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:32:03.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:03 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:03.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:03 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:32:03.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:03 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:32:03.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:03 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:03.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:03 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:32:03.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:03 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:03.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:03 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:32:03.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:03 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:32:03.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:03 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:03.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:03 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:32:03.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:03 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:03.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:03 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:32:03.757 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:32:04.016 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:04.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:04 smithi001 ceph-mon[30934]: pgmap v460: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:32:04.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:04 smithi153 ceph-mon[35538]: pgmap v460: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.537+0000 7f0d2fad2640 1 -- 172.21.15.1:0/179766592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d28101b60 msgr2=0x7f0d2810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.537+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/179766592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d28101b60 0x7f0d2810d970 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009ca0 tx=0x7f0d2002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.538+0000 7f0d2fad2640 1 -- 172.21.15.1:0/179766592 shutdown_connections 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.538+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/179766592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d28101b60 0x7f0d2810d970 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.538+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/179766592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d281011c0 0x7f0d28101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.538+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/179766592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d28105ad0 0x7f0d28105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.538+0000 7f0d2fad2640 1 -- 172.21.15.1:0/179766592 >> 172.21.15.1:0/179766592 conn(0x7f0d28077210 msgr2=0x7f0d280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.538+0000 7f0d2fad2640 1 -- 172.21.15.1:0/179766592 shutdown_connections 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.538+0000 7f0d2fad2640 1 -- 172.21.15.1:0/179766592 wait complete. 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.538+0000 7f0d2fad2640 1 Processor -- start 2024-09-16T00:32:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.538+0000 7f0d2fad2640 1 -- start start 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2fad2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d281011c0 0x7f0d281991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2fad2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d28101b60 0x7f0d28199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2fad2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d28105ad0 0x7f0d281a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2fad2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d28110ef0 con 0x7f0d281011c0 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2fad2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d28110d70 con 0x7f0d28101b60 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2fad2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d28111070 con 0x7f0d28105ad0 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d86e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d281011c0 0x7f0d281991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d86e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d281011c0 0x7f0d281991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48722/0 (socket says 172.21.15.1:48722) 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d86e640 1 -- 172.21.15.1:0/582329260 learned_addr learned my addr 172.21.15.1:0/582329260 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d06d640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d28101b60 0x7f0d28199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d28105ad0 0x7f0d281a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d86e640 1 -- 172.21.15.1:0/582329260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d28105ad0 msgr2=0x7f0d281a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d28105ad0 0x7f0d281a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d86e640 1 -- 172.21.15.1:0/582329260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d28101b60 msgr2=0x7f0d28199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d28101b60 0x7f0d28199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d86e640 1 -- 172.21.15.1:0/582329260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d281a47b0 con 0x7f0d281011c0 2024-09-16T00:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.539+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d281011c0 0x7f0d281991f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c890 tx=0x7f0d2400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.540+0000 7f0d1effd640 1 -- 172.21.15.1:0/582329260 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24004220 con 0x7f0d281011c0 2024-09-16T00:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.540+0000 7f0d1effd640 1 -- 172.21.15.1:0/582329260 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d240043c0 con 0x7f0d281011c0 2024-09-16T00:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.540+0000 7f0d1effd640 1 -- 172.21.15.1:0/582329260 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d240049e0 con 0x7f0d281011c0 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.540+0000 7f0d2fad2640 1 -- 172.21.15.1:0/582329260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d281a4980 con 0x7f0d281011c0 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.540+0000 7f0d2fad2640 1 -- 172.21.15.1:0/582329260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d281a4e40 con 0x7f0d281011c0 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.543+0000 7f0d1effd640 1 -- 172.21.15.1:0/582329260 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f0d240079b0 con 0x7f0d281011c0 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.543+0000 7f0d31eab740 1 Processor -- start 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.543+0000 7f0d31eab740 1 -- start start 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.543+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f14a0 0x555c9b9f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.543+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f1dc0 0x555c9b9fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.543+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 0x555c9b9fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d31eab740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555c9b9572e0 con 0x555c9b9f1dc0 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d31eab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c9b8fece0 con 0x555c9b9f14a0 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d31eab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555c9b9532a0 con 0x555c9b9fc680 2024-09-16T00:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d1effd640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0d140a32c0 0x7f0d140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d86e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f1dc0 0x555c9b9fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d86e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f1dc0 0x555c9b9fc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48738/0 (socket says 172.21.15.1:48738) 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d86e640 1 -- 172.21.15.1:0/1632838546 learned_addr learned my addr 172.21.15.1:0/1632838546 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/1632838546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f14a0 0x555c9b9f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0d140a32c0 0x7f0d140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d06d640 1 --2- 172.21.15.1:0/1632838546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 0x555c9b9fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d1effd640 1 -- 172.21.15.1:0/582329260 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0d240cf7f0 con 0x7f0d281011c0 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d86e640 1 -- 172.21.15.1:0/1632838546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 msgr2=0x555c9b9fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/1632838546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 0x555c9b9fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d86e640 1 -- 172.21.15.1:0/1632838546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f14a0 msgr2=0x555c9b9f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/1632838546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f14a0 0x555c9b9f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d86e640 1 -- 172.21.15.1:0/1632838546 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c9b8e3440 con 0x555c9b9f1dc0 2024-09-16T00:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0d140a32c0 0x7f0d140a5780 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0d200098d0 tx=0x7f0d20005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.544+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/1632838546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f1dc0 0x555c9b9fc140 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0d24098490 tx=0x7f0d24097e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e32f680cee0ab83 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d13fff640 1 -- 172.21.15.1:0/1632838546 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2401d040 con 0x555c9b9f1dc0 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d13fff640 1 -- 172.21.15.1:0/1632838546 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d240a4680 con 0x555c9b9f1dc0 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d13fff640 1 -- 172.21.15.1:0/1632838546 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24007ca0 con 0x555c9b9f1dc0 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d31eab740 1 -- 172.21.15.1:0/1632838546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f1dc0 msgr2=0x555c9b9fc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d31eab740 1 --2- 172.21.15.1:0/1632838546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f1dc0 0x555c9b9fc140 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0d24098490 tx=0x7f0d24097e90 comp rx=0 tx=0).stop 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d31eab740 1 -- 172.21.15.1:0/1632838546 shutdown_connections 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d31eab740 1 --2- 172.21.15.1:0/1632838546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 0x555c9b9fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d31eab740 1 --2- 172.21.15.1:0/1632838546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f1dc0 0x555c9b9fc140 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d31eab740 1 --2- 172.21.15.1:0/1632838546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f14a0 0x555c9b9f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d31eab740 1 -- 172.21.15.1:0/1632838546 >> 172.21.15.1:0/1632838546 conn(0x555c9b9eda30 msgr2=0x555c9b9ede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.545+0000 7f0d31eab740 1 -- 172.21.15.1:0/1632838546 shutdown_connections 2024-09-16T00:32:04.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 1 -- 172.21.15.1:0/1632838546 wait complete. 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 1 -- start start 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 1 Processor -- start 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f14a0 0x555c9b9fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9b9f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 0x555c9b9f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555c9b9ff250 con 0x555c9b9f14a0 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c9b912050 con 0x555c9b9f1dc0 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555c9b9ff3d0 con 0x555c9b9fc680 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d31eab740 10 client.0 _ll_register_callbacks cb 0x555c9b952160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.546+0000 7f0d2d86e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f14a0 0x555c9b9fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d86e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f14a0 0x555c9b9fbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48754/0 (socket says 172.21.15.1:48754) 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d86e640 1 -- 172.21.15.1:0/662970628 learned_addr learned my addr 172.21.15.1:0/662970628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 0x555c9b9f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d06d640 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9b9f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d86e640 1 -- 172.21.15.1:0/662970628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 msgr2=0x555c9b9f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 0x555c9b9f9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d86e640 1 -- 172.21.15.1:0/662970628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 msgr2=0x555c9b9f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9b9f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d86e640 1 -- 172.21.15.1:0/662970628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c9b6c83c0 con 0x555c9b9f14a0 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d06d640 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9b9f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f14a0 0x555c9b9fbf80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0d240a45f0 tx=0x7f0d24005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6e55a317baefcc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d117fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d117fa640 1 -- 172.21.15.1:0/662970628 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d240b8030 con 0x555c9b9f14a0 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d117fa640 1 -- 172.21.15.1:0/662970628 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0d240ba2c0 con 0x555c9b9f14a0 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d31eab740 1 -- 172.21.15.1:0/662970628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c9b964850 con 0x555c9b9f14a0 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d117fa640 1 -- 172.21.15.1:0/662970628 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d2401d040 con 0x555c9b9f14a0 2024-09-16T00:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d31eab740 10 client.16116 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.547+0000 7f0d31eab740 1 -- 172.21.15.1:0/662970628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c9ba0abf0 con 0x555c9b9f14a0 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.548+0000 7f0d117fa640 1 -- 172.21.15.1:0/662970628 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555c9b964850 con 0x555c9b9f14a0 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.548+0000 7f0d117fa640 1 -- 172.21.15.1:0/662970628 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0d240a0f50 con 0x555c9b9f14a0 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.548+0000 7f0d31eab740 10 client.16116 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.548+0000 7f0d31eab740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.548+0000 7f0d31eab740 20 client.16116 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.548+0000 7f0d31eab740 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555c9ba0bf40 0x555c9ba0e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.548+0000 7f0d31eab740 4 client.16116 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.548+0000 7f0d31eab740 1 -- 172.21.15.1:0/662970628 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c9b95ed60 con 0x555c9ba0bf40 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.548+0000 7f0d2d06d640 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555c9ba0bf40 0x555c9ba0e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.549+0000 7f0d2d06d640 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555c9ba0bf40 0x555c9ba0e330 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcda4678892b7b5c server_cookie=a7b2b4246bd97942 in_seq=0 out_seq=0 2024-09-16T00:32:04.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.549+0000 7f0d117fa640 10 client.16116 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:04.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.549+0000 7f0d117fa640 1 -- 172.21.15.1:0/662970628 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c9b95ed60 con 0x555c9ba0bf40 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.549+0000 7f0d117fa640 10 client.16116 handle_command_reply: tid=0 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.549+0000 7f0d31eab740 1 client.16116 shutdown 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.549+0000 7f0d31eab740 1 -- 172.21.15.1:0/662970628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f14a0 msgr2=0x555c9b9fbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.549+0000 7f0d31eab740 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f14a0 0x555c9b9fbf80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0d240a45f0 tx=0x7f0d24005300 comp rx=0 tx=0).stop 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d31eab740 1 -- 172.21.15.1:0/662970628 shutdown_connections 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d31eab740 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555c9ba0bf40 0x555c9ba0e330 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d31eab740 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9fc680 0x555c9b9f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d31eab740 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9b9f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d31eab740 1 --2- 172.21.15.1:0/662970628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9f14a0 0x555c9b9fbf80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d117fa640 0 client.16116 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d31eab740 1 -- 172.21.15.1:0/662970628 >> 172.21.15.1:0/662970628 conn(0x555c9b9eda30 msgr2=0x555c9b9f3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d31eab740 1 -- 172.21.15.1:0/662970628 shutdown_connections 2024-09-16T00:32:04.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d31eab740 1 -- 172.21.15.1:0/662970628 wait complete. 2024-09-16T00:32:04.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.550+0000 7f0d31eab740 20 client.16116 trim_cache size 0 max 16384 2024-09-16T00:32:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d31eab740 1 Processor -- start 2024-09-16T00:32:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d31eab740 1 -- start start 2024-09-16T00:32:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 0x555c9b9eca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9ba10f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d31eab740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281b1820 con 0x555c9b9fc680 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d31eab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281af0f0 con 0x555c9b9f1dc0 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d31eab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281acf40 con 0x555c9b9f14a0 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d2e06f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d2e06f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba114b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48768/0 (socket says 172.21.15.1:48768) 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.567+0000 7f0d2e06f640 1 -- 172.21.15.1:0/2475864677 learned_addr learned my addr 172.21.15.1:0/2475864677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 0x555c9b9eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2d06d640 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9ba10f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2e06f640 1 -- 172.21.15.1:0/2475864677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 msgr2=0x555c9b9eca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 0x555c9b9eca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2e06f640 1 -- 172.21.15.1:0/2475864677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 msgr2=0x555c9ba10f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9ba10f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2e06f640 1 -- 172.21.15.1:0/2475864677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c9b9ed860 con 0x555c9b9fc680 2024-09-16T00:32:04.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 0x555c9b9eca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:04.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2d06d640 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9ba10f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:04.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba114b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0d28106710 tx=0x7f0d200095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c5f83f7745afb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:04.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d137fe640 1 -- 172.21.15.1:0/2475864677 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2005a070 con 0x555c9b9fc680 2024-09-16T00:32:04.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d137fe640 1 -- 172.21.15.1:0/2475864677 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d20036070 con 0x555c9b9fc680 2024-09-16T00:32:04.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.568+0000 7f0d137fe640 1 -- 172.21.15.1:0/2475864677 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20045040 con 0x555c9b9fc680 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 -- 172.21.15.1:0/2475864677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 msgr2=0x555c9ba114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba114b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0d28106710 tx=0x7f0d200095c0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 -- 172.21.15.1:0/2475864677 shutdown_connections 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba114b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9ba10f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 --2- 172.21.15.1:0/2475864677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 0x555c9b9eca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 -- 172.21.15.1:0/2475864677 >> 172.21.15.1:0/2475864677 conn(0x555c9b9eda30 msgr2=0x555c9b9ec350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 -- 172.21.15.1:0/2475864677 shutdown_connections 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 -- 172.21.15.1:0/2475864677 wait complete. 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 -- start start 2024-09-16T00:32:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.569+0000 7f0d31eab740 1 Processor -- start 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 0x555c9ba11440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9ba0e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d31eab740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba0ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d31eab740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281ad5b0 con 0x555c9b9fc680 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d31eab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281aea20 con 0x555c9b9f1dc0 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d31eab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281adc80 con 0x555c9b9f14a0 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d31eab740 10 client.0 _ll_register_callbacks cb 0x7f0d20032d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2e06f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba0ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2e06f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba0ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48784/0 (socket says 172.21.15.1:48784) 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2e06f640 1 -- 172.21.15.1:0/901585912 learned_addr learned my addr 172.21.15.1:0/901585912 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 0x555c9ba11440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2d06d640 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9ba0e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2e06f640 1 -- 172.21.15.1:0/901585912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 msgr2=0x555c9ba11440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 0x555c9ba11440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2e06f640 1 -- 172.21.15.1:0/901585912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 msgr2=0x555c9ba0e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9ba0e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.570+0000 7f0d2e06f640 1 -- 172.21.15.1:0/901585912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c9ba0f240 con 0x555c9b9fc680 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d2e06f640 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba0ecd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0d20031040 tx=0x7f0d20005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f26349dadf881b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d117fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d117fa640 1 -- 172.21.15.1:0/901585912 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20045070 con 0x555c9b9fc680 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d117fa640 1 -- 172.21.15.1:0/901585912 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0d2002f860 con 0x555c9b9fc680 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d31eab740 1 -- 172.21.15.1:0/901585912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d20036070 con 0x555c9b9fc680 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d117fa640 1 -- 172.21.15.1:0/901585912 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d2006c050 con 0x555c9b9fc680 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d31eab740 10 client.16122 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d31eab740 1 -- 172.21.15.1:0/901585912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c9ba76560 con 0x555c9b9fc680 2024-09-16T00:32:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d117fa640 1 -- 172.21.15.1:0/901585912 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d20036070 con 0x555c9b9fc680 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d117fa640 1 -- 172.21.15.1:0/901585912 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0d2005e420 con 0x555c9b9fc680 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d31eab740 10 client.16122 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d31eab740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d31eab740 20 client.16122 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d31eab740 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555c9ba76bc0 0x555c9ba78fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d31eab740 4 client.16122 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.571+0000 7f0d31eab740 1 -- 172.21.15.1:0/901585912 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c9b9f9b00 con 0x555c9ba76bc0 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555c9ba76bc0 0x555c9ba78fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d2d86e640 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555c9ba76bc0 0x555c9ba78fb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5714425d89d26772 server_cookie=6f301762bfdfe444 in_seq=0 out_seq=0 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d117fa640 10 client.16122 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d117fa640 1 -- 172.21.15.1:0/901585912 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555c9b9f9b00 con 0x555c9ba76bc0 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d117fa640 10 client.16122 handle_command_reply: tid=0 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 client.16122 shutdown 2024-09-16T00:32:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 -- 172.21.15.1:0/901585912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 msgr2=0x555c9ba0ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba0ecd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0d20031040 tx=0x7f0d20005ef0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 -- 172.21.15.1:0/901585912 shutdown_connections 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555c9ba76bc0 0x555c9ba78fb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c9b9fc680 0x555c9ba0ecd0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c9b9f1dc0 0x555c9ba0e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d117fa640 0 client.16122 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 --2- 172.21.15.1:0/901585912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c9b9f14a0 0x555c9ba11440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 -- 172.21.15.1:0/901585912 >> 172.21.15.1:0/901585912 conn(0x555c9b9eda30 msgr2=0x555c9b9f79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.572+0000 7f0d31eab740 1 -- 172.21.15.1:0/901585912 shutdown_connections 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d31eab740 1 -- 172.21.15.1:0/901585912 wait complete. 2024-09-16T00:32:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d31eab740 20 client.16122 trim_cache size 0 max 16384 2024-09-16T00:32:04.579 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:32:04.579 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:32:04.579 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:32:04.579 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:32:04.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 -- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0d140a32c0 msgr2=0x7f0d140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0d140a32c0 0x7f0d140a5780 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0d200098d0 tx=0x7f0d20005f60 comp rx=0 tx=0).stop 2024-09-16T00:32:04.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 -- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d281011c0 msgr2=0x7f0d281991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d281011c0 0x7f0d281991f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c890 tx=0x7f0d2400cd50 comp rx=0 tx=0).stop 2024-09-16T00:32:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 -- 172.21.15.1:0/582329260 shutdown_connections 2024-09-16T00:32:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0d140a32c0 0x7f0d140a5780 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d28105ad0 0x7f0d281a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d28101b60 0x7f0d28199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 --2- 172.21.15.1:0/582329260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d281011c0 0x7f0d281991f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 -- 172.21.15.1:0/582329260 >> 172.21.15.1:0/582329260 conn(0x7f0d28077210 msgr2=0x7f0d28103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 -- 172.21.15.1:0/582329260 shutdown_connections 2024-09-16T00:32:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:04.573+0000 7f0d2fad2640 1 -- 172.21.15.1:0/582329260 wait complete. 2024-09-16T00:32:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:04 smithi019 ceph-mon[35186]: pgmap v460: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:32:05.498 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:32:05.498 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:05.762 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:06.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.256+0000 7f674c8d4640 1 -- 172.21.15.1:0/3563951119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6748103660 msgr2=0x7f674810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:06.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.256+0000 7f674c8d4640 1 --2- 172.21.15.1:0/3563951119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6748103660 0x7f674810bb50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f673c009900 tx=0x7f673c02a480 comp rx=0 tx=0).stop 2024-09-16T00:32:06.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.256+0000 7f674c8d4640 1 -- 172.21.15.1:0/3563951119 shutdown_connections 2024-09-16T00:32:06.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.256+0000 7f674c8d4640 1 --2- 172.21.15.1:0/3563951119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f674810c170 0x7f674810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.256+0000 7f674c8d4640 1 --2- 172.21.15.1:0/3563951119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6748103660 0x7f674810bb50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.256+0000 7f674c8d4640 1 --2- 172.21.15.1:0/3563951119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6748102d40 0x7f6748103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.256+0000 7f674c8d4640 1 -- 172.21.15.1:0/3563951119 >> 172.21.15.1:0/3563951119 conn(0x7f67480fc7a0 msgr2=0x7f67480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:06.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.256+0000 7f674c8d4640 1 -- 172.21.15.1:0/3563951119 shutdown_connections 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.256+0000 7f674c8d4640 1 -- 172.21.15.1:0/3563951119 wait complete. 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.257+0000 7f674c8d4640 1 Processor -- start 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.257+0000 7f674c8d4640 1 -- start start 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f674c8d4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6748102d40 0x7f67481983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f674c8d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6748103660 0x7f6748198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f674c8d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f674810c170 0x7f674819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f674c8d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6748111f70 con 0x7f674810c170 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f674c8d4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6748111df0 con 0x7f6748103660 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f674c8d4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f67481120f0 con 0x7f6748102d40 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6746d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f674810c170 0x7f674819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6746d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f674810c170 0x7f674819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48804/0 (socket says 172.21.15.1:48804) 2024-09-16T00:32:06.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6746d9d640 1 -- 172.21.15.1:0/103366050 learned_addr learned my addr 172.21.15.1:0/103366050 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f674659c640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6748102d40 0x7f67481983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6745d9b640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6748103660 0x7f6748198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6746d9d640 1 -- 172.21.15.1:0/103366050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6748102d40 msgr2=0x7f67481983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6746d9d640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6748102d40 0x7f67481983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6746d9d640 1 -- 172.21.15.1:0/103366050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6748103660 msgr2=0x7f6748198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6746d9d640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6748103660 0x7f6748198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6746d9d640 1 -- 172.21.15.1:0/103366050 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674819d380 con 0x7f674810c170 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6745d9b640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6748103660 0x7f6748198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f674659c640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6748102d40 0x7f67481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.258+0000 7f6746d9d640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f674810c170 0x7f674819cca0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f674000eaa0 tx=0x7f674000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.259+0000 7f67377fe640 1 -- 172.21.15.1:0/103366050 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674000cdb0 con 0x7f674810c170 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.259+0000 7f67377fe640 1 -- 172.21.15.1:0/103366050 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6740005020 con 0x7f674810c170 2024-09-16T00:32:06.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.259+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674819d670 con 0x7f674810c170 2024-09-16T00:32:06.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.259+0000 7f67377fe640 1 -- 172.21.15.1:0/103366050 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67400116b0 con 0x7f674810c170 2024-09-16T00:32:06.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.259+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67481a4f50 con 0x7f674810c170 2024-09-16T00:32:06.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.260+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6748103120 con 0x7f674810c170 2024-09-16T00:32:06.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.261+0000 7f67377fe640 1 -- 172.21.15.1:0/103366050 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6740005560 con 0x7f674810c170 2024-09-16T00:32:06.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.262+0000 7f67377fe640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f67300a3240 0x7f67300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:06.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.262+0000 7f674659c640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f67300a3240 0x7f67300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:06.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.262+0000 7f67377fe640 1 -- 172.21.15.1:0/103366050 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f67400cffe0 con 0x7f674810c170 2024-09-16T00:32:06.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.262+0000 7f674659c640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f67300a3240 0x7f67300a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f673800abe0 tx=0x7f673800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:06.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.266+0000 7f67377fe640 1 -- 172.21.15.1:0/103366050 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6740016030 con 0x7f674810c170 2024-09-16T00:32:06.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:06 smithi001 ceph-mon[30934]: pgmap v461: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-16T00:32:06.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:06 smithi153 ceph-mon[35538]: pgmap v461: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-16T00:32:06.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.479+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6748107240 con 0x7f674810c170 2024-09-16T00:32:06.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.479+0000 7f67377fe640 1 -- 172.21.15.1:0/103366050 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f674009c8c0 con 0x7f674810c170 2024-09-16T00:32:06.480 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:32:06.480 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:32:06.480 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:32:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f67300a3240 msgr2=0x7f67300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f67300a3240 0x7f67300a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f673800abe0 tx=0x7f673800a5c0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f674810c170 msgr2=0x7f674819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f674810c170 0x7f674819cca0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f674000eaa0 tx=0x7f674000ef60 comp rx=0 tx=0).stop 2024-09-16T00:32:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 shutdown_connections 2024-09-16T00:32:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f674810c170 0x7f674819cca0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6748103660 0x7f6748198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f67300a3240 0x7f67300a5700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 --2- 172.21.15.1:0/103366050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6748102d40 0x7f67481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 >> 172.21.15.1:0/103366050 conn(0x7f67480fc7a0 msgr2=0x7f67480fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 shutdown_connections 2024-09-16T00:32:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:06.482+0000 7f674c8d4640 1 -- 172.21.15.1:0/103366050 wait complete. 2024-09-16T00:32:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:06 smithi019 ceph-mon[35186]: pgmap v461: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-16T00:32:07.289 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:32:07.554 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:07.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:07 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/103366050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:07.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:07 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/103366050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:07.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:07 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/103366050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:08.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 -- 172.21.15.1:0/1179295523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 msgr2=0x7f49c4100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/1179295523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 0x7f49c4100f00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009900 tx=0x7f49c002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 -- 172.21.15.1:0/1179295523 shutdown_connections 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/1179295523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c410e500 0x7f49c4110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/1179295523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c4101440 0x7f49c410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/1179295523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 0x7f49c4100f00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 -- 172.21.15.1:0/1179295523 >> 172.21.15.1:0/1179295523 conn(0x7f49c40fc7d0 msgr2=0x7f49c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 -- 172.21.15.1:0/1179295523 shutdown_connections 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 -- 172.21.15.1:0/1179295523 wait complete. 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 Processor -- start 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.080+0000 7f49cbeb6640 1 -- start start 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49cbeb6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 0x7f49c41a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49cbeb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c4101440 0x7f49c41a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49cbeb6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c410e500 0x7f49c419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49cbeb6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49c4114260 con 0x7f49c4100b20 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49cbeb6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41140e0 con 0x7f49c4101440 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49cbeb6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41143e0 con 0x7f49c410e500 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9c52640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 0x7f49c41a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9c52640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 0x7f49c41a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48814/0 (socket says 172.21.15.1:48814) 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9c52640 1 -- 172.21.15.1:0/2760580328 learned_addr learned my addr 172.21.15.1:0/2760580328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9451640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c4101440 0x7f49c41a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49ca453640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c410e500 0x7f49c419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9c52640 1 -- 172.21.15.1:0/2760580328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c410e500 msgr2=0x7f49c419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9c52640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c410e500 0x7f49c419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9c52640 1 -- 172.21.15.1:0/2760580328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c4101440 msgr2=0x7f49c41a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9c52640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c4101440 0x7f49c41a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9c52640 1 -- 172.21.15.1:0/2760580328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c419ad00 con 0x7f49c4100b20 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49c9c52640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 0x7f49c41a0410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006010 tx=0x7f49c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49b6ffd640 1 -- 172.21.15.1:0/2760580328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c003f030 con 0x7f49c4100b20 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.081+0000 7f49b6ffd640 1 -- 172.21.15.1:0/2760580328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c0003ce0 con 0x7f49c4100b20 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.082+0000 7f49cbeb6640 1 -- 172.21.15.1:0/2760580328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c419af90 con 0x7f49c4100b20 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.082+0000 7f49b6ffd640 1 -- 172.21.15.1:0/2760580328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0035070 con 0x7f49c4100b20 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.082+0000 7f49cbeb6640 1 -- 172.21.15.1:0/2760580328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c41a72e0 con 0x7f49c4100b20 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.084+0000 7f49b6ffd640 1 -- 172.21.15.1:0/2760580328 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f49c0042030 con 0x7f49c4100b20 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.084+0000 7f49ce28f740 1 Processor -- start 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.084+0000 7f49ce28f740 1 -- start start 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 0x55fb6d997090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d997a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9a2370 0x55fb6d9a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49ce28f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fb6d8fd2f0 con 0x55fb6d9a2370 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49ce28f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb6d8a4cf0 con 0x55fb6d998b40 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49ce28f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb6d8f92b0 con 0x55fb6d9975d0 2024-09-16T00:32:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49c9451640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9a2370 0x55fb6d9a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49c9451640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9a2370 0x55fb6d9a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48828/0 (socket says 172.21.15.1:48828) 2024-09-16T00:32:08.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49c9451640 1 -- 172.21.15.1:0/2864436241 learned_addr learned my addr 172.21.15.1:0/2864436241 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:08.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49ca453640 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 0x55fb6d997090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49b6ffd640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49b00a3240 0x7f49b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49c9c52640 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d997a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49c9451640 1 -- 172.21.15.1:0/2864436241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 msgr2=0x55fb6d997a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:08.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49c9451640 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d997a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49c9451640 1 -- 172.21.15.1:0/2864436241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 msgr2=0x55fb6d997090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:08.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49c9451640 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 0x55fb6d997090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49c9451640 1 -- 172.21.15.1:0/2864436241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6d8893a0 con 0x55fb6d9a2370 2024-09-16T00:32:08.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49ca453640 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 0x55fb6d997090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:08.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49ca453640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49b00a3240 0x7f49b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.085+0000 7f49b6ffd640 1 -- 172.21.15.1:0/2760580328 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f49c00f0380 con 0x7f49c4100b20 2024-09-16T00:32:08.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.086+0000 7f49c9451640 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9a2370 0x55fb6d9a4760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f49b800bc30 tx=0x7f49b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a26f9b5e138f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:08.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.086+0000 7f49affff640 1 -- 172.21.15.1:0/2864436241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8004030 con 0x55fb6d9a2370 2024-09-16T00:32:08.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.086+0000 7f49affff640 1 -- 172.21.15.1:0/2864436241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b80041d0 con 0x55fb6d9a2370 2024-09-16T00:32:08.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.086+0000 7f49affff640 1 -- 172.21.15.1:0/2864436241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80116e0 con 0x55fb6d9a2370 2024-09-16T00:32:08.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.086+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864436241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9a2370 msgr2=0x55fb6d9a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.086+0000 7f49ca453640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49b00a3240 0x7f49b00a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f49c4069760 tx=0x7f49bc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9a2370 0x55fb6d9a4760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f49b800bc30 tx=0x7f49b80107a0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864436241 shutdown_connections 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9a2370 0x55fb6d9a4760 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d997a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 --2- 172.21.15.1:0/2864436241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 0x55fb6d997090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864436241 >> 172.21.15.1:0/2864436241 conn(0x55fb6d960310 msgr2=0x55fb6d993c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864436241 shutdown_connections 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864436241 wait complete. 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 -- start start 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.087+0000 7f49ce28f740 1 Processor -- start 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d9a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ce28f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fb6d9a5070 con 0x55fb6d998b40 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ce28f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb6d8b8060 con 0x55fb6d9a2370 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ce28f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb6d9a51f0 con 0x55fb6d9975d0 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ce28f740 10 client.0 _ll_register_callbacks cb 0x55fb6d60b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9451640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9451640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48832/0 (socket says 172.21.15.1:48832) 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9451640 1 -- 172.21.15.1:0/974646229 learned_addr learned my addr 172.21.15.1:0/974646229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9c52640 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d9a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ca453640 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9451640 1 -- 172.21.15.1:0/974646229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 msgr2=0x55fb6d9a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9451640 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d9a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9451640 1 -- 172.21.15.1:0/974646229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 msgr2=0x55fb6d99f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9451640 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9451640 1 -- 172.21.15.1:0/974646229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6d66e3c0 con 0x55fb6d998b40 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ca453640 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49c9451640 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99ef80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f49b8011010 tx=0x7f49b800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469e97de118fcf35 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ad7fa640 1 -- 172.21.15.1:0/974646229 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b802a060 con 0x55fb6d998b40 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ad7fa640 1 -- 172.21.15.1:0/974646229 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f49b8005040 con 0x55fb6d998b40 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.088+0000 7f49ce28f740 1 -- 172.21.15.1:0/974646229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b80041d0 con 0x55fb6d998b40 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ad7fa640 1 -- 172.21.15.1:0/974646229 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f49b8031050 con 0x55fb6d998b40 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ce28f740 10 client.16140 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ce28f740 1 -- 172.21.15.1:0/974646229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb6d9b0ae0 con 0x55fb6d998b40 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ad7fa640 1 -- 172.21.15.1:0/974646229 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b80041d0 con 0x55fb6d998b40 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ad7fa640 1 -- 172.21.15.1:0/974646229 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f49b8004880 con 0x55fb6d998b40 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ce28f740 10 client.16140 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ce28f740 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ce28f740 20 client.16140 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:08.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ce28f740 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55fb6d9b1e30 0x55fb6d9b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ce28f740 4 client.16140 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49ce28f740 1 -- 172.21.15.1:0/974646229 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb6d90a860 con 0x55fb6d9b1e30 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.089+0000 7f49c9c52640 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55fb6d9b1e30 0x55fb6d9b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.090+0000 7f49ad7fa640 10 client.16140 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.090+0000 7f49c9c52640 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55fb6d9b1e30 0x55fb6d9b4220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2d3b3eb0646078d server_cookie=d33d70de8087e4fc in_seq=0 out_seq=0 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.090+0000 7f49ad7fa640 1 -- 172.21.15.1:0/974646229 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb6d90a860 con 0x55fb6d9b1e30 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.090+0000 7f49ad7fa640 10 client.16140 handle_command_reply: tid=0 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.090+0000 7f49ce28f740 1 client.16140 shutdown 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.090+0000 7f49ce28f740 1 -- 172.21.15.1:0/974646229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 msgr2=0x55fb6d99ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.090+0000 7f49ce28f740 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99ef80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f49b8011010 tx=0x7f49b800bde0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ce28f740 1 -- 172.21.15.1:0/974646229 shutdown_connections 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ce28f740 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55fb6d9b1e30 0x55fb6d9b4220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ce28f740 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ad7fa640 0 client.16140 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:32:08.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ce28f740 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99ef80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ce28f740 1 --2- 172.21.15.1:0/974646229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d9a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ce28f740 1 -- 172.21.15.1:0/974646229 >> 172.21.15.1:0/974646229 conn(0x55fb6d960310 msgr2=0x55fb6d999960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:08.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ce28f740 1 -- 172.21.15.1:0/974646229 shutdown_connections 2024-09-16T00:32:08.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ce28f740 1 -- 172.21.15.1:0/974646229 wait complete. 2024-09-16T00:32:08.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.091+0000 7f49ce28f740 20 client.16140 trim_cache size 0 max 16384 2024-09-16T00:32:08.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49ce28f740 1 Processor -- start 2024-09-16T00:32:08.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49ce28f740 1 -- start start 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9975d0 0x55fb6d9ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 0x55fb6d99dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9a2370 0x55fb6d9b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49ce28f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41b38d0 con 0x55fb6d9975d0 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49ce28f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41b11a0 con 0x55fb6d998b40 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49ce28f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41aeff0 con 0x55fb6d9a2370 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49c9c52640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9975d0 0x55fb6d9ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49c9c52640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9975d0 0x55fb6d9ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48848/0 (socket says 172.21.15.1:48848) 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49c9c52640 1 -- 172.21.15.1:0/2864025754 learned_addr learned my addr 172.21.15.1:0/2864025754 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49ca453640 1 --2- 172.21.15.1:0/2864025754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9a2370 0x55fb6d9b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.104+0000 7f49c9451640 1 --2- 172.21.15.1:0/2864025754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 0x55fb6d99dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49c9c52640 1 -- 172.21.15.1:0/2864025754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9a2370 msgr2=0x55fb6d9b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49c9c52640 1 --2- 172.21.15.1:0/2864025754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9a2370 0x55fb6d9b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49c9c52640 1 -- 172.21.15.1:0/2864025754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 msgr2=0x55fb6d99dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49c9c52640 1 --2- 172.21.15.1:0/2864025754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 0x55fb6d99dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49c9c52640 1 -- 172.21.15.1:0/2864025754 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6d9934f0 con 0x55fb6d9975d0 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49c9c52640 1 --2- 172.21.15.1:0/2864025754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9975d0 0x55fb6d9ae370 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f49c00eda20 tx=0x7f49c002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5ffba16f863945 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49af7fe640 1 -- 172.21.15.1:0/2864025754 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0049040 con 0x55fb6d9975d0 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49af7fe640 1 -- 172.21.15.1:0/2864025754 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c00dc040 con 0x55fb6d9975d0 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49af7fe640 1 -- 172.21.15.1:0/2864025754 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00c09b0 con 0x55fb6d9975d0 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.105+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864025754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9975d0 msgr2=0x55fb6d9ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 --2- 172.21.15.1:0/2864025754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9975d0 0x55fb6d9ae370 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f49c00eda20 tx=0x7f49c002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864025754 shutdown_connections 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 --2- 172.21.15.1:0/2864025754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9a2370 0x55fb6d9b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 --2- 172.21.15.1:0/2864025754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d998b40 0x55fb6d99dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 --2- 172.21.15.1:0/2864025754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d9975d0 0x55fb6d9ae370 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864025754 >> 172.21.15.1:0/2864025754 conn(0x55fb6d960310 msgr2=0x55fb6d99d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:08.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864025754 shutdown_connections 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 -- 172.21.15.1:0/2864025754 wait complete. 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 -- start start 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 Processor -- start 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d9aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.106+0000 7f49ce28f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ce28f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41af660 con 0x55fb6d998b40 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ce28f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41b0ad0 con 0x55fb6d9a2370 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ce28f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41afd30 con 0x55fb6d9975d0 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ce28f740 10 client.0 _ll_register_callbacks cb 0x7f49bc009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9451640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9451640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48856/0 (socket says 172.21.15.1:48856) 2024-09-16T00:32:08.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9451640 1 -- 172.21.15.1:0/150301860 learned_addr learned my addr 172.21.15.1:0/150301860 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9c52640 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d9aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ca453640 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9451640 1 -- 172.21.15.1:0/150301860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 msgr2=0x55fb6d9aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9451640 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d9aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9451640 1 -- 172.21.15.1:0/150301860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 msgr2=0x55fb6d99bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9451640 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9451640 1 -- 172.21.15.1:0/150301860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6d99c420 con 0x55fb6d998b40 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ca453640 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49c9451640 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99b9a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x55fb6d9a05c0 tx=0x7f49b800bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f1e055b1ba48db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ad7fa640 1 -- 172.21.15.1:0/150301860 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b802a050 con 0x55fb6d998b40 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ad7fa640 1 -- 172.21.15.1:0/150301860 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f49b80047a0 con 0x55fb6d998b40 2024-09-16T00:32:08.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.107+0000 7f49ce28f740 1 -- 172.21.15.1:0/150301860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49c00dc040 con 0x55fb6d998b40 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ad7fa640 1 -- 172.21.15.1:0/150301860 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f49b80041d0 con 0x55fb6d998b40 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ce28f740 10 client.16146 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ce28f740 1 -- 172.21.15.1:0/150301860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb6da1c4d0 con 0x55fb6d998b40 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ad7fa640 1 -- 172.21.15.1:0/150301860 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c00dc040 con 0x55fb6d998b40 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ad7fa640 1 -- 172.21.15.1:0/150301860 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f49b8011680 con 0x55fb6d998b40 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ce28f740 10 client.16146 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ce28f740 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ce28f740 20 client.16146 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ce28f740 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55fb6da1cbe0 0x55fb6da1f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ce28f740 4 client.16146 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49ce28f740 1 -- 172.21.15.1:0/150301860 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb6d904d70 con 0x55fb6da1cbe0 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.108+0000 7f49c9c52640 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55fb6da1cbe0 0x55fb6da1f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49c9c52640 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55fb6da1cbe0 0x55fb6da1f000 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a11d7c7e293d28e0 server_cookie=2afc245cf37ec946 in_seq=0 out_seq=0 2024-09-16T00:32:08.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ad7fa640 10 client.16146 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ad7fa640 1 -- 172.21.15.1:0/150301860 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb6d904d70 con 0x55fb6da1cbe0 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ad7fa640 10 client.16146 handle_command_reply: tid=0 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 client.16146 shutdown 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 -- 172.21.15.1:0/150301860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 msgr2=0x55fb6d99b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99b9a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x55fb6d9a05c0 tx=0x7f49b800bec0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 -- 172.21.15.1:0/150301860 shutdown_connections 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55fb6da1cbe0 0x55fb6da1f000 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb6d9a2370 0x55fb6d99bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fb6d998b40 0x55fb6d99b9a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 --2- 172.21.15.1:0/150301860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb6d9975d0 0x55fb6d9aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ad7fa640 0 client.16146 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 -- 172.21.15.1:0/150301860 >> 172.21.15.1:0/150301860 conn(0x55fb6d960310 msgr2=0x55fb6d99e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 -- 172.21.15.1:0/150301860 shutdown_connections 2024-09-16T00:32:08.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 1 -- 172.21.15.1:0/150301860 wait complete. 2024-09-16T00:32:08.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.109+0000 7f49ce28f740 20 client.16146 trim_cache size 0 max 16384 2024-09-16T00:32:08.124 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 -- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49b00a3240 msgr2=0x7f49b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49b00a3240 0x7f49b00a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f49c4069760 tx=0x7f49bc004a40 comp rx=0 tx=0).stop 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 -- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 msgr2=0x7f49c41a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 0x7f49c41a0410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006010 tx=0x7f49c00042f0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 -- 172.21.15.1:0/2760580328 shutdown_connections 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49b00a3240 0x7f49b00a5700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c410e500 0x7f49c419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c4101440 0x7f49c41a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 --2- 172.21.15.1:0/2760580328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4100b20 0x7f49c41a0410 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 -- 172.21.15.1:0/2760580328 >> 172.21.15.1:0/2760580328 conn(0x7f49c40fc7d0 msgr2=0x7f49c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:08.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 -- 172.21.15.1:0/2760580328 shutdown_connections 2024-09-16T00:32:08.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:08.110+0000 7f49cbeb6640 1 -- 172.21.15.1:0/2760580328 wait complete. 2024-09-16T00:32:08.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:08 smithi001 ceph-mon[30934]: pgmap v462: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:32:08.508 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:08 smithi019 ceph-mon[35186]: pgmap v462: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:32:08.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:08 smithi153 ceph-mon[35538]: pgmap v462: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:32:08.973 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.747+0000 7f9997b7b640 1 -- 172.21.15.1:0/3124450648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990101420 msgr2=0x7f999010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.747+0000 7f9997b7b640 1 --2- 172.21.15.1:0/3124450648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990101420 0x7f999010d950 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9988009cd0 tx=0x7f998802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.747+0000 7f9997b7b640 1 -- 172.21.15.1:0/3124450648 shutdown_connections 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.747+0000 7f9997b7b640 1 --2- 172.21.15.1:0/3124450648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990101420 0x7f999010d950 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.747+0000 7f9997b7b640 1 --2- 172.21.15.1:0/3124450648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9990100a80 0x7f9990100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.747+0000 7f9997b7b640 1 --2- 172.21.15.1:0/3124450648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9990106a80 0x7f9990106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.747+0000 7f9997b7b640 1 -- 172.21.15.1:0/3124450648 >> 172.21.15.1:0/3124450648 conn(0x7f99900fc7a0 msgr2=0x7f99900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.747+0000 7f9997b7b640 1 -- 172.21.15.1:0/3124450648 shutdown_connections 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.747+0000 7f9997b7b640 1 -- 172.21.15.1:0/3124450648 wait complete. 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9997b7b640 1 Processor -- start 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9997b7b640 1 -- start start 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9997b7b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990100a80 0x7f999019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9997b7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9990101420 0x7f999019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9997b7b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9990106a80 0x7f999019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9997b7b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9990110f20 con 0x7f9990100a80 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9997b7b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9990110da0 con 0x7f9990106a80 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9997b7b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99901110a0 con 0x7f9990101420 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9995917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990100a80 0x7f999019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9995917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990100a80 0x7f999019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48886/0 (socket says 172.21.15.1:48886) 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9995917640 1 -- 172.21.15.1:0/2864732003 learned_addr learned my addr 172.21.15.1:0/2864732003 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9995116640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9990101420 0x7f999019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9996118640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9990106a80 0x7f999019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9995917640 1 -- 172.21.15.1:0/2864732003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9990101420 msgr2=0x7f999019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9995917640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9990101420 0x7f999019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9995917640 1 -- 172.21.15.1:0/2864732003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9990106a80 msgr2=0x7f999019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9995917640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9990106a80 0x7f999019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.748+0000 7f9995917640 1 -- 172.21.15.1:0/2864732003 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999019f6f0 con 0x7f9990100a80 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.749+0000 7f9995917640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990100a80 0x7f999019a740 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f998c00b4a0 tx=0x7f998c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.749+0000 7f9986ffd640 1 -- 172.21.15.1:0/2864732003 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c014070 con 0x7f9990100a80 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.749+0000 7f9986ffd640 1 -- 172.21.15.1:0/2864732003 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f998c010af0 con 0x7f9990100a80 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.749+0000 7f9997b7b640 1 -- 172.21.15.1:0/2864732003 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999019f9e0 con 0x7f9990100a80 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.749+0000 7f9986ffd640 1 -- 172.21.15.1:0/2864732003 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c017d10 con 0x7f9990100a80 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.749+0000 7f9997b7b640 1 -- 172.21.15.1:0/2864732003 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99901a72c0 con 0x7f9990100a80 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9986ffd640 1 -- 172.21.15.1:0/2864732003 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f998c017070 con 0x7f9990100a80 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9999f54740 1 Processor -- start 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9999f54740 1 -- start start 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9999f54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769ecb40 0x55f0769eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9999f54740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769eba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9999f54740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769f6370 0x55f0769f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9999f54740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f0769512f0 con 0x55f0769f6370 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9999f54740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f0768f8cf0 con 0x55f0769eb5d0 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9999f54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f07694d2b0 con 0x55f0769ecb40 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.752+0000 7f9995116640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769f6370 0x55f0769f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9996118640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769ecb40 0x55f0769eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9995116640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769f6370 0x55f0769f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48896/0 (socket says 172.21.15.1:48896) 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9995116640 1 -- 172.21.15.1:0/2815895447 learned_addr learned my addr 172.21.15.1:0/2815895447 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9995917640 1 --2- 172.21.15.1:0/2815895447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9986ffd640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f997c0a32c0 0x7f997c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9995116640 1 -- 172.21.15.1:0/2815895447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769ecb40 msgr2=0x55f0769eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9995116640 1 --2- 172.21.15.1:0/2815895447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769ecb40 0x55f0769eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9995116640 1 -- 172.21.15.1:0/2815895447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 msgr2=0x55f0769eba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9995116640 1 --2- 172.21.15.1:0/2815895447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769eba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9995116640 1 -- 172.21.15.1:0/2815895447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0768dd3a0 con 0x55f0769f6370 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9996118640 1 --2- 172.21.15.1:0/2815895447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769ecb40 0x55f0769eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9996118640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f997c0a32c0 0x7f997c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9986ffd640 1 -- 172.21.15.1:0/2864732003 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f998c0cf0b0 con 0x7f9990100a80 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9995116640 1 --2- 172.21.15.1:0/2815895447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769f6370 0x55f0769f8760 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f998000b7f0 tx=0x7f998000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d257b43b2e97b24d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f997bfff640 1 -- 172.21.15.1:0/2815895447 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998000d950 con 0x55f0769f6370 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.753+0000 7f9996118640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f997c0a32c0 0x7f997c0a5780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9988006010 tx=0x7f99880095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f997bfff640 1 -- 172.21.15.1:0/2815895447 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9980018070 con 0x55f0769f6370 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f997bfff640 1 -- 172.21.15.1:0/2815895447 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800123e0 con 0x55f0769f6370 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f9999f54740 1 -- 172.21.15.1:0/2815895447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769f6370 msgr2=0x55f0769f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f9999f54740 1 --2- 172.21.15.1:0/2815895447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769f6370 0x55f0769f8760 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f998000b7f0 tx=0x7f998000bcb0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f9999f54740 1 -- 172.21.15.1:0/2815895447 shutdown_connections 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f9999f54740 1 --2- 172.21.15.1:0/2815895447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769f6370 0x55f0769f8760 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f9999f54740 1 --2- 172.21.15.1:0/2815895447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f9999f54740 1 --2- 172.21.15.1:0/2815895447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769ecb40 0x55f0769eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f9999f54740 1 -- 172.21.15.1:0/2815895447 >> 172.21.15.1:0/2815895447 conn(0x55f0769b4310 msgr2=0x55f0769e7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f9999f54740 1 -- 172.21.15.1:0/2815895447 shutdown_connections 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.754+0000 7f9999f54740 1 -- 172.21.15.1:0/2815895447 wait complete. 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.755+0000 7f9999f54740 1 -- start start 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.755+0000 7f9999f54740 1 Processor -- start 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.755+0000 7f9999f54740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.755+0000 7f9999f54740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.755+0000 7f9999f54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f0769f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.755+0000 7f9999f54740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f0769f9070 con 0x55f0769ecb40 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.755+0000 7f9999f54740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f07690c060 con 0x55f0769eb5d0 2024-09-16T00:32:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.755+0000 7f9999f54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f0769f91f0 con 0x55f0769f6370 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.755+0000 7f9999f54740 10 client.0 _ll_register_callbacks cb 0x55f07665f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995116640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995116640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48912/0 (socket says 172.21.15.1:48912) 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995116640 1 -- 172.21.15.1:0/1974584586 learned_addr learned my addr 172.21.15.1:0/1974584586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995917640 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9996118640 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f0769f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995116640 1 -- 172.21.15.1:0/1974584586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 msgr2=0x55f0769f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995116640 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f0769f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995116640 1 -- 172.21.15.1:0/1974584586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 msgr2=0x55f0769f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995116640 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995116640 1 -- 172.21.15.1:0/1974584586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0766c23c0 con 0x55f0769ecb40 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995917640 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9995116640 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f2f80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9980013b30 tx=0x7f9980013730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb91c509affa9504 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f99797fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f99797fa640 1 -- 172.21.15.1:0/1974584586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980023070 con 0x55f0769ecb40 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f99797fa640 1 -- 172.21.15.1:0/1974584586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f998000dd60 con 0x55f0769ecb40 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.756+0000 7f9999f54740 1 -- 172.21.15.1:0/1974584586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9980018070 con 0x55f0769ecb40 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f99797fa640 1 -- 172.21.15.1:0/1974584586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f998002f050 con 0x55f0769ecb40 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f9999f54740 10 client.16158 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f9999f54740 1 -- 172.21.15.1:0/1974584586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f076a04a80 con 0x55f0769ecb40 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f99797fa640 1 -- 172.21.15.1:0/1974584586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9980018070 con 0x55f0769ecb40 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f99797fa640 1 -- 172.21.15.1:0/1974584586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f998001b380 con 0x55f0769ecb40 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f9999f54740 10 client.16158 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f9999f54740 10 client.16158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f9999f54740 20 client.16158 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f9999f54740 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f076a05dd0 0x55f076a081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f9999f54740 4 client.16158 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:09.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f9999f54740 1 -- 172.21.15.1:0/1974584586 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f07695e860 con 0x55f076a05dd0 2024-09-16T00:32:09.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.757+0000 7f9995917640 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f076a05dd0 0x55f076a081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.758+0000 7f9995917640 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f076a05dd0 0x55f076a081c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d1f977a35a6ef71 server_cookie=2319d9dbdff4f340 in_seq=0 out_seq=0 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.758+0000 7f99797fa640 10 client.16158 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.758+0000 7f99797fa640 1 -- 172.21.15.1:0/1974584586 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f07695e860 con 0x55f076a05dd0 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.758+0000 7f99797fa640 10 client.16158 handle_command_reply: tid=0 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 client.16158 shutdown 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 -- 172.21.15.1:0/1974584586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 msgr2=0x55f0769f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f2f80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9980013b30 tx=0x7f9980013730 comp rx=0 tx=0).stop 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 -- 172.21.15.1:0/1974584586 shutdown_connections 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f0769f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f076a05dd0 0x55f076a081c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f2f80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 --2- 172.21.15.1:0/1974584586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f99797fa640 0 client.16158 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:32:09.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 -- 172.21.15.1:0/1974584586 >> 172.21.15.1:0/1974584586 conn(0x55f0769b4310 msgr2=0x55f0769ed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:09.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 -- 172.21.15.1:0/1974584586 shutdown_connections 2024-09-16T00:32:09.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 1 -- 172.21.15.1:0/1974584586 wait complete. 2024-09-16T00:32:09.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.759+0000 7f9999f54740 20 client.16158 trim_cache size 0 max 16384 2024-09-16T00:32:09.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.772+0000 7f9999f54740 1 Processor -- start 2024-09-16T00:32:09.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.772+0000 7f9999f54740 1 -- start start 2024-09-16T00:32:09.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9999f54740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769eb5d0 0x55f076a02310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9999f54740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769ecb40 0x55f0769f2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9999f54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f076a0d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9999f54740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99901b3a70 con 0x55f0769eb5d0 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9999f54740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99901b1340 con 0x55f0769ecb40 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9999f54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99901af190 con 0x55f0769f6370 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769eb5d0 0x55f076a02310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769eb5d0 0x55f076a02310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48924/0 (socket says 172.21.15.1:48924) 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995917640 1 -- 172.21.15.1:0/1706531503 learned_addr learned my addr 172.21.15.1:0/1706531503 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995116640 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769ecb40 0x55f0769f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995917640 1 -- 172.21.15.1:0/1706531503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 msgr2=0x55f076a0d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9996118640 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f076a0d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995917640 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f076a0d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995917640 1 -- 172.21.15.1:0/1706531503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769ecb40 msgr2=0x55f0769f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995917640 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769ecb40 0x55f0769f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995917640 1 -- 172.21.15.1:0/1706531503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0769f14d0 con 0x55f0769eb5d0 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995116640 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769ecb40 0x55f0769f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9996118640 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f076a0d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f9995917640 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769eb5d0 0x55f076a02310 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f998c009420 tx=0x7f998c0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7e1f65d286b2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f997b7fe640 1 -- 172.21.15.1:0/1706531503 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c01e040 con 0x55f0769eb5d0 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f997b7fe640 1 -- 172.21.15.1:0/1706531503 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f998c09b9d0 con 0x55f0769eb5d0 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.773+0000 7f997b7fe640 1 -- 172.21.15.1:0/1706531503 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c017360 con 0x55f0769eb5d0 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 -- 172.21.15.1:0/1706531503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769eb5d0 msgr2=0x55f076a02310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769eb5d0 0x55f076a02310 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f998c009420 tx=0x7f998c0bb040 comp rx=0 tx=0).stop 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 -- 172.21.15.1:0/1706531503 shutdown_connections 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f076a0d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769ecb40 0x55f0769f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 --2- 172.21.15.1:0/1706531503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769eb5d0 0x55f076a02310 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 -- 172.21.15.1:0/1706531503 >> 172.21.15.1:0/1706531503 conn(0x55f0769b4310 msgr2=0x55f0769e2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:09.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 -- 172.21.15.1:0/1706531503 shutdown_connections 2024-09-16T00:32:09.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 -- 172.21.15.1:0/1706531503 wait complete. 2024-09-16T00:32:09.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.774+0000 7f9999f54740 1 -- start start 2024-09-16T00:32:09.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9999f54740 1 Processor -- start 2024-09-16T00:32:09.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9999f54740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9999f54740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9999f54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f076a6e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9999f54740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99901af800 con 0x55f0769ecb40 2024-09-16T00:32:09.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9999f54740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99901b0c70 con 0x55f0769eb5d0 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9999f54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99901afed0 con 0x55f0769f6370 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9999f54740 10 client.0 _ll_register_callbacks cb 0x7f998c0d2a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9995116640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9995116640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48926/0 (socket says 172.21.15.1:48926) 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9995116640 1 -- 172.21.15.1:0/1180487570 learned_addr learned my addr 172.21.15.1:0/1180487570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9995917640 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9995116640 1 -- 172.21.15.1:0/1180487570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 msgr2=0x55f076a6e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9996118640 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f076a6e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9995116640 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f076a6e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.775+0000 7f9995116640 1 -- 172.21.15.1:0/1180487570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 msgr2=0x55f0769f99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f9995116640 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f9995116640 1 -- 172.21.15.1:0/1180487570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f076a6ee60 con 0x55f0769ecb40 2024-09-16T00:32:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f9995917640 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f9995116640 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f9f20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9980000c00 tx=0x7f9980013a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219f3a4c32dc63c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f99797fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f99797fa640 1 -- 172.21.15.1:0/1180487570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980020020 con 0x55f0769ecb40 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f99797fa640 1 -- 172.21.15.1:0/1180487570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f998001bbc0 con 0x55f0769ecb40 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f9999f54740 1 -- 172.21.15.1:0/1180487570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f998c09b9d0 con 0x55f0769ecb40 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f99797fa640 1 -- 172.21.15.1:0/1180487570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9980023040 con 0x55f0769ecb40 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f9999f54740 10 client.16164 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f9999f54740 1 -- 172.21.15.1:0/1180487570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f076a700b0 con 0x55f0769ecb40 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f99797fa640 1 -- 172.21.15.1:0/1180487570 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f998c09b9d0 con 0x55f0769ecb40 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f99797fa640 1 -- 172.21.15.1:0/1180487570 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f998000dc40 con 0x55f0769ecb40 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.776+0000 7f9999f54740 10 client.16164 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f9999f54740 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f9999f54740 20 client.16164 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f9999f54740 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f076a70830 0x55f076a72c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f9999f54740 4 client.16164 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f9999f54740 1 -- 172.21.15.1:0/1180487570 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f076958d70 con 0x55f076a70830 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f9995917640 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f076a70830 0x55f076a72c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f9995917640 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f076a70830 0x55f076a72c50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4628d21b2b53655 server_cookie=666f13a6732b92f2 in_seq=0 out_seq=0 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f99797fa640 10 client.16164 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f99797fa640 1 -- 172.21.15.1:0/1180487570 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f076958d70 con 0x55f076a70830 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f99797fa640 10 client.16164 handle_command_reply: tid=0 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.777+0000 7f9999f54740 1 client.16164 shutdown 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 -- 172.21.15.1:0/1180487570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 msgr2=0x55f0769f9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f9f20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9980000c00 tx=0x7f9980013a90 comp rx=0 tx=0).stop 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 -- 172.21.15.1:0/1180487570 shutdown_connections 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0769f6370 0x55f076a6e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f076a70830 0x55f076a72c50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0769ecb40 0x55f0769f9f20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 --2- 172.21.15.1:0/1180487570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0769eb5d0 0x55f0769f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f99797fa640 0 client.16164 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:32:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 -- 172.21.15.1:0/1180487570 >> 172.21.15.1:0/1180487570 conn(0x55f0769b4310 msgr2=0x55f0769e64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 -- 172.21.15.1:0/1180487570 shutdown_connections 2024-09-16T00:32:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 1 -- 172.21.15.1:0/1180487570 wait complete. 2024-09-16T00:32:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9999f54740 20 client.16164 trim_cache size 0 max 16384 2024-09-16T00:32:09.784 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:32:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9997b7b640 1 -- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f997c0a32c0 msgr2=0x7f997c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9997b7b640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f997c0a32c0 0x7f997c0a5780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9988006010 tx=0x7f99880095c0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9997b7b640 1 -- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990100a80 msgr2=0x7f999019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.778+0000 7f9997b7b640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990100a80 0x7f999019a740 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f998c00b4a0 tx=0x7f998c00b960 comp rx=0 tx=0).stop 2024-09-16T00:32:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.779+0000 7f9997b7b640 1 -- 172.21.15.1:0/2864732003 shutdown_connections 2024-09-16T00:32:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.779+0000 7f9997b7b640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9990106a80 0x7f999019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.779+0000 7f9997b7b640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f997c0a32c0 0x7f997c0a5780 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.779+0000 7f9997b7b640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9990101420 0x7f999019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.779+0000 7f9997b7b640 1 --2- 172.21.15.1:0/2864732003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9990100a80 0x7f999019a740 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.779+0000 7f9997b7b640 1 -- 172.21.15.1:0/2864732003 >> 172.21.15.1:0/2864732003 conn(0x7f99900fc7a0 msgr2=0x7f999006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.779+0000 7f9997b7b640 1 -- 172.21.15.1:0/2864732003 shutdown_connections 2024-09-16T00:32:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:09.779+0000 7f9997b7b640 1 -- 172.21.15.1:0/2864732003 wait complete. 2024-09-16T00:32:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:10 smithi019 ceph-mon[35186]: pgmap v463: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:32:10.611 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:10 smithi001 ceph-mon[30934]: pgmap v463: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:32:10.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:32:10.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:10 smithi153 ceph-mon[35538]: pgmap v463: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:32:10.895 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:11.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 -- 172.21.15.1:0/1423999239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0100e20 msgr2=0x7fadb0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:11.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 --2- 172.21.15.1:0/1423999239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0100e20 0x7fadb0101200 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fada0009900 tx=0x7fada002a480 comp rx=0 tx=0).stop 2024-09-16T00:32:11.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 -- 172.21.15.1:0/1423999239 shutdown_connections 2024-09-16T00:32:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 --2- 172.21.15.1:0/1423999239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb0102ef0 0x7fadb010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 --2- 172.21.15.1:0/1423999239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb0102570 0x7fadb01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 --2- 172.21.15.1:0/1423999239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0100e20 0x7fadb0101200 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 -- 172.21.15.1:0/1423999239 >> 172.21.15.1:0/1423999239 conn(0x7fadb00fc7d0 msgr2=0x7fadb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 -- 172.21.15.1:0/1423999239 shutdown_connections 2024-09-16T00:32:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 -- 172.21.15.1:0/1423999239 wait complete. 2024-09-16T00:32:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 Processor -- start 2024-09-16T00:32:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.416+0000 7fadb697e640 1 -- start start 2024-09-16T00:32:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadb697e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb0100e20 0x7fadb01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadb697e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0102570 0x7fadb0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadb697e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb0102ef0 0x7fadb019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadb697e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01120a0 con 0x7fadb0102570 2024-09-16T00:32:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadb697e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0111f20 con 0x7fadb0100e20 2024-09-16T00:32:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadb697e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0112220 con 0x7fadb0102ef0 2024-09-16T00:32:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadaf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0102570 0x7fadb0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadaf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0102570 0x7fadb0198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58416/0 (socket says 172.21.15.1:58416) 2024-09-16T00:32:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadaf7fe640 1 -- 172.21.15.1:0/2235014197 learned_addr learned my addr 172.21.15.1:0/2235014197 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadaffff640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb0100e20 0x7fadb01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadb4f1b640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb0102ef0 0x7fadb019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadaf7fe640 1 -- 172.21.15.1:0/2235014197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb0102ef0 msgr2=0x7fadb019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb0102ef0 0x7fadb019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadaf7fe640 1 -- 172.21.15.1:0/2235014197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb0100e20 msgr2=0x7fadb01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:11.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb0100e20 0x7fadb01984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadaf7fe640 1 -- 172.21.15.1:0/2235014197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb019d4a0 con 0x7fadb0102570 2024-09-16T00:32:11.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.417+0000 7fadb4f1b640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb0102ef0 0x7fadb019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:11.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.418+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0102570 0x7fadb0198a30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fada400d770 tx=0x7fada400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:11.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.418+0000 7fadad7fa640 1 -- 172.21.15.1:0/2235014197 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4017070 con 0x7fadb0102570 2024-09-16T00:32:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.418+0000 7fadb697e640 1 -- 172.21.15.1:0/2235014197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb019d790 con 0x7fadb0102570 2024-09-16T00:32:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.418+0000 7fadad7fa640 1 -- 172.21.15.1:0/2235014197 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada4005500 con 0x7fadb0102570 2024-09-16T00:32:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.418+0000 7fadad7fa640 1 -- 172.21.15.1:0/2235014197 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4004e00 con 0x7fadb0102570 2024-09-16T00:32:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.418+0000 7fadb697e640 1 -- 172.21.15.1:0/2235014197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb01a5070 con 0x7fadb0102570 2024-09-16T00:32:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.420+0000 7fadad7fa640 1 -- 172.21.15.1:0/2235014197 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fada4004fa0 con 0x7fadb0102570 2024-09-16T00:32:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb8d57740 1 Processor -- start 2024-09-16T00:32:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb8d57740 1 -- start start 2024-09-16T00:32:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619db40 0x5634c619c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 0x5634c61a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb8d57740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5634c61022f0 con 0x5634c619c5d0 2024-09-16T00:32:11.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb8d57740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5634c60a9cf0 con 0x5634c619db40 2024-09-16T00:32:11.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb8d57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634c60fe2b0 con 0x5634c61a7370 2024-09-16T00:32:11.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadaffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadad7fa640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad980a3240 0x7fad980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb4f1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619db40 0x5634c619c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadaffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58422/0 (socket says 172.21.15.1:58422) 2024-09-16T00:32:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadaffff640 1 -- 172.21.15.1:0/2627397428 learned_addr learned my addr 172.21.15.1:0/2627397428 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/2627397428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 0x5634c61a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadb4f1b640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad980a3240 0x7fad980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadad7fa640 1 -- 172.21.15.1:0/2235014197 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fada40cf790 con 0x7fadb0102570 2024-09-16T00:32:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadaffff640 1 -- 172.21.15.1:0/2627397428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 msgr2=0x5634c61a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadaffff640 1 --2- 172.21.15.1:0/2627397428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 0x5634c61a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadaffff640 1 -- 172.21.15.1:0/2627397428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619db40 msgr2=0x5634c619c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadaffff640 1 --2- 172.21.15.1:0/2627397428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619db40 0x5634c619c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.421+0000 7fadaffff640 1 -- 172.21.15.1:0/2627397428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c608e3a0 con 0x5634c619c5d0 2024-09-16T00:32:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.422+0000 7fadaffff640 1 --2- 172.21.15.1:0/2627397428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619ca10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fada002a9a0 tx=0x7fada00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95ae851192c6c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.422+0000 7fadb4f1b640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad980a3240 0x7fad980a5700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fada8005fd0 tx=0x7fada8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.422+0000 7fad967fc640 1 -- 172.21.15.1:0/2627397428 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0035030 con 0x5634c619c5d0 2024-09-16T00:32:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.422+0000 7fad967fc640 1 -- 172.21.15.1:0/2627397428 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada0040070 con 0x5634c619c5d0 2024-09-16T00:32:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.422+0000 7fad967fc640 1 -- 172.21.15.1:0/2627397428 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada00319d0 con 0x5634c619c5d0 2024-09-16T00:32:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 -- 172.21.15.1:0/2627397428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 msgr2=0x5634c619ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2627397428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619ca10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fada002a9a0 tx=0x7fada00040c0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 -- 172.21.15.1:0/2627397428 shutdown_connections 2024-09-16T00:32:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2627397428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 0x5634c61a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2627397428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619ca10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2627397428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619db40 0x5634c619c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 -- 172.21.15.1:0/2627397428 >> 172.21.15.1:0/2627397428 conn(0x5634c6165310 msgr2=0x5634c6198c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 -- 172.21.15.1:0/2627397428 shutdown_connections 2024-09-16T00:32:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 -- 172.21.15.1:0/2627397428 wait complete. 2024-09-16T00:32:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 -- start start 2024-09-16T00:32:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.423+0000 7fadb8d57740 1 Processor -- start 2024-09-16T00:32:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619c5d0 0x5634c61a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619db40 0x5634c61a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 0x5634c61a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadb8d57740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5634c61aa010 con 0x5634c619db40 2024-09-16T00:32:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadb8d57740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5634c60bd060 con 0x5634c619c5d0 2024-09-16T00:32:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadb8d57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634c61aa190 con 0x5634c61a7370 2024-09-16T00:32:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadb8d57740 10 client.0 _ll_register_callbacks cb 0x5634c5e108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619db40 0x5634c61a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619db40 0x5634c61a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58426/0 (socket says 172.21.15.1:58426) 2024-09-16T00:32:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaf7fe640 1 -- 172.21.15.1:0/3879822363 learned_addr learned my addr 172.21.15.1:0/3879822363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadb4f1b640 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 0x5634c61a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaffff640 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619c5d0 0x5634c61a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaf7fe640 1 -- 172.21.15.1:0/3879822363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 msgr2=0x5634c61a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 0x5634c61a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaf7fe640 1 -- 172.21.15.1:0/3879822363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619c5d0 msgr2=0x5634c61a6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619c5d0 0x5634c61a6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaf7fe640 1 -- 172.21.15.1:0/3879822363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c5e733c0 con 0x5634c619db40 2024-09-16T00:32:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaffff640 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619c5d0 0x5634c61a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619db40 0x5634c61a3fb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fada409b6b0 tx=0x7fada406b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bfce98b22d7c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fad8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fad8f7fe640 1 -- 172.21.15.1:0/3879822363 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4020070 con 0x5634c619db40 2024-09-16T00:32:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.424+0000 7fad8f7fe640 1 -- 172.21.15.1:0/3879822363 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fada40860b0 con 0x5634c619db40 2024-09-16T00:32:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadb8d57740 1 -- 172.21.15.1:0/3879822363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada0040070 con 0x5634c619db40 2024-09-16T00:32:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fad8f7fe640 1 -- 172.21.15.1:0/3879822363 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fada40046c0 con 0x5634c619db40 2024-09-16T00:32:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadb8d57740 10 client.16176 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadb8d57740 1 -- 172.21.15.1:0/3879822363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634c61b5a90 con 0x5634c619db40 2024-09-16T00:32:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fad8f7fe640 1 -- 172.21.15.1:0/3879822363 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada0040070 con 0x5634c619db40 2024-09-16T00:32:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fad8f7fe640 1 -- 172.21.15.1:0/3879822363 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fada409cce0 con 0x5634c619db40 2024-09-16T00:32:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadb8d57740 10 client.16176 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadb8d57740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadb8d57740 20 client.16176 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5634c61b6de0 0x5634c61b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadb8d57740 4 client.16176 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadb8d57740 1 -- 172.21.15.1:0/3879822363 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c610f860 con 0x5634c61b6de0 2024-09-16T00:32:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.425+0000 7fadaffff640 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5634c61b6de0 0x5634c61b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.426+0000 7fad8f7fe640 10 client.16176 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.426+0000 7fadaffff640 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5634c61b6de0 0x5634c61b91d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74688e42a706e65f server_cookie=a0b6d083818e428e in_seq=0 out_seq=0 2024-09-16T00:32:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.426+0000 7fad8f7fe640 1 -- 172.21.15.1:0/3879822363 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634c610f860 con 0x5634c61b6de0 2024-09-16T00:32:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.426+0000 7fad8f7fe640 10 client.16176 handle_command_reply: tid=0 2024-09-16T00:32:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.426+0000 7fadb8d57740 1 client.16176 shutdown 2024-09-16T00:32:11.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 -- 172.21.15.1:0/3879822363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619db40 msgr2=0x5634c61a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:11.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619db40 0x5634c61a3fb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fada409b6b0 tx=0x7fada406b620 comp rx=0 tx=0).stop 2024-09-16T00:32:11.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 -- 172.21.15.1:0/3879822363 shutdown_connections 2024-09-16T00:32:11.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5634c61b6de0 0x5634c61b91d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c61a7370 0x5634c61a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619db40 0x5634c61a3fb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fad8f7fe640 0 client.16176 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:11.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3879822363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c619c5d0 0x5634c61a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 -- 172.21.15.1:0/3879822363 >> 172.21.15.1:0/3879822363 conn(0x5634c6165310 msgr2=0x5634c619e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:11.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 -- 172.21.15.1:0/3879822363 shutdown_connections 2024-09-16T00:32:11.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 1 -- 172.21.15.1:0/3879822363 wait complete. 2024-09-16T00:32:11.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.427+0000 7fadb8d57740 20 client.16176 trim_cache size 0 max 16384 2024-09-16T00:32:11.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadb8d57740 1 Processor -- start 2024-09-16T00:32:11.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadb8d57740 1 -- start start 2024-09-16T00:32:11.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c619db40 0x5634c619f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c61a7370 0x5634c61beaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadb8d57740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01b1620 con 0x5634c619c5d0 2024-09-16T00:32:11.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadb8d57740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01aeef0 con 0x5634c61a7370 2024-09-16T00:32:11.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadb8d57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01acd40 con 0x5634c619db40 2024-09-16T00:32:11.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadaffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadaffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58442/0 (socket says 172.21.15.1:58442) 2024-09-16T00:32:11.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadaffff640 1 -- 172.21.15.1:0/3407726477 learned_addr learned my addr 172.21.15.1:0/3407726477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:11.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadaffff640 1 -- 172.21.15.1:0/3407726477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c619db40 msgr2=0x5634c619f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:11.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadaffff640 1 --2- 172.21.15.1:0/3407726477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c619db40 0x5634c619f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadaffff640 1 -- 172.21.15.1:0/3407726477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c61a7370 msgr2=0x5634c61beaa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:32:11.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadaffff640 1 --2- 172.21.15.1:0/3407726477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c61a7370 0x5634c61beaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.440+0000 7fadaffff640 1 -- 172.21.15.1:0/3407726477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c619fba0 con 0x5634c619c5d0 2024-09-16T00:32:11.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fadaffff640 1 --2- 172.21.15.1:0/3407726477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619f120 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fada002a9a0 tx=0x7fada0003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f102de6c267e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:11.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fad95ffb640 1 -- 172.21.15.1:0/3407726477 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0054070 con 0x5634c619c5d0 2024-09-16T00:32:11.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fad95ffb640 1 -- 172.21.15.1:0/3407726477 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada0040070 con 0x5634c619c5d0 2024-09-16T00:32:11.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fad95ffb640 1 -- 172.21.15.1:0/3407726477 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0035040 con 0x5634c619c5d0 2024-09-16T00:32:11.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fadb8d57740 1 -- 172.21.15.1:0/3407726477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 msgr2=0x5634c619f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:11.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3407726477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619f120 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fada002a9a0 tx=0x7fada0003780 comp rx=0 tx=0).stop 2024-09-16T00:32:11.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fadb8d57740 1 -- 172.21.15.1:0/3407726477 shutdown_connections 2024-09-16T00:32:11.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3407726477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c61a7370 0x5634c61beaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3407726477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c619db40 0x5634c619f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fadb8d57740 1 --2- 172.21.15.1:0/3407726477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c619f120 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.441+0000 7fadb8d57740 1 -- 172.21.15.1:0/3407726477 >> 172.21.15.1:0/3407726477 conn(0x5634c6165310 msgr2=0x5634c61a6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:11.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 -- 172.21.15.1:0/3407726477 shutdown_connections 2024-09-16T00:32:11.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 -- 172.21.15.1:0/3407726477 wait complete. 2024-09-16T00:32:11.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 -- start start 2024-09-16T00:32:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 Processor -- start 2024-09-16T00:32:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c61abd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c619db40 0x5634c61ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c61a7370 0x5634c6219a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01ad3b0 con 0x5634c619c5d0 2024-09-16T00:32:11.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01ae820 con 0x5634c61a7370 2024-09-16T00:32:11.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01ada80 con 0x5634c619db40 2024-09-16T00:32:11.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.442+0000 7fadb8d57740 10 client.0 _ll_register_callbacks cb 0x7fada400d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:11.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c61abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c61abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58446/0 (socket says 172.21.15.1:58446) 2024-09-16T00:32:11.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaffff640 1 -- 172.21.15.1:0/2303173110 learned_addr learned my addr 172.21.15.1:0/2303173110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:11.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadb4f1b640 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c61a7370 0x5634c6219a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c619db40 0x5634c61ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaffff640 1 -- 172.21.15.1:0/2303173110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c619db40 msgr2=0x5634c61ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:11.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaffff640 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c619db40 0x5634c61ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaffff640 1 -- 172.21.15.1:0/2303173110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c61a7370 msgr2=0x5634c6219a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:11.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaffff640 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c61a7370 0x5634c6219a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaffff640 1 -- 172.21.15.1:0/2303173110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c621a260 con 0x5634c619c5d0 2024-09-16T00:32:11.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadaffff640 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c61abd30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fada0009900 tx=0x7fada0004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f24e6a6a2c8b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:11.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fad8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:11.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fad8f7fe640 1 -- 172.21.15.1:0/2303173110 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0035070 con 0x5634c619c5d0 2024-09-16T00:32:11.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadb8d57740 1 -- 172.21.15.1:0/2303173110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada0040070 con 0x5634c619c5d0 2024-09-16T00:32:11.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fad8f7fe640 1 -- 172.21.15.1:0/2303173110 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fada0005230 con 0x5634c619c5d0 2024-09-16T00:32:11.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fad8f7fe640 1 -- 172.21.15.1:0/2303173110 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fada0054040 con 0x5634c619c5d0 2024-09-16T00:32:11.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadb8d57740 10 client.16182 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:11.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.443+0000 7fadb8d57740 1 -- 172.21.15.1:0/2303173110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634c62225d0 con 0x5634c619c5d0 2024-09-16T00:32:11.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.444+0000 7fad8f7fe640 1 -- 172.21.15.1:0/2303173110 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada0040070 con 0x5634c619c5d0 2024-09-16T00:32:11.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.444+0000 7fad8f7fe640 1 -- 172.21.15.1:0/2303173110 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fada0057070 con 0x5634c619c5d0 2024-09-16T00:32:11.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.444+0000 7fadb8d57740 10 client.16182 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:11.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.444+0000 7fadb8d57740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:11.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.444+0000 7fadb8d57740 20 client.16182 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:11.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.444+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5634c6222800 0x5634c6224bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:11.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.444+0000 7fadb8d57740 4 client.16182 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:32:11.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.444+0000 7fadb8d57740 1 -- 172.21.15.1:0/2303173110 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634c6109d70 con 0x5634c6222800 2024-09-16T00:32:11.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.444+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5634c6222800 0x5634c6224bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:11.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fad8f7fe640 10 client.16182 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:11.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadaf7fe640 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5634c6222800 0x5634c6224bf0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96147cf8dde1f077 server_cookie=695e07a313185271 in_seq=0 out_seq=0 2024-09-16T00:32:11.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fad8f7fe640 1 -- 172.21.15.1:0/2303173110 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634c6109d70 con 0x5634c6222800 2024-09-16T00:32:11.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fad8f7fe640 10 client.16182 handle_command_reply: tid=0 2024-09-16T00:32:11.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 client.16182 shutdown 2024-09-16T00:32:11.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 -- 172.21.15.1:0/2303173110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 msgr2=0x5634c61abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:11.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c61abd30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fada0009900 tx=0x7fada0004370 comp rx=0 tx=0).stop 2024-09-16T00:32:11.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 -- 172.21.15.1:0/2303173110 shutdown_connections 2024-09-16T00:32:11.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5634c6222800 0x5634c6224bf0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634c61a7370 0x5634c6219a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634c619db40 0x5634c61ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 --2- 172.21.15.1:0/2303173110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634c619c5d0 0x5634c61abd30 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 -- 172.21.15.1:0/2303173110 >> 172.21.15.1:0/2303173110 conn(0x5634c6165310 msgr2=0x5634c61a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:11.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fad8f7fe640 0 client.16182 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:11.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 -- 172.21.15.1:0/2303173110 shutdown_connections 2024-09-16T00:32:11.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 1 -- 172.21.15.1:0/2303173110 wait complete. 2024-09-16T00:32:11.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.445+0000 7fadb8d57740 20 client.16182 trim_cache size 0 max 16384 2024-09-16T00:32:11.489 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:32:11.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 -- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad980a3240 msgr2=0x7fad980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:11.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad980a3240 0x7fad980a5700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fada8005fd0 tx=0x7fada8004ab0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 -- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0102570 msgr2=0x7fadb0198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0102570 0x7fadb0198a30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fada400d770 tx=0x7fada400dc30 comp rx=0 tx=0).stop 2024-09-16T00:32:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 -- 172.21.15.1:0/2235014197 shutdown_connections 2024-09-16T00:32:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fad980a3240 0x7fad980a5700 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadb0102ef0 0x7fadb019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb0102570 0x7fadb0198a30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 --2- 172.21.15.1:0/2235014197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb0100e20 0x7fadb01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 -- 172.21.15.1:0/2235014197 >> 172.21.15.1:0/2235014197 conn(0x7fadb00fc7d0 msgr2=0x7fadb00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 -- 172.21.15.1:0/2235014197 shutdown_connections 2024-09-16T00:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:11.446+0000 7fadb697e640 1 -- 172.21.15.1:0/2235014197 wait complete. 2024-09-16T00:32:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:12 smithi019 ceph-mon[35186]: pgmap v464: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:12.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:12 smithi001 ceph-mon[30934]: pgmap v464: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:12.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:12 smithi153 ceph-mon[35538]: pgmap v464: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:13.257 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:32:13.257 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:32:13.530 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 -- 172.21.15.1:0/352696075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 msgr2=0x7f5570110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 --2- 172.21.15.1:0/352696075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 0x7f5570110900 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5568009900 tx=0x7f556802a410 comp rx=0 tx=0).stop 2024-09-16T00:32:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 -- 172.21.15.1:0/352696075 shutdown_connections 2024-09-16T00:32:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 --2- 172.21.15.1:0/352696075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 0x7f5570110900 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 --2- 172.21.15.1:0/352696075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5570101380 0x7f557010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 --2- 172.21.15.1:0/352696075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55701009d0 0x7f5570100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 -- 172.21.15.1:0/352696075 >> 172.21.15.1:0/352696075 conn(0x7f55700fc7d0 msgr2=0x7f55700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 -- 172.21.15.1:0/352696075 shutdown_connections 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 -- 172.21.15.1:0/352696075 wait complete. 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 Processor -- start 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.083+0000 7f557540f640 1 -- start start 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f557540f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55701009d0 0x7f557019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f557540f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5570101380 0x7f557019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f557540f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 0x7f557019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f557540f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5570114250 con 0x7f557010e4f0 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f557540f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55701140d0 con 0x7f55701009d0 2024-09-16T00:32:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f557540f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55701143d0 con 0x7f5570101380 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 0x7f557019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 0x7f557019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58460/0 (socket says 172.21.15.1:58460) 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556f7fe640 1 -- 172.21.15.1:0/3884420174 learned_addr learned my addr 172.21.15.1:0/3884420174 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556effd640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55701009d0 0x7f557019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556e7fc640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5570101380 0x7f557019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556f7fe640 1 -- 172.21.15.1:0/3884420174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5570101380 msgr2=0x7f557019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556f7fe640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5570101380 0x7f557019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556f7fe640 1 -- 172.21.15.1:0/3884420174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55701009d0 msgr2=0x7f557019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556f7fe640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55701009d0 0x7f557019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556f7fe640 1 -- 172.21.15.1:0/3884420174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557019f6f0 con 0x7f557010e4f0 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556effd640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55701009d0 0x7f557019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f556f7fe640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 0x7f557019f010 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5568005ab0 tx=0x7f55680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f5557fff640 1 -- 172.21.15.1:0/3884420174 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556803f030 con 0x7f557010e4f0 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f5557fff640 1 -- 172.21.15.1:0/3884420174 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5568003ce0 con 0x7f557010e4f0 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f557540f640 1 -- 172.21.15.1:0/3884420174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557019f980 con 0x7f557010e4f0 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.084+0000 7f5557fff640 1 -- 172.21.15.1:0/3884420174 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568035070 con 0x7f557010e4f0 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.085+0000 7f557540f640 1 -- 172.21.15.1:0/3884420174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55701a72c0 con 0x7f557010e4f0 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.087+0000 7f5557fff640 1 -- 172.21.15.1:0/3884420174 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f5568042030 con 0x7f557010e4f0 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.087+0000 7f55777e8740 1 Processor -- start 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.087+0000 7f55777e8740 1 -- start start 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.087+0000 7f55777e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3eb60 0x55f5eae3ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.087+0000 7f55777e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 0x55f5eae49800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.087+0000 7f55777e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae49d40 0x55f5eae4c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.087+0000 7f55777e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f5eada4280 con 0x55f5eae3eb60 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.087+0000 7f55777e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f5ead4bb10 con 0x55f5eae3f480 2024-09-16T00:32:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.087+0000 7f55777e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f5eada0240 con 0x55f5eae49d40 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f5557fff640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55580a3240 0x7f55580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3eb60 0x55f5eae3ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3eb60 0x55f5eae3ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58466/0 (socket says 172.21.15.1:58466) 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 -- 172.21.15.1:0/822313366 learned_addr learned my addr 172.21.15.1:0/822313366 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556f7fe640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55580a3240 0x7f55580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae49d40 0x55f5eae4c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f5557fff640 1 -- 172.21.15.1:0/3884420174 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f55680f0290 con 0x7f557010e4f0 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556effd640 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 0x55f5eae49800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 -- 172.21.15.1:0/822313366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae49d40 msgr2=0x55f5eae4c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae49d40 0x55f5eae4c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 -- 172.21.15.1:0/822313366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 msgr2=0x55f5eae49800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 0x55f5eae49800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 -- 172.21.15.1:0/822313366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ead581f0 con 0x55f5eae3eb60 2024-09-16T00:32:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556effd640 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 0x55f5eae49800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556e7fc640 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3eb60 0x55f5eae3ef40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5564005fd0 tx=0x7f556400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0372dd88ccae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f556f7fe640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55580a3240 0x7f55580a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f55680bc1b0 tx=0x7f55680b1990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f55557fa640 1 -- 172.21.15.1:0/822313366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564019030 con 0x55f5eae3eb60 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f55557fa640 1 -- 172.21.15.1:0/822313366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5564005500 con 0x55f5eae3eb60 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.088+0000 7f55557fa640 1 -- 172.21.15.1:0/822313366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55640116a0 con 0x55f5eae3eb60 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 -- 172.21.15.1:0/822313366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3eb60 msgr2=0x55f5eae3ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3eb60 0x55f5eae3ef40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5564005fd0 tx=0x7f556400b610 comp rx=0 tx=0).stop 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 -- 172.21.15.1:0/822313366 shutdown_connections 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae49d40 0x55f5eae4c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 0x55f5eae49800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 --2- 172.21.15.1:0/822313366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3eb60 0x55f5eae3ef40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 -- 172.21.15.1:0/822313366 >> 172.21.15.1:0/822313366 conn(0x55f5eae3b0f0 msgr2=0x55f5eae3b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 -- 172.21.15.1:0/822313366 shutdown_connections 2024-09-16T00:32:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 -- 172.21.15.1:0/822313366 wait complete. 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.089+0000 7f55777e8740 1 -- start start 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f55777e8740 1 Processor -- start 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f55777e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 0x55f5eae46be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f55777e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 0x55f5eae47120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f55777e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae49d40 0x55f5eae47660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f55777e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f5eae4c910 con 0x55f5eae49d40 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f55777e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f5ead5ef10 con 0x55f5eae3f480 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f55777e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f5eae4ca90 con 0x55f5eae3eb60 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f55777e8740 10 client.0 _ll_register_callbacks cb 0x55f5ead27a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae49d40 0x55f5eae47660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae49d40 0x55f5eae47660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58474/0 (socket says 172.21.15.1:58474) 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556e7fc640 1 -- 172.21.15.1:0/1181231407 learned_addr learned my addr 172.21.15.1:0/1181231407 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556effd640 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 0x55f5eae47120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556e7fc640 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 0x55f5eae46be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556e7fc640 1 -- 172.21.15.1:0/1181231407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 msgr2=0x55f5eae46be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556e7fc640 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 0x55f5eae46be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556e7fc640 1 -- 172.21.15.1:0/1181231407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 msgr2=0x55f5eae47120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556e7fc640 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 0x55f5eae47120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.090+0000 7f556e7fc640 1 -- 172.21.15.1:0/1181231407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5eab156a0 con 0x55f5eae49d40 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f556e7fc640 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae49d40 0x55f5eae47660 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5564013fd0 tx=0x7f5564004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd19362a90057f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f5552ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f5552ff5640 1 -- 172.21.15.1:0/1181231407 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564019050 con 0x55f5eae49d40 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f5552ff5640 1 -- 172.21.15.1:0/1181231407 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f55640051b0 con 0x55f5eae49d40 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f55777e8740 1 -- 172.21.15.1:0/1181231407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5564005500 con 0x55f5eae49d40 2024-09-16T00:32:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f5552ff5640 1 -- 172.21.15.1:0/1181231407 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5564035050 con 0x55f5eae49d40 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f55777e8740 10 client.16194 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f55777e8740 1 -- 172.21.15.1:0/1181231407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5eae58450 con 0x55f5eae49d40 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f5552ff5640 1 -- 172.21.15.1:0/1181231407 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5564005500 con 0x55f5eae49d40 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f5552ff5640 1 -- 172.21.15.1:0/1181231407 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5564011440 con 0x55f5eae49d40 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f55777e8740 10 client.16194 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f55777e8740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f55777e8740 20 client.16194 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f55777e8740 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f5eae59800 0x55f5eae5bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f55777e8740 4 client.16194 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.091+0000 7f55777e8740 1 -- 172.21.15.1:0/1181231407 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5eadb17f0 con 0x55f5eae59800 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.092+0000 7f556effd640 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f5eae59800 0x55f5eae5bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.092+0000 7f556effd640 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f5eae59800 0x55f5eae5bbf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65bc4004bd603f4 server_cookie=14c3e5bd3f30b83b in_seq=0 out_seq=0 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.092+0000 7f5552ff5640 10 client.16194 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f5552ff5640 1 -- 172.21.15.1:0/1181231407 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5eadb17f0 con 0x55f5eae59800 2024-09-16T00:32:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f5552ff5640 10 client.16194 handle_command_reply: tid=0 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 client.16194 shutdown 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 -- 172.21.15.1:0/1181231407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae49d40 msgr2=0x55f5eae47660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae49d40 0x55f5eae47660 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5564013fd0 tx=0x7f5564004c90 comp rx=0 tx=0).stop 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 -- 172.21.15.1:0/1181231407 shutdown_connections 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f5eae59800 0x55f5eae5bbf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae49d40 0x55f5eae47660 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3f480 0x55f5eae47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 --2- 172.21.15.1:0/1181231407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 0x55f5eae46be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f5552ff5640 0 client.16194 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 -- 172.21.15.1:0/1181231407 >> 172.21.15.1:0/1181231407 conn(0x55f5eae3b0f0 msgr2=0x55f5eae40e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 -- 172.21.15.1:0/1181231407 shutdown_connections 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 1 -- 172.21.15.1:0/1181231407 wait complete. 2024-09-16T00:32:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.093+0000 7f55777e8740 20 client.16194 trim_cache size 0 max 16384 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.110+0000 7f55777e8740 1 Processor -- start 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.110+0000 7f55777e8740 1 -- start start 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f55777e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3eb60 0x55f5eae41580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f55777e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae3a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f55777e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae49d40 0x55f5eae61330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f55777e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f55701b3980 con 0x55f5eae3f480 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f55777e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55701b1250 con 0x55f5eae3eb60 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f55777e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55701af0a0 con 0x55f5eae49d40 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae3a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae3a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58486/0 (socket says 172.21.15.1:58486) 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556effd640 1 -- 172.21.15.1:0/334508911 learned_addr learned my addr 172.21.15.1:0/334508911 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556e7fc640 1 --2- 172.21.15.1:0/334508911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3eb60 0x55f5eae41580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556effd640 1 -- 172.21.15.1:0/334508911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae49d40 msgr2=0x55f5eae61330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556effd640 1 --2- 172.21.15.1:0/334508911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae49d40 0x55f5eae61330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556effd640 1 -- 172.21.15.1:0/334508911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3eb60 msgr2=0x55f5eae41580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556effd640 1 --2- 172.21.15.1:0/334508911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3eb60 0x55f5eae41580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556effd640 1 -- 172.21.15.1:0/334508911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5eae3af20 con 0x55f5eae3f480 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f556effd640 1 --2- 172.21.15.1:0/334508911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae3a160 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5560003a80 tx=0x7f5560013bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2a181dce83c838 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f5554ff9640 1 -- 172.21.15.1:0/334508911 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560013db0 con 0x55f5eae3f480 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f5554ff9640 1 -- 172.21.15.1:0/334508911 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55600130d0 con 0x55f5eae3f480 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.111+0000 7f5554ff9640 1 -- 172.21.15.1:0/334508911 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560005280 con 0x55f5eae3f480 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 -- 172.21.15.1:0/334508911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 msgr2=0x55f5eae3a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 --2- 172.21.15.1:0/334508911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae3a160 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5560003a80 tx=0x7f5560013bc0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 -- 172.21.15.1:0/334508911 shutdown_connections 2024-09-16T00:32:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 --2- 172.21.15.1:0/334508911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae49d40 0x55f5eae61330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 --2- 172.21.15.1:0/334508911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae3a160 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 --2- 172.21.15.1:0/334508911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae3eb60 0x55f5eae41580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 -- 172.21.15.1:0/334508911 >> 172.21.15.1:0/334508911 conn(0x55f5eae3b0f0 msgr2=0x55f5eae39a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 -- 172.21.15.1:0/334508911 shutdown_connections 2024-09-16T00:32:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 -- 172.21.15.1:0/334508911 wait complete. 2024-09-16T00:32:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.112+0000 7f55777e8740 1 -- start start 2024-09-16T00:32:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f55777e8740 1 Processor -- start 2024-09-16T00:32:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f55777e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 0x55f5eae4da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f55777e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae4df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f55777e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae49d40 0x55f5eae4e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f55777e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f55701af710 con 0x55f5eae3f480 2024-09-16T00:32:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f55777e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55701b0b80 con 0x55f5eae49d40 2024-09-16T00:32:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f55777e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55701afde0 con 0x55f5eae3eb60 2024-09-16T00:32:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f55777e8740 10 client.0 _ll_register_callbacks cb 0x7f5564013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae4df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae4df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58494/0 (socket says 172.21.15.1:58494) 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556effd640 1 -- 172.21.15.1:0/3544197371 learned_addr learned my addr 172.21.15.1:0/3544197371 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556e7fc640 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae49d40 0x55f5eae4e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556e7fc640 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 0x55f5eae4da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556effd640 1 -- 172.21.15.1:0/3544197371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 msgr2=0x55f5eae4da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556effd640 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 0x55f5eae4da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556effd640 1 -- 172.21.15.1:0/3544197371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae49d40 msgr2=0x55f5eae4e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556effd640 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae49d40 0x55f5eae4e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556effd640 1 -- 172.21.15.1:0/3544197371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5eaec3300 con 0x55f5eae3f480 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556e7fc640 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae49d40 0x55f5eae4e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:14.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f556effd640 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae4df70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5560023000 tx=0x7f5560013f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dd191cadc5287d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.113+0000 7f5552ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f5552ff5640 1 -- 172.21.15.1:0/3544197371 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560010af0 con 0x55f5eae3f480 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f5552ff5640 1 -- 172.21.15.1:0/3544197371 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5560021040 con 0x55f5eae3f480 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f55777e8740 1 -- 172.21.15.1:0/3544197371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55600130d0 con 0x55f5eae3f480 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f5552ff5640 1 -- 172.21.15.1:0/3544197371 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5560013420 con 0x55f5eae3f480 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f55777e8740 10 client.16200 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f55777e8740 1 -- 172.21.15.1:0/3544197371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5eaec4620 con 0x55f5eae3f480 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f5552ff5640 1 -- 172.21.15.1:0/3544197371 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55600130d0 con 0x55f5eae3f480 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f5552ff5640 1 -- 172.21.15.1:0/3544197371 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5560031420 con 0x55f5eae3f480 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f55777e8740 10 client.16200 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f55777e8740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f55777e8740 20 client.16200 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f55777e8740 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f5eaec4d00 0x55f5eaec7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f55777e8740 4 client.16200 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.114+0000 7f55777e8740 1 -- 172.21.15.1:0/3544197371 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f5eadabd00 con 0x55f5eaec4d00 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.115+0000 7f556e7fc640 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f5eaec4d00 0x55f5eaec7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.115+0000 7f5552ff5640 10 client.16200 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.115+0000 7f556e7fc640 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f5eaec4d00 0x55f5eaec7120 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b11d8dd155b22b server_cookie=82ec1e752bd8fea8 in_seq=0 out_seq=0 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f5552ff5640 1 -- 172.21.15.1:0/3544197371 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f5eadabd00 con 0x55f5eaec4d00 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f5552ff5640 10 client.16200 handle_command_reply: tid=0 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 client.16200 shutdown 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 -- 172.21.15.1:0/3544197371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 msgr2=0x55f5eae4df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae4df70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5560023000 tx=0x7f5560013f10 comp rx=0 tx=0).stop 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 -- 172.21.15.1:0/3544197371 shutdown_connections 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f5eaec4d00 0x55f5eaec7120 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5eae49d40 0x55f5eae4e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f5552ff5640 0 client.16200 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5eae3f480 0x55f5eae4df70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 --2- 172.21.15.1:0/3544197371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5eae3eb60 0x55f5eae4da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 -- 172.21.15.1:0/3544197371 >> 172.21.15.1:0/3544197371 conn(0x55f5eae3b0f0 msgr2=0x55f5eae473c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 -- 172.21.15.1:0/3544197371 shutdown_connections 2024-09-16T00:32:14.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.116+0000 7f55777e8740 1 -- 172.21.15.1:0/3544197371 wait complete. 2024-09-16T00:32:14.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f55777e8740 20 client.16200 trim_cache size 0 max 16384 2024-09-16T00:32:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:32:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:32:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "311679b3-bcb9-4197-9d1c-b9654e0dfb90", 2024-09-16T00:32:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:32:14.123 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:32:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 -- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55580a3240 msgr2=0x7f55580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55580a3240 0x7f55580a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f55680bc1b0 tx=0x7f55680b1990 comp rx=0 tx=0).stop 2024-09-16T00:32:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 -- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 msgr2=0x7f557019f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 0x7f557019f010 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5568005ab0 tx=0x7f55680042f0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 -- 172.21.15.1:0/3884420174 shutdown_connections 2024-09-16T00:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f557010e4f0 0x7f557019f010 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f55580a3240 0x7f55580a5700 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5570101380 0x7f557019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 --2- 172.21.15.1:0/3884420174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55701009d0 0x7f557019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 -- 172.21.15.1:0/3884420174 >> 172.21.15.1:0/3884420174 conn(0x7f55700fc7d0 msgr2=0x7f55700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 -- 172.21.15.1:0/3884420174 shutdown_connections 2024-09-16T00:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:14.117+0000 7f557540f640 1 -- 172.21.15.1:0/3884420174 wait complete. 2024-09-16T00:32:14.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:14 smithi001 ceph-mon[30934]: pgmap v465: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:32:14.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:14 smithi019 ceph-mon[35186]: pgmap v465: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:32:14.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:14 smithi153 ceph-mon[35538]: pgmap v465: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:32:15.058 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:32:15.317 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:15.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:15 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:32:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:15 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:15 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:32:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:15 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:15 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:32:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:15 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:15 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:32:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:15 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:15.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:15 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:32:15.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:15 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:15.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:15 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:32:15.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:15 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:15.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.816+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4078120807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 msgr2=0x7f2cc0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.816+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4078120807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 0x7f2cc0100ee0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009930 tx=0x7f2cb002a420 comp rx=0 tx=0).stop 2024-09-16T00:32:15.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4078120807 shutdown_connections 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4078120807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc0101420 0x7f2cc010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4078120807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 0x7f2cc0100ee0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4078120807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc0106a80 0x7f2cc0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4078120807 >> 172.21.15.1:0/4078120807 conn(0x7f2cc00fc7a0 msgr2=0x7f2cc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4078120807 shutdown_connections 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4078120807 wait complete. 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 Processor -- start 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 -- start start 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 0x7f2cc00fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc0101420 0x7f2cc00fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc0106a80 0x7f2cc00fd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0110ea0 con 0x7f2cc0100a80 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0110d20 con 0x7f2cc0106a80 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.817+0000 7f2cc82c2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0111020 con 0x7f2cc0101420 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc605e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 0x7f2cc00fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc605e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 0x7f2cc00fcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58500/0 (socket says 172.21.15.1:58500) 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc605e640 1 -- 172.21.15.1:0/4272255689 learned_addr learned my addr 172.21.15.1:0/4272255689 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc685f640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc0106a80 0x7f2cc00fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc585d640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc0101420 0x7f2cc00fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc605e640 1 -- 172.21.15.1:0/4272255689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc0101420 msgr2=0x7f2cc00fd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc605e640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc0101420 0x7f2cc00fd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc605e640 1 -- 172.21.15.1:0/4272255689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc0106a80 msgr2=0x7f2cc00fd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc605e640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc0106a80 0x7f2cc00fd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc605e640 1 -- 172.21.15.1:0/4272255689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc01a6940 con 0x7f2cc0100a80 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc585d640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc0101420 0x7f2cc00fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc605e640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 0x7f2cc00fcd80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00b4a0 tx=0x7f2cbc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cb77fe640 1 -- 172.21.15.1:0/4272255689 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc014070 con 0x7f2cc0100a80 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cb77fe640 1 -- 172.21.15.1:0/4272255689 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cbc010af0 con 0x7f2cc0100a80 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4272255689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01a6c30 con 0x7f2cc0100a80 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.818+0000 7f2cb77fe640 1 -- 172.21.15.1:0/4272255689 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc017d10 con 0x7f2cc0100a80 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.819+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4272255689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc01a7190 con 0x7f2cc0100a80 2024-09-16T00:32:15.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.821+0000 7f2cb77fe640 1 -- 172.21.15.1:0/4272255689 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2cbc017070 con 0x7f2cc0100a80 2024-09-16T00:32:15.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.821+0000 7f2cca69b740 1 Processor -- start 2024-09-16T00:32:15.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.821+0000 7f2cca69b740 1 -- start start 2024-09-16T00:32:15.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.821+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 0x5575e1201880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e1201dc0 0x5575e120c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e120c680 0x5575e120ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cca69b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5575e11672e0 con 0x5575e1201dc0 2024-09-16T00:32:15.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cca69b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5575e110ece0 con 0x5575e12014a0 2024-09-16T00:32:15.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cca69b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5575e11632a0 con 0x5575e120c680 2024-09-16T00:32:15.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc605e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e1201dc0 0x5575e120c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc605e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e1201dc0 0x5575e120c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58506/0 (socket says 172.21.15.1:58506) 2024-09-16T00:32:15.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc605e640 1 -- 172.21.15.1:0/3964892998 learned_addr learned my addr 172.21.15.1:0/3964892998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:15.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc685f640 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 0x5575e1201880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cb77fe640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cac0a32c0 0x7f2cac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc585d640 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e120c680 0x5575e120ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc685f640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cac0a32c0 0x7f2cac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cb77fe640 1 -- 172.21.15.1:0/4272255689 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2cbc0cf060 con 0x7f2cc0100a80 2024-09-16T00:32:15.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc685f640 1 -- 172.21.15.1:0/3964892998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e120c680 msgr2=0x5575e120ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:15.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc685f640 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e120c680 0x5575e120ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc685f640 1 -- 172.21.15.1:0/3964892998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e1201dc0 msgr2=0x5575e120c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:15.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc685f640 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e1201dc0 0x5575e120c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc685f640 1 -- 172.21.15.1:0/3964892998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e10f3440 con 0x5575e12014a0 2024-09-16T00:32:15.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc605e640 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e1201dc0 0x5575e120c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:15.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.822+0000 7f2cc685f640 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 0x5575e1201880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005fd0 tx=0x7f2cb8013ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76acadb7aad1212a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.823+0000 7f2cc685f640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cac0a32c0 0x7f2cac0a5780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2cb801e000 tx=0x7f2cb8020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.823+0000 7f2cb4dea640 1 -- 172.21.15.1:0/3964892998 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb801fce0 con 0x5575e12014a0 2024-09-16T00:32:15.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.823+0000 7f2cb4dea640 1 -- 172.21.15.1:0/3964892998 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb8023020 con 0x5575e12014a0 2024-09-16T00:32:15.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.823+0000 7f2cb4dea640 1 -- 172.21.15.1:0/3964892998 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8004290 con 0x5575e12014a0 2024-09-16T00:32:15.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.823+0000 7f2cca69b740 1 -- 172.21.15.1:0/3964892998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 msgr2=0x5575e1201880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.823+0000 7f2cca69b740 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 0x5575e1201880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005fd0 tx=0x7f2cb8013ef0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 -- 172.21.15.1:0/3964892998 shutdown_connections 2024-09-16T00:32:15.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e120c680 0x5575e120ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e1201dc0 0x5575e120c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 --2- 172.21.15.1:0/3964892998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 0x5575e1201880 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 -- 172.21.15.1:0/3964892998 >> 172.21.15.1:0/3964892998 conn(0x5575e11fda30 msgr2=0x5575e11fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 -- 172.21.15.1:0/3964892998 shutdown_connections 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 -- 172.21.15.1:0/3964892998 wait complete. 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 -- start start 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 Processor -- start 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e12014a0 0x5575e120bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e1201dc0 0x5575e12091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e12096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5575e120f250 con 0x5575e120c680 2024-09-16T00:32:15.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5575e1122050 con 0x5575e1201dc0 2024-09-16T00:32:15.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5575e120f3d0 con 0x5575e12014a0 2024-09-16T00:32:15.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.824+0000 7f2cca69b740 10 client.0 _ll_register_callbacks cb 0x5575e1162160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:15.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc685f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e12096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc685f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e12096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58522/0 (socket says 172.21.15.1:58522) 2024-09-16T00:32:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc685f640 1 -- 172.21.15.1:0/4200059841 learned_addr learned my addr 172.21.15.1:0/4200059841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc585d640 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e1201dc0 0x5575e12091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc605e640 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e12014a0 0x5575e120bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc685f640 1 -- 172.21.15.1:0/4200059841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e12014a0 msgr2=0x5575e120bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc685f640 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e12014a0 0x5575e120bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc685f640 1 -- 172.21.15.1:0/4200059841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e1201dc0 msgr2=0x5575e12091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc685f640 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e1201dc0 0x5575e12091b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc685f640 1 -- 172.21.15.1:0/4200059841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e0ed83c0 con 0x5575e120c680 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc605e640 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e12014a0 0x5575e120bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cc685f640 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e12096f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005f50 tx=0x7f2cb801ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908dc5d35538c975 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2ca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/4200059841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80135e0 con 0x5575e120c680 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/4200059841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2cb8003b80 con 0x5575e120c680 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.825+0000 7f2cca69b740 1 -- 172.21.15.1:0/4200059841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cb8023020 con 0x5575e120c680 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/4200059841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cb8039050 con 0x5575e120c680 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2cca69b740 10 client.16212 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2cca69b740 1 -- 172.21.15.1:0/4200059841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575e121abe0 con 0x5575e120c680 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/4200059841 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb8023020 con 0x5575e120c680 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/4200059841 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2cb8027450 con 0x5575e120c680 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2cca69b740 10 client.16212 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2cca69b740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2cca69b740 20 client.16212 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2cca69b740 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5575e121c040 0x5575e121e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2cca69b740 4 client.16212 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2cca69b740 1 -- 172.21.15.1:0/4200059841 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575e1174850 con 0x5575e121c040 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.826+0000 7f2cc605e640 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5575e121c040 0x5575e121e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.827+0000 7f2cc605e640 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5575e121c040 0x5575e121e430 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31dd9518cd599aad server_cookie=79af3f5ada15a65e in_seq=0 out_seq=0 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.827+0000 7f2ca9ffb640 10 client.16212 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.827+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/4200059841 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2cbc079e50 con 0x5575e121c040 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.827+0000 7f2ca9ffb640 10 client.16212 handle_command_reply: tid=0 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.827+0000 7f2cca69b740 1 client.16212 shutdown 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 -- 172.21.15.1:0/4200059841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 msgr2=0x5575e12096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e12096f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005f50 tx=0x7f2cb801ff70 comp rx=0 tx=0).stop 2024-09-16T00:32:15.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 -- 172.21.15.1:0/4200059841 shutdown_connections 2024-09-16T00:32:15.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5575e121c040 0x5575e121e430 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e12096f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e1201dc0 0x5575e12091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 --2- 172.21.15.1:0/4200059841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e12014a0 0x5575e120bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 -- 172.21.15.1:0/4200059841 >> 172.21.15.1:0/4200059841 conn(0x5575e11fda30 msgr2=0x5575e11fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2ca9ffb640 0 client.16212 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:15.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 -- 172.21.15.1:0/4200059841 shutdown_connections 2024-09-16T00:32:15.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 1 -- 172.21.15.1:0/4200059841 wait complete. 2024-09-16T00:32:15.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.828+0000 7f2cca69b740 20 client.16212 trim_cache size 0 max 16384 2024-09-16T00:32:15.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cca69b740 1 Processor -- start 2024-09-16T00:32:15.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cca69b740 1 -- start start 2024-09-16T00:32:15.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e12014a0 0x5575e12184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 0x5575e1203600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e120c680 0x5575e1223bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cca69b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01b3890 con 0x5575e12014a0 2024-09-16T00:32:15.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cca69b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01b1160 con 0x5575e120c680 2024-09-16T00:32:15.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cca69b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01aefb0 con 0x5575e1201dc0 2024-09-16T00:32:15.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cc605e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e12014a0 0x5575e12184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cc605e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e12014a0 0x5575e12184e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58524/0 (socket says 172.21.15.1:58524) 2024-09-16T00:32:15.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.845+0000 7f2cc605e640 1 -- 172.21.15.1:0/2283086922 learned_addr learned my addr 172.21.15.1:0/2283086922 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:15.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cc585d640 1 --2- 172.21.15.1:0/2283086922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 0x5575e1203600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cc685f640 1 --2- 172.21.15.1:0/2283086922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e120c680 0x5575e1223bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cc605e640 1 -- 172.21.15.1:0/2283086922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 msgr2=0x5575e1203600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cc605e640 1 --2- 172.21.15.1:0/2283086922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 0x5575e1203600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cc605e640 1 -- 172.21.15.1:0/2283086922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e120c680 msgr2=0x5575e1223bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cc605e640 1 --2- 172.21.15.1:0/2283086922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e120c680 0x5575e1223bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cc605e640 1 -- 172.21.15.1:0/2283086922 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e1201bf0 con 0x5575e12014a0 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cc605e640 1 --2- 172.21.15.1:0/2283086922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e12014a0 0x5575e12184e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc000c00 tx=0x7f2cbc0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bc48b689a7c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cabfff640 1 -- 172.21.15.1:0/2283086922 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc01e040 con 0x5575e12014a0 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cabfff640 1 -- 172.21.15.1:0/2283086922 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5575e1174850 con 0x5575e12014a0 2024-09-16T00:32:15.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.846+0000 7f2cabfff640 1 -- 172.21.15.1:0/2283086922 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0b9e60 con 0x5575e12014a0 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 -- 172.21.15.1:0/2283086922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e12014a0 msgr2=0x5575e12184e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 --2- 172.21.15.1:0/2283086922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e12014a0 0x5575e12184e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc000c00 tx=0x7f2cbc0bb040 comp rx=0 tx=0).stop 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 -- 172.21.15.1:0/2283086922 shutdown_connections 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 --2- 172.21.15.1:0/2283086922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e120c680 0x5575e1223bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 --2- 172.21.15.1:0/2283086922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 0x5575e1203600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 --2- 172.21.15.1:0/2283086922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e12014a0 0x5575e12184e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 -- 172.21.15.1:0/2283086922 >> 172.21.15.1:0/2283086922 conn(0x5575e11fda30 msgr2=0x5575e1202ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 -- 172.21.15.1:0/2283086922 shutdown_connections 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 -- 172.21.15.1:0/2283086922 wait complete. 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 -- start start 2024-09-16T00:32:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 Processor -- start 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 0x5575e12103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.847+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 0x5575e1210900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cca69b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e1210e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cca69b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01af620 con 0x5575e120c680 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cca69b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01b0a90 con 0x5575e12014a0 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cca69b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01afcf0 con 0x5575e1201dc0 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cca69b740 10 client.0 _ll_register_callbacks cb 0x7f2cb8013070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc685f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e1210e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc685f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e1210e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58532/0 (socket says 172.21.15.1:58532) 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc685f640 1 -- 172.21.15.1:0/64984424 learned_addr learned my addr 172.21.15.1:0/64984424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc605e640 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 0x5575e12103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc585d640 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 0x5575e1210900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc685f640 1 -- 172.21.15.1:0/64984424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 msgr2=0x5575e1210900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc685f640 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 0x5575e1210900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc685f640 1 -- 172.21.15.1:0/64984424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 msgr2=0x5575e12103c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc685f640 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 0x5575e12103c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc685f640 1 -- 172.21.15.1:0/64984424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e120c270 con 0x5575e120c680 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cc685f640 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e1210e40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8016090 tx=0x7f2cb8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fec66f653069b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2ca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/64984424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8027660 con 0x5575e120c680 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/64984424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2cb8003d80 con 0x5575e120c680 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.848+0000 7f2cca69b740 1 -- 172.21.15.1:0/64984424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575e1174850 con 0x5575e120c680 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/64984424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cb8045050 con 0x5575e120c680 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2cca69b740 10 client.16218 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2cca69b740 1 -- 172.21.15.1:0/64984424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575e1286cc0 con 0x5575e120c680 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/64984424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5575e1174850 con 0x5575e120c680 2024-09-16T00:32:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/64984424 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2cb8005030 con 0x5575e120c680 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2cca69b740 10 client.16218 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2cca69b740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2cca69b740 20 client.16218 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2cca69b740 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5575e1287320 0x5575e1289710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2cca69b740 4 client.16218 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2cca69b740 1 -- 172.21.15.1:0/64984424 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5575e116ed60 con 0x5575e1287320 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.849+0000 7f2cc605e640 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5575e1287320 0x5575e1289710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2ca9ffb640 10 client.16218 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cc605e640 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5575e1287320 0x5575e1289710 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40340acd3093d21 server_cookie=9be814a1d3a7484b in_seq=0 out_seq=0 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2ca9ffb640 1 -- 172.21.15.1:0/64984424 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5575e116ed60 con 0x5575e1287320 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2ca9ffb640 10 client.16218 handle_command_reply: tid=0 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 client.16218 shutdown 2024-09-16T00:32:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 -- 172.21.15.1:0/64984424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 msgr2=0x5575e1210e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e1210e40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8016090 tx=0x7f2cb8009b30 comp rx=0 tx=0).stop 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 -- 172.21.15.1:0/64984424 shutdown_connections 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5575e1287320 0x5575e1289710 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575e120c680 0x5575e1210e40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e1201dc0 0x5575e1210900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 --2- 172.21.15.1:0/64984424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e12014a0 0x5575e12103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 -- 172.21.15.1:0/64984424 >> 172.21.15.1:0/64984424 conn(0x5575e11fda30 msgr2=0x5575e11fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2ca9ffb640 0 client.16218 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 -- 172.21.15.1:0/64984424 shutdown_connections 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 1 -- 172.21.15.1:0/64984424 wait complete. 2024-09-16T00:32:15.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.850+0000 7f2cca69b740 20 client.16218 trim_cache size 0 max 16384 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "311679b3-bcb9-4197-9d1c-b9654e0dfb90": { 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "311679b3-bcb9-4197-9d1c-b9654e0dfb90", 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:32:15.858 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:32:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cac0a32c0 msgr2=0x7f2cac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cac0a32c0 0x7f2cac0a5780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2cb801e000 tx=0x7f2cb8020000 comp rx=0 tx=0).stop 2024-09-16T00:32:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 msgr2=0x7f2cc00fcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 0x7f2cc00fcd80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00b4a0 tx=0x7f2cbc00b960 comp rx=0 tx=0).stop 2024-09-16T00:32:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4272255689 shutdown_connections 2024-09-16T00:32:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc0106a80 0x7f2cc00fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cac0a32c0 0x7f2cac0a5780 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc0101420 0x7f2cc00fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 --2- 172.21.15.1:0/4272255689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc0100a80 0x7f2cc00fcd80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4272255689 >> 172.21.15.1:0/4272255689 conn(0x7f2cc00fc7a0 msgr2=0x7f2cc006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4272255689 shutdown_connections 2024-09-16T00:32:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:15.851+0000 7f2cc82c2640 1 -- 172.21.15.1:0/4272255689 wait complete. 2024-09-16T00:32:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:16 smithi019 ceph-mon[35186]: pgmap v466: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:16.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:16 smithi001 ceph-mon[30934]: pgmap v466: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:16.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:16 smithi153 ceph-mon[35538]: pgmap v466: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:16.766 INFO:tasks.cephfs.filesystem:scrub status for tag:311679b3-bcb9-4197-9d1c-b9654e0dfb90 - {'path': '/', 'tag': '311679b3-bcb9-4197-9d1c-b9654e0dfb90', 'options': 'recursive,force'} 2024-09-16T00:32:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:18 smithi001 ceph-mon[30934]: pgmap v467: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:18 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:32:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:18 smithi019 ceph-mon[35186]: pgmap v467: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:18 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:32:18.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:18 smithi153 ceph-mon[35538]: pgmap v467: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:18.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:18 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:32:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:20 smithi019 ceph-mon[35186]: pgmap v468: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:20.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:20 smithi001 ceph-mon[30934]: pgmap v468: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:20.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:20 smithi153 ceph-mon[35538]: pgmap v468: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:22.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:22 smithi019 ceph-mon[35186]: pgmap v469: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:22 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:22.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:22 smithi001 ceph-mon[30934]: pgmap v469: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:22.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:22 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:22.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:22 smithi153 ceph-mon[35538]: pgmap v469: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:22.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:22 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:23.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:23.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:23.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:32:23.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:23.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:23.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:32:23.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:23.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:23.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:32:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:24 smithi019 ceph-mon[35186]: pgmap v470: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:24.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:24 smithi001 ceph-mon[30934]: pgmap v470: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:24.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:24 smithi153 ceph-mon[35538]: pgmap v470: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:26.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:26 smithi019 ceph-mon[35186]: pgmap v471: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:26.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:26 smithi001 ceph-mon[30934]: pgmap v471: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:26.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:26 smithi153 ceph-mon[35538]: pgmap v471: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:28.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:28 smithi001 ceph-mon[30934]: pgmap v472: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:28 smithi019 ceph-mon[35186]: pgmap v472: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:28.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:28 smithi153 ceph-mon[35538]: pgmap v472: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:30.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:30 smithi019 ceph-mon[35186]: pgmap v473: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:30.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:30 smithi001 ceph-mon[30934]: pgmap v473: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:30.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:30 smithi153 ceph-mon[35538]: pgmap v473: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:32.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:32 smithi019 ceph-mon[35186]: pgmap v474: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:32.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:32 smithi001 ceph-mon[30934]: pgmap v474: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:32.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:32 smithi153 ceph-mon[35538]: pgmap v474: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:34.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:34 smithi019 ceph-mon[35186]: pgmap v475: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:32:34.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:34 smithi001 ceph-mon[30934]: pgmap v475: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:32:34.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:34 smithi153 ceph-mon[35538]: pgmap v475: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:32:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:36 smithi019 ceph-mon[35186]: pgmap v476: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:36 smithi001 ceph-mon[30934]: pgmap v476: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:36.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:36 smithi153 ceph-mon[35538]: pgmap v476: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:38.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:38 smithi019 ceph-mon[35186]: pgmap v477: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:38.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:38 smithi001 ceph-mon[30934]: pgmap v477: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:38.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:38 smithi153 ceph-mon[35538]: pgmap v477: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:40.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:40 smithi019 ceph-mon[35186]: pgmap v478: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:40.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:40 smithi001 ceph-mon[30934]: pgmap v478: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:40.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:40 smithi153 ceph-mon[35538]: pgmap v478: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:42 smithi019 ceph-mon[35186]: pgmap v479: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:42.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:42 smithi001 ceph-mon[30934]: pgmap v479: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:42.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:42 smithi153 ceph-mon[35538]: pgmap v479: 129 pgs: 129 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:44.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:44 smithi019 ceph-mon[35186]: pgmap v480: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:32:44.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:44 smithi001 ceph-mon[30934]: pgmap v480: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:32:44.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:44 smithi153 ceph-mon[35538]: pgmap v480: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:32:46.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:46 smithi019 ceph-mon[35186]: pgmap v481: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:46.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:46 smithi001 ceph-mon[30934]: pgmap v481: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:46.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:46 smithi153 ceph-mon[35538]: pgmap v481: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:46.768 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:32:47.044 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/2676609562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 msgr2=0x7f6fc4110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/2676609562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 0x7f6fc4110870 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009ca0 tx=0x7f6fbc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/2676609562 shutdown_connections 2024-09-16T00:32:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/2676609562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 0x7f6fc4110870 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/2676609562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fc4101280 0x7f6fc410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/2676609562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fc41008d0 0x7f6fc4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/2676609562 >> 172.21.15.1:0/2676609562 conn(0x7f6fc40fc7a0 msgr2=0x7f6fc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/2676609562 shutdown_connections 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/2676609562 wait complete. 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 Processor -- start 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.717+0000 7f6fcb5dc640 1 -- start start 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fcb5dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fc41008d0 0x7f6fc419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fcb5dc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fc4101280 0x7f6fc419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fcb5dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 0x7f6fc419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fcb5dc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4114180 con 0x7f6fc410e480 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fcb5dc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4114000 con 0x7f6fc41008d0 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fcb5dc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4114300 con 0x7f6fc4101280 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9b79640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 0x7f6fc419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9b79640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 0x7f6fc419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39992/0 (socket says 172.21.15.1:39992) 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9b79640 1 -- 172.21.15.1:0/469226900 learned_addr learned my addr 172.21.15.1:0/469226900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fc4101280 0x7f6fc419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9378640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fc41008d0 0x7f6fc419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9b79640 1 -- 172.21.15.1:0/469226900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fc4101280 msgr2=0x7f6fc419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9b79640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fc4101280 0x7f6fc419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9b79640 1 -- 172.21.15.1:0/469226900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fc41008d0 msgr2=0x7f6fc419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9b79640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fc41008d0 0x7f6fc419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9b79640 1 -- 172.21.15.1:0/469226900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc419f650 con 0x7f6fc410e480 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9378640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fc41008d0 0x7f6fc419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fc9b79640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 0x7f6fc419ef70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc02ade0 tx=0x7f6fbc0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fba7fc640 1 -- 172.21.15.1:0/469226900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc003c30 con 0x7f6fc410e480 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fba7fc640 1 -- 172.21.15.1:0/469226900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fbc003dd0 con 0x7f6fc410e480 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/469226900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc419f8e0 con 0x7f6fc410e480 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.718+0000 7f6fba7fc640 1 -- 172.21.15.1:0/469226900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc02fd60 con 0x7f6fc410e480 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.719+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/469226900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc41a7220 con 0x7f6fc410e480 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.721+0000 7f6fba7fc640 1 -- 172.21.15.1:0/469226900 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6fbc004050 con 0x7f6fc410e480 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.721+0000 7f6fcd9b5740 1 Processor -- start 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.721+0000 7f6fcd9b5740 1 -- start start 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9b490 0x55ba75f9b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9bdb0 0x55ba75fa6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ba75f012e0 con 0x55ba75fa6670 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba75ea8ce0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba75efd2a0 con 0x55ba75f9b490 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fba7fc640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fb00a3240 0x7f6fb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc9b79640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fb00a3240 0x7f6fb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39998/0 (socket says 172.21.15.1:39998) 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 -- 172.21.15.1:0/4085933119 learned_addr learned my addr 172.21.15.1:0/4085933119 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc9378640 1 --2- 172.21.15.1:0/4085933119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9bdb0 0x55ba75fa6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/4085933119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9b490 0x55ba75f9b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fba7fc640 1 -- 172.21.15.1:0/469226900 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6fbc0f1200 con 0x7f6fc410e480 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 -- 172.21.15.1:0/4085933119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9b490 msgr2=0x55ba75f9b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/4085933119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9b490 0x55ba75f9b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 -- 172.21.15.1:0/4085933119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9bdb0 msgr2=0x55ba75fa6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/4085933119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9bdb0 0x55ba75fa6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 -- 172.21.15.1:0/4085933119 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba75e8d440 con 0x55ba75fa6670 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc9b79640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fb00a3240 0x7f6fb00a5700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0bd2d0 tx=0x7f6fbc08d360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.722+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/4085933119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa8a60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4012720 tx=0x7f6fb400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f447d32d5dd7205c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6faf7fe640 1 -- 172.21.15.1:0/4085933119 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4012ca0 con 0x55ba75fa6670 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6faf7fe640 1 -- 172.21.15.1:0/4085933119 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fb4012e40 con 0x55ba75fa6670 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6faf7fe640 1 -- 172.21.15.1:0/4085933119 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40049e0 con 0x55ba75fa6670 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/4085933119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 msgr2=0x55ba75fa8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/4085933119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa8a60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4012720 tx=0x7f6fb400b6e0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/4085933119 shutdown_connections 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/4085933119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa8a60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/4085933119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9bdb0 0x55ba75fa6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/4085933119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9b490 0x55ba75f9b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.723+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/4085933119 >> 172.21.15.1:0/4085933119 conn(0x55ba75f97a20 msgr2=0x55ba75f97e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/4085933119 shutdown_connections 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/4085933119 wait complete. 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 -- start start 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 Processor -- start 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fa6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9bdb0 0x55ba75fa32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ba75fa9240 con 0x55ba75fa6670 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba75ebc050 con 0x55ba75f9b490 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba75fa93c0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.724+0000 7f6fcd9b5740 10 client.0 _ll_register_callbacks cb 0x55ba75efc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fa6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fa6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:42120/0 (socket says 172.21.15.1:42120) 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 -- 172.21.15.1:0/3827005215 learned_addr learned my addr 172.21.15.1:0/3827005215 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc9378640 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9bdb0 0x55ba75fa32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 -- 172.21.15.1:0/3827005215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9bdb0 msgr2=0x55ba75fa32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9bdb0 0x55ba75fa32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 -- 172.21.15.1:0/3827005215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 msgr2=0x55ba75fa3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 -- 172.21.15.1:0/3827005215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba75c723c0 con 0x55ba75f9b490 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fa6020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4005560 tx=0x7f6fb4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9febffc8fe1e3aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.725+0000 7f6facff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6facff9640 1 -- 172.21.15.1:0/3827005215 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4004050 con 0x55ba75f9b490 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6facff9640 1 -- 172.21.15.1:0/3827005215 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6fb40041f0 con 0x55ba75f9b490 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3827005215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba75f0e850 con 0x55ba75f9b490 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6facff9640 1 -- 172.21.15.1:0/3827005215 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6fb4034050 con 0x55ba75f9b490 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6fcd9b5740 10 client.25700 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3827005215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba75fb4c30 con 0x55ba75f9b490 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6facff9640 1 -- 172.21.15.1:0/3827005215 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ba75f0e850 con 0x55ba75f9b490 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6facff9640 1 -- 172.21.15.1:0/3827005215 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6fb4004750 con 0x55ba75f9b490 2024-09-16T00:32:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6fcd9b5740 10 client.25700 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6fcd9b5740 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6fcd9b5740 20 client.25700 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ba75fb5f80 0x55ba75fb8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6fcd9b5740 4 client.25700 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.726+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3827005215 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba75f08d60 con 0x55ba75fb5f80 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.727+0000 7f6fc9378640 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ba75fb5f80 0x55ba75fb8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.727+0000 7f6facff9640 10 client.25700 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.727+0000 7f6fc9378640 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ba75fb5f80 0x55ba75fb8370 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eec2e4e2bf8a419 server_cookie=1db4fa1b324cbe93 in_seq=0 out_seq=0 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6facff9640 1 -- 172.21.15.1:0/3827005215 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba75f08d60 con 0x55ba75fb5f80 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6facff9640 10 client.25700 handle_command_reply: tid=0 2024-09-16T00:32:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 client.25700 shutdown 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3827005215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 msgr2=0x55ba75fa6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fa6020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4005560 tx=0x7f6fb4005fd0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3827005215 shutdown_connections 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ba75fb5f80 0x55ba75fb8370 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75fa6670 0x55ba75fa3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75f9bdb0 0x55ba75fa32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6facff9640 0 client.25700 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3827005215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fa6020 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3827005215 >> 172.21.15.1:0/3827005215 conn(0x55ba75f97a20 msgr2=0x55ba75f9d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3827005215 shutdown_connections 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3827005215 wait complete. 2024-09-16T00:32:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.728+0000 7f6fcd9b5740 20 client.25700 trim_cache size 0 max 16384 2024-09-16T00:32:47.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.746+0000 7f6fcd9b5740 1 Processor -- start 2024-09-16T00:32:47.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.746+0000 7f6fcd9b5740 1 -- start start 2024-09-16T00:32:47.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.746+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75f9eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.746+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75fb24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 0x55ba75fbdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc41b39e0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc41b12b0 con 0x55ba75f9b490 2024-09-16T00:32:47.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc41af100 con 0x55ba75fa6670 2024-09-16T00:32:47.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc9378640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75fb24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc9378640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75fb24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40020/0 (socket says 172.21.15.1:40020) 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc9378640 1 -- 172.21.15.1:0/3220637331 learned_addr learned my addr 172.21.15.1:0/3220637331 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75f9eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 0x55ba75fbdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc9378640 1 -- 172.21.15.1:0/3220637331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 msgr2=0x55ba75fbdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc9378640 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 0x55ba75fbdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc9378640 1 -- 172.21.15.1:0/3220637331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 msgr2=0x55ba75f9eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc9378640 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75f9eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc9378640 1 -- 172.21.15.1:0/3220637331 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba75fa3be0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75f9eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:47.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6fc9378640 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75fb24a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x55ba75f98880 tx=0x7f6fc000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1c32920c0b1b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:47.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.747+0000 7f6faeffd640 1 -- 172.21.15.1:0/3220637331 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0003e40 con 0x55ba75f9bdb0 2024-09-16T00:32:47.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6faeffd640 1 -- 172.21.15.1:0/3220637331 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fc0005500 con 0x55ba75f9bdb0 2024-09-16T00:32:47.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6faeffd640 1 -- 172.21.15.1:0/3220637331 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00116a0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3220637331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 msgr2=0x55ba75fb24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75fb24a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x55ba75f98880 tx=0x7f6fc000bde0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3220637331 shutdown_connections 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 0x55ba75fbdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75fb24a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/3220637331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75f9eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3220637331 >> 172.21.15.1:0/3220637331 conn(0x55ba75f97a20 msgr2=0x55ba75fa5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3220637331 shutdown_connections 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.748+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/3220637331 wait complete. 2024-09-16T00:32:47.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fcd9b5740 1 -- start start 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fcd9b5740 1 Processor -- start 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75faed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fcd9b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 0x55ba75fab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc41af770 con 0x55ba75f9bdb0 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc41b0be0 con 0x55ba75f9b490 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fcd9b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc41afe40 con 0x55ba75fa6670 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fc9378640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75faed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.749+0000 7f6fc9378640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75faed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40026/0 (socket says 172.21.15.1:40026) 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc9378640 1 -- 172.21.15.1:0/158745696 learned_addr learned my addr 172.21.15.1:0/158745696 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fcd9b5740 10 client.0 _ll_register_callbacks cb 0x7f6fc000b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 0x55ba75fab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc9378640 1 -- 172.21.15.1:0/158745696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 msgr2=0x55ba75fab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc9378640 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 0x55ba75fab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc9378640 1 -- 172.21.15.1:0/158745696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 msgr2=0x55ba75fae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc9378640 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc9378640 1 -- 172.21.15.1:0/158745696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba75fa8dd0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fc9378640 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75faed30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000b900 tx=0x7f6fc00056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7048271560dd083 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6facff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6facff9640 1 -- 172.21.15.1:0/158745696 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00051e0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6facff9640 1 -- 172.21.15.1:0/158745696 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6fc00046c0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.750+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/158745696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fc0005500 con 0x55ba75f9bdb0 2024-09-16T00:32:47.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6facff9640 1 -- 172.21.15.1:0/158745696 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6fc0037050 con 0x55ba75f9bdb0 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6fcd9b5740 10 client.16236 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/158745696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba760213a0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6facff9640 1 -- 172.21.15.1:0/158745696 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fc0005500 con 0x55ba75f9bdb0 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6facff9640 1 -- 172.21.15.1:0/158745696 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6fc0011ae0 con 0x55ba75f9bdb0 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6fcd9b5740 10 client.16236 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6fcd9b5740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6fcd9b5740 20 client.16236 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ba76021a40 0x55ba76023e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6fcd9b5740 4 client.16236 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/158745696 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba75e85210 con 0x55ba76021a40 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.751+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ba76021a40 0x55ba76023e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fc8b77640 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ba76021a40 0x55ba76023e60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f7303d5724c90e server_cookie=66e92e29120804a8 in_seq=0 out_seq=0 2024-09-16T00:32:47.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6facff9640 10 client.16236 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6facff9640 1 -- 172.21.15.1:0/158745696 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ba75e85210 con 0x55ba76021a40 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6facff9640 10 client.16236 handle_command_reply: tid=0 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 client.16236 shutdown 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/158745696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 msgr2=0x55ba75faed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75faed30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000b900 tx=0x7f6fc00056b0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/158745696 shutdown_connections 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ba76021a40 0x55ba76023e60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba75fa6670 0x55ba75fab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba75f9bdb0 0x55ba75faed30 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 --2- 172.21.15.1:0/158745696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba75f9b490 0x55ba75fae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6facff9640 0 client.16236 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:47.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/158745696 >> 172.21.15.1:0/158745696 conn(0x55ba75f97a20 msgr2=0x55ba75f9cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:47.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/158745696 shutdown_connections 2024-09-16T00:32:47.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 1 -- 172.21.15.1:0/158745696 wait complete. 2024-09-16T00:32:47.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.752+0000 7f6fcd9b5740 20 client.16236 trim_cache size 0 max 16384 2024-09-16T00:32:47.759 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:32:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:32:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:32:47.759 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:32:47.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fb00a3240 msgr2=0x7f6fb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fb00a3240 0x7f6fb00a5700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0bd2d0 tx=0x7f6fbc08d360 comp rx=0 tx=0).stop 2024-09-16T00:32:47.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 msgr2=0x7f6fc419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 0x7f6fc419ef70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc02ade0 tx=0x7f6fbc0319f0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/469226900 shutdown_connections 2024-09-16T00:32:47.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fc410e480 0x7f6fc419ef70 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6fb00a3240 0x7f6fb00a5700 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fc4101280 0x7f6fc419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 --2- 172.21.15.1:0/469226900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fc41008d0 0x7f6fc419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/469226900 >> 172.21.15.1:0/469226900 conn(0x7f6fc40fc7a0 msgr2=0x7f6fc40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:47.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/469226900 shutdown_connections 2024-09-16T00:32:47.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:47.753+0000 7f6fcb5dc640 1 -- 172.21.15.1:0/469226900 wait complete. 2024-09-16T00:32:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:48 smithi019 ceph-mon[35186]: pgmap v482: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:48.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:48 smithi001 ceph-mon[30934]: pgmap v482: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:48.700 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:32:48.701 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:48.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:48 smithi153 ceph-mon[35538]: pgmap v482: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:48.962 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:49.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 -- 172.21.15.1:0/2253432845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510100e20 msgr2=0x7f2510101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 --2- 172.21.15.1:0/2253432845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510100e20 0x7f2510101200 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2500009900 tx=0x7f250002a480 comp rx=0 tx=0).stop 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 -- 172.21.15.1:0/2253432845 shutdown_connections 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 --2- 172.21.15.1:0/2253432845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2510102ef0 0x7f251010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 --2- 172.21.15.1:0/2253432845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2510102570 0x7f25101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 --2- 172.21.15.1:0/2253432845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510100e20 0x7f2510101200 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 -- 172.21.15.1:0/2253432845 >> 172.21.15.1:0/2253432845 conn(0x7f25100fc7d0 msgr2=0x7f25100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 -- 172.21.15.1:0/2253432845 shutdown_connections 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 -- 172.21.15.1:0/2253432845 wait complete. 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.468+0000 7f2514f3e640 1 Processor -- start 2024-09-16T00:32:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f2514f3e640 1 -- start start 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f2514f3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2510100e20 0x7f2510197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f2514f3e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2510102570 0x7f2510197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f2514f3e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510102ef0 0x7f251019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f2514f3e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2510110fd0 con 0x7f2510102ef0 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f2514f3e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2510110e50 con 0x7f2510100e20 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f2514f3e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2510111150 con 0x7f2510102570 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510102ef0 0x7f251019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510102ef0 0x7f251019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40050/0 (socket says 172.21.15.1:40050) 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2510100e20 0x7f2510197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250ed9d640 1 -- 172.21.15.1:0/1906313588 learned_addr learned my addr 172.21.15.1:0/1906313588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f2507fff640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2510102570 0x7f2510197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250ed9d640 1 -- 172.21.15.1:0/1906313588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2510102570 msgr2=0x7f2510197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250ed9d640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2510102570 0x7f2510197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250ed9d640 1 -- 172.21.15.1:0/1906313588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2510100e20 msgr2=0x7f2510197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250ed9d640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2510100e20 0x7f2510197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250ed9d640 1 -- 172.21.15.1:0/1906313588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251019c3f0 con 0x7f2510102ef0 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250e59c640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2510100e20 0x7f2510197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.469+0000 7f250ed9d640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510102ef0 0x7f251019bd10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f250800eb10 tx=0x7f250800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.470+0000 7f25077fe640 1 -- 172.21.15.1:0/1906313588 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508004b20 con 0x7f2510102ef0 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.470+0000 7f25077fe640 1 -- 172.21.15.1:0/1906313588 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2508004cc0 con 0x7f2510102ef0 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.470+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251019c6e0 con 0x7f2510102ef0 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.470+0000 7f25077fe640 1 -- 172.21.15.1:0/1906313588 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508010760 con 0x7f2510102ef0 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.470+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25101a3fc0 con 0x7f2510102ef0 2024-09-16T00:32:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.471+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2510101200 con 0x7f2510102ef0 2024-09-16T00:32:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.472+0000 7f25077fe640 1 -- 172.21.15.1:0/1906313588 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2508010900 con 0x7f2510102ef0 2024-09-16T00:32:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.473+0000 7f25077fe640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24fc0a32c0 0x7f24fc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.473+0000 7f250e59c640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24fc0a32c0 0x7f24fc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.473+0000 7f25077fe640 1 -- 172.21.15.1:0/1906313588 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f25080d0450 con 0x7f2510102ef0 2024-09-16T00:32:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.473+0000 7f250e59c640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24fc0a32c0 0x7f24fc0a5780 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2500000c00 tx=0x7f2500005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.476+0000 7f25077fe640 1 -- 172.21.15.1:0/1906313588 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f250809cd70 con 0x7f2510102ef0 2024-09-16T00:32:49.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.687+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25101987a0 con 0x7f2510102ef0 2024-09-16T00:32:49.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.688+0000 7f25077fe640 1 -- 172.21.15.1:0/1906313588 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f25101987a0 con 0x7f2510102ef0 2024-09-16T00:32:49.688 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:32:49.688 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:32:49.689 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:32:49.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24fc0a32c0 msgr2=0x7f24fc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24fc0a32c0 0x7f24fc0a5780 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2500000c00 tx=0x7f2500005b50 comp rx=0 tx=0).stop 2024-09-16T00:32:49.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510102ef0 msgr2=0x7f251019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510102ef0 0x7f251019bd10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f250800eb10 tx=0x7f250800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 shutdown_connections 2024-09-16T00:32:49.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2510102ef0 0x7f251019bd10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24fc0a32c0 0x7f24fc0a5780 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2510102570 0x7f2510197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 --2- 172.21.15.1:0/1906313588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2510100e20 0x7f2510197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 >> 172.21.15.1:0/1906313588 conn(0x7f25100fc7d0 msgr2=0x7f25100fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:49.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 shutdown_connections 2024-09-16T00:32:49.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:49.690+0000 7f2514f3e640 1 -- 172.21.15.1:0/1906313588 wait complete. 2024-09-16T00:32:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:50 smithi001 ceph-mon[30934]: pgmap v483: 129 pgs: 129 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:32:50.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:50 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1906313588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:50.506 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:32:50.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:50 smithi019 ceph-mon[35186]: pgmap v483: 129 pgs: 129 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:32:50.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:50 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1906313588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:50.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:50 smithi153 ceph-mon[35538]: pgmap v483: 129 pgs: 129 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:32:50.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:50 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1906313588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.286+0000 7fc324c54640 1 -- 172.21.15.1:0/1297956952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320101240 msgr2=0x7fc320101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.286+0000 7fc324c54640 1 --2- 172.21.15.1:0/1297956952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320101240 0x7fc320101680 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc314009900 tx=0x7fc31402a410 comp rx=0 tx=0).stop 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.286+0000 7fc324c54640 1 -- 172.21.15.1:0/1297956952 shutdown_connections 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.286+0000 7fc324c54640 1 --2- 172.21.15.1:0/1297956952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320101bc0 0x7fc32010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.286+0000 7fc324c54640 1 --2- 172.21.15.1:0/1297956952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320101240 0x7fc320101680 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.286+0000 7fc324c54640 1 --2- 172.21.15.1:0/1297956952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc320105a90 0x7fc320105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.286+0000 7fc324c54640 1 -- 172.21.15.1:0/1297956952 >> 172.21.15.1:0/1297956952 conn(0x7fc320077210 msgr2=0x7fc3200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.286+0000 7fc324c54640 1 -- 172.21.15.1:0/1297956952 shutdown_connections 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.286+0000 7fc324c54640 1 -- 172.21.15.1:0/1297956952 wait complete. 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.287+0000 7fc324c54640 1 Processor -- start 2024-09-16T00:32:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.287+0000 7fc324c54640 1 -- start start 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.287+0000 7fc324c54640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc320101240 0x7fc320073970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.287+0000 7fc324c54640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320101bc0 0x7fc320073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.287+0000 7fc324c54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320105a90 0x7fc32006cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.287+0000 7fc324c54640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc320110f70 con 0x7fc320105a90 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.287+0000 7fc324c54640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc320110df0 con 0x7fc320101240 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.287+0000 7fc324c54640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc3201110f0 con 0x7fc320101bc0 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320105a90 0x7fc32006cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320105a90 0x7fc32006cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37642/0 (socket says 172.21.15.1:37642) 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31ed9d640 1 -- 172.21.15.1:0/2862099138 learned_addr learned my addr 172.21.15.1:0/2862099138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31e59c640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc320101240 0x7fc320073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320101bc0 0x7fc320073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31ed9d640 1 -- 172.21.15.1:0/2862099138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320101bc0 msgr2=0x7fc320073eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31ed9d640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320101bc0 0x7fc320073eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31ed9d640 1 -- 172.21.15.1:0/2862099138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc320101240 msgr2=0x7fc320073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31ed9d640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc320101240 0x7fc320073970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31ed9d640 1 -- 172.21.15.1:0/2862099138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32006d470 con 0x7fc320105a90 2024-09-16T00:32:51.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc31ed9d640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320105a90 0x7fc32006cbb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc31800eaa0 tx=0x7fc31800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc30f7fe640 1 -- 172.21.15.1:0/2862099138 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31800cdb0 con 0x7fc320105a90 2024-09-16T00:32:51.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc30f7fe640 1 -- 172.21.15.1:0/2862099138 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc318005020 con 0x7fc320105a90 2024-09-16T00:32:51.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc324c54640 1 -- 172.21.15.1:0/2862099138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32006d760 con 0x7fc320105a90 2024-09-16T00:32:51.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc30f7fe640 1 -- 172.21.15.1:0/2862099138 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3180116b0 con 0x7fc320105a90 2024-09-16T00:32:51.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.288+0000 7fc324c54640 1 -- 172.21.15.1:0/2862099138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc320071780 con 0x7fc320105a90 2024-09-16T00:32:51.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.291+0000 7fc30f7fe640 1 -- 172.21.15.1:0/2862099138 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc318005560 con 0x7fc320105a90 2024-09-16T00:32:51.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc32702d740 1 Processor -- start 2024-09-16T00:32:51.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc32702d740 1 -- start start 2024-09-16T00:32:51.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc32702d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfcb40 0x55ce63bfb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc32702d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfb5d0 0x55ce63bfba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc32702d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c0a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc32702d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ce63b612f0 con 0x55ce63bfb5d0 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc32702d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ce63b08cf0 con 0x55ce63bfcb40 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc32702d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ce63b5d2b0 con 0x55ce63c08580 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc31e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfb5d0 0x55ce63bfba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc31e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfb5d0 0x55ce63bfba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37656/0 (socket says 172.21.15.1:37656) 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc31e59c640 1 -- 172.21.15.1:0/229952910 learned_addr learned my addr 172.21.15.1:0/229952910 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc30f7fe640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc3080a32c0 0x7fc3080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.292+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfcb40 0x55ce63bfb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31ed9d640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc3080a32c0 0x7fc3080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c0a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31e59c640 1 -- 172.21.15.1:0/229952910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 msgr2=0x55ce63c0a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:51.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31e59c640 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c0a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc30f7fe640 1 -- 172.21.15.1:0/2862099138 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc3180cff40 con 0x7fc320105a90 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31e59c640 1 -- 172.21.15.1:0/229952910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfcb40 msgr2=0x55ce63bfb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31e59c640 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfcb40 0x55ce63bfb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31e59c640 1 -- 172.21.15.1:0/229952910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce63aed3a0 con 0x55ce63bfb5d0 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c0a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31e59c640 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfb5d0 0x55ce63bfba10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc31000aba0 tx=0x7fc31000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4dcd318a816748 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc30cff9640 1 -- 172.21.15.1:0/229952910 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31001d070 con 0x55ce63bfb5d0 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc31ed9d640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc3080a32c0 0x7fc3080a5780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc318005510 tx=0x7fc318066b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc30cff9640 1 -- 172.21.15.1:0/229952910 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc310018b20 con 0x55ce63bfb5d0 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.293+0000 7fc30cff9640 1 -- 172.21.15.1:0/229952910 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310010b30 con 0x55ce63bfb5d0 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 -- 172.21.15.1:0/229952910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfb5d0 msgr2=0x55ce63bfba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfb5d0 0x55ce63bfba10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc31000aba0 tx=0x7fc31000f790 comp rx=0 tx=0).stop 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 -- 172.21.15.1:0/229952910 shutdown_connections 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c0a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfb5d0 0x55ce63bfba10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 --2- 172.21.15.1:0/229952910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfcb40 0x55ce63bfb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 -- 172.21.15.1:0/229952910 >> 172.21.15.1:0/229952910 conn(0x55ce63bc4310 msgr2=0x55ce63bf7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 -- 172.21.15.1:0/229952910 shutdown_connections 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 -- 172.21.15.1:0/229952910 wait complete. 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.294+0000 7fc32702d740 1 -- start start 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc32702d740 1 Processor -- start 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc32702d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfb5d0 0x55ce63c07f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc32702d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc32702d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc32702d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ce63c0b280 con 0x55ce63bfcb40 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc32702d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ce63b1c060 con 0x55ce63bfb5d0 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc32702d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ce63c0b400 con 0x55ce63c08580 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc32702d740 10 client.0 _ll_register_callbacks cb 0x55ce6386f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37668/0 (socket says 172.21.15.1:37668) 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31e59c640 1 -- 172.21.15.1:0/298553558 learned_addr learned my addr 172.21.15.1:0/298553558 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfb5d0 0x55ce63c07f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31e59c640 1 -- 172.21.15.1:0/298553558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 msgr2=0x55ce63c057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31e59c640 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31e59c640 1 -- 172.21.15.1:0/298553558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfb5d0 msgr2=0x55ce63c07f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31e59c640 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfb5d0 0x55ce63c07f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31e59c640 1 -- 172.21.15.1:0/298553558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce638d23c0 con 0x55ce63bfcb40 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.295+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc31e59c640 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c051f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc31000a7a0 tx=0x7fc310018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ab12e8fc24294b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc305ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc305ffb640 1 -- 172.21.15.1:0/298553558 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31001d050 con 0x55ce63bfcb40 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc32702d740 1 -- 172.21.15.1:0/298553558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc310018b20 con 0x55ce63bfcb40 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc305ffb640 1 -- 172.21.15.1:0/298553558 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc310010870 con 0x55ce63bfcb40 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc305ffb640 1 -- 172.21.15.1:0/298553558 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc310031050 con 0x55ce63bfcb40 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc32702d740 10 client.16254 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc32702d740 1 -- 172.21.15.1:0/298553558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce63c16c80 con 0x55ce63bfcb40 2024-09-16T00:32:51.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc305ffb640 1 -- 172.21.15.1:0/298553558 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc310018b20 con 0x55ce63bfcb40 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc305ffb640 1 -- 172.21.15.1:0/298553558 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc3100046c0 con 0x55ce63bfcb40 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc32702d740 10 client.16254 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc32702d740 10 client.16254 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc32702d740 20 client.16254 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc32702d740 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ce63c18030 0x55ce63c1a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc32702d740 4 client.16254 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.296+0000 7fc32702d740 1 -- 172.21.15.1:0/298553558 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce63b6e860 con 0x55ce63c18030 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.297+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ce63c18030 0x55ce63c1a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.297+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ce63c18030 0x55ce63c1a420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1d736d98949eafb server_cookie=b7d6c5770b23c329 in_seq=0 out_seq=0 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.297+0000 7fc305ffb640 10 client.16254 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:32:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc305ffb640 1 -- 172.21.15.1:0/298553558 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce63b6e860 con 0x55ce63c18030 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc305ffb640 10 client.16254 handle_command_reply: tid=0 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 client.16254 shutdown 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 -- 172.21.15.1:0/298553558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 msgr2=0x55ce63c051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c051f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc31000a7a0 tx=0x7fc310018670 comp rx=0 tx=0).stop 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 -- 172.21.15.1:0/298553558 shutdown_connections 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63c08580 0x55ce63c057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c051f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ce63c18030 0x55ce63c1a420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc305ffb640 0 client.16254 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 --2- 172.21.15.1:0/298553558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63bfb5d0 0x55ce63c07f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 -- 172.21.15.1:0/298553558 >> 172.21.15.1:0/298553558 conn(0x55ce63bc4310 msgr2=0x55ce63bfd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 -- 172.21.15.1:0/298553558 shutdown_connections 2024-09-16T00:32:51.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.298+0000 7fc32702d740 1 -- 172.21.15.1:0/298553558 wait complete. 2024-09-16T00:32:51.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.299+0000 7fc32702d740 20 client.16254 trim_cache size 0 max 16384 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc32702d740 1 Processor -- start 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc32702d740 1 -- start start 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc32702d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 0x55ce63c14550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc32702d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c04ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc32702d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63c08580 0x55ce63c041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc32702d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3201b7dc0 con 0x55ce63bfcb40 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc32702d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3201b5690 con 0x55ce63c08580 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc32702d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc3201b34e0 con 0x55ce63bfb5d0 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c04ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c04ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37678/0 (socket says 172.21.15.1:37678) 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31e59c640 1 -- 172.21.15.1:0/3499234892 learned_addr learned my addr 172.21.15.1:0/3499234892 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/3499234892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 0x55ce63c14550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31e59c640 1 -- 172.21.15.1:0/3499234892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 msgr2=0x55ce63c14550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31e59c640 1 --2- 172.21.15.1:0/3499234892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 0x55ce63c14550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31e59c640 1 -- 172.21.15.1:0/3499234892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63c08580 msgr2=0x55ce63c041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31e59c640 1 --2- 172.21.15.1:0/3499234892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63c08580 0x55ce63c041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31e59c640 1 -- 172.21.15.1:0/3499234892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce63bf6310 con 0x55ce63bfcb40 2024-09-16T00:32:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.312+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/3499234892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 0x55ce63c14550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.313+0000 7fc31e59c640 1 --2- 172.21.15.1:0/3499234892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c04ab0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc31000a7a0 tx=0x7fc31000fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8aea14e9f3ddd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.313+0000 7fc307fff640 1 -- 172.21.15.1:0/3499234892 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310039070 con 0x55ce63bfcb40 2024-09-16T00:32:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.313+0000 7fc307fff640 1 -- 172.21.15.1:0/3499234892 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc31001e040 con 0x55ce63bfcb40 2024-09-16T00:32:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.313+0000 7fc307fff640 1 -- 172.21.15.1:0/3499234892 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31001d070 con 0x55ce63bfcb40 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.313+0000 7fc32702d740 1 -- 172.21.15.1:0/3499234892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 msgr2=0x55ce63c04ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.313+0000 7fc32702d740 1 --2- 172.21.15.1:0/3499234892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c04ab0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc31000a7a0 tx=0x7fc31000fdc0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 -- 172.21.15.1:0/3499234892 shutdown_connections 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 --2- 172.21.15.1:0/3499234892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63c08580 0x55ce63c041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 --2- 172.21.15.1:0/3499234892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c04ab0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 --2- 172.21.15.1:0/3499234892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 0x55ce63c14550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 -- 172.21.15.1:0/3499234892 >> 172.21.15.1:0/3499234892 conn(0x55ce63bc4310 msgr2=0x55ce63c03ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 -- 172.21.15.1:0/3499234892 shutdown_connections 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 -- 172.21.15.1:0/3499234892 wait complete. 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 -- start start 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 Processor -- start 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 0x55ce63c10bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c0bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63c08580 0x55ce63c0c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3201b3b50 con 0x55ce63bfcb40 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3201b4fc0 con 0x55ce63c08580 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc3201b4220 con 0x55ce63bfb5d0 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.314+0000 7fc32702d740 10 client.0 _ll_register_callbacks cb 0x7fc31001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c0bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c0bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37692/0 (socket says 172.21.15.1:37692) 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31e59c640 1 -- 172.21.15.1:0/4172205383 learned_addr learned my addr 172.21.15.1:0/4172205383 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 0x55ce63c10bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63c08580 0x55ce63c0c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31e59c640 1 -- 172.21.15.1:0/4172205383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 msgr2=0x55ce63c10bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31e59c640 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 0x55ce63c10bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31e59c640 1 -- 172.21.15.1:0/4172205383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63c08580 msgr2=0x55ce63c0c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31e59c640 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63c08580 0x55ce63c0c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31e59c640 1 -- 172.21.15.1:0/4172205383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce63bf7510 con 0x55ce63bfcb40 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc31e59c640 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c0bf20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc310011000 tx=0x7fc31000a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c64bafb60cd7009 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc305ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc305ffb640 1 -- 172.21.15.1:0/4172205383 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310039070 con 0x55ce63bfcb40 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc305ffb640 1 -- 172.21.15.1:0/4172205383 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc310004660 con 0x55ce63bfcb40 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc32702d740 1 -- 172.21.15.1:0/4172205383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc31001e040 con 0x55ce63bfcb40 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc305ffb640 1 -- 172.21.15.1:0/4172205383 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc31004a050 con 0x55ce63bfcb40 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc32702d740 10 client.16260 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.315+0000 7fc32702d740 1 -- 172.21.15.1:0/4172205383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce63c1a880 con 0x55ce63bfcb40 2024-09-16T00:32:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.316+0000 7fc305ffb640 1 -- 172.21.15.1:0/4172205383 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc31001e040 con 0x55ce63bfcb40 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.316+0000 7fc305ffb640 1 -- 172.21.15.1:0/4172205383 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc310045030 con 0x55ce63bfcb40 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.316+0000 7fc32702d740 10 client.16260 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.316+0000 7fc32702d740 10 client.16260 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.316+0000 7fc32702d740 20 client.16260 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.316+0000 7fc32702d740 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ce63c18030 0x55ce63c84aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.316+0000 7fc32702d740 4 client.16260 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.316+0000 7fc32702d740 1 -- 172.21.15.1:0/4172205383 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce63b68d70 con 0x55ce63c18030 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.316+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ce63c18030 0x55ce63c84aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc31dd9b640 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ce63c18030 0x55ce63c84aa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ca71b77cb5dbe53 server_cookie=7791eaab3b455af8 in_seq=0 out_seq=0 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc305ffb640 10 client.16260 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc305ffb640 1 -- 172.21.15.1:0/4172205383 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce63b68d70 con 0x55ce63c18030 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc305ffb640 10 client.16260 handle_command_reply: tid=0 2024-09-16T00:32:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 client.16260 shutdown 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 -- 172.21.15.1:0/4172205383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 msgr2=0x55ce63c0bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c0bf20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc310011000 tx=0x7fc31000a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 -- 172.21.15.1:0/4172205383 shutdown_connections 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ce63c18030 0x55ce63c84aa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce63c08580 0x55ce63c0c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce63bfcb40 0x55ce63c0bf20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 --2- 172.21.15.1:0/4172205383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce63bfb5d0 0x55ce63c10bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 -- 172.21.15.1:0/4172205383 >> 172.21.15.1:0/4172205383 conn(0x55ce63bc4310 msgr2=0x55ce63bf6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc305ffb640 0 client.16260 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 -- 172.21.15.1:0/4172205383 shutdown_connections 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 1 -- 172.21.15.1:0/4172205383 wait complete. 2024-09-16T00:32:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.317+0000 7fc32702d740 20 client.16260 trim_cache size 0 max 16384 2024-09-16T00:32:51.325 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:32:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 -- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc3080a32c0 msgr2=0x7fc3080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc3080a32c0 0x7fc3080a5780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc318005510 tx=0x7fc318066b40 comp rx=0 tx=0).stop 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 -- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320105a90 msgr2=0x7fc32006cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320105a90 0x7fc32006cbb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc31800eaa0 tx=0x7fc31800ef60 comp rx=0 tx=0).stop 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 -- 172.21.15.1:0/2862099138 shutdown_connections 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc320105a90 0x7fc32006cbb0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc3080a32c0 0x7fc3080a5780 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320101bc0 0x7fc320073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 --2- 172.21.15.1:0/2862099138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc320101240 0x7fc320073970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 -- 172.21.15.1:0/2862099138 >> 172.21.15.1:0/2862099138 conn(0x7fc320077210 msgr2=0x7fc320103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 -- 172.21.15.1:0/2862099138 shutdown_connections 2024-09-16T00:32:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:51.318+0000 7fc324c54640 1 -- 172.21.15.1:0/2862099138 wait complete. 2024-09-16T00:32:52.268 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:32:52.531 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:52.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:52 smithi019 ceph-mon[35186]: pgmap v484: 129 pgs: 129 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:52.640 INFO:tasks.workunit.client.0.smithi001.stdout:64/600: open fd = 3 2024-09-16T00:32:52.641 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.641 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.641 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.641 INFO:tasks.workunit.client.0.smithi001.stdout:65/600: open fd = 3 2024-09-16T00:32:52.641 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.641 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.641 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:66/600: open fd = 3 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:67/600: open fd = 3 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:68/600: open fd = 3 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:69/600: open fd = 3 2024-09-16T00:32:52.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:70/600: open fd = 3 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:71/600: open fd = 3 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:72/600: open fd = 3 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.643 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:73/600: open fd = 3 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:74/600: open fd = 3 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:75/600: open fd = 3 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:76/600: open fd = 3 2024-09-16T00:32:52.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:77/600: open fd = 3 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:78/600: open fd = 3 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:79/600: open fd = 3 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:80/600: open fd = 3 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:81/600: open fd = 3 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:82/600: open fd = 3 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:83/600: open fd = 3 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:84/600: open fd = 3 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:85/600: open fd = 3 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.647 INFO:tasks.workunit.client.0.smithi001.stdout:86/600: open fd = 3 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:87/600: open fd = 3 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:88/600: open fd = 3 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:89/600: open fd = 3 2024-09-16T00:32:52.648 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:90/600: open fd = 3 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:91/600: open fd = 3 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:92/600: open fd = 3 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.649 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:93/600: open fd = 3 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:94/600: open fd = 3 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:95/600: open fd = 3 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.650 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:96/600: open fd = 3 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:97/600: open fd = 3 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:98/600: open fd = 3 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.651 INFO:tasks.workunit.client.0.smithi001.stdout:99/600: open fd = 3 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:100/600: open fd = 3 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:101/600: open fd = 3 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.652 INFO:tasks.workunit.client.0.smithi001.stdout:102/600: open fd = 3 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:103/600: open fd = 3 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:104/600: open fd = 3 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:105/600: open fd = 3 2024-09-16T00:32:52.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:106/600: open fd = 3 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:107/600: open fd = 3 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:108/600: open fd = 3 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.654 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:109/600: open fd = 3 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:110/600: open fd = 3 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:111/600: open fd = 3 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:112/600: open fd = 3 2024-09-16T00:32:52.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:113/600: open fd = 3 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:114/600: open fd = 3 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:115/600: open fd = 3 2024-09-16T00:32:52.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:116/600: open fd = 3 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:117/600: open fd = 3 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:118/600: open fd = 3 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.657 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:119/600: open fd = 3 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:120/600: open fd = 3 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:121/600: open fd = 3 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:122/600: open fd = 3 2024-09-16T00:32:52.658 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:123/600: open fd = 3 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:124/600: open fd = 3 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:125/600: open fd = 3 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.659 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.660 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:32:52.660 INFO:tasks.workunit.client.0.smithi001.stdout:126/600: open fd = 3 2024-09-16T00:32:52.660 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:32:52.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:52 smithi001 ceph-mon[30934]: pgmap v484: 129 pgs: 129 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:52.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:52 smithi153 ceph-mon[35538]: pgmap v484: 129 pgs: 129 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:32:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.074+0000 7fa903290640 1 -- 172.21.15.1:0/4132129559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 msgr2=0x7fa8fc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.074+0000 7fa903290640 1 --2- 172.21.15.1:0/4132129559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 0x7fa8fc10ea90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009ca0 tx=0x7fa8f402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.074+0000 7fa903290640 1 -- 172.21.15.1:0/4132129559 shutdown_connections 2024-09-16T00:32:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.074+0000 7fa903290640 1 --2- 172.21.15.1:0/4132129559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 0x7fa8fc10ea90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.074+0000 7fa903290640 1 --2- 172.21.15.1:0/4132129559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8fc102790 0x7fa8fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.074+0000 7fa903290640 1 --2- 172.21.15.1:0/4132129559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc101de0 0x7fa8fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.074+0000 7fa903290640 1 -- 172.21.15.1:0/4132129559 >> 172.21.15.1:0/4132129559 conn(0x7fa8fc0fd8c0 msgr2=0x7fa8fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.074+0000 7fa903290640 1 -- 172.21.15.1:0/4132129559 shutdown_connections 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.074+0000 7fa903290640 1 -- 172.21.15.1:0/4132129559 wait complete. 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa903290640 1 Processor -- start 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa903290640 1 -- start start 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa903290640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8fc101de0 0x7fa8fc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa903290640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc102790 0x7fa8fc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa903290640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 0x7fa8fc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa903290640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc111f70 con 0x7fa8fc106f10 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa903290640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc111df0 con 0x7fa8fc101de0 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa903290640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1120f0 con 0x7fa8fc102790 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa90182d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 0x7fa8fc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa90182d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 0x7fa8fc19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37710/0 (socket says 172.21.15.1:37710) 2024-09-16T00:32:53.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa90102c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8fc101de0 0x7fa8fc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.075+0000 7fa90182d640 1 -- 172.21.15.1:0/3773975471 learned_addr learned my addr 172.21.15.1:0/3773975471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa90082b640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc102790 0x7fa8fc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa90182d640 1 -- 172.21.15.1:0/3773975471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc102790 msgr2=0x7fa8fc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa90182d640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc102790 0x7fa8fc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa90182d640 1 -- 172.21.15.1:0/3773975471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8fc101de0 msgr2=0x7fa8fc198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa90182d640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8fc101de0 0x7fa8fc198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa90182d640 1 -- 172.21.15.1:0/3773975471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8fc19d2e0 con 0x7fa8fc106f10 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa90182d640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 0x7fa8fc19cc00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4031a20 tx=0x7fa8f4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa8f27fc640 1 -- 172.21.15.1:0/3773975471 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4003b70 con 0x7fa8fc106f10 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa8f27fc640 1 -- 172.21.15.1:0/3773975471 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f4003d10 con 0x7fa8fc106f10 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa8f27fc640 1 -- 172.21.15.1:0/3773975471 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f402fc80 con 0x7fa8fc106f10 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa903290640 1 -- 172.21.15.1:0/3773975471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8fc19d570 con 0x7fa8fc106f10 2024-09-16T00:32:53.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.076+0000 7fa903290640 1 -- 172.21.15.1:0/3773975471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8fc1a4eb0 con 0x7fa8fc106f10 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa8f27fc640 1 -- 172.21.15.1:0/3773975471 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa8f40046a0 con 0x7fa8fc106f10 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa905669740 1 Processor -- start 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa905669740 1 -- start start 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa905669740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564016b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa905669740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564016ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa905669740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x555640178760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa905669740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5556400d12f0 con 0x55564016b5d0 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa905669740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555640078cf0 con 0x555640176370 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa905669740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5556400cd2b0 con 0x55564016cb40 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa90102c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564016ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa90102c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564016ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37724/0 (socket says 172.21.15.1:37724) 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa90102c640 1 -- 172.21.15.1:0/1535484170 learned_addr learned my addr 172.21.15.1:0/1535484170 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.079+0000 7fa90082b640 1 --2- 172.21.15.1:0/1535484170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x555640178760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa90082b640 1 --2- 172.21.15.1:0/1535484170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564016b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa90102c640 1 -- 172.21.15.1:0/1535484170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 msgr2=0x55564016b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa90102c640 1 --2- 172.21.15.1:0/1535484170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564016b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa90102c640 1 -- 172.21.15.1:0/1535484170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 msgr2=0x555640178760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa90102c640 1 --2- 172.21.15.1:0/1535484170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x555640178760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa90102c640 1 -- 172.21.15.1:0/1535484170 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564005d3a0 con 0x55564016b5d0 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa8f27fc640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa8e80a3240 0x7fa8e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa90102c640 1 --2- 172.21.15.1:0/1535484170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564016ba10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800ad00 tx=0x7fa8f800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b06af67dbb19b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa90082b640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa8e80a3240 0x7fa8e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa8f27fc640 1 -- 172.21.15.1:0/3773975471 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa8f40f11a0 con 0x7fa8fc106f10 2024-09-16T00:32:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa8e77fe640 1 -- 172.21.15.1:0/1535484170 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f801d070 con 0x55564016b5d0 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa8e77fe640 1 -- 172.21.15.1:0/1535484170 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f8018b20 con 0x55564016b5d0 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.080+0000 7fa8e77fe640 1 -- 172.21.15.1:0/1535484170 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8010b80 con 0x55564016b5d0 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 -- 172.21.15.1:0/1535484170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 msgr2=0x55564016ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 --2- 172.21.15.1:0/1535484170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564016ba10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800ad00 tx=0x7fa8f800f790 comp rx=0 tx=0).stop 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa90082b640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa8e80a3240 0x7fa8e80a5700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55564016c070 tx=0x7fa8ec016040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 -- 172.21.15.1:0/1535484170 shutdown_connections 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 --2- 172.21.15.1:0/1535484170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x555640178760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 --2- 172.21.15.1:0/1535484170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564016ba10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 --2- 172.21.15.1:0/1535484170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564016b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 -- 172.21.15.1:0/1535484170 >> 172.21.15.1:0/1535484170 conn(0x555640134310 msgr2=0x555640167c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 -- 172.21.15.1:0/1535484170 shutdown_connections 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 -- 172.21.15.1:0/1535484170 wait complete. 2024-09-16T00:32:53.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 -- start start 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.081+0000 7fa905669740 1 Processor -- start 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa905669740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564016b5d0 0x555640175d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa905669740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016cb40 0x555640173010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa905669740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555640176370 0x555640173600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa905669740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555640179070 con 0x55564016cb40 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa905669740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55564008c060 con 0x55564016b5d0 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa905669740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5556401791f0 con 0x555640176370 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90082b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016cb40 0x555640173010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90082b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016cb40 0x555640173010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37728/0 (socket says 172.21.15.1:37728) 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa905669740 10 client.0 _ll_register_callbacks cb 0x55563fddf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90082b640 1 -- 172.21.15.1:0/3320251249 learned_addr learned my addr 172.21.15.1:0/3320251249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90102c640 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564016b5d0 0x555640175d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90182d640 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555640176370 0x555640173600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90082b640 1 -- 172.21.15.1:0/3320251249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555640176370 msgr2=0x555640173600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90082b640 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555640176370 0x555640173600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90082b640 1 -- 172.21.15.1:0/3320251249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564016b5d0 msgr2=0x555640175d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90082b640 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564016b5d0 0x555640175d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.082+0000 7fa90082b640 1 -- 172.21.15.1:0/3320251249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563fe423c0 con 0x55564016cb40 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa90082b640 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016cb40 0x555640173010 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec011b50 tx=0x7fa8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27982bd7d06c9eb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa8e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/3320251249 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec025070 con 0x55564016cb40 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/3320251249 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa8ec004020 con 0x55564016cb40 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa905669740 1 -- 172.21.15.1:0/3320251249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f8018b20 con 0x55564016cb40 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/3320251249 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8ec030050 con 0x55564016cb40 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa905669740 10 client.16272 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa905669740 1 -- 172.21.15.1:0/3320251249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555640184b70 con 0x55564016cb40 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/3320251249 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f8018b20 con 0x55564016cb40 2024-09-16T00:32:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/3320251249 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa8ec01e050 con 0x55564016cb40 2024-09-16T00:32:53.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa905669740 10 client.16272 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:53.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa905669740 10 client.16272 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:32:53.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa905669740 20 client.16272 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:53.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa905669740 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555640186020 0x555640188480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa905669740 4 client.16272 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:53.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.083+0000 7fa905669740 1 -- 172.21.15.1:0/3320251249 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556400de860 con 0x555640186020 2024-09-16T00:32:53.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.084+0000 7fa90102c640 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555640186020 0x555640188480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.084+0000 7fa90102c640 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555640186020 0x555640188480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8ef3652ddf5fe79 server_cookie=b092821c02f0b267 in_seq=0 out_seq=0 2024-09-16T00:32:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.084+0000 7fa8e4ff9640 10 client.16272 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:32:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/3320251249 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556400de860 con 0x555640186020 2024-09-16T00:32:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa8e4ff9640 10 client.16272 handle_command_reply: tid=0 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 client.16272 shutdown 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 -- 172.21.15.1:0/3320251249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016cb40 msgr2=0x555640173010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016cb40 0x555640173010 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec011b50 tx=0x7fa8ec0046c0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 -- 172.21.15.1:0/3320251249 shutdown_connections 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555640176370 0x555640173600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555640186020 0x555640188480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016cb40 0x555640173010 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 --2- 172.21.15.1:0/3320251249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564016b5d0 0x555640175d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 -- 172.21.15.1:0/3320251249 >> 172.21.15.1:0/3320251249 conn(0x555640134310 msgr2=0x55564016d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa8e4ff9640 0 client.16272 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 -- 172.21.15.1:0/3320251249 shutdown_connections 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 1 -- 172.21.15.1:0/3320251249 wait complete. 2024-09-16T00:32:53.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.085+0000 7fa905669740 20 client.16272 trim_cache size 0 max 16384 2024-09-16T00:32:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa905669740 1 Processor -- start 2024-09-16T00:32:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa905669740 1 -- start start 2024-09-16T00:32:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa905669740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x555640182310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa905669740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564016e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa905669740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x55564016e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa905669740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1b1690 con 0x55564016b5d0 2024-09-16T00:32:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa905669740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1aef60 con 0x555640176370 2024-09-16T00:32:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa905669740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1acdb0 con 0x55564016cb40 2024-09-16T00:32:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90102c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x555640182310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90102c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x555640182310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37736/0 (socket says 172.21.15.1:37736) 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90102c640 1 -- 172.21.15.1:0/249885512 learned_addr learned my addr 172.21.15.1:0/249885512 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90082b640 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564016e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90182d640 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x55564016e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90102c640 1 -- 172.21.15.1:0/249885512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 msgr2=0x55564016e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90102c640 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564016e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90102c640 1 -- 172.21.15.1:0/249885512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 msgr2=0x55564016e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90102c640 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x55564016e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90102c640 1 -- 172.21.15.1:0/249885512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555640176090 con 0x55564016b5d0 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.099+0000 7fa90182d640 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x55564016e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa90102c640 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x555640182310 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8010510 tx=0x7fa8f800a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59be7ab7e9dd03d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa8e6ffd640 1 -- 172.21.15.1:0/249885512 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f801d040 con 0x55564016b5d0 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa8e6ffd640 1 -- 172.21.15.1:0/249885512 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f802f070 con 0x55564016b5d0 2024-09-16T00:32:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa8e6ffd640 1 -- 172.21.15.1:0/249885512 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8010660 con 0x55564016b5d0 2024-09-16T00:32:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa905669740 1 -- 172.21.15.1:0/249885512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 msgr2=0x555640182310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa905669740 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x555640182310 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8010510 tx=0x7fa8f800a510 comp rx=0 tx=0).stop 2024-09-16T00:32:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa905669740 1 -- 172.21.15.1:0/249885512 shutdown_connections 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa905669740 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x55564016e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa905669740 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564016e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa905669740 1 --2- 172.21.15.1:0/249885512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x555640182310 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.100+0000 7fa905669740 1 -- 172.21.15.1:0/249885512 >> 172.21.15.1:0/249885512 conn(0x555640134310 msgr2=0x555640172140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 -- 172.21.15.1:0/249885512 shutdown_connections 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 -- 172.21.15.1:0/249885512 wait complete. 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 -- start start 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 Processor -- start 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564017e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564017eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x55564016f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1ad420 con 0x55564016b5d0 2024-09-16T00:32:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1ae890 con 0x555640176370 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1adaf0 con 0x55564016cb40 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa905669740 10 client.0 _ll_register_callbacks cb 0x7fa8ec00cd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa90102c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564017e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa90102c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564017e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37738/0 (socket says 172.21.15.1:37738) 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.101+0000 7fa90102c640 1 -- 172.21.15.1:0/634410057 learned_addr learned my addr 172.21.15.1:0/634410057 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa90182d640 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x55564016f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa90082b640 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564017eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa90102c640 1 -- 172.21.15.1:0/634410057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 msgr2=0x55564017eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa90102c640 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564017eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa90102c640 1 -- 172.21.15.1:0/634410057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 msgr2=0x55564016f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa90102c640 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x55564016f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa90102c640 1 -- 172.21.15.1:0/634410057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55564016ffe0 con 0x55564016b5d0 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa90102c640 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564017e660 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa8f802c040 tx=0x7fa8f800a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d54be7857f81eaa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa8e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/634410057 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8038060 con 0x55564016b5d0 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa905669740 1 -- 172.21.15.1:0/634410057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f802f070 con 0x55564016b5d0 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/634410057 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa8f8005040 con 0x55564016b5d0 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/634410057 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8f801d030 con 0x55564016b5d0 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa905669740 10 client.16278 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.102+0000 7fa905669740 1 -- 172.21.15.1:0/634410057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5556401f0760 con 0x55564016b5d0 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.103+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/634410057 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f802f070 con 0x55564016b5d0 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.103+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/634410057 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa8f8010bb0 con 0x55564016b5d0 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.103+0000 7fa905669740 10 client.16278 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.103+0000 7fa905669740 10 client.16278 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.103+0000 7fa905669740 20 client.16278 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.103+0000 7fa905669740 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5556401f0c90 0x5556401f3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.103+0000 7fa905669740 4 client.16278 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.103+0000 7fa905669740 1 -- 172.21.15.1:0/634410057 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556400d8d70 con 0x5556401f0c90 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.103+0000 7fa90082b640 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5556401f0c90 0x5556401f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa90082b640 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5556401f0c90 0x5556401f3080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3111b418be478fdc server_cookie=25e90e9e37d9bddb in_seq=0 out_seq=0 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa8e4ff9640 10 client.16278 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa8e4ff9640 1 -- 172.21.15.1:0/634410057 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556400d8d70 con 0x5556401f0c90 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa8e4ff9640 10 client.16278 handle_command_reply: tid=0 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 client.16278 shutdown 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 -- 172.21.15.1:0/634410057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 msgr2=0x55564017e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564017e660 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa8f802c040 tx=0x7fa8f800a880 comp rx=0 tx=0).stop 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 -- 172.21.15.1:0/634410057 shutdown_connections 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555640176370 0x55564016f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55564016cb40 0x55564017eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5556401f0c90 0x5556401f3080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 --2- 172.21.15.1:0/634410057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55564016b5d0 0x55564017e660 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa8e4ff9640 0 client.16278 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:32:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 -- 172.21.15.1:0/634410057 >> 172.21.15.1:0/634410057 conn(0x555640134310 msgr2=0x5556401664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 -- 172.21.15.1:0/634410057 shutdown_connections 2024-09-16T00:32:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 1 -- 172.21.15.1:0/634410057 wait complete. 2024-09-16T00:32:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.104+0000 7fa905669740 20 client.16278 trim_cache size 0 max 16384 2024-09-16T00:32:53.110 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:32:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 -- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa8e80a3240 msgr2=0x7fa8e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa8e80a3240 0x7fa8e80a5700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55564016c070 tx=0x7fa8ec016040 comp rx=0 tx=0).stop 2024-09-16T00:32:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 -- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 msgr2=0x7fa8fc19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 0x7fa8fc19cc00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4031a20 tx=0x7fa8f4031a50 comp rx=0 tx=0).stop 2024-09-16T00:32:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 -- 172.21.15.1:0/3773975471 shutdown_connections 2024-09-16T00:32:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8fc106f10 0x7fa8fc19cc00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa8e80a3240 0x7fa8e80a5700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc102790 0x7fa8fc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 --2- 172.21.15.1:0/3773975471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8fc101de0 0x7fa8fc198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 -- 172.21.15.1:0/3773975471 >> 172.21.15.1:0/3773975471 conn(0x7fa8fc0fd8c0 msgr2=0x7fa8fc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 -- 172.21.15.1:0/3773975471 shutdown_connections 2024-09-16T00:32:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:53.105+0000 7fa903290640 1 -- 172.21.15.1:0/3773975471 wait complete. 2024-09-16T00:32:53.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:32:54.250 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:54.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:54 smithi001 ceph-mon[30934]: pgmap v485: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:54 smithi019 ceph-mon[35186]: pgmap v485: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:54.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:54 smithi153 ceph-mon[35538]: pgmap v485: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:32:54.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.776+0000 7fc0892b8640 1 -- 172.21.15.1:0/2871163028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084105a90 msgr2=0x7fc084105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.776+0000 7fc0892b8640 1 --2- 172.21.15.1:0/2871163028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084105a90 0x7fc084105e70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc074009900 tx=0x7fc07402a410 comp rx=0 tx=0).stop 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.776+0000 7fc0892b8640 1 -- 172.21.15.1:0/2871163028 shutdown_connections 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.776+0000 7fc0892b8640 1 --2- 172.21.15.1:0/2871163028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc084101bc0 0x7fc08410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.776+0000 7fc0892b8640 1 --2- 172.21.15.1:0/2871163028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc084101240 0x7fc084101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.776+0000 7fc0892b8640 1 --2- 172.21.15.1:0/2871163028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084105a90 0x7fc084105e70 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.776+0000 7fc0892b8640 1 -- 172.21.15.1:0/2871163028 >> 172.21.15.1:0/2871163028 conn(0x7fc084077210 msgr2=0x7fc0840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.776+0000 7fc0892b8640 1 -- 172.21.15.1:0/2871163028 shutdown_connections 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.776+0000 7fc0892b8640 1 -- 172.21.15.1:0/2871163028 wait complete. 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.777+0000 7fc0892b8640 1 Processor -- start 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.777+0000 7fc0892b8640 1 -- start start 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.777+0000 7fc0892b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084101240 0x7fc08419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.777+0000 7fc0892b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc084101bc0 0x7fc08419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.777+0000 7fc0892b8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc084105a90 0x7fc08419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.777+0000 7fc0892b8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc084110fb0 con 0x7fc084101240 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.777+0000 7fc0892b8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc084110e30 con 0x7fc084101bc0 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.777+0000 7fc0892b8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc084111130 con 0x7fc084105a90 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc082ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084101240 0x7fc08419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc082ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084101240 0x7fc08419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37762/0 (socket says 172.21.15.1:37762) 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc082ffd640 1 -- 172.21.15.1:0/3868830584 learned_addr learned my addr 172.21.15.1:0/3868830584 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc0827fc640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc084101bc0 0x7fc08419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc0837fe640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc084105a90 0x7fc08419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc082ffd640 1 -- 172.21.15.1:0/3868830584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc084105a90 msgr2=0x7fc08419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc082ffd640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc084105a90 0x7fc08419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc082ffd640 1 -- 172.21.15.1:0/3868830584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc084101bc0 msgr2=0x7fc08419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc082ffd640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc084101bc0 0x7fc08419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc082ffd640 1 -- 172.21.15.1:0/3868830584 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08419f6a0 con 0x7fc084101240 2024-09-16T00:32:54.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc0837fe640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc084105a90 0x7fc08419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:54.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc082ffd640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084101240 0x7fc08419a6f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc074006010 tx=0x7fc074003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc06bfff640 1 -- 172.21.15.1:0/3868830584 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07403f070 con 0x7fc084101240 2024-09-16T00:32:54.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc06bfff640 1 -- 172.21.15.1:0/3868830584 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc074031c30 con 0x7fc084101240 2024-09-16T00:32:54.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc06bfff640 1 -- 172.21.15.1:0/3868830584 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074035030 con 0x7fc084101240 2024-09-16T00:32:54.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc0892b8640 1 -- 172.21.15.1:0/3868830584 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08419f930 con 0x7fc084101240 2024-09-16T00:32:54.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.778+0000 7fc0892b8640 1 -- 172.21.15.1:0/3868830584 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08419fdc0 con 0x7fc084101240 2024-09-16T00:32:54.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc06bfff640 1 -- 172.21.15.1:0/3868830584 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc074042070 con 0x7fc084101240 2024-09-16T00:32:54.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc08b691740 1 Processor -- start 2024-09-16T00:32:54.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc08b691740 1 -- start start 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc08b691740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 0x564246749090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc08b691740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 0x564246749a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc08b691740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246756760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc08b691740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5642466af2f0 con 0x564246754370 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc08b691740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564246656cf0 con 0x56424674ab40 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc08b691740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642466ab2b0 con 0x5642467495d0 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.781+0000 7fc0827fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246756760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0827fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246756760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37770/0 (socket says 172.21.15.1:37770) 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0827fc640 1 -- 172.21.15.1:0/1941834171 learned_addr learned my addr 172.21.15.1:0/1941834171 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0837fe640 1 --2- 172.21.15.1:0/1941834171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 0x564246749090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc06bfff640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc06c0a3240 0x7fc06c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc082ffd640 1 --2- 172.21.15.1:0/1941834171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 0x564246749a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0827fc640 1 -- 172.21.15.1:0/1941834171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 msgr2=0x564246749a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0827fc640 1 --2- 172.21.15.1:0/1941834171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 0x564246749a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0827fc640 1 -- 172.21.15.1:0/1941834171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 msgr2=0x564246749090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0827fc640 1 --2- 172.21.15.1:0/1941834171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 0x564246749090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0827fc640 1 -- 172.21.15.1:0/1941834171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424663b3a0 con 0x564246754370 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0837fe640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc06c0a3240 0x7fc06c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc06bfff640 1 -- 172.21.15.1:0/3868830584 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc0740f05d0 con 0x7fc084101240 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0827fc640 1 --2- 172.21.15.1:0/1941834171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246756760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc07800bcb0 tx=0x7fc0780107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637d314152662e15 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0697fa640 1 -- 172.21.15.1:0/1941834171 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078004030 con 0x564246754370 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0697fa640 1 -- 172.21.15.1:0/1941834171 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0780041d0 con 0x564246754370 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.782+0000 7fc0697fa640 1 -- 172.21.15.1:0/1941834171 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078011800 con 0x564246754370 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc08b691740 1 -- 172.21.15.1:0/1941834171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 msgr2=0x564246756760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc0837fe640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc06c0a3240 0x7fc06c0a5700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00a4c0 tx=0x7fc07c00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc08b691740 1 --2- 172.21.15.1:0/1941834171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246756760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc07800bcb0 tx=0x7fc0780107a0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc08b691740 1 -- 172.21.15.1:0/1941834171 shutdown_connections 2024-09-16T00:32:54.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc08b691740 1 --2- 172.21.15.1:0/1941834171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246756760 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc08b691740 1 --2- 172.21.15.1:0/1941834171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 0x564246749a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc08b691740 1 --2- 172.21.15.1:0/1941834171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 0x564246749090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc08b691740 1 -- 172.21.15.1:0/1941834171 >> 172.21.15.1:0/1941834171 conn(0x564246712310 msgr2=0x564246745c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc08b691740 1 -- 172.21.15.1:0/1941834171 shutdown_connections 2024-09-16T00:32:54.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.783+0000 7fc08b691740 1 -- 172.21.15.1:0/1941834171 wait complete. 2024-09-16T00:32:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc08b691740 1 -- start start 2024-09-16T00:32:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc08b691740 1 Processor -- start 2024-09-16T00:32:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc08b691740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642467495d0 0x564246753d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc08b691740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56424674ab40 0x564246751000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc08b691740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564246754370 0x5642467515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc08b691740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564246757070 con 0x5642467495d0 2024-09-16T00:32:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc08b691740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56424666a060 con 0x564246754370 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc08b691740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642467571f0 con 0x56424674ab40 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc082ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642467495d0 0x564246753d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc08b691740 10 client.0 _ll_register_callbacks cb 0x5642463bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc082ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642467495d0 0x564246753d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37774/0 (socket says 172.21.15.1:37774) 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc082ffd640 1 -- 172.21.15.1:0/4115801884 learned_addr learned my addr 172.21.15.1:0/4115801884 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc0827fc640 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56424674ab40 0x564246751000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc0837fe640 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564246754370 0x5642467515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc082ffd640 1 -- 172.21.15.1:0/4115801884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56424674ab40 msgr2=0x564246751000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc082ffd640 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56424674ab40 0x564246751000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc082ffd640 1 -- 172.21.15.1:0/4115801884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564246754370 msgr2=0x5642467515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc082ffd640 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564246754370 0x5642467515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.784+0000 7fc082ffd640 1 -- 172.21.15.1:0/4115801884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642464203c0 con 0x5642467495d0 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc082ffd640 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642467495d0 0x564246753d50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc0740c4ed0 tx=0x7fc0740bc560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14926314db373ea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc066ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc066ff5640 1 -- 172.21.15.1:0/4115801884 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074049070 con 0x5642467495d0 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc066ff5640 1 -- 172.21.15.1:0/4115801884 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc0740edd80 con 0x5642467495d0 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc08b691740 1 -- 172.21.15.1:0/4115801884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0780041d0 con 0x5642467495d0 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc066ff5640 1 -- 172.21.15.1:0/4115801884 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc0740bce90 con 0x5642467495d0 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc08b691740 10 client.16290 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc08b691740 1 -- 172.21.15.1:0/4115801884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564246762af0 con 0x5642467495d0 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc066ff5640 1 -- 172.21.15.1:0/4115801884 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0780041d0 con 0x5642467495d0 2024-09-16T00:32:54.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc066ff5640 1 -- 172.21.15.1:0/4115801884 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc0740f0230 con 0x5642467495d0 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc08b691740 10 client.16290 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc08b691740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc08b691740 20 client.16290 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc08b691740 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564246763fa0 0x5642467663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc08b691740 4 client.16290 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.785+0000 7fc08b691740 1 -- 172.21.15.1:0/4115801884 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642466bc860 con 0x564246763fa0 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.786+0000 7fc0827fc640 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564246763fa0 0x5642467663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.786+0000 7fc0827fc640 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564246763fa0 0x5642467663f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b976eaa67675d30f server_cookie=45886bd6689efbc7 in_seq=0 out_seq=0 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.786+0000 7fc066ff5640 10 client.16290 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc066ff5640 1 -- 172.21.15.1:0/4115801884 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642466bc860 con 0x564246763fa0 2024-09-16T00:32:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc066ff5640 10 client.16290 handle_command_reply: tid=0 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 client.16290 shutdown 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 -- 172.21.15.1:0/4115801884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642467495d0 msgr2=0x564246753d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642467495d0 0x564246753d50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc0740c4ed0 tx=0x7fc0740bc560 comp rx=0 tx=0).stop 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 -- 172.21.15.1:0/4115801884 shutdown_connections 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564246763fa0 0x5642467663f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc066ff5640 0 client.16290 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564246754370 0x5642467515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56424674ab40 0x564246751000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 --2- 172.21.15.1:0/4115801884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642467495d0 0x564246753d50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 -- 172.21.15.1:0/4115801884 >> 172.21.15.1:0/4115801884 conn(0x564246712310 msgr2=0x56424674b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 -- 172.21.15.1:0/4115801884 shutdown_connections 2024-09-16T00:32:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 1 -- 172.21.15.1:0/4115801884 wait complete. 2024-09-16T00:32:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.787+0000 7fc08b691740 20 client.16290 trim_cache size 0 max 16384 2024-09-16T00:32:54.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc08b691740 1 Processor -- start 2024-09-16T00:32:54.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc08b691740 1 -- start start 2024-09-16T00:32:54.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc08b691740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642467495d0 0x56424674a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc08b691740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56424674ab40 0x564246760310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc08b691740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246745700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc08b691740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841b3940 con 0x564246754370 2024-09-16T00:32:54.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc08b691740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841b1210 con 0x5642467495d0 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc08b691740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841af060 con 0x56424674ab40 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc0837fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246745700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc0837fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246745700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37788/0 (socket says 172.21.15.1:37788) 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.800+0000 7fc0837fe640 1 -- 172.21.15.1:0/1349314411 learned_addr learned my addr 172.21.15.1:0/1349314411 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc082ffd640 1 --2- 172.21.15.1:0/1349314411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642467495d0 0x56424674a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc0837fe640 1 -- 172.21.15.1:0/1349314411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56424674ab40 msgr2=0x564246760310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc0837fe640 1 --2- 172.21.15.1:0/1349314411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56424674ab40 0x564246760310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc0837fe640 1 -- 172.21.15.1:0/1349314411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642467495d0 msgr2=0x56424674a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc0837fe640 1 --2- 172.21.15.1:0/1349314411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642467495d0 0x56424674a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc0837fe640 1 -- 172.21.15.1:0/1349314411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642467454f0 con 0x564246754370 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc082ffd640 1 --2- 172.21.15.1:0/1349314411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642467495d0 0x56424674a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc0837fe640 1 --2- 172.21.15.1:0/1349314411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246745700 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0175e0 tx=0x7fc07c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52a309af51424d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc068ff9640 1 -- 172.21.15.1:0/1349314411 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c007780 con 0x564246754370 2024-09-16T00:32:54.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc068ff9640 1 -- 172.21.15.1:0/1349314411 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc07c028070 con 0x564246754370 2024-09-16T00:32:54.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.801+0000 7fc068ff9640 1 -- 172.21.15.1:0/1349314411 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c023de0 con 0x564246754370 2024-09-16T00:32:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 -- 172.21.15.1:0/1349314411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 msgr2=0x564246745700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 --2- 172.21.15.1:0/1349314411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246745700 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0175e0 tx=0x7fc07c015000 comp rx=0 tx=0).stop 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 -- 172.21.15.1:0/1349314411 shutdown_connections 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 --2- 172.21.15.1:0/1349314411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x564246745700 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 --2- 172.21.15.1:0/1349314411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56424674ab40 0x564246760310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 --2- 172.21.15.1:0/1349314411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642467495d0 0x56424674a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 -- 172.21.15.1:0/1349314411 >> 172.21.15.1:0/1349314411 conn(0x564246712310 msgr2=0x564246740a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 -- 172.21.15.1:0/1349314411 shutdown_connections 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 -- 172.21.15.1:0/1349314411 wait complete. 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 -- start start 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 Processor -- start 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 0x56424675c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 0x56424675cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x56424674dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841af6d0 con 0x564246754370 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841b0b40 con 0x56424674ab40 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841afda0 con 0x5642467495d0 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.802+0000 7fc08b691740 10 client.0 _ll_register_callbacks cb 0x7fc074005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0837fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x56424674dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0837fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x56424674dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37790/0 (socket says 172.21.15.1:37790) 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0837fe640 1 -- 172.21.15.1:0/1572869764 learned_addr learned my addr 172.21.15.1:0/1572869764 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0827fc640 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 0x56424675cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc082ffd640 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 0x56424675c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0837fe640 1 -- 172.21.15.1:0/1572869764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 msgr2=0x56424675c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0837fe640 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 0x56424675c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0837fe640 1 -- 172.21.15.1:0/1572869764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 msgr2=0x56424675cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0837fe640 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 0x56424675cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0837fe640 1 -- 172.21.15.1:0/1572869764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424674e440 con 0x564246754370 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0827fc640 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 0x56424675cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc0837fe640 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x56424674dc10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc07c004b90 tx=0x7fc07c023650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c25079595ade99 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc066ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc066ff5640 1 -- 172.21.15.1:0/1572869764 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c032070 con 0x564246754370 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc066ff5640 1 -- 172.21.15.1:0/1572869764 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc07c023d70 con 0x564246754370 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc08b691740 1 -- 172.21.15.1:0/1572869764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc07c028070 con 0x564246754370 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc066ff5640 1 -- 172.21.15.1:0/1572869764 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc07c03e050 con 0x564246754370 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc08b691740 10 client.16296 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.803+0000 7fc08b691740 1 -- 172.21.15.1:0/1572869764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642467ce8b0 con 0x564246754370 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.804+0000 7fc066ff5640 1 -- 172.21.15.1:0/1572869764 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc07c028070 con 0x564246754370 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.804+0000 7fc066ff5640 1 -- 172.21.15.1:0/1572869764 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc07c02e7e0 con 0x564246754370 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.804+0000 7fc08b691740 10 client.16296 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.804+0000 7fc08b691740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.804+0000 7fc08b691740 20 client.16296 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.804+0000 7fc08b691740 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564246763fa0 0x5642467d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.804+0000 7fc08b691740 4 client.16296 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.804+0000 7fc08b691740 1 -- 172.21.15.1:0/1572869764 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642466b6d70 con 0x564246763fa0 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.804+0000 7fc082ffd640 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564246763fa0 0x5642467d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc066ff5640 10 client.16296 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc082ffd640 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564246763fa0 0x5642467d0f80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df53e3ff7bef1bec server_cookie=650aa1e6a5ed7004 in_seq=0 out_seq=0 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc066ff5640 1 -- 172.21.15.1:0/1572869764 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642466b6d70 con 0x564246763fa0 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc066ff5640 10 client.16296 handle_command_reply: tid=0 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 client.16296 shutdown 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 -- 172.21.15.1:0/1572869764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 msgr2=0x56424674dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x56424674dc10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc07c004b90 tx=0x7fc07c023650 comp rx=0 tx=0).stop 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 -- 172.21.15.1:0/1572869764 shutdown_connections 2024-09-16T00:32:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564246763fa0 0x5642467d0f80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564246754370 0x56424674dc10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56424674ab40 0x56424675cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 --2- 172.21.15.1:0/1572869764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642467495d0 0x56424675c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 -- 172.21.15.1:0/1572869764 >> 172.21.15.1:0/1572869764 conn(0x564246712310 msgr2=0x564246745700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc066ff5640 0 client.16296 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 -- 172.21.15.1:0/1572869764 shutdown_connections 2024-09-16T00:32:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 1 -- 172.21.15.1:0/1572869764 wait complete. 2024-09-16T00:32:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.805+0000 7fc08b691740 20 client.16296 trim_cache size 0 max 16384 2024-09-16T00:32:54.811 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:32:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 -- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc06c0a3240 msgr2=0x7fc06c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc06c0a3240 0x7fc06c0a5700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00a4c0 tx=0x7fc07c00c040 comp rx=0 tx=0).stop 2024-09-16T00:32:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 -- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084101240 msgr2=0x7fc08419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084101240 0x7fc08419a6f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc074006010 tx=0x7fc074003c10 comp rx=0 tx=0).stop 2024-09-16T00:32:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 -- 172.21.15.1:0/3868830584 shutdown_connections 2024-09-16T00:32:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc06c0a3240 0x7fc06c0a5700 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc084105a90 0x7fc08419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc084101bc0 0x7fc08419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 --2- 172.21.15.1:0/3868830584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc084101240 0x7fc08419a6f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 -- 172.21.15.1:0/3868830584 >> 172.21.15.1:0/3868830584 conn(0x7fc084077210 msgr2=0x7fc084100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 -- 172.21.15.1:0/3868830584 shutdown_connections 2024-09-16T00:32:54.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:54.806+0000 7fc0892b8640 1 -- 172.21.15.1:0/3868830584 wait complete. 2024-09-16T00:32:56.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:56 smithi019 ceph-mon[35186]: pgmap v486: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:56.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:56 smithi001 ceph-mon[30934]: pgmap v486: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:56.709 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:32:56.709 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:32:56.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:56 smithi153 ceph-mon[35538]: pgmap v486: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:56.975 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:57.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.497+0000 7fc22549b640 1 -- 172.21.15.1:0/3079875360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220106f10 msgr2=0x7fc22010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.497+0000 7fc22549b640 1 --2- 172.21.15.1:0/3079875360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220106f10 0x7fc22010ea90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc218009ca0 tx=0x7fc21802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.497+0000 7fc22549b640 1 -- 172.21.15.1:0/3079875360 shutdown_connections 2024-09-16T00:32:57.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.497+0000 7fc22549b640 1 --2- 172.21.15.1:0/3079875360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220106f10 0x7fc22010ea90 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.497+0000 7fc22549b640 1 --2- 172.21.15.1:0/3079875360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc220102790 0x7fc2201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.497+0000 7fc22549b640 1 --2- 172.21.15.1:0/3079875360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc220101de0 0x7fc2201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.497+0000 7fc22549b640 1 -- 172.21.15.1:0/3079875360 >> 172.21.15.1:0/3079875360 conn(0x7fc2200fd8c0 msgr2=0x7fc2200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.497+0000 7fc22549b640 1 -- 172.21.15.1:0/3079875360 shutdown_connections 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.497+0000 7fc22549b640 1 -- 172.21.15.1:0/3079875360 wait complete. 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc22549b640 1 Processor -- start 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc22549b640 1 -- start start 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc22549b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220101de0 0x7fc2200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc22549b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc220102790 0x7fc220074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc22549b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc220106f10 0x7fc220074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc22549b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc220111f90 con 0x7fc220101de0 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc22549b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc220111e10 con 0x7fc220106f10 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc22549b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc220112110 con 0x7fc220102790 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc21effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220101de0 0x7fc2200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc21effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220101de0 0x7fc2200744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37798/0 (socket says 172.21.15.1:37798) 2024-09-16T00:32:57.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc21effd640 1 -- 172.21.15.1:0/814463501 learned_addr learned my addr 172.21.15.1:0/814463501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc217fff640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc220102790 0x7fc220074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.498+0000 7fc21f7fe640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc220106f10 0x7fc220074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc21effd640 1 -- 172.21.15.1:0/814463501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc220102790 msgr2=0x7fc220074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc21effd640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc220102790 0x7fc220074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc21effd640 1 -- 172.21.15.1:0/814463501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc220106f10 msgr2=0x7fc220074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc21effd640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc220106f10 0x7fc220074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc21effd640 1 -- 172.21.15.1:0/814463501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2201a6930 con 0x7fc220101de0 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc21effd640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220101de0 0x7fc2200744f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d950 tx=0x7fc21000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc21cff9640 1 -- 172.21.15.1:0/814463501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210017030 con 0x7fc220101de0 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc21cff9640 1 -- 172.21.15.1:0/814463501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc210004020 con 0x7fc220101de0 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc22549b640 1 -- 172.21.15.1:0/814463501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2201a6c20 con 0x7fc220101de0 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc21cff9640 1 -- 172.21.15.1:0/814463501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100047a0 con 0x7fc220101de0 2024-09-16T00:32:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.499+0000 7fc22549b640 1 -- 172.21.15.1:0/814463501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2201a7160 con 0x7fc220101de0 2024-09-16T00:32:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.501+0000 7fc21cff9640 1 -- 172.21.15.1:0/814463501 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc2100041d0 con 0x7fc220101de0 2024-09-16T00:32:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc227874740 1 Processor -- start 2024-09-16T00:32:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc227874740 1 -- start start 2024-09-16T00:32:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc227874740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223057b60 0x561223057f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc227874740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223058480 0x561223062800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc227874740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x561223065130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc227874740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561222fbd280 con 0x561223062d40 2024-09-16T00:32:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc227874740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561222f64b10 con 0x561223057b60 2024-09-16T00:32:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc227874740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561222fb9240 con 0x561223058480 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc217fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x561223065130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc217fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x561223065130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37814/0 (socket says 172.21.15.1:37814) 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc21f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223057b60 0x561223057f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc21cff9640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc20c0a3240 0x7fc20c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc21effd640 1 --2- 172.21.15.1:0/1763664564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223058480 0x561223062800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.502+0000 7fc217fff640 1 -- 172.21.15.1:0/1763664564 learned_addr learned my addr 172.21.15.1:0/1763664564 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc21f7fe640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc20c0a3240 0x7fc20c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc217fff640 1 -- 172.21.15.1:0/1763664564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223058480 msgr2=0x561223062800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc21cff9640 1 -- 172.21.15.1:0/814463501 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc2100cf790 con 0x7fc220101de0 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc217fff640 1 --2- 172.21.15.1:0/1763664564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223058480 0x561223062800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc217fff640 1 -- 172.21.15.1:0/1763664564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223057b60 msgr2=0x561223057f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc217fff640 1 --2- 172.21.15.1:0/1763664564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223057b60 0x561223057f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc217fff640 1 -- 172.21.15.1:0/1763664564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561222f711f0 con 0x561223062d40 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc21f7fe640 1 --2- 172.21.15.1:0/1763664564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223057b60 0x561223057f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc217fff640 1 --2- 172.21.15.1:0/1763664564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x561223065130 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc20800b7f0 tx=0x7fc20800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f47ad20985104d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc21f7fe640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc20c0a3240 0x7fc20c0a5700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc218009430 tx=0x7fc21802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc2157fa640 1 -- 172.21.15.1:0/1763664564 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20800d950 con 0x561223062d40 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc2157fa640 1 -- 172.21.15.1:0/1763664564 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc208018070 con 0x561223062d40 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.503+0000 7fc2157fa640 1 -- 172.21.15.1:0/1763664564 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080123e0 con 0x561223062d40 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 -- 172.21.15.1:0/1763664564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 msgr2=0x561223065130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 --2- 172.21.15.1:0/1763664564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x561223065130 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc20800b7f0 tx=0x7fc20800bcb0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 -- 172.21.15.1:0/1763664564 shutdown_connections 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 --2- 172.21.15.1:0/1763664564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x561223065130 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 --2- 172.21.15.1:0/1763664564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223058480 0x561223062800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 --2- 172.21.15.1:0/1763664564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223057b60 0x561223057f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 -- 172.21.15.1:0/1763664564 >> 172.21.15.1:0/1763664564 conn(0x5612230540f0 msgr2=0x5612230544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 -- 172.21.15.1:0/1763664564 shutdown_connections 2024-09-16T00:32:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 -- 172.21.15.1:0/1763664564 wait complete. 2024-09-16T00:32:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 -- start start 2024-09-16T00:32:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.504+0000 7fc227874740 1 Processor -- start 2024-09-16T00:32:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc227874740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 0x56122305fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc227874740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223058480 0x561223060110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc227874740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223062d40 0x561223060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc227874740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561223065910 con 0x561223058480 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc227874740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561222f77f10 con 0x561223062d40 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc227874740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561223065a90 con 0x561223057b60 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc227874740 10 client.0 _ll_register_callbacks cb 0x561222f40a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc217fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223058480 0x561223060110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc217fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223058480 0x561223060110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37816/0 (socket says 172.21.15.1:37816) 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc217fff640 1 -- 172.21.15.1:0/1698880695 learned_addr learned my addr 172.21.15.1:0/1698880695 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc21f7fe640 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223062d40 0x561223060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc21effd640 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 0x56122305fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc217fff640 1 -- 172.21.15.1:0/1698880695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 msgr2=0x56122305fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc217fff640 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 0x56122305fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc217fff640 1 -- 172.21.15.1:0/1698880695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223062d40 msgr2=0x561223060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc217fff640 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223062d40 0x561223060650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc217fff640 1 -- 172.21.15.1:0/1698880695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561222d2e6a0 con 0x561223058480 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc21effd640 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 0x56122305fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc217fff640 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223058480 0x561223060110 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc208013c20 tx=0x7fc2080136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae28375aec1cc87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.505+0000 7fc2067fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc2067fc640 1 -- 172.21.15.1:0/1698880695 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20800dd60 con 0x561223058480 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc2067fc640 1 -- 172.21.15.1:0/1698880695 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc20801c420 con 0x561223058480 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc227874740 1 -- 172.21.15.1:0/1698880695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc208018070 con 0x561223058480 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc2067fc640 1 -- 172.21.15.1:0/1698880695 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc20802d050 con 0x561223058480 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc227874740 10 client.16308 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc227874740 1 -- 172.21.15.1:0/1698880695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5612230714c0 con 0x561223058480 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc2067fc640 1 -- 172.21.15.1:0/1698880695 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc208018070 con 0x561223058480 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc2067fc640 1 -- 172.21.15.1:0/1698880695 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc2080273e0 con 0x561223058480 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc227874740 10 client.16308 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc227874740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc227874740 20 client.16308 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc227874740 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561223072810 0x561223074c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc227874740 4 client.16308 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.506+0000 7fc227874740 1 -- 172.21.15.1:0/1698880695 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561222fca7f0 con 0x561223072810 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.507+0000 7fc21effd640 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561223072810 0x561223074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.507+0000 7fc2067fc640 10 client.16308 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.507+0000 7fc21effd640 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561223072810 0x561223074c00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73804950089adfd8 server_cookie=91ac43b542983f in_seq=0 out_seq=0 2024-09-16T00:32:57.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc2067fc640 1 -- 172.21.15.1:0/1698880695 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561222fca7f0 con 0x561223072810 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc2067fc640 10 client.16308 handle_command_reply: tid=0 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 client.16308 shutdown 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 -- 172.21.15.1:0/1698880695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223058480 msgr2=0x561223060110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223058480 0x561223060110 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc208013c20 tx=0x7fc2080136b0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 -- 172.21.15.1:0/1698880695 shutdown_connections 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561223072810 0x561223074c00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223062d40 0x561223060650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223058480 0x561223060110 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc2067fc640 0 client.16308 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 --2- 172.21.15.1:0/1698880695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 0x56122305fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 -- 172.21.15.1:0/1698880695 >> 172.21.15.1:0/1698880695 conn(0x5612230540f0 msgr2=0x561223059dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 -- 172.21.15.1:0/1698880695 shutdown_connections 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 1 -- 172.21.15.1:0/1698880695 wait complete. 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.508+0000 7fc227874740 20 client.16308 trim_cache size 0 max 16384 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc227874740 1 Processor -- start 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc227874740 1 -- start start 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc227874740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 0x561223059cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc227874740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122307a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc227874740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x56122307a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc227874740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201b3950 con 0x561223062d40 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc227874740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201b1220 con 0x561223058480 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc227874740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201af070 con 0x561223057b60 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc21f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x56122307a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc21f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x56122307a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37822/0 (socket says 172.21.15.1:37822) 2024-09-16T00:32:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc21f7fe640 1 -- 172.21.15.1:0/3446149553 learned_addr learned my addr 172.21.15.1:0/3446149553 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc217fff640 1 --2- 172.21.15.1:0/3446149553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122307a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc21f7fe640 1 -- 172.21.15.1:0/3446149553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 msgr2=0x561223059cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc21f7fe640 1 --2- 172.21.15.1:0/3446149553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 0x561223059cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc21f7fe640 1 -- 172.21.15.1:0/3446149553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 msgr2=0x56122307a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc21f7fe640 1 --2- 172.21.15.1:0/3446149553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122307a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.526+0000 7fc21f7fe640 1 -- 172.21.15.1:0/3446149553 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612230582b0 con 0x561223062d40 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.527+0000 7fc217fff640 1 --2- 172.21.15.1:0/3446149553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122307a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.527+0000 7fc21f7fe640 1 --2- 172.21.15.1:0/3446149553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x56122307a710 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc218009ca0 tx=0x7fc218009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb42320d80c6cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.527+0000 7fc214ff9640 1 -- 172.21.15.1:0/3446149553 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218050030 con 0x561223062d40 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.527+0000 7fc214ff9640 1 -- 172.21.15.1:0/3446149553 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc218036040 con 0x561223062d40 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.527+0000 7fc214ff9640 1 -- 172.21.15.1:0/3446149553 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218030cc0 con 0x561223062d40 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.527+0000 7fc227874740 1 -- 172.21.15.1:0/3446149553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 msgr2=0x56122307a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.527+0000 7fc227874740 1 --2- 172.21.15.1:0/3446149553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x56122307a710 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc218009ca0 tx=0x7fc218009360 comp rx=0 tx=0).stop 2024-09-16T00:32:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 -- 172.21.15.1:0/3446149553 shutdown_connections 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 --2- 172.21.15.1:0/3446149553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223062d40 0x56122307a710 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 --2- 172.21.15.1:0/3446149553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122307a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 --2- 172.21.15.1:0/3446149553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223057b60 0x561223059cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 -- 172.21.15.1:0/3446149553 >> 172.21.15.1:0/3446149553 conn(0x5612230540f0 msgr2=0x5612230595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 -- 172.21.15.1:0/3446149553 shutdown_connections 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 -- 172.21.15.1:0/3446149553 wait complete. 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 -- start start 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 Processor -- start 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223057b60 0x56122305cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122305d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223062d40 0x56122305d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201af6e0 con 0x561223057b60 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201b0b50 con 0x561223058480 2024-09-16T00:32:57.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201afdb0 con 0x561223062d40 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc227874740 10 client.0 _ll_register_callbacks cb 0x7fc218009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc21effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223057b60 0x56122305cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc21effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223057b60 0x56122305cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37824/0 (socket says 172.21.15.1:37824) 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.528+0000 7fc21effd640 1 -- 172.21.15.1:0/2656758495 learned_addr learned my addr 172.21.15.1:0/2656758495 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc21effd640 1 -- 172.21.15.1:0/2656758495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223062d40 msgr2=0x56122305d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc217fff640 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122305d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc21effd640 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223062d40 0x56122305d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc21effd640 1 -- 172.21.15.1:0/2656758495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 msgr2=0x56122305d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc21effd640 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122305d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc21effd640 1 -- 172.21.15.1:0/2656758495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612230dc250 con 0x561223057b60 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc217fff640 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122305d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc21effd640 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223057b60 0x56122305cc60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc210098380 tx=0x7fc21000b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e2c8ec385f2c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc2067fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc2067fc640 1 -- 172.21.15.1:0/2656758495 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100b9060 con 0x561223057b60 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc2067fc640 1 -- 172.21.15.1:0/2656758495 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc210099cb0 con 0x561223057b60 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc227874740 1 -- 172.21.15.1:0/2656758495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc218036040 con 0x561223057b60 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc2067fc640 1 -- 172.21.15.1:0/2656758495 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2100b3030 con 0x561223057b60 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc227874740 10 client.16314 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc227874740 1 -- 172.21.15.1:0/2656758495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5612230dd4c0 con 0x561223057b60 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc2067fc640 1 -- 172.21.15.1:0/2656758495 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc218036040 con 0x561223057b60 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.529+0000 7fc2067fc640 1 -- 172.21.15.1:0/2656758495 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc21009cce0 con 0x561223057b60 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.530+0000 7fc227874740 10 client.16314 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.530+0000 7fc227874740 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.530+0000 7fc227874740 20 client.16314 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.530+0000 7fc227874740 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5612230ddb40 0x5612230dff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.530+0000 7fc227874740 4 client.16314 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:32:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.530+0000 7fc227874740 1 -- 172.21.15.1:0/2656758495 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561222fc4d00 con 0x5612230ddb40 2024-09-16T00:32:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.530+0000 7fc217fff640 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5612230ddb40 0x5612230dff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.530+0000 7fc217fff640 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5612230ddb40 0x5612230dff60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c89ccef43b98d5 server_cookie=42c7dfbf956706 in_seq=0 out_seq=0 2024-09-16T00:32:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.530+0000 7fc2067fc640 10 client.16314 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.531+0000 7fc2067fc640 1 -- 172.21.15.1:0/2656758495 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561222fc4d00 con 0x5612230ddb40 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.531+0000 7fc2067fc640 10 client.16314 handle_command_reply: tid=0 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.531+0000 7fc227874740 1 client.16314 shutdown 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.531+0000 7fc227874740 1 -- 172.21.15.1:0/2656758495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223057b60 msgr2=0x56122305cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.531+0000 7fc227874740 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223057b60 0x56122305cc60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc210098380 tx=0x7fc21000b530 comp rx=0 tx=0).stop 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc227874740 1 -- 172.21.15.1:0/2656758495 shutdown_connections 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc227874740 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5612230ddb40 0x5612230dff60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc227874740 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561223062d40 0x56122305d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc227874740 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561223058480 0x56122305d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc2067fc640 0 client.16314 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc227874740 1 --2- 172.21.15.1:0/2656758495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561223057b60 0x56122305cc60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc227874740 1 -- 172.21.15.1:0/2656758495 >> 172.21.15.1:0/2656758495 conn(0x5612230540f0 msgr2=0x561223061a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc227874740 1 -- 172.21.15.1:0/2656758495 shutdown_connections 2024-09-16T00:32:57.565 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:32:57.565 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:32:57.565 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "de9c7ae7-a0ee-457d-b395-873a3aeaa48c", 2024-09-16T00:32:57.565 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:32:57.565 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:32:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc227874740 1 -- 172.21.15.1:0/2656758495 wait complete. 2024-09-16T00:32:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc227874740 20 client.16314 trim_cache size 0 max 16384 2024-09-16T00:32:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 -- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc20c0a3240 msgr2=0x7fc20c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc20c0a3240 0x7fc20c0a5700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc218009430 tx=0x7fc21802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 -- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220101de0 msgr2=0x7fc2200744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220101de0 0x7fc2200744f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d950 tx=0x7fc21000de10 comp rx=0 tx=0).stop 2024-09-16T00:32:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 -- 172.21.15.1:0/814463501 shutdown_connections 2024-09-16T00:32:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc220106f10 0x7fc220074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc20c0a3240 0x7fc20c0a5700 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc220102790 0x7fc220074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 --2- 172.21.15.1:0/814463501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc220101de0 0x7fc2200744f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 -- 172.21.15.1:0/814463501 >> 172.21.15.1:0/814463501 conn(0x7fc2200fd8c0 msgr2=0x7fc2200fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 -- 172.21.15.1:0/814463501 shutdown_connections 2024-09-16T00:32:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:57.532+0000 7fc22549b640 1 -- 172.21.15.1:0/814463501 wait complete. 2024-09-16T00:32:57.641 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.8 2024-09-16T00:32:57.641 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:58.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:32:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:58 smithi019 ceph-mon[35186]: pgmap v487: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:58 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:32:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:58 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:58 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:32:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:58 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:58 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:32:58.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:58 smithi001 ceph-mon[30934]: pgmap v487: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:58.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:58 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:32:58.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:58 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:58.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:58 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:32:58.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:58 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:58.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:58 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:32:58.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:58 smithi153 ceph-mon[35538]: pgmap v487: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:32:58.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:58 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:32:58.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:58 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:58.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:58 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:32:58.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:58 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:58.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:58 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.922+0000 7f0dcf346640 1 -- 172.21.15.1:0/3329946431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8106a80 msgr2=0x7f0dc8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.922+0000 7f0dcf346640 1 --2- 172.21.15.1:0/3329946431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8106a80 0x7f0dc8106e60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009930 tx=0x7f0dc402a420 comp rx=0 tx=0).stop 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.922+0000 7f0dcf346640 1 -- 172.21.15.1:0/3329946431 shutdown_connections 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.922+0000 7f0dcf346640 1 --2- 172.21.15.1:0/3329946431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dc8101420 0x7f0dc810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.922+0000 7f0dcf346640 1 --2- 172.21.15.1:0/3329946431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc8100a80 0x7f0dc8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.922+0000 7f0dcf346640 1 --2- 172.21.15.1:0/3329946431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8106a80 0x7f0dc8106e60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.922+0000 7f0dcf346640 1 -- 172.21.15.1:0/3329946431 >> 172.21.15.1:0/3329946431 conn(0x7f0dc80fc7a0 msgr2=0x7f0dc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.922+0000 7f0dcf346640 1 -- 172.21.15.1:0/3329946431 shutdown_connections 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.922+0000 7f0dcf346640 1 -- 172.21.15.1:0/3329946431 wait complete. 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcf346640 1 Processor -- start 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcf346640 1 -- start start 2024-09-16T00:32:58.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcf346640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8100a80 0x7f0dc819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcf346640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc8101420 0x7f0dc819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcf346640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dc8106a80 0x7f0dc819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcf346640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8110ea0 con 0x7f0dc8100a80 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcf346640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8110d20 con 0x7f0dc8101420 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcf346640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8111020 con 0x7f0dc8106a80 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcd0e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8100a80 0x7f0dc819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcd0e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8100a80 0x7f0dc819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37848/0 (socket says 172.21.15.1:37848) 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcd0e2640 1 -- 172.21.15.1:0/1714273558 learned_addr learned my addr 172.21.15.1:0/1714273558 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.923+0000 7f0dcc8e1640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc8101420 0x7f0dc819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcd8e3640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dc8106a80 0x7f0dc819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcd0e2640 1 -- 172.21.15.1:0/1714273558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dc8106a80 msgr2=0x7f0dc819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:58.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcd0e2640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dc8106a80 0x7f0dc819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcd0e2640 1 -- 172.21.15.1:0/1714273558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc8101420 msgr2=0x7f0dc819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcd0e2640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc8101420 0x7f0dc819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcd0e2640 1 -- 172.21.15.1:0/1714273558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc819f6e0 con 0x7f0dc8100a80 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcc8e1640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc8101420 0x7f0dc819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcd8e3640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dc8106a80 0x7f0dc819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcd0e2640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8100a80 0x7f0dc819a730 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009230 tx=0x7f0dc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dbe7fc640 1 -- 172.21.15.1:0/1714273558 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc403f030 con 0x7f0dc8100a80 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc819f970 con 0x7f0dc8100a80 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dbe7fc640 1 -- 172.21.15.1:0/1714273558 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc4003ad0 con 0x7f0dc8100a80 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dbe7fc640 1 -- 172.21.15.1:0/1714273558 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4035070 con 0x7f0dc8100a80 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.924+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc81a72b0 con 0x7f0dc8100a80 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.925+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc8100ee0 con 0x7f0dc8100a80 2024-09-16T00:32:58.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.926+0000 7f0dbe7fc640 1 -- 172.21.15.1:0/1714273558 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f0dc4042030 con 0x7f0dc8100a80 2024-09-16T00:32:58.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.927+0000 7f0dbe7fc640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0db40a3240 0x7f0db40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:58.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.927+0000 7f0dbe7fc640 1 -- 172.21.15.1:0/1714273558 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0dc40f06d0 con 0x7f0dc8100a80 2024-09-16T00:32:58.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.927+0000 7f0dcc8e1640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0db40a3240 0x7f0db40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:58.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.928+0000 7f0dcc8e1640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0db40a3240 0x7f0db40a5700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0db80058c0 tx=0x7f0db8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:58.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:58.930+0000 7f0dbe7fc640 1 -- 172.21.15.1:0/1714273558 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0dc40bcff0 con 0x7f0dc8100a80 2024-09-16T00:32:58.996 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:59.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.158+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dc81a7480 con 0x7f0dc8100a80 2024-09-16T00:32:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.158+0000 7f0dbe7fc640 1 -- 172.21.15.1:0/1714273558 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f0dc40bc8f0 con 0x7f0dc8100a80 2024-09-16T00:32:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:32:59.159 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:32:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:32:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0db40a3240 msgr2=0x7f0db40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0db40a3240 0x7f0db40a5700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0db80058c0 tx=0x7f0db8005850 comp rx=0 tx=0).stop 2024-09-16T00:32:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8100a80 msgr2=0x7f0dc819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8100a80 0x7f0dc819a730 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009230 tx=0x7f0dc4004270 comp rx=0 tx=0).stop 2024-09-16T00:32:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 shutdown_connections 2024-09-16T00:32:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f0db40a3240 0x7f0db40a5700 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dc8106a80 0x7f0dc819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc8101420 0x7f0dc819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 --2- 172.21.15.1:0/1714273558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dc8100a80 0x7f0dc819a730 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 >> 172.21.15.1:0/1714273558 conn(0x7f0dc80fc7a0 msgr2=0x7f0dc806a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:59.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 shutdown_connections 2024-09-16T00:32:59.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.161+0000 7f0dcf346640 1 -- 172.21.15.1:0/1714273558 wait complete. 2024-09-16T00:32:59.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:32:59 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1714273558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:59.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:32:59 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1714273558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:59.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:32:59 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1714273558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.706+0000 7f94c646d640 1 -- 172.21.15.1:0/4107130351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0101de0 msgr2=0x7f94c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.706+0000 7f94c646d640 1 --2- 172.21.15.1:0/4107130351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0101de0 0x7f94c01021c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009860 tx=0x7f94b002a340 comp rx=0 tx=0).stop 2024-09-16T00:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.706+0000 7f94c646d640 1 -- 172.21.15.1:0/4107130351 shutdown_connections 2024-09-16T00:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.706+0000 7f94c646d640 1 --2- 172.21.15.1:0/4107130351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c0106f10 0x7f94c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.706+0000 7f94c646d640 1 --2- 172.21.15.1:0/4107130351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c0102790 0x7f94c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.706+0000 7f94c646d640 1 --2- 172.21.15.1:0/4107130351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0101de0 0x7f94c01021c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.706+0000 7f94c646d640 1 -- 172.21.15.1:0/4107130351 >> 172.21.15.1:0/4107130351 conn(0x7f94c00fd8c0 msgr2=0x7f94c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.706+0000 7f94c646d640 1 -- 172.21.15.1:0/4107130351 shutdown_connections 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.706+0000 7f94c646d640 1 -- 172.21.15.1:0/4107130351 wait complete. 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c646d640 1 Processor -- start 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c646d640 1 -- start start 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c646d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c0101de0 0x7f94c0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c646d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c0102790 0x7f94c0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c646d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0106f10 0x7f94c019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c646d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0111f70 con 0x7f94c0106f10 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c646d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0111df0 con 0x7f94c0101de0 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c646d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01120f0 con 0x7f94c0102790 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c4a0a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0106f10 0x7f94c019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c0101de0 0x7f94c0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c4a0a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0106f10 0x7f94c019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37866/0 (socket says 172.21.15.1:37866) 2024-09-16T00:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94c4a0a640 1 -- 172.21.15.1:0/3978635153 learned_addr learned my addr 172.21.15.1:0/3978635153 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.707+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c0102790 0x7f94c0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94c4a0a640 1 -- 172.21.15.1:0/3978635153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c0102790 msgr2=0x7f94c0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94c4a0a640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c0102790 0x7f94c0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94c4a0a640 1 -- 172.21.15.1:0/3978635153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c0101de0 msgr2=0x7f94c0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94c4a0a640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c0101de0 0x7f94c0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94c4a0a640 1 -- 172.21.15.1:0/3978635153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c019d3e0 con 0x7f94c0106f10 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94bffff640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c0101de0 0x7f94c0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94c4a0a640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0106f10 0x7f94c019cd00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f94b800efc0 tx=0x7f94b800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94bd7fa640 1 -- 172.21.15.1:0/3978635153 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8003970 con 0x7f94c0106f10 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94c646d640 1 -- 172.21.15.1:0/3978635153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c019d6d0 con 0x7f94c0106f10 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94bd7fa640 1 -- 172.21.15.1:0/3978635153 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b8004020 con 0x7f94c0106f10 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94bd7fa640 1 -- 172.21.15.1:0/3978635153 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b800b9f0 con 0x7f94c0106f10 2024-09-16T00:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.708+0000 7f94c646d640 1 -- 172.21.15.1:0/3978635153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c01a4fb0 con 0x7f94c0106f10 2024-09-16T00:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.711+0000 7f94bd7fa640 1 -- 172.21.15.1:0/3978635153 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f94b801d020 con 0x7f94c0106f10 2024-09-16T00:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.711+0000 7f94c8846740 1 Processor -- start 2024-09-16T00:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.711+0000 7f94c8846740 1 -- start start 2024-09-16T00:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.711+0000 7f94c8846740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 0x558162581880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.711+0000 7f94c8846740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558162581dc0 0x55816258c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.711+0000 7f94c8846740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x55816258ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.711+0000 7f94c8846740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5581624e72e0 con 0x55816258c680 2024-09-16T00:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.711+0000 7f94c8846740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55816248ece0 con 0x5581625814a0 2024-09-16T00:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.711+0000 7f94c8846740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5581624e32a0 con 0x558162581dc0 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x55816258ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x55816258ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37870/0 (socket says 172.21.15.1:37870) 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 -- 172.21.15.1:0/1589134009 learned_addr learned my addr 172.21.15.1:0/1589134009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bffff640 1 --2- 172.21.15.1:0/1589134009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558162581dc0 0x55816258c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/1589134009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 0x558162581880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bd7fa640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f94a80a3240 0x7f94a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 -- 172.21.15.1:0/1589134009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558162581dc0 msgr2=0x55816258c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/1589134009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558162581dc0 0x55816258c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 -- 172.21.15.1:0/1589134009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 msgr2=0x558162581880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/1589134009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 0x558162581880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 -- 172.21.15.1:0/1589134009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558162473440 con 0x55816258c680 2024-09-16T00:32:59.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94c4a0a640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f94a80a3240 0x7f94a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/1589134009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x55816258ea70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f94b4011470 tx=0x7f94b40097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5b43cad01538b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94bd7fa640 1 -- 172.21.15.1:0/3978635153 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f94b80cfb20 con 0x7f94c0106f10 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94a67fc640 1 -- 172.21.15.1:0/1589134009 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4004030 con 0x55816258c680 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94a67fc640 1 -- 172.21.15.1:0/1589134009 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b40041d0 con 0x55816258c680 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.712+0000 7f94a67fc640 1 -- 172.21.15.1:0/1589134009 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b400f740 con 0x55816258c680 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c8846740 1 -- 172.21.15.1:0/1589134009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 msgr2=0x55816258ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c4a0a640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f94a80a3240 0x7f94a80a5700 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f94b8098710 tx=0x7f94b806ebc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c8846740 1 --2- 172.21.15.1:0/1589134009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x55816258ea70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f94b4011470 tx=0x7f94b40097d0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c8846740 1 -- 172.21.15.1:0/1589134009 shutdown_connections 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c8846740 1 --2- 172.21.15.1:0/1589134009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x55816258ea70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c8846740 1 --2- 172.21.15.1:0/1589134009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558162581dc0 0x55816258c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c8846740 1 --2- 172.21.15.1:0/1589134009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 0x558162581880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c8846740 1 -- 172.21.15.1:0/1589134009 >> 172.21.15.1:0/1589134009 conn(0x55816257da30 msgr2=0x55816257de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:59.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c8846740 1 -- 172.21.15.1:0/1589134009 shutdown_connections 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.713+0000 7f94c8846740 1 -- 172.21.15.1:0/1589134009 wait complete. 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94c8846740 1 -- start start 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94c8846740 1 Processor -- start 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94c8846740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581625814a0 0x55816258c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94c8846740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 0x558162589330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94c8846740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 0x558162589920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94c8846740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55816258f250 con 0x5581625814a0 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94c8846740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5581624a2050 con 0x558162581dc0 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94c8846740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55816258f3d0 con 0x55816258c680 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94c8846740 10 client.0 _ll_register_callbacks cb 0x5581624e2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581625814a0 0x55816258c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581625814a0 0x55816258c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37878/0 (socket says 172.21.15.1:37878) 2024-09-16T00:32:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.714+0000 7f94bf7fe640 1 -- 172.21.15.1:0/2798290006 learned_addr learned my addr 172.21.15.1:0/2798290006 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94bffff640 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 0x558162589330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 0x558162589920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94bf7fe640 1 -- 172.21.15.1:0/2798290006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 msgr2=0x558162589920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 0x558162589920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94bf7fe640 1 -- 172.21.15.1:0/2798290006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 msgr2=0x558162589330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 0x558162589330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94bf7fe640 1 -- 172.21.15.1:0/2798290006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581622583c0 con 0x5581625814a0 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581625814a0 0x55816258c080 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47da7f209437b132 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f949f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f949f7fe640 1 -- 172.21.15.1:0/2798290006 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4026040 con 0x5581625814a0 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f949f7fe640 1 -- 172.21.15.1:0/2798290006 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f94b4005030 con 0x5581625814a0 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94c8846740 1 -- 172.21.15.1:0/2798290006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b40041d0 con 0x5581625814a0 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f949f7fe640 1 -- 172.21.15.1:0/2798290006 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94b4031050 con 0x5581625814a0 2024-09-16T00:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94c8846740 10 client.16332 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f94c8846740 1 -- 172.21.15.1:0/2798290006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55816259acb0 con 0x5581625814a0 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.715+0000 7f949f7fe640 1 -- 172.21.15.1:0/2798290006 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b40041d0 con 0x5581625814a0 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f949f7fe640 1 -- 172.21.15.1:0/2798290006 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f94b400f4c0 con 0x5581625814a0 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f94c8846740 10 client.16332 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f94c8846740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f94c8846740 20 client.16332 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f94c8846740 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55816259c0c0 0x55816259e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f94c8846740 4 client.16332 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f94c8846740 1 -- 172.21.15.1:0/2798290006 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581624f4850 con 0x55816259c0c0 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f94bffff640 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55816259c0c0 0x55816259e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f949f7fe640 10 client.16332 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.716+0000 7f94bffff640 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55816259c0c0 0x55816259e4b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29bd06ffcb397277 server_cookie=2a9a0943a344e702 in_seq=0 out_seq=0 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f949f7fe640 1 -- 172.21.15.1:0/2798290006 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581624f4850 con 0x55816259c0c0 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f949f7fe640 10 client.16332 handle_command_reply: tid=0 2024-09-16T00:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 client.16332 shutdown 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 -- 172.21.15.1:0/2798290006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581625814a0 msgr2=0x55816258c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581625814a0 0x55816258c080 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4001e50 comp rx=0 tx=0).stop 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 -- 172.21.15.1:0/2798290006 shutdown_connections 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55816259c0c0 0x55816259e4b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 0x558162589920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 0x558162589330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 --2- 172.21.15.1:0/2798290006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581625814a0 0x55816258c080 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f949f7fe640 0 client.16332 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 -- 172.21.15.1:0/2798290006 >> 172.21.15.1:0/2798290006 conn(0x55816257da30 msgr2=0x558162583710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 -- 172.21.15.1:0/2798290006 shutdown_connections 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 1 -- 172.21.15.1:0/2798290006 wait complete. 2024-09-16T00:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.717+0000 7f94c8846740 20 client.16332 trim_cache size 0 max 16384 2024-09-16T00:32:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.736+0000 7f94c8846740 1 Processor -- start 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.736+0000 7f94c8846740 1 -- start start 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.736+0000 7f94c8846740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 0x5581625984b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.736+0000 7f94c8846740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558162581dc0 0x5581625a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.736+0000 7f94c8846740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 0x5581625a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.736+0000 7f94c8846740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b1720 con 0x558162581dc0 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.736+0000 7f94c8846740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01aeff0 con 0x5581625814a0 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.736+0000 7f94c8846740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01ace40 con 0x55816258c680 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558162581dc0 0x5581625a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558162581dc0 0x5581625a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37890/0 (socket says 172.21.15.1:37890) 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bffff640 1 -- 172.21.15.1:0/3245937856 learned_addr learned my addr 172.21.15.1:0/3245937856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 0x5581625984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 0x5581625a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bffff640 1 -- 172.21.15.1:0/3245937856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 msgr2=0x5581625a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bffff640 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 0x5581625a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bffff640 1 -- 172.21.15.1:0/3245937856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 msgr2=0x5581625984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bffff640 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 0x5581625984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bffff640 1 -- 172.21.15.1:0/3245937856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816258c3f0 con 0x558162581dc0 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 0x5581625984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94bffff640 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558162581dc0 0x5581625a3bf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f94b0031000 tx=0x7f94b0030460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ca7b1b1f88caf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94a5ffb640 1 -- 172.21.15.1:0/3245937856 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0037040 con 0x558162581dc0 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94a5ffb640 1 -- 172.21.15.1:0/3245937856 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b0049070 con 0x558162581dc0 2024-09-16T00:32:59.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.737+0000 7f94a5ffb640 1 -- 172.21.15.1:0/3245937856 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0003de0 con 0x558162581dc0 2024-09-16T00:32:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.738+0000 7f94c8846740 1 -- 172.21.15.1:0/3245937856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558162581dc0 msgr2=0x5581625a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.738+0000 7f94c8846740 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558162581dc0 0x5581625a3bf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f94b0031000 tx=0x7f94b0030460 comp rx=0 tx=0).stop 2024-09-16T00:32:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.738+0000 7f94c8846740 1 -- 172.21.15.1:0/3245937856 shutdown_connections 2024-09-16T00:32:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.738+0000 7f94c8846740 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55816258c680 0x5581625a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.738+0000 7f94c8846740 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558162581dc0 0x5581625a3bf0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.738+0000 7f94c8846740 1 --2- 172.21.15.1:0/3245937856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581625814a0 0x5581625984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.738+0000 7f94c8846740 1 -- 172.21.15.1:0/3245937856 >> 172.21.15.1:0/3245937856 conn(0x55816257da30 msgr2=0x5581625848b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.738+0000 7f94c8846740 1 -- 172.21.15.1:0/3245937856 shutdown_connections 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.738+0000 7f94c8846740 1 -- 172.21.15.1:0/3245937856 wait complete. 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.739+0000 7f94c8846740 1 -- start start 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.739+0000 7f94c8846740 1 Processor -- start 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.739+0000 7f94c8846740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581625814a0 0x55816258fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.739+0000 7f94c8846740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 0x5581625902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.739+0000 7f94c8846740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x558162608fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.739+0000 7f94c8846740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01ad4b0 con 0x55816258c680 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.739+0000 7f94c8846740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01ae920 con 0x558162581dc0 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.739+0000 7f94c8846740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01adb80 con 0x5581625814a0 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.739+0000 7f94c8846740 10 client.0 _ll_register_callbacks cb 0x7f94b8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x558162608fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x558162608fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37906/0 (socket says 172.21.15.1:37906) 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 -- 172.21.15.1:0/421501282 learned_addr learned my addr 172.21.15.1:0/421501282 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581625814a0 0x55816258fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bffff640 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 0x5581625902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 -- 172.21.15.1:0/421501282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581625814a0 msgr2=0x55816258fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581625814a0 0x55816258fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 -- 172.21.15.1:0/421501282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 msgr2=0x5581625902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 0x5581625902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 -- 172.21.15.1:0/421501282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558162581bf0 con 0x55816258c680 2024-09-16T00:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bffff640 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 0x5581625902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94bf7fe640 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x558162608fc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d4c5dbfcc35a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f949f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f949f7fe640 1 -- 172.21.15.1:0/421501282 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4011230 con 0x55816258c680 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f949f7fe640 1 -- 172.21.15.1:0/421501282 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f94b400f3c0 con 0x55816258c680 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.740+0000 7f94c8846740 1 -- 172.21.15.1:0/421501282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b0049070 con 0x55816258c680 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f949f7fe640 1 -- 172.21.15.1:0/421501282 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94b400f5c0 con 0x55816258c680 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f94c8846740 10 client.16338 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f94c8846740 1 -- 172.21.15.1:0/421501282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55816260a720 con 0x55816258c680 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f949f7fe640 1 -- 172.21.15.1:0/421501282 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b0049070 con 0x55816258c680 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f949f7fe640 1 -- 172.21.15.1:0/421501282 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f94b401e450 con 0x55816258c680 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f94c8846740 10 client.16338 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f94c8846740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f94c8846740 20 client.16338 populate_metadata read hostname 'smithi001' 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f94c8846740 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55816260ad80 0x55816260d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f94c8846740 4 client.16338 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f94c8846740 1 -- 172.21.15.1:0/421501282 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5581624eed60 con 0x55816260ad80 2024-09-16T00:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.741+0000 7f94bffff640 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55816260ad80 0x55816260d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f949f7fe640 10 client.16338 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94bffff640 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55816260ad80 0x55816260d170 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d95b9980b41c41e server_cookie=4af409c20b8a6b8f in_seq=0 out_seq=0 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f949f7fe640 1 -- 172.21.15.1:0/421501282 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5581624eed60 con 0x55816260ad80 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f949f7fe640 10 client.16338 handle_command_reply: tid=0 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 client.16338 shutdown 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 -- 172.21.15.1:0/421501282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 msgr2=0x558162608fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x558162608fc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4009620 comp rx=0 tx=0).stop 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 -- 172.21.15.1:0/421501282 shutdown_connections 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55816260ad80 0x55816260d170 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55816258c680 0x558162608fc0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558162581dc0 0x5581625902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 --2- 172.21.15.1:0/421501282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581625814a0 0x55816258fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f949f7fe640 0 client.16338 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:32:59.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 -- 172.21.15.1:0/421501282 >> 172.21.15.1:0/421501282 conn(0x55816257da30 msgr2=0x558162587ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:59.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 -- 172.21.15.1:0/421501282 shutdown_connections 2024-09-16T00:32:59.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 1 -- 172.21.15.1:0/421501282 wait complete. 2024-09-16T00:32:59.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.742+0000 7f94c8846740 20 client.16338 trim_cache size 0 max 16384 2024-09-16T00:32:59.749 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:32:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:32:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:32:59.749 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:32:59.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 -- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f94a80a3240 msgr2=0x7f94a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f94a80a3240 0x7f94a80a5700 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f94b8098710 tx=0x7f94b806ebc0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 -- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0106f10 msgr2=0x7f94c019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0106f10 0x7f94c019cd00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f94b800efc0 tx=0x7f94b800c800 comp rx=0 tx=0).stop 2024-09-16T00:32:59.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 -- 172.21.15.1:0/3978635153 shutdown_connections 2024-09-16T00:32:59.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c0106f10 0x7f94c019cd00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f94a80a3240 0x7f94a80a5700 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c0102790 0x7f94c0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 --2- 172.21.15.1:0/3978635153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c0101de0 0x7f94c0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 -- 172.21.15.1:0/3978635153 >> 172.21.15.1:0/3978635153 conn(0x7f94c00fd8c0 msgr2=0x7f94c00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 -- 172.21.15.1:0/3978635153 shutdown_connections 2024-09-16T00:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:32:59.743+0000 7f94c646d640 1 -- 172.21.15.1:0/3978635153 wait complete. 2024-09-16T00:33:00.604 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:00 smithi001 ceph-mon[30934]: pgmap v488: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:33:00.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:00 smithi019 ceph-mon[35186]: pgmap v488: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:33:00.637 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:33:00.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:00.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:00 smithi153 ceph-mon[35538]: pgmap v488: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:33:00.999 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0' 2024-09-16T00:33:00.999 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0 2024-09-16T00:33:01.066 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.584+0000 7f37da9f2640 1 -- 172.21.15.1:0/1402384876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4101de0 msgr2=0x7f37d41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.584+0000 7f37da9f2640 1 --2- 172.21.15.1:0/1402384876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4101de0 0x7f37d41021c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009900 tx=0x7f37c402a410 comp rx=0 tx=0).stop 2024-09-16T00:33:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.585+0000 7f37da9f2640 1 -- 172.21.15.1:0/1402384876 shutdown_connections 2024-09-16T00:33:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.585+0000 7f37da9f2640 1 --2- 172.21.15.1:0/1402384876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d4106f10 0x7f37d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.585+0000 7f37da9f2640 1 --2- 172.21.15.1:0/1402384876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d4102790 0x7f37d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.585+0000 7f37da9f2640 1 --2- 172.21.15.1:0/1402384876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4101de0 0x7f37d41021c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.585+0000 7f37da9f2640 1 -- 172.21.15.1:0/1402384876 >> 172.21.15.1:0/1402384876 conn(0x7f37d40fd8c0 msgr2=0x7f37d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.585+0000 7f37da9f2640 1 -- 172.21.15.1:0/1402384876 shutdown_connections 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.585+0000 7f37da9f2640 1 -- 172.21.15.1:0/1402384876 wait complete. 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.585+0000 7f37da9f2640 1 Processor -- start 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.585+0000 7f37da9f2640 1 -- start start 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37da9f2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d4101de0 0x7f37d4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37da9f2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d4102790 0x7f37d4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37da9f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4106f10 0x7f37d419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37da9f2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37d4111f70 con 0x7f37d4106f10 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37da9f2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37d4111df0 con 0x7f37d4101de0 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37da9f2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37d41120f0 con 0x7f37d4102790 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d8f8f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4106f10 0x7f37d419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:01.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d8f8f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4106f10 0x7f37d419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58482/0 (socket says 172.21.15.1:58482) 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d8f8f640 1 -- 172.21.15.1:0/3302378422 learned_addr learned my addr 172.21.15.1:0/3302378422 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d37fe640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d4102790 0x7f37d4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d3fff640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d4101de0 0x7f37d4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d8f8f640 1 -- 172.21.15.1:0/3302378422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d4102790 msgr2=0x7f37d4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d8f8f640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d4102790 0x7f37d4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d8f8f640 1 -- 172.21.15.1:0/3302378422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d4101de0 msgr2=0x7f37d4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d8f8f640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d4101de0 0x7f37d4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d8f8f640 1 -- 172.21.15.1:0/3302378422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d419d400 con 0x7f37d4106f10 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d8f8f640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4106f10 0x7f37d419cd20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00eaa0 tx=0x7f37cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d17fa640 1 -- 172.21.15.1:0/3302378422 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00cdb0 con 0x7f37d4106f10 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.586+0000 7f37d17fa640 1 -- 172.21.15.1:0/3302378422 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37cc005020 con 0x7f37d4106f10 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.587+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d419d6f0 con 0x7f37d4106f10 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.587+0000 7f37d17fa640 1 -- 172.21.15.1:0/3302378422 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0116b0 con 0x7f37d4106f10 2024-09-16T00:33:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.587+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d41a4fd0 con 0x7f37d4106f10 2024-09-16T00:33:01.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.588+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d41021c0 con 0x7f37d4106f10 2024-09-16T00:33:01.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.589+0000 7f37d17fa640 1 -- 172.21.15.1:0/3302378422 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f37cc005560 con 0x7f37d4106f10 2024-09-16T00:33:01.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.589+0000 7f37d17fa640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37bc0a32c0 0x7f37bc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:01.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.590+0000 7f37d17fa640 1 -- 172.21.15.1:0/3302378422 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f37cc0cffb0 con 0x7f37d4106f10 2024-09-16T00:33:01.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.590+0000 7f37d3fff640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37bc0a32c0 0x7f37bc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:01.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.590+0000 7f37d3fff640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37bc0a32c0 0x7f37bc0a5780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f37c4005ab0 tx=0x7f37c4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:01.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.594+0000 7f37d17fa640 1 -- 172.21.15.1:0/3302378422 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37cc016030 con 0x7f37d4106f10 2024-09-16T00:33:01.651 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:01.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.816+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37d402d7c0 con 0x7f37d4106f10 2024-09-16T00:33:01.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.817+0000 7f37d17fa640 1 -- 172.21.15.1:0/3302378422 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f37cc09c870 con 0x7f37d4106f10 2024-09-16T00:33:01.817 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:33:01.817 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:33:01.818 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:33:01.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37bc0a32c0 msgr2=0x7f37bc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:01.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37bc0a32c0 0x7f37bc0a5780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f37c4005ab0 tx=0x7f37c4005a20 comp rx=0 tx=0).stop 2024-09-16T00:33:01.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4106f10 msgr2=0x7f37d419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:01.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4106f10 0x7f37d419cd20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00eaa0 tx=0x7f37cc00ef60 comp rx=0 tx=0).stop 2024-09-16T00:33:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 shutdown_connections 2024-09-16T00:33:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4106f10 0x7f37d419cd20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37bc0a32c0 0x7f37bc0a5780 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d4102790 0x7f37d4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 --2- 172.21.15.1:0/3302378422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d4101de0 0x7f37d4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 >> 172.21.15.1:0/3302378422 conn(0x7f37d40fd8c0 msgr2=0x7f37d40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 shutdown_connections 2024-09-16T00:33:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:01.819+0000 7f37da9f2640 1 -- 172.21.15.1:0/3302378422 wait complete. 2024-09-16T00:33:02.556 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:02 smithi001 ceph-mon[30934]: pgmap v489: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:02.557 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:02 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3302378422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:02.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:02 smithi019 ceph-mon[35186]: pgmap v489: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:02.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:02 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3302378422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:02.702 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:33:02.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:02 smithi153 ceph-mon[35538]: pgmap v489: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:02.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:02 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3302378422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.890+0000 7fc8a6dda640 1 -- 172.21.15.1:0/55441142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 msgr2=0x7fc8a01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.890+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/55441142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 0x7fc8a01021c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc888009900 tx=0x7fc88802a410 comp rx=0 tx=0).stop 2024-09-16T00:33:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.890+0000 7fc8a6dda640 1 -- 172.21.15.1:0/55441142 shutdown_connections 2024-09-16T00:33:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.890+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/55441142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8a0106f10 0x7fc8a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.890+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/55441142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a0102790 0x7fc8a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.890+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/55441142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 0x7fc8a01021c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.890+0000 7fc8a6dda640 1 -- 172.21.15.1:0/55441142 >> 172.21.15.1:0/55441142 conn(0x7fc8a00fd8c0 msgr2=0x7fc8a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.890+0000 7fc8a6dda640 1 -- 172.21.15.1:0/55441142 shutdown_connections 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.890+0000 7fc8a6dda640 1 -- 172.21.15.1:0/55441142 wait complete. 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a6dda640 1 Processor -- start 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a6dda640 1 -- start start 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a6dda640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 0x7fc8a019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a6dda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a0102790 0x7fc8a019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a6dda640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8a0106f10 0x7fc8a010dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a6dda640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a0111f70 con 0x7fc8a0101de0 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a6dda640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a0111df0 con 0x7fc8a0102790 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a6dda640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01120f0 con 0x7fc8a0106f10 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a4b76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 0x7fc8a019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a4b76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 0x7fc8a019eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58490/0 (socket says 172.21.15.1:58490) 2024-09-16T00:33:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a4b76640 1 -- 172.21.15.1:0/2450067115 learned_addr learned my addr 172.21.15.1:0/2450067115 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc89ffff640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a0102790 0x7fc8a019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.891+0000 7fc8a5377640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8a0106f10 0x7fc8a010dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc8a4b76640 1 -- 172.21.15.1:0/2450067115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8a0106f10 msgr2=0x7fc8a010dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8a0106f10 0x7fc8a010dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc8a4b76640 1 -- 172.21.15.1:0/2450067115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a0102790 msgr2=0x7fc8a019f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a0102790 0x7fc8a019f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc8a4b76640 1 -- 172.21.15.1:0/2450067115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a010e480 con 0x7fc8a0101de0 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 0x7fc8a019eff0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc888006010 tx=0x7fc8880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc89dffb640 1 -- 172.21.15.1:0/2450067115 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88803f030 con 0x7fc8a0101de0 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc89dffb640 1 -- 172.21.15.1:0/2450067115 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc888003ce0 con 0x7fc8a0101de0 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc8a6dda640 1 -- 172.21.15.1:0/2450067115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a010e710 con 0x7fc8a0101de0 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc89dffb640 1 -- 172.21.15.1:0/2450067115 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888035070 con 0x7fc8a0101de0 2024-09-16T00:33:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.892+0000 7fc8a6dda640 1 -- 172.21.15.1:0/2450067115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a019ac10 con 0x7fc8a0101de0 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.895+0000 7fc89dffb640 1 -- 172.21.15.1:0/2450067115 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc888042030 con 0x7fc8a0101de0 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.895+0000 7fc8a91b3740 1 Processor -- start 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.895+0000 7fc8a91b3740 1 -- start start 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.895+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bcf7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.895+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd02400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd04d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56012bc5d2a0 con 0x56012bd02940 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56012bc04b10 con 0x56012bcf8080 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56012bc59260 con 0x56012bcf7760 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc89ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd04d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc89ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd04d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58492/0 (socket says 172.21.15.1:58492) 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc89ffff640 1 -- 172.21.15.1:0/129715355 learned_addr learned my addr 172.21.15.1:0/129715355 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd02400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc8a5377640 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bcf7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc89dffb640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc88c0a3240 0x7fc88c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc89ffff640 1 -- 172.21.15.1:0/129715355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 msgr2=0x56012bcf7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc89ffff640 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bcf7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc89ffff640 1 -- 172.21.15.1:0/129715355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 msgr2=0x56012bd02400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc89ffff640 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd02400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc89ffff640 1 -- 172.21.15.1:0/129715355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012b9ceb90 con 0x56012bd02940 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.896+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd02400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.897+0000 7fc89dffb640 1 -- 172.21.15.1:0/2450067115 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc8880efda0 con 0x7fc8a0101de0 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.897+0000 7fc8a5377640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc88c0a3240 0x7fc88c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.897+0000 7fc89ffff640 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd04d30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc89000ac70 tx=0x7fc89000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135d5b475eb24eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.897+0000 7fc896ffd640 1 -- 172.21.15.1:0/129715355 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89001d070 con 0x56012bd02940 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.897+0000 7fc896ffd640 1 -- 172.21.15.1:0/129715355 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890018b20 con 0x56012bd02940 2024-09-16T00:33:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.897+0000 7fc896ffd640 1 -- 172.21.15.1:0/129715355 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890010cb0 con 0x56012bd02940 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 -- 172.21.15.1:0/129715355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 msgr2=0x56012bd04d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a5377640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc88c0a3240 0x7fc88c0a5700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc898009fd0 tx=0x7fc898004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd04d30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc89000ac70 tx=0x7fc89000f790 comp rx=0 tx=0).stop 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 -- 172.21.15.1:0/129715355 shutdown_connections 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd04d30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd02400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/129715355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bcf7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 -- 172.21.15.1:0/129715355 >> 172.21.15.1:0/129715355 conn(0x56012bcf3cf0 msgr2=0x56012bcf40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 -- 172.21.15.1:0/129715355 shutdown_connections 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 -- 172.21.15.1:0/129715355 wait complete. 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.898+0000 7fc8a91b3740 1 -- start start 2024-09-16T00:33:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc8a91b3740 1 Processor -- start 2024-09-16T00:33:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bcff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bcf8080 0x56012bcffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bd02940 0x56012bd00320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc89ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bcf8080 0x56012bcffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56012bd05510 con 0x56012bcf8080 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56012bbe56e0 con 0x56012bd02940 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc89ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bcf8080 0x56012bcffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58506/0 (socket says 172.21.15.1:58506) 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56012bd05690 con 0x56012bcf7760 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc89ffff640 1 -- 172.21.15.1:0/1783252176 learned_addr learned my addr 172.21.15.1:0/1783252176 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.899+0000 7fc8a91b3740 10 client.0 _ll_register_callbacks cb 0x56012bc09a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.900+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bcff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.900+0000 7fc89ffff640 1 -- 172.21.15.1:0/1783252176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 msgr2=0x56012bcff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.900+0000 7fc89ffff640 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bcff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.900+0000 7fc89ffff640 1 -- 172.21.15.1:0/1783252176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bd02940 msgr2=0x56012bd00320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.900+0000 7fc89ffff640 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bd02940 0x56012bd00320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.900+0000 7fc89ffff640 1 -- 172.21.15.1:0/1783252176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56012bd0ea80 con 0x56012bcf8080 2024-09-16T00:33:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.900+0000 7fc89ffff640 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bcf8080 0x56012bcffde0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc890000c00 tx=0x7fc890010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63637949a1d359f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.900+0000 7fc87ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.900+0000 7fc87ffff640 1 -- 172.21.15.1:0/1783252176 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89001d050 con 0x56012bcf8080 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.901+0000 7fc87ffff640 1 -- 172.21.15.1:0/1783252176 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc890018850 con 0x56012bcf8080 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.901+0000 7fc8a91b3740 1 -- 172.21.15.1:0/1783252176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc890018b20 con 0x56012bcf8080 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.901+0000 7fc87ffff640 1 -- 172.21.15.1:0/1783252176 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc890032050 con 0x56012bcf8080 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.901+0000 7fc8a91b3740 10 client.16356 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.901+0000 7fc8a91b3740 1 -- 172.21.15.1:0/1783252176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56012bd113f0 con 0x56012bcf8080 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.901+0000 7fc87ffff640 1 -- 172.21.15.1:0/1783252176 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890018b20 con 0x56012bcf8080 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.901+0000 7fc87ffff640 1 -- 172.21.15.1:0/1783252176 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc8900046c0 con 0x56012bcf8080 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.901+0000 7fc8a91b3740 10 client.16356 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.902+0000 7fc8a91b3740 10 client.16356 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.902+0000 7fc8a91b3740 20 client.16356 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.902+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56012bd12850 0x56012bd14c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.902+0000 7fc8a91b3740 4 client.16356 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.902+0000 7fc8a91b3740 1 -- 172.21.15.1:0/1783252176 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56012bc6a9d0 con 0x56012bd12850 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.902+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56012bd12850 0x56012bd14c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.902+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56012bd12850 0x56012bd14c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fcb1fea420df434 server_cookie=732a8c91103c934a in_seq=0 out_seq=0 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.902+0000 7fc87ffff640 10 client.16356 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc87ffff640 1 -- 172.21.15.1:0/1783252176 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56012bc6a9d0 con 0x56012bd12850 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc87ffff640 10 client.16356 handle_command_reply: tid=0 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc8a91b3740 1 client.16356 shutdown 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc8a91b3740 1 -- 172.21.15.1:0/1783252176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bcf8080 msgr2=0x56012bcffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bcf8080 0x56012bcffde0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc890000c00 tx=0x7fc890010c20 comp rx=0 tx=0).stop 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc8a91b3740 1 -- 172.21.15.1:0/1783252176 shutdown_connections 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56012bd12850 0x56012bd14c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bd02940 0x56012bd00320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bcf8080 0x56012bcffde0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc87ffff640 0 client.16356 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/1783252176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bcff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.903+0000 7fc8a91b3740 1 -- 172.21.15.1:0/1783252176 >> 172.21.15.1:0/1783252176 conn(0x56012bcf3cf0 msgr2=0x56012bcf99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.904+0000 7fc8a91b3740 1 -- 172.21.15.1:0/1783252176 shutdown_connections 2024-09-16T00:33:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.904+0000 7fc8a91b3740 1 -- 172.21.15.1:0/1783252176 wait complete. 2024-09-16T00:33:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.904+0000 7fc8a91b3740 20 client.16356 trim_cache size 0 max 16384 2024-09-16T00:33:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a91b3740 1 Processor -- start 2024-09-16T00:33:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a91b3740 1 -- start start 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf7760 0x56012bd00170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf8080 0x56012bcf98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd1a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01c0560 con 0x56012bd02940 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01bde30 con 0x56012bcf7760 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01bbc80 con 0x56012bcf8080 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a5377640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd1a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a5377640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd1a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58508/0 (socket says 172.21.15.1:58508) 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a5377640 1 -- 172.21.15.1:0/3200316412 learned_addr learned my addr 172.21.15.1:0/3200316412 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf7760 0x56012bd00170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.922+0000 7fc89ffff640 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf8080 0x56012bcf98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8a5377640 1 -- 172.21.15.1:0/3200316412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf8080 msgr2=0x56012bcf98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8a5377640 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf8080 0x56012bcf98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8a5377640 1 -- 172.21.15.1:0/3200316412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf7760 msgr2=0x56012bd00170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8a5377640 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf7760 0x56012bd00170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8a5377640 1 -- 172.21.15.1:0/3200316412 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012bcfe850 con 0x56012bd02940 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf7760 0x56012bd00170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8a5377640 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd1a2a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc8980143d0 tx=0x7fc898009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210ae3021f5049c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8967fc640 1 -- 172.21.15.1:0/3200316412 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89801a070 con 0x56012bd02940 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8967fc640 1 -- 172.21.15.1:0/3200316412 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc898027070 con 0x56012bd02940 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.923+0000 7fc8967fc640 1 -- 172.21.15.1:0/3200316412 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc898022bc0 con 0x56012bd02940 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 -- 172.21.15.1:0/3200316412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 msgr2=0x56012bd1a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd1a2a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc8980143d0 tx=0x7fc898009600 comp rx=0 tx=0).stop 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 -- 172.21.15.1:0/3200316412 shutdown_connections 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd1a2a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf8080 0x56012bcf98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/3200316412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf7760 0x56012bd00170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 -- 172.21.15.1:0/3200316412 >> 172.21.15.1:0/3200316412 conn(0x56012bcf3cf0 msgr2=0x56012bcf91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 -- 172.21.15.1:0/3200316412 shutdown_connections 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 -- 172.21.15.1:0/3200316412 wait complete. 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.924+0000 7fc8a91b3740 1 -- start start 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a91b3740 1 Processor -- start 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bd0a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd0ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a91b3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01bc2f0 con 0x56012bd02940 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01bd760 con 0x56012bcf8080 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a91b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01bc9c0 con 0x56012bcf7760 2024-09-16T00:33:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a91b3740 10 client.0 _ll_register_callbacks cb 0x7fc89000aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a5377640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a5377640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58514/0 (socket says 172.21.15.1:58514) 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a5377640 1 -- 172.21.15.1:0/79109415 learned_addr learned my addr 172.21.15.1:0/79109415 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.925+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bd0a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc89ffff640 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd0ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc8a5377640 1 -- 172.21.15.1:0/79109415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 msgr2=0x56012bd0a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc8a5377640 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bd0a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc8a5377640 1 -- 172.21.15.1:0/79109415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 msgr2=0x56012bd0ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc8a5377640 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd0ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc8a5377640 1 -- 172.21.15.1:0/79109415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56012bcf7eb0 con 0x56012bd02940 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc89ffff640 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd0ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc8a5377640 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd078b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc898012810 tx=0x7fc898014400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83200ef94ef01105 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc87ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc87ffff640 1 -- 172.21.15.1:0/79109415 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89801a030 con 0x56012bd02940 2024-09-16T00:33:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc87ffff640 1 -- 172.21.15.1:0/79109415 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc898004d20 con 0x56012bd02940 2024-09-16T00:33:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc8a91b3740 1 -- 172.21.15.1:0/79109415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc898027070 con 0x56012bd02940 2024-09-16T00:33:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc87ffff640 1 -- 172.21.15.1:0/79109415 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc89803c050 con 0x56012bd02940 2024-09-16T00:33:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc8a91b3740 10 client.16362 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.926+0000 7fc8a91b3740 1 -- 172.21.15.1:0/79109415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56012bd7d750 con 0x56012bd02940 2024-09-16T00:33:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.927+0000 7fc87ffff640 1 -- 172.21.15.1:0/79109415 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc898027070 con 0x56012bd02940 2024-09-16T00:33:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.927+0000 7fc87ffff640 1 -- 172.21.15.1:0/79109415 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc89800f040 con 0x56012bd02940 2024-09-16T00:33:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.927+0000 7fc8a91b3740 10 client.16362 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.927+0000 7fc8a91b3740 10 client.16362 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:33:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.927+0000 7fc8a91b3740 20 client.16362 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.927+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56012bd12850 0x56012bd7fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.927+0000 7fc8a91b3740 4 client.16362 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}] 2024-09-16T00:33:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.927+0000 7fc8a91b3740 1 -- 172.21.15.1:0/79109415 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}) -- 0x56012bc64d20 con 0x56012bd12850 2024-09-16T00:33:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.927+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56012bd12850 0x56012bd7fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.928+0000 7fc87ffff640 10 client.16362 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.928+0000 7fc8a4b76640 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56012bd12850 0x56012bd7fef0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4edbc60141dda75 server_cookie=59f9eecd677392cb in_seq=0 out_seq=0 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc87ffff640 1 -- 172.21.15.1:0/79109415 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56012bc64d20 con 0x56012bd12850 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc87ffff640 10 client.16362 handle_command_reply: tid=0 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 client.16362 shutdown 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 -- 172.21.15.1:0/79109415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 msgr2=0x56012bd078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd078b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc898012810 tx=0x7fc898014400 comp rx=0 tx=0).stop 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 -- 172.21.15.1:0/79109415 shutdown_connections 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56012bd12850 0x56012bd7fef0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56012bd02940 0x56012bd078b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56012bcf8080 0x56012bd0ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 --2- 172.21.15.1:0/79109415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56012bcf7760 0x56012bd0a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 -- 172.21.15.1:0/79109415 >> 172.21.15.1:0/79109415 conn(0x56012bcf3cf0 msgr2=0x56012bcf2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc87ffff640 0 client.16362 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 -- 172.21.15.1:0/79109415 shutdown_connections 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 1 -- 172.21.15.1:0/79109415 wait complete. 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.940+0000 7fc8a91b3740 20 client.16362 trim_cache size 0 max 16384 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 14, 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "ec419676": { 2024-09-16T00:33:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 14, 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 133.8, 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:33:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:02.944 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:33:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 -- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc88c0a3240 msgr2=0x7fc88c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc88c0a3240 0x7fc88c0a5700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc898009fd0 tx=0x7fc898004ab0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 -- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 msgr2=0x7fc8a019eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 0x7fc8a019eff0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc888006010 tx=0x7fc8880042f0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 -- 172.21.15.1:0/2450067115 shutdown_connections 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc88c0a3240 0x7fc88c0a5700 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8a0106f10 0x7fc8a010dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a0102790 0x7fc8a019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 --2- 172.21.15.1:0/2450067115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8a0101de0 0x7fc8a019eff0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 -- 172.21.15.1:0/2450067115 >> 172.21.15.1:0/2450067115 conn(0x7fc8a00fd8c0 msgr2=0x7fc8a00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 -- 172.21.15.1:0/2450067115 shutdown_connections 2024-09-16T00:33:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:02.941+0000 7fc8a6dda640 1 -- 172.21.15.1:0/2450067115 wait complete. 2024-09-16T00:33:03.155 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:03.656 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ec419676', quiesce took 6.0 seconds. Will release after: 7.800000000000001 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.183+0000 7f1607d07640 1 -- 172.21.15.1:0/1832541675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f160010c170 msgr2=0x7f160010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.183+0000 7f1607d07640 1 --2- 172.21.15.1:0/1832541675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f160010c170 0x7f160010e610 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009ca0 tx=0x7f15f802a950 comp rx=0 tx=0).stop 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.183+0000 7f1607d07640 1 -- 172.21.15.1:0/1832541675 shutdown_connections 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.183+0000 7f1607d07640 1 --2- 172.21.15.1:0/1832541675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f160010c170 0x7f160010e610 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.183+0000 7f1607d07640 1 --2- 172.21.15.1:0/1832541675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600103660 0x7f160010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.183+0000 7f1607d07640 1 --2- 172.21.15.1:0/1832541675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1600102d40 0x7f1600103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.183+0000 7f1607d07640 1 -- 172.21.15.1:0/1832541675 >> 172.21.15.1:0/1832541675 conn(0x7f16000fc7a0 msgr2=0x7f16000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.183+0000 7f1607d07640 1 -- 172.21.15.1:0/1832541675 shutdown_connections 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.183+0000 7f1607d07640 1 -- 172.21.15.1:0/1832541675 wait complete. 2024-09-16T00:33:04.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f1607d07640 1 Processor -- start 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f1607d07640 1 -- start start 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f1607d07640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1600102d40 0x7f16001983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f1607d07640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1600103660 0x7f1600198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f1607d07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f160010c170 0x7f160019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f1607d07640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1600111f70 con 0x7f1600103660 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f1607d07640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1600111df0 con 0x7f160010c170 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f1607d07640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16001120f0 con 0x7f1600102d40 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16052a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1600103660 0x7f1600198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16052a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1600103660 0x7f1600198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58528/0 (socket says 172.21.15.1:58528) 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16052a2640 1 -- 172.21.15.1:0/420319159 learned_addr learned my addr 172.21.15.1:0/420319159 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f1605aa3640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1600102d40 0x7f16001983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16062a4640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f160010c170 0x7f160019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16052a2640 1 -- 172.21.15.1:0/420319159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1600102d40 msgr2=0x7f16001983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16052a2640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1600102d40 0x7f16001983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16052a2640 1 -- 172.21.15.1:0/420319159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f160010c170 msgr2=0x7f160019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16052a2640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f160010c170 0x7f160019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16052a2640 1 -- 172.21.15.1:0/420319159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160019d380 con 0x7f1600103660 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.184+0000 7f16062a4640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f160010c170 0x7f160019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.185+0000 7f16052a2640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1600103660 0x7f1600198910 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c940 tx=0x7f15f000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.185+0000 7f15f6ffd640 1 -- 172.21.15.1:0/420319159 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004220 con 0x7f1600103660 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.185+0000 7f15f6ffd640 1 -- 172.21.15.1:0/420319159 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f00043c0 con 0x7f1600103660 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.185+0000 7f1607d07640 1 -- 172.21.15.1:0/420319159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160019d670 con 0x7f1600103660 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.185+0000 7f15f6ffd640 1 -- 172.21.15.1:0/420319159 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004a00 con 0x7f1600103660 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.185+0000 7f1607d07640 1 -- 172.21.15.1:0/420319159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16001a4f50 con 0x7f1600103660 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.187+0000 7f15f6ffd640 1 -- 172.21.15.1:0/420319159 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f15f00079b0 con 0x7f1600103660 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.187+0000 7f160a0e0740 1 Processor -- start 2024-09-16T00:33:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f160a0e0740 1 -- start start 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 0x556195631090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561956315d0 0x556195631a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195640780 0x556195642b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f160a0e0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5561955972f0 con 0x5561956315d0 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f160a0e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55619553ecf0 con 0x556195632b40 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f160a0e0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5561955932b0 con 0x556195640780 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f1605aa3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561956315d0 0x556195631a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f1605aa3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561956315d0 0x556195631a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58530/0 (socket says 172.21.15.1:58530) 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f1605aa3640 1 -- 172.21.15.1:0/3595254718 learned_addr learned my addr 172.21.15.1:0/3595254718 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f15f6ffd640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15ec0a3240 0x7f15ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f16052a2640 1 --2- 172.21.15.1:0/3595254718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195640780 0x556195642b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f1605aa3640 1 -- 172.21.15.1:0/3595254718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195640780 msgr2=0x556195642b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f1605aa3640 1 --2- 172.21.15.1:0/3595254718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195640780 0x556195642b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f1605aa3640 1 -- 172.21.15.1:0/3595254718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 msgr2=0x556195631090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f1605aa3640 1 --2- 172.21.15.1:0/3595254718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 0x556195631090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f16062a4640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15ec0a3240 0x7f15ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.188+0000 7f1605aa3640 1 -- 172.21.15.1:0/3595254718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561955233a0 con 0x5561956315d0 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.189+0000 7f15f6ffd640 1 -- 172.21.15.1:0/420319159 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f15f00cfd70 con 0x7f1600103660 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.189+0000 7f1605aa3640 1 --2- 172.21.15.1:0/3595254718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561956315d0 0x556195631a10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00cbf0 tx=0x7f15fc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d40260167fe72a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.189+0000 7f15ebfff640 1 -- 172.21.15.1:0/3595254718 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc010630 con 0x5561956315d0 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.189+0000 7f15ebfff640 1 -- 172.21.15.1:0/3595254718 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc010cb0 con 0x5561956315d0 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.189+0000 7f16062a4640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15ec0a3240 0x7f15ec0a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f15f8006010 tx=0x7f15f8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.189+0000 7f15ebfff640 1 -- 172.21.15.1:0/3595254718 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc00fc60 con 0x5561956315d0 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 -- 172.21.15.1:0/3595254718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561956315d0 msgr2=0x556195631a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3595254718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561956315d0 0x556195631a10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00cbf0 tx=0x7f15fc00e720 comp rx=0 tx=0).stop 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 -- 172.21.15.1:0/3595254718 shutdown_connections 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3595254718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195640780 0x556195642b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3595254718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561956315d0 0x556195631a10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3595254718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 0x556195631090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 -- 172.21.15.1:0/3595254718 >> 172.21.15.1:0/3595254718 conn(0x5561955fa310 msgr2=0x55619562dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 -- 172.21.15.1:0/3595254718 shutdown_connections 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 -- 172.21.15.1:0/3595254718 wait complete. 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 -- start start 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.190+0000 7f160a0e0740 1 Processor -- start 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 0x5561956400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 0x55619563d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f160a0e0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556195643480 con 0x556195640780 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f160a0e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556195552060 con 0x5561956315d0 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f160a0e0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556195643600 con 0x556195632b40 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f160a0e0740 10 client.0 _ll_register_callbacks cb 0x5561952a58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16062a4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16062a4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58540/0 (socket says 172.21.15.1:58540) 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16062a4640 1 -- 172.21.15.1:0/3270103817 learned_addr learned my addr 172.21.15.1:0/3270103817 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f1605aa3640 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 0x5561956400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16052a2640 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 0x55619563d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16062a4640 1 -- 172.21.15.1:0/3270103817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 msgr2=0x55619563d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16062a4640 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 0x55619563d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16062a4640 1 -- 172.21.15.1:0/3270103817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 msgr2=0x5561956400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16062a4640 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 0x5561956400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16062a4640 1 -- 172.21.15.1:0/3270103817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561953083c0 con 0x556195640780 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f16062a4640 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563d8e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f15f802f7e0 tx=0x7f15f802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da141c9fd479966f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.191+0000 7f15e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f15e97fa640 1 -- 172.21.15.1:0/3270103817 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8034040 con 0x556195640780 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f160a0e0740 1 -- 172.21.15.1:0/3270103817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15fc010cb0 con 0x556195640780 2024-09-16T00:33:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f15e97fa640 1 -- 172.21.15.1:0/3270103817 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f15f8004240 con 0x556195640780 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f15e97fa640 1 -- 172.21.15.1:0/3270103817 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f15f8004440 con 0x556195640780 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f160a0e0740 10 client.16374 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f160a0e0740 1 -- 172.21.15.1:0/3270103817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55619564eee0 con 0x556195640780 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f15e97fa640 1 -- 172.21.15.1:0/3270103817 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc010cb0 con 0x556195640780 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f15e97fa640 1 -- 172.21.15.1:0/3270103817 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f15f8030440 con 0x556195640780 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f160a0e0740 10 client.16374 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f160a0e0740 10 client.16374 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f160a0e0740 20 client.16374 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5561956506d0 0x556195652b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f160a0e0740 4 client.16374 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.192+0000 7f160a0e0740 1 -- 172.21.15.1:0/3270103817 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561955a4860 con 0x5561956506d0 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.193+0000 7f1605aa3640 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5561956506d0 0x556195652b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.193+0000 7f15e97fa640 10 client.16374 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:33:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.193+0000 7f1605aa3640 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5561956506d0 0x556195652b70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72e4716e9c161389 server_cookie=29dd844f74bd9bb9 in_seq=0 out_seq=0 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f15e97fa640 1 -- 172.21.15.1:0/3270103817 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561955a4860 con 0x5561956506d0 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f15e97fa640 10 client.16374 handle_command_reply: tid=0 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 client.16374 shutdown 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 -- 172.21.15.1:0/3270103817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 msgr2=0x55619563d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563d8e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f15f802f7e0 tx=0x7f15f802a950 comp rx=0 tx=0).stop 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 -- 172.21.15.1:0/3270103817 shutdown_connections 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563d8e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 0x55619563d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5561956506d0 0x556195652b70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3270103817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 0x5561956400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f15e97fa640 0 client.16374 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 -- 172.21.15.1:0/3270103817 >> 172.21.15.1:0/3270103817 conn(0x5561955fa310 msgr2=0x556195633960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 -- 172.21.15.1:0/3270103817 shutdown_connections 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 1 -- 172.21.15.1:0/3270103817 wait complete. 2024-09-16T00:33:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.194+0000 7f160a0e0740 20 client.16374 trim_cache size 0 max 16384 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f160a0e0740 1 Processor -- start 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f160a0e0740 1 -- start start 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561956315d0 0x55619564c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 0x55619562d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619562dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f160a0e0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16001af500 con 0x556195640780 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f160a0e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16001acdd0 con 0x556195632b40 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f160a0e0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16001aac20 con 0x5561956315d0 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16062a4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619562dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16062a4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619562dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58542/0 (socket says 172.21.15.1:58542) 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16062a4640 1 -- 172.21.15.1:0/2026869011 learned_addr learned my addr 172.21.15.1:0/2026869011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f1605aa3640 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561956315d0 0x55619564c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16052a2640 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 0x55619562d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16062a4640 1 -- 172.21.15.1:0/2026869011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561956315d0 msgr2=0x55619564c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16062a4640 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561956315d0 0x55619564c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16062a4640 1 -- 172.21.15.1:0/2026869011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 msgr2=0x55619562d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16062a4640 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 0x55619562d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16062a4640 1 -- 172.21.15.1:0/2026869011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55619562e180 con 0x556195640780 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.208+0000 7f16052a2640 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 0x55619562d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.209+0000 7f16062a4640 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619562dc40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f15f80308b0 tx=0x7f15f8009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9637d34dfe1d08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.209+0000 7f15eb7fe640 1 -- 172.21.15.1:0/2026869011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8030640 con 0x556195640780 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.209+0000 7f15eb7fe640 1 -- 172.21.15.1:0/2026869011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f804a070 con 0x556195640780 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.209+0000 7f15eb7fe640 1 -- 172.21.15.1:0/2026869011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f80045e0 con 0x556195640780 2024-09-16T00:33:04.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 -- 172.21.15.1:0/2026869011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 msgr2=0x55619562dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619562dc40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f15f80308b0 tx=0x7f15f8009660 comp rx=0 tx=0).stop 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 -- 172.21.15.1:0/2026869011 shutdown_connections 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619562dc40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556195632b40 0x55619562d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 --2- 172.21.15.1:0/2026869011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561956315d0 0x55619564c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 -- 172.21.15.1:0/2026869011 >> 172.21.15.1:0/2026869011 conn(0x5561955fa310 msgr2=0x55619563d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 -- 172.21.15.1:0/2026869011 shutdown_connections 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 -- 172.21.15.1:0/2026869011 wait complete. 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 -- start start 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.210+0000 7f160a0e0740 1 Processor -- start 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 0x556195649040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 0x55619563a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f160a0e0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f160a0e0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ab290 con 0x556195640780 2024-09-16T00:33:04.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f160a0e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ac700 con 0x5561956315d0 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f160a0e0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ab960 con 0x556195632b40 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f160a0e0740 10 client.0 _ll_register_callbacks cb 0x7f15f8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16062a4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16052a2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 0x55619563a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16062a4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58556/0 (socket says 172.21.15.1:58556) 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16062a4640 1 -- 172.21.15.1:0/3371874490 learned_addr learned my addr 172.21.15.1:0/3371874490 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f1605aa3640 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 0x556195649040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16052a2640 1 -- 172.21.15.1:0/3371874490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 msgr2=0x556195649040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16052a2640 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 0x556195649040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16052a2640 1 -- 172.21.15.1:0/3371874490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 msgr2=0x55619563a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16052a2640 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16052a2640 1 -- 172.21.15.1:0/3371874490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55619563ae90 con 0x556195632b40 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.211+0000 7f16062a4640 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f16052a2640 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 0x55619563a120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f15f00a4b10 tx=0x7f15f00053f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76ac264d655e7ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f15e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f15e97fa640 1 -- 172.21.15.1:0/3371874490 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f001d040 con 0x556195632b40 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f160a0e0740 1 -- 172.21.15.1:0/3371874490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15f804a070 con 0x556195632b40 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f15e97fa640 1 -- 172.21.15.1:0/3371874490 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f15f0007ca0 con 0x556195632b40 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f15e97fa640 1 -- 172.21.15.1:0/3371874490 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f15f009bf30 con 0x556195632b40 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f160a0e0740 10 client.25696 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f160a0e0740 1 -- 172.21.15.1:0/3371874490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5561956bb4d0 con 0x556195632b40 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f15e97fa640 1 -- 172.21.15.1:0/3371874490 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f804a070 con 0x556195632b40 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.212+0000 7f15e97fa640 1 -- 172.21.15.1:0/3371874490 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f15f00bab70 con 0x556195632b40 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f160a0e0740 10 client.25696 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f160a0e0740 10 client.25696 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f160a0e0740 20 client.25696 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5561956bbbf0 0x5561956bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f160a0e0740 4 client.25696 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f160a0e0740 1 -- 172.21.15.1:0/3371874490 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55619559ed70 con 0x5561956bbbf0 2024-09-16T00:33:04.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f1605aa3640 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5561956bbbf0 0x5561956bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f15e97fa640 10 client.25696 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f1605aa3640 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5561956bbbf0 0x5561956bdfe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34ec33e5aad62a6c server_cookie=36d8f4b35fef51f2 in_seq=0 out_seq=0 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f15e97fa640 1 -- 172.21.15.1:0/3371874490 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55619559ed70 con 0x5561956bbbf0 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f15e97fa640 10 client.25696 handle_command_reply: tid=0 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.213+0000 7f160a0e0740 1 client.25696 shutdown 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 -- 172.21.15.1:0/3371874490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 msgr2=0x55619563a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 0x55619563a120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f15f00a4b10 tx=0x7f15f00053f0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 -- 172.21.15.1:0/3371874490 shutdown_connections 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556195640780 0x55619563a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556195632b40 0x55619563a120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5561956bbbf0 0x5561956bdfe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 --2- 172.21.15.1:0/3371874490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561956315d0 0x556195649040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f15e97fa640 0 client.25696 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 -- 172.21.15.1:0/3371874490 >> 172.21.15.1:0/3371874490 conn(0x5561955fa310 msgr2=0x55619563d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 -- 172.21.15.1:0/3371874490 shutdown_connections 2024-09-16T00:33:04.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 1 -- 172.21.15.1:0/3371874490 wait complete. 2024-09-16T00:33:04.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f160a0e0740 20 client.25696 trim_cache size 0 max 16384 2024-09-16T00:33:04.226 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f1607d07640 1 -- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15ec0a3240 msgr2=0x7f15ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f1607d07640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15ec0a3240 0x7f15ec0a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f15f8006010 tx=0x7f15f8004a40 comp rx=0 tx=0).stop 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f1607d07640 1 -- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1600103660 msgr2=0x7f1600198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.214+0000 7f1607d07640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1600103660 0x7f1600198910 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c940 tx=0x7f15f000ce00 comp rx=0 tx=0).stop 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.215+0000 7f1607d07640 1 -- 172.21.15.1:0/420319159 shutdown_connections 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.215+0000 7f1607d07640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f160010c170 0x7f160019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.215+0000 7f1607d07640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1600103660 0x7f1600198910 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.215+0000 7f1607d07640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15ec0a3240 0x7f15ec0a5700 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.215+0000 7f1607d07640 1 --2- 172.21.15.1:0/420319159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1600102d40 0x7f16001983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.215+0000 7f1607d07640 1 -- 172.21.15.1:0/420319159 >> 172.21.15.1:0/420319159 conn(0x7f16000fc7a0 msgr2=0x7f16000fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:04.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.215+0000 7f1607d07640 1 -- 172.21.15.1:0/420319159 shutdown_connections 2024-09-16T00:33:04.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:04.215+0000 7f1607d07640 1 -- 172.21.15.1:0/420319159 wait complete. 2024-09-16T00:33:04.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:04 smithi019 ceph-mon[35186]: pgmap v490: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:33:04.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:04 smithi001 ceph-mon[30934]: pgmap v490: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:33:04.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:04 smithi153 ceph-mon[35538]: pgmap v490: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:33:05.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:33:05.418 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:05.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.893+0000 7f31304c4640 1 -- 172.21.15.1:0/2677462666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128106a80 msgr2=0x7f3128106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.893+0000 7f31304c4640 1 --2- 172.21.15.1:0/2677462666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128106a80 0x7f3128106e60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3124009930 tx=0x7f312402a420 comp rx=0 tx=0).stop 2024-09-16T00:33:05.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.893+0000 7f31304c4640 1 -- 172.21.15.1:0/2677462666 shutdown_connections 2024-09-16T00:33:05.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.893+0000 7f31304c4640 1 --2- 172.21.15.1:0/2677462666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3128101420 0x7f312810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.893+0000 7f31304c4640 1 --2- 172.21.15.1:0/2677462666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3128100a80 0x7f3128100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.893+0000 7f31304c4640 1 --2- 172.21.15.1:0/2677462666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128106a80 0x7f3128106e60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.893+0000 7f31304c4640 1 -- 172.21.15.1:0/2677462666 >> 172.21.15.1:0/2677462666 conn(0x7f31280fc7a0 msgr2=0x7f31280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.893+0000 7f31304c4640 1 -- 172.21.15.1:0/2677462666 shutdown_connections 2024-09-16T00:33:05.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.893+0000 7f31304c4640 1 -- 172.21.15.1:0/2677462666 wait complete. 2024-09-16T00:33:05.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.894+0000 7f31304c4640 1 Processor -- start 2024-09-16T00:33:05.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.894+0000 7f31304c4640 1 -- start start 2024-09-16T00:33:05.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f31304c4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3128100a80 0x7f31281992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f31304c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128101420 0x7f31281997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f31304c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3128106a80 0x7f31281a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f31304c4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3128110e90 con 0x7f3128101420 2024-09-16T00:33:05.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f31304c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3128110d10 con 0x7f3128106a80 2024-09-16T00:33:05.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f31304c4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3128111010 con 0x7f3128100a80 2024-09-16T00:33:05.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312da5f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128101420 0x7f31281997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312da5f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128101420 0x7f31281997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58576/0 (socket says 172.21.15.1:58576) 2024-09-16T00:33:05.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312da5f640 1 -- 172.21.15.1:0/1829762187 learned_addr learned my addr 172.21.15.1:0/1829762187 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:05.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312e260640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3128100a80 0x7f31281992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312ea61640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3128106a80 0x7f31281a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312da5f640 1 -- 172.21.15.1:0/1829762187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3128100a80 msgr2=0x7f31281992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312da5f640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3128100a80 0x7f31281992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312da5f640 1 -- 172.21.15.1:0/1829762187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3128106a80 msgr2=0x7f31281a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312da5f640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3128106a80 0x7f31281a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312da5f640 1 -- 172.21.15.1:0/1829762187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31281a4860 con 0x7f3128101420 2024-09-16T00:33:05.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312ea61640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3128106a80 0x7f31281a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:05.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f312da5f640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128101420 0x7f31281997e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f311800c920 tx=0x7f311800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f311f7fe640 1 -- 172.21.15.1:0/1829762187 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118007c70 con 0x7f3128101420 2024-09-16T00:33:05.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f311f7fe640 1 -- 172.21.15.1:0/1829762187 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3118007e10 con 0x7f3128101420 2024-09-16T00:33:05.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.895+0000 7f31304c4640 1 -- 172.21.15.1:0/1829762187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31281a4a30 con 0x7f3128101420 2024-09-16T00:33:05.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.896+0000 7f311f7fe640 1 -- 172.21.15.1:0/1829762187 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118018890 con 0x7f3128101420 2024-09-16T00:33:05.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.896+0000 7f31304c4640 1 -- 172.21.15.1:0/1829762187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31281a4f40 con 0x7f3128101420 2024-09-16T00:33:05.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.898+0000 7f311f7fe640 1 -- 172.21.15.1:0/1829762187 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3118018a70 con 0x7f3128101420 2024-09-16T00:33:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.898+0000 7f313289d740 1 Processor -- start 2024-09-16T00:33:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.898+0000 7f313289d740 1 -- start start 2024-09-16T00:33:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f313289d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 0x557d6ad1c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f313289d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad1ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f313289d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad2b780 0x557d6ad2db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f313289d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557d6ac822f0 con 0x557d6ad1c5d0 2024-09-16T00:33:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f313289d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d6ac29cf0 con 0x557d6ad1db40 2024-09-16T00:33:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f313289d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557d6ac7e2b0 con 0x557d6ad2b780 2024-09-16T00:33:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312e260640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad1ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312e260640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad1ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58580/0 (socket says 172.21.15.1:58580) 2024-09-16T00:33:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312e260640 1 -- 172.21.15.1:0/487760382 learned_addr learned my addr 172.21.15.1:0/487760382 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f311f7fe640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f31140a3240 0x7f31140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312da5f640 1 --2- 172.21.15.1:0/487760382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad2b780 0x557d6ad2db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312ea61640 1 --2- 172.21.15.1:0/487760382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 0x557d6ad1c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312ea61640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f31140a3240 0x7f31140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312e260640 1 -- 172.21.15.1:0/487760382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad2b780 msgr2=0x557d6ad2db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312e260640 1 --2- 172.21.15.1:0/487760382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad2b780 0x557d6ad2db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312e260640 1 -- 172.21.15.1:0/487760382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 msgr2=0x557d6ad1c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312e260640 1 --2- 172.21.15.1:0/487760382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 0x557d6ad1c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312e260640 1 -- 172.21.15.1:0/487760382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6ac0e3a0 con 0x557d6ad1c5d0 2024-09-16T00:33:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f311f7fe640 1 -- 172.21.15.1:0/1829762187 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f31180cfec0 con 0x7f3128101420 2024-09-16T00:33:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312e260640 1 --2- 172.21.15.1:0/487760382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad1ca10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3124009930 tx=0x7f3124005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c499a004714de8f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.899+0000 7f312ea61640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f31140a3240 0x7f31140a5700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3120013540 tx=0x7f3120005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f311cff9640 1 -- 172.21.15.1:0/487760382 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124040070 con 0x557d6ad1c5d0 2024-09-16T00:33:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f311cff9640 1 -- 172.21.15.1:0/487760382 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124004490 con 0x557d6ad1c5d0 2024-09-16T00:33:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f311cff9640 1 -- 172.21.15.1:0/487760382 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124035040 con 0x557d6ad1c5d0 2024-09-16T00:33:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f313289d740 1 -- 172.21.15.1:0/487760382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 msgr2=0x557d6ad1ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f313289d740 1 --2- 172.21.15.1:0/487760382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad1ca10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3124009930 tx=0x7f3124005c00 comp rx=0 tx=0).stop 2024-09-16T00:33:05.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f313289d740 1 -- 172.21.15.1:0/487760382 shutdown_connections 2024-09-16T00:33:05.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f313289d740 1 --2- 172.21.15.1:0/487760382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad2b780 0x557d6ad2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f313289d740 1 --2- 172.21.15.1:0/487760382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad1ca10 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f313289d740 1 --2- 172.21.15.1:0/487760382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 0x557d6ad1c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f313289d740 1 -- 172.21.15.1:0/487760382 >> 172.21.15.1:0/487760382 conn(0x557d6ace5310 msgr2=0x557d6ad18c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f313289d740 1 -- 172.21.15.1:0/487760382 shutdown_connections 2024-09-16T00:33:05.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.900+0000 7f313289d740 1 -- 172.21.15.1:0/487760382 wait complete. 2024-09-16T00:33:05.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f313289d740 1 -- start start 2024-09-16T00:33:05.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f313289d740 1 Processor -- start 2024-09-16T00:33:05.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f313289d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 0x557d6ad2b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f313289d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 0x557d6ad283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f313289d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f313289d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557d6ad2e420 con 0x557d6ad2b780 2024-09-16T00:33:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f313289d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d6ac3d060 con 0x557d6ad1c5d0 2024-09-16T00:33:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f313289d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557d6ad2e5a0 con 0x557d6ad1db40 2024-09-16T00:33:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f313289d740 10 client.0 _ll_register_callbacks cb 0x557d6a9908c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f312ea61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f312ea61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad289e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58586/0 (socket says 172.21.15.1:58586) 2024-09-16T00:33:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f312ea61640 1 -- 172.21.15.1:0/1706464781 learned_addr learned my addr 172.21.15.1:0/1706464781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:05.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.901+0000 7f312da5f640 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 0x557d6ad283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f312e260640 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 0x557d6ad2b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f312ea61640 1 -- 172.21.15.1:0/1706464781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 msgr2=0x557d6ad283f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f312ea61640 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 0x557d6ad283f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f312ea61640 1 -- 172.21.15.1:0/1706464781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 msgr2=0x557d6ad2b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f312ea61640 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 0x557d6ad2b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f312ea61640 1 -- 172.21.15.1:0/1706464781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6a9f33c0 con 0x557d6ad2b780 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f312ea61640 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad289e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f312000ac70 tx=0x7f3120004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f73b843eeec691 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f3111ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f3111ffb640 1 -- 172.21.15.1:0/1706464781 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120024020 con 0x557d6ad2b780 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f3111ffb640 1 -- 172.21.15.1:0/1706464781 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f312000ed10 con 0x557d6ad2b780 2024-09-16T00:33:05.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f313289d740 1 -- 172.21.15.1:0/1706464781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3124004490 con 0x557d6ad2b780 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f3111ffb640 1 -- 172.21.15.1:0/1706464781 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f312002f050 con 0x557d6ad2b780 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f313289d740 10 client.16392 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.902+0000 7f313289d740 1 -- 172.21.15.1:0/1706464781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d6ad39eb0 con 0x557d6ad2b780 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.903+0000 7f3111ffb640 1 -- 172.21.15.1:0/1706464781 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124004490 con 0x557d6ad2b780 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.903+0000 7f3111ffb640 1 -- 172.21.15.1:0/1706464781 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f312001f030 con 0x557d6ad2b780 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.903+0000 7f313289d740 10 client.16392 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.903+0000 7f313289d740 10 client.16392 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.903+0000 7f313289d740 20 client.16392 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.903+0000 7f313289d740 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x557d6ad3b370 0x557d6ad3d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.903+0000 7f313289d740 4 client.16392 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.903+0000 7f313289d740 1 -- 172.21.15.1:0/1706464781 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d6ac8f860 con 0x557d6ad3b370 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.903+0000 7f312e260640 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x557d6ad3b370 0x557d6ad3d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.904+0000 7f312e260640 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x557d6ad3b370 0x557d6ad3d810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a916df2f14d44b38 server_cookie=e8ae6a1b82ba7f42 in_seq=0 out_seq=0 2024-09-16T00:33:05.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.904+0000 7f3111ffb640 10 client.16392 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.904+0000 7f3111ffb640 1 -- 172.21.15.1:0/1706464781 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d6ac8f860 con 0x557d6ad3b370 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.904+0000 7f3111ffb640 10 client.16392 handle_command_reply: tid=0 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.904+0000 7f313289d740 1 client.16392 shutdown 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.904+0000 7f313289d740 1 -- 172.21.15.1:0/1706464781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 msgr2=0x557d6ad289e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.904+0000 7f313289d740 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad289e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f312000ac70 tx=0x7f3120004a40 comp rx=0 tx=0).stop 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f313289d740 1 -- 172.21.15.1:0/1706464781 shutdown_connections 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f313289d740 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x557d6ad3b370 0x557d6ad3d810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f313289d740 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad289e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f313289d740 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 0x557d6ad283f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f3111ffb640 0 client.16392 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:33:05.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f313289d740 1 --2- 172.21.15.1:0/1706464781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 0x557d6ad2b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f313289d740 1 -- 172.21.15.1:0/1706464781 >> 172.21.15.1:0/1706464781 conn(0x557d6ace5310 msgr2=0x557d6ad1e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f313289d740 1 -- 172.21.15.1:0/1706464781 shutdown_connections 2024-09-16T00:33:05.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f313289d740 1 -- 172.21.15.1:0/1706464781 wait complete. 2024-09-16T00:33:05.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.905+0000 7f313289d740 20 client.16392 trim_cache size 0 max 16384 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f313289d740 1 Processor -- start 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f313289d740 1 -- start start 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f313289d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad18e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f313289d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 0x557d6ad376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f313289d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad2b780 0x557d6ad27cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f313289d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f31281af510 con 0x557d6ad1c5d0 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f313289d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31281acde0 con 0x557d6ad1db40 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f313289d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31281aac30 con 0x557d6ad2b780 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312e260640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad18e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312e260640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad18e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58590/0 (socket says 172.21.15.1:58590) 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312e260640 1 -- 172.21.15.1:0/2996095859 learned_addr learned my addr 172.21.15.1:0/2996095859 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:05.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312da5f640 1 --2- 172.21.15.1:0/2996095859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 0x557d6ad376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312e260640 1 -- 172.21.15.1:0/2996095859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad2b780 msgr2=0x557d6ad27cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312e260640 1 --2- 172.21.15.1:0/2996095859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad2b780 0x557d6ad27cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312e260640 1 -- 172.21.15.1:0/2996095859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 msgr2=0x557d6ad376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312e260640 1 --2- 172.21.15.1:0/2996095859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 0x557d6ad376f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312e260640 1 -- 172.21.15.1:0/2996095859 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6ad273a0 con 0x557d6ad1c5d0 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.918+0000 7f312e260640 1 --2- 172.21.15.1:0/2996095859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad18e10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x557d6ad19ac0 tx=0x7f3124004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8715dcd1bc535f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f3113fff640 1 -- 172.21.15.1:0/2996095859 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124054070 con 0x557d6ad1c5d0 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f3113fff640 1 -- 172.21.15.1:0/2996095859 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124004ae0 con 0x557d6ad1c5d0 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f3113fff640 1 -- 172.21.15.1:0/2996095859 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124040040 con 0x557d6ad1c5d0 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f313289d740 1 -- 172.21.15.1:0/2996095859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 msgr2=0x557d6ad18e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f313289d740 1 --2- 172.21.15.1:0/2996095859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad18e10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x557d6ad19ac0 tx=0x7f3124004db0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f313289d740 1 -- 172.21.15.1:0/2996095859 shutdown_connections 2024-09-16T00:33:05.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f313289d740 1 --2- 172.21.15.1:0/2996095859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad2b780 0x557d6ad27cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f313289d740 1 --2- 172.21.15.1:0/2996095859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1db40 0x557d6ad376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f313289d740 1 --2- 172.21.15.1:0/2996095859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad1c5d0 0x557d6ad18e10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f313289d740 1 -- 172.21.15.1:0/2996095859 >> 172.21.15.1:0/2996095859 conn(0x557d6ace5310 msgr2=0x557d6ad18700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f313289d740 1 -- 172.21.15.1:0/2996095859 shutdown_connections 2024-09-16T00:33:05.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.919+0000 7f313289d740 1 -- 172.21.15.1:0/2996095859 wait complete. 2024-09-16T00:33:05.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f313289d740 1 -- start start 2024-09-16T00:33:05.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f313289d740 1 Processor -- start 2024-09-16T00:33:05.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f313289d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 0x557d6ad2f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f313289d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 0x557d6ad2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f313289d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad30240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f313289d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f31281ab2a0 con 0x557d6ad2b780 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f313289d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31281ac710 con 0x557d6ad1c5d0 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f313289d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31281ab970 con 0x557d6ad1db40 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f313289d740 10 client.0 _ll_register_callbacks cb 0x7f312000a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312ea61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad30240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312ea61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad30240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58600/0 (socket says 172.21.15.1:58600) 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312ea61640 1 -- 172.21.15.1:0/619388693 learned_addr learned my addr 172.21.15.1:0/619388693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312da5f640 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 0x557d6ad2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312e260640 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 0x557d6ad2f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312ea61640 1 -- 172.21.15.1:0/619388693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 msgr2=0x557d6ad2fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312ea61640 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 0x557d6ad2fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312ea61640 1 -- 172.21.15.1:0/619388693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 msgr2=0x557d6ad2f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312ea61640 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 0x557d6ad2f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.920+0000 7f312ea61640 1 -- 172.21.15.1:0/619388693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6ada5300 con 0x557d6ad2b780 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f312e260640 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 0x557d6ad2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f312ea61640 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad30240 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3120015040 tx=0x7f312000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943b6a20c07df422 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f3111ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f3111ffb640 1 -- 172.21.15.1:0/619388693 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120024030 con 0x557d6ad2b780 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f3111ffb640 1 -- 172.21.15.1:0/619388693 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3120013040 con 0x557d6ad2b780 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f313289d740 1 -- 172.21.15.1:0/619388693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3124004ae0 con 0x557d6ad2b780 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f3111ffb640 1 -- 172.21.15.1:0/619388693 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3120041050 con 0x557d6ad2b780 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f313289d740 10 client.16398 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:05.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f313289d740 1 -- 172.21.15.1:0/619388693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d6ada65e0 con 0x557d6ad2b780 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f3111ffb640 1 -- 172.21.15.1:0/619388693 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124004ae0 con 0x557d6ad2b780 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.921+0000 7f3111ffb640 1 -- 172.21.15.1:0/619388693 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f312001f040 con 0x557d6ad2b780 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.922+0000 7f313289d740 10 client.16398 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.922+0000 7f313289d740 10 client.16398 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.922+0000 7f313289d740 20 client.16398 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.922+0000 7f313289d740 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x557d6ada6cf0 0x557d6ada90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.922+0000 7f313289d740 4 client.16398 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.922+0000 7f313289d740 1 -- 172.21.15.1:0/619388693 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d6ac89d70 con 0x557d6ada6cf0 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.922+0000 7f312e260640 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x557d6ada6cf0 0x557d6ada90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.922+0000 7f312e260640 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x557d6ada6cf0 0x557d6ada90e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=943c89d4d40a3a2b server_cookie=26b7847d99d92fca in_seq=0 out_seq=0 2024-09-16T00:33:05.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.922+0000 7f3111ffb640 10 client.16398 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f3111ffb640 1 -- 172.21.15.1:0/619388693 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d6ac89d70 con 0x557d6ada6cf0 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f3111ffb640 10 client.16398 handle_command_reply: tid=0 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 client.16398 shutdown 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 -- 172.21.15.1:0/619388693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 msgr2=0x557d6ad30240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad30240 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3120015040 tx=0x7f312000da90 comp rx=0 tx=0).stop 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 -- 172.21.15.1:0/619388693 shutdown_connections 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x557d6ada6cf0 0x557d6ada90e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d6ad2b780 0x557d6ad30240 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d6ad1db40 0x557d6ad2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 --2- 172.21.15.1:0/619388693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6ad1c5d0 0x557d6ad2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 -- 172.21.15.1:0/619388693 >> 172.21.15.1:0/619388693 conn(0x557d6ace5310 msgr2=0x557d6ad29c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f3111ffb640 0 client.16398 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:33:05.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 -- 172.21.15.1:0/619388693 shutdown_connections 2024-09-16T00:33:05.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 1 -- 172.21.15.1:0/619388693 wait complete. 2024-09-16T00:33:05.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.923+0000 7f313289d740 20 client.16398 trim_cache size 0 max 16384 2024-09-16T00:33:05.936 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:33:05.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 -- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f31140a3240 msgr2=0x7f31140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f31140a3240 0x7f31140a5700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3120013540 tx=0x7f3120005c60 comp rx=0 tx=0).stop 2024-09-16T00:33:05.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 -- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128101420 msgr2=0x7f31281997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128101420 0x7f31281997e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f311800c920 tx=0x7f311800cde0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 -- 172.21.15.1:0/1829762187 shutdown_connections 2024-09-16T00:33:05.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3128106a80 0x7f31281a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3128101420 0x7f31281997e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f31140a3240 0x7f31140a5700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 --2- 172.21.15.1:0/1829762187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3128100a80 0x7f31281992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 -- 172.21.15.1:0/1829762187 >> 172.21.15.1:0/1829762187 conn(0x7f31280fc7a0 msgr2=0x7f31280fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 -- 172.21.15.1:0/1829762187 shutdown_connections 2024-09-16T00:33:05.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:05.924+0000 7f31304c4640 1 -- 172.21.15.1:0/1829762187 wait complete. 2024-09-16T00:33:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:06 smithi019 ceph-mon[35186]: pgmap v491: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:06.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:06 smithi001 ceph-mon[30934]: pgmap v491: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:06 smithi153 ceph-mon[35538]: pgmap v491: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:06.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:33:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:07.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 -- 172.21.15.1:0/2908089117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854101420 msgr2=0x7f885410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 --2- 172.21.15.1:0/2908089117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854101420 0x7f885410d950 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f884c009cd0 tx=0x7f884c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 -- 172.21.15.1:0/2908089117 shutdown_connections 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 --2- 172.21.15.1:0/2908089117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854101420 0x7f885410d950 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 --2- 172.21.15.1:0/2908089117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8854100a80 0x7f8854100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 --2- 172.21.15.1:0/2908089117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8854106a80 0x7f8854106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 -- 172.21.15.1:0/2908089117 >> 172.21.15.1:0/2908089117 conn(0x7f88540fc7a0 msgr2=0x7f88540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 -- 172.21.15.1:0/2908089117 shutdown_connections 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 -- 172.21.15.1:0/2908089117 wait complete. 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.589+0000 7f88588bb640 1 Processor -- start 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f88588bb640 1 -- start start 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f88588bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854100a80 0x7f885419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f88588bb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8854101420 0x7f885419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f88588bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8854106a80 0x7f885419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f88588bb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8854110f10 con 0x7f8854100a80 2024-09-16T00:33:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f88588bb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8854110d90 con 0x7f8854106a80 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f88588bb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8854111090 con 0x7f8854101420 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f885259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854100a80 0x7f885419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f885259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854100a80 0x7f885419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58608/0 (socket says 172.21.15.1:58608) 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f885259c640 1 -- 172.21.15.1:0/285549890 learned_addr learned my addr 172.21.15.1:0/285549890 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f8851d9b640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8854101420 0x7f885419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f8852d9d640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8854106a80 0x7f885419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f885259c640 1 -- 172.21.15.1:0/285549890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8854101420 msgr2=0x7f885419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f885259c640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8854101420 0x7f885419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f885259c640 1 -- 172.21.15.1:0/285549890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8854106a80 msgr2=0x7f885419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f885259c640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8854106a80 0x7f885419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f885259c640 1 -- 172.21.15.1:0/285549890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885419f650 con 0x7f8854100a80 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f8852d9d640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8854106a80 0x7f885419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.590+0000 7f885259c640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854100a80 0x7f885419a6a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f884400b4a0 tx=0x7f884400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.591+0000 7f88437fe640 1 -- 172.21.15.1:0/285549890 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8844014070 con 0x7f8854100a80 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.591+0000 7f88588bb640 1 -- 172.21.15.1:0/285549890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885419f940 con 0x7f8854100a80 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.591+0000 7f88437fe640 1 -- 172.21.15.1:0/285549890 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8844010af0 con 0x7f8854100a80 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.591+0000 7f88437fe640 1 -- 172.21.15.1:0/285549890 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8844017d10 con 0x7f8854100a80 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.591+0000 7f88588bb640 1 -- 172.21.15.1:0/285549890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88541a7220 con 0x7f8854100a80 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.593+0000 7f88437fe640 1 -- 172.21.15.1:0/285549890 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8844017070 con 0x7f8854100a80 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.593+0000 7f885ac94740 1 Processor -- start 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f885ac94740 1 -- start start 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f885ac94740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f885ac94740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 0x565394c07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f885ac94740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c14760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f885ac94740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x565394b6d2f0 con 0x565394c08b40 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f885ac94740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565394b14cf0 con 0x565394c075d0 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f885ac94740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x565394b692b0 con 0x565394c12370 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f8852d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f8852d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c07090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58620/0 (socket says 172.21.15.1:58620) 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f8852d9d640 1 -- 172.21.15.1:0/4144866897 learned_addr learned my addr 172.21.15.1:0/4144866897 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f885259c640 1 --2- 172.21.15.1:0/4144866897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 0x565394c07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f88437fe640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f883c0a3240 0x7f883c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f8851d9b640 1 --2- 172.21.15.1:0/4144866897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c14760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f8852d9d640 1 -- 172.21.15.1:0/4144866897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 msgr2=0x565394c14760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f8852d9d640 1 --2- 172.21.15.1:0/4144866897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c14760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f8852d9d640 1 -- 172.21.15.1:0/4144866897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 msgr2=0x565394c07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f8852d9d640 1 --2- 172.21.15.1:0/4144866897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 0x565394c07a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.594+0000 7f8852d9d640 1 -- 172.21.15.1:0/4144866897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565394af93a0 con 0x565394c08b40 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.595+0000 7f8852d9d640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f883c0a3240 0x7f883c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.595+0000 7f88437fe640 1 -- 172.21.15.1:0/285549890 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f88440cf0e0 con 0x7f8854100a80 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.595+0000 7f8852d9d640 1 --2- 172.21.15.1:0/4144866897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c07090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f884c009df0 tx=0x7f884c031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e296754e7d88cec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.595+0000 7f8840ff9640 1 -- 172.21.15.1:0/4144866897 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c04d070 con 0x565394c08b40 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.595+0000 7f8840ff9640 1 -- 172.21.15.1:0/4144866897 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f884c043070 con 0x565394c08b40 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.595+0000 7f8840ff9640 1 -- 172.21.15.1:0/4144866897 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c030dd0 con 0x565394c08b40 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.595+0000 7f8852d9d640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f883c0a3240 0x7f883c0a5700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f884c04a000 tx=0x7f884c04b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 -- 172.21.15.1:0/4144866897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 msgr2=0x565394c07090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 --2- 172.21.15.1:0/4144866897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c07090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f884c009df0 tx=0x7f884c031660 comp rx=0 tx=0).stop 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 -- 172.21.15.1:0/4144866897 shutdown_connections 2024-09-16T00:33:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 --2- 172.21.15.1:0/4144866897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c14760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 --2- 172.21.15.1:0/4144866897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 0x565394c07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 --2- 172.21.15.1:0/4144866897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c07090 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 -- 172.21.15.1:0/4144866897 >> 172.21.15.1:0/4144866897 conn(0x565394bd0310 msgr2=0x565394c03c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 -- 172.21.15.1:0/4144866897 shutdown_connections 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 -- 172.21.15.1:0/4144866897 wait complete. 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 -- start start 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 Processor -- start 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.596+0000 7f885ac94740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 0x565394c11d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f885ac94740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c0f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f885ac94740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c0f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f885ac94740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x565394c15070 con 0x565394c08b40 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f885ac94740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565394b28060 con 0x565394c075d0 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f885ac94740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x565394c151f0 con 0x565394c12370 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f885ac94740 10 client.0 _ll_register_callbacks cb 0x56539487b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8851d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c0f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8851d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c0f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58634/0 (socket says 172.21.15.1:58634) 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8851d9b640 1 -- 172.21.15.1:0/302347050 learned_addr learned my addr 172.21.15.1:0/302347050 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8852d9d640 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f885259c640 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 0x565394c11d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8851d9b640 1 -- 172.21.15.1:0/302347050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 msgr2=0x565394c0f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8851d9b640 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c0f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8851d9b640 1 -- 172.21.15.1:0/302347050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 msgr2=0x565394c11d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8851d9b640 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 0x565394c11d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8851d9b640 1 -- 172.21.15.1:0/302347050 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653948de3c0 con 0x565394c08b40 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f885259c640 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 0x565394c11d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8851d9b640 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c0f010 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8848012750 tx=0x7f8848012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b9a632f3b7e2b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8839ffb640 1 -- 172.21.15.1:0/302347050 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884801f060 con 0x565394c08b40 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f885ac94740 1 -- 172.21.15.1:0/302347050 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f884c043070 con 0x565394c08b40 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8839ffb640 1 -- 172.21.15.1:0/302347050 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8848010dc0 con 0x565394c08b40 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.597+0000 7f8839ffb640 1 -- 172.21.15.1:0/302347050 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8848026050 con 0x565394c08b40 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f885ac94740 10 client.16410 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f885ac94740 1 -- 172.21.15.1:0/302347050 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565394c20bc0 con 0x565394c08b40 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f8839ffb640 1 -- 172.21.15.1:0/302347050 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f884c043070 con 0x565394c08b40 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f8839ffb640 1 -- 172.21.15.1:0/302347050 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f884800d5e0 con 0x565394c08b40 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f885ac94740 10 client.16410 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f885ac94740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f885ac94740 20 client.16410 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f885ac94740 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565394c22020 0x565394c24460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f885ac94740 4 client.16410 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f885ac94740 1 -- 172.21.15.1:0/302347050 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565394b7a860 con 0x565394c22020 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.598+0000 7f885259c640 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565394c22020 0x565394c24460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.599+0000 7f8839ffb640 10 client.16410 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.599+0000 7f885259c640 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565394c22020 0x565394c24460 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64f369a9bd98ff1 server_cookie=9056f6f23983978d in_seq=0 out_seq=0 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.599+0000 7f8839ffb640 1 -- 172.21.15.1:0/302347050 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565394b7a860 con 0x565394c22020 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.599+0000 7f8839ffb640 10 client.16410 handle_command_reply: tid=0 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.599+0000 7f885ac94740 1 client.16410 shutdown 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.599+0000 7f885ac94740 1 -- 172.21.15.1:0/302347050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 msgr2=0x565394c0f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.599+0000 7f885ac94740 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c0f010 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8848012750 tx=0x7f8848012c10 comp rx=0 tx=0).stop 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f885ac94740 1 -- 172.21.15.1:0/302347050 shutdown_connections 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f885ac94740 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565394c22020 0x565394c24460 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f885ac94740 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c0f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f885ac94740 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c08b40 0x565394c0f010 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f8839ffb640 0 client.16410 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f885ac94740 1 --2- 172.21.15.1:0/302347050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c075d0 0x565394c11d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f885ac94740 1 -- 172.21.15.1:0/302347050 >> 172.21.15.1:0/302347050 conn(0x565394bd0310 msgr2=0x565394c09930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f885ac94740 1 -- 172.21.15.1:0/302347050 shutdown_connections 2024-09-16T00:33:07.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f885ac94740 1 -- 172.21.15.1:0/302347050 wait complete. 2024-09-16T00:33:07.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.600+0000 7f885ac94740 20 client.16410 trim_cache size 0 max 16384 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.612+0000 7f885ac94740 1 Processor -- start 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.612+0000 7f885ac94740 1 -- start start 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f885ac94740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c075d0 0x565394c10f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f885ac94740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f885ac94740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c12370 0x565394c0a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f885ac94740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88541b39c0 con 0x565394c12370 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f885ac94740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88541b1290 con 0x565394c08b40 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f885ac94740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f88541af0e0 con 0x565394c075d0 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8852d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c12370 0x565394c0a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8852d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c12370 0x565394c0a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58648/0 (socket says 172.21.15.1:58648) 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8852d9d640 1 -- 172.21.15.1:0/2855122291 learned_addr learned my addr 172.21.15.1:0/2855122291 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f885259c640 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c075d0 0x565394c10f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8851d9b640 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8852d9d640 1 -- 172.21.15.1:0/2855122291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c075d0 msgr2=0x565394c10f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8852d9d640 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c075d0 0x565394c10f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8852d9d640 1 -- 172.21.15.1:0/2855122291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 msgr2=0x565394c0a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8852d9d640 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8852d9d640 1 -- 172.21.15.1:0/2855122291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565394c0f970 con 0x565394c12370 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8851d9b640 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.613+0000 7f8852d9d640 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c12370 0x565394c0a660 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f884c006010 tx=0x7f884c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d92b5191f16d30c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f883bfff640 1 -- 172.21.15.1:0/2855122291 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c04d030 con 0x565394c12370 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f883bfff640 1 -- 172.21.15.1:0/2855122291 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f884c043070 con 0x565394c12370 2024-09-16T00:33:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f883bfff640 1 -- 172.21.15.1:0/2855122291 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c030890 con 0x565394c12370 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f885ac94740 1 -- 172.21.15.1:0/2855122291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c12370 msgr2=0x565394c0a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f885ac94740 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c12370 0x565394c0a660 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f884c006010 tx=0x7f884c009800 comp rx=0 tx=0).stop 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f885ac94740 1 -- 172.21.15.1:0/2855122291 shutdown_connections 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f885ac94740 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c12370 0x565394c0a660 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f885ac94740 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f885ac94740 1 --2- 172.21.15.1:0/2855122291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c075d0 0x565394c10f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f885ac94740 1 -- 172.21.15.1:0/2855122291 >> 172.21.15.1:0/2855122291 conn(0x565394bd0310 msgr2=0x565394c10870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f885ac94740 1 -- 172.21.15.1:0/2855122291 shutdown_connections 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.614+0000 7f885ac94740 1 -- 172.21.15.1:0/2855122291 wait complete. 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885ac94740 1 -- start start 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885ac94740 1 Processor -- start 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885ac94740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c075d0 0x565394c0c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885ac94740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885ac94740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885ac94740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88541af750 con 0x565394c075d0 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885ac94740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88541b0bc0 con 0x565394c08b40 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885ac94740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f88541afe20 con 0x565394c12370 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885ac94740 10 client.0 _ll_register_callbacks cb 0x7f884800a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c075d0 0x565394c0c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c075d0 0x565394c0c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58662/0 (socket says 172.21.15.1:58662) 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885259c640 1 -- 172.21.15.1:0/2332308491 learned_addr learned my addr 172.21.15.1:0/2332308491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f8852d9d640 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f8851d9b640 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.615+0000 7f885259c640 1 -- 172.21.15.1:0/2332308491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 msgr2=0x565394c0cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f885259c640 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c0cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f885259c640 1 -- 172.21.15.1:0/2332308491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 msgr2=0x565394c0c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f885259c640 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f885259c640 1 -- 172.21.15.1:0/2332308491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565394c8bdb0 con 0x565394c075d0 2024-09-16T00:33:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f8852d9d640 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f8851d9b640 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f885259c640 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c075d0 0x565394c0c100 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8844097760 tx=0x7f8844000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57331326d791de51 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f8839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f8839ffb640 1 -- 172.21.15.1:0/2332308491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884401e040 con 0x565394c075d0 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f8839ffb640 1 -- 172.21.15.1:0/2332308491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8844017360 con 0x565394c075d0 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f885ac94740 1 -- 172.21.15.1:0/2332308491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f884c043070 con 0x565394c075d0 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f8839ffb640 1 -- 172.21.15.1:0/2332308491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f884409f6b0 con 0x565394c075d0 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f885ac94740 10 client.16416 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f885ac94740 1 -- 172.21.15.1:0/2332308491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565394c8d060 con 0x565394c075d0 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f8839ffb640 1 -- 172.21.15.1:0/2332308491 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f884c043070 con 0x565394c075d0 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.616+0000 7f8839ffb640 1 -- 172.21.15.1:0/2332308491 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f88440a1ea0 con 0x565394c075d0 2024-09-16T00:33:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f885ac94740 10 client.16416 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f885ac94740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f885ac94740 20 client.16416 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f885ac94740 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565394c8d740 0x565394c8fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f885ac94740 4 client.16416 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f885ac94740 1 -- 172.21.15.1:0/2332308491 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565394b74d70 con 0x565394c8d740 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f8851d9b640 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565394c8d740 0x565394c8fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f8851d9b640 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565394c8d740 0x565394c8fb60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb9a1b0e167a470 server_cookie=ce33208ac471ab88 in_seq=0 out_seq=0 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f8839ffb640 10 client.16416 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f8839ffb640 1 -- 172.21.15.1:0/2332308491 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565394b74d70 con 0x565394c8d740 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f8839ffb640 10 client.16416 handle_command_reply: tid=0 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.617+0000 7f885ac94740 1 client.16416 shutdown 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 -- 172.21.15.1:0/2332308491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c075d0 msgr2=0x565394c0c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c075d0 0x565394c0c100 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8844097760 tx=0x7f8844000bc0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 -- 172.21.15.1:0/2332308491 shutdown_connections 2024-09-16T00:33:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565394c8d740 0x565394c8fb60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565394c12370 0x565394c0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565394c08b40 0x565394c0c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 --2- 172.21.15.1:0/2332308491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565394c075d0 0x565394c0c100 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 -- 172.21.15.1:0/2332308491 >> 172.21.15.1:0/2332308491 conn(0x565394bd0310 msgr2=0x565394c11090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f8839ffb640 0 client.16416 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 -- 172.21.15.1:0/2332308491 shutdown_connections 2024-09-16T00:33:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 1 -- 172.21.15.1:0/2332308491 wait complete. 2024-09-16T00:33:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f885ac94740 20 client.16416 trim_cache size 0 max 16384 2024-09-16T00:33:07.627 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:33:07.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 -- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f883c0a3240 msgr2=0x7f883c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f883c0a3240 0x7f883c0a5700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f884c04a000 tx=0x7f884c04b000 comp rx=0 tx=0).stop 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 -- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854100a80 msgr2=0x7f885419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854100a80 0x7f885419a6a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f884400b4a0 tx=0x7f884400b960 comp rx=0 tx=0).stop 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 -- 172.21.15.1:0/285549890 shutdown_connections 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8854106a80 0x7f885419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f883c0a3240 0x7f883c0a5700 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8854101420 0x7f885419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 --2- 172.21.15.1:0/285549890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8854100a80 0x7f885419a6a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.618+0000 7f88588bb640 1 -- 172.21.15.1:0/285549890 >> 172.21.15.1:0/285549890 conn(0x7f88540fc7a0 msgr2=0x7f885406a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.619+0000 7f88588bb640 1 -- 172.21.15.1:0/285549890 shutdown_connections 2024-09-16T00:33:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:07.619+0000 7f88588bb640 1 -- 172.21.15.1:0/285549890 wait complete. 2024-09-16T00:33:08.520 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:08 smithi001 ceph-mon[30934]: pgmap v492: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:33:08.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:08 smithi019 ceph-mon[35186]: pgmap v492: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:33:08.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:08 smithi153 ceph-mon[35538]: pgmap v492: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:33:10.603 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:33:10.604 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:33:10.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:10 smithi019 ceph-mon[35186]: pgmap v493: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:33:10.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:10 smithi001 ceph-mon[30934]: pgmap v493: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:33:10.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:10 smithi153 ceph-mon[35538]: pgmap v493: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:33:10.875 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 -- 172.21.15.1:0/2044692757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c100e20 msgr2=0x7f547c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 --2- 172.21.15.1:0/2044692757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c100e20 0x7f547c101200 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5478009900 tx=0x7f547802a480 comp rx=0 tx=0).stop 2024-09-16T00:33:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 -- 172.21.15.1:0/2044692757 shutdown_connections 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 --2- 172.21.15.1:0/2044692757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f547c102ef0 0x7f547c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 --2- 172.21.15.1:0/2044692757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f547c102570 0x7f547c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 --2- 172.21.15.1:0/2044692757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c100e20 0x7f547c101200 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 -- 172.21.15.1:0/2044692757 >> 172.21.15.1:0/2044692757 conn(0x7f547c0fc7d0 msgr2=0x7f547c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 -- 172.21.15.1:0/2044692757 shutdown_connections 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 -- 172.21.15.1:0/2044692757 wait complete. 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.376+0000 7f548412a640 1 Processor -- start 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f548412a640 1 -- start start 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f548412a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f547c100e20 0x7f547c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f548412a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f547c102570 0x7f547c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f548412a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c102ef0 0x7f547c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f548412a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f547c110fa0 con 0x7f547c102ef0 2024-09-16T00:33:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f548412a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f547c110e20 con 0x7f547c102570 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f548412a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f547c111120 con 0x7f547c100e20 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54826c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c102ef0 0x7f547c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54826c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c102ef0 0x7f547c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54616/0 (socket says 172.21.15.1:54616) 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54826c7640 1 -- 172.21.15.1:0/2000649749 learned_addr learned my addr 172.21.15.1:0/2000649749 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f5481ec6640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f547c100e20 0x7f547c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54816c5640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f547c102570 0x7f547c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54826c7640 1 -- 172.21.15.1:0/2000649749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f547c100e20 msgr2=0x7f547c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54826c7640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f547c100e20 0x7f547c1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54826c7640 1 -- 172.21.15.1:0/2000649749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f547c102570 msgr2=0x7f547c1a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54826c7640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f547c102570 0x7f547c1a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54826c7640 1 -- 172.21.15.1:0/2000649749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c19acd0 con 0x7f547c102ef0 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f5481ec6640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f547c100e20 0x7f547c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54816c5640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f547c102570 0x7f547c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.377+0000 7f54826c7640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c102ef0 0x7f547c19a5a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f547400cad0 tx=0x7f547400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.378+0000 7f5472ffd640 1 -- 172.21.15.1:0/2000649749 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547400ae20 con 0x7f547c102ef0 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.378+0000 7f5472ffd640 1 -- 172.21.15.1:0/2000649749 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5474005020 con 0x7f547c102ef0 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.378+0000 7f5472ffd640 1 -- 172.21.15.1:0/2000649749 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547400f6f0 con 0x7f547c102ef0 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.378+0000 7f548412a640 1 -- 172.21.15.1:0/2000649749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c19afc0 con 0x7f547c102ef0 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.378+0000 7f548412a640 1 -- 172.21.15.1:0/2000649749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c1a72b0 con 0x7f547c102ef0 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.380+0000 7f5472ffd640 1 -- 172.21.15.1:0/2000649749 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f54740051d0 con 0x7f547c102ef0 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.380+0000 7f5486503740 1 Processor -- start 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.380+0000 7f5486503740 1 -- start start 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.380+0000 7f5486503740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed00f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.380+0000 7f5486503740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 0x55b04ed0b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.380+0000 7f5486503740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed0bd40 0x55b04ed0e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.380+0000 7f5486503740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b04ec66280 con 0x55b04ed00b60 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.380+0000 7f5486503740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b04ec0db10 con 0x55b04ed0bd40 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.380+0000 7f5486503740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b04ec62240 con 0x55b04ed01480 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54816c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed00f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54816c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed00f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54624/0 (socket says 172.21.15.1:54624) 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54816c5640 1 -- 172.21.15.1:0/3920384268 learned_addr learned my addr 172.21.15.1:0/3920384268 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f5481ec6640 1 --2- 172.21.15.1:0/3920384268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 0x55b04ed0b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f5472ffd640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f54680a32c0 0x7f54680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54816c5640 1 -- 172.21.15.1:0/3920384268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 msgr2=0x55b04ed0b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54816c5640 1 --2- 172.21.15.1:0/3920384268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 0x55b04ed0b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54816c5640 1 -- 172.21.15.1:0/3920384268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed0bd40 msgr2=0x55b04ed0e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54816c5640 1 --2- 172.21.15.1:0/3920384268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed0bd40 0x55b04ed0e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54816c5640 1 -- 172.21.15.1:0/3920384268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04ec1a1f0 con 0x55b04ed00b60 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f5481ec6640 1 --2- 172.21.15.1:0/3920384268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 0x55b04ed0b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54826c7640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f54680a32c0 0x7f54680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f5472ffd640 1 -- 172.21.15.1:0/2000649749 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f54740cfe30 con 0x7f547c102ef0 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f54816c5640 1 --2- 172.21.15.1:0/3920384268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed00f40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f546c0079d0 tx=0x7f546c007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25badafc0612ead8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f5467fff640 1 -- 172.21.15.1:0/3920384268 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c00fc90 con 0x55b04ed00b60 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f5467fff640 1 -- 172.21.15.1:0/3920384268 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f546c010040 con 0x55b04ed00b60 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.381+0000 7f5467fff640 1 -- 172.21.15.1:0/3920384268 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c0136c0 con 0x55b04ed00b60 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f5486503740 1 -- 172.21.15.1:0/3920384268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 msgr2=0x55b04ed00f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f5486503740 1 --2- 172.21.15.1:0/3920384268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed00f40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f546c0079d0 tx=0x7f546c007e90 comp rx=0 tx=0).stop 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f54826c7640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f54680a32c0 0x7f54680a5780 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f547400caa0 tx=0x7f54740914d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f5486503740 1 -- 172.21.15.1:0/3920384268 shutdown_connections 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f5486503740 1 --2- 172.21.15.1:0/3920384268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed0bd40 0x55b04ed0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f5486503740 1 --2- 172.21.15.1:0/3920384268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 0x55b04ed0b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f5486503740 1 --2- 172.21.15.1:0/3920384268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed00f40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f5486503740 1 -- 172.21.15.1:0/3920384268 >> 172.21.15.1:0/3920384268 conn(0x55b04ecfd0f0 msgr2=0x55b04ecfd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f5486503740 1 -- 172.21.15.1:0/3920384268 shutdown_connections 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.382+0000 7f5486503740 1 -- 172.21.15.1:0/3920384268 wait complete. 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f5486503740 1 -- start start 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f5486503740 1 Processor -- start 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f5486503740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed08c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f5486503740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 0x55b04ed09140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f5486503740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed0bd40 0x55b04ed09680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f5486503740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b04ed0e910 con 0x55b04ed00b60 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f5486503740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b04ec20f10 con 0x55b04ed0bd40 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f5486503740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b04ed0ea90 con 0x55b04ed01480 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f5486503740 10 client.0 _ll_register_callbacks cb 0x55b04ebe9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f54816c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed08c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f54816c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed08c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54636/0 (socket says 172.21.15.1:54636) 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.383+0000 7f54816c5640 1 -- 172.21.15.1:0/1848541771 learned_addr learned my addr 172.21.15.1:0/1848541771 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f5481ec6640 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 0x55b04ed09140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54816c5640 1 -- 172.21.15.1:0/1848541771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 msgr2=0x55b04ed09140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54816c5640 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 0x55b04ed09140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54816c5640 1 -- 172.21.15.1:0/1848541771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed0bd40 msgr2=0x55b04ed09680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54816c5640 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed0bd40 0x55b04ed09680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54816c5640 1 -- 172.21.15.1:0/1848541771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04e9d76a0 con 0x55b04ed00b60 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54816c5640 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed08c00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f546c00f550 tx=0x7f546c0146b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd063a5e53af107e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54657fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54657fa640 1 -- 172.21.15.1:0/1848541771 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c023070 con 0x55b04ed00b60 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54657fa640 1 -- 172.21.15.1:0/1848541771 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f546c028660 con 0x55b04ed00b60 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f5486503740 1 -- 172.21.15.1:0/1848541771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f546c010040 con 0x55b04ed00b60 2024-09-16T00:33:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54657fa640 1 -- 172.21.15.1:0/1848541771 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f546c031050 con 0x55b04ed00b60 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f5486503740 10 client.16428 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f5486503740 1 -- 172.21.15.1:0/1848541771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b04ed1a4f0 con 0x55b04ed00b60 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54657fa640 1 -- 172.21.15.1:0/1848541771 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f546c010040 con 0x55b04ed00b60 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f54657fa640 1 -- 172.21.15.1:0/1848541771 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f546c01f800 con 0x55b04ed00b60 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f5486503740 10 client.16428 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f5486503740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f5486503740 20 client.16428 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.384+0000 7f5486503740 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b04ed1b950 0x55b04ed1dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.385+0000 7f5486503740 4 client.16428 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.385+0000 7f5486503740 1 -- 172.21.15.1:0/1848541771 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b04ec737f0 con 0x55b04ed1b950 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.385+0000 7f5481ec6640 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b04ed1b950 0x55b04ed1dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.385+0000 7f54657fa640 10 client.16428 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.385+0000 7f5481ec6640 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b04ed1b950 0x55b04ed1dd40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53da6927012eb8a server_cookie=898fddd8621e42bb in_seq=0 out_seq=0 2024-09-16T00:33:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f54657fa640 1 -- 172.21.15.1:0/1848541771 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b04ec737f0 con 0x55b04ed1b950 2024-09-16T00:33:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f54657fa640 10 client.16428 handle_command_reply: tid=0 2024-09-16T00:33:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 client.16428 shutdown 2024-09-16T00:33:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 -- 172.21.15.1:0/1848541771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 msgr2=0x55b04ed08c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed08c00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f546c00f550 tx=0x7f546c0146b0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 -- 172.21.15.1:0/1848541771 shutdown_connections 2024-09-16T00:33:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b04ed1b950 0x55b04ed1dd40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed0bd40 0x55b04ed09680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed01480 0x55b04ed09140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 --2- 172.21.15.1:0/1848541771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed00b60 0x55b04ed08c00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f54657fa640 0 client.16428 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 -- 172.21.15.1:0/1848541771 >> 172.21.15.1:0/1848541771 conn(0x55b04ecfd0f0 msgr2=0x55b04ed02dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 -- 172.21.15.1:0/1848541771 shutdown_connections 2024-09-16T00:33:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 1 -- 172.21.15.1:0/1848541771 wait complete. 2024-09-16T00:33:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.386+0000 7f5486503740 20 client.16428 trim_cache size 0 max 16384 2024-09-16T00:33:11.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5486503740 1 Processor -- start 2024-09-16T00:33:11.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5486503740 1 -- start start 2024-09-16T00:33:11.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5486503740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed00b60 0x55b04ed07ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5486503740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed01480 0x55b04ed233f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5486503740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed0bd40 0x55b04ed23930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5486503740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f547c1b3980 con 0x55b04ed01480 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5486503740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f547c1b1250 con 0x55b04ed00b60 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5486503740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f547c1af0a0 con 0x55b04ed0bd40 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5481ec6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed01480 0x55b04ed233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5481ec6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed01480 0x55b04ed233f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54642/0 (socket says 172.21.15.1:54642) 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5481ec6640 1 -- 172.21.15.1:0/3656510407 learned_addr learned my addr 172.21.15.1:0/3656510407 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f54816c5640 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed00b60 0x55b04ed07ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f54816c5640 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed0bd40 0x55b04ed23930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5481ec6640 1 -- 172.21.15.1:0/3656510407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed0bd40 msgr2=0x55b04ed23930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5481ec6640 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed0bd40 0x55b04ed23930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5481ec6640 1 -- 172.21.15.1:0/3656510407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed00b60 msgr2=0x55b04ed07ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5481ec6640 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed00b60 0x55b04ed07ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f5481ec6640 1 -- 172.21.15.1:0/3656510407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04ed0b1e0 con 0x55b04ed01480 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.404+0000 7f54816c5640 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed00b60 0x55b04ed07ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f5481ec6640 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed01480 0x55b04ed233f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f547802a9a0 tx=0x7f5478005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bf0bd3fa47eea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f54677fe640 1 -- 172.21.15.1:0/3656510407 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478035030 con 0x55b04ed01480 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f54677fe640 1 -- 172.21.15.1:0/3656510407 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5478005500 con 0x55b04ed01480 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f54677fe640 1 -- 172.21.15.1:0/3656510407 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478003850 con 0x55b04ed01480 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f5486503740 1 -- 172.21.15.1:0/3656510407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed01480 msgr2=0x55b04ed233f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f5486503740 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed01480 0x55b04ed233f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f547802a9a0 tx=0x7f5478005d50 comp rx=0 tx=0).stop 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f5486503740 1 -- 172.21.15.1:0/3656510407 shutdown_connections 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f5486503740 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed0bd40 0x55b04ed23930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f5486503740 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed01480 0x55b04ed233f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f5486503740 1 --2- 172.21.15.1:0/3656510407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed00b60 0x55b04ed07ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f5486503740 1 -- 172.21.15.1:0/3656510407 >> 172.21.15.1:0/3656510407 conn(0x55b04ecfd0f0 msgr2=0x55b04ed073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.405+0000 7f5486503740 1 -- 172.21.15.1:0/3656510407 shutdown_connections 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 1 -- 172.21.15.1:0/3656510407 wait complete. 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 1 -- start start 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 1 Processor -- start 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed00b60 0x55b04ed14390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed01480 0x55b04ed05890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed0bd40 0x55b04ed05dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f547c1af710 con 0x55b04ed0bd40 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f547c1b0b80 con 0x55b04ed01480 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f547c1afde0 con 0x55b04ed00b60 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f5486503740 10 client.0 _ll_register_callbacks cb 0x7f546c0078d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.406+0000 7f54816c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed0bd40 0x55b04ed05dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54816c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed0bd40 0x55b04ed05dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54654/0 (socket says 172.21.15.1:54654) 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54816c5640 1 -- 172.21.15.1:0/1839635959 learned_addr learned my addr 172.21.15.1:0/1839635959 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f5481ec6640 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed01480 0x55b04ed05890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54816c5640 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed00b60 0x55b04ed14390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54816c5640 1 -- 172.21.15.1:0/1839635959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed00b60 msgr2=0x55b04ed14390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54816c5640 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed00b60 0x55b04ed14390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54816c5640 1 -- 172.21.15.1:0/1839635959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed01480 msgr2=0x55b04ed05890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54816c5640 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed01480 0x55b04ed05890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54816c5640 1 -- 172.21.15.1:0/1839635959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04ed06600 con 0x55b04ed0bd40 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54816c5640 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed0bd40 0x55b04ed05dd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f546c004700 tx=0x7f546c00fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181a3c5c648b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54657fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54657fa640 1 -- 172.21.15.1:0/1839635959 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c03c070 con 0x55b04ed0bd40 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54657fa640 1 -- 172.21.15.1:0/1839635959 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f546c01f420 con 0x55b04ed0bd40 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f5486503740 1 -- 172.21.15.1:0/1839635959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5478005500 con 0x55b04ed0bd40 2024-09-16T00:33:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f54657fa640 1 -- 172.21.15.1:0/1839635959 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f546c045050 con 0x55b04ed0bd40 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f5486503740 10 client.16434 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.407+0000 7f5486503740 1 -- 172.21.15.1:0/1839635959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b04ed86350 con 0x55b04ed0bd40 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.408+0000 7f54657fa640 1 -- 172.21.15.1:0/1839635959 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5478005500 con 0x55b04ed0bd40 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.408+0000 7f54657fa640 1 -- 172.21.15.1:0/1839635959 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f546c03f020 con 0x55b04ed0bd40 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.408+0000 7f5486503740 10 client.16434 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.408+0000 7f5486503740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.408+0000 7f5486503740 20 client.16434 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.408+0000 7f5486503740 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b04ed86a40 0x55b04ed88e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.408+0000 7f5486503740 4 client.16434 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.408+0000 7f5486503740 1 -- 172.21.15.1:0/1839635959 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b04ec6dd00 con 0x55b04ed86a40 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.408+0000 7f5481ec6640 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b04ed86a40 0x55b04ed88e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.409+0000 7f5481ec6640 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b04ed86a40 0x55b04ed88e60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74834ffe5fe61ad3 server_cookie=84382b4ef6684501 in_seq=0 out_seq=0 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.409+0000 7f54657fa640 10 client.16434 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.409+0000 7f54657fa640 1 -- 172.21.15.1:0/1839635959 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b04ec6dd00 con 0x55b04ed86a40 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.409+0000 7f54657fa640 10 client.16434 handle_command_reply: tid=0 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.409+0000 7f5486503740 1 client.16434 shutdown 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 -- 172.21.15.1:0/1839635959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed0bd40 msgr2=0x55b04ed05dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed0bd40 0x55b04ed05dd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f546c004700 tx=0x7f546c00fde0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 -- 172.21.15.1:0/1839635959 shutdown_connections 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b04ed86a40 0x55b04ed88e60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b04ed0bd40 0x55b04ed05dd0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b04ed01480 0x55b04ed05890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 --2- 172.21.15.1:0/1839635959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04ed00b60 0x55b04ed14390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f54657fa640 0 client.16434 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 -- 172.21.15.1:0/1839635959 >> 172.21.15.1:0/1839635959 conn(0x55b04ecfd0f0 msgr2=0x55b04ed02dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 -- 172.21.15.1:0/1839635959 shutdown_connections 2024-09-16T00:33:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 1 -- 172.21.15.1:0/1839635959 wait complete. 2024-09-16T00:33:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.410+0000 7f5486503740 20 client.16434 trim_cache size 0 max 16384 2024-09-16T00:33:11.420 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:33:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:33:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "595af8b5-c0ba-426f-a6b0-ab736324a46f", 2024-09-16T00:33:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:33:11.420 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:33:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 -- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f54680a32c0 msgr2=0x7f54680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f54680a32c0 0x7f54680a5780 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f547400caa0 tx=0x7f54740914d0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 -- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c102ef0 msgr2=0x7f547c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c102ef0 0x7f547c19a5a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f547400cad0 tx=0x7f547400cf90 comp rx=0 tx=0).stop 2024-09-16T00:33:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 -- 172.21.15.1:0/2000649749 shutdown_connections 2024-09-16T00:33:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f547c102ef0 0x7f547c19a5a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f547c102570 0x7f547c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f54680a32c0 0x7f54680a5780 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 --2- 172.21.15.1:0/2000649749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f547c100e20 0x7f547c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:11.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 -- 172.21.15.1:0/2000649749 >> 172.21.15.1:0/2000649749 conn(0x7f547c0fc7d0 msgr2=0x7f547c06a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:11.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 -- 172.21.15.1:0/2000649749 shutdown_connections 2024-09-16T00:33:11.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:11.411+0000 7f548412a640 1 -- 172.21.15.1:0/2000649749 wait complete. 2024-09-16T00:33:11.457 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:12.314 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:33:12.316 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:12.602 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:12 smithi001 ceph-mon[30934]: pgmap v494: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 9 op/s 2024-09-16T00:33:12.602 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:12 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:33:12.602 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:12 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:12.603 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:12 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:33:12.603 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:12 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:12.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:12 smithi019 ceph-mon[35186]: pgmap v494: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 9 op/s 2024-09-16T00:33:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:12 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:33:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:12 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:12 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:33:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:12 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:12.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:12 smithi153 ceph-mon[35538]: pgmap v494: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 9 op/s 2024-09-16T00:33:12.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:12 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:33:12.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:12 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:12.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:12 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:33:12.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:12 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:12.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.825+0000 7fd394182640 1 -- 172.21.15.1:0/2733389866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 msgr2=0x7fd38c110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:12.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.825+0000 7fd394182640 1 --2- 172.21.15.1:0/2733389866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 0x7fd38c110810 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd384009ca0 tx=0x7fd38402a950 comp rx=0 tx=0).stop 2024-09-16T00:33:12.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.825+0000 7fd394182640 1 -- 172.21.15.1:0/2733389866 shutdown_connections 2024-09-16T00:33:12.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.825+0000 7fd394182640 1 --2- 172.21.15.1:0/2733389866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 0x7fd38c110810 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:12.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.825+0000 7fd394182640 1 --2- 172.21.15.1:0/2733389866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c074f40 0x7fd38c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:12.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.825+0000 7fd394182640 1 --2- 172.21.15.1:0/2733389866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38c074620 0x7fd38c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:12.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.825+0000 7fd394182640 1 -- 172.21.15.1:0/2733389866 >> 172.21.15.1:0/2733389866 conn(0x7fd38c0fd8c0 msgr2=0x7fd38c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.825+0000 7fd394182640 1 -- 172.21.15.1:0/2733389866 shutdown_connections 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.825+0000 7fd394182640 1 -- 172.21.15.1:0/2733389866 wait complete. 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd394182640 1 Processor -- start 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd394182640 1 -- start start 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd394182640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38c074620 0x7fd38c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd394182640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c074f40 0x7fd38c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd394182640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 0x7fd38c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd394182640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c114140 con 0x7fd38c10e3c0 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd394182640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c113fc0 con 0x7fd38c074f40 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd394182640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1142c0 con 0x7fd38c074620 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd39271f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 0x7fd38c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd391f1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38c074620 0x7fd38c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd39271f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 0x7fd38c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54672/0 (socket says 172.21.15.1:54672) 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd39271f640 1 -- 172.21.15.1:0/250724593 learned_addr learned my addr 172.21.15.1:0/250724593 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.826+0000 7fd39171d640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c074f40 0x7fd38c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd39271f640 1 -- 172.21.15.1:0/250724593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38c074620 msgr2=0x7fd38c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd39271f640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38c074620 0x7fd38c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd39271f640 1 -- 172.21.15.1:0/250724593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c074f40 msgr2=0x7fd38c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd39271f640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c074f40 0x7fd38c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd39271f640 1 -- 172.21.15.1:0/250724593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c19f5c0 con 0x7fd38c10e3c0 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd39171d640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c074f40 0x7fd38c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd39271f640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 0x7fd38c19eee0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd38402ae70 tx=0x7fd38402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd382ffd640 1 -- 172.21.15.1:0/250724593 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384034040 con 0x7fd38c10e3c0 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd382ffd640 1 -- 172.21.15.1:0/250724593 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd384030920 con 0x7fd38c10e3c0 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c19f850 con 0x7fd38c10e3c0 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd382ffd640 1 -- 172.21.15.1:0/250724593 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384045480 con 0x7fd38c10e3c0 2024-09-16T00:33:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.827+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c1a7190 con 0x7fd38c10e3c0 2024-09-16T00:33:12.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.828+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c075cd0 con 0x7fd38c10e3c0 2024-09-16T00:33:12.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.829+0000 7fd382ffd640 1 -- 172.21.15.1:0/250724593 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd384037020 con 0x7fd38c10e3c0 2024-09-16T00:33:12.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.830+0000 7fd382ffd640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3780a3240 0x7fd3780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:12.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.830+0000 7fd391f1e640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3780a3240 0x7fd3780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:12.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.830+0000 7fd382ffd640 1 -- 172.21.15.1:0/250724593 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd3840f0cd0 con 0x7fd38c10e3c0 2024-09-16T00:33:12.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.830+0000 7fd391f1e640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3780a3240 0x7fd3780a5700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd38800bbe0 tx=0x7fd38800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:12.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:12.834+0000 7fd382ffd640 1 -- 172.21.15.1:0/250724593 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3840bd590 con 0x7fd38c10e3c0 2024-09-16T00:33:12.902 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:13.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.054+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd38c19b970 con 0x7fd38c10e3c0 2024-09-16T00:33:13.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.055+0000 7fd382ffd640 1 -- 172.21.15.1:0/250724593 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fd3840bce90 con 0x7fd38c10e3c0 2024-09-16T00:33:13.056 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:33:13.056 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:33:13.056 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:33:13.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3780a3240 msgr2=0x7fd3780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3780a3240 0x7fd3780a5700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd38800bbe0 tx=0x7fd38800b390 comp rx=0 tx=0).stop 2024-09-16T00:33:13.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 msgr2=0x7fd38c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 0x7fd38c19eee0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd38402ae70 tx=0x7fd38402f5b0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 shutdown_connections 2024-09-16T00:33:13.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd38c10e3c0 0x7fd38c19eee0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c074f40 0x7fd38c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd3780a3240 0x7fd3780a5700 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 --2- 172.21.15.1:0/250724593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38c074620 0x7fd38c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 >> 172.21.15.1:0/250724593 conn(0x7fd38c0fd8c0 msgr2=0x7fd38c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:13.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 shutdown_connections 2024-09-16T00:33:13.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.057+0000 7fd394182640 1 -- 172.21.15.1:0/250724593 wait complete. 2024-09-16T00:33:13.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:13 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:33:13.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:13 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/250724593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:13.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 -- 172.21.15.1:0/1413982914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 msgr2=0x7f4eac10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 --2- 172.21.15.1:0/1413982914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 0x7f4eac10bb50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009900 tx=0x7f4ea002a480 comp rx=0 tx=0).stop 2024-09-16T00:33:13.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 -- 172.21.15.1:0/1413982914 shutdown_connections 2024-09-16T00:33:13.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 --2- 172.21.15.1:0/1413982914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eac10c170 0x7f4eac10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 --2- 172.21.15.1:0/1413982914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 0x7f4eac10bb50 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 --2- 172.21.15.1:0/1413982914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac102d40 0x7f4eac103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 -- 172.21.15.1:0/1413982914 >> 172.21.15.1:0/1413982914 conn(0x7f4eac0fc7a0 msgr2=0x7f4eac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 -- 172.21.15.1:0/1413982914 shutdown_connections 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 -- 172.21.15.1:0/1413982914 wait complete. 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 Processor -- start 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.560+0000 7f4eb2413640 1 -- start start 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eb2413640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eac102d40 0x7f4eac198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eb2413640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 0x7f4eac1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eb2413640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac10c170 0x7f4eac19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eb2413640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac111f70 con 0x7f4eac103660 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eb2413640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac111df0 con 0x7f4eac10c170 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eb2413640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1120f0 con 0x7f4eac102d40 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 0x7f4eac1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eac102d40 0x7f4eac198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eb09b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac10c170 0x7f4eac19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 0x7f4eac1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54688/0 (socket says 172.21.15.1:54688) 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eb09b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac10c170 0x7f4eac19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:34058/0 (socket says 172.21.15.1:34058) 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eab7fe640 1 -- 172.21.15.1:0/2194738402 learned_addr learned my addr 172.21.15.1:0/2194738402 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eab7fe640 1 -- 172.21.15.1:0/2194738402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eac102d40 msgr2=0x7f4eac198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eab7fe640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eac102d40 0x7f4eac198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eab7fe640 1 -- 172.21.15.1:0/2194738402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac10c170 msgr2=0x7f4eac19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eab7fe640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac10c170 0x7f4eac19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.561+0000 7f4eab7fe640 1 -- 172.21.15.1:0/2194738402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac19d420 con 0x7f4eac103660 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.562+0000 7f4eab7fe640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 0x7f4eac1989b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0006010 tx=0x7f4ea00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.562+0000 7f4ea97fa640 1 -- 172.21.15.1:0/2194738402 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea003f030 con 0x7f4eac103660 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.562+0000 7f4eb2413640 1 -- 172.21.15.1:0/2194738402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac19d6b0 con 0x7f4eac103660 2024-09-16T00:33:13.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.562+0000 7f4ea97fa640 1 -- 172.21.15.1:0/2194738402 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea0003ce0 con 0x7f4eac103660 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.562+0000 7f4ea97fa640 1 -- 172.21.15.1:0/2194738402 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0035070 con 0x7f4eac103660 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.562+0000 7f4eb2413640 1 -- 172.21.15.1:0/2194738402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac1a4ff0 con 0x7f4eac103660 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4ea97fa640 1 -- 172.21.15.1:0/2194738402 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4ea0042030 con 0x7f4eac103660 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb47ec740 1 Processor -- start 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb47ec740 1 -- start start 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d697c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d697cdc0 0x5635d6987140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d6987680 0x5635d6989a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5635d68e22e0 con 0x5635d697c4a0 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5635d6889ce0 con 0x5635d697cdc0 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5635d68de2a0 con 0x5635d6987680 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb09b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d697c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb09b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d697c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54690/0 (socket says 172.21.15.1:54690) 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.565+0000 7f4eb09b0640 1 -- 172.21.15.1:0/2727640182 learned_addr learned my addr 172.21.15.1:0/2727640182 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eab7fe640 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d6987680 0x5635d6989a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eabfff640 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d697cdc0 0x5635d6987140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eb09b0640 1 -- 172.21.15.1:0/2727640182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d6987680 msgr2=0x5635d6989a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d6987680 0x5635d6989a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4ea97fa640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e940a3240 0x7f4e940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eb09b0640 1 -- 172.21.15.1:0/2727640182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d697cdc0 msgr2=0x5635d6987140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d697cdc0 0x5635d6987140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eb09b0640 1 -- 172.21.15.1:0/2727640182 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635d686e440 con 0x5635d697c4a0 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eabfff640 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d697cdc0 0x5635d6987140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4ea97fa640 1 -- 172.21.15.1:0/2194738402 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4ea00efd80 con 0x7f4eac103660 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d697c880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009f70 tx=0x7f4ea4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462c45ddf08e1ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:13.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e940a3240 0x7f4e940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4e927fc640 1 -- 172.21.15.1:0/2727640182 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4016070 con 0x5635d697c4a0 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4e927fc640 1 -- 172.21.15.1:0/2727640182 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea401bc40 con 0x5635d697c4a0 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.566+0000 7f4e927fc640 1 -- 172.21.15.1:0/2727640182 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40113e0 con 0x5635d697c4a0 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 -- 172.21.15.1:0/2727640182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 msgr2=0x5635d697c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e940a3240 0x7f4e940a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4ea401bfd0 tx=0x7f4ea40116c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d697c880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009f70 tx=0x7f4ea4010800 comp rx=0 tx=0).stop 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 -- 172.21.15.1:0/2727640182 shutdown_connections 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d6987680 0x5635d6989a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d697cdc0 0x5635d6987140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/2727640182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d697c880 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 -- 172.21.15.1:0/2727640182 >> 172.21.15.1:0/2727640182 conn(0x5635d6978a30 msgr2=0x5635d6978e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 -- 172.21.15.1:0/2727640182 shutdown_connections 2024-09-16T00:33:13.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 -- 172.21.15.1:0/2727640182 wait complete. 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.567+0000 7f4eb47ec740 1 -- start start 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb47ec740 1 Processor -- start 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d6986fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697cdc0 0x5635d6984260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6987680 0x5635d69847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5635d698a250 con 0x5635d697c4a0 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5635d689d050 con 0x5635d6987680 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5635d698a3d0 con 0x5635d697cdc0 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb47ec740 10 client.0 _ll_register_callbacks cb 0x5635d68dd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d6986fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d6986fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54694/0 (socket says 172.21.15.1:54694) 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eabfff640 1 -- 172.21.15.1:0/516621286 learned_addr learned my addr 172.21.15.1:0/516621286 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eab7fe640 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697cdc0 0x5635d6984260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6987680 0x5635d69847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eabfff640 1 -- 172.21.15.1:0/516621286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697cdc0 msgr2=0x5635d6984260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eabfff640 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697cdc0 0x5635d6984260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eabfff640 1 -- 172.21.15.1:0/516621286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6987680 msgr2=0x5635d69847a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eabfff640 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6987680 0x5635d69847a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eabfff640 1 -- 172.21.15.1:0/516621286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635d66533c0 con 0x5635d697c4a0 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6987680 0x5635d69847a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4eabfff640 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d6986fb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00b980 tx=0x7f4e9c00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6b72bc10c80f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.568+0000 7f4e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/516621286 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c005220 con 0x5635d697c4a0 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/516621286 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4e9c0053c0 con 0x5635d697c4a0 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4eb47ec740 1 -- 172.21.15.1:0/516621286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea401bc40 con 0x5635d697c4a0 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/516621286 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e9c003e40 con 0x5635d697c4a0 2024-09-16T00:33:13.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4eb47ec740 10 client.16452 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4eb47ec740 1 -- 172.21.15.1:0/516621286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5635d6995bd0 con 0x5635d697c4a0 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/516621286 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea401bc40 con 0x5635d697c4a0 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/516621286 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4e9c012840 con 0x5635d697c4a0 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4eb47ec740 10 client.16452 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4eb47ec740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4eb47ec740 20 client.16452 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5635d6997090 0x5635d6999500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4eb47ec740 4 client.16452 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.569+0000 7f4eb47ec740 1 -- 172.21.15.1:0/516621286 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635d68ef850 con 0x5635d6997090 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.570+0000 7f4eab7fe640 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5635d6997090 0x5635d6999500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.570+0000 7f4eab7fe640 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5635d6997090 0x5635d6999500 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588ff633449c2f2 server_cookie=9c998241d209d941 in_seq=0 out_seq=0 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.570+0000 7f4e8b7fe640 10 client.16452 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:13.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/516621286 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635d68ef850 con 0x5635d6997090 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4e8b7fe640 10 client.16452 handle_command_reply: tid=0 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 client.16452 shutdown 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 -- 172.21.15.1:0/516621286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 msgr2=0x5635d6986fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d6986fb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00b980 tx=0x7f4e9c00be40 comp rx=0 tx=0).stop 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 -- 172.21.15.1:0/516621286 shutdown_connections 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5635d6997090 0x5635d6999500 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6987680 0x5635d69847a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697cdc0 0x5635d6984260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4e8b7fe640 0 client.16452 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/516621286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697c4a0 0x5635d6986fb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 -- 172.21.15.1:0/516621286 >> 172.21.15.1:0/516621286 conn(0x5635d6978a30 msgr2=0x5635d697e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 -- 172.21.15.1:0/516621286 shutdown_connections 2024-09-16T00:33:13.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 1 -- 172.21.15.1:0/516621286 wait complete. 2024-09-16T00:33:13.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.571+0000 7f4eb47ec740 20 client.16452 trim_cache size 0 max 16384 2024-09-16T00:33:13.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.588+0000 7f4eb47ec740 1 Processor -- start 2024-09-16T00:33:13.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.588+0000 7f4eb47ec740 1 -- start start 2024-09-16T00:33:13.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 0x5635d69934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697c4a0 0x5635d697f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d699edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1b1730 con 0x5635d697cdc0 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1af000 con 0x5635d6978a30 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1ace50 con 0x5635d697c4a0 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb09b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d699edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb09b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d699edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54710/0 (socket says 172.21.15.1:54710) 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb09b0640 1 -- 172.21.15.1:0/3088023986 learned_addr learned my addr 172.21.15.1:0/3088023986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eabfff640 1 --2- 172.21.15.1:0/3088023986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 0x5635d69934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb09b0640 1 -- 172.21.15.1:0/3088023986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697c4a0 msgr2=0x5635d697f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/3088023986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697c4a0 0x5635d697f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb09b0640 1 -- 172.21.15.1:0/3088023986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 msgr2=0x5635d69934b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/3088023986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 0x5635d69934b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb09b0640 1 -- 172.21.15.1:0/3088023986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635d698a2a0 con 0x5635d697cdc0 2024-09-16T00:33:13.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/3088023986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d699edd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009830 tx=0x7f4ea4027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b4c849e6c18773 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.589+0000 7f4e91ffb640 1 -- 172.21.15.1:0/3088023986 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea402d040 con 0x5635d697cdc0 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4e91ffb640 1 -- 172.21.15.1:0/3088023986 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea4003e50 con 0x5635d697cdc0 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4e91ffb640 1 -- 172.21.15.1:0/3088023986 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4016040 con 0x5635d697cdc0 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3088023986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 msgr2=0x5635d699edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3088023986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d699edd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009830 tx=0x7f4ea4027000 comp rx=0 tx=0).stop 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3088023986 shutdown_connections 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3088023986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d699edd0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3088023986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697c4a0 0x5635d697f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3088023986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 0x5635d69934b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3088023986 >> 172.21.15.1:0/3088023986 conn(0x5635d6984900 msgr2=0x5635d697f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3088023986 shutdown_connections 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.590+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3088023986 wait complete. 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb47ec740 1 -- start start 2024-09-16T00:33:13.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb47ec740 1 Processor -- start 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 0x5635d698b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697c4a0 0x5635d698b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb47ec740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d698be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1ad4c0 con 0x5635d697cdc0 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1ae930 con 0x5635d6978a30 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb47ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1adb90 con 0x5635d697c4a0 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb47ec740 10 client.0 _ll_register_callbacks cb 0x7f4e9c0040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb09b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d698be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb09b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d698be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54714/0 (socket says 172.21.15.1:54714) 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.591+0000 7f4eb09b0640 1 -- 172.21.15.1:0/3384138387 learned_addr learned my addr 172.21.15.1:0/3384138387 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eabfff640 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 0x5635d698b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eab7fe640 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697c4a0 0x5635d698b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eb09b0640 1 -- 172.21.15.1:0/3384138387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697c4a0 msgr2=0x5635d698b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697c4a0 0x5635d698b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eb09b0640 1 -- 172.21.15.1:0/3384138387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 msgr2=0x5635d698b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 0x5635d698b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eb09b0640 1 -- 172.21.15.1:0/3384138387 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635d6a00dd0 con 0x5635d697cdc0 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eabfff640 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 0x5635d698b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eb09b0640 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d698be70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4011010 tx=0x7f4ea4010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea06a3578b84bffb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/3384138387 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea403f060 con 0x5635d697cdc0 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/3384138387 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4ea4003b80 con 0x5635d697cdc0 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3384138387 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea4003e50 con 0x5635d697cdc0 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/3384138387 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ea4016040 con 0x5635d697cdc0 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eb47ec740 10 client.16458 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:13.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.592+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3384138387 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5635d6a020a0 con 0x5635d697cdc0 2024-09-16T00:33:13.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.593+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/3384138387 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea4003e50 con 0x5635d697cdc0 2024-09-16T00:33:13.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.593+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/3384138387 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4ea4030070 con 0x5635d697cdc0 2024-09-16T00:33:13.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.593+0000 7f4eb47ec740 10 client.16458 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:13.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.593+0000 7f4eb47ec740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:13.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.593+0000 7f4eb47ec740 20 client.16458 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:13.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.593+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5635d6987680 0x5635d6a045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:13.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.593+0000 7f4eb47ec740 4 client.16458 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:33:13.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.593+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3384138387 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635d68e9d60 con 0x5635d6987680 2024-09-16T00:33:13.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.593+0000 7f4eabfff640 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5635d6987680 0x5635d6a045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eabfff640 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5635d6987680 0x5635d6a045d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e29daeec358a64 server_cookie=7a8778debe0828e in_seq=0 out_seq=0 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4e8b7fe640 10 client.16458 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4e8b7fe640 1 -- 172.21.15.1:0/3384138387 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5635d68e9d60 con 0x5635d6987680 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4e8b7fe640 10 client.16458 handle_command_reply: tid=0 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 client.16458 shutdown 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3384138387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 msgr2=0x5635d698be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d698be70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4011010 tx=0x7f4ea4010d90 comp rx=0 tx=0).stop 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3384138387 shutdown_connections 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5635d6987680 0x5635d6a045d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635d697cdc0 0x5635d698be70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635d697c4a0 0x5635d698b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4e8b7fe640 0 client.16458 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:13.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 --2- 172.21.15.1:0/3384138387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635d6978a30 0x5635d698b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3384138387 >> 172.21.15.1:0/3384138387 conn(0x5635d6984900 msgr2=0x5635d6977b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:13.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3384138387 shutdown_connections 2024-09-16T00:33:13.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 1 -- 172.21.15.1:0/3384138387 wait complete. 2024-09-16T00:33:13.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.594+0000 7f4eb47ec740 20 client.16458 trim_cache size 0 max 16384 2024-09-16T00:33:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:33:13.605 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:33:13.606 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:33:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:33:13.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:13 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:33:13.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:13 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/250724593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:13.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 -- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e940a3240 msgr2=0x7f4e940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e940a3240 0x7f4e940a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4ea401bfd0 tx=0x7f4ea40116c0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 -- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 msgr2=0x7f4eac1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:13.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 0x7f4eac1989b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0006010 tx=0x7f4ea00042f0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 -- 172.21.15.1:0/2194738402 shutdown_connections 2024-09-16T00:33:13.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac10c170 0x7f4eac19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eac103660 0x7f4eac1989b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e940a3240 0x7f4e940a5700 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 --2- 172.21.15.1:0/2194738402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eac102d40 0x7f4eac198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:13.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 -- 172.21.15.1:0/2194738402 >> 172.21.15.1:0/2194738402 conn(0x7f4eac0fc7a0 msgr2=0x7f4eac0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:13.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 -- 172.21.15.1:0/2194738402 shutdown_connections 2024-09-16T00:33:13.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:13.595+0000 7f4eb2413640 1 -- 172.21.15.1:0/2194738402 wait complete. 2024-09-16T00:33:13.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:13 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:33:13.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:13 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/250724593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:14.151 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id ec419676 --release --await' 2024-09-16T00:33:14.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id ec419676 --release --await 2024-09-16T00:33:14.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:14 smithi019 ceph-mon[35186]: pgmap v495: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-16T00:33:14.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:14 smithi001 ceph-mon[30934]: pgmap v495: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-16T00:33:14.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:14 smithi153 ceph-mon[35538]: pgmap v495: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-16T00:33:14.930 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:33:14.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:15.047 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:15.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.545+0000 7f2b7e926640 1 -- 172.21.15.1:0/2133598984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b781055b0 msgr2=0x7f2b7810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:15.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.545+0000 7f2b7e926640 1 --2- 172.21.15.1:0/2133598984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b781055b0 0x7f2b7810d130 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009ca0 tx=0x7f2b7002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.546+0000 7f2b7e926640 1 -- 172.21.15.1:0/2133598984 shutdown_connections 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.546+0000 7f2b7e926640 1 --2- 172.21.15.1:0/2133598984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b781055b0 0x7f2b7810d130 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.546+0000 7f2b7e926640 1 --2- 172.21.15.1:0/2133598984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b78100e30 0x7f2b78104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.546+0000 7f2b7e926640 1 --2- 172.21.15.1:0/2133598984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b78100480 0x7f2b78100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.546+0000 7f2b7e926640 1 -- 172.21.15.1:0/2133598984 >> 172.21.15.1:0/2133598984 conn(0x7f2b78077210 msgr2=0x7f2b780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.546+0000 7f2b7e926640 1 -- 172.21.15.1:0/2133598984 shutdown_connections 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.546+0000 7f2b7e926640 1 -- 172.21.15.1:0/2133598984 wait complete. 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.546+0000 7f2b7e926640 1 Processor -- start 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.546+0000 7f2b7e926640 1 -- start start 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b7e926640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b78100480 0x7f2b780fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b7e926640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b78100e30 0x7f2b780fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b7e926640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b781055b0 0x7f2b780fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b7e926640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7810fd90 con 0x7f2b78100e30 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b7e926640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7810fc10 con 0x7f2b78100480 2024-09-16T00:33:15.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b7e926640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7810ff10 con 0x7f2b781055b0 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b78100e30 0x7f2b780fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b78100e30 0x7f2b780fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54720/0 (socket says 172.21.15.1:54720) 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b777fe640 1 -- 172.21.15.1:0/3068017550 learned_addr learned my addr 172.21.15.1:0/3068017550 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b781055b0 0x7f2b780fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b77fff640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b78100480 0x7f2b780fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b777fe640 1 -- 172.21.15.1:0/3068017550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b781055b0 msgr2=0x7f2b780fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b777fe640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b781055b0 0x7f2b780fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b777fe640 1 -- 172.21.15.1:0/3068017550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b78100480 msgr2=0x7f2b780fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b777fe640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b78100480 0x7f2b780fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b777fe640 1 -- 172.21.15.1:0/3068017550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b781a6940 con 0x7f2b78100e30 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b777fe640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b78100e30 0x7f2b780fcbd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00d770 tx=0x7f2b6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.547+0000 7f2b757fa640 1 -- 172.21.15.1:0/3068017550 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c017070 con 0x7f2b78100e30 2024-09-16T00:33:15.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.548+0000 7f2b757fa640 1 -- 172.21.15.1:0/3068017550 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c005500 con 0x7f2b78100e30 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.548+0000 7f2b7e926640 1 -- 172.21.15.1:0/3068017550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b781a6c30 con 0x7f2b78100e30 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.548+0000 7f2b757fa640 1 -- 172.21.15.1:0/3068017550 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c004e00 con 0x7f2b78100e30 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.548+0000 7f2b7e926640 1 -- 172.21.15.1:0/3068017550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b781a7120 con 0x7f2b78100e30 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.550+0000 7f2b757fa640 1 -- 172.21.15.1:0/3068017550 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2b6c004fa0 con 0x7f2b78100e30 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b80cff740 1 Processor -- start 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b80cff740 1 -- start start 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd078d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b80cff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5616dcfd3620 con 0x5616dd06e060 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b80cff740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5616dcf7ae50 con 0x5616dd079240 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b80cff740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5616dcfcf5e0 con 0x5616dd06e980 2024-09-16T00:33:15.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b7cec3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b7cec3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54732/0 (socket says 172.21.15.1:54732) 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b7cec3640 1 -- 172.21.15.1:0/2065700498 learned_addr learned my addr 172.21.15.1:0/2065700498 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b77fff640 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd078d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b777fe640 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b7cec3640 1 -- 172.21.15.1:0/2065700498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 msgr2=0x5616dd078d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd078d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b7cec3640 1 -- 172.21.15.1:0/2065700498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 msgr2=0x5616dd07b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b757fa640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2b600a3240 0x7f2b600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b7cec3640 1 -- 172.21.15.1:0/2065700498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616dcf6bd40 con 0x5616dd06e060 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.551+0000 7f2b777fe640 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.552+0000 7f2b757fa640 1 -- 172.21.15.1:0/3068017550 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2b6c0cf790 con 0x7f2b78100e30 2024-09-16T00:33:15.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.552+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd06e440 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2b70005cc0 tx=0x7f2b70005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd23d5f782dd55d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.552+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2b600a3240 0x7f2b600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.552+0000 7f2b5e7fc640 1 -- 172.21.15.1:0/2065700498 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70049040 con 0x5616dd06e060 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.552+0000 7f2b5e7fc640 1 -- 172.21.15.1:0/2065700498 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b70036050 con 0x5616dd06e060 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.552+0000 7f2b5e7fc640 1 -- 172.21.15.1:0/2065700498 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70004640 con 0x5616dd06e060 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 -- 172.21.15.1:0/2065700498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 msgr2=0x5616dd06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd06e440 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2b70005cc0 tx=0x7f2b70005ef0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2b600a3240 0x7f2b600a5700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2b70044000 tx=0x7f2b70004990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 -- 172.21.15.1:0/2065700498 shutdown_connections 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd078d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 --2- 172.21.15.1:0/2065700498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd06e440 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 -- 172.21.15.1:0/2065700498 >> 172.21.15.1:0/2065700498 conn(0x5616dd06a5f0 msgr2=0x5616dd06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 -- 172.21.15.1:0/2065700498 shutdown_connections 2024-09-16T00:33:15.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 -- 172.21.15.1:0/2065700498 wait complete. 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 -- start start 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.553+0000 7f2b80cff740 1 Processor -- start 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd076040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd076580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b80cff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5616dd07bc90 con 0x5616dd06e060 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b80cff740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5616dcf5bf10 con 0x5616dd079240 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b80cff740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5616dd07be10 con 0x5616dd06e980 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b80cff740 10 client.0 _ll_register_callbacks cb 0x5616dcf7fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd076040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54746/0 (socket says 172.21.15.1:54746) 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b77fff640 1 -- 172.21.15.1:0/3822440534 learned_addr learned my addr 172.21.15.1:0/3822440534 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:15.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b777fe640 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd076580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b77fff640 1 -- 172.21.15.1:0/3822440534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 msgr2=0x5616dd076580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b77fff640 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd076580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b77fff640 1 -- 172.21.15.1:0/3822440534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 msgr2=0x5616dd076ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b77fff640 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd076ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b77fff640 1 -- 172.21.15.1:0/3822440534 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616dcce2580 con 0x5616dd06e060 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b77fff640 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd076040 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800e750 tx=0x7f2b6800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2985380fac32d0de server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b577fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b577fe640 1 -- 172.21.15.1:0/3822440534 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6801e060 con 0x5616dd06e060 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b577fe640 1 -- 172.21.15.1:0/3822440534 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2b68004460 con 0x5616dd06e060 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.554+0000 7f2b80cff740 1 -- 172.21.15.1:0/3822440534 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b70036050 con 0x5616dd06e060 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b577fe640 1 -- 172.21.15.1:0/3822440534 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b68025050 con 0x5616dd06e060 2024-09-16T00:33:15.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b80cff740 10 client.16470 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b80cff740 1 -- 172.21.15.1:0/3822440534 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5616dcdac960 con 0x5616dd06e060 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b577fe640 1 -- 172.21.15.1:0/3822440534 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b70036050 con 0x5616dd06e060 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b577fe640 1 -- 172.21.15.1:0/3822440534 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2b6800b040 con 0x5616dd06e060 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b80cff740 10 client.16470 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b80cff740 10 client.16470 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b80cff740 20 client.16470 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b80cff740 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5616dd088ba0 0x5616dd08af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b80cff740 4 client.16470 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b80cff740 1 -- 172.21.15.1:0/3822440534 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616dcfe0b90 con 0x5616dd088ba0 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.555+0000 7f2b777fe640 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5616dd088ba0 0x5616dd08af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.556+0000 7f2b777fe640 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5616dd088ba0 0x5616dd08af90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcdebacb11faf040 server_cookie=559aa5f461a17495 in_seq=0 out_seq=0 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.556+0000 7f2b577fe640 10 client.16470 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:15.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.556+0000 7f2b577fe640 1 -- 172.21.15.1:0/3822440534 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616dcfe0b90 con 0x5616dd088ba0 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.556+0000 7f2b577fe640 10 client.16470 handle_command_reply: tid=0 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.556+0000 7f2b80cff740 1 client.16470 shutdown 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 -- 172.21.15.1:0/3822440534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 msgr2=0x5616dd076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd076040 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800e750 tx=0x7f2b6800ec10 comp rx=0 tx=0).stop 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 -- 172.21.15.1:0/3822440534 shutdown_connections 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5616dd088ba0 0x5616dd08af90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd076ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b577fe640 0 client.16470 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 --2- 172.21.15.1:0/3822440534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd076040 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 -- 172.21.15.1:0/3822440534 >> 172.21.15.1:0/3822440534 conn(0x5616dd06a5f0 msgr2=0x5616dd0702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 -- 172.21.15.1:0/3822440534 shutdown_connections 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 1 -- 172.21.15.1:0/3822440534 wait complete. 2024-09-16T00:33:15.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.557+0000 7f2b80cff740 20 client.16470 trim_cache size 0 max 16384 2024-09-16T00:33:15.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.574+0000 7f2b80cff740 1 Processor -- start 2024-09-16T00:33:15.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.574+0000 7f2b80cff740 1 -- start start 2024-09-16T00:33:15.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e060 0x5616dd074910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e980 0x5616dd069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd090840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b80cff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781b3880 con 0x5616dd06e980 2024-09-16T00:33:15.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b80cff740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781b1150 con 0x5616dd079240 2024-09-16T00:33:15.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b80cff740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781aefa0 con 0x5616dd06e060 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e980 0x5616dd069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e980 0x5616dd069640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54754/0 (socket says 172.21.15.1:54754) 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b777fe640 1 -- 172.21.15.1:0/519698716 learned_addr learned my addr 172.21.15.1:0/519698716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b77fff640 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e060 0x5616dd074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd090840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b777fe640 1 -- 172.21.15.1:0/519698716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e060 msgr2=0x5616dd074910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b777fe640 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e060 0x5616dd074910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b777fe640 1 -- 172.21.15.1:0/519698716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 msgr2=0x5616dd090840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b777fe640 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd090840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b777fe640 1 -- 172.21.15.1:0/519698716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616dd06a420 con 0x5616dd06e980 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b77fff640 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e060 0x5616dd074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd090840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:15.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b777fe640 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e980 0x5616dd069640 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00d740 tx=0x7f2b6c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be083caf259651d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.575+0000 7f2b5dffb640 1 -- 172.21.15.1:0/519698716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c020040 con 0x5616dd06e980 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b5dffb640 1 -- 172.21.15.1:0/519698716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c09fd60 con 0x5616dd06e980 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b5dffb640 1 -- 172.21.15.1:0/519698716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c0860b0 con 0x5616dd06e980 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b80cff740 1 -- 172.21.15.1:0/519698716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e980 msgr2=0x5616dd069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b80cff740 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e980 0x5616dd069640 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00d740 tx=0x7f2b6c086040 comp rx=0 tx=0).stop 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b80cff740 1 -- 172.21.15.1:0/519698716 shutdown_connections 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b80cff740 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd090840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b80cff740 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e980 0x5616dd069640 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b80cff740 1 --2- 172.21.15.1:0/519698716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e060 0x5616dd074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b80cff740 1 -- 172.21.15.1:0/519698716 >> 172.21.15.1:0/519698716 conn(0x5616dd06a5f0 msgr2=0x5616dd068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b80cff740 1 -- 172.21.15.1:0/519698716 shutdown_connections 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.576+0000 7f2b80cff740 1 -- 172.21.15.1:0/519698716 wait complete. 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b80cff740 1 -- start start 2024-09-16T00:33:15.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b80cff740 1 Processor -- start 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd081890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd07c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b80cff740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b80cff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781af610 con 0x5616dd06e060 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b80cff740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781b0a80 con 0x5616dd079240 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b80cff740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781afce0 con 0x5616dd06e980 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b80cff740 10 client.0 _ll_register_callbacks cb 0x7f2b6800aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd081890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd081890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54760/0 (socket says 172.21.15.1:54760) 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.577+0000 7f2b77fff640 1 -- 172.21.15.1:0/310176177 learned_addr learned my addr 172.21.15.1:0/310176177 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:15.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b777fe640 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd07c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b77fff640 1 -- 172.21.15.1:0/310176177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 msgr2=0x5616dd07c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b77fff640 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd07c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b77fff640 1 -- 172.21.15.1:0/310176177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 msgr2=0x5616dd07c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b77fff640 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b77fff640 1 -- 172.21.15.1:0/310176177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616dd087740 con 0x5616dd06e060 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b7cec3640 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b77fff640 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd081890 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2b68003d00 tx=0x7f2b68005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c375db05d24b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b577fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b577fe640 1 -- 172.21.15.1:0/310176177 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6800ee60 con 0x5616dd06e060 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b80cff740 1 -- 172.21.15.1:0/310176177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b6c09fd60 con 0x5616dd06e060 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b577fe640 1 -- 172.21.15.1:0/310176177 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2b68005510 con 0x5616dd06e060 2024-09-16T00:33:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b577fe640 1 -- 172.21.15.1:0/310176177 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b68037050 con 0x5616dd06e060 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b80cff740 10 client.16476 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.578+0000 7f2b80cff740 1 -- 172.21.15.1:0/310176177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5616dd0f4690 con 0x5616dd06e060 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.579+0000 7f2b577fe640 1 -- 172.21.15.1:0/310176177 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c09fd60 con 0x5616dd06e060 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.579+0000 7f2b577fe640 1 -- 172.21.15.1:0/310176177 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2b68019460 con 0x5616dd06e060 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.579+0000 7f2b80cff740 10 client.16476 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.579+0000 7f2b80cff740 10 client.16476 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.579+0000 7f2b80cff740 20 client.16476 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.579+0000 7f2b80cff740 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5616dd0f4da0 0x5616dd0f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.579+0000 7f2b80cff740 4 client.16476 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ec419676", "release": true, "await": true}] 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.579+0000 7f2b80cff740 1 -- 172.21.15.1:0/310176177 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ec419676", "release": true, "await": true}) -- 0x5616dcfdb0a0 con 0x5616dd0f4da0 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.579+0000 7f2b777fe640 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5616dd0f4da0 0x5616dd0f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.580+0000 7f2b777fe640 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5616dd0f4da0 0x5616dd0f71c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1844085fd3f3e64b server_cookie=4b7c9f1b9772647a in_seq=0 out_seq=0 2024-09-16T00:33:15.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.580+0000 7f2b577fe640 10 client.16476 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:15.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b577fe640 1 -- 172.21.15.1:0/310176177 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5616dcfdb0a0 con 0x5616dd0f4da0 2024-09-16T00:33:15.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b577fe640 10 client.16476 handle_command_reply: tid=0 2024-09-16T00:33:15.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 client.16476 shutdown 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 -- 172.21.15.1:0/310176177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 msgr2=0x5616dd081890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd081890 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2b68003d00 tx=0x7f2b68005b20 comp rx=0 tx=0).stop 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 -- 172.21.15.1:0/310176177 shutdown_connections 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5616dd0f4da0 0x5616dd0f71c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616dd079240 0x5616dd07c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616dd06e980 0x5616dd07c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 --2- 172.21.15.1:0/310176177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5616dd06e060 0x5616dd081890 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 -- 172.21.15.1:0/310176177 >> 172.21.15.1:0/310176177 conn(0x5616dd06a5f0 msgr2=0x5616dd069750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b577fe640 0 client.16476 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 -- 172.21.15.1:0/310176177 shutdown_connections 2024-09-16T00:33:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 1 -- 172.21.15.1:0/310176177 wait complete. 2024-09-16T00:33:15.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.582+0000 7f2b80cff740 20 client.16476 trim_cache size 0 max 16384 2024-09-16T00:33:15.589 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:33:15.589 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:33:15.589 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:33:15.589 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 16, 2024-09-16T00:33:15.589 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:33:15.589 INFO:teuthology.orchestra.run.smithi001.stdout: "ec419676": { 2024-09-16T00:33:15.589 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 16, 2024-09-16T00:33:15.589 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 133.8, 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:33:15.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:15.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:15.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:15.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:15.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:15.591 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:33:15.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 -- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2b600a3240 msgr2=0x7f2b600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:15.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2b600a3240 0x7f2b600a5700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2b70044000 tx=0x7f2b70004990 comp rx=0 tx=0).stop 2024-09-16T00:33:15.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 -- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b78100e30 msgr2=0x7f2b780fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:15.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b78100e30 0x7f2b780fcbd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00d770 tx=0x7f2b6c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:33:15.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 -- 172.21.15.1:0/3068017550 shutdown_connections 2024-09-16T00:33:15.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2b600a3240 0x7f2b600a5700 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b781055b0 0x7f2b780fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b78100e30 0x7f2b780fcbd0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 --2- 172.21.15.1:0/3068017550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b78100480 0x7f2b780fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:15.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 -- 172.21.15.1:0/3068017550 >> 172.21.15.1:0/3068017550 conn(0x7f2b78077210 msgr2=0x7f2b78103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:15.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 -- 172.21.15.1:0/3068017550 shutdown_connections 2024-09-16T00:33:15.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:15.583+0000 7f2b7e926640 1 -- 172.21.15.1:0/3068017550 wait complete. 2024-09-16T00:33:16.428 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ec419676', total seconds elapsed: 18.8 2024-09-16T00:33:16.509 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:16 smithi019 ceph-mon[35186]: pgmap v496: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-09-16T00:33:16.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:16 smithi001 ceph-mon[30934]: pgmap v496: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-09-16T00:33:16.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:16 smithi153 ceph-mon[35538]: pgmap v496: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-09-16T00:33:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:16.999+0000 7f4ea848b640 1 -- 172.21.15.1:0/4294803931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 msgr2=0x7f4ea0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:16.999+0000 7f4ea848b640 1 --2- 172.21.15.1:0/4294803931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 0x7f4ea0100ee0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009930 tx=0x7f4e9002a420 comp rx=0 tx=0).stop 2024-09-16T00:33:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:16.999+0000 7f4ea848b640 1 -- 172.21.15.1:0/4294803931 shutdown_connections 2024-09-16T00:33:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:16.999+0000 7f4ea848b640 1 --2- 172.21.15.1:0/4294803931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea0101420 0x7f4ea010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.000+0000 7f4ea848b640 1 --2- 172.21.15.1:0/4294803931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 0x7f4ea0100ee0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.000+0000 7f4ea848b640 1 --2- 172.21.15.1:0/4294803931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea0106a80 0x7f4ea0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.000+0000 7f4ea848b640 1 -- 172.21.15.1:0/4294803931 >> 172.21.15.1:0/4294803931 conn(0x7f4ea00fc7a0 msgr2=0x7f4ea00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.000+0000 7f4ea848b640 1 -- 172.21.15.1:0/4294803931 shutdown_connections 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.000+0000 7f4ea848b640 1 -- 172.21.15.1:0/4294803931 wait complete. 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.000+0000 7f4ea848b640 1 Processor -- start 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.000+0000 7f4ea848b640 1 -- start start 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.000+0000 7f4ea848b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 0x7f4ea01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea848b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea0101420 0x7f4ea0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea848b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea0106a80 0x7f4ea019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea848b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0111fa0 con 0x7f4ea0100a80 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea848b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0111e20 con 0x7f4ea0101420 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea848b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0112120 con 0x7f4ea0106a80 2024-09-16T00:33:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6227640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 0x7f4ea01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6227640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 0x7f4ea01984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54772/0 (socket says 172.21.15.1:54772) 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6227640 1 -- 172.21.15.1:0/54393549 learned_addr learned my addr 172.21.15.1:0/54393549 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea5a26640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea0101420 0x7f4ea0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6a28640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea0106a80 0x7f4ea019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6227640 1 -- 172.21.15.1:0/54393549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea0106a80 msgr2=0x7f4ea019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6227640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea0106a80 0x7f4ea019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6227640 1 -- 172.21.15.1:0/54393549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea0101420 msgr2=0x7f4ea0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6227640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea0101420 0x7f4ea0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6227640 1 -- 172.21.15.1:0/54393549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea019d490 con 0x7f4ea0100a80 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6a28640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea0106a80 0x7f4ea019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea6227640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 0x7f4ea01984e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00b4a0 tx=0x7f4e9c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4e977fe640 1 -- 172.21.15.1:0/54393549 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c014070 con 0x7f4ea0100a80 2024-09-16T00:33:17.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4e977fe640 1 -- 172.21.15.1:0/54393549 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c010af0 con 0x7f4ea0100a80 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea019d780 con 0x7f4ea0100a80 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.001+0000 7f4e977fe640 1 -- 172.21.15.1:0/54393549 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c017d10 con 0x7f4ea0100a80 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.002+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea01a5060 con 0x7f4ea0100a80 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.003+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea0100ee0 con 0x7f4ea0100a80 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.004+0000 7f4e977fe640 1 -- 172.21.15.1:0/54393549 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4e9c017070 con 0x7f4ea0100a80 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.004+0000 7f4e977fe640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e8c0a32c0 0x7f4e8c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.005+0000 7f4ea5a26640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e8c0a32c0 0x7f4e8c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.005+0000 7f4e977fe640 1 -- 172.21.15.1:0/54393549 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4e9c0cf480 con 0x7f4ea0100a80 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.005+0000 7f4ea5a26640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e8c0a32c0 0x7f4e8c0a5780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009900 tx=0x7f4e900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:17.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.008+0000 7f4e977fe640 1 -- 172.21.15.1:0/54393549 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e9c09bd40 con 0x7f4ea0100a80 2024-09-16T00:33:17.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.220+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ea0109040 con 0x7f4ea0100a80 2024-09-16T00:33:17.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.220+0000 7f4e977fe640 1 -- 172.21.15.1:0/54393549 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f4e9c097070 con 0x7f4ea0100a80 2024-09-16T00:33:17.221 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:33:17.221 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:33:17.221 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:33:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e8c0a32c0 msgr2=0x7f4e8c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e8c0a32c0 0x7f4e8c0a5780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009900 tx=0x7f4e900058d0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 msgr2=0x7f4ea01984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 0x7f4ea01984e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00b4a0 tx=0x7f4e9c00b960 comp rx=0 tx=0).stop 2024-09-16T00:33:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 shutdown_connections 2024-09-16T00:33:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e8c0a32c0 0x7f4e8c0a5780 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea0106a80 0x7f4ea019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea0101420 0x7f4ea0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 --2- 172.21.15.1:0/54393549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ea0100a80 0x7f4ea01984e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 >> 172.21.15.1:0/54393549 conn(0x7f4ea00fc7a0 msgr2=0x7f4ea00fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 shutdown_connections 2024-09-16T00:33:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:17.223+0000 7f4ea848b640 1 -- 172.21.15.1:0/54393549 wait complete. 2024-09-16T00:33:17.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:17 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/54393549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:17.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:17 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/54393549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:17.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:17 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/54393549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:17.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:33:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:18 smithi019 ceph-mon[35186]: pgmap v497: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:33:18.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:18 smithi001 ceph-mon[30934]: pgmap v497: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:33:18.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:18 smithi153 ceph-mon[35538]: pgmap v497: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.809+0000 7f9d22514640 1 -- 172.21.15.1:0/2868989118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c102790 msgr2=0x7f9d1c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.809+0000 7f9d22514640 1 --2- 172.21.15.1:0/2868989118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c102790 0x7f9d1c1067e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009900 tx=0x7f9d1002a410 comp rx=0 tx=0).stop 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.809+0000 7f9d22514640 1 -- 172.21.15.1:0/2868989118 shutdown_connections 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.809+0000 7f9d22514640 1 --2- 172.21.15.1:0/2868989118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d1c106f10 0x7f9d1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.809+0000 7f9d22514640 1 --2- 172.21.15.1:0/2868989118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c102790 0x7f9d1c1067e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.809+0000 7f9d22514640 1 --2- 172.21.15.1:0/2868989118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d1c101de0 0x7f9d1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.809+0000 7f9d22514640 1 -- 172.21.15.1:0/2868989118 >> 172.21.15.1:0/2868989118 conn(0x7f9d1c0fd8c0 msgr2=0x7f9d1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.809+0000 7f9d22514640 1 -- 172.21.15.1:0/2868989118 shutdown_connections 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.809+0000 7f9d22514640 1 -- 172.21.15.1:0/2868989118 wait complete. 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.810+0000 7f9d22514640 1 Processor -- start 2024-09-16T00:33:18.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.810+0000 7f9d22514640 1 -- start start 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d22514640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c101de0 0x7f9d1c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d22514640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d1c102790 0x7f9d1c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d22514640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d1c106f10 0x7f9d1c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d22514640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c111f70 con 0x7f9d1c101de0 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d22514640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c111df0 con 0x7f9d1c102790 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d22514640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1120f0 con 0x7f9d1c106f10 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c101de0 0x7f9d1c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c101de0 0x7f9d1c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54780/0 (socket says 172.21.15.1:54780) 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1bfff640 1 -- 172.21.15.1:0/648453514 learned_addr learned my addr 172.21.15.1:0/648453514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d1c106f10 0x7f9d1c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1b7fe640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d1c102790 0x7f9d1c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1bfff640 1 -- 172.21.15.1:0/648453514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d1c106f10 msgr2=0x7f9d1c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d1c106f10 0x7f9d1c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1bfff640 1 -- 172.21.15.1:0/648453514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d1c102790 msgr2=0x7f9d1c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d1c102790 0x7f9d1c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1bfff640 1 -- 172.21.15.1:0/648453514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c19d380 con 0x7f9d1c101de0 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c101de0 0x7f9d1c1983d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00d770 tx=0x7f9d0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.811+0000 7f9d197fa640 1 -- 172.21.15.1:0/648453514 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c017070 con 0x7f9d1c101de0 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.812+0000 7f9d22514640 1 -- 172.21.15.1:0/648453514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c19d670 con 0x7f9d1c101de0 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.812+0000 7f9d197fa640 1 -- 172.21.15.1:0/648453514 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d0c005500 con 0x7f9d1c101de0 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.812+0000 7f9d197fa640 1 -- 172.21.15.1:0/648453514 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c004e00 con 0x7f9d1c101de0 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.812+0000 7f9d22514640 1 -- 172.21.15.1:0/648453514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c1a4f50 con 0x7f9d1c101de0 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d197fa640 1 -- 172.21.15.1:0/648453514 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9d0c004fa0 con 0x7f9d1c101de0 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d248ed740 1 Processor -- start 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d248ed740 1 -- start start 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f2b40 0x55b6884f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 0x55b6884f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 0x55b688502b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d248ed740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b6884572f0 con 0x55b6884f2b40 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d248ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b6883fecf0 con 0x55b6884f15d0 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d248ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b6884532b0 con 0x55b688500780 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d197fa640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9d040a3240 0x7f9d040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d20ab1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f2b40 0x55b6884f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.814+0000 7f9d20ab1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f2b40 0x55b6884f1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54792/0 (socket says 172.21.15.1:54792) 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d20ab1640 1 -- 172.21.15.1:0/870857166 learned_addr learned my addr 172.21.15.1:0/870857166 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9d040a3240 0x7f9d040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d197fa640 1 -- 172.21.15.1:0/648453514 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9d0c0cf690 con 0x7f9d1c101de0 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d1b7fe640 1 --2- 172.21.15.1:0/870857166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 0x55b688502b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/870857166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 0x55b6884f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d20ab1640 1 -- 172.21.15.1:0/870857166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 msgr2=0x55b688502b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/870857166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 0x55b688502b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d20ab1640 1 -- 172.21.15.1:0/870857166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 msgr2=0x55b6884f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/870857166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 0x55b6884f1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d20ab1640 1 -- 172.21.15.1:0/870857166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6883e33a0 con 0x55b6884f2b40 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/870857166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f2b40 0x55b6884f1090 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9d14014e80 tx=0x7f9d140117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d85f29053d9d845 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9d040a3240 0x7f9d040a5700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d14011cb0 tx=0x7f9d14016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d027fc640 1 -- 172.21.15.1:0/870857166 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14024070 con 0x55b6884f2b40 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d027fc640 1 -- 172.21.15.1:0/870857166 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d14007c60 con 0x55b6884f2b40 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.815+0000 7f9d027fc640 1 -- 172.21.15.1:0/870857166 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14005880 con 0x55b6884f2b40 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 -- 172.21.15.1:0/870857166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f2b40 msgr2=0x55b6884f1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:18.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 --2- 172.21.15.1:0/870857166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f2b40 0x55b6884f1090 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9d14014e80 tx=0x7f9d140117b0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 -- 172.21.15.1:0/870857166 shutdown_connections 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 --2- 172.21.15.1:0/870857166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 0x55b688502b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 --2- 172.21.15.1:0/870857166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 0x55b6884f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 --2- 172.21.15.1:0/870857166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f2b40 0x55b6884f1090 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 -- 172.21.15.1:0/870857166 >> 172.21.15.1:0/870857166 conn(0x55b6884ba310 msgr2=0x55b6884edc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 -- 172.21.15.1:0/870857166 shutdown_connections 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 -- 172.21.15.1:0/870857166 wait complete. 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.816+0000 7f9d248ed740 1 -- start start 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d248ed740 1 Processor -- start 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 0x55b6885001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f2b40 0x55b6884fd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b6884fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d248ed740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b688503420 con 0x55b688500780 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d248ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b688412060 con 0x55b6884f15d0 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d248ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b6885035a0 con 0x55b6884f2b40 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d248ed740 10 client.0 _ll_register_callbacks cb 0x55b6881658c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d20ab1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b6884fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d20ab1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b6884fdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54808/0 (socket says 172.21.15.1:54808) 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d20ab1640 1 -- 172.21.15.1:0/3652516717 learned_addr learned my addr 172.21.15.1:0/3652516717 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 0x55b6885001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d1b7fe640 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f2b40 0x55b6884fd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d20ab1640 1 -- 172.21.15.1:0/3652516717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f2b40 msgr2=0x55b6884fd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f2b40 0x55b6884fd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d20ab1640 1 -- 172.21.15.1:0/3652516717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 msgr2=0x55b6885001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 0x55b6885001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d20ab1640 1 -- 172.21.15.1:0/3652516717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6881c83c0 con 0x55b688500780 2024-09-16T00:33:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.817+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b6884fdbe0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9d140053c0 tx=0x7f9d140052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f523733e803c1927 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3652516717 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14024050 con 0x55b688500780 2024-09-16T00:33:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3652516717 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9d140057c0 con 0x55b688500780 2024-09-16T00:33:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9d248ed740 1 -- 172.21.15.1:0/3652516717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d14007c60 con 0x55b688500780 2024-09-16T00:33:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3652516717 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d1400ce00 con 0x55b688500780 2024-09-16T00:33:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9d248ed740 10 client.16494 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9d248ed740 1 -- 172.21.15.1:0/3652516717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b68850ef50 con 0x55b688500780 2024-09-16T00:33:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3652516717 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d14007c60 con 0x55b688500780 2024-09-16T00:33:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3652516717 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9d1400f900 con 0x55b688500780 2024-09-16T00:33:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9d248ed740 10 client.16494 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9d248ed740 10 client.16494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:33:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9d248ed740 20 client.16494 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b688510410 0x55b688512800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9d248ed740 4 client.16494 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.818+0000 7f9d248ed740 1 -- 172.21.15.1:0/3652516717 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b688464860 con 0x55b688510410 2024-09-16T00:33:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.819+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b688510410 0x55b688512800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.819+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b688510410 0x55b688512800 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ec3393ecabb3103 server_cookie=c8c99ec0368781f2 in_seq=0 out_seq=0 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.819+0000 7f9cfb7fe640 10 client.16494 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3652516717 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b688464860 con 0x55b688510410 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9cfb7fe640 10 client.16494 handle_command_reply: tid=0 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 client.16494 shutdown 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 -- 172.21.15.1:0/3652516717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 msgr2=0x55b6884fdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b6884fdbe0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9d140053c0 tx=0x7f9d140052e0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 -- 172.21.15.1:0/3652516717 shutdown_connections 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b6884fdbe0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f2b40 0x55b6884fd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b688510410 0x55b688512800 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3652516717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f15d0 0x55b6885001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9cfb7fe640 0 client.16494 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:33:18.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 -- 172.21.15.1:0/3652516717 >> 172.21.15.1:0/3652516717 conn(0x55b6884ba310 msgr2=0x55b6884f3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:18.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 -- 172.21.15.1:0/3652516717 shutdown_connections 2024-09-16T00:33:18.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 1 -- 172.21.15.1:0/3652516717 wait complete. 2024-09-16T00:33:18.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.820+0000 7f9d248ed740 20 client.16494 trim_cache size 0 max 16384 2024-09-16T00:33:18.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.833+0000 7f9d248ed740 1 Processor -- start 2024-09-16T00:33:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.833+0000 7f9d248ed740 1 -- start start 2024-09-16T00:33:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f15d0 0x55b68850c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 0x55b688517dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b688518310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d248ed740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1af4f0 con 0x55b688500780 2024-09-16T00:33:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d248ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1acdc0 con 0x55b6884f2b40 2024-09-16T00:33:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d248ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1aac10 con 0x55b6884f15d0 2024-09-16T00:33:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d20ab1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b688518310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d20ab1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b688518310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54816/0 (socket says 172.21.15.1:54816) 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d20ab1640 1 -- 172.21.15.1:0/3231348788 learned_addr learned my addr 172.21.15.1:0/3231348788 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d1b7fe640 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 0x55b688517dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f15d0 0x55b68850c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d20ab1640 1 -- 172.21.15.1:0/3231348788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f15d0 msgr2=0x55b68850c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f15d0 0x55b68850c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d20ab1640 1 -- 172.21.15.1:0/3231348788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 msgr2=0x55b688517dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 0x55b688517dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d20ab1640 1 -- 172.21.15.1:0/3231348788 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68850cc00 con 0x55b688500780 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f15d0 0x55b68850c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:18.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b688518310 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9d14005010 tx=0x7f9d14014e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927837755b238a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.834+0000 7f9d01ffb640 1 -- 172.21.15.1:0/3231348788 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14007860 con 0x55b688500780 2024-09-16T00:33:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d01ffb640 1 -- 172.21.15.1:0/3231348788 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d14007c60 con 0x55b688500780 2024-09-16T00:33:18.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d01ffb640 1 -- 172.21.15.1:0/3231348788 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400f450 con 0x55b688500780 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d248ed740 1 -- 172.21.15.1:0/3231348788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 msgr2=0x55b688518310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b688518310 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9d14005010 tx=0x7f9d14014e40 comp rx=0 tx=0).stop 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d248ed740 1 -- 172.21.15.1:0/3231348788 shutdown_connections 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b688500780 0x55b688518310 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 0x55b688517dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3231348788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6884f15d0 0x55b68850c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d248ed740 1 -- 172.21.15.1:0/3231348788 >> 172.21.15.1:0/3231348788 conn(0x55b6884ba310 msgr2=0x55b6884ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d248ed740 1 -- 172.21.15.1:0/3231348788 shutdown_connections 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.835+0000 7f9d248ed740 1 -- 172.21.15.1:0/3231348788 wait complete. 2024-09-16T00:33:18.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d248ed740 1 -- start start 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d248ed740 1 Processor -- start 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f15d0 0x55b688503d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 0x55b6885042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d248ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 0x55b688578d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d248ed740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1ab280 con 0x55b6884f15d0 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d248ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1ac6f0 con 0x55b6884f2b40 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d248ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1ab950 con 0x55b688500780 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d248ed740 10 client.0 _ll_register_callbacks cb 0x7f9d0c0962e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f15d0 0x55b688503d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f15d0 0x55b688503d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54826/0 (socket says 172.21.15.1:54826) 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d1bfff640 1 -- 172.21.15.1:0/3752062582 learned_addr learned my addr 172.21.15.1:0/3752062582 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d20ab1640 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 0x55b688578d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.836+0000 7f9d1b7fe640 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 0x55b6885042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d1bfff640 1 -- 172.21.15.1:0/3752062582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 msgr2=0x55b688578d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 0x55b688578d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d1bfff640 1 -- 172.21.15.1:0/3752062582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 msgr2=0x55b6885042a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 0x55b6885042a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d1bfff640 1 -- 172.21.15.1:0/3752062582 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b688579350 con 0x55b6884f15d0 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d1b7fe640 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 0x55b6885042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d1bfff640 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f15d0 0x55b688503d60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c09b8f0 tx=0x7f9d0c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8bdc6b1039d01f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3752062582 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0b9060 con 0x55b6884f15d0 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3752062582 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9d0c005290 con 0x55b6884f15d0 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d248ed740 1 -- 172.21.15.1:0/3752062582 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d14007c60 con 0x55b6884f15d0 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3752062582 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d0c020040 con 0x55b6884f15d0 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d248ed740 10 client.16500 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:18.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9d248ed740 1 -- 172.21.15.1:0/3752062582 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b68857a620 con 0x55b6884f15d0 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3752062582 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d14007c60 con 0x55b6884f15d0 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.837+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3752062582 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9d0c097fd0 con 0x55b6884f15d0 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9d248ed740 10 client.16500 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9d248ed740 10 client.16500 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9d248ed740 20 client.16500 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b68857aca0 0x55b68857d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9d248ed740 4 client.16500 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9d248ed740 1 -- 172.21.15.1:0/3752062582 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b68845ed70 con 0x55b68857aca0 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9d1b7fe640 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b68857aca0 0x55b68857d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9cfb7fe640 10 client.16500 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9d1b7fe640 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b68857aca0 0x55b68857d0c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76fe6b42166f8961 server_cookie=601e477347522b14 in_seq=0 out_seq=0 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9cfb7fe640 1 -- 172.21.15.1:0/3752062582 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b68845ed70 con 0x55b68857aca0 2024-09-16T00:33:18.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9cfb7fe640 10 client.16500 handle_command_reply: tid=0 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.838+0000 7f9d248ed740 1 client.16500 shutdown 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 -- 172.21.15.1:0/3752062582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f15d0 msgr2=0x55b688503d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f15d0 0x55b688503d60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c09b8f0 tx=0x7f9d0c009c30 comp rx=0 tx=0).stop 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 -- 172.21.15.1:0/3752062582 shutdown_connections 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b688500780 0x55b688578d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b68857aca0 0x55b68857d0c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6884f2b40 0x55b6885042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 --2- 172.21.15.1:0/3752062582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6884f15d0 0x55b688503d60 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 -- 172.21.15.1:0/3752062582 >> 172.21.15.1:0/3752062582 conn(0x55b6884ba310 msgr2=0x55b6884fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9cfb7fe640 0 client.16500 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 -- 172.21.15.1:0/3752062582 shutdown_connections 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 1 -- 172.21.15.1:0/3752062582 wait complete. 2024-09-16T00:33:18.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d248ed740 20 client.16500 trim_cache size 0 max 16384 2024-09-16T00:33:18.846 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:33:18.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d22514640 1 -- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9d040a3240 msgr2=0x7f9d040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d22514640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9d040a3240 0x7f9d040a5700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d14011cb0 tx=0x7f9d14016000 comp rx=0 tx=0).stop 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d22514640 1 -- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c101de0 msgr2=0x7f9d1c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.839+0000 7f9d22514640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c101de0 0x7f9d1c1983d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00d770 tx=0x7f9d0c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.840+0000 7f9d22514640 1 -- 172.21.15.1:0/648453514 shutdown_connections 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.840+0000 7f9d22514640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9d040a3240 0x7f9d040a5700 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.840+0000 7f9d22514640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d1c106f10 0x7f9d1c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.840+0000 7f9d22514640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d1c102790 0x7f9d1c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.840+0000 7f9d22514640 1 --2- 172.21.15.1:0/648453514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c101de0 0x7f9d1c1983d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.840+0000 7f9d22514640 1 -- 172.21.15.1:0/648453514 >> 172.21.15.1:0/648453514 conn(0x7f9d1c0fd8c0 msgr2=0x7f9d1c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.840+0000 7f9d22514640 1 -- 172.21.15.1:0/648453514 shutdown_connections 2024-09-16T00:33:18.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:18.840+0000 7f9d22514640 1 -- 172.21.15.1:0/648453514 wait complete. 2024-09-16T00:33:19.656 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:33:19.917 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:20.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.457+0000 7fe9ab178640 1 -- 172.21.15.1:0/2562075483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 msgr2=0x7fe9a4103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.457+0000 7fe9ab178640 1 --2- 172.21.15.1:0/2562075483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 0x7fe9a4103180 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe998009900 tx=0x7fe99802a480 comp rx=0 tx=0).stop 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.457+0000 7fe9ab178640 1 -- 172.21.15.1:0/2562075483 shutdown_connections 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.457+0000 7fe9ab178640 1 --2- 172.21.15.1:0/2562075483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9a410c180 0x7fe9a410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.457+0000 7fe9ab178640 1 --2- 172.21.15.1:0/2562075483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a41036c0 0x7fe9a410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.457+0000 7fe9ab178640 1 --2- 172.21.15.1:0/2562075483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 0x7fe9a4103180 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.457+0000 7fe9ab178640 1 -- 172.21.15.1:0/2562075483 >> 172.21.15.1:0/2562075483 conn(0x7fe9a40fc7a0 msgr2=0x7fe9a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.457+0000 7fe9ab178640 1 -- 172.21.15.1:0/2562075483 shutdown_connections 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.457+0000 7fe9ab178640 1 -- 172.21.15.1:0/2562075483 wait complete. 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9ab178640 1 Processor -- start 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9ab178640 1 -- start start 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9ab178640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 0x7fe9a41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9ab178640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9a41036c0 0x7fe9a41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9ab178640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a410c180 0x7fe9a419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9ab178640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4111f60 con 0x7fe9a4102da0 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9ab178640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4111de0 con 0x7fe9a410c180 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9ab178640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41120e0 con 0x7fe9a41036c0 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9a8f14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 0x7fe9a41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9a8f14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 0x7fe9a41983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57552/0 (socket says 172.21.15.1:57552) 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.458+0000 7fe9a8f14640 1 -- 172.21.15.1:0/1043957397 learned_addr learned my addr 172.21.15.1:0/1043957397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a9715640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a410c180 0x7fe9a419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9a41036c0 0x7fe9a41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a8f14640 1 -- 172.21.15.1:0/1043957397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9a41036c0 msgr2=0x7fe9a41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9a41036c0 0x7fe9a41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a8f14640 1 -- 172.21.15.1:0/1043957397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a410c180 msgr2=0x7fe9a419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a410c180 0x7fe9a419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a8f14640 1 -- 172.21.15.1:0/1043957397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a419d350 con 0x7fe9a4102da0 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9a41036c0 0x7fe9a41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 0x7fe9a41983a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe998006010 tx=0x7fe9980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a1ffb640 1 -- 172.21.15.1:0/1043957397 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99803f030 con 0x7fe9a4102da0 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a1ffb640 1 -- 172.21.15.1:0/1043957397 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe998003ce0 con 0x7fe9a4102da0 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9ab178640 1 -- 172.21.15.1:0/1043957397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a419d5e0 con 0x7fe9a4102da0 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9a1ffb640 1 -- 172.21.15.1:0/1043957397 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998035070 con 0x7fe9a4102da0 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.459+0000 7fe9ab178640 1 -- 172.21.15.1:0/1043957397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a41a4f20 con 0x7fe9a4102da0 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9a1ffb640 1 -- 172.21.15.1:0/1043957397 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe998042030 con 0x7fe9a4102da0 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9ad551740 1 Processor -- start 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9ad551740 1 -- start start 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 0x55bb6c940090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c940a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c94d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9ad551740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c8a62f0 con 0x55bb6c94b370 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9ad551740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c84dcf0 con 0x55bb6c9405d0 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9ad551740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c8a22b0 con 0x55bb6c941b40 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c94d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c94d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57556/0 (socket says 172.21.15.1:57556) 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9a3fff640 1 -- 172.21.15.1:0/4280591967 learned_addr learned my addr 172.21.15.1:0/4280591967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.462+0000 7fe9a1ffb640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9940a3240 0x7fe9940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/4280591967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c940a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a9715640 1 --2- 172.21.15.1:0/4280591967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 0x55bb6c940090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a9715640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9940a3240 0x7fe9940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a3fff640 1 -- 172.21.15.1:0/4280591967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 msgr2=0x55bb6c940090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/4280591967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 0x55bb6c940090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a3fff640 1 -- 172.21.15.1:0/4280591967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 msgr2=0x55bb6c940a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:20.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/4280591967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c940a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a3fff640 1 -- 172.21.15.1:0/4280591967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb6c8323a0 con 0x55bb6c94b370 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a1ffb640 1 -- 172.21.15.1:0/1043957397 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe9980f03e0 con 0x7fe9a4102da0 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/4280591967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c94d760 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe990009c20 tx=0x7fe99000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfba849e7dd453e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe98effd640 1 -- 172.21.15.1:0/4280591967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990016070 con 0x55bb6c94b370 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe9a9715640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9940a3240 0x7fe9940a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe99c005cc0 tx=0x7fe99c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe98effd640 1 -- 172.21.15.1:0/4280591967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe990011950 con 0x55bb6c94b370 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.463+0000 7fe98effd640 1 -- 172.21.15.1:0/4280591967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990010800 con 0x55bb6c94b370 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 -- 172.21.15.1:0/4280591967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 msgr2=0x55bb6c94d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 --2- 172.21.15.1:0/4280591967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c94d760 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe990009c20 tx=0x7fe99000e6c0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 -- 172.21.15.1:0/4280591967 shutdown_connections 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 --2- 172.21.15.1:0/4280591967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c94d760 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 --2- 172.21.15.1:0/4280591967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c940a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 --2- 172.21.15.1:0/4280591967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 0x55bb6c940090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 -- 172.21.15.1:0/4280591967 >> 172.21.15.1:0/4280591967 conn(0x55bb6c909310 msgr2=0x55bb6c93cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 -- 172.21.15.1:0/4280591967 shutdown_connections 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 -- 172.21.15.1:0/4280591967 wait complete. 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 -- start start 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.464+0000 7fe9ad551740 1 Processor -- start 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c9405d0 0x55bb6c94ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c941b40 0x55bb6c947f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 0x55bb6c9484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9ad551740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c94e070 con 0x55bb6c9405d0 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9ad551740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c861060 con 0x55bb6c941b40 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9ad551740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c94e1f0 con 0x55bb6c94b370 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9ad551740 10 client.0 _ll_register_callbacks cb 0x55bb6c5b48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a8f14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c9405d0 0x55bb6c94ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a8f14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c9405d0 0x55bb6c94ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57566/0 (socket says 172.21.15.1:57566) 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a8f14640 1 -- 172.21.15.1:0/1041906531 learned_addr learned my addr 172.21.15.1:0/1041906531 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c941b40 0x55bb6c947f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a9715640 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 0x55bb6c9484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a8f14640 1 -- 172.21.15.1:0/1041906531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 msgr2=0x55bb6c9484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 0x55bb6c9484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a8f14640 1 -- 172.21.15.1:0/1041906531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c941b40 msgr2=0x55bb6c947f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c941b40 0x55bb6c947f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a8f14640 1 -- 172.21.15.1:0/1041906531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb6c6173c0 con 0x55bb6c9405d0 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c9405d0 0x55bb6c94ace0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe998005ab0 tx=0x7fe99808b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce56a567fbbc244 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe983fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.465+0000 7fe983fff640 1 -- 172.21.15.1:0/1041906531 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998049070 con 0x55bb6c9405d0 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe983fff640 1 -- 172.21.15.1:0/1041906531 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe9980c4f40 con 0x55bb6c9405d0 2024-09-16T00:33:20.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9ad551740 1 -- 172.21.15.1:0/1041906531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe990011950 con 0x55bb6c9405d0 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe983fff640 1 -- 172.21.15.1:0/1041906531 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9980bc2a0 con 0x55bb6c9405d0 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9ad551740 10 client.16512 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9ad551740 1 -- 172.21.15.1:0/1041906531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bb6c959a80 con 0x55bb6c9405d0 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe983fff640 1 -- 172.21.15.1:0/1041906531 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe990011950 con 0x55bb6c9405d0 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe983fff640 1 -- 172.21.15.1:0/1041906531 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe9980bbbb0 con 0x55bb6c9405d0 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9ad551740 10 client.16512 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9ad551740 10 client.16512 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9ad551740 20 client.16512 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9ad551740 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bb6c95ae70 0x55bb6c95d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9ad551740 4 client.16512 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9ad551740 1 -- 172.21.15.1:0/1041906531 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb6c8b3860 con 0x55bb6c95ae70 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.466+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bb6c95ae70 0x55bb6c95d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.467+0000 7fe983fff640 10 client.16512 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:33:20.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.467+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bb6c95ae70 0x55bb6c95d2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a507a96b1feb7066 server_cookie=a3bebdddf04defad in_seq=0 out_seq=0 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.467+0000 7fe983fff640 1 -- 172.21.15.1:0/1041906531 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb6c8b3860 con 0x55bb6c95ae70 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.467+0000 7fe983fff640 10 client.16512 handle_command_reply: tid=0 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 client.16512 shutdown 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 -- 172.21.15.1:0/1041906531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c9405d0 msgr2=0x55bb6c94ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c9405d0 0x55bb6c94ace0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe998005ab0 tx=0x7fe99808b560 comp rx=0 tx=0).stop 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 -- 172.21.15.1:0/1041906531 shutdown_connections 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 0x55bb6c9484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c941b40 0x55bb6c947f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bb6c95ae70 0x55bb6c95d2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 --2- 172.21.15.1:0/1041906531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c9405d0 0x55bb6c94ace0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 -- 172.21.15.1:0/1041906531 >> 172.21.15.1:0/1041906531 conn(0x55bb6c909310 msgr2=0x55bb6c942960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe983fff640 0 client.16512 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:33:20.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 -- 172.21.15.1:0/1041906531 shutdown_connections 2024-09-16T00:33:20.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 1 -- 172.21.15.1:0/1041906531 wait complete. 2024-09-16T00:33:20.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.468+0000 7fe9ad551740 20 client.16512 trim_cache size 0 max 16384 2024-09-16T00:33:20.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.481+0000 7fe9ad551740 1 Processor -- start 2024-09-16T00:33:20.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.481+0000 7fe9ad551740 1 -- start start 2024-09-16T00:33:20.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.481+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c946860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.481+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 0x55bb6c949f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.481+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c943120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.481+0000 7fe9ad551740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41b16e0 con 0x55bb6c94b370 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.481+0000 7fe9ad551740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41aefb0 con 0x55bb6c9405d0 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.481+0000 7fe9ad551740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41ace00 con 0x55bb6c941b40 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.481+0000 7fe9a9715640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c943120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a9715640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c943120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57568/0 (socket says 172.21.15.1:57568) 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a9715640 1 -- 172.21.15.1:0/2738766995 learned_addr learned my addr 172.21.15.1:0/2738766995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c946860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 0x55bb6c949f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a9715640 1 -- 172.21.15.1:0/2738766995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 msgr2=0x55bb6c949f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a9715640 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 0x55bb6c949f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a9715640 1 -- 172.21.15.1:0/2738766995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 msgr2=0x55bb6c946860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a9715640 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c946860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a9715640 1 -- 172.21.15.1:0/2738766995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb6c946f40 con 0x55bb6c94b370 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 0x55bb6c949f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c946860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe9a9715640 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c943120 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00e960 tx=0x7fe99c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599e7e560277e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe98e7fc640 1 -- 172.21.15.1:0/2738766995 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c017040 con 0x55bb6c94b370 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe98e7fc640 1 -- 172.21.15.1:0/2738766995 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe99c004d20 con 0x55bb6c94b370 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.482+0000 7fe98e7fc640 1 -- 172.21.15.1:0/2738766995 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c005400 con 0x55bb6c94b370 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 -- 172.21.15.1:0/2738766995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 msgr2=0x55bb6c943120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c943120 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00e960 tx=0x7fe99c00e000 comp rx=0 tx=0).stop 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 -- 172.21.15.1:0/2738766995 shutdown_connections 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c94b370 0x55bb6c943120 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c941b40 0x55bb6c949f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 --2- 172.21.15.1:0/2738766995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c946860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 -- 172.21.15.1:0/2738766995 >> 172.21.15.1:0/2738766995 conn(0x55bb6c909310 msgr2=0x55bb6c9497f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 -- 172.21.15.1:0/2738766995 shutdown_connections 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 -- 172.21.15.1:0/2738766995 wait complete. 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.483+0000 7fe9ad551740 1 -- start start 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9ad551740 1 Processor -- start 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c944f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c941b40 0x55bb6c945480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9ad551740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 0x55bb6c9459c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9ad551740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41ad470 con 0x55bb6c941b40 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9ad551740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41ae8e0 con 0x55bb6c9405d0 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9ad551740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41adb40 con 0x55bb6c94b370 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9ad551740 10 client.0 _ll_register_callbacks cb 0x7fe998005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c941b40 0x55bb6c945480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c941b40 0x55bb6c945480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57574/0 (socket says 172.21.15.1:57574) 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a3fff640 1 -- 172.21.15.1:0/3814454513 learned_addr learned my addr 172.21.15.1:0/3814454513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c944f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a9715640 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 0x55bb6c9459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a3fff640 1 -- 172.21.15.1:0/3814454513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 msgr2=0x55bb6c9459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 0x55bb6c9459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a3fff640 1 -- 172.21.15.1:0/3814454513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 msgr2=0x55bb6c944f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c944f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a3fff640 1 -- 172.21.15.1:0/3814454513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb6c9c4b40 con 0x55bb6c941b40 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c944f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.484+0000 7fe9a9715640 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 0x55bb6c9459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe9a3fff640 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c941b40 0x55bb6c945480 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe9900097a0 tx=0x7fe990005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350168360abc7b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe983fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe983fff640 1 -- 172.21.15.1:0/3814454513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990016040 con 0x55bb6c941b40 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe983fff640 1 -- 172.21.15.1:0/3814454513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe990005240 con 0x55bb6c941b40 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe9ad551740 1 -- 172.21.15.1:0/3814454513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe99c004d20 con 0x55bb6c941b40 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe983fff640 1 -- 172.21.15.1:0/3814454513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe990039050 con 0x55bb6c941b40 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe9ad551740 10 client.16518 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe9ad551740 1 -- 172.21.15.1:0/3814454513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bb6c9c5de0 con 0x55bb6c941b40 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe983fff640 1 -- 172.21.15.1:0/3814454513 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe99c004d20 con 0x55bb6c941b40 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.485+0000 7fe983fff640 1 -- 172.21.15.1:0/3814454513 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe99000f040 con 0x55bb6c941b40 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe9ad551740 10 client.16518 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe9ad551740 10 client.16518 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe9ad551740 20 client.16518 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe9ad551740 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bb6c9c6230 0x55bb6c9c8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe9ad551740 4 client.16518 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe9ad551740 1 -- 172.21.15.1:0/3814454513 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb6c8add70 con 0x55bb6c9c6230 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bb6c9c6230 0x55bb6c9c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe9a8f14640 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bb6c9c6230 0x55bb6c9c8650 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4b594890781c0be server_cookie=3c2efc5f70ba9323 in_seq=0 out_seq=0 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe983fff640 10 client.16518 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe983fff640 1 -- 172.21.15.1:0/3814454513 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb6c8add70 con 0x55bb6c9c6230 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe983fff640 10 client.16518 handle_command_reply: tid=0 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.486+0000 7fe9ad551740 1 client.16518 shutdown 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 -- 172.21.15.1:0/3814454513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c941b40 msgr2=0x55bb6c945480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c941b40 0x55bb6c945480 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe9900097a0 tx=0x7fe990005c50 comp rx=0 tx=0).stop 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 -- 172.21.15.1:0/3814454513 shutdown_connections 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb6c94b370 0x55bb6c9459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55bb6c9c6230 0x55bb6c9c8650 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe983fff640 0 client.16518 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:33:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb6c941b40 0x55bb6c945480 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 --2- 172.21.15.1:0/3814454513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb6c9405d0 0x55bb6c944f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 -- 172.21.15.1:0/3814454513 >> 172.21.15.1:0/3814454513 conn(0x55bb6c909310 msgr2=0x55bb6c94a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 -- 172.21.15.1:0/3814454513 shutdown_connections 2024-09-16T00:33:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 1 -- 172.21.15.1:0/3814454513 wait complete. 2024-09-16T00:33:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.487+0000 7fe9ad551740 20 client.16518 trim_cache size 0 max 16384 2024-09-16T00:33:20.494 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:33:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 -- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9940a3240 msgr2=0x7fe9940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9940a3240 0x7fe9940a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe99c005cc0 tx=0x7fe99c004a40 comp rx=0 tx=0).stop 2024-09-16T00:33:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 -- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 msgr2=0x7fe9a41983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 0x7fe9a41983a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe998006010 tx=0x7fe9980042f0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 -- 172.21.15.1:0/1043957397 shutdown_connections 2024-09-16T00:33:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a410c180 0x7fe9a419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe9940a3240 0x7fe9940a5700 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9a41036c0 0x7fe9a41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 --2- 172.21.15.1:0/1043957397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9a4102da0 0x7fe9a41983a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 -- 172.21.15.1:0/1043957397 >> 172.21.15.1:0/1043957397 conn(0x7fe9a40fc7a0 msgr2=0x7fe9a40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:20.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 -- 172.21.15.1:0/1043957397 shutdown_connections 2024-09-16T00:33:20.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:20.488+0000 7fe9ab178640 1 -- 172.21.15.1:0/1043957397 wait complete. 2024-09-16T00:33:20.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:20 smithi019 ceph-mon[35186]: pgmap v498: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-16T00:33:20.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:20 smithi001 ceph-mon[30934]: pgmap v498: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-16T00:33:20.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:20 smithi153 ceph-mon[35538]: pgmap v498: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-16T00:33:21.269 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:33:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:22.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.033+0000 7fc300d8a640 1 -- 172.21.15.1:0/2089801089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc101de0 msgr2=0x7fc2fc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.033+0000 7fc300d8a640 1 --2- 172.21.15.1:0/2089801089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc101de0 0x7fc2fc1021c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009900 tx=0x7fc2ec02a410 comp rx=0 tx=0).stop 2024-09-16T00:33:22.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.034+0000 7fc300d8a640 1 -- 172.21.15.1:0/2089801089 shutdown_connections 2024-09-16T00:33:22.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.034+0000 7fc300d8a640 1 --2- 172.21.15.1:0/2089801089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2fc106f10 0x7fc2fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.034+0000 7fc300d8a640 1 --2- 172.21.15.1:0/2089801089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2fc102790 0x7fc2fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.034+0000 7fc300d8a640 1 --2- 172.21.15.1:0/2089801089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc101de0 0x7fc2fc1021c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.034+0000 7fc300d8a640 1 -- 172.21.15.1:0/2089801089 >> 172.21.15.1:0/2089801089 conn(0x7fc2fc0fd8c0 msgr2=0x7fc2fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.034+0000 7fc300d8a640 1 -- 172.21.15.1:0/2089801089 shutdown_connections 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.034+0000 7fc300d8a640 1 -- 172.21.15.1:0/2089801089 wait complete. 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.034+0000 7fc300d8a640 1 Processor -- start 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.034+0000 7fc300d8a640 1 -- start start 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc300d8a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2fc101de0 0x7fc2fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc300d8a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc102790 0x7fc2fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc300d8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2fc106f10 0x7fc2fc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc300d8a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc111f30 con 0x7fc2fc102790 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc300d8a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc111db0 con 0x7fc2fc106f10 2024-09-16T00:33:22.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc300d8a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1120b0 con 0x7fc2fc101de0 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc102790 0x7fc2fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc102790 0x7fc2fc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57588/0 (socket says 172.21.15.1:57588) 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2f3fff640 1 -- 172.21.15.1:0/1907264099 learned_addr learned my addr 172.21.15.1:0/1907264099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2fa59c640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2fc101de0 0x7fc2fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2fc106f10 0x7fc2fc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2f3fff640 1 -- 172.21.15.1:0/1907264099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2fc101de0 msgr2=0x7fc2fc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2fc101de0 0x7fc2fc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2f3fff640 1 -- 172.21.15.1:0/1907264099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2fc106f10 msgr2=0x7fc2fc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2fc106f10 0x7fc2fc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2f3fff640 1 -- 172.21.15.1:0/1907264099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc1a68f0 con 0x7fc2fc102790 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.035+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc102790 0x7fc2fc074a30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c920 tx=0x7fc2e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.036+0000 7fc2f37fe640 1 -- 172.21.15.1:0/1907264099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4007c70 con 0x7fc2fc102790 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.036+0000 7fc300d8a640 1 -- 172.21.15.1:0/1907264099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc1a6be0 con 0x7fc2fc102790 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.036+0000 7fc2f37fe640 1 -- 172.21.15.1:0/1907264099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e4007e10 con 0x7fc2fc102790 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.036+0000 7fc2f37fe640 1 -- 172.21.15.1:0/1907264099 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4018890 con 0x7fc2fc102790 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.036+0000 7fc300d8a640 1 -- 172.21.15.1:0/1907264099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc1a7170 con 0x7fc2fc102790 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.038+0000 7fc2f37fe640 1 -- 172.21.15.1:0/1907264099 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc2e4018a70 con 0x7fc2fc102790 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.038+0000 7fc303163740 1 Processor -- start 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.038+0000 7fc303163740 1 -- start start 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.038+0000 7fc303163740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc303163740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 0x5570abac1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc303163740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abace760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc303163740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5570aba272f0 con 0x5570abacc370 2024-09-16T00:33:22.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc303163740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570ab9cecf0 con 0x5570abac2b40 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc303163740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5570aba232b0 con 0x5570abac15d0 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abace760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:43076/0 (socket says 172.21.15.1:43076) 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abace760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57590/0 (socket says 172.21.15.1:57590) 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 -- 172.21.15.1:0/2576614814 learned_addr learned my addr 172.21.15.1:0/2576614814 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fa59c640 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 0x5570abac1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2f37fe640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2e80a3240 0x7fc2e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 -- 172.21.15.1:0/2576614814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 msgr2=0x5570abac1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 0x5570abac1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 -- 172.21.15.1:0/2576614814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 msgr2=0x5570abace760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abace760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 -- 172.21.15.1:0/2576614814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ab9b33a0 con 0x5570abac2b40 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fa59c640 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 0x5570abac1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abace760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2f37fe640 1 -- 172.21.15.1:0/1907264099 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc2e40cff10 con 0x7fc2fc102790 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2e80a3240 0x7fc2e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.039+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac1090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009f20 tx=0x7fc2f400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b225d4a0dac84ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.040+0000 7fc2f0ff9640 1 -- 172.21.15.1:0/2576614814 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4021070 con 0x5570abac2b40 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.040+0000 7fc2f0ff9640 1 -- 172.21.15.1:0/2576614814 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f4011e10 con 0x5570abac2b40 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.040+0000 7fc2f0ff9640 1 -- 172.21.15.1:0/2576614814 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4015070 con 0x5570abac2b40 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.040+0000 7fc303163740 1 -- 172.21.15.1:0/2576614814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 msgr2=0x5570abac1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.040+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2e80a3240 0x7fc2e80a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc2f401e000 tx=0x7fc2f4021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.040+0000 7fc303163740 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac1090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009f20 tx=0x7fc2f400f7f0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 -- 172.21.15.1:0/2576614814 shutdown_connections 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abace760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 0x5570abac1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 --2- 172.21.15.1:0/2576614814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac1090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 -- 172.21.15.1:0/2576614814 >> 172.21.15.1:0/2576614814 conn(0x5570aba8a310 msgr2=0x5570ababdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 -- 172.21.15.1:0/2576614814 shutdown_connections 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 -- 172.21.15.1:0/2576614814 wait complete. 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 -- start start 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 Processor -- start 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abacbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 0x5570abac9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5570abacf070 con 0x5570abac15d0 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570ab9e2060 con 0x5570abacc370 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5570abacf1f0 con 0x5570abac2b40 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc303163740 10 client.0 _ll_register_callbacks cb 0x5570ab7358c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.041+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abacbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abacbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57602/0 (socket says 172.21.15.1:57602) 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2fa59c640 1 -- 172.21.15.1:0/2563569275 learned_addr learned my addr 172.21.15.1:0/2563569275 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 0x5570abac9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2fad9d640 1 -- 172.21.15.1:0/2563569275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 msgr2=0x5570abac9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 0x5570abac9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2fad9d640 1 -- 172.21.15.1:0/2563569275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 msgr2=0x5570abacbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abacbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2fad9d640 1 -- 172.21.15.1:0/2563569275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ab7983c0 con 0x5570abacc370 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac9600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400fcf0 tx=0x7fc2f4010930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637e9453887f48f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2563569275 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4015050 con 0x5570abacc370 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2563569275 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc2f40103d0 con 0x5570abacc370 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.042+0000 7fc303163740 1 -- 172.21.15.1:0/2563569275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f4011e10 con 0x5570abacc370 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2563569275 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2f4009a00 con 0x5570abacc370 2024-09-16T00:33:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc303163740 10 client.25916 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc303163740 1 -- 172.21.15.1:0/2563569275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570abadab40 con 0x5570abacc370 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2563569275 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f4011e10 con 0x5570abacc370 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2563569275 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc2f4005040 con 0x5570abacc370 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc303163740 10 client.25916 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc303163740 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc303163740 20 client.25916 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc303163740 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5570abadbf40 0x5570abade330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc303163740 4 client.25916 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc303163740 1 -- 172.21.15.1:0/2563569275 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570aba34860 con 0x5570abadbf40 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.043+0000 7fc2fa59c640 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5570abadbf40 0x5570abade330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.044+0000 7fc2e1ffb640 10 client.25916 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.044+0000 7fc2fa59c640 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5570abadbf40 0x5570abade330 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3d962823a7f50e server_cookie=feaafb6b1c1886f0 in_seq=0 out_seq=0 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.044+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2563569275 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570aba34860 con 0x5570abadbf40 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.044+0000 7fc2e1ffb640 10 client.25916 handle_command_reply: tid=0 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.044+0000 7fc303163740 1 client.25916 shutdown 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 -- 172.21.15.1:0/2563569275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 msgr2=0x5570abac9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac9600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400fcf0 tx=0x7fc2f4010930 comp rx=0 tx=0).stop 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 -- 172.21.15.1:0/2563569275 shutdown_connections 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5570abadbf40 0x5570abade330 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac9600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 0x5570abac9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc2e1ffb640 0 client.25916 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:22.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 --2- 172.21.15.1:0/2563569275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abacbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 -- 172.21.15.1:0/2563569275 >> 172.21.15.1:0/2563569275 conn(0x5570aba8a310 msgr2=0x5570abac3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 -- 172.21.15.1:0/2563569275 shutdown_connections 2024-09-16T00:33:22.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 1 -- 172.21.15.1:0/2563569275 wait complete. 2024-09-16T00:33:22.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.045+0000 7fc303163740 20 client.25916 trim_cache size 0 max 16384 2024-09-16T00:33:22.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc303163740 1 Processor -- start 2024-09-16T00:33:22.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc303163740 1 -- start start 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc303163740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 0x5570abac88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc303163740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc303163740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abac4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc303163740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1b3830 con 0x5570abacc370 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc303163740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1b1100 con 0x5570abac2b40 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc303163740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1aef50 con 0x5570abac15d0 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abac4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abac4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57608/0 (socket says 172.21.15.1:57608) 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.058+0000 7fc2fad9d640 1 -- 172.21.15.1:0/3685995660 learned_addr learned my addr 172.21.15.1:0/3685995660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2fa59c640 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 0x5570abac88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2fad9d640 1 -- 172.21.15.1:0/3685995660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 msgr2=0x5570abac88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:22.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 0x5570abac88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2fad9d640 1 -- 172.21.15.1:0/3685995660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 msgr2=0x5570abac4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2fad9d640 1 -- 172.21.15.1:0/3685995660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570abacb980 con 0x5570abacc370 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abac4660 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005b40 tx=0x7fc2f4010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43474735ec0c4192 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2e3fff640 1 -- 172.21.15.1:0/3685995660 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4042070 con 0x5570abacc370 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2e3fff640 1 -- 172.21.15.1:0/3685995660 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f400fd20 con 0x5570abacc370 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.059+0000 7fc2e3fff640 1 -- 172.21.15.1:0/3685995660 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4015040 con 0x5570abacc370 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 -- 172.21.15.1:0/3685995660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 msgr2=0x5570abac4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abac4660 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005b40 tx=0x7fc2f4010c70 comp rx=0 tx=0).stop 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 -- 172.21.15.1:0/3685995660 shutdown_connections 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abacc370 0x5570abac4660 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abac2b40 0x5570abac4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 --2- 172.21.15.1:0/3685995660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac15d0 0x5570abac88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 -- 172.21.15.1:0/3685995660 >> 172.21.15.1:0/3685995660 conn(0x5570aba8a310 msgr2=0x5570abacb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 -- 172.21.15.1:0/3685995660 shutdown_connections 2024-09-16T00:33:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 -- 172.21.15.1:0/3685995660 wait complete. 2024-09-16T00:33:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 -- start start 2024-09-16T00:33:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.060+0000 7fc303163740 1 Processor -- start 2024-09-16T00:33:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc303163740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abad49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc303163740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 0x5570abac5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc303163740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc303163740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1af5c0 con 0x5570abac15d0 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc303163740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1b0a30 con 0x5570abacc370 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc303163740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1afc90 con 0x5570abac2b40 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc303163740 10 client.0 _ll_register_callbacks cb 0x7fc2f401f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abad49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abad49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57610/0 (socket says 172.21.15.1:57610) 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fa59c640 1 -- 172.21.15.1:0/2183944030 learned_addr learned my addr 172.21.15.1:0/2183944030 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 0x5570abac5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fa59c640 1 -- 172.21.15.1:0/2183944030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 msgr2=0x5570abac5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fa59c640 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 0x5570abac5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fa59c640 1 -- 172.21.15.1:0/2183944030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 msgr2=0x5570abac60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fa59c640 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fa59c640 1 -- 172.21.15.1:0/2183944030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570abac6690 con 0x5570abac15d0 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fad9d640 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 0x5570abac5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2fa59c640 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abad49e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec032dc0 tx=0x7fc2ec009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e176d176f9a58e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2183944030 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec035040 con 0x5570abac15d0 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2183944030 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc2ec005040 con 0x5570abac15d0 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.061+0000 7fc303163740 1 -- 172.21.15.1:0/2183944030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f400fd20 con 0x5570abac15d0 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2183944030 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec005240 con 0x5570abac15d0 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc303163740 10 client.16536 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc303163740 1 -- 172.21.15.1:0/2183944030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570abb467e0 con 0x5570abac15d0 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2183944030 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f400fd20 con 0x5570abac15d0 2024-09-16T00:33:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2183944030 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc2ec031630 con 0x5570abac15d0 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc303163740 10 client.16536 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc303163740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc303163740 20 client.16536 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc303163740 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5570abb46f90 0x5570abb493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc303163740 4 client.16536 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc303163740 1 -- 172.21.15.1:0/2183944030 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570aba2ed70 con 0x5570abb46f90 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.062+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5570abb46f90 0x5570abb493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc2f3fff640 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5570abb46f90 0x5570abb493b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe8b50b0ded1076 server_cookie=9e13bbb34ac38a14 in_seq=0 out_seq=0 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc2e1ffb640 10 client.16536 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc2e1ffb640 1 -- 172.21.15.1:0/2183944030 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570aba2ed70 con 0x5570abb46f90 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc2e1ffb640 10 client.16536 handle_command_reply: tid=0 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 client.16536 shutdown 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 -- 172.21.15.1:0/2183944030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 msgr2=0x5570abad49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abad49e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec032dc0 tx=0x7fc2ec009680 comp rx=0 tx=0).stop 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 -- 172.21.15.1:0/2183944030 shutdown_connections 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5570abb46f90 0x5570abb493b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570abacc370 0x5570abac60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570abac2b40 0x5570abac5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 --2- 172.21.15.1:0/2183944030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570abac15d0 0x5570abad49e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 -- 172.21.15.1:0/2183944030 >> 172.21.15.1:0/2183944030 conn(0x5570aba8a310 msgr2=0x5570ababc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc2e1ffb640 0 client.16536 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 -- 172.21.15.1:0/2183944030 shutdown_connections 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 1 -- 172.21.15.1:0/2183944030 wait complete. 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.063+0000 7fc303163740 20 client.16536 trim_cache size 0 max 16384 2024-09-16T00:33:22.073 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:33:22.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 -- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2e80a3240 msgr2=0x7fc2e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2e80a3240 0x7fc2e80a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc2f401e000 tx=0x7fc2f4021000 comp rx=0 tx=0).stop 2024-09-16T00:33:22.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 -- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc102790 msgr2=0x7fc2fc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc102790 0x7fc2fc074a30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c920 tx=0x7fc2e400cde0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 -- 172.21.15.1:0/1907264099 shutdown_connections 2024-09-16T00:33:22.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2fc106f10 0x7fc2fc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2fc102790 0x7fc2fc074a30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2e80a3240 0x7fc2e80a5700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 --2- 172.21.15.1:0/1907264099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2fc101de0 0x7fc2fc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 -- 172.21.15.1:0/1907264099 >> 172.21.15.1:0/1907264099 conn(0x7fc2fc0fd8c0 msgr2=0x7fc2fc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 -- 172.21.15.1:0/1907264099 shutdown_connections 2024-09-16T00:33:22.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:22.064+0000 7fc300d8a640 1 -- 172.21.15.1:0/1907264099 wait complete. 2024-09-16T00:33:22.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:22 smithi019 ceph-mon[35186]: pgmap v499: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:33:22.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:22 smithi001 ceph-mon[30934]: pgmap v499: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:33:22.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:22 smithi153 ceph-mon[35538]: pgmap v499: 129 pgs: 129 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:33:23.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:23.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:23.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:23.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:33:23.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:23.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:23.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:23.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:33:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:33:23.953 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:33:23.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:33:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:24.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:24 smithi001 ceph-mon[30934]: pgmap v500: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-09-16T00:33:24.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:24 smithi153 ceph-mon[35538]: pgmap v500: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-09-16T00:33:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.755+0000 7f622e6d9640 1 -- 172.21.15.1:0/1877340053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62281037a0 msgr2=0x7f622810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.755+0000 7f622e6d9640 1 --2- 172.21.15.1:0/1877340053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62281037a0 0x7f622810bc00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6210009900 tx=0x7f621002a410 comp rx=0 tx=0).stop 2024-09-16T00:33:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.755+0000 7f622e6d9640 1 -- 172.21.15.1:0/1877340053 shutdown_connections 2024-09-16T00:33:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.755+0000 7f622e6d9640 1 --2- 172.21.15.1:0/1877340053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f622810c2f0 0x7f622810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.755+0000 7f622e6d9640 1 --2- 172.21.15.1:0/1877340053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62281037a0 0x7f622810bc00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.755+0000 7f622e6d9640 1 --2- 172.21.15.1:0/1877340053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6228102df0 0x7f62281031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.755+0000 7f622e6d9640 1 -- 172.21.15.1:0/1877340053 >> 172.21.15.1:0/1877340053 conn(0x7f62280fc7d0 msgr2=0x7f62280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.755+0000 7f622e6d9640 1 -- 172.21.15.1:0/1877340053 shutdown_connections 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.755+0000 7f622e6d9640 1 -- 172.21.15.1:0/1877340053 wait complete. 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622e6d9640 1 Processor -- start 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622e6d9640 1 -- start start 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622e6d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6228102df0 0x7f6228100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622e6d9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62281037a0 0x7f6228100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622e6d9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f622810c2f0 0x7f62281014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622e6d9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62281120d0 con 0x7f622810c2f0 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622e6d9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6228111f50 con 0x7f6228102df0 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622e6d9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6228112250 con 0x7f62281037a0 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622cc76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f622810c2f0 0x7f62281014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622cc76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f622810c2f0 0x7f62281014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57626/0 (socket says 172.21.15.1:57626) 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622cc76640 1 -- 172.21.15.1:0/2374032676 learned_addr learned my addr 172.21.15.1:0/2374032676 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f6227fff640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6228102df0 0x7f6228100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f621ffff640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62281037a0 0x7f6228100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622cc76640 1 -- 172.21.15.1:0/2374032676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62281037a0 msgr2=0x7f6228100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622cc76640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62281037a0 0x7f6228100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622cc76640 1 -- 172.21.15.1:0/2374032676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6228102df0 msgr2=0x7f6228100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622cc76640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6228102df0 0x7f6228100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f622cc76640 1 -- 172.21.15.1:0/2374032676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62281a6a90 con 0x7f622810c2f0 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.756+0000 7f6227fff640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6228102df0 0x7f6228100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.757+0000 7f622cc76640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f622810c2f0 0x7f62281014d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f622000cce0 tx=0x7f622000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.757+0000 7f6225ffb640 1 -- 172.21.15.1:0/2374032676 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622000f450 con 0x7f622810c2f0 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.757+0000 7f6225ffb640 1 -- 172.21.15.1:0/2374032676 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f622000fad0 con 0x7f622810c2f0 2024-09-16T00:33:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.757+0000 7f622e6d9640 1 -- 172.21.15.1:0/2374032676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62281a6d80 con 0x7f622810c2f0 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.757+0000 7f6225ffb640 1 -- 172.21.15.1:0/2374032676 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220018c10 con 0x7f622810c2f0 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.757+0000 7f622e6d9640 1 -- 172.21.15.1:0/2374032676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62281a7240 con 0x7f622810c2f0 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.759+0000 7f6225ffb640 1 -- 172.21.15.1:0/2374032676 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f622000f5f0 con 0x7f622810c2f0 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6230ab2740 1 Processor -- start 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6230ab2740 1 -- start start 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 0x55f036b94f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6225ffb640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62140a3240 0x7f62140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036ba2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6230ab2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f036afa280 con 0x55f036b95480 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6230ab2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f036aa1b10 con 0x55f036b9fd40 2024-09-16T00:33:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6230ab2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f036af6240 con 0x55f036b94b60 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6227fff640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62140a3240 0x7f62140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f621ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f621ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57628/0 (socket says 172.21.15.1:57628) 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f6225ffb640 1 -- 172.21.15.1:0/2374032676 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f62200d04c0 con 0x7f622810c2f0 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f621ffff640 1 -- 172.21.15.1:0/2047033137 learned_addr learned my addr 172.21.15.1:0/2047033137 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f621ffff640 1 --2- 172.21.15.1:0/2047033137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 0x55f036b94f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.760+0000 7f622cc76640 1 --2- 172.21.15.1:0/2047033137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036ba2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f621ffff640 1 -- 172.21.15.1:0/2047033137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 msgr2=0x55f036b94f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f621ffff640 1 --2- 172.21.15.1:0/2047033137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 0x55f036b94f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f621ffff640 1 -- 172.21.15.1:0/2047033137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 msgr2=0x55f036ba2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f621ffff640 1 --2- 172.21.15.1:0/2047033137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036ba2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f621ffff640 1 -- 172.21.15.1:0/2047033137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f036aae1f0 con 0x55f036b95480 2024-09-16T00:33:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f6227fff640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62140a3240 0x7f62140a5700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f62180058c0 tx=0x7f6218005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f621ffff640 1 --2- 172.21.15.1:0/2047033137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9f800 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6210000c00 tx=0x7f621002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505449e9da3174a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f621e7fc640 1 -- 172.21.15.1:0/2047033137 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210035040 con 0x55f036b95480 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f621e7fc640 1 -- 172.21.15.1:0/2047033137 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62100035f0 con 0x55f036b95480 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.761+0000 7f621e7fc640 1 -- 172.21.15.1:0/2047033137 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621003d6e0 con 0x55f036b95480 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 -- 172.21.15.1:0/2047033137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 msgr2=0x55f036b9f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2047033137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9f800 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6210000c00 tx=0x7f621002a410 comp rx=0 tx=0).stop 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 -- 172.21.15.1:0/2047033137 shutdown_connections 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2047033137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036ba2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2047033137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9f800 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2047033137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 0x55f036b94f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 -- 172.21.15.1:0/2047033137 >> 172.21.15.1:0/2047033137 conn(0x55f036b910f0 msgr2=0x55f036b914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 -- 172.21.15.1:0/2047033137 shutdown_connections 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 -- 172.21.15.1:0/2047033137 wait complete. 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 -- start start 2024-09-16T00:33:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.762+0000 7f6230ab2740 1 Processor -- start 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b94b60 0x55f036b9cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b9fd40 0x55f036b9d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f6230ab2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f036ba2910 con 0x55f036b95480 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f6230ab2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f036ab4f10 con 0x55f036b94b60 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f6230ab2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f036ba2a90 con 0x55f036b9fd40 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f6230ab2740 10 client.0 _ll_register_callbacks cb 0x55f036a7da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f621ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f621ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57634/0 (socket says 172.21.15.1:57634) 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f621ffff640 1 -- 172.21.15.1:0/962760051 learned_addr learned my addr 172.21.15.1:0/962760051 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f6227fff640 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b94b60 0x55f036b9cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f622cc76640 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b9fd40 0x55f036b9d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f621ffff640 1 -- 172.21.15.1:0/962760051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b9fd40 msgr2=0x55f036b9d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f621ffff640 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b9fd40 0x55f036b9d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f621ffff640 1 -- 172.21.15.1:0/962760051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b94b60 msgr2=0x55f036b9cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f621ffff640 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b94b60 0x55f036b9cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f621ffff640 1 -- 172.21.15.1:0/962760051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03686b6a0 con 0x55f036b95480 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f621ffff640 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9d1b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f62100057c0 tx=0x7f6210004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25eb75caf35797e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f62077fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f62077fe640 1 -- 172.21.15.1:0/962760051 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621004d020 con 0x55f036b95480 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f62077fe640 1 -- 172.21.15.1:0/962760051 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f62100039b0 con 0x55f036b95480 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.763+0000 7f6230ab2740 1 -- 172.21.15.1:0/962760051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62100035f0 con 0x55f036b95480 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f62077fe640 1 -- 172.21.15.1:0/962760051 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6210058050 con 0x55f036b95480 2024-09-16T00:33:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f6230ab2740 10 client.16548 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f6230ab2740 1 -- 172.21.15.1:0/962760051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f036bae550 con 0x55f036b95480 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f62077fe640 1 -- 172.21.15.1:0/962760051 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62100035f0 con 0x55f036b95480 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f62077fe640 1 -- 172.21.15.1:0/962760051 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f621003a040 con 0x55f036b95480 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f6230ab2740 10 client.16548 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f6230ab2740 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f6230ab2740 20 client.16548 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f6230ab2740 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f036baf8a0 0x55f036bb1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f6230ab2740 4 client.16548 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f6230ab2740 1 -- 172.21.15.1:0/962760051 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f036b077f0 con 0x55f036baf8a0 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.764+0000 7f6227fff640 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f036baf8a0 0x55f036bb1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.765+0000 7f6227fff640 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f036baf8a0 0x55f036bb1c90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3311e402eada298d server_cookie=7f7405d2b68672f9 in_seq=0 out_seq=0 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.765+0000 7f62077fe640 10 client.16548 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.765+0000 7f62077fe640 1 -- 172.21.15.1:0/962760051 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f036b077f0 con 0x55f036baf8a0 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.765+0000 7f62077fe640 10 client.16548 handle_command_reply: tid=0 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 client.16548 shutdown 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 -- 172.21.15.1:0/962760051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 msgr2=0x55f036b9d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9d1b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f62100057c0 tx=0x7f6210004290 comp rx=0 tx=0).stop 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 -- 172.21.15.1:0/962760051 shutdown_connections 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f036baf8a0 0x55f036bb1c90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b9fd40 0x55f036b9d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036b9d1b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 --2- 172.21.15.1:0/962760051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b94b60 0x55f036b9cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f62077fe640 0 client.16548 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 -- 172.21.15.1:0/962760051 >> 172.21.15.1:0/962760051 conn(0x55f036b910f0 msgr2=0x55f036b96e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 -- 172.21.15.1:0/962760051 shutdown_connections 2024-09-16T00:33:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 1 -- 172.21.15.1:0/962760051 wait complete. 2024-09-16T00:33:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.766+0000 7f6230ab2740 20 client.16548 trim_cache size 0 max 16384 2024-09-16T00:33:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6230ab2740 1 Processor -- start 2024-09-16T00:33:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6230ab2740 1 -- start start 2024-09-16T00:33:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b94b60 0x55f036bab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b95480 0x55f036bb4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036bb51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6230ab2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62281b38d0 con 0x55f036b94b60 2024-09-16T00:33:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6230ab2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62281b11a0 con 0x55f036b9fd40 2024-09-16T00:33:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6230ab2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62281aeff0 con 0x55f036b95480 2024-09-16T00:33:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6227fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b94b60 0x55f036bab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6227fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b94b60 0x55f036bab3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57650/0 (socket says 172.21.15.1:57650) 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6227fff640 1 -- 172.21.15.1:0/4292239336 learned_addr learned my addr 172.21.15.1:0/4292239336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f622cc76640 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036bb51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f621ffff640 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b95480 0x55f036bb4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6227fff640 1 -- 172.21.15.1:0/4292239336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b95480 msgr2=0x55f036bb4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6227fff640 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b95480 0x55f036bb4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6227fff640 1 -- 172.21.15.1:0/4292239336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 msgr2=0x55f036bb51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6227fff640 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036bb51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f6227fff640 1 -- 172.21.15.1:0/4292239336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f036b9bc20 con 0x55f036b94b60 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.784+0000 7f622cc76640 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036bb51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f6227fff640 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b94b60 0x55f036bab3f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55f036ba2b00 tx=0x7f6218015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356c31204e6a8efe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f621dffb640 1 -- 172.21.15.1:0/4292239336 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218011440 con 0x55f036b94b60 2024-09-16T00:33:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f621dffb640 1 -- 172.21.15.1:0/4292239336 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6218011ac0 con 0x55f036b94b60 2024-09-16T00:33:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f621dffb640 1 -- 172.21.15.1:0/4292239336 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62180268c0 con 0x55f036b94b60 2024-09-16T00:33:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f6230ab2740 1 -- 172.21.15.1:0/4292239336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b94b60 msgr2=0x55f036bab3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f6230ab2740 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b94b60 0x55f036bab3f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55f036ba2b00 tx=0x7f6218015000 comp rx=0 tx=0).stop 2024-09-16T00:33:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f6230ab2740 1 -- 172.21.15.1:0/4292239336 shutdown_connections 2024-09-16T00:33:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f6230ab2740 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036bb51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f6230ab2740 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b95480 0x55f036bb4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.785+0000 7f6230ab2740 1 --2- 172.21.15.1:0/4292239336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b94b60 0x55f036bab3f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 -- 172.21.15.1:0/4292239336 >> 172.21.15.1:0/4292239336 conn(0x55f036b910f0 msgr2=0x55f036baace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 -- 172.21.15.1:0/4292239336 shutdown_connections 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 -- 172.21.15.1:0/4292239336 wait complete. 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 -- start start 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 Processor -- start 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 0x55f036bb2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036bb3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036c191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62281af660 con 0x55f036b95480 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62281b0ad0 con 0x55f036b9fd40 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62281afd30 con 0x55f036b94b60 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f6230ab2740 10 client.0 _ll_register_callbacks cb 0x7f62200077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.786+0000 7f621ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036bb3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f621ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036bb3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57662/0 (socket says 172.21.15.1:57662) 2024-09-16T00:33:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f621ffff640 1 -- 172.21.15.1:0/2324426958 learned_addr learned my addr 172.21.15.1:0/2324426958 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f622cc76640 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036c191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f6227fff640 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 0x55f036bb2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f621ffff640 1 -- 172.21.15.1:0/2324426958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 msgr2=0x55f036bb2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f621ffff640 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 0x55f036bb2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f621ffff640 1 -- 172.21.15.1:0/2324426958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 msgr2=0x55f036c191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f621ffff640 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036c191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f621ffff640 1 -- 172.21.15.1:0/2324426958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f036c19720 con 0x55f036b95480 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f622cc76640 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036c191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f6227fff640 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 0x55f036bb2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f621ffff640 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036bb3370 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f6210004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5948f8ec295264 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f62077fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f62077fe640 1 -- 172.21.15.1:0/2324426958 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621004d050 con 0x55f036b95480 2024-09-16T00:33:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f6230ab2740 1 -- 172.21.15.1:0/2324426958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6218011ac0 con 0x55f036b95480 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f62077fe640 1 -- 172.21.15.1:0/2324426958 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6210049620 con 0x55f036b95480 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f62077fe640 1 -- 172.21.15.1:0/2324426958 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6210068050 con 0x55f036b95480 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f6230ab2740 10 client.16554 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.787+0000 7f6230ab2740 1 -- 172.21.15.1:0/2324426958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f036c1a900 con 0x55f036b95480 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f62077fe640 1 -- 172.21.15.1:0/2324426958 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6218011ac0 con 0x55f036b95480 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f62077fe640 1 -- 172.21.15.1:0/2324426958 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6210049a20 con 0x55f036b95480 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f6230ab2740 10 client.16554 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f6230ab2740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f6230ab2740 20 client.16554 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f036c1af90 0x55f036c1d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f6230ab2740 4 client.16554 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f6230ab2740 1 -- 172.21.15.1:0/2324426958 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f036b01d00 con 0x55f036c1af90 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f6227fff640 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f036c1af90 0x55f036c1d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f6227fff640 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f036c1af90 0x55f036c1d3b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ef9f9dedf26222 server_cookie=5930cc53c0ab6e5e in_seq=0 out_seq=0 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.788+0000 7f62077fe640 10 client.16554 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.789+0000 7f62077fe640 1 -- 172.21.15.1:0/2324426958 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f036b01d00 con 0x55f036c1af90 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.789+0000 7f62077fe640 10 client.16554 handle_command_reply: tid=0 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.789+0000 7f6230ab2740 1 client.16554 shutdown 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 -- 172.21.15.1:0/2324426958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 msgr2=0x55f036bb3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036bb3370 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f6210004530 comp rx=0 tx=0).stop 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 -- 172.21.15.1:0/2324426958 shutdown_connections 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f036c1af90 0x55f036c1d3b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f036b9fd40 0x55f036c191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f62077fe640 0 client.16554 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f036b95480 0x55f036bb3370 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 --2- 172.21.15.1:0/2324426958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f036b94b60 0x55f036bb2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 -- 172.21.15.1:0/2324426958 >> 172.21.15.1:0/2324426958 conn(0x55f036b910f0 msgr2=0x55f036bab500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:24.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 -- 172.21.15.1:0/2324426958 shutdown_connections 2024-09-16T00:33:24.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 1 -- 172.21.15.1:0/2324426958 wait complete. 2024-09-16T00:33:24.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f6230ab2740 20 client.16554 trim_cache size 0 max 16384 2024-09-16T00:33:24.795 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:33:24.795 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:33:24.795 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "cf77f3fd-068a-4a02-9c74-623be8326cb0", 2024-09-16T00:33:24.795 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:33:24.795 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:33:24.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f622e6d9640 1 -- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62140a3240 msgr2=0x7f62140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f622e6d9640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62140a3240 0x7f62140a5700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f62180058c0 tx=0x7f6218005850 comp rx=0 tx=0).stop 2024-09-16T00:33:24.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f622e6d9640 1 -- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f622810c2f0 msgr2=0x7f62281014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.790+0000 7f622e6d9640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f622810c2f0 0x7f62281014d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f622000cce0 tx=0x7f622000a620 comp rx=0 tx=0).stop 2024-09-16T00:33:24.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.791+0000 7f622e6d9640 1 -- 172.21.15.1:0/2374032676 shutdown_connections 2024-09-16T00:33:24.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.791+0000 7f622e6d9640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f622810c2f0 0x7f62281014d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.791+0000 7f622e6d9640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f62140a3240 0x7f62140a5700 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.791+0000 7f622e6d9640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62281037a0 0x7f6228100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.791+0000 7f622e6d9640 1 --2- 172.21.15.1:0/2374032676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6228102df0 0x7f6228100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.791+0000 7f622e6d9640 1 -- 172.21.15.1:0/2374032676 >> 172.21.15.1:0/2374032676 conn(0x7f62280fc7d0 msgr2=0x7f62280fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:24.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.791+0000 7f622e6d9640 1 -- 172.21.15.1:0/2374032676 shutdown_connections 2024-09-16T00:33:24.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:24.791+0000 7f622e6d9640 1 -- 172.21.15.1:0/2374032676 wait complete. 2024-09-16T00:33:24.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:24 smithi019 ceph-mon[35186]: pgmap v500: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-09-16T00:33:25.604 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:33:25.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:25 smithi001 ceph-mon[30934]: pgmap v501: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-09-16T00:33:25.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:25 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:33:25.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:25 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:25.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:25 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:33:25.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:25 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:25.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:25 smithi153 ceph-mon[35538]: pgmap v501: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-09-16T00:33:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:25 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:33:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:25 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:25 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:33:25.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:25 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:25.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:25 smithi019 ceph-mon[35186]: pgmap v501: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-09-16T00:33:25.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:25 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:33:25.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:25 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:25.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:25 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:33:25.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:25 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:26.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.355+0000 7f5fd988d640 1 -- 172.21.15.1:0/3263195580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 msgr2=0x7f5fd410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.355+0000 7f5fd988d640 1 --2- 172.21.15.1:0/3263195580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 0x7f5fd410bb50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009900 tx=0x7f5fbc02a480 comp rx=0 tx=0).stop 2024-09-16T00:33:26.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.355+0000 7f5fd988d640 1 -- 172.21.15.1:0/3263195580 shutdown_connections 2024-09-16T00:33:26.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.355+0000 7f5fd988d640 1 --2- 172.21.15.1:0/3263195580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fd410c170 0x7f5fd410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.355+0000 7f5fd988d640 1 --2- 172.21.15.1:0/3263195580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 0x7f5fd410bb50 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.355+0000 7f5fd988d640 1 --2- 172.21.15.1:0/3263195580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fd4102d40 0x7f5fd4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.355+0000 7f5fd988d640 1 -- 172.21.15.1:0/3263195580 >> 172.21.15.1:0/3263195580 conn(0x7f5fd40fc7a0 msgr2=0x7f5fd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.355+0000 7f5fd988d640 1 -- 172.21.15.1:0/3263195580 shutdown_connections 2024-09-16T00:33:26.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.355+0000 7f5fd988d640 1 -- 172.21.15.1:0/3263195580 wait complete. 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd988d640 1 Processor -- start 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd988d640 1 -- start start 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd988d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fd4102d40 0x7f5fd41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd988d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 0x7f5fd4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd988d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fd410c170 0x7f5fd4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd988d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd4111f70 con 0x7f5fd4103660 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd988d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd4111df0 con 0x7f5fd410c170 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd988d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41120f0 con 0x7f5fd4102d40 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 0x7f5fd4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 0x7f5fd4100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57680/0 (socket says 172.21.15.1:57680) 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fcbfff640 1 -- 172.21.15.1:0/64560543 learned_addr learned my addr 172.21.15.1:0/64560543 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fd4102d40 0x7f5fd41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.356+0000 7f5fd37fe640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fd410c170 0x7f5fd4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fcbfff640 1 -- 172.21.15.1:0/64560543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fd4102d40 msgr2=0x7f5fd41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fd4102d40 0x7f5fd41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fcbfff640 1 -- 172.21.15.1:0/64560543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fd410c170 msgr2=0x7f5fd4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fd410c170 0x7f5fd4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fcbfff640 1 -- 172.21.15.1:0/64560543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd41a6830 con 0x7f5fd4103660 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fd4102d40 0x7f5fd41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 0x7f5fd4100ee0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc005ab0 tx=0x7f5fbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fd888b640 1 -- 172.21.15.1:0/64560543 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc03f030 con 0x7f5fd4103660 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fd888b640 1 -- 172.21.15.1:0/64560543 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc003ce0 con 0x7f5fd4103660 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fd888b640 1 -- 172.21.15.1:0/64560543 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc035070 con 0x7f5fd4103660 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fd988d640 1 -- 172.21.15.1:0/64560543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd41a6ac0 con 0x7f5fd4103660 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.357+0000 7f5fd988d640 1 -- 172.21.15.1:0/64560543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd41a6fa0 con 0x7f5fd4103660 2024-09-16T00:33:26.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fd888b640 1 -- 172.21.15.1:0/64560543 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f5fbc042030 con 0x7f5fd4103660 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fdbc66740 1 Processor -- start 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fdbc66740 1 -- start start 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf0804a0 0x5566bf080880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf08b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf08b680 0x5566bf08da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5566befe62e0 con 0x5566bf080dc0 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5566bef8dce0 con 0x5566bf08b680 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5566befe22a0 con 0x5566bf0804a0 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fd2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf08b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fd2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf08b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57688/0 (socket says 172.21.15.1:57688) 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.360+0000 7f5fd2ffd640 1 -- 172.21.15.1:0/480368188 learned_addr learned my addr 172.21.15.1:0/480368188 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd888b640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5fc00a3240 0x7f5fc00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf08b680 0x5566bf08da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd37fe640 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf0804a0 0x5566bf080880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd2ffd640 1 -- 172.21.15.1:0/480368188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf0804a0 msgr2=0x5566bf080880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf0804a0 0x5566bf080880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd2ffd640 1 -- 172.21.15.1:0/480368188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf08b680 msgr2=0x5566bf08da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf08b680 0x5566bf08da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd2ffd640 1 -- 172.21.15.1:0/480368188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bef72440 con 0x5566bf080dc0 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd37fe640 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf0804a0 0x5566bf080880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd37fe640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5fc00a3240 0x7f5fc00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd888b640 1 -- 172.21.15.1:0/64560543 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5fbc0f0620 con 0x7f5fd4103660 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf08b140 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4008c60 tx=0x7f5fc400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af73b4fd6d12caf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fc9ffb640 1 -- 172.21.15.1:0/480368188 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4014070 con 0x5566bf080dc0 2024-09-16T00:33:26.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fc9ffb640 1 -- 172.21.15.1:0/480368188 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc40047a0 con 0x5566bf080dc0 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.361+0000 7f5fc9ffb640 1 -- 172.21.15.1:0/480368188 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4010760 con 0x5566bf080dc0 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 -- 172.21.15.1:0/480368188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 msgr2=0x5566bf08b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fd37fe640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5fc00a3240 0x7f5fc00a5700 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc005cc0 tx=0x7f5fcc00b470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf08b140 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4008c60 tx=0x7f5fc400e720 comp rx=0 tx=0).stop 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 -- 172.21.15.1:0/480368188 shutdown_connections 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf08b680 0x5566bf08da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf08b140 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/480368188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf0804a0 0x5566bf080880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 -- 172.21.15.1:0/480368188 >> 172.21.15.1:0/480368188 conn(0x5566bf07ca30 msgr2=0x5566bf07ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 -- 172.21.15.1:0/480368188 shutdown_connections 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 -- 172.21.15.1:0/480368188 wait complete. 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.362+0000 7f5fdbc66740 1 -- start start 2024-09-16T00:33:26.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fdbc66740 1 Processor -- start 2024-09-16T00:33:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf08af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf088270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf08b680 0x5566bf0887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5566bf08e250 con 0x5566bf080dc0 2024-09-16T00:33:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5566befa1050 con 0x5566bf0804a0 2024-09-16T00:33:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5566bf08e3d0 con 0x5566bf08b680 2024-09-16T00:33:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fdbc66740 10 client.0 _ll_register_callbacks cb 0x5566befe1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf088270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf088270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57704/0 (socket says 172.21.15.1:57704) 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fcbfff640 1 -- 172.21.15.1:0/4001079499 learned_addr learned my addr 172.21.15.1:0/4001079499 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf08af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fd37fe640 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf08b680 0x5566bf0887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fcbfff640 1 -- 172.21.15.1:0/4001079499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf08b680 msgr2=0x5566bf0887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf08b680 0x5566bf0887b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fcbfff640 1 -- 172.21.15.1:0/4001079499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 msgr2=0x5566bf08af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf08af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fcbfff640 1 -- 172.21.15.1:0/4001079499 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bed573c0 con 0x5566bf080dc0 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf08af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf088270 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc000c00 tx=0x7f5fbc009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b5da31933abe95 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.363+0000 7f5fb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/4001079499 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc049070 con 0x5566bf080dc0 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/4001079499 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5fbc03bd00 con 0x5566bf080dc0 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fdbc66740 1 -- 172.21.15.1:0/4001079499 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc40047a0 con 0x5566bf080dc0 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/4001079499 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5fbc0044e0 con 0x5566bf080dc0 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fdbc66740 10 client.16566 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fdbc66740 1 -- 172.21.15.1:0/4001079499 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566bf099ba0 con 0x5566bf080dc0 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/4001079499 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc40047a0 con 0x5566bf080dc0 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/4001079499 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5fbc0b8eb0 con 0x5566bf080dc0 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fdbc66740 10 client.16566 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fdbc66740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fdbc66740 20 client.16566 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5566bf09afb0 0x5566bf09d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fdbc66740 4 client.16566 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.364+0000 7f5fdbc66740 1 -- 172.21.15.1:0/4001079499 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566beff3850 con 0x5566bf09afb0 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.365+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5566bf09afb0 0x5566bf09d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.365+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5566bf09afb0 0x5566bf09d3e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af567c5bf0a5c7a3 server_cookie=3885925a30815af0 in_seq=0 out_seq=0 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.365+0000 7f5fb2ffd640 10 client.16566 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/4001079499 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566beff3850 con 0x5566bf09afb0 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fb2ffd640 10 client.16566 handle_command_reply: tid=0 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 client.16566 shutdown 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 -- 172.21.15.1:0/4001079499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 msgr2=0x5566bf088270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf088270 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc000c00 tx=0x7f5fbc009300 comp rx=0 tx=0).stop 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 -- 172.21.15.1:0/4001079499 shutdown_connections 2024-09-16T00:33:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5566bf09afb0 0x5566bf09d3e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf08b680 0x5566bf0887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf088270 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fb2ffd640 0 client.16566 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/4001079499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf08af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 -- 172.21.15.1:0/4001079499 >> 172.21.15.1:0/4001079499 conn(0x5566bf07ca30 msgr2=0x5566bf07bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 -- 172.21.15.1:0/4001079499 shutdown_connections 2024-09-16T00:33:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 1 -- 172.21.15.1:0/4001079499 wait complete. 2024-09-16T00:33:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.366+0000 7f5fdbc66740 20 client.16566 trim_cache size 0 max 16384 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fdbc66740 1 Processor -- start 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fdbc66740 1 -- start start 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf0a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf08b680 0x5566bf0854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41b37d0 con 0x5566bf080dc0 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41b10a0 con 0x5566bf0804a0 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41aeef0 con 0x5566bf08b680 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf0a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf0a3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57708/0 (socket says 172.21.15.1:57708) 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.384+0000 7f5fcbfff640 1 -- 172.21.15.1:0/2051897517 learned_addr learned my addr 172.21.15.1:0/2051897517 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/2051897517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fcbfff640 1 -- 172.21.15.1:0/2051897517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf08b680 msgr2=0x5566bf0854d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/2051897517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf08b680 0x5566bf0854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fcbfff640 1 -- 172.21.15.1:0/2051897517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 msgr2=0x5566bf0a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/2051897517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf0a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fcbfff640 1 -- 172.21.15.1:0/2051897517 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bf08ab70 con 0x5566bf080dc0 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/2051897517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf0a3230 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0b9210 tx=0x7f5fbc0bad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3110b5543b9f1be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fc97fa640 1 -- 172.21.15.1:0/2051897517 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc049070 con 0x5566bf080dc0 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fc97fa640 1 -- 172.21.15.1:0/2051897517 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc0bcf40 con 0x5566bf080dc0 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.385+0000 7f5fc97fa640 1 -- 172.21.15.1:0/2051897517 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0bc610 con 0x5566bf080dc0 2024-09-16T00:33:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2051897517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 msgr2=0x5566bf0a3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2051897517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf0a3230 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0b9210 tx=0x7f5fbc0bad50 comp rx=0 tx=0).stop 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2051897517 shutdown_connections 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2051897517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf08b680 0x5566bf0854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2051897517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf080dc0 0x5566bf0a3230 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2051897517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf0a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2051897517 >> 172.21.15.1:0/2051897517 conn(0x5566bf07ca30 msgr2=0x5566bf08a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2051897517 shutdown_connections 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2051897517 wait complete. 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 -- start start 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.386+0000 7f5fdbc66740 1 Processor -- start 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf091770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf080dc0 0x5566bf091cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fdbc66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf08b680 0x5566bf104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41af560 con 0x5566bf08b680 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41b09d0 con 0x5566bf0804a0 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fdbc66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41afc30 con 0x5566bf080dc0 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fdbc66740 10 client.0 _ll_register_callbacks cb 0x7f5fbc005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf08b680 0x5566bf104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf08b680 0x5566bf104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57710/0 (socket says 172.21.15.1:57710) 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf091770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd37fe640 1 -- 172.21.15.1:0/2383478869 learned_addr learned my addr 172.21.15.1:0/2383478869 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fcbfff640 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf080dc0 0x5566bf091cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd37fe640 1 -- 172.21.15.1:0/2383478869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf080dc0 msgr2=0x5566bf091cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd37fe640 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf080dc0 0x5566bf091cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd37fe640 1 -- 172.21.15.1:0/2383478869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 msgr2=0x5566bf091770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd37fe640 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf091770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd37fe640 1 -- 172.21.15.1:0/2383478869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bf0925b0 con 0x5566bf08b680 2024-09-16T00:33:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fd37fe640 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf08b680 0x5566bf104cd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00b330 tx=0x7f5fcc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e44b0ad66443b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.387+0000 7f5fb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/2383478869 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc01cde0 con 0x5566bf08b680 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/2383478869 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5fcc01a470 con 0x5566bf08b680 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2383478869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fbc0bcf40 con 0x5566bf08b680 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/2383478869 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5fcc02e050 con 0x5566bf08b680 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fdbc66740 10 client.16572 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2383478869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566bf1063b0 con 0x5566bf08b680 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/2383478869 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc0bcf40 con 0x5566bf08b680 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/2383478869 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5fcc023020 con 0x5566bf08b680 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fdbc66740 10 client.16572 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fdbc66740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fdbc66740 20 client.16572 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:26.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5566bf106ae0 0x5566bf108f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fdbc66740 4 client.16572 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.388+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2383478869 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566befedd60 con 0x5566bf106ae0 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5566bf106ae0 0x5566bf108f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fd2ffd640 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5566bf106ae0 0x5566bf108f00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb16a136290c4a5a server_cookie=41ac529caa1740b3 in_seq=0 out_seq=0 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fb2ffd640 10 client.16572 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fb2ffd640 1 -- 172.21.15.1:0/2383478869 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5566befedd60 con 0x5566bf106ae0 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fb2ffd640 10 client.16572 handle_command_reply: tid=0 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fdbc66740 1 client.16572 shutdown 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2383478869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf08b680 msgr2=0x5566bf104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf08b680 0x5566bf104cd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00b330 tx=0x7f5fcc004ab0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2383478869 shutdown_connections 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5566bf106ae0 0x5566bf108f00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566bf08b680 0x5566bf104cd0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566bf080dc0 0x5566bf091cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fb2ffd640 0 client.16572 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fdbc66740 1 --2- 172.21.15.1:0/2383478869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566bf0804a0 0x5566bf091770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.389+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2383478869 >> 172.21.15.1:0/2383478869 conn(0x5566bf07ca30 msgr2=0x5566bf07b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2383478869 shutdown_connections 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fdbc66740 1 -- 172.21.15.1:0/2383478869 wait complete. 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fdbc66740 20 client.16572 trim_cache size 0 max 16384 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "cf77f3fd-068a-4a02-9c74-623be8326cb0": { 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:33:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "cf77f3fd-068a-4a02-9c74-623be8326cb0", 2024-09-16T00:33:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:33:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:26.403 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:33:26.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 -- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5fc00a3240 msgr2=0x7f5fc00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5fc00a3240 0x7f5fc00a5700 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc005cc0 tx=0x7f5fcc00b470 comp rx=0 tx=0).stop 2024-09-16T00:33:26.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 -- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 msgr2=0x7f5fd4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 0x7f5fd4100ee0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc005ab0 tx=0x7f5fbc0042f0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 -- 172.21.15.1:0/64560543 shutdown_connections 2024-09-16T00:33:26.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fd410c170 0x7f5fd4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd4103660 0x7f5fd4100ee0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5fc00a3240 0x7f5fc00a5700 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 --2- 172.21.15.1:0/64560543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fd4102d40 0x7f5fd41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 -- 172.21.15.1:0/64560543 >> 172.21.15.1:0/64560543 conn(0x7f5fd40fc7a0 msgr2=0x7f5fd40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 -- 172.21.15.1:0/64560543 shutdown_connections 2024-09-16T00:33:26.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:26.390+0000 7f5fd988d640 1 -- 172.21.15.1:0/64560543 wait complete. 2024-09-16T00:33:27.157 INFO:tasks.cephfs.filesystem:scrub status for tag:cf77f3fd-068a-4a02-9c74-623be8326cb0 - {'path': '/', 'tag': 'cf77f3fd-068a-4a02-9c74-623be8326cb0', 'options': 'recursive,force'} 2024-09-16T00:33:27.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:27 smithi019 ceph-mon[35186]: pgmap v502: 129 pgs: 129 active+clean; 2.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 KiB/s wr, 9 op/s 2024-09-16T00:33:27.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:27 smithi001 ceph-mon[30934]: pgmap v502: 129 pgs: 129 active+clean; 2.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 KiB/s wr, 9 op/s 2024-09-16T00:33:27.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:27 smithi153 ceph-mon[35538]: pgmap v502: 129 pgs: 129 active+clean; 2.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 KiB/s wr, 9 op/s 2024-09-16T00:33:28.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:28 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:33:28.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:28 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:33:28.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:28 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:33:29.742 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:29 smithi001 ceph-mon[30934]: pgmap v503: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:33:29.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:29 smithi019 ceph-mon[35186]: pgmap v503: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:33:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:29 smithi153 ceph-mon[35538]: pgmap v503: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:33:31.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:31 smithi019 ceph-mon[35186]: pgmap v504: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:33:31.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:31 smithi001 ceph-mon[30934]: pgmap v504: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:33:31.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:31 smithi153 ceph-mon[35538]: pgmap v504: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:33:33.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:33 smithi019 ceph-mon[35186]: pgmap v505: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:33:33.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:33 smithi001 ceph-mon[30934]: pgmap v505: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:33:33.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:33 smithi153 ceph-mon[35538]: pgmap v505: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:33:35.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:35 smithi019 ceph-mon[35186]: pgmap v506: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:35.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:35 smithi001 ceph-mon[30934]: pgmap v506: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:35.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:35 smithi153 ceph-mon[35538]: pgmap v506: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:37.764 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:37 smithi153 ceph-mon[35538]: pgmap v507: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:33:37.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:37 smithi019 ceph-mon[35186]: pgmap v507: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:33:37.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:37 smithi001 ceph-mon[30934]: pgmap v507: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:33:39.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:39 smithi019 ceph-mon[35186]: pgmap v508: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:33:39.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:39 smithi001 ceph-mon[30934]: pgmap v508: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:33:39.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:39 smithi153 ceph-mon[35538]: pgmap v508: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:33:41.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:41 smithi019 ceph-mon[35186]: pgmap v509: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:33:41.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:41 smithi001 ceph-mon[30934]: pgmap v509: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:33:41.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:41 smithi153 ceph-mon[35538]: pgmap v509: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:33:42.058 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:33:42.058 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-16T00:33:42.058 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:42.332 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 -- 172.21.15.1:0/1224694344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101240 msgr2=0x7f04dc101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 --2- 172.21.15.1:0/1224694344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101240 0x7f04dc101680 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009ca0 tx=0x7f04cc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 -- 172.21.15.1:0/1224694344 shutdown_connections 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 --2- 172.21.15.1:0/1224694344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04dc101bc0 0x7f04dc10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 --2- 172.21.15.1:0/1224694344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101240 0x7f04dc101680 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 --2- 172.21.15.1:0/1224694344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc105a90 0x7f04dc105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 -- 172.21.15.1:0/1224694344 >> 172.21.15.1:0/1224694344 conn(0x7f04dc077210 msgr2=0x7f04dc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 -- 172.21.15.1:0/1224694344 shutdown_connections 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 -- 172.21.15.1:0/1224694344 wait complete. 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 Processor -- start 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.006+0000 7f04e2e61640 1 -- start start 2024-09-16T00:33:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04e2e61640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc101240 0x7f04dc0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04e2e61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101bc0 0x7f04dc0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04e2e61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04dc105a90 0x7f04dc0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04e2e61640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc110fb0 con 0x7f04dc101bc0 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04e2e61640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc110e30 con 0x7f04dc105a90 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04e2e61640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc111130 con 0x7f04dc101240 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101bc0 0x7f04dc0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101bc0 0x7f04dc0fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48732/0 (socket says 172.21.15.1:48732) 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04dbfff640 1 -- 172.21.15.1:0/3887117962 learned_addr learned my addr 172.21.15.1:0/3887117962 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04e0bfd640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc101240 0x7f04dc0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04e13fe640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04dc105a90 0x7f04dc0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04dbfff640 1 -- 172.21.15.1:0/3887117962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc101240 msgr2=0x7f04dc0fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04dbfff640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc101240 0x7f04dc0fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04dbfff640 1 -- 172.21.15.1:0/3887117962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04dc105a90 msgr2=0x7f04dc0fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04dbfff640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04dc105a90 0x7f04dc0fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04dbfff640 1 -- 172.21.15.1:0/3887117962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc1a69c0 con 0x7f04dc101bc0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04dbfff640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101bc0 0x7f04dc0fcbd0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006010 tx=0x7f04cc003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04d9ffb640 1 -- 172.21.15.1:0/3887117962 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0317d0 con 0x7f04dc101bc0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04d9ffb640 1 -- 172.21.15.1:0/3887117962 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc031970 con 0x7f04dc101bc0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.007+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc1a6c50 con 0x7f04dc101bc0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.008+0000 7f04d9ffb640 1 -- 172.21.15.1:0/3887117962 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc030c70 con 0x7f04dc101bc0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.008+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc1a7190 con 0x7f04dc101bc0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.009+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04dc101680 con 0x7f04dc101bc0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.010+0000 7f04d9ffb640 1 -- 172.21.15.1:0/3887117962 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f04cc004040 con 0x7f04dc101bc0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.010+0000 7f04d9ffb640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f04c80a32c0 0x7f04c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.011+0000 7f04d9ffb640 1 -- 172.21.15.1:0/3887117962 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f04cc0f1280 con 0x7f04dc101bc0 2024-09-16T00:33:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.011+0000 7f04e0bfd640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f04c80a32c0 0x7f04c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.011+0000 7f04e0bfd640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f04c80a32c0 0x7f04c80a5780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009990 tx=0x7f04c4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.015+0000 7f04d9ffb640 1 -- 172.21.15.1:0/3887117962 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04cc0bdba0 con 0x7f04dc101bc0 2024-09-16T00:33:43.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.231+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04dc1a7480 con 0x7f04dc101bc0 2024-09-16T00:33:43.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.231+0000 7f04d9ffb640 1 -- 172.21.15.1:0/3887117962 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f04cc0bd4a0 con 0x7f04dc101bc0 2024-09-16T00:33:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:33:43.232 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:33:43.232 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:33:43.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f04c80a32c0 msgr2=0x7f04c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f04c80a32c0 0x7f04c80a5780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009990 tx=0x7f04c4009110 comp rx=0 tx=0).stop 2024-09-16T00:33:43.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101bc0 msgr2=0x7f04dc0fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101bc0 0x7f04dc0fcbd0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006010 tx=0x7f04cc003f80 comp rx=0 tx=0).stop 2024-09-16T00:33:43.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 shutdown_connections 2024-09-16T00:33:43.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04dc105a90 0x7f04dc0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04dc101bc0 0x7f04dc0fcbd0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f04c80a32c0 0x7f04c80a5780 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 --2- 172.21.15.1:0/3887117962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc101240 0x7f04dc0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 >> 172.21.15.1:0/3887117962 conn(0x7f04dc077210 msgr2=0x7f04dc100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:43.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 shutdown_connections 2024-09-16T00:33:43.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:43.234+0000 7f04e2e61640 1 -- 172.21.15.1:0/3887117962 wait complete. 2024-09-16T00:33:43.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:43 smithi019 ceph-mon[35186]: pgmap v510: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:33:43.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:43 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3887117962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:43.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:43 smithi001 ceph-mon[30934]: pgmap v510: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:33:43.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:43 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3887117962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:43.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:43 smithi153 ceph-mon[35538]: pgmap v510: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:33:43.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:43 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3887117962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:44.140 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-16T00:33:44.140 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-16T00:33:44.403 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.917+0000 7fed410f5640 1 -- 172.21.15.1:0/2168485187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c103660 msgr2=0x7fed3c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.917+0000 7fed410f5640 1 --2- 172.21.15.1:0/2168485187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c103660 0x7fed3c10bb50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fed30009900 tx=0x7fed3002a480 comp rx=0 tx=0).stop 2024-09-16T00:33:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.917+0000 7fed410f5640 1 -- 172.21.15.1:0/2168485187 shutdown_connections 2024-09-16T00:33:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.917+0000 7fed410f5640 1 --2- 172.21.15.1:0/2168485187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c10c170 0x7fed3c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.917+0000 7fed410f5640 1 --2- 172.21.15.1:0/2168485187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c103660 0x7fed3c10bb50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.917+0000 7fed410f5640 1 --2- 172.21.15.1:0/2168485187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed3c102d40 0x7fed3c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.917+0000 7fed410f5640 1 -- 172.21.15.1:0/2168485187 >> 172.21.15.1:0/2168485187 conn(0x7fed3c0fc7a0 msgr2=0x7fed3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.917+0000 7fed410f5640 1 -- 172.21.15.1:0/2168485187 shutdown_connections 2024-09-16T00:33:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.917+0000 7fed410f5640 1 -- 172.21.15.1:0/2168485187 wait complete. 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.918+0000 7fed410f5640 1 Processor -- start 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.918+0000 7fed410f5640 1 -- start start 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.918+0000 7fed410f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed3c102d40 0x7fed3c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.918+0000 7fed410f5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c103660 0x7fed3c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.918+0000 7fed410f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c10c170 0x7fed3c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.918+0000 7fed410f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c111f70 con 0x7fed3c10c170 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.918+0000 7fed410f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c111df0 con 0x7fed3c102d40 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.918+0000 7fed410f5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1120f0 con 0x7fed3c103660 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c10c170 0x7fed3c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c10c170 0x7fed3c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48740/0 (socket says 172.21.15.1:48740) 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3b59e640 1 -- 172.21.15.1:0/2333527318 learned_addr learned my addr 172.21.15.1:0/2333527318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed3c102d40 0x7fed3c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3a59c640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c103660 0x7fed3c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3b59e640 1 -- 172.21.15.1:0/2333527318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c103660 msgr2=0x7fed3c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3b59e640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c103660 0x7fed3c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3b59e640 1 -- 172.21.15.1:0/2333527318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed3c102d40 msgr2=0x7fed3c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3b59e640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed3c102d40 0x7fed3c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3b59e640 1 -- 172.21.15.1:0/2333527318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c19d390 con 0x7fed3c10c170 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed3b59e640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c10c170 0x7fed3c19ccb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fed3400cce0 tx=0x7fed3400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed23fff640 1 -- 172.21.15.1:0/2333527318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3400f450 con 0x7fed3c10c170 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed410f5640 1 -- 172.21.15.1:0/2333527318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c19d680 con 0x7fed3c10c170 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed23fff640 1 -- 172.21.15.1:0/2333527318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3400fad0 con 0x7fed3c10c170 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed23fff640 1 -- 172.21.15.1:0/2333527318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34018c10 con 0x7fed3c10c170 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.919+0000 7fed410f5640 1 -- 172.21.15.1:0/2333527318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c1a4f60 con 0x7fed3c10c170 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed434ce740 1 Processor -- start 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed23fff640 1 -- 172.21.15.1:0/2333527318 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fed3400f5f0 con 0x7fed3c10c170 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed434ce740 1 -- start start 2024-09-16T00:33:44.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed434ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702cd730 0x55b5702cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed434ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 0x55b5702dc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed434ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702dce40 0x55b5702df230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed434ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b570232160 con 0x55b5702cd730 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed434ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5701d9720 con 0x55b5702ce050 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed434ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b57022e120 con 0x55b5702dce40 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed3a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702cd730 0x55b5702cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed3a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702cd730 0x55b5702cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48750/0 (socket says 172.21.15.1:48750) 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed3a59c640 1 -- 172.21.15.1:0/37143250 learned_addr learned my addr 172.21.15.1:0/37143250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.922+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/37143250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 0x55b5702dc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed3a59c640 1 -- 172.21.15.1:0/37143250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702dce40 msgr2=0x55b5702df230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed3a59c640 1 --2- 172.21.15.1:0/37143250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702dce40 0x55b5702df230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed3a59c640 1 -- 172.21.15.1:0/37143250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 msgr2=0x55b5702dc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:44.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed3a59c640 1 --2- 172.21.15.1:0/37143250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 0x55b5702dc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed3a59c640 1 -- 172.21.15.1:0/37143250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56ffa3ba0 con 0x55b5702cd730 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/37143250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 0x55b5702dc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed23fff640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fed240a3240 0x7fed240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fed240a3240 0x7fed240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed23fff640 1 -- 172.21.15.1:0/2333527318 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fed340d0390 con 0x7fed3c10c170 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed3a59c640 1 --2- 172.21.15.1:0/37143250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702cd730 0x55b5702cdb10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fed3002a9a0 tx=0x7fed30032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125a1b0be9e349f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed217fa640 1 -- 172.21.15.1:0/37143250 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3003f070 con 0x55b5702cd730 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed217fa640 1 -- 172.21.15.1:0/37143250 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed300317b0 con 0x55b5702cd730 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.923+0000 7fed217fa640 1 -- 172.21.15.1:0/37143250 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30035070 con 0x55b5702cd730 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 -- 172.21.15.1:0/37143250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702cd730 msgr2=0x55b5702cdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fed240a3240 0x7fed240a5700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55b5702d0440 tx=0x7fed2c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 --2- 172.21.15.1:0/37143250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702cd730 0x55b5702cdb10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fed3002a9a0 tx=0x7fed30032d00 comp rx=0 tx=0).stop 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 -- 172.21.15.1:0/37143250 shutdown_connections 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 --2- 172.21.15.1:0/37143250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702dce40 0x55b5702df230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 --2- 172.21.15.1:0/37143250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 0x55b5702dc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 --2- 172.21.15.1:0/37143250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702cd730 0x55b5702cdb10 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 -- 172.21.15.1:0/37143250 >> 172.21.15.1:0/37143250 conn(0x55b5702c9de0 msgr2=0x55b5702ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 -- 172.21.15.1:0/37143250 shutdown_connections 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 -- 172.21.15.1:0/37143250 wait complete. 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.924+0000 7fed434ce740 1 -- start start 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed434ce740 1 Processor -- start 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed434ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702cd730 0x55b5702d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed434ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 0x55b5702da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed434ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b5702da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed434ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b5702dfa10 con 0x55b5702dce40 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed434ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5701ba900 con 0x55b5702ce050 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed434ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b5702dfb90 con 0x55b5702cd730 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed434ce740 10 client.0 _ll_register_callbacks cb 0x55b5701b8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b5702da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b5702da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48754/0 (socket says 172.21.15.1:48754) 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3b59e640 1 -- 172.21.15.1:0/3447986997 learned_addr learned my addr 172.21.15.1:0/3447986997 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3a59c640 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 0x55b5702da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702cd730 0x55b5702d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3b59e640 1 -- 172.21.15.1:0/3447986997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702cd730 msgr2=0x55b5702d9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3b59e640 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702cd730 0x55b5702d9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3b59e640 1 -- 172.21.15.1:0/3447986997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 msgr2=0x55b5702da2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3b59e640 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 0x55b5702da2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.925+0000 7fed3b59e640 1 -- 172.21.15.1:0/3447986997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5701fac50 con 0x55b5702dce40 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed3b59e640 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b5702da810 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fed3409c2b0 tx=0x7fed3409bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd5ef398543fe83 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed1eff5640 1 -- 172.21.15.1:0/3447986997 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340b5060 con 0x55b5702dce40 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed1eff5640 1 -- 172.21.15.1:0/3447986997 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fed3400fc80 con 0x55b5702dce40 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed434ce740 1 -- 172.21.15.1:0/3447986997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed300317b0 con 0x55b5702dce40 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed1eff5640 1 -- 172.21.15.1:0/3447986997 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fed340a0960 con 0x55b5702dce40 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed434ce740 10 client.16590 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed434ce740 1 -- 172.21.15.1:0/3447986997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b5702eb780 con 0x55b5702dce40 2024-09-16T00:33:44.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed1eff5640 1 -- 172.21.15.1:0/3447986997 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed300317b0 con 0x55b5702dce40 2024-09-16T00:33:44.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed1eff5640 1 -- 172.21.15.1:0/3447986997 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fed340672b0 con 0x55b5702dce40 2024-09-16T00:33:44.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed434ce740 10 client.16590 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:44.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed434ce740 10 client.16590 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:33:44.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed434ce740 20 client.16590 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:44.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed434ce740 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5702ecd80 0x55b5702ef170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed434ce740 4 client.16590 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:44.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.926+0000 7fed434ce740 1 -- 172.21.15.1:0/3447986997 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b57023f890 con 0x55b5702ecd80 2024-09-16T00:33:44.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.927+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5702ecd80 0x55b5702ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.927+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5702ecd80 0x55b5702ef170 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325fec88a5c4efed server_cookie=a381815988a2e547 in_seq=0 out_seq=0 2024-09-16T00:33:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.927+0000 7fed1eff5640 10 client.16590 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed1eff5640 1 -- 172.21.15.1:0/3447986997 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b57023f890 con 0x55b5702ecd80 2024-09-16T00:33:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed1eff5640 10 client.16590 handle_command_reply: tid=0 2024-09-16T00:33:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 client.16590 shutdown 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 -- 172.21.15.1:0/3447986997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 msgr2=0x55b5702da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b5702da810 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fed3409c2b0 tx=0x7fed3409bf10 comp rx=0 tx=0).stop 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 -- 172.21.15.1:0/3447986997 shutdown_connections 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5702ecd80 0x55b5702ef170 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b5702da810 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702ce050 0x55b5702da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 --2- 172.21.15.1:0/3447986997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702cd730 0x55b5702d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 -- 172.21.15.1:0/3447986997 >> 172.21.15.1:0/3447986997 conn(0x55b5702c9de0 msgr2=0x55b5702cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed1eff5640 0 client.16590 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 -- 172.21.15.1:0/3447986997 shutdown_connections 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 1 -- 172.21.15.1:0/3447986997 wait complete. 2024-09-16T00:33:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.928+0000 7fed434ce740 20 client.16590 trim_cache size 0 max 16384 2024-09-16T00:33:44.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed434ce740 1 Processor -- start 2024-09-16T00:33:44.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed434ce740 1 -- start start 2024-09-16T00:33:44.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed434ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 0x55b5702d8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed434ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702ce050 0x55b5702f4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed434ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702dce40 0x55b5702f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed434ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1af360 con 0x55b5702ce050 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed434ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1acc30 con 0x55b5702cd730 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed434ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1aaa80 con 0x55b5702dce40 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed3a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702ce050 0x55b5702f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed3a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702ce050 0x55b5702f4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48760/0 (socket says 172.21.15.1:48760) 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.945+0000 7fed3a59c640 1 -- 172.21.15.1:0/1509219108 learned_addr learned my addr 172.21.15.1:0/1509219108 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/1509219108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 0x55b5702d8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed3b59e640 1 --2- 172.21.15.1:0/1509219108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702dce40 0x55b5702f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed3a59c640 1 -- 172.21.15.1:0/1509219108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702dce40 msgr2=0x55b5702f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed3a59c640 1 --2- 172.21.15.1:0/1509219108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702dce40 0x55b5702f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed3a59c640 1 -- 172.21.15.1:0/1509219108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 msgr2=0x55b5702d8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:44.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed3a59c640 1 --2- 172.21.15.1:0/1509219108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 0x55b5702d8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed3a59c640 1 -- 172.21.15.1:0/1509219108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5702c9880 con 0x55b5702ce050 2024-09-16T00:33:44.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed3a59c640 1 --2- 172.21.15.1:0/1509219108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702ce050 0x55b5702f4890 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fed300052c0 tx=0x7fed30005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e147c5b0da1cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:44.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed20ff9640 1 -- 172.21.15.1:0/1509219108 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3003f070 con 0x55b5702ce050 2024-09-16T00:33:44.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed20ff9640 1 -- 172.21.15.1:0/1509219108 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed300035f0 con 0x55b5702ce050 2024-09-16T00:33:44.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.946+0000 7fed20ff9640 1 -- 172.21.15.1:0/1509219108 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30035040 con 0x55b5702ce050 2024-09-16T00:33:44.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 -- 172.21.15.1:0/1509219108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702ce050 msgr2=0x55b5702f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 --2- 172.21.15.1:0/1509219108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702ce050 0x55b5702f4890 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fed300052c0 tx=0x7fed30005fd0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 -- 172.21.15.1:0/1509219108 shutdown_connections 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 --2- 172.21.15.1:0/1509219108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702dce40 0x55b5702f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 --2- 172.21.15.1:0/1509219108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702ce050 0x55b5702f4890 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 --2- 172.21.15.1:0/1509219108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 0x55b5702d8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 -- 172.21.15.1:0/1509219108 >> 172.21.15.1:0/1509219108 conn(0x55b5702c9de0 msgr2=0x55b5702dbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 -- 172.21.15.1:0/1509219108 shutdown_connections 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 -- 172.21.15.1:0/1509219108 wait complete. 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 -- start start 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 Processor -- start 2024-09-16T00:33:44.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 0x55b5702e1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702ce050 0x55b5702e1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b570356860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1ab0f0 con 0x55b5702dce40 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1ac560 con 0x55b5702cd730 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.947+0000 7fed434ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1ab7c0 con 0x55b5702ce050 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed434ce740 10 client.0 _ll_register_callbacks cb 0x7fed340077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b570356860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b570356860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48764/0 (socket says 172.21.15.1:48764) 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3b59e640 1 -- 172.21.15.1:0/1781571156 learned_addr learned my addr 172.21.15.1:0/1781571156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 0x55b5702e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3a59c640 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702ce050 0x55b5702e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3b59e640 1 -- 172.21.15.1:0/1781571156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702ce050 msgr2=0x55b5702e1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:44.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3b59e640 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702ce050 0x55b5702e1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3b59e640 1 -- 172.21.15.1:0/1781571156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 msgr2=0x55b5702e1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3b59e640 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 0x55b5702e1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3b59e640 1 -- 172.21.15.1:0/1781571156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5702df8f0 con 0x55b5702dce40 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 0x55b5702e1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed3b59e640 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b570356860 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fed3400ccb0 tx=0x7fed3409be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9669fe5bb7cf006d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed1eff5640 1 -- 172.21.15.1:0/1781571156 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340b5040 con 0x55b5702dce40 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed1eff5640 1 -- 172.21.15.1:0/1781571156 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fed3400fc80 con 0x55b5702dce40 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed434ce740 1 -- 172.21.15.1:0/1781571156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed300035f0 con 0x55b5702dce40 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed1eff5640 1 -- 172.21.15.1:0/1781571156 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fed34082050 con 0x55b5702dce40 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed434ce740 10 client.16596 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.948+0000 7fed434ce740 1 -- 172.21.15.1:0/1781571156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b570357e10 con 0x55b5702dce40 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.949+0000 7fed1eff5640 1 -- 172.21.15.1:0/1781571156 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed300035f0 con 0x55b5702dce40 2024-09-16T00:33:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.949+0000 7fed1eff5640 1 -- 172.21.15.1:0/1781571156 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fed340672f0 con 0x55b5702dce40 2024-09-16T00:33:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.949+0000 7fed434ce740 10 client.16596 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.949+0000 7fed434ce740 10 client.16596 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:33:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.949+0000 7fed434ce740 20 client.16596 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.949+0000 7fed434ce740 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5703584e0 0x55b57035a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.949+0000 7fed434ce740 4 client.16596 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-16T00:33:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.949+0000 7fed434ce740 1 -- 172.21.15.1:0/1781571156 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x55b570239be0 con 0x55b5703584e0 2024-09-16T00:33:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.949+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5703584e0 0x55b57035a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.950+0000 7fed1eff5640 10 client.16596 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.950+0000 7fed3ad9d640 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5703584e0 0x55b57035a8d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9e90bff5e9cd1c server_cookie=6113bb37a883305d in_seq=0 out_seq=0 2024-09-16T00:33:44.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed1eff5640 1 -- 172.21.15.1:0/1781571156 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b570239be0 con 0x55b5703584e0 2024-09-16T00:33:44.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed1eff5640 10 client.16596 handle_command_reply: tid=0 2024-09-16T00:33:44.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 client.16596 shutdown 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 -- 172.21.15.1:0/1781571156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 msgr2=0x55b570356860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b570356860 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fed3400ccb0 tx=0x7fed3409be20 comp rx=0 tx=0).stop 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 -- 172.21.15.1:0/1781571156 shutdown_connections 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5703584e0 0x55b57035a8d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5702dce40 0x55b570356860 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5702ce050 0x55b5702e1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 --2- 172.21.15.1:0/1781571156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5702cd730 0x55b5702e1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 -- 172.21.15.1:0/1781571156 >> 172.21.15.1:0/1781571156 conn(0x55b5702c9de0 msgr2=0x55b5702db5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed1eff5640 0 client.16596 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 -- 172.21.15.1:0/1781571156 shutdown_connections 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 1 -- 172.21.15.1:0/1781571156 wait complete. 2024-09-16T00:33:44.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.961+0000 7fed434ce740 20 client.16596 trim_cache size 0 max 16384 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 18, 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "8e4ce410": { 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 18, 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:33:44.964 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 145.3, 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:44.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:44.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:33:44.966 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:33:44.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 -- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fed240a3240 msgr2=0x7fed240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fed240a3240 0x7fed240a5700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55b5702d0440 tx=0x7fed2c00b390 comp rx=0 tx=0).stop 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 -- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c10c170 msgr2=0x7fed3c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c10c170 0x7fed3c19ccb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fed3400cce0 tx=0x7fed3400a620 comp rx=0 tx=0).stop 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 -- 172.21.15.1:0/2333527318 shutdown_connections 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c10c170 0x7fed3c19ccb0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fed240a3240 0x7fed240a5700 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c103660 0x7fed3c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 --2- 172.21.15.1:0/2333527318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed3c102d40 0x7fed3c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 -- 172.21.15.1:0/2333527318 >> 172.21.15.1:0/2333527318 conn(0x7fed3c0fc7a0 msgr2=0x7fed3c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 -- 172.21.15.1:0/2333527318 shutdown_connections 2024-09-16T00:33:44.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:44.962+0000 7fed410f5640 1 -- 172.21.15.1:0/2333527318 wait complete. 2024-09-16T00:33:45.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:45 smithi001 ceph-mon[30934]: pgmap v511: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:45.838 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8e4ce410', quiesce took 3.8 seconds. Will release after: 21.55 2024-09-16T00:33:45.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:45 smithi019 ceph-mon[35186]: pgmap v511: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:45.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:45 smithi153 ceph-mon[35538]: pgmap v511: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:47.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:47 smithi019 ceph-mon[35186]: pgmap v512: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:47.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:47 smithi001 ceph-mon[30934]: pgmap v512: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:47.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:47 smithi153 ceph-mon[35538]: pgmap v512: 129 pgs: 129 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:33:49.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:49 smithi019 ceph-mon[35186]: pgmap v513: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-16T00:33:49.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:49 smithi001 ceph-mon[30934]: pgmap v513: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-16T00:33:49.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:49 smithi153 ceph-mon[35538]: pgmap v513: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-16T00:33:51.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:51 smithi019 ceph-mon[35186]: pgmap v514: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:33:51.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:51 smithi001 ceph-mon[30934]: pgmap v514: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:33:51.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:51 smithi153 ceph-mon[35538]: pgmap v514: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:33:53.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:53 smithi019 ceph-mon[35186]: pgmap v515: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 10 op/s 2024-09-16T00:33:53.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:53 smithi001 ceph-mon[30934]: pgmap v515: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 10 op/s 2024-09-16T00:33:53.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:53 smithi153 ceph-mon[35538]: pgmap v515: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 10 op/s 2024-09-16T00:33:55.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:55 smithi019 ceph-mon[35186]: pgmap v516: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-16T00:33:55.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:55 smithi001 ceph-mon[30934]: pgmap v516: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-16T00:33:55.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:55 smithi153 ceph-mon[35538]: pgmap v516: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-16T00:33:57.157 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:33:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:57.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:57 smithi019 ceph-mon[35186]: pgmap v517: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-16T00:33:57.862 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:57 smithi001 ceph-mon[30934]: pgmap v517: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-16T00:33:57.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:57 smithi153 ceph-mon[35538]: pgmap v517: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-16T00:33:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.950+0000 7f00ec193640 1 -- 172.21.15.1:0/2226811604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e4106f20 msgr2=0x7f00e410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.950+0000 7f00ec193640 1 --2- 172.21.15.1:0/2226811604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e4106f20 0x7f00e410eaa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009ca0 tx=0x7f00dc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.950+0000 7f00ec193640 1 -- 172.21.15.1:0/2226811604 shutdown_connections 2024-09-16T00:33:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.950+0000 7f00ec193640 1 --2- 172.21.15.1:0/2226811604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e4106f20 0x7f00e410eaa0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.950+0000 7f00ec193640 1 --2- 172.21.15.1:0/2226811604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e41027a0 0x7f00e41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.950+0000 7f00ec193640 1 --2- 172.21.15.1:0/2226811604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e4101df0 0x7f00e41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.950+0000 7f00ec193640 1 -- 172.21.15.1:0/2226811604 >> 172.21.15.1:0/2226811604 conn(0x7f00e40fd8f0 msgr2=0x7f00e40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.950+0000 7f00ec193640 1 -- 172.21.15.1:0/2226811604 shutdown_connections 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.950+0000 7f00ec193640 1 -- 172.21.15.1:0/2226811604 wait complete. 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00ec193640 1 Processor -- start 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00ec193640 1 -- start start 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00ec193640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e4101df0 0x7f00e4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00ec193640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e41027a0 0x7f00e41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00ec193640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e4106f20 0x7f00e419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00ec193640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4111f80 con 0x7f00e41027a0 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00ec193640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4111e00 con 0x7f00e4101df0 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00ec193640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4112100 con 0x7f00e4106f20 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00e972e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e41027a0 0x7f00e41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00e972e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e41027a0 0x7f00e41988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53588/0 (socket says 172.21.15.1:53588) 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00e972e640 1 -- 172.21.15.1:0/2044416291 learned_addr learned my addr 172.21.15.1:0/2044416291 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e4101df0 0x7f00e4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00ea730640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e4106f20 0x7f00e419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00e972e640 1 -- 172.21.15.1:0/2044416291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e4106f20 msgr2=0x7f00e419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00e972e640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e4106f20 0x7f00e419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00e972e640 1 -- 172.21.15.1:0/2044416291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e4101df0 msgr2=0x7f00e4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00e972e640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e4101df0 0x7f00e4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.951+0000 7f00e972e640 1 -- 172.21.15.1:0/2044416291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e419d310 con 0x7f00e41027a0 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.952+0000 7f00e972e640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e41027a0 0x7f00e41988a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f00d800c920 tx=0x7f00d800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.952+0000 7f00d6ffd640 1 -- 172.21.15.1:0/2044416291 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8007c70 con 0x7f00e41027a0 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.952+0000 7f00ec193640 1 -- 172.21.15.1:0/2044416291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e419d600 con 0x7f00e41027a0 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.952+0000 7f00d6ffd640 1 -- 172.21.15.1:0/2044416291 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d8007e10 con 0x7f00e41027a0 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.952+0000 7f00d6ffd640 1 -- 172.21.15.1:0/2044416291 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8018890 con 0x7f00e41027a0 2024-09-16T00:33:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.952+0000 7f00ec193640 1 -- 172.21.15.1:0/2044416291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e41a4ee0 con 0x7f00e41027a0 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.954+0000 7f00d6ffd640 1 -- 172.21.15.1:0/2044416291 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f00d8018a70 con 0x7f00e41027a0 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00ee56c740 1 Processor -- start 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00ee56c740 1 -- start start 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da53c0 0x55fd58da57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 0x55fd58db4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58db4a90 0x55fd58db6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00ee56c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fd58d0b2e0 con 0x55fd58db4a90 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00ee56c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fd58cb2ce0 con 0x55fd58da5ce0 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00ee56c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd58d072a0 con 0x55fd58da53c0 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00e972e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58db4a90 0x55fd58db6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00e972e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58db4a90 0x55fd58db6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53590/0 (socket says 172.21.15.1:53590) 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00e972e640 1 -- 172.21.15.1:0/1823305324 learned_addr learned my addr 172.21.15.1:0/1823305324 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00d6ffd640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d00a3240 0x7f00d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.955+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/1823305324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 0x55fd58db4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00ea730640 1 --2- 172.21.15.1:0/1823305324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da53c0 0x55fd58da57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00ea730640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d00a3240 0x7f00d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00d6ffd640 1 -- 172.21.15.1:0/2044416291 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f00d80cfef0 con 0x7f00e41027a0 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00e9f2f640 1 -- 172.21.15.1:0/1823305324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da53c0 msgr2=0x55fd58da57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/1823305324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da53c0 0x55fd58da57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00e9f2f640 1 -- 172.21.15.1:0/1823305324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58db4a90 msgr2=0x55fd58db6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/1823305324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58db4a90 0x55fd58db6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00e9f2f640 1 -- 172.21.15.1:0/1823305324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd58c97440 con 0x55fd58da5ce0 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/1823305324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 0x55fd58db4550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00e000bbe0 tx=0x7f00e000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e6866c636ee2f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00ea730640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d00a3240 0x7f00d00a5700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009a60 tx=0x7f00dc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00cffff640 1 -- 172.21.15.1:0/1823305324 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0015070 con 0x55fd58da5ce0 2024-09-16T00:33:57.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00cffff640 1 -- 172.21.15.1:0/1823305324 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e00035f0 con 0x55fd58da5ce0 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.956+0000 7f00cffff640 1 -- 172.21.15.1:0/1823305324 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0011640 con 0x55fd58da5ce0 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.957+0000 7f00ee56c740 1 -- 172.21.15.1:0/1823305324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 msgr2=0x55fd58db4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.957+0000 7f00ee56c740 1 --2- 172.21.15.1:0/1823305324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 0x55fd58db4550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00e000bbe0 tx=0x7f00e000f730 comp rx=0 tx=0).stop 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.957+0000 7f00ee56c740 1 -- 172.21.15.1:0/1823305324 shutdown_connections 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.957+0000 7f00ee56c740 1 --2- 172.21.15.1:0/1823305324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58db4a90 0x55fd58db6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.957+0000 7f00ee56c740 1 --2- 172.21.15.1:0/1823305324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 0x55fd58db4550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.957+0000 7f00ee56c740 1 --2- 172.21.15.1:0/1823305324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da53c0 0x55fd58da57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.957+0000 7f00ee56c740 1 -- 172.21.15.1:0/1823305324 >> 172.21.15.1:0/1823305324 conn(0x55fd58da1a30 msgr2=0x55fd58da1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.957+0000 7f00ee56c740 1 -- 172.21.15.1:0/1823305324 shutdown_connections 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.957+0000 7f00ee56c740 1 -- 172.21.15.1:0/1823305324 wait complete. 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ee56c740 1 -- start start 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ee56c740 1 Processor -- start 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da53c0 0x55fd58db4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da5ce0 0x55fd58db16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58db1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ee56c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fd58db7600 con 0x55fd58da5ce0 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ee56c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fd58cc6050 con 0x55fd58da53c0 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ee56c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd58db7780 con 0x55fd58db4a90 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ee56c740 10 client.0 _ll_register_callbacks cb 0x55fd58d06160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00e972e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da5ce0 0x55fd58db16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00e972e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da5ce0 0x55fd58db16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53598/0 (socket says 172.21.15.1:53598) 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00e972e640 1 -- 172.21.15.1:0/2769671528 learned_addr learned my addr 172.21.15.1:0/2769671528 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00ea730640 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58db1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.958+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da53c0 0x55fd58db4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00e972e640 1 -- 172.21.15.1:0/2769671528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 msgr2=0x55fd58db1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00e972e640 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58db1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00e972e640 1 -- 172.21.15.1:0/2769671528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da53c0 msgr2=0x55fd58db4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00e972e640 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da53c0 0x55fd58db4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00e972e640 1 -- 172.21.15.1:0/2769671528 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd58a7c3c0 con 0x55fd58da5ce0 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da53c0 0x55fd58db4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00ea730640 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58db1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00e972e640 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da5ce0 0x55fd58db16f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f00d8000c00 tx=0x7f00d809be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7520caada3331f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2769671528 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d809c7b0 con 0x55fd58da5ce0 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2769671528 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f00d800fb30 con 0x55fd58da5ce0 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00ee56c740 1 -- 172.21.15.1:0/2769671528 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e00035f0 con 0x55fd58da5ce0 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2769671528 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f00d809c0b0 con 0x55fd58da5ce0 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00ee56c740 10 client.16608 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:57.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00ee56c740 1 -- 172.21.15.1:0/2769671528 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd58dc30b0 con 0x55fd58da5ce0 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.959+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2769671528 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e00035f0 con 0x55fd58da5ce0 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2769671528 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f00d8098720 con 0x55fd58da5ce0 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00ee56c740 10 client.16608 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00ee56c740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00ee56c740 20 client.16608 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fd58dc4570 0x55fd58dc6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00ee56c740 4 client.16608 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00ee56c740 1 -- 172.21.15.1:0/2769671528 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd58d18850 con 0x55fd58dc4570 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fd58dc4570 0x55fd58dc6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00cd7fa640 10 client.16608 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.960+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fd58dc4570 0x55fd58dc6990 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f3761643cf1427 server_cookie=2b54c5e167c4476e in_seq=0 out_seq=0 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2769671528 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd58d18850 con 0x55fd58dc4570 2024-09-16T00:33:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00cd7fa640 10 client.16608 handle_command_reply: tid=0 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00ee56c740 1 client.16608 shutdown 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00ee56c740 1 -- 172.21.15.1:0/2769671528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da5ce0 msgr2=0x55fd58db16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da5ce0 0x55fd58db16f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f00d8000c00 tx=0x7f00d809be80 comp rx=0 tx=0).stop 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00ee56c740 1 -- 172.21.15.1:0/2769671528 shutdown_connections 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fd58dc4570 0x55fd58dc6990 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58db1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da5ce0 0x55fd58db16f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00cd7fa640 0 client.16608 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2769671528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da53c0 0x55fd58db4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.961+0000 7f00ee56c740 1 -- 172.21.15.1:0/2769671528 >> 172.21.15.1:0/2769671528 conn(0x55fd58da1a30 msgr2=0x55fd58da76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.962+0000 7f00ee56c740 1 -- 172.21.15.1:0/2769671528 shutdown_connections 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.962+0000 7f00ee56c740 1 -- 172.21.15.1:0/2769671528 wait complete. 2024-09-16T00:33:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.962+0000 7f00ee56c740 20 client.16608 trim_cache size 0 max 16384 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.979+0000 7f00ee56c740 1 Processor -- start 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.979+0000 7f00ee56c740 1 -- start start 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.979+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dc08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.979+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 0x55fd58dcc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.979+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58dcc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.979+0000 7f00ee56c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41af480 con 0x55fd58da53c0 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.979+0000 7f00ee56c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41acd50 con 0x55fd58da5ce0 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.979+0000 7f00ee56c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41aaba0 con 0x55fd58db4a90 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e9f2f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dc08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e9f2f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dc08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53600/0 (socket says 172.21.15.1:53600) 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e9f2f640 1 -- 172.21.15.1:0/1573669091 learned_addr learned my addr 172.21.15.1:0/1573669091 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e972e640 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 0x55fd58dcc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00ea730640 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58dcc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e9f2f640 1 -- 172.21.15.1:0/1573669091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 msgr2=0x55fd58dcc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58dcc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e9f2f640 1 -- 172.21.15.1:0/1573669091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 msgr2=0x55fd58dcc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 0x55fd58dcc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e9f2f640 1 -- 172.21.15.1:0/1573669091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd58db06c0 con 0x55fd58da53c0 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00ea730640 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58dcc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dc08a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005fa0 tx=0x7f00e0005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8098a9b94b37402 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00cf7fe640 1 -- 172.21.15.1:0/1573669091 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e002c070 con 0x55fd58da53c0 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.980+0000 7f00cf7fe640 1 -- 172.21.15.1:0/1573669091 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e00035f0 con 0x55fd58da53c0 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.981+0000 7f00cf7fe640 1 -- 172.21.15.1:0/1573669091 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0015040 con 0x55fd58da53c0 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.981+0000 7f00ee56c740 1 -- 172.21.15.1:0/1573669091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 msgr2=0x55fd58dc08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.981+0000 7f00ee56c740 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dc08a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005fa0 tx=0x7f00e0005f30 comp rx=0 tx=0).stop 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.982+0000 7f00ee56c740 1 -- 172.21.15.1:0/1573669091 shutdown_connections 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.982+0000 7f00ee56c740 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58db4a90 0x55fd58dcc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.982+0000 7f00ee56c740 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58da5ce0 0x55fd58dcc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.982+0000 7f00ee56c740 1 --2- 172.21.15.1:0/1573669091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dc08a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.982+0000 7f00ee56c740 1 -- 172.21.15.1:0/1573669091 >> 172.21.15.1:0/1573669091 conn(0x55fd58da1a30 msgr2=0x55fd58dacd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.982+0000 7f00ee56c740 1 -- 172.21.15.1:0/1573669091 shutdown_connections 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.982+0000 7f00ee56c740 1 -- 172.21.15.1:0/1573669091 wait complete. 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.982+0000 7f00ee56c740 1 -- start start 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.982+0000 7f00ee56c740 1 Processor -- start 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dacf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da5ce0 0x55fd58dbb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00ee56c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58db4a90 0x55fd58dbb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00ee56c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41ab210 con 0x55fd58da53c0 2024-09-16T00:33:57.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00ee56c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41ac680 con 0x55fd58db4a90 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00ee56c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41ab8e0 con 0x55fd58da5ce0 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00ee56c740 10 client.0 _ll_register_callbacks cb 0x7f00d800c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00e9f2f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dacf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00e9f2f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dacf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53612/0 (socket says 172.21.15.1:53612) 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00e9f2f640 1 -- 172.21.15.1:0/2098879319 learned_addr learned my addr 172.21.15.1:0/2098879319 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00e972e640 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da5ce0 0x55fd58dbb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00ea730640 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58db4a90 0x55fd58dbb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00e9f2f640 1 -- 172.21.15.1:0/2098879319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da5ce0 msgr2=0x55fd58dbb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da5ce0 0x55fd58dbb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00e9f2f640 1 -- 172.21.15.1:0/2098879319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58db4a90 msgr2=0x55fd58dbb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58db4a90 0x55fd58dbb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.983+0000 7f00e9f2f640 1 -- 172.21.15.1:0/2098879319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd58da9fe0 con 0x55fd58da53c0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00e9f2f640 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dacf60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f00e000fe10 tx=0x7f00e000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbe724cc18d32b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2098879319 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e002c070 con 0x55fd58da53c0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2098879319 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f00e0003bb0 con 0x55fd58da53c0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00ee56c740 1 -- 172.21.15.1:0/2098879319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e00035f0 con 0x55fd58da53c0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2098879319 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f00e0003f10 con 0x55fd58da53c0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00ee56c740 10 client.16614 fetch_fsmap learned FSMap version 25 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00ee56c740 1 -- 172.21.15.1:0/2098879319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd58e33e40 con 0x55fd58da53c0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2098879319 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e00035f0 con 0x55fd58da53c0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2098879319 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f00e0025af0 con 0x55fd58da53c0 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00ee56c740 10 client.16614 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00ee56c740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00ee56c740 20 client.16614 populate_metadata read hostname 'smithi001' 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fd58e342b0 0x55fd58e366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00ee56c740 4 client.16614 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.984+0000 7f00ee56c740 1 -- 172.21.15.1:0/2098879319 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fd58d12d60 con 0x55fd58e342b0 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.985+0000 7f00e972e640 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fd58e342b0 0x55fd58e366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.985+0000 7f00e972e640 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fd58e342b0 0x55fd58e366a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c9ce89910643510 server_cookie=75772d1b7089dd2f in_seq=0 out_seq=0 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.985+0000 7f00cd7fa640 10 client.16614 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.985+0000 7f00cd7fa640 1 -- 172.21.15.1:0/2098879319 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fd58d12d60 con 0x55fd58e342b0 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.985+0000 7f00cd7fa640 10 client.16614 handle_command_reply: tid=0 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.985+0000 7f00ee56c740 1 client.16614 shutdown 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.985+0000 7f00ee56c740 1 -- 172.21.15.1:0/2098879319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 msgr2=0x55fd58dacf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.985+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dacf60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f00e000fe10 tx=0x7f00e000b820 comp rx=0 tx=0).stop 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ee56c740 1 -- 172.21.15.1:0/2098879319 shutdown_connections 2024-09-16T00:33:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fd58e342b0 0x55fd58e366a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd58db4a90 0x55fd58dbb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd58da5ce0 0x55fd58dbb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00cd7fa640 0 client.16614 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ee56c740 1 --2- 172.21.15.1:0/2098879319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd58da53c0 0x55fd58dacf60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ee56c740 1 -- 172.21.15.1:0/2098879319 >> 172.21.15.1:0/2098879319 conn(0x55fd58da1a30 msgr2=0x55fd58da0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ee56c740 1 -- 172.21.15.1:0/2098879319 shutdown_connections 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ee56c740 1 -- 172.21.15.1:0/2098879319 wait complete. 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ee56c740 20 client.16614 trim_cache size 0 max 16384 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:33:58.000 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:33:58.001 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:33:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:33:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ec193640 1 -- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d00a3240 msgr2=0x7f00d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ec193640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d00a3240 0x7f00d00a5700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009a60 tx=0x7f00dc0095c0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ec193640 1 -- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e41027a0 msgr2=0x7f00e41988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ec193640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e41027a0 0x7f00e41988a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f00d800c920 tx=0x7f00d800cde0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ec193640 1 -- 172.21.15.1:0/2044416291 shutdown_connections 2024-09-16T00:33:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ec193640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00d00a3240 0x7f00d00a5700 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.986+0000 7f00ec193640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e4106f20 0x7f00e419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.987+0000 7f00ec193640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e41027a0 0x7f00e41988a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.987+0000 7f00ec193640 1 --2- 172.21.15.1:0/2044416291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e4101df0 0x7f00e4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.987+0000 7f00ec193640 1 -- 172.21.15.1:0/2044416291 >> 172.21.15.1:0/2044416291 conn(0x7f00e40fd8f0 msgr2=0x7f00e40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.987+0000 7f00ec193640 1 -- 172.21.15.1:0/2044416291 shutdown_connections 2024-09-16T00:33:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:57.987+0000 7f00ec193640 1 -- 172.21.15.1:0/2044416291 wait complete. 2024-09-16T00:33:58.839 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:33:58.840 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:59.101 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:59.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/1471443571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 msgr2=0x7fcb6410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:59.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/1471443571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 0x7fcb6410df40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009900 tx=0x7fcb5402a410 comp rx=0 tx=0).stop 2024-09-16T00:33:59.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/1471443571 shutdown_connections 2024-09-16T00:33:59.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/1471443571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410e480 0x7fcb64110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/1471443571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 0x7fcb6410df40 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/1471443571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb641008d0 0x7fcb64100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/1471443571 >> 172.21.15.1:0/1471443571 conn(0x7fcb640fc7a0 msgr2=0x7fcb640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:59.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/1471443571 shutdown_connections 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/1471443571 wait complete. 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.606+0000 7fcb6bd6a640 1 Processor -- start 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb6bd6a640 1 -- start start 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb6bd6a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb641008d0 0x7fcb6419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb6bd6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 0x7fcb6419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb6bd6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410e480 0x7fcb6419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb6bd6a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64114190 con 0x7fcb64101280 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb6bd6a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64114010 con 0x7fcb6410e480 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb6bd6a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64114310 con 0x7fcb641008d0 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69305640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 0x7fcb6419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69305640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 0x7fcb6419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53628/0 (socket says 172.21.15.1:53628) 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69305640 1 -- 172.21.15.1:0/2246385166 learned_addr learned my addr 172.21.15.1:0/2246385166 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:33:59.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69b06640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb641008d0 0x7fcb6419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb6a307640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410e480 0x7fcb6419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69305640 1 -- 172.21.15.1:0/2246385166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb641008d0 msgr2=0x7fcb6419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69305640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb641008d0 0x7fcb6419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69305640 1 -- 172.21.15.1:0/2246385166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410e480 msgr2=0x7fcb6419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69305640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410e480 0x7fcb6419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69305640 1 -- 172.21.15.1:0/2246385166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6419f640 con 0x7fcb64101280 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.607+0000 7fcb69305640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 0x7fcb6419abd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcb54006010 tx=0x7fcb540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.608+0000 7fcb5affd640 1 -- 172.21.15.1:0/2246385166 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5403f030 con 0x7fcb64101280 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.608+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6419f8d0 con 0x7fcb64101280 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.608+0000 7fcb5affd640 1 -- 172.21.15.1:0/2246385166 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb54003ce0 con 0x7fcb64101280 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.608+0000 7fcb5affd640 1 -- 172.21.15.1:0/2246385166 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54035070 con 0x7fcb64101280 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.608+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb641a7210 con 0x7fcb64101280 2024-09-16T00:33:59.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.609+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb64100cb0 con 0x7fcb64101280 2024-09-16T00:33:59.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.610+0000 7fcb5affd640 1 -- 172.21.15.1:0/2246385166 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fcb54042030 con 0x7fcb64101280 2024-09-16T00:33:59.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.610+0000 7fcb5affd640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb500a3240 0x7fcb500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:59.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.611+0000 7fcb5affd640 1 -- 172.21.15.1:0/2246385166 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fcb540f0130 con 0x7fcb64101280 2024-09-16T00:33:59.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.611+0000 7fcb69b06640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb500a3240 0x7fcb500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:59.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.611+0000 7fcb69b06640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb500a3240 0x7fcb500a5700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000abe0 tx=0x7fcb6000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:59.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.615+0000 7fcb5affd640 1 -- 172.21.15.1:0/2246385166 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb540bca50 con 0x7fcb64101280 2024-09-16T00:33:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.827+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb64109610 con 0x7fcb64101280 2024-09-16T00:33:59.829 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:33:59 smithi001 ceph-mon[30934]: pgmap v518: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-16T00:33:59.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.828+0000 7fcb5affd640 1 -- 172.21.15.1:0/2246385166 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fcb540bc350 con 0x7fcb64101280 2024-09-16T00:33:59.830 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:33:59.830 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:33:59.830 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:33:59.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb500a3240 msgr2=0x7fcb500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:59.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb500a3240 0x7fcb500a5700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000abe0 tx=0x7fcb6000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 msgr2=0x7fcb6419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:59.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 0x7fcb6419abd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcb54006010 tx=0x7fcb540042f0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 shutdown_connections 2024-09-16T00:33:59.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410e480 0x7fcb6419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb64101280 0x7fcb6419abd0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb500a3240 0x7fcb500a5700 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 --2- 172.21.15.1:0/2246385166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb641008d0 0x7fcb6419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:59.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 >> 172.21.15.1:0/2246385166 conn(0x7fcb640fc7a0 msgr2=0x7fcb640feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:59.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 shutdown_connections 2024-09-16T00:33:59.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:33:59.831+0000 7fcb6bd6a640 1 -- 172.21.15.1:0/2246385166 wait complete. 2024-09-16T00:33:59.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:33:59 smithi019 ceph-mon[35186]: pgmap v518: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-16T00:33:59.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:33:59 smithi153 ceph-mon[35538]: pgmap v518: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-16T00:34:00.598 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:34:00.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:00 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2246385166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:00.863 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:00.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:00 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2246385166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:00.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:00 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2246385166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:01.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.427+0000 7f65a9f62640 1 -- 172.21.15.1:0/2773670883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 msgr2=0x7f65a4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:01.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.427+0000 7f65a9f62640 1 --2- 172.21.15.1:0/2773670883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 0x7f65a4103120 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6594009900 tx=0x7f659402a480 comp rx=0 tx=0).stop 2024-09-16T00:34:01.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.427+0000 7f65a9f62640 1 -- 172.21.15.1:0/2773670883 shutdown_connections 2024-09-16T00:34:01.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.427+0000 7f65a9f62640 1 --2- 172.21.15.1:0/2773670883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a410c170 0x7f65a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.427+0000 7f65a9f62640 1 --2- 172.21.15.1:0/2773670883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4103660 0x7f65a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.427+0000 7f65a9f62640 1 --2- 172.21.15.1:0/2773670883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 0x7f65a4103120 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.427+0000 7f65a9f62640 1 -- 172.21.15.1:0/2773670883 >> 172.21.15.1:0/2773670883 conn(0x7f65a40fc7a0 msgr2=0x7f65a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:01.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.427+0000 7f65a9f62640 1 -- 172.21.15.1:0/2773670883 shutdown_connections 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.427+0000 7f65a9f62640 1 -- 172.21.15.1:0/2773670883 wait complete. 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a9f62640 1 Processor -- start 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a9f62640 1 -- start start 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a9f62640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 0x7f65a41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a9f62640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4103660 0x7f65a4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a9f62640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a410c170 0x7f65a4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a9f62640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4111fb0 con 0x7f65a4102d40 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a9f62640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4111e30 con 0x7f65a4103660 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a9f62640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4112130 con 0x7f65a410c170 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 0x7f65a41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 0x7f65a41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38828/0 (socket says 172.21.15.1:38828) 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.428+0000 7f65a37fe640 1 -- 172.21.15.1:0/1409058709 learned_addr learned my addr 172.21.15.1:0/1409058709 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:01.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a3fff640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a410c170 0x7f65a4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4103660 0x7f65a4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a37fe640 1 -- 172.21.15.1:0/1409058709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a410c170 msgr2=0x7f65a4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a37fe640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a410c170 0x7f65a4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a37fe640 1 -- 172.21.15.1:0/1409058709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4103660 msgr2=0x7f65a4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a37fe640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4103660 0x7f65a4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a37fe640 1 -- 172.21.15.1:0/1409058709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a41a6950 con 0x7f65a4102d40 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4103660 0x7f65a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a37fe640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 0x7f65a41009a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6594006010 tx=0x7f65940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a0ff9640 1 -- 172.21.15.1:0/1409058709 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659403f030 con 0x7f65a4102d40 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a9f62640 1 -- 172.21.15.1:0/1409058709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a41a6be0 con 0x7f65a4102d40 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a0ff9640 1 -- 172.21.15.1:0/1409058709 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6594003ce0 con 0x7f65a4102d40 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a0ff9640 1 -- 172.21.15.1:0/1409058709 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594035070 con 0x7f65a4102d40 2024-09-16T00:34:01.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.429+0000 7f65a9f62640 1 -- 172.21.15.1:0/1409058709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a41a70c0 con 0x7f65a4102d40 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.431+0000 7f65a0ff9640 1 -- 172.21.15.1:0/1409058709 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6594042030 con 0x7f65a4102d40 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65ac33b740 1 Processor -- start 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65ac33b740 1 -- start start 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3db40 0x5613caf3c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3c5d0 0x5613caf3ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf49760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65ac33b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5613caea22f0 con 0x5613caf3db40 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65ac33b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5613cae49cf0 con 0x5613caf47370 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65ac33b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5613cae9e2b0 con 0x5613caf3c5d0 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3db40 0x5613caf3c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65a0ff9640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f658c0a3240 0x7f658c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3db40 0x5613caf3c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38834/0 (socket says 172.21.15.1:38834) 2024-09-16T00:34:01.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65a3fff640 1 -- 172.21.15.1:0/4154093075 learned_addr learned my addr 172.21.15.1:0/4154093075 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/4154093075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf49760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65a0ff9640 1 -- 172.21.15.1:0/1409058709 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f65940f0220 con 0x7f65a4102d40 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65a37fe640 1 --2- 172.21.15.1:0/4154093075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3c5d0 0x5613caf3ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.432+0000 7f65a3fff640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f658c0a3240 0x7f658c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f65a3fff640 1 -- 172.21.15.1:0/4154093075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3c5d0 msgr2=0x5613caf3ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f65a3fff640 1 --2- 172.21.15.1:0/4154093075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3c5d0 0x5613caf3ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f65a3fff640 1 -- 172.21.15.1:0/4154093075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 msgr2=0x5613caf49760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f65a3fff640 1 --2- 172.21.15.1:0/4154093075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf49760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f65a3fff640 1 -- 172.21.15.1:0/4154093075 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613cae2e3a0 con 0x5613caf3db40 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f65a3fff640 1 --2- 172.21.15.1:0/4154093075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3db40 0x5613caf3c090 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f659c00a990 tx=0x7f659c00ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d72108dc1bb9e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f6589ffb640 1 -- 172.21.15.1:0/4154093075 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c01d030 con 0x5613caf3db40 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f6589ffb640 1 -- 172.21.15.1:0/4154093075 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c015c60 con 0x5613caf3db40 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f6589ffb640 1 -- 172.21.15.1:0/4154093075 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0213c0 con 0x5613caf3db40 2024-09-16T00:34:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.433+0000 7f65a3fff640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f658c0a3240 0x7f658c0a5700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f659c020000 tx=0x7f659c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:01.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 -- 172.21.15.1:0/4154093075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3db40 msgr2=0x5613caf3c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:01.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 --2- 172.21.15.1:0/4154093075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3db40 0x5613caf3c090 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f659c00a990 tx=0x7f659c00ae50 comp rx=0 tx=0).stop 2024-09-16T00:34:01.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 -- 172.21.15.1:0/4154093075 shutdown_connections 2024-09-16T00:34:01.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 --2- 172.21.15.1:0/4154093075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 --2- 172.21.15.1:0/4154093075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3c5d0 0x5613caf3ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 --2- 172.21.15.1:0/4154093075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3db40 0x5613caf3c090 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 -- 172.21.15.1:0/4154093075 >> 172.21.15.1:0/4154093075 conn(0x5613caf05310 msgr2=0x5613caf38c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:01.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 -- 172.21.15.1:0/4154093075 shutdown_connections 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 -- 172.21.15.1:0/4154093075 wait complete. 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 -- start start 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.434+0000 7f65ac33b740 1 Processor -- start 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf46d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf3db40 0x5613caf44010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf47370 0x5613caf44600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65ac33b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5613caf4a070 con 0x5613caf3c5d0 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65ac33b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5613cae5d060 con 0x5613caf3db40 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65ac33b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5613caf4a1f0 con 0x5613caf47370 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65ac33b740 10 client.0 _ll_register_callbacks cb 0x5613cabb08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf46d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf46d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38838/0 (socket says 172.21.15.1:38838) 2024-09-16T00:34:01.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a37fe640 1 -- 172.21.15.1:0/2880011889 learned_addr learned my addr 172.21.15.1:0/2880011889 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf3db40 0x5613caf44010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a3fff640 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf47370 0x5613caf44600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a37fe640 1 -- 172.21.15.1:0/2880011889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf47370 msgr2=0x5613caf44600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a37fe640 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf47370 0x5613caf44600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a37fe640 1 -- 172.21.15.1:0/2880011889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf3db40 msgr2=0x5613caf44010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a37fe640 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf3db40 0x5613caf44010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a37fe640 1 -- 172.21.15.1:0/2880011889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613cac133c0 con 0x5613caf3c5d0 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f65a37fe640 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf46d60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f65940b8ed0 tx=0x7f65940b8860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5396251fffab9c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.435+0000 7f6582ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f6582ffd640 1 -- 172.21.15.1:0/2880011889 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594049040 con 0x5613caf3c5d0 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f6582ffd640 1 -- 172.21.15.1:0/2880011889 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f65940c4ce0 con 0x5613caf3c5d0 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f65ac33b740 1 -- 172.21.15.1:0/2880011889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c015c60 con 0x5613caf3c5d0 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f6582ffd640 1 -- 172.21.15.1:0/2880011889 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f659403bcb0 con 0x5613caf3c5d0 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f65ac33b740 10 client.16632 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f65ac33b740 1 -- 172.21.15.1:0/2880011889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5613caf55b10 con 0x5613caf3c5d0 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f6582ffd640 1 -- 172.21.15.1:0/2880011889 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c015c60 con 0x5613caf3c5d0 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f6582ffd640 1 -- 172.21.15.1:0/2880011889 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f65940bb950 con 0x5613caf3c5d0 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f65ac33b740 10 client.16632 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f65ac33b740 10 client.16632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f65ac33b740 20 client.16632 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f65ac33b740 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5613caf56f20 0x5613caf59330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f65ac33b740 4 client.16632 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.436+0000 7f65ac33b740 1 -- 172.21.15.1:0/2880011889 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613caeaf860 con 0x5613caf56f20 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.437+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5613caf56f20 0x5613caf59330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.437+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5613caf56f20 0x5613caf59330 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb4690aabe701fb8 server_cookie=b283385cbfc822f in_seq=0 out_seq=0 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.437+0000 7f6582ffd640 10 client.16632 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f6582ffd640 1 -- 172.21.15.1:0/2880011889 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613caeaf860 con 0x5613caf56f20 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f6582ffd640 10 client.16632 handle_command_reply: tid=0 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 client.16632 shutdown 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 -- 172.21.15.1:0/2880011889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 msgr2=0x5613caf46d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf46d60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f65940b8ed0 tx=0x7f65940b8860 comp rx=0 tx=0).stop 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 -- 172.21.15.1:0/2880011889 shutdown_connections 2024-09-16T00:34:01.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf47370 0x5613caf44600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5613caf56f20 0x5613caf59330 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf3db40 0x5613caf44010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 --2- 172.21.15.1:0/2880011889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf46d60 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f6582ffd640 0 client.16632 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:01.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 -- 172.21.15.1:0/2880011889 >> 172.21.15.1:0/2880011889 conn(0x5613caf05310 msgr2=0x5613caf3e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:01.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 -- 172.21.15.1:0/2880011889 shutdown_connections 2024-09-16T00:34:01.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 1 -- 172.21.15.1:0/2880011889 wait complete. 2024-09-16T00:34:01.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.438+0000 7f65ac33b740 20 client.16632 trim_cache size 0 max 16384 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65ac33b740 1 Processor -- start 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65ac33b740 1 -- start start 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf38c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 0x5613caf43fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf3f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65ac33b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41b38b0 con 0x5613caf3c5d0 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65ac33b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41b1180 con 0x5613caf47370 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65ac33b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41aefd0 con 0x5613caf3db40 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf38c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf38c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38844/0 (socket says 172.21.15.1:38844) 2024-09-16T00:34:01.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.451+0000 7f65a37fe640 1 -- 172.21.15.1:0/3773811789 learned_addr learned my addr 172.21.15.1:0/3773811789 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 0x5613caf43fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65a3fff640 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf3f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65a37fe640 1 -- 172.21.15.1:0/3773811789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 msgr2=0x5613caf43fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65a37fe640 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 0x5613caf43fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65a37fe640 1 -- 172.21.15.1:0/3773811789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 msgr2=0x5613caf3f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65a37fe640 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf3f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65a37fe640 1 -- 172.21.15.1:0/3773811789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613caf425c0 con 0x5613caf3c5d0 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65a3fff640 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf3f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65a37fe640 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf38c60 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6594005ab0 tx=0x7f65940baa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabf90fde295a65a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:01.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65897fa640 1 -- 172.21.15.1:0/3773811789 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594049030 con 0x5613caf3c5d0 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65897fa640 1 -- 172.21.15.1:0/3773811789 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659403b580 con 0x5613caf3c5d0 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.452+0000 7f65897fa640 1 -- 172.21.15.1:0/3773811789 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65940044e0 con 0x5613caf3c5d0 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 -- 172.21.15.1:0/3773811789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 msgr2=0x5613caf38c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf38c60 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6594005ab0 tx=0x7f65940baa10 comp rx=0 tx=0).stop 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 -- 172.21.15.1:0/3773811789 shutdown_connections 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf3f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 0x5613caf43fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3773811789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf38c60 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 -- 172.21.15.1:0/3773811789 >> 172.21.15.1:0/3773811789 conn(0x5613caf05310 msgr2=0x5613caf37d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 -- 172.21.15.1:0/3773811789 shutdown_connections 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 -- 172.21.15.1:0/3773811789 wait complete. 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.453+0000 7f65ac33b740 1 -- start start 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65ac33b740 1 Processor -- start 2024-09-16T00:34:01.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf4f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 0x5613caf4f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65ac33b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf4c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65ac33b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41af640 con 0x5613caf3c5d0 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65ac33b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41b0ab0 con 0x5613caf47370 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65ac33b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41afd10 con 0x5613caf3db40 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65ac33b740 10 client.0 _ll_register_callbacks cb 0x7f65940dc040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf4f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf4f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38858/0 (socket says 172.21.15.1:38858) 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a37fe640 1 -- 172.21.15.1:0/3524180867 learned_addr learned my addr 172.21.15.1:0/3524180867 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 0x5613caf4f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a3fff640 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf4c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a37fe640 1 -- 172.21.15.1:0/3524180867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 msgr2=0x5613caf4f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a37fe640 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 0x5613caf4f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a37fe640 1 -- 172.21.15.1:0/3524180867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 msgr2=0x5613caf4c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a37fe640 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf4c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.454+0000 7f65a37fe640 1 -- 172.21.15.1:0/3524180867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613caf4c990 con 0x5613caf3c5d0 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f65a37fe640 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf4f480 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f65940bc490 tx=0x7f65940b8e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0b9231c7299ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f6582ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f6582ffd640 1 -- 172.21.15.1:0/3524180867 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65940b3060 con 0x5613caf3c5d0 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f6582ffd640 1 -- 172.21.15.1:0/3524180867 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6594003a00 con 0x5613caf3c5d0 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f65ac33b740 1 -- 172.21.15.1:0/3524180867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5613caea9d70 con 0x5613caf3c5d0 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f6582ffd640 1 -- 172.21.15.1:0/3524180867 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6594049070 con 0x5613caf3c5d0 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f65ac33b740 10 client.16638 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f65ac33b740 1 -- 172.21.15.1:0/3524180867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5613caf4c990 con 0x5613caf3c5d0 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f6582ffd640 1 -- 172.21.15.1:0/3524180867 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5613caea9d70 con 0x5613caf3c5d0 2024-09-16T00:34:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f6582ffd640 1 -- 172.21.15.1:0/3524180867 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f65940ad030 con 0x5613caf3c5d0 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f65ac33b740 10 client.16638 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.455+0000 7f65ac33b740 10 client.16638 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f65ac33b740 20 client.16638 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5613cafc25f0 0x5613cafc49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f65ac33b740 4 client.16638 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f65ac33b740 1 -- 172.21.15.1:0/3524180867 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613cae26170 con 0x5613cafc25f0 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5613cafc25f0 0x5613cafc49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f65a2ffd640 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5613cafc25f0 0x5613cafc49e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=977185c15e451552 server_cookie=54e9ddc23815ff6c in_seq=0 out_seq=0 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f6582ffd640 10 client.16638 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f6582ffd640 1 -- 172.21.15.1:0/3524180867 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613cae26170 con 0x5613cafc25f0 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f6582ffd640 10 client.16638 handle_command_reply: tid=0 2024-09-16T00:34:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.456+0000 7f65ac33b740 1 client.16638 shutdown 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 -- 172.21.15.1:0/3524180867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 msgr2=0x5613caf4f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf4f480 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f65940bc490 tx=0x7f65940b8e20 comp rx=0 tx=0).stop 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 -- 172.21.15.1:0/3524180867 shutdown_connections 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5613cafc25f0 0x5613cafc49e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613caf47370 0x5613caf4c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613caf3db40 0x5613caf4f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 --2- 172.21.15.1:0/3524180867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613caf3c5d0 0x5613caf4f480 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 -- 172.21.15.1:0/3524180867 >> 172.21.15.1:0/3524180867 conn(0x5613caf05310 msgr2=0x5613caf45870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f6582ffd640 0 client.16638 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 -- 172.21.15.1:0/3524180867 shutdown_connections 2024-09-16T00:34:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 1 -- 172.21.15.1:0/3524180867 wait complete. 2024-09-16T00:34:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65ac33b740 20 client.16638 trim_cache size 0 max 16384 2024-09-16T00:34:01.467 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:34:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65a9f62640 1 -- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f658c0a3240 msgr2=0x7f658c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65a9f62640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f658c0a3240 0x7f658c0a5700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f659c020000 tx=0x7f659c01e000 comp rx=0 tx=0).stop 2024-09-16T00:34:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65a9f62640 1 -- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 msgr2=0x7f65a41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.457+0000 7f65a9f62640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 0x7f65a41009a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6594006010 tx=0x7f65940042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.458+0000 7f65a9f62640 1 -- 172.21.15.1:0/1409058709 shutdown_connections 2024-09-16T00:34:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.458+0000 7f65a9f62640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f658c0a3240 0x7f658c0a5700 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.458+0000 7f65a9f62640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a410c170 0x7f65a4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.458+0000 7f65a9f62640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4103660 0x7f65a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.458+0000 7f65a9f62640 1 --2- 172.21.15.1:0/1409058709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a4102d40 0x7f65a41009a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.458+0000 7f65a9f62640 1 -- 172.21.15.1:0/1409058709 >> 172.21.15.1:0/1409058709 conn(0x7f65a40fc7a0 msgr2=0x7f65a40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.458+0000 7f65a9f62640 1 -- 172.21.15.1:0/1409058709 shutdown_connections 2024-09-16T00:34:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:01.458+0000 7f65a9f62640 1 -- 172.21.15.1:0/1409058709 wait complete. 2024-09-16T00:34:01.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:01 smithi001 ceph-mon[30934]: pgmap v519: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:34:01.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:01 smithi019 ceph-mon[35186]: pgmap v519: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:34:01.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:01 smithi153 ceph-mon[35538]: pgmap v519: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:34:02.289 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:34:02.548 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:03.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 -- 172.21.15.1:0/3987130947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108101de0 msgr2=0x7fe1081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:03.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 --2- 172.21.15.1:0/3987130947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108101de0 0x7fe1081021c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009ca0 tx=0x7fe0f802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 -- 172.21.15.1:0/3987130947 shutdown_connections 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 --2- 172.21.15.1:0/3987130947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe108106f10 0x7fe10810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 --2- 172.21.15.1:0/3987130947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe108102790 0x7fe1081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 --2- 172.21.15.1:0/3987130947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108101de0 0x7fe1081021c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 -- 172.21.15.1:0/3987130947 >> 172.21.15.1:0/3987130947 conn(0x7fe1080fd8c0 msgr2=0x7fe1080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 -- 172.21.15.1:0/3987130947 shutdown_connections 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 -- 172.21.15.1:0/3987130947 wait complete. 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.062+0000 7fe10e22c640 1 Processor -- start 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe10e22c640 1 -- start start 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe10e22c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe108101de0 0x7fe1080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe10e22c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108102790 0x7fe108074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe10e22c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe108106f10 0x7fe108074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe10e22c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe108111f70 con 0x7fe108102790 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe10e22c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe108111df0 con 0x7fe108106f10 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe10e22c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081120f0 con 0x7fe108101de0 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe106ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108102790 0x7fe108074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe106ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108102790 0x7fe108074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38864/0 (socket says 172.21.15.1:38864) 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe106ffd640 1 -- 172.21.15.1:0/2812087526 learned_addr learned my addr 172.21.15.1:0/2812087526 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe1077fe640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe108101de0 0x7fe1080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe107fff640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe108106f10 0x7fe108074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe106ffd640 1 -- 172.21.15.1:0/2812087526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe108101de0 msgr2=0x7fe1080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:03.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe106ffd640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe108101de0 0x7fe1080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe106ffd640 1 -- 172.21.15.1:0/2812087526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe108106f10 msgr2=0x7fe108074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:03.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe106ffd640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe108106f10 0x7fe108074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.063+0000 7fe106ffd640 1 -- 172.21.15.1:0/2812087526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1081a68f0 con 0x7fe108102790 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.064+0000 7fe106ffd640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108102790 0x7fe108074a30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000d950 tx=0x7fe0f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.064+0000 7fe104ff9640 1 -- 172.21.15.1:0/2812087526 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0017070 con 0x7fe108102790 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.064+0000 7fe104ff9640 1 -- 172.21.15.1:0/2812087526 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f0009c30 con 0x7fe108102790 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.064+0000 7fe10e22c640 1 -- 172.21.15.1:0/2812087526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1081a6be0 con 0x7fe108102790 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.064+0000 7fe104ff9640 1 -- 172.21.15.1:0/2812087526 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000f6b0 con 0x7fe108102790 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.064+0000 7fe10e22c640 1 -- 172.21.15.1:0/2812087526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1081a7120 con 0x7fe108102790 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe104ff9640 1 -- 172.21.15.1:0/2812087526 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe0f00109d0 con 0x7fe108102790 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe110605740 1 Processor -- start 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe110605740 1 -- start start 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe110605740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db49b40 0x559d0db48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe110605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 0x559d0db48a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe110605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db55760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe110605740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559d0daae2f0 con 0x559d0db49b40 2024-09-16T00:34:03.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe110605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d0da55cf0 con 0x559d0db53370 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe110605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559d0daaa2b0 con 0x559d0db485d0 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe107fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db49b40 0x559d0db48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe107fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db49b40 0x559d0db48090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38872/0 (socket says 172.21.15.1:38872) 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe107fff640 1 -- 172.21.15.1:0/2234294793 learned_addr learned my addr 172.21.15.1:0/2234294793 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe106ffd640 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db55760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.067+0000 7fe1077fe640 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 0x559d0db48a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe107fff640 1 -- 172.21.15.1:0/2234294793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 msgr2=0x559d0db48a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe107fff640 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 0x559d0db48a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe107fff640 1 -- 172.21.15.1:0/2234294793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 msgr2=0x559d0db55760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe107fff640 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db55760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe107fff640 1 -- 172.21.15.1:0/2234294793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0da3a3a0 con 0x559d0db49b40 2024-09-16T00:34:03.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe106ffd640 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe104ff9640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe0f40a3240 0x7fe0f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe107fff640 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db49b40 0x559d0db48090 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe100009b90 tx=0x7fe1000117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f4bcfe5f784a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe1077fe640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe0f40a3240 0x7fe0f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe104ff9640 1 -- 172.21.15.1:0/2812087526 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe0f00cf940 con 0x7fe108102790 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe0fdffb640 1 -- 172.21.15.1:0/2234294793 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100012450 con 0x559d0db49b40 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe0fdffb640 1 -- 172.21.15.1:0/2234294793 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe100012ad0 con 0x559d0db49b40 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.068+0000 7fe0fdffb640 1 -- 172.21.15.1:0/2234294793 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10000fa50 con 0x559d0db49b40 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 -- 172.21.15.1:0/2234294793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db49b40 msgr2=0x559d0db48090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe1077fe640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe0f40a3240 0x7fe0f40a5700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8006010 tx=0x7fe0f803d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db49b40 0x559d0db48090 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe100009b90 tx=0x7fe1000117b0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 -- 172.21.15.1:0/2234294793 shutdown_connections 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 0x559d0db48a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 --2- 172.21.15.1:0/2234294793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db49b40 0x559d0db48090 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 -- 172.21.15.1:0/2234294793 >> 172.21.15.1:0/2234294793 conn(0x559d0db11310 msgr2=0x559d0db44c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 -- 172.21.15.1:0/2234294793 shutdown_connections 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 -- 172.21.15.1:0/2234294793 wait complete. 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 -- start start 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.069+0000 7fe110605740 1 Processor -- start 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe110605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db485d0 0x559d0db52d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe110605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 0x559d0db4ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe110605740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe110605740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559d0db56070 con 0x559d0db53370 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe110605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d0da69060 con 0x559d0db485d0 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe110605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559d0db561f0 con 0x559d0db49b40 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe110605740 10 client.0 _ll_register_callbacks cb 0x559d0d7bc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:03.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe107fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe1077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db485d0 0x559d0db52d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe1077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db485d0 0x559d0db52d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:42370/0 (socket says 172.21.15.1:42370) 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe1077fe640 1 -- 172.21.15.1:0/1404911323 learned_addr learned my addr 172.21.15.1:0/1404911323 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe107fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db50560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38880/0 (socket says 172.21.15.1:38880) 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe106ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 0x559d0db4ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe107fff640 1 -- 172.21.15.1:0/1404911323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 msgr2=0x559d0db4ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe107fff640 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 0x559d0db4ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe107fff640 1 -- 172.21.15.1:0/1404911323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db485d0 msgr2=0x559d0db52d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe107fff640 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db485d0 0x559d0db52d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.070+0000 7fe107fff640 1 -- 172.21.15.1:0/1404911323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0d81f3c0 con 0x559d0db53370 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe107fff640 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db50560 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe100009b60 tx=0x7fe100012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74eb1f001d0acb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe0e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:03.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/1404911323 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100029020 con 0x559d0db53370 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/1404911323 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe10000f9e0 con 0x559d0db53370 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe110605740 1 -- 172.21.15.1:0/1404911323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe100012ad0 con 0x559d0db53370 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/1404911323 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe100034050 con 0x559d0db53370 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe110605740 10 client.16650 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe110605740 1 -- 172.21.15.1:0/1404911323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d0db61ac0 con 0x559d0db53370 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/1404911323 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe100012ad0 con 0x559d0db53370 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/1404911323 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe1000046c0 con 0x559d0db53370 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe110605740 10 client.16650 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe110605740 10 client.16650 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe110605740 20 client.16650 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe110605740 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x559d0db62f20 0x559d0db65310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe110605740 4 client.16650 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.071+0000 7fe110605740 1 -- 172.21.15.1:0/1404911323 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d0dabb860 con 0x559d0db62f20 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.072+0000 7fe1077fe640 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x559d0db62f20 0x559d0db65310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.072+0000 7fe0e6ffd640 10 client.16650 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.072+0000 7fe1077fe640 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x559d0db62f20 0x559d0db65310 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a17bb0b4f94673f server_cookie=5bd9dd8489563a9d in_seq=0 out_seq=0 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/1404911323 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d0dabb860 con 0x559d0db62f20 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe0e6ffd640 10 client.16650 handle_command_reply: tid=0 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 client.16650 shutdown 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 -- 172.21.15.1:0/1404911323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 msgr2=0x559d0db50560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db50560 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe100009b60 tx=0x7fe100012a00 comp rx=0 tx=0).stop 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 -- 172.21.15.1:0/1404911323 shutdown_connections 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x559d0db62f20 0x559d0db65310 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db50560 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 0x559d0db4ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 --2- 172.21.15.1:0/1404911323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db485d0 0x559d0db52d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe0e6ffd640 0 client.16650 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:03.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 -- 172.21.15.1:0/1404911323 >> 172.21.15.1:0/1404911323 conn(0x559d0db11310 msgr2=0x559d0db4a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:03.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 -- 172.21.15.1:0/1404911323 shutdown_connections 2024-09-16T00:34:03.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 1 -- 172.21.15.1:0/1404911323 wait complete. 2024-09-16T00:34:03.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.073+0000 7fe110605740 20 client.16650 trim_cache size 0 max 16384 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.086+0000 7fe110605740 1 Processor -- start 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe110605740 1 -- start start 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe110605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 0x559d0db5f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe110605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db49b40 0x559d0db4b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe110605740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db4b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe110605740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081b3a10 con 0x559d0db53370 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe110605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081b12e0 con 0x559d0db49b40 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe110605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081af130 con 0x559d0db485d0 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe107fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db4b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe107fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db4b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38896/0 (socket says 172.21.15.1:38896) 2024-09-16T00:34:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe107fff640 1 -- 172.21.15.1:0/2919836347 learned_addr learned my addr 172.21.15.1:0/2919836347 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe1077fe640 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 0x559d0db5f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe106ffd640 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db49b40 0x559d0db4b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe107fff640 1 -- 172.21.15.1:0/2919836347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 msgr2=0x559d0db5f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe107fff640 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 0x559d0db5f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe107fff640 1 -- 172.21.15.1:0/2919836347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db49b40 msgr2=0x559d0db4b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe107fff640 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db49b40 0x559d0db4b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe107fff640 1 -- 172.21.15.1:0/2919836347 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0db52930 con 0x559d0db53370 2024-09-16T00:34:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe106ffd640 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db49b40 0x559d0db4b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.087+0000 7fe107fff640 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db4b660 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe100006010 tx=0x7fe100012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa66830fb6ce6ce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe0fd7fa640 1 -- 172.21.15.1:0/2919836347 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100029040 con 0x559d0db53370 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe0fd7fa640 1 -- 172.21.15.1:0/2919836347 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe100011cb0 con 0x559d0db53370 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe0fd7fa640 1 -- 172.21.15.1:0/2919836347 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10000fe30 con 0x559d0db53370 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe110605740 1 -- 172.21.15.1:0/2919836347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 msgr2=0x559d0db4b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe110605740 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db4b660 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe100006010 tx=0x7fe100012920 comp rx=0 tx=0).stop 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe110605740 1 -- 172.21.15.1:0/2919836347 shutdown_connections 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe110605740 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db53370 0x559d0db4b660 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe110605740 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db49b40 0x559d0db4b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe110605740 1 --2- 172.21.15.1:0/2919836347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db485d0 0x559d0db5f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe110605740 1 -- 172.21.15.1:0/2919836347 >> 172.21.15.1:0/2919836347 conn(0x559d0db11310 msgr2=0x559d0db4f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe110605740 1 -- 172.21.15.1:0/2919836347 shutdown_connections 2024-09-16T00:34:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.088+0000 7fe110605740 1 -- 172.21.15.1:0/2919836347 wait complete. 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe110605740 1 -- start start 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe110605740 1 Processor -- start 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe110605740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db485d0 0x559d0db5baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe110605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 0x559d0db4cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe110605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db4d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe110605740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081af7a0 con 0x559d0db485d0 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe110605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081b0c10 con 0x559d0db53370 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe110605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081afe70 con 0x559d0db49b40 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe110605740 10 client.0 _ll_register_callbacks cb 0x7fe100009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe1077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db485d0 0x559d0db5baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe1077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db485d0 0x559d0db5baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38900/0 (socket says 172.21.15.1:38900) 2024-09-16T00:34:03.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe1077fe640 1 -- 172.21.15.1:0/3670656342 learned_addr learned my addr 172.21.15.1:0/3670656342 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.089+0000 7fe107fff640 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db4d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe106ffd640 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 0x559d0db4cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe1077fe640 1 -- 172.21.15.1:0/3670656342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 msgr2=0x559d0db4cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe1077fe640 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 0x559d0db4cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe1077fe640 1 -- 172.21.15.1:0/3670656342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 msgr2=0x559d0db4d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe1077fe640 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db4d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe1077fe640 1 -- 172.21.15.1:0/3670656342 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0db4d8d0 con 0x559d0db485d0 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe107fff640 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db4d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe106ffd640 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 0x559d0db4cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe1077fe640 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db485d0 0x559d0db5baa0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009430 tx=0x7fe0f802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267fe42c92bb6527 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe0e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/3670656342 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8057070 con 0x559d0db485d0 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe110605740 1 -- 172.21.15.1:0/3670656342 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe100011cb0 con 0x559d0db485d0 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/3670656342 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe0f80305c0 con 0x559d0db485d0 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/3670656342 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0f80307c0 con 0x559d0db485d0 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe110605740 10 client.16656 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe110605740 1 -- 172.21.15.1:0/3670656342 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d0dbcd890 con 0x559d0db485d0 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.090+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/3670656342 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe100011cb0 con 0x559d0db485d0 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/3670656342 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe0f8053a40 con 0x559d0db485d0 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe110605740 10 client.16656 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe110605740 10 client.16656 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:34:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe110605740 20 client.16656 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe110605740 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x559d0dbcdf90 0x559d0dbd03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe110605740 4 client.16656 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe110605740 1 -- 172.21.15.1:0/3670656342 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d0dab5d70 con 0x559d0dbcdf90 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe106ffd640 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x559d0dbcdf90 0x559d0dbd03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe106ffd640 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x559d0dbcdf90 0x559d0dbd03b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9b00d7463c9e08c server_cookie=33ab8e90b98c2137 in_seq=0 out_seq=0 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.091+0000 7fe0e6ffd640 10 client.16656 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe0e6ffd640 1 -- 172.21.15.1:0/3670656342 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d0dab5d70 con 0x559d0dbcdf90 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe0e6ffd640 10 client.16656 handle_command_reply: tid=0 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 client.16656 shutdown 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 -- 172.21.15.1:0/3670656342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db485d0 msgr2=0x559d0db5baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db485d0 0x559d0db5baa0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009430 tx=0x7fe0f802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 -- 172.21.15.1:0/3670656342 shutdown_connections 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0db53370 0x559d0db4d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d0db49b40 0x559d0db4cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x559d0dbcdf90 0x559d0dbd03b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 --2- 172.21.15.1:0/3670656342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0db485d0 0x559d0db5baa0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 -- 172.21.15.1:0/3670656342 >> 172.21.15.1:0/3670656342 conn(0x559d0db11310 msgr2=0x559d0db51820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:03.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe0e6ffd640 0 client.16656 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:03.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 -- 172.21.15.1:0/3670656342 shutdown_connections 2024-09-16T00:34:03.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 1 -- 172.21.15.1:0/3670656342 wait complete. 2024-09-16T00:34:03.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe110605740 20 client.16656 trim_cache size 0 max 16384 2024-09-16T00:34:03.102 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:34:03.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.092+0000 7fe10e22c640 1 -- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe0f40a3240 msgr2=0x7fe0f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:03.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe0f40a3240 0x7fe0f40a5700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8006010 tx=0x7fe0f803d040 comp rx=0 tx=0).stop 2024-09-16T00:34:03.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 -- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108102790 msgr2=0x7fe108074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:03.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108102790 0x7fe108074a30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000d950 tx=0x7fe0f000de10 comp rx=0 tx=0).stop 2024-09-16T00:34:03.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 -- 172.21.15.1:0/2812087526 shutdown_connections 2024-09-16T00:34:03.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe108106f10 0x7fe108074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe108102790 0x7fe108074a30 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe0f40a3240 0x7fe0f40a5700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 --2- 172.21.15.1:0/2812087526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe108101de0 0x7fe1080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:03.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 -- 172.21.15.1:0/2812087526 >> 172.21.15.1:0/2812087526 conn(0x7fe1080fd8c0 msgr2=0x7fe1080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:03.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 -- 172.21.15.1:0/2812087526 shutdown_connections 2024-09-16T00:34:03.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:03.093+0000 7fe10e22c640 1 -- 172.21.15.1:0/2812087526 wait complete. 2024-09-16T00:34:03.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:03 smithi001 ceph-mon[30934]: pgmap v520: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-16T00:34:03.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:03 smithi019 ceph-mon[35186]: pgmap v520: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-16T00:34:03.924 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:34:03.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:03 smithi153 ceph-mon[35538]: pgmap v520: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-16T00:34:04.182 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 -- 172.21.15.1:0/2954664815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28102570 msgr2=0x7fdf281029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/2954664815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28102570 0x7fdf281029b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009900 tx=0x7fdf1002a480 comp rx=0 tx=0).stop 2024-09-16T00:34:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 -- 172.21.15.1:0/2954664815 shutdown_connections 2024-09-16T00:34:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/2954664815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102ef0 0x7fdf2810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/2954664815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28102570 0x7fdf281029b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/2954664815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf28100e20 0x7fdf28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 -- 172.21.15.1:0/2954664815 >> 172.21.15.1:0/2954664815 conn(0x7fdf280fc7d0 msgr2=0x7fdf280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 -- 172.21.15.1:0/2954664815 shutdown_connections 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 -- 172.21.15.1:0/2954664815 wait complete. 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.698+0000 7fdf2dd76640 1 Processor -- start 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf2dd76640 1 -- start start 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf2dd76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28100e20 0x7fdf280fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf2dd76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102570 0x7fdf280fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf2dd76640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf28102ef0 0x7fdf280fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf2dd76640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28110f80 con 0x7fdf28100e20 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf2dd76640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28110e00 con 0x7fdf28102570 2024-09-16T00:34:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf2dd76640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28111100 con 0x7fdf28102ef0 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28100e20 0x7fdf280fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28100e20 0x7fdf280fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38914/0 (socket says 172.21.15.1:38914) 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf277fe640 1 -- 172.21.15.1:0/391536553 learned_addr learned my addr 172.21.15.1:0/391536553 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf27fff640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf28102ef0 0x7fdf280fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf1effd640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102570 0x7fdf280fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf277fe640 1 -- 172.21.15.1:0/391536553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf28102ef0 msgr2=0x7fdf280fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf277fe640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf28102ef0 0x7fdf280fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf277fe640 1 -- 172.21.15.1:0/391536553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102570 msgr2=0x7fdf280fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf277fe640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102570 0x7fdf280fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf277fe640 1 -- 172.21.15.1:0/391536553 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf281a69c0 con 0x7fdf28100e20 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.699+0000 7fdf1effd640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102570 0x7fdf280fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.700+0000 7fdf277fe640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28100e20 0x7fdf280fcc80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e9f0 tx=0x7fdf1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.700+0000 7fdf257fa640 1 -- 172.21.15.1:0/391536553 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800cd80 con 0x7fdf28100e20 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.700+0000 7fdf257fa640 1 -- 172.21.15.1:0/391536553 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf18005500 con 0x7fdf28100e20 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.700+0000 7fdf2dd76640 1 -- 172.21.15.1:0/391536553 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf281a6cb0 con 0x7fdf28100e20 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.700+0000 7fdf257fa640 1 -- 172.21.15.1:0/391536553 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18010690 con 0x7fdf28100e20 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.700+0000 7fdf2dd76640 1 -- 172.21.15.1:0/391536553 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf281a71a0 con 0x7fdf28100e20 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.702+0000 7fdf257fa640 1 -- 172.21.15.1:0/391536553 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fdf180046c0 con 0x7fdf28100e20 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf3014f740 1 Processor -- start 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf3014f740 1 -- start start 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f456783b40 0x55f456782090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456782a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f45678d370 0x55f45678f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf3014f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f4566e82f0 con 0x55f4567825d0 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf3014f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f45668fcf0 con 0x55f456783b40 2024-09-16T00:34:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf3014f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f4566e42b0 con 0x55f45678d370 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456782a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456782a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38918/0 (socket says 172.21.15.1:38918) 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf277fe640 1 -- 172.21.15.1:0/3752553269 learned_addr learned my addr 172.21.15.1:0/3752553269 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf27fff640 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f456783b40 0x55f456782090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.703+0000 7fdf1effd640 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f45678d370 0x55f45678f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf257fa640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdf140a32c0 0x7fdf140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf277fe640 1 -- 172.21.15.1:0/3752553269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f45678d370 msgr2=0x55f45678f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf277fe640 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f45678d370 0x55f45678f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf277fe640 1 -- 172.21.15.1:0/3752553269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f456783b40 msgr2=0x55f456782090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf277fe640 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f456783b40 0x55f456782090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf277fe640 1 -- 172.21.15.1:0/3752553269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4566743a0 con 0x55f4567825d0 2024-09-16T00:34:04.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf27fff640 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f456783b40 0x55f456782090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf27fff640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdf140a32c0 0x7fdf140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf257fa640 1 -- 172.21.15.1:0/391536553 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdf180d01a0 con 0x7fdf28100e20 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf277fe640 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456782a10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdf1807dab0 tx=0x7fdf18066db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de67d1f56a912fe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf1dffb640 1 -- 172.21.15.1:0/3752553269 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1809bea0 con 0x55f4567825d0 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf27fff640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdf140a32c0 0x7fdf140a5780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005fd0 tx=0x7fdf20004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf1dffb640 1 -- 172.21.15.1:0/3752553269 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf180672c0 con 0x55f4567825d0 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.704+0000 7fdf1dffb640 1 -- 172.21.15.1:0/3752553269 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180d3cd0 con 0x55f4567825d0 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 -- 172.21.15.1:0/3752553269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 msgr2=0x55f456782a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456782a10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdf1807dab0 tx=0x7fdf18066db0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 -- 172.21.15.1:0/3752553269 shutdown_connections 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f45678d370 0x55f45678f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456782a10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3752553269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f456783b40 0x55f456782090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 -- 172.21.15.1:0/3752553269 >> 172.21.15.1:0/3752553269 conn(0x55f45674b310 msgr2=0x55f45677ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 -- 172.21.15.1:0/3752553269 shutdown_connections 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 -- 172.21.15.1:0/3752553269 wait complete. 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.705+0000 7fdf3014f740 1 -- start start 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf3014f740 1 Processor -- start 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 0x55f45678cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456789f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f45678a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf3014f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f456790010 con 0x55f45678d370 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf3014f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4566a3060 con 0x55f4567825d0 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf3014f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f456790190 con 0x55f456783b40 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf3014f740 10 client.0 _ll_register_callbacks cb 0x55f4563f68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:04.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f45678a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f45678a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38928/0 (socket says 172.21.15.1:38928) 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf27fff640 1 -- 172.21.15.1:0/3885885255 learned_addr learned my addr 172.21.15.1:0/3885885255 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf277fe640 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 0x55f45678cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf1effd640 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456789f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf27fff640 1 -- 172.21.15.1:0/3885885255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 msgr2=0x55f456789f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf27fff640 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456789f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf27fff640 1 -- 172.21.15.1:0/3885885255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 msgr2=0x55f45678cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf27fff640 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 0x55f45678cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf27fff640 1 -- 172.21.15.1:0/3885885255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4564593c0 con 0x55f45678d370 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.706+0000 7fdf1effd640 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456789f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf27fff640 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f45678a480 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009d80 tx=0x7fdf20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15f430ba4db9aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf06ffd640 1 -- 172.21.15.1:0/3885885255 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20011ac0 con 0x55f45678d370 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf06ffd640 1 -- 172.21.15.1:0/3885885255 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdf20011c60 con 0x55f45678d370 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf3014f740 1 -- 172.21.15.1:0/3885885255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf180672c0 con 0x55f45678d370 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf06ffd640 1 -- 172.21.15.1:0/3885885255 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdf20030050 con 0x55f45678d370 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf3014f740 10 client.16668 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf3014f740 1 -- 172.21.15.1:0/3885885255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f45679ba90 con 0x55f45678d370 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf06ffd640 1 -- 172.21.15.1:0/3885885255 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf180672c0 con 0x55f45678d370 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf06ffd640 1 -- 172.21.15.1:0/3885885255 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdf2001f540 con 0x55f45678d370 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf3014f740 10 client.16668 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf3014f740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf3014f740 20 client.16668 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f45679cf50 0x55f45679f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf3014f740 4 client.16668 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.707+0000 7fdf3014f740 1 -- 172.21.15.1:0/3885885255 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4566f5860 con 0x55f45679cf50 2024-09-16T00:34:04.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.708+0000 7fdf277fe640 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f45679cf50 0x55f45679f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.708+0000 7fdf277fe640 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f45679cf50 0x55f45679f3d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e271351d638381 server_cookie=31184b3a1f85226c in_seq=0 out_seq=0 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.708+0000 7fdf06ffd640 10 client.16668 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf06ffd640 1 -- 172.21.15.1:0/3885885255 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4566f5860 con 0x55f45679cf50 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf06ffd640 10 client.16668 handle_command_reply: tid=0 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 client.16668 shutdown 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 -- 172.21.15.1:0/3885885255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 msgr2=0x55f45678a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f45678a480 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009d80 tx=0x7fdf20004ab0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 -- 172.21.15.1:0/3885885255 shutdown_connections 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f45679cf50 0x55f45679f3d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f45678a480 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456789f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 --2- 172.21.15.1:0/3885885255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 0x55f45678cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf06ffd640 0 client.16668 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:04.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 -- 172.21.15.1:0/3885885255 >> 172.21.15.1:0/3885885255 conn(0x55f45674b310 msgr2=0x55f456784930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:04.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 -- 172.21.15.1:0/3885885255 shutdown_connections 2024-09-16T00:34:04.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 1 -- 172.21.15.1:0/3885885255 wait complete. 2024-09-16T00:34:04.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.709+0000 7fdf3014f740 20 client.16668 trim_cache size 0 max 16384 2024-09-16T00:34:04.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.722+0000 7fdf3014f740 1 Processor -- start 2024-09-16T00:34:04.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.722+0000 7fdf3014f740 1 -- start start 2024-09-16T00:34:04.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 0x55f456799310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456785120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f456785660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf3014f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281b3990 con 0x55f45678d370 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf3014f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281b1260 con 0x55f4567825d0 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf3014f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281af0b0 con 0x55f456783b40 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f456785660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f456785660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38934/0 (socket says 172.21.15.1:38934) 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf27fff640 1 -- 172.21.15.1:0/1729895384 learned_addr learned my addr 172.21.15.1:0/1729895384 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf277fe640 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 0x55f456799310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf1effd640 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456785120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf27fff640 1 -- 172.21.15.1:0/1729895384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 msgr2=0x55f456785120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf27fff640 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456785120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf27fff640 1 -- 172.21.15.1:0/1729895384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 msgr2=0x55f456799310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf27fff640 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 0x55f456799310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf27fff640 1 -- 172.21.15.1:0/1729895384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f456788490 con 0x55f45678d370 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf277fe640 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 0x55f456799310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.723+0000 7fdf27fff640 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f456785660 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fdf20000f80 tx=0x7fdf20009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc42a38404f3a50c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf1d7fa640 1 -- 172.21.15.1:0/1729895384 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2001fda0 con 0x55f45678d370 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf1d7fa640 1 -- 172.21.15.1:0/1729895384 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2002d070 con 0x55f45678d370 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf1d7fa640 1 -- 172.21.15.1:0/1729895384 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20028680 con 0x55f45678d370 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf3014f740 1 -- 172.21.15.1:0/1729895384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 msgr2=0x55f456785660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf3014f740 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f456785660 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fdf20000f80 tx=0x7fdf20009c10 comp rx=0 tx=0).stop 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf3014f740 1 -- 172.21.15.1:0/1729895384 shutdown_connections 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf3014f740 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f45678d370 0x55f456785660 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf3014f740 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456785120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf3014f740 1 --2- 172.21.15.1:0/1729895384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4567825d0 0x55f456799310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.724+0000 7fdf3014f740 1 -- 172.21.15.1:0/1729895384 >> 172.21.15.1:0/1729895384 conn(0x55f45674b310 msgr2=0x55f456789f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:04.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 -- 172.21.15.1:0/1729895384 shutdown_connections 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 -- 172.21.15.1:0/1729895384 wait complete. 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 -- start start 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 Processor -- start 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456795bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456790f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f45678d370 0x55f456791480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281af720 con 0x55f4567825d0 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281b0b90 con 0x55f45678d370 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281afdf0 con 0x55f456783b40 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.725+0000 7fdf3014f740 10 client.0 _ll_register_callbacks cb 0x7fdf20009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456795bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456795bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38950/0 (socket says 172.21.15.1:38950) 2024-09-16T00:34:04.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf277fe640 1 -- 172.21.15.1:0/2556735693 learned_addr learned my addr 172.21.15.1:0/2556735693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:04.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf1effd640 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456790f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf27fff640 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f45678d370 0x55f456791480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf277fe640 1 -- 172.21.15.1:0/2556735693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 msgr2=0x55f456790f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:04.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf277fe640 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456790f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf277fe640 1 -- 172.21.15.1:0/2556735693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f45678d370 msgr2=0x55f456791480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf277fe640 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f45678d370 0x55f456791480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf277fe640 1 -- 172.21.15.1:0/2556735693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f456791cb0 con 0x55f4567825d0 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf1effd640 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456790f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf277fe640 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456795bc0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdf1809c2f0 tx=0x7fdf180bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5af1c7b00557bb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf06ffd640 1 -- 172.21.15.1:0/2556735693 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1807ce60 con 0x55f4567825d0 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf06ffd640 1 -- 172.21.15.1:0/2556735693 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdf180b0540 con 0x55f4567825d0 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf3014f740 1 -- 172.21.15.1:0/2556735693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf2002d070 con 0x55f4567825d0 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf06ffd640 1 -- 172.21.15.1:0/2556735693 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdf1806c100 con 0x55f4567825d0 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf3014f740 10 client.16674 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:04.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.726+0000 7fdf3014f740 1 -- 172.21.15.1:0/2556735693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f456807cb0 con 0x55f4567825d0 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.727+0000 7fdf06ffd640 1 -- 172.21.15.1:0/2556735693 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2002d070 con 0x55f4567825d0 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.727+0000 7fdf06ffd640 1 -- 172.21.15.1:0/2556735693 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdf180989d0 con 0x55f4567825d0 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.727+0000 7fdf3014f740 10 client.16674 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.727+0000 7fdf3014f740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.727+0000 7fdf3014f740 20 client.16674 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.727+0000 7fdf3014f740 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f456807f40 0x55f45680a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.727+0000 7fdf3014f740 4 client.16674 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.727+0000 7fdf3014f740 1 -- 172.21.15.1:0/2556735693 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4566efd70 con 0x55f456807f40 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.727+0000 7fdf1effd640 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f456807f40 0x55f45680a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf1effd640 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f456807f40 0x55f45680a330 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714ae0a9109d284e server_cookie=7c6914733bd8cbbf in_seq=0 out_seq=0 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf06ffd640 10 client.16674 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf06ffd640 1 -- 172.21.15.1:0/2556735693 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4566efd70 con 0x55f456807f40 2024-09-16T00:34:04.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf06ffd640 10 client.16674 handle_command_reply: tid=0 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 client.16674 shutdown 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 -- 172.21.15.1:0/2556735693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 msgr2=0x55f456795bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456795bc0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdf1809c2f0 tx=0x7fdf180bc040 comp rx=0 tx=0).stop 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 -- 172.21.15.1:0/2556735693 shutdown_connections 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f456807f40 0x55f45680a330 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f45678d370 0x55f456791480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f456783b40 0x55f456790f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 --2- 172.21.15.1:0/2556735693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4567825d0 0x55f456795bc0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 -- 172.21.15.1:0/2556735693 >> 172.21.15.1:0/2556735693 conn(0x55f45674b310 msgr2=0x55f45678b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf06ffd640 0 client.16674 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 -- 172.21.15.1:0/2556735693 shutdown_connections 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 1 -- 172.21.15.1:0/2556735693 wait complete. 2024-09-16T00:34:04.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.728+0000 7fdf3014f740 20 client.16674 trim_cache size 0 max 16384 2024-09-16T00:34:04.740 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:34:04.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 -- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdf140a32c0 msgr2=0x7fdf140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdf140a32c0 0x7fdf140a5780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005fd0 tx=0x7fdf20004a40 comp rx=0 tx=0).stop 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 -- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28100e20 msgr2=0x7fdf280fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28100e20 0x7fdf280fcc80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e9f0 tx=0x7fdf1800edf0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 -- 172.21.15.1:0/391536553 shutdown_connections 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdf140a32c0 0x7fdf140a5780 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf28102ef0 0x7fdf280fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102570 0x7fdf280fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 --2- 172.21.15.1:0/391536553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf28100e20 0x7fdf280fcc80 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 -- 172.21.15.1:0/391536553 >> 172.21.15.1:0/391536553 conn(0x7fdf280fc7d0 msgr2=0x7fdf2806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 -- 172.21.15.1:0/391536553 shutdown_connections 2024-09-16T00:34:04.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:04.729+0000 7fdf2dd76640 1 -- 172.21.15.1:0/391536553 wait complete. 2024-09-16T00:34:05.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:05 smithi019 ceph-mon[35186]: pgmap v521: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-16T00:34:05.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:05 smithi001 ceph-mon[30934]: pgmap v521: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-16T00:34:05.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:05 smithi153 ceph-mon[35538]: pgmap v521: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-16T00:34:06.556 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:34:06.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:34:06.825 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:07.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.336+0000 7f40a0d88640 1 -- 172.21.15.1:0/2965877161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 msgr2=0x7f409c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.336+0000 7f40a0d88640 1 --2- 172.21.15.1:0/2965877161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 0x7f409c10ea90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4094009ca0 tx=0x7f409402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.336+0000 7f40a0d88640 1 -- 172.21.15.1:0/2965877161 shutdown_connections 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.336+0000 7f40a0d88640 1 --2- 172.21.15.1:0/2965877161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 0x7f409c10ea90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.336+0000 7f40a0d88640 1 --2- 172.21.15.1:0/2965877161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c102790 0x7f409c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.336+0000 7f40a0d88640 1 --2- 172.21.15.1:0/2965877161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c101de0 0x7f409c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.336+0000 7f40a0d88640 1 -- 172.21.15.1:0/2965877161 >> 172.21.15.1:0/2965877161 conn(0x7f409c0fd8c0 msgr2=0x7f409c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.336+0000 7f40a0d88640 1 -- 172.21.15.1:0/2965877161 shutdown_connections 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.336+0000 7f40a0d88640 1 -- 172.21.15.1:0/2965877161 wait complete. 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.337+0000 7f40a0d88640 1 Processor -- start 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.337+0000 7f40a0d88640 1 -- start start 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f40a0d88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c101de0 0x7f409c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f40a0d88640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c102790 0x7f409c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f40a0d88640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 0x7f409c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f40a0d88640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f409c111f70 con 0x7f409c106f10 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f40a0d88640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f409c111df0 con 0x7f409c101de0 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f40a0d88640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1120f0 con 0x7f409c102790 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 0x7f409c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 0x7f409c19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38978/0 (socket says 172.21.15.1:38978) 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409ad9d640 1 -- 172.21.15.1:0/3208803576 learned_addr learned my addr 172.21.15.1:0/3208803576 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409a59c640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c101de0 0x7f409c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f4099d9b640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c102790 0x7f409c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409ad9d640 1 -- 172.21.15.1:0/3208803576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c102790 msgr2=0x7f409c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409ad9d640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c102790 0x7f409c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409ad9d640 1 -- 172.21.15.1:0/3208803576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c101de0 msgr2=0x7f409c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:07.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409ad9d640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c101de0 0x7f409c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409ad9d640 1 -- 172.21.15.1:0/3208803576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c19d290 con 0x7f409c106f10 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f409ad9d640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 0x7f409c19cbb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4094031a60 tx=0x7f4094031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.338+0000 7f408b7fe640 1 -- 172.21.15.1:0/3208803576 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094003b70 con 0x7f409c106f10 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.339+0000 7f408b7fe640 1 -- 172.21.15.1:0/3208803576 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4094003d10 con 0x7f409c106f10 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.339+0000 7f408b7fe640 1 -- 172.21.15.1:0/3208803576 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094004770 con 0x7f409c106f10 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.339+0000 7f40a0d88640 1 -- 172.21.15.1:0/3208803576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c19d520 con 0x7f409c106f10 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.339+0000 7f40a0d88640 1 -- 172.21.15.1:0/3208803576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a4e60 con 0x7f409c106f10 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.341+0000 7f408b7fe640 1 -- 172.21.15.1:0/3208803576 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f409402f790 con 0x7f409c106f10 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.341+0000 7f40a3161740 1 Processor -- start 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.341+0000 7f40a3161740 1 -- start start 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f40a3161740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43f9b60 0x55c3d43f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f40a3161740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43fa480 0x55c3d4404800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43f9b60 0x55c3d43f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43f9b60 0x55c3d43f9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38992/0 (socket says 172.21.15.1:38992) 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f4099d9b640 1 -- 172.21.15.1:0/2507992014 learned_addr learned my addr 172.21.15.1:0/2507992014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f408b7fe640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40840a3240 0x7f40840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f409a59c640 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43fa480 0x55c3d4404800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f408b7fe640 1 -- 172.21.15.1:0/3208803576 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f40940f10a0 con 0x7f409c106f10 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f40a3161740 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d4404d40 0x55c3d4407130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f40a3161740 1 -- 172.21.15.1:0/2507992014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c3d435f280 con 0x55c3d43f9b60 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f40a3161740 1 -- 172.21.15.1:0/2507992014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c3d4306b10 con 0x55c3d4404d40 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.342+0000 7f40a3161740 1 -- 172.21.15.1:0/2507992014 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c3d435b240 con 0x55c3d43fa480 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f4099d9b640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40840a3240 0x7f40840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f409ad9d640 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d4404d40 0x55c3d4407130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f409a59c640 1 -- 172.21.15.1:0/2507992014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d4404d40 msgr2=0x55c3d4407130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f409a59c640 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d4404d40 0x55c3d4407130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f409a59c640 1 -- 172.21.15.1:0/2507992014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43f9b60 msgr2=0x55c3d43f9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f409a59c640 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43f9b60 0x55c3d43f9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f409a59c640 1 -- 172.21.15.1:0/2507992014 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d43131f0 con 0x55c3d43fa480 2024-09-16T00:34:07.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f4099d9b640 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43f9b60 0x55c3d43f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:07.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f409a59c640 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43fa480 0x55c3d4404800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f408c00abf0 tx=0x7f408c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25d7e2a76963e509 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:07.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f4099d9b640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40840a3240 0x7f40840a5700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4090004650 tx=0x7f4090004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:07.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f4088ff9640 1 -- 172.21.15.1:0/2507992014 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c005220 con 0x55c3d43fa480 2024-09-16T00:34:07.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f4088ff9640 1 -- 172.21.15.1:0/2507992014 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408c004c80 con 0x55c3d43fa480 2024-09-16T00:34:07.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.343+0000 7f4088ff9640 1 -- 172.21.15.1:0/2507992014 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c018830 con 0x55c3d43fa480 2024-09-16T00:34:07.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 -- 172.21.15.1:0/2507992014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43fa480 msgr2=0x55c3d4404800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:07.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43fa480 0x55c3d4404800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f408c00abf0 tx=0x7f408c00f790 comp rx=0 tx=0).stop 2024-09-16T00:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 -- 172.21.15.1:0/2507992014 shutdown_connections 2024-09-16T00:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d4404d40 0x55c3d4407130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43fa480 0x55c3d4404800 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 --2- 172.21.15.1:0/2507992014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43f9b60 0x55c3d43f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 -- 172.21.15.1:0/2507992014 >> 172.21.15.1:0/2507992014 conn(0x55c3d43f60f0 msgr2=0x55c3d43f64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 -- 172.21.15.1:0/2507992014 shutdown_connections 2024-09-16T00:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 -- 172.21.15.1:0/2507992014 wait complete. 2024-09-16T00:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.344+0000 7f40a3161740 1 -- start start 2024-09-16T00:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f40a3161740 1 Processor -- start 2024-09-16T00:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f40a3161740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43f9b60 0x55c3d4404800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f40a3161740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43fa480 0x55c3d4401b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f40a3161740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d4404d40 0x55c3d44022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f40a3161740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c3d4407910 con 0x55c3d4404d40 2024-09-16T00:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f40a3161740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c3d4319f10 con 0x55c3d43fa480 2024-09-16T00:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f40a3161740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c3d4407a90 con 0x55c3d43f9b60 2024-09-16T00:34:07.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f40a3161740 10 client.0 _ll_register_callbacks cb 0x55c3d42e2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:07.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d4404d40 0x55c3d44022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d4404d40 0x55c3d44022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38998/0 (socket says 172.21.15.1:38998) 2024-09-16T00:34:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409ad9d640 1 -- 172.21.15.1:0/3294721647 learned_addr learned my addr 172.21.15.1:0/3294721647 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409a59c640 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43f9b60 0x55c3d4404800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f4099d9b640 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43fa480 0x55c3d4401b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409ad9d640 1 -- 172.21.15.1:0/3294721647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43f9b60 msgr2=0x55c3d4404800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409ad9d640 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43f9b60 0x55c3d4404800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409ad9d640 1 -- 172.21.15.1:0/3294721647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43fa480 msgr2=0x55c3d4401b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409ad9d640 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43fa480 0x55c3d4401b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409ad9d640 1 -- 172.21.15.1:0/3294721647 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d40d06a0 con 0x55c3d4404d40 2024-09-16T00:34:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f409ad9d640 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d4404d40 0x55c3d44022e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4094031b50 tx=0x7f409402ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55528c36679dc95f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.345+0000 7f4081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f4081ffb640 1 -- 172.21.15.1:0/3294721647 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40940d6060 con 0x55c3d4404d40 2024-09-16T00:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f4081ffb640 1 -- 172.21.15.1:0/3294721647 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f40940bb560 con 0x55c3d4404d40 2024-09-16T00:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f40a3161740 1 -- 172.21.15.1:0/3294721647 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408c004c80 con 0x55c3d4404d40 2024-09-16T00:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f4081ffb640 1 -- 172.21.15.1:0/3294721647 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40940bd260 con 0x55c3d4404d40 2024-09-16T00:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f40a3161740 10 client.16686 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f40a3161740 1 -- 172.21.15.1:0/3294721647 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c3d4407910 con 0x55c3d4404d40 2024-09-16T00:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f4081ffb640 1 -- 172.21.15.1:0/3294721647 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408c004c80 con 0x55c3d4404d40 2024-09-16T00:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f4081ffb640 1 -- 172.21.15.1:0/3294721647 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f40940dd0f0 con 0x55c3d4404d40 2024-09-16T00:34:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f40a3161740 10 client.16686 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f40a3161740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f40a3161740 20 client.16686 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f40a3161740 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c3d4418cb0 0x55c3d441b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f40a3161740 4 client.16686 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:07.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f40a3161740 1 -- 172.21.15.1:0/3294721647 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3d436c7f0 con 0x55c3d4418cb0 2024-09-16T00:34:07.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.346+0000 7f409a59c640 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c3d4418cb0 0x55c3d441b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.347+0000 7f409a59c640 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c3d4418cb0 0x55c3d441b120 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73540e8fd9c1a42b server_cookie=fdae36117a0aef4 in_seq=0 out_seq=0 2024-09-16T00:34:07.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.347+0000 7f4081ffb640 10 client.16686 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:07.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f4081ffb640 1 -- 172.21.15.1:0/3294721647 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3d436c7f0 con 0x55c3d4418cb0 2024-09-16T00:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f4081ffb640 10 client.16686 handle_command_reply: tid=0 2024-09-16T00:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 client.16686 shutdown 2024-09-16T00:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 -- 172.21.15.1:0/3294721647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d4404d40 msgr2=0x55c3d44022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d4404d40 0x55c3d44022e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4094031b50 tx=0x7f409402ff50 comp rx=0 tx=0).stop 2024-09-16T00:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 -- 172.21.15.1:0/3294721647 shutdown_connections 2024-09-16T00:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c3d4418cb0 0x55c3d441b120 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d4404d40 0x55c3d44022e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43fa480 0x55c3d4401b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 --2- 172.21.15.1:0/3294721647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d43f9b60 0x55c3d4404800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 -- 172.21.15.1:0/3294721647 >> 172.21.15.1:0/3294721647 conn(0x55c3d43f60f0 msgr2=0x55c3d43fbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f4081ffb640 0 client.16686 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 -- 172.21.15.1:0/3294721647 shutdown_connections 2024-09-16T00:34:07.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 1 -- 172.21.15.1:0/3294721647 wait complete. 2024-09-16T00:34:07.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.348+0000 7f40a3161740 20 client.16686 trim_cache size 0 max 16384 2024-09-16T00:34:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.365+0000 7f40a3161740 1 Processor -- start 2024-09-16T00:34:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.365+0000 7f40a3161740 1 -- start start 2024-09-16T00:34:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.365+0000 7f40a3161740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 0x55c3d43fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.365+0000 7f40a3161740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d4420a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.365+0000 7f40a3161740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 0x55c3d4420f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.365+0000 7f40a3161740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1af420 con 0x55c3d43fa480 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.365+0000 7f40a3161740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1accf0 con 0x55c3d43f9b60 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.365+0000 7f40a3161740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1aab40 con 0x55c3d4404d40 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d4420a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d4420a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39006/0 (socket says 172.21.15.1:39006) 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4099d9b640 1 -- 172.21.15.1:0/2246076823 learned_addr learned my addr 172.21.15.1:0/2246076823 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f409a59c640 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 0x55c3d43fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f409ad9d640 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 0x55c3d4420f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4099d9b640 1 -- 172.21.15.1:0/2246076823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 msgr2=0x55c3d4420f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4099d9b640 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 0x55c3d4420f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4099d9b640 1 -- 172.21.15.1:0/2246076823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 msgr2=0x55c3d43fc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4099d9b640 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 0x55c3d43fc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4099d9b640 1 -- 172.21.15.1:0/2246076823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d43f1830 con 0x55c3d43fa480 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f409ad9d640 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 0x55c3d4420f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4099d9b640 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d4420a30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4090005940 tx=0x7f409000c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259b1920bcb18ec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4083fff640 1 -- 172.21.15.1:0/2246076823 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090018070 con 0x55c3d43fa480 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4083fff640 1 -- 172.21.15.1:0/2246076823 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4090024070 con 0x55c3d43fa480 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.366+0000 7f4083fff640 1 -- 172.21.15.1:0/2246076823 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40900209b0 con 0x55c3d43fa480 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 -- 172.21.15.1:0/2246076823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 msgr2=0x55c3d4420a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d4420a30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4090005940 tx=0x7f409000c5d0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 -- 172.21.15.1:0/2246076823 shutdown_connections 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 0x55c3d4420f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d4420a30 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 --2- 172.21.15.1:0/2246076823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 0x55c3d43fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 -- 172.21.15.1:0/2246076823 >> 172.21.15.1:0/2246076823 conn(0x55c3d43f60f0 msgr2=0x55c3d4400420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 -- 172.21.15.1:0/2246076823 shutdown_connections 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 -- 172.21.15.1:0/2246076823 wait complete. 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 -- start start 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.367+0000 7f40a3161740 1 Processor -- start 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f40a3161740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 0x55c3d440d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f40a3161740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d440dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f40a3161740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 0x55c3d4482a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f40a3161740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ab1b0 con 0x55c3d43fa480 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f40a3161740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ac620 con 0x55c3d43f9b60 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f40a3161740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ab880 con 0x55c3d4404d40 2024-09-16T00:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f40a3161740 10 client.0 _ll_register_callbacks cb 0x7f4094004910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d440dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d440dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39010/0 (socket says 172.21.15.1:39010) 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4099d9b640 1 -- 172.21.15.1:0/4164479981 learned_addr learned my addr 172.21.15.1:0/4164479981 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f409a59c640 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 0x55c3d440d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f409ad9d640 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 0x55c3d4482a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4099d9b640 1 -- 172.21.15.1:0/4164479981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 msgr2=0x55c3d4482a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4099d9b640 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 0x55c3d4482a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4099d9b640 1 -- 172.21.15.1:0/4164479981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 msgr2=0x55c3d440d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4099d9b640 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 0x55c3d440d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4099d9b640 1 -- 172.21.15.1:0/4164479981 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d43fa2b0 con 0x55c3d43fa480 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4099d9b640 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d440dba0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4090004600 tx=0x7f409000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d971a35a017f816 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4081ffb640 1 -- 172.21.15.1:0/4164479981 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090018040 con 0x55c3d43fa480 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f4081ffb640 1 -- 172.21.15.1:0/4164479981 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4090005220 con 0x55c3d43fa480 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.368+0000 7f40a3161740 1 -- 172.21.15.1:0/4164479981 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4090024070 con 0x55c3d43fa480 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f4081ffb640 1 -- 172.21.15.1:0/4164479981 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4090005420 con 0x55c3d43fa480 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f40a3161740 10 client.16692 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f40a3161740 1 -- 172.21.15.1:0/4164479981 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c3d4484070 con 0x55c3d43fa480 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f4081ffb640 1 -- 172.21.15.1:0/4164479981 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4090024070 con 0x55c3d43fa480 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f4081ffb640 1 -- 172.21.15.1:0/4164479981 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4090029470 con 0x55c3d43fa480 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f40a3161740 10 client.16692 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f40a3161740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f40a3161740 20 client.16692 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f40a3161740 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c3d4484760 0x55c3d4486b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f40a3161740 4 client.16692 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f40a3161740 1 -- 172.21.15.1:0/4164479981 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c3d4366d00 con 0x55c3d4484760 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.369+0000 7f409a59c640 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c3d4484760 0x55c3d4486b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.370+0000 7f409a59c640 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c3d4484760 0x55c3d4486b80 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c3b3b1458733c4 server_cookie=e7fdaa81dd974e08 in_seq=0 out_seq=0 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.370+0000 7f4081ffb640 10 client.16692 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f4081ffb640 1 -- 172.21.15.1:0/4164479981 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c3d4366d00 con 0x55c3d4484760 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f4081ffb640 10 client.16692 handle_command_reply: tid=0 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 client.16692 shutdown 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 -- 172.21.15.1:0/4164479981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 msgr2=0x55c3d440dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d440dba0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4090004600 tx=0x7f409000cae0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 -- 172.21.15.1:0/4164479981 shutdown_connections 2024-09-16T00:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c3d4484760 0x55c3d4486b80 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3d4404d40 0x55c3d4482a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3d43fa480 0x55c3d440dba0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 --2- 172.21.15.1:0/4164479981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3d43f9b60 0x55c3d440d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 -- 172.21.15.1:0/4164479981 >> 172.21.15.1:0/4164479981 conn(0x55c3d43f60f0 msgr2=0x55c3d4414810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f4081ffb640 0 client.16692 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 -- 172.21.15.1:0/4164479981 shutdown_connections 2024-09-16T00:34:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 1 -- 172.21.15.1:0/4164479981 wait complete. 2024-09-16T00:34:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.371+0000 7f40a3161740 20 client.16692 trim_cache size 0 max 16384 2024-09-16T00:34:07.382 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:34:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:34:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "b9e69fdf-212d-4992-ae9b-b21bfaa42bc0", 2024-09-16T00:34:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:34:07.382 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 -- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40840a3240 msgr2=0x7f40840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40840a3240 0x7f40840a5700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4090004650 tx=0x7f4090004730 comp rx=0 tx=0).stop 2024-09-16T00:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 -- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 msgr2=0x7f409c19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 0x7f409c19cbb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4094031a60 tx=0x7f4094031a90 comp rx=0 tx=0).stop 2024-09-16T00:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 -- 172.21.15.1:0/3208803576 shutdown_connections 2024-09-16T00:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f409c106f10 0x7f409c19cbb0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f40840a3240 0x7f40840a5700 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c102790 0x7f409c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 --2- 172.21.15.1:0/3208803576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c101de0 0x7f409c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 -- 172.21.15.1:0/3208803576 >> 172.21.15.1:0/3208803576 conn(0x7f409c0fd8c0 msgr2=0x7f409c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 -- 172.21.15.1:0/3208803576 shutdown_connections 2024-09-16T00:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:07.372+0000 7f40a0d88640 1 -- 172.21.15.1:0/3208803576 wait complete. 2024-09-16T00:34:07.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:34:07.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:07 smithi019 ceph-mon[35186]: pgmap v522: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T00:34:07.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:07 smithi001 ceph-mon[30934]: pgmap v522: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T00:34:07.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:07 smithi153 ceph-mon[35538]: pgmap v522: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T00:34:08.209 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:08.225 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:34:08.775 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:08 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:34:08.775 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:08 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:08.775 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:08 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:34:08.775 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:08 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:08.776 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:08 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:34:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 -- 172.21.15.1:0/2683649345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee841008d0 msgr2=0x7fee84100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 --2- 172.21.15.1:0/2683649345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee841008d0 0x7fee84100cb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fee74009ca0 tx=0x7fee7402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:34:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 -- 172.21.15.1:0/2683649345 shutdown_connections 2024-09-16T00:34:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 --2- 172.21.15.1:0/2683649345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8410e480 0x7fee84110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 --2- 172.21.15.1:0/2683649345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee84101280 0x7fee8410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 --2- 172.21.15.1:0/2683649345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee841008d0 0x7fee84100cb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 -- 172.21.15.1:0/2683649345 >> 172.21.15.1:0/2683649345 conn(0x7fee840fc7a0 msgr2=0x7fee840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 -- 172.21.15.1:0/2683649345 shutdown_connections 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 -- 172.21.15.1:0/2683649345 wait complete. 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 Processor -- start 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.770+0000 7fee8b84c640 1 -- start start 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee8b84c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee841008d0 0x7fee8419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee8b84c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee84101280 0x7fee8419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee8b84c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee8410e480 0x7fee8419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee8b84c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fee84114190 con 0x7fee84101280 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee8b84c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee84114010 con 0x7fee841008d0 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee8b84c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee84114310 con 0x7fee8410e480 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee88de7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee84101280 0x7fee8419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee88de7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee84101280 0x7fee8419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39038/0 (socket says 172.21.15.1:39038) 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee88de7640 1 -- 172.21.15.1:0/1842944303 learned_addr learned my addr 172.21.15.1:0/1842944303 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee89de9640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee8410e480 0x7fee8419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:08.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee895e8640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee841008d0 0x7fee8419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee88de7640 1 -- 172.21.15.1:0/1842944303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee8410e480 msgr2=0x7fee8419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee88de7640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee8410e480 0x7fee8419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee88de7640 1 -- 172.21.15.1:0/1842944303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee841008d0 msgr2=0x7fee8419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee88de7640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee841008d0 0x7fee8419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee88de7640 1 -- 172.21.15.1:0/1842944303 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8419f620 con 0x7fee84101280 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee89de9640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee8410e480 0x7fee8419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee895e8640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee841008d0 0x7fee8419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee88de7640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee84101280 0x7fee8419abb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fee7800d950 tx=0x7fee7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee827fc640 1 -- 172.21.15.1:0/1842944303 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78017070 con 0x7fee84101280 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee827fc640 1 -- 172.21.15.1:0/1842944303 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee78009c30 con 0x7fee84101280 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8419f910 con 0x7fee84101280 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.771+0000 7fee827fc640 1 -- 172.21.15.1:0/1842944303 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7800f6b0 con 0x7fee84101280 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.772+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee841a71f0 con 0x7fee84101280 2024-09-16T00:34:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.773+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee84100cb0 con 0x7fee84101280 2024-09-16T00:34:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.774+0000 7fee827fc640 1 -- 172.21.15.1:0/1842944303 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fee780109d0 con 0x7fee84101280 2024-09-16T00:34:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.775+0000 7fee827fc640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fee6c0a32c0 0x7fee6c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.775+0000 7fee895e8640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fee6c0a32c0 0x7fee6c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.775+0000 7fee827fc640 1 -- 172.21.15.1:0/1842944303 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fee780cf9b0 con 0x7fee84101280 2024-09-16T00:34:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.775+0000 7fee895e8640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fee6c0a32c0 0x7fee6c0a5780 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee74006010 tx=0x7fee740093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:08.779+0000 7fee827fc640 1 -- 172.21.15.1:0/1842944303 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee78009de0 con 0x7fee84101280 2024-09-16T00:34:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:08 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:34:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:08 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:08 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:34:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:08 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:08 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:34:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:08.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:08 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:34:08.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:08 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:08.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:08 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:34:08.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:08 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:08.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:08 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:34:09.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.009+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee8402d7c0 con 0x7fee84101280 2024-09-16T00:34:09.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.009+0000 7fee827fc640 1 -- 172.21.15.1:0/1842944303 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fee7809c2d0 con 0x7fee84101280 2024-09-16T00:34:09.010 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:34:09.010 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:34:09.010 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:34:09.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fee6c0a32c0 msgr2=0x7fee6c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fee6c0a32c0 0x7fee6c0a5780 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee74006010 tx=0x7fee740093f0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee84101280 msgr2=0x7fee8419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee84101280 0x7fee8419abb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fee7800d950 tx=0x7fee7800de10 comp rx=0 tx=0).stop 2024-09-16T00:34:09.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 shutdown_connections 2024-09-16T00:34:09.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fee6c0a32c0 0x7fee6c0a5780 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee8410e480 0x7fee8419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee84101280 0x7fee8419abb0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 --2- 172.21.15.1:0/1842944303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee841008d0 0x7fee8419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 >> 172.21.15.1:0/1842944303 conn(0x7fee840fc7a0 msgr2=0x7fee840feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:09.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 shutdown_connections 2024-09-16T00:34:09.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.012+0000 7fee8b84c640 1 -- 172.21.15.1:0/1842944303 wait complete. 2024-09-16T00:34:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.576+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/2494612401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 msgr2=0x7fc6e8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.576+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/2494612401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 0x7fc6e8107b90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009930 tx=0x7fc6d802a420 comp rx=0 tx=0).stop 2024-09-16T00:34:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.576+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/2494612401 shutdown_connections 2024-09-16T00:34:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.576+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/2494612401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6e81082a0 0x7fc6e810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.576+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/2494612401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 0x7fc6e8107b90 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.576+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/2494612401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e8102d30 0x7fc6e8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.576+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/2494612401 >> 172.21.15.1:0/2494612401 conn(0x7fc6e80fc7d0 msgr2=0x7fc6e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:09.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.576+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/2494612401 shutdown_connections 2024-09-16T00:34:09.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.576+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/2494612401 wait complete. 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ef2cb640 1 Processor -- start 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ef2cb640 1 -- start start 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ef2cb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6e8102d30 0x7fc6e8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ef2cb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 0x7fc6e8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ef2cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e81082a0 0x7fc6e819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ef2cb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81120c0 con 0x7fc6e81036e0 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ef2cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8111f40 con 0x7fc6e81082a0 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ef2cb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8112240 con 0x7fc6e8102d30 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ec866640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 0x7fc6e8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ec866640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 0x7fc6e8198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39054/0 (socket says 172.21.15.1:39054) 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ec866640 1 -- 172.21.15.1:0/3120889047 learned_addr learned my addr 172.21.15.1:0/3120889047 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ed868640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e81082a0 0x7fc6e819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ed067640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6e8102d30 0x7fc6e8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ec866640 1 -- 172.21.15.1:0/3120889047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6e8102d30 msgr2=0x7fc6e8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ec866640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6e8102d30 0x7fc6e8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ec866640 1 -- 172.21.15.1:0/3120889047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e81082a0 msgr2=0x7fc6e819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ec866640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e81082a0 0x7fc6e819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ec866640 1 -- 172.21.15.1:0/3120889047 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e819d5f0 con 0x7fc6e81036e0 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.577+0000 7fc6ed067640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6e8102d30 0x7fc6e8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.578+0000 7fc6ec866640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 0x7fc6e8198b80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8005940 tx=0x7fc6d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.578+0000 7fc6de7fc640 1 -- 172.21.15.1:0/3120889047 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d803f030 con 0x7fc6e81036e0 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.578+0000 7fc6de7fc640 1 -- 172.21.15.1:0/3120889047 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d8003ce0 con 0x7fc6e81036e0 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.578+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/3120889047 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e819d880 con 0x7fc6e81036e0 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.578+0000 7fc6de7fc640 1 -- 172.21.15.1:0/3120889047 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8035070 con 0x7fc6e81036e0 2024-09-16T00:34:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.578+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/3120889047 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e81a51c0 con 0x7fc6e81036e0 2024-09-16T00:34:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.581+0000 7fc6f16a4740 1 Processor -- start 2024-09-16T00:34:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.581+0000 7fc6de7fc640 1 -- 172.21.15.1:0/3120889047 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc6d8042030 con 0x7fc6e81036e0 2024-09-16T00:34:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.581+0000 7fc6f16a4740 1 -- start start 2024-09-16T00:34:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458b380 0x55ff0458b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 0x55ff0459a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0459aa90 0x55ff0459ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ff044f12e0 con 0x55ff0459aa90 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff04498ce0 con 0x55ff0458bca0 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ff044ed2a0 con 0x55ff0458b380 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ec866640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0459aa90 0x55ff0459ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ec866640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0459aa90 0x55ff0459ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39056/0 (socket says 172.21.15.1:39056) 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ec866640 1 -- 172.21.15.1:0/3621767892 learned_addr learned my addr 172.21.15.1:0/3621767892 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ed868640 1 --2- 172.21.15.1:0/3621767892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458b380 0x55ff0458b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ed067640 1 --2- 172.21.15.1:0/3621767892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 0x55ff0459a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ec866640 1 -- 172.21.15.1:0/3621767892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458b380 msgr2=0x55ff0458b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ec866640 1 --2- 172.21.15.1:0/3621767892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458b380 0x55ff0458b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ec866640 1 -- 172.21.15.1:0/3621767892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 msgr2=0x55ff0459a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ec866640 1 --2- 172.21.15.1:0/3621767892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 0x55ff0459a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6de7fc640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6d40a3240 0x7fc6d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ec866640 1 -- 172.21.15.1:0/3621767892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0447d440 con 0x55ff0459aa90 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ed868640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6d40a3240 0x7fc6d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.582+0000 7fc6ec866640 1 --2- 172.21.15.1:0/3621767892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0459aa90 0x55ff0459ce80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80f80e0 tx=0x7fc6d80b83a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760cd115aeb473b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.583+0000 7fc6de7fc640 1 -- 172.21.15.1:0/3120889047 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc6d80efee0 con 0x7fc6e81036e0 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.583+0000 7fc6d37fe640 1 -- 172.21.15.1:0/3621767892 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8049020 con 0x55ff0459aa90 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.583+0000 7fc6d37fe640 1 -- 172.21.15.1:0/3621767892 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d80bc800 con 0x55ff0459aa90 2024-09-16T00:34:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.583+0000 7fc6d37fe640 1 -- 172.21.15.1:0/3621767892 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80bbed0 con 0x55ff0459aa90 2024-09-16T00:34:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.583+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3621767892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0459aa90 msgr2=0x55ff0459ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.583+0000 7fc6ed868640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6d40a3240 0x7fc6d40a5700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000af10 tx=0x7fc6e000a4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.583+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3621767892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0459aa90 0x55ff0459ce80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80f80e0 tx=0x7fc6d80b83a0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3621767892 shutdown_connections 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3621767892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0459aa90 0x55ff0459ce80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3621767892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 0x55ff0459a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3621767892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458b380 0x55ff0458b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3621767892 >> 172.21.15.1:0/3621767892 conn(0x55ff04587a30 msgr2=0x55ff04587e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3621767892 shutdown_connections 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3621767892 wait complete. 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 -- start start 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 Processor -- start 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.584+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 0x55ff04597680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0459aa90 0x55ff04597bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ff0459d420 con 0x55ff0458b380 2024-09-16T00:34:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff044ac050 con 0x55ff0459aa90 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ff0459d5a0 con 0x55ff0458bca0 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6f16a4740 10 client.0 _ll_register_callbacks cb 0x55ff044ec160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed067640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed067640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39060/0 (socket says 172.21.15.1:39060) 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed067640 1 -- 172.21.15.1:0/3567075684 learned_addr learned my addr 172.21.15.1:0/3567075684 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed868640 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0459aa90 0x55ff04597bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ec866640 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 0x55ff04597680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed067640 1 -- 172.21.15.1:0/3567075684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 msgr2=0x55ff04597680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed067640 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 0x55ff04597680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed067640 1 -- 172.21.15.1:0/3567075684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0459aa90 msgr2=0x55ff04597bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed067640 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0459aa90 0x55ff04597bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed067640 1 -- 172.21.15.1:0/3567075684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff042623c0 con 0x55ff0458b380 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6ed067640 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459a3d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400f570 tx=0x7fc6e400f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc2c2106f9e5ea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/3567075684 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4013850 con 0x55ff0458b380 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/3567075684 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc6e40139f0 con 0x55ff0458b380 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.585+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3567075684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6d80bc800 con 0x55ff0458b380 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/3567075684 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc6e4024050 con 0x55ff0458b380 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6f16a4740 10 client.16710 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3567075684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ff045a8db0 con 0x55ff0458b380 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/3567075684 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d80bc800 con 0x55ff0458b380 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/3567075684 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc6e40113e0 con 0x55ff0458b380 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6f16a4740 10 client.16710 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6f16a4740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6f16a4740 20 client.16710 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ff045aa270 0x55ff045ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6f16a4740 4 client.16710 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3567075684 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff044fe850 con 0x55ff045aa270 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.586+0000 7fc6ec866640 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ff045aa270 0x55ff045ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.587+0000 7fc6d0ff9640 10 client.16710 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.587+0000 7fc6ec866640 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ff045aa270 0x55ff045ac6c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef10364c667c9ef8 server_cookie=1f26f7d9721852fa in_seq=0 out_seq=0 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.587+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/3567075684 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff044fe850 con 0x55ff045aa270 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.587+0000 7fc6d0ff9640 10 client.16710 handle_command_reply: tid=0 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.587+0000 7fc6f16a4740 1 client.16710 shutdown 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3567075684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 msgr2=0x55ff0459a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459a3d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400f570 tx=0x7fc6e400f970 comp rx=0 tx=0).stop 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3567075684 shutdown_connections 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ff045aa270 0x55ff045ac6c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0459aa90 0x55ff04597bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 0x55ff04597680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/3567075684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459a3d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3567075684 >> 172.21.15.1:0/3567075684 conn(0x55ff04587a30 msgr2=0x55ff0458d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6d0ff9640 0 client.16710 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3567075684 shutdown_connections 2024-09-16T00:34:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 1 -- 172.21.15.1:0/3567075684 wait complete. 2024-09-16T00:34:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.588+0000 7fc6f16a4740 20 client.16710 trim_cache size 0 max 16384 2024-09-16T00:34:09.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6f16a4740 1 Processor -- start 2024-09-16T00:34:09.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6f16a4740 1 -- start start 2024-09-16T00:34:09.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff04587a30 0x55ff045a6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff04595f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 0x55ff04586a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81af650 con 0x55ff0458b380 2024-09-16T00:34:09.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81acf20 con 0x55ff0458bca0 2024-09-16T00:34:09.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81aad70 con 0x55ff04587a30 2024-09-16T00:34:09.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6ec866640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff04595f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6ec866640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff04595f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39072/0 (socket says 172.21.15.1:39072) 2024-09-16T00:34:09.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.605+0000 7fc6ec866640 1 -- 172.21.15.1:0/1746538469 learned_addr learned my addr 172.21.15.1:0/1746538469 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:09.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6ed067640 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff04587a30 0x55ff045a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6ed868640 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 0x55ff04586a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6ec866640 1 -- 172.21.15.1:0/1746538469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff04587a30 msgr2=0x55ff045a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:09.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6ec866640 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff04587a30 0x55ff045a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6ec866640 1 -- 172.21.15.1:0/1746538469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 msgr2=0x55ff04586a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:09.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6ec866640 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 0x55ff04586a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6ec866640 1 -- 172.21.15.1:0/1746538469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff04596630 con 0x55ff0458b380 2024-09-16T00:34:09.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6ed868640 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 0x55ff04586a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:09.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6ec866640 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff04595f50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8000c00 tx=0x7fc6d80f39c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5309d4be08e7514 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:09.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6d2ffd640 1 -- 172.21.15.1:0/1746538469 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8049040 con 0x55ff0458b380 2024-09-16T00:34:09.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6d2ffd640 1 -- 172.21.15.1:0/1746538469 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d80bc800 con 0x55ff0458b380 2024-09-16T00:34:09.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.606+0000 7fc6d2ffd640 1 -- 172.21.15.1:0/1746538469 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80bba60 con 0x55ff0458b380 2024-09-16T00:34:09.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 -- 172.21.15.1:0/1746538469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 msgr2=0x55ff04595f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff04595f50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8000c00 tx=0x7fc6d80f39c0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 -- 172.21.15.1:0/1746538469 shutdown_connections 2024-09-16T00:34:09.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff0458bca0 0x55ff04586a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff04595f50 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/1746538469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff04587a30 0x55ff045a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 -- 172.21.15.1:0/1746538469 >> 172.21.15.1:0/1746538469 conn(0x55ff04597d20 msgr2=0x55ff04586350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:09.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 -- 172.21.15.1:0/1746538469 shutdown_connections 2024-09-16T00:34:09.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 -- 172.21.15.1:0/1746538469 wait complete. 2024-09-16T00:34:09.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 -- start start 2024-09-16T00:34:09.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.607+0000 7fc6f16a4740 1 Processor -- start 2024-09-16T00:34:09.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff04587a30 0x55ff0459e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6f16a4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 0x55ff0459f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81ab3e0 con 0x55ff0458b380 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81ac850 con 0x55ff04587a30 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6f16a4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81abab0 con 0x55ff0458bca0 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6f16a4740 10 client.0 _ll_register_callbacks cb 0x7fc6e400a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ec866640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ec866640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39074/0 (socket says 172.21.15.1:39074) 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ec866640 1 -- 172.21.15.1:0/179431737 learned_addr learned my addr 172.21.15.1:0/179431737 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ed067640 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff04587a30 0x55ff0459e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ed868640 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 0x55ff0459f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ec866640 1 -- 172.21.15.1:0/179431737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 msgr2=0x55ff0459f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ec866640 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 0x55ff0459f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ec866640 1 -- 172.21.15.1:0/179431737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff04587a30 msgr2=0x55ff0459e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ec866640 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff04587a30 0x55ff0459e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ec866640 1 -- 172.21.15.1:0/179431737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff04614230 con 0x55ff0458b380 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6ec866640 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459ed70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80bc090 tx=0x7fc6d802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d01a59f3a3ade03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.608+0000 7fc6d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/179431737 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8049050 con 0x55ff0458b380 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/179431737 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc6d80ba400 con 0x55ff0458b380 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6f16a4740 1 -- 172.21.15.1:0/179431737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6d80bc800 con 0x55ff0458b380 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/179431737 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc6d80bbc40 con 0x55ff0458b380 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6f16a4740 10 client.16716 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6f16a4740 1 -- 172.21.15.1:0/179431737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ff046154b0 con 0x55ff0458b380 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/179431737 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d80bc800 con 0x55ff0458b380 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/179431737 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc6d80b8770 con 0x55ff0458b380 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6f16a4740 10 client.16716 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6f16a4740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6f16a4740 20 client.16716 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:09.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ff0459aa90 0x55ff04617850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6f16a4740 4 client.16716 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6f16a4740 1 -- 172.21.15.1:0/179431737 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff044f8d60 con 0x55ff0459aa90 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.609+0000 7fc6ed067640 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ff0459aa90 0x55ff04617850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6ed067640 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ff0459aa90 0x55ff04617850 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75127313c83a401d server_cookie=96497fef8db8ef6e in_seq=0 out_seq=0 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6d0ff9640 10 client.16716 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6d0ff9640 1 -- 172.21.15.1:0/179431737 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ff044f8d60 con 0x55ff0459aa90 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6d0ff9640 10 client.16716 handle_command_reply: tid=0 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 client.16716 shutdown 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 -- 172.21.15.1:0/179431737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 msgr2=0x55ff0459ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459ed70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80bc090 tx=0x7fc6d802a420 comp rx=0 tx=0).stop 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 -- 172.21.15.1:0/179431737 shutdown_connections 2024-09-16T00:34:09.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ff0459aa90 0x55ff04617850 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff0458bca0 0x55ff0459f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ff0458b380 0x55ff0459ed70 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 --2- 172.21.15.1:0/179431737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff04587a30 0x55ff0459e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 -- 172.21.15.1:0/179431737 >> 172.21.15.1:0/179431737 conn(0x55ff04597d20 msgr2=0x55ff04586350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6d0ff9640 0 client.16716 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 -- 172.21.15.1:0/179431737 shutdown_connections 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 1 -- 172.21.15.1:0/179431737 wait complete. 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.610+0000 7fc6f16a4740 20 client.16716 trim_cache size 0 max 16384 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:34:09.618 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:34:09.619 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:34:09.619 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:34:09.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6d40a3240 msgr2=0x7fc6d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6d40a3240 0x7fc6d40a5700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000af10 tx=0x7fc6e000a4d0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 msgr2=0x7fc6e8198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:09.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 0x7fc6e8198b80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8005940 tx=0x7fc6d80042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/3120889047 shutdown_connections 2024-09-16T00:34:09.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e81082a0 0x7fc6e819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e81036e0 0x7fc6e8198b80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc6d40a3240 0x7fc6d40a5700 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 --2- 172.21.15.1:0/3120889047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6e8102d30 0x7fc6e8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:09.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/3120889047 >> 172.21.15.1:0/3120889047 conn(0x7fc6e80fc7d0 msgr2=0x7fc6e8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:09.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/3120889047 shutdown_connections 2024-09-16T00:34:09.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:09.611+0000 7fc6ef2cb640 1 -- 172.21.15.1:0/3120889047 wait complete. 2024-09-16T00:34:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:09 smithi019 ceph-mon[35186]: pgmap v523: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T00:34:09.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:09 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1842944303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:09.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:09 smithi001 ceph-mon[30934]: pgmap v523: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T00:34:09.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:09 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1842944303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:09.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:09 smithi153 ceph-mon[35538]: pgmap v523: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T00:34:09.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:09 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1842944303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:10.159 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 8e4ce410 --release --await' 2024-09-16T00:34:10.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 8e4ce410 --release --await 2024-09-16T00:34:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:10.922 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:34:10.923 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.404+0000 7f5bd94f5640 1 -- 172.21.15.1:0/2086780927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100480 msgr2=0x7f5bd4100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.404+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/2086780927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100480 0x7f5bd4100860 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009900 tx=0x7f5bc402a410 comp rx=0 tx=0).stop 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.404+0000 7f5bd94f5640 1 -- 172.21.15.1:0/2086780927 shutdown_connections 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.404+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/2086780927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bd41055b0 0x7f5bd410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.404+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/2086780927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bd4100e30 0x7f5bd4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.404+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/2086780927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100480 0x7f5bd4100860 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.404+0000 7f5bd94f5640 1 -- 172.21.15.1:0/2086780927 >> 172.21.15.1:0/2086780927 conn(0x7f5bd4077210 msgr2=0x7f5bd40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.404+0000 7f5bd94f5640 1 -- 172.21.15.1:0/2086780927 shutdown_connections 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.404+0000 7f5bd94f5640 1 -- 172.21.15.1:0/2086780927 wait complete. 2024-09-16T00:34:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd94f5640 1 Processor -- start 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd94f5640 1 -- start start 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd94f5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bd4100480 0x7f5bd4196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd94f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100e30 0x7f5bd4196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd94f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bd41055b0 0x7f5bd4196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd94f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd410fd90 con 0x7f5bd4100e30 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd94f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd410fc10 con 0x7f5bd41055b0 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd94f5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd410ff10 con 0x7f5bd4100480 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100e30 0x7f5bd4196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100e30 0x7f5bd4196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56136/0 (socket says 172.21.15.1:56136) 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd27fc640 1 -- 172.21.15.1:0/397444396 learned_addr learned my addr 172.21.15.1:0/397444396 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd37fe640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bd41055b0 0x7f5bd4196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.405+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bd4100480 0x7f5bd4196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bd27fc640 1 -- 172.21.15.1:0/397444396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bd4100480 msgr2=0x7f5bd4196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bd4100480 0x7f5bd4196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bd27fc640 1 -- 172.21.15.1:0/397444396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bd41055b0 msgr2=0x7f5bd4196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bd41055b0 0x7f5bd4196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bd27fc640 1 -- 172.21.15.1:0/397444396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd41a68c0 con 0x7f5bd4100e30 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100e30 0x7f5bd4196560 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800d950 tx=0x7f5bc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bbbfff640 1 -- 172.21.15.1:0/397444396 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8017030 con 0x7f5bd4100e30 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bbbfff640 1 -- 172.21.15.1:0/397444396 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc8004020 con 0x7f5bd4100e30 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bbbfff640 1 -- 172.21.15.1:0/397444396 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80047a0 con 0x7f5bd4100e30 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bd94f5640 1 -- 172.21.15.1:0/397444396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd41a6bb0 con 0x7f5bd4100e30 2024-09-16T00:34:11.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.406+0000 7f5bd94f5640 1 -- 172.21.15.1:0/397444396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd41a70f0 con 0x7f5bd4100e30 2024-09-16T00:34:11.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bdb8ce740 1 Processor -- start 2024-09-16T00:34:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bdb8ce740 1 -- start start 2024-09-16T00:34:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bbbfff640 1 -- 172.21.15.1:0/397444396 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f5bc80041d0 con 0x7f5bd4100e30 2024-09-16T00:34:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 0x56001d328440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d332d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 0x56001d335630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56001d28d620 con 0x56001d328980 2024-09-16T00:34:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56001d234e50 con 0x56001d333240 2024-09-16T00:34:11.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56001d2895e0 con 0x56001d328060 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bd2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d332d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bd2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d332d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56146/0 (socket says 172.21.15.1:56146) 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bd2ffd640 1 -- 172.21.15.1:0/3285035481 learned_addr learned my addr 172.21.15.1:0/3285035481 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 0x56001d335630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.409+0000 7f5bd37fe640 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 0x56001d328440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bd2ffd640 1 -- 172.21.15.1:0/3285035481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 msgr2=0x56001d328440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 0x56001d328440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bd2ffd640 1 -- 172.21.15.1:0/3285035481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 msgr2=0x56001d335630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 0x56001d335630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bd2ffd640 1 -- 172.21.15.1:0/3285035481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56001d225d40 con 0x56001d328980 2024-09-16T00:34:11.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 0x56001d335630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bbbfff640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5bbc0a3240 0x7f5bbc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d332d00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009900 tx=0x7f5bc4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86151b860f31dbd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bbbfff640 1 -- 172.21.15.1:0/397444396 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5bc80cf6e0 con 0x7f5bd4100e30 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bb97fa640 1 -- 172.21.15.1:0/3285035481 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc403f070 con 0x56001d328980 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bd37fe640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5bbc0a3240 0x7f5bbc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bb97fa640 1 -- 172.21.15.1:0/3285035481 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc4004030 con 0x56001d328980 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.410+0000 7f5bb97fa640 1 -- 172.21.15.1:0/3285035481 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4035070 con 0x56001d328980 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/3285035481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 msgr2=0x56001d332d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bd37fe640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5bbc0a3240 0x7f5bbc0a5700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005fd0 tx=0x7f5bcc00a280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d332d00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009900 tx=0x7f5bc4005e80 comp rx=0 tx=0).stop 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/3285035481 shutdown_connections 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 0x56001d335630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d332d00 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/3285035481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 0x56001d328440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/3285035481 >> 172.21.15.1:0/3285035481 conn(0x56001d3245f0 msgr2=0x56001d3249d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/3285035481 shutdown_connections 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.411+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/3285035481 wait complete. 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bdb8ce740 1 -- start start 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bdb8ce740 1 Processor -- start 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d3300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d3305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 0x56001d330b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56001d335e10 con 0x56001d328980 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56001d215f10 con 0x56001d328060 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56001d335f90 con 0x56001d333240 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bdb8ce740 10 client.0 _ll_register_callbacks cb 0x56001d239db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bd27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d3305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bd27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d3305e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56148/0 (socket says 172.21.15.1:56148) 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.412+0000 7f5bd27fc640 1 -- 172.21.15.1:0/328322129 learned_addr learned my addr 172.21.15.1:0/328322129 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d3300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bd27fc640 1 -- 172.21.15.1:0/328322129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 msgr2=0x56001d330b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bd37fe640 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 0x56001d330b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 0x56001d330b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bd27fc640 1 -- 172.21.15.1:0/328322129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 msgr2=0x56001d3300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d3300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bd27fc640 1 -- 172.21.15.1:0/328322129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56001cf9c580 con 0x56001d328980 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d3300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d3305e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80982d0 tx=0x7f5bc809b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396b205dea88335e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/328322129 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc809b900 con 0x56001d328980 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/328322129 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5bc8097f70 con 0x56001d328980 2024-09-16T00:34:11.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/328322129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc4004030 con 0x56001d328980 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/328322129 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5bc809fcb0 con 0x56001d328980 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bdb8ce740 10 client.16728 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.413+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/328322129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56001d066960 con 0x56001d328980 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.414+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/328322129 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc4004030 con 0x56001d328980 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.414+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/328322129 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5bc80cfcb0 con 0x56001d328980 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.414+0000 7f5bdb8ce740 10 client.16728 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.414+0000 7f5bdb8ce740 10 client.16728 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.414+0000 7f5bdb8ce740 20 client.16728 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.414+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56001d342d10 0x56001d3451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.414+0000 7f5bdb8ce740 4 client.16728 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.414+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/328322129 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56001d29ab90 con 0x56001d342d10 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.414+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56001d342d10 0x56001d3451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.415+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56001d342d10 0x56001d3451d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faff756ac6f18587 server_cookie=8e8880f40d748047 in_seq=0 out_seq=0 2024-09-16T00:34:11.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.415+0000 7f5bb6ff5640 10 client.16728 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:11.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.415+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/328322129 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56001d29ab90 con 0x56001d342d10 2024-09-16T00:34:11.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.415+0000 7f5bb6ff5640 10 client.16728 handle_command_reply: tid=0 2024-09-16T00:34:11.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.415+0000 7f5bdb8ce740 1 client.16728 shutdown 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/328322129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 msgr2=0x56001d3305e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d3305e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80982d0 tx=0x7f5bc809b790 comp rx=0 tx=0).stop 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/328322129 shutdown_connections 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56001d342d10 0x56001d3451d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 0x56001d330b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d328980 0x56001d3305e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/328322129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d3300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/328322129 >> 172.21.15.1:0/328322129 conn(0x56001d3245f0 msgr2=0x56001d32a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bb6ff5640 0 client.16728 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/328322129 shutdown_connections 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/328322129 wait complete. 2024-09-16T00:34:11.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.416+0000 7f5bdb8ce740 20 client.16728 trim_cache size 0 max 16384 2024-09-16T00:34:11.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bdb8ce740 1 Processor -- start 2024-09-16T00:34:11.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bdb8ce740 1 -- start start 2024-09-16T00:34:11.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 0x56001d32b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 0x56001d332010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d34aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41b3960 con 0x56001d332550 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41b1230 con 0x56001d333240 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41af080 con 0x56001d328060 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d34aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d34aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56160/0 (socket says 172.21.15.1:56160) 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bd37fe640 1 -- 172.21.15.1:0/2671330236 learned_addr learned my addr 172.21.15.1:0/2671330236 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/2671330236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 0x56001d332010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/2671330236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 0x56001d32b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bd37fe640 1 -- 172.21.15.1:0/2671330236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 msgr2=0x56001d32b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bd37fe640 1 --2- 172.21.15.1:0/2671330236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 0x56001d32b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.435+0000 7f5bd37fe640 1 -- 172.21.15.1:0/2671330236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 msgr2=0x56001d332010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:11.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.436+0000 7f5bd37fe640 1 --2- 172.21.15.1:0/2671330236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 0x56001d332010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.436+0000 7f5bd37fe640 1 -- 172.21.15.1:0/2671330236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56001d323610 con 0x56001d332550 2024-09-16T00:34:11.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.436+0000 7f5bd37fe640 1 --2- 172.21.15.1:0/2671330236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d34aaa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00d790 tx=0x7f5bcc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a3f0911d817680 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.436+0000 7f5bb8ff9640 1 -- 172.21.15.1:0/2671330236 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc02b070 con 0x56001d332550 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.436+0000 7f5bb8ff9640 1 -- 172.21.15.1:0/2671330236 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc026400 con 0x56001d332550 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.436+0000 7f5bb8ff9640 1 -- 172.21.15.1:0/2671330236 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc01a610 con 0x56001d332550 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2671330236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 msgr2=0x56001d34aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2671330236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d34aaa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00d790 tx=0x7f5bcc016000 comp rx=0 tx=0).stop 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2671330236 shutdown_connections 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2671330236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d34aaa0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2671330236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d333240 0x56001d332010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2671330236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d328060 0x56001d32b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2671330236 >> 172.21.15.1:0/2671330236 conn(0x56001d3245f0 msgr2=0x56001d331900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2671330236 shutdown_connections 2024-09-16T00:34:11.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2671330236 wait complete. 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.437+0000 7f5bdb8ce740 1 -- start start 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bdb8ce740 1 Processor -- start 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d33b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d337d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bdb8ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 0x56001d338290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41af6f0 con 0x56001d332550 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41b0b60 con 0x56001d328060 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bdb8ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41afdc0 con 0x56001d333240 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bdb8ce740 10 client.0 _ll_register_callbacks cb 0x7f5bc8004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bd27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d337d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bd27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d337d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56166/0 (socket says 172.21.15.1:56166) 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bd27fc640 1 -- 172.21.15.1:0/2834565407 learned_addr learned my addr 172.21.15.1:0/2834565407 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d33b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.438+0000 7f5bd37fe640 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 0x56001d338290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bd27fc640 1 -- 172.21.15.1:0/2834565407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 msgr2=0x56001d338290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 0x56001d338290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bd27fc640 1 -- 172.21.15.1:0/2834565407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 msgr2=0x56001d33b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d33b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bd27fc640 1 -- 172.21.15.1:0/2834565407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56001d338ac0 con 0x56001d332550 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d33b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bd27fc640 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d337d30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5bc809b890 tx=0x7f5bc8097c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d28fa2e25dedefa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/2834565407 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc809b900 con 0x56001d332550 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/2834565407 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5bc8097650 con 0x56001d332550 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2834565407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bcc026400 con 0x56001d332550 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/2834565407 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5bc809b260 con 0x56001d332550 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bdb8ce740 10 client.16734 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.439+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2834565407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56001d3aeb50 con 0x56001d332550 2024-09-16T00:34:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.440+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/2834565407 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc026400 con 0x56001d332550 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.440+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/2834565407 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5bc809a7c0 con 0x56001d332550 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.440+0000 7f5bdb8ce740 10 client.16734 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.440+0000 7f5bdb8ce740 10 client.16734 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.440+0000 7f5bdb8ce740 20 client.16734 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.440+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56001d3af0e0 0x56001d3b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.440+0000 7f5bdb8ce740 4 client.16734 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8e4ce410", "release": true, "await": true}] 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.440+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2834565407 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8e4ce410", "release": true, "await": true}) -- 0x56001d2950a0 con 0x56001d3af0e0 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.440+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56001d3af0e0 0x56001d3b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.441+0000 7f5bd2ffd640 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56001d3af0e0 0x56001d3b1500 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed933c3304672b8 server_cookie=2ed1cb4ae3acd801 in_seq=0 out_seq=0 2024-09-16T00:34:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.441+0000 7f5bb6ff5640 10 client.16734 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:11.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bb6ff5640 1 -- 172.21.15.1:0/2834565407 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56001d2950a0 con 0x56001d3af0e0 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bb6ff5640 10 client.16734 handle_command_reply: tid=0 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 client.16734 shutdown 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2834565407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 msgr2=0x56001d337d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d337d30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5bc809b890 tx=0x7f5bc8097c60 comp rx=0 tx=0).stop 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2834565407 shutdown_connections 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56001d3af0e0 0x56001d3b1500 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56001d333240 0x56001d338290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56001d332550 0x56001d337d30 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 --2- 172.21.15.1:0/2834565407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56001d328060 0x56001d33b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2834565407 >> 172.21.15.1:0/2834565407 conn(0x56001d3245f0 msgr2=0x56001d330060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bb6ff5640 0 client.16734 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2834565407 shutdown_connections 2024-09-16T00:34:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 1 -- 172.21.15.1:0/2834565407 wait complete. 2024-09-16T00:34:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.444+0000 7f5bdb8ce740 20 client.16734 trim_cache size 0 max 16384 2024-09-16T00:34:11.447 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:34:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:34:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:34:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 20, 2024-09-16T00:34:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:34:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: "8e4ce410": { 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 20, 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 145.3, 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:34:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:34:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:34:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:34:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:11.449 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:34:11.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 -- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5bbc0a3240 msgr2=0x7f5bbc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:11.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5bbc0a3240 0x7f5bbc0a5700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005fd0 tx=0x7f5bcc00a280 comp rx=0 tx=0).stop 2024-09-16T00:34:11.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 -- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100e30 msgr2=0x7f5bd4196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100e30 0x7f5bd4196560 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800d950 tx=0x7f5bc800de10 comp rx=0 tx=0).stop 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 -- 172.21.15.1:0/397444396 shutdown_connections 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bd41055b0 0x7f5bd4196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bd4100e30 0x7f5bd4196560 secure :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800d950 tx=0x7f5bc800de10 comp rx=0 tx=0).stop 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5bbc0a3240 0x7f5bbc0a5700 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 --2- 172.21.15.1:0/397444396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bd4100480 0x7f5bd4196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.445+0000 7f5bd94f5640 1 -- 172.21.15.1:0/397444396 >> 172.21.15.1:0/397444396 conn(0x7f5bd4077210 msgr2=0x7f5bd4103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.446+0000 7f5bd94f5640 1 -- 172.21.15.1:0/397444396 shutdown_connections 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:11.446+0000 7f5bd94f5640 1 -- 172.21.15.1:0/397444396 wait complete. 2024-09-16T00:34:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:11.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:11 smithi001 ceph-mon[30934]: pgmap v524: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:34:11.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:11 smithi019 ceph-mon[35186]: pgmap v524: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:34:11.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:11 smithi153 ceph-mon[35538]: pgmap v524: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:34:12.387 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8e4ce410', total seconds elapsed: 30.3 2024-09-16T00:34:12.388 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-16T00:34:12.388 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:34:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.611+0000 7ff085073640 1 -- 172.21.15.1:0/3015853272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 msgr2=0x7ff080103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.611+0000 7ff085073640 1 --2- 172.21.15.1:0/3015853272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 0x7ff080103120 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff070009900 tx=0x7ff07002a480 comp rx=0 tx=0).stop 2024-09-16T00:34:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.611+0000 7ff085073640 1 -- 172.21.15.1:0/3015853272 shutdown_connections 2024-09-16T00:34:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.611+0000 7ff085073640 1 --2- 172.21.15.1:0/3015853272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff08010c170 0x7ff08010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.611+0000 7ff085073640 1 --2- 172.21.15.1:0/3015853272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff080103660 0x7ff08010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.611+0000 7ff085073640 1 --2- 172.21.15.1:0/3015853272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 0x7ff080103120 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.611+0000 7ff085073640 1 -- 172.21.15.1:0/3015853272 >> 172.21.15.1:0/3015853272 conn(0x7ff0800fc7a0 msgr2=0x7ff0800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.611+0000 7ff085073640 1 -- 172.21.15.1:0/3015853272 shutdown_connections 2024-09-16T00:34:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.611+0000 7ff085073640 1 -- 172.21.15.1:0/3015853272 wait complete. 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff085073640 1 Processor -- start 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff085073640 1 -- start start 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff085073640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 0x7ff080198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff085073640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff080103660 0x7ff080198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff085073640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff08010c170 0x7ff08019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff085073640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff080111f70 con 0x7ff080102d40 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff085073640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff080111df0 con 0x7ff080103660 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff085073640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801120f0 con 0x7ff08010c170 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 0x7ff080198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 0x7ff080198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56184/0 (socket says 172.21.15.1:56184) 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff07e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff080103660 0x7ff080198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.612+0000 7ff07ed9d640 1 -- 172.21.15.1:0/1656342418 learned_addr learned my addr 172.21.15.1:0/1656342418 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff07f59e640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff08010c170 0x7ff08019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff07ed9d640 1 -- 172.21.15.1:0/1656342418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff08010c170 msgr2=0x7ff08019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff07ed9d640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff08010c170 0x7ff08019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff07ed9d640 1 -- 172.21.15.1:0/1656342418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff080103660 msgr2=0x7ff080198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff07ed9d640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff080103660 0x7ff080198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff07ed9d640 1 -- 172.21.15.1:0/1656342418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08019d3e0 con 0x7ff080102d40 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff07e59c640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff080103660 0x7ff080198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff07ed9d640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 0x7ff080198430 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff070006010 tx=0x7ff070004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff067fff640 1 -- 172.21.15.1:0/1656342418 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07003f030 con 0x7ff080102d40 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08019d670 con 0x7ff080102d40 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff067fff640 1 -- 172.21.15.1:0/1656342418 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff070004440 con 0x7ff080102d40 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff067fff640 1 -- 172.21.15.1:0/1656342418 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070035070 con 0x7ff080102d40 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.613+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0801a4fb0 con 0x7ff080102d40 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.614+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08006b070 con 0x7ff080102d40 2024-09-16T00:34:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.615+0000 7ff067fff640 1 -- 172.21.15.1:0/1656342418 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff070042030 con 0x7ff080102d40 2024-09-16T00:34:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.616+0000 7ff067fff640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0680a3240 0x7ff0680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.616+0000 7ff07e59c640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0680a3240 0x7ff0680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.616+0000 7ff067fff640 1 -- 172.21.15.1:0/1656342418 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff0700f00c0 con 0x7ff080102d40 2024-09-16T00:34:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.617+0000 7ff07e59c640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0680a3240 0x7ff0680a5700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff08010d5b0 tx=0x7ff074005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.620+0000 7ff067fff640 1 -- 172.21.15.1:0/1656342418 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0700bc330 con 0x7ff080102d40 2024-09-16T00:34:12.703 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:12.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.847+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff080107210 con 0x7ff080102d40 2024-09-16T00:34:12.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.848+0000 7ff067fff640 1 -- 172.21.15.1:0/1656342418 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7ff0700c0630 con 0x7ff080102d40 2024-09-16T00:34:12.849 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:34:12.849 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:34:12.849 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0680a3240 msgr2=0x7ff0680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0680a3240 0x7ff0680a5700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff08010d5b0 tx=0x7ff074005f90 comp rx=0 tx=0).stop 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 msgr2=0x7ff080198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 0x7ff080198430 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff070006010 tx=0x7ff070004040 comp rx=0 tx=0).stop 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 shutdown_connections 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0680a3240 0x7ff0680a5700 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff08010c170 0x7ff08019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff080103660 0x7ff080198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 --2- 172.21.15.1:0/1656342418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff080102d40 0x7ff080198430 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.850+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 >> 172.21.15.1:0/1656342418 conn(0x7ff0800fc7a0 msgr2=0x7ff0800fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:12.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.851+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 shutdown_connections 2024-09-16T00:34:12.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:12.851+0000 7ff085073640 1 -- 172.21.15.1:0/1656342418 wait complete. 2024-09-16T00:34:13.724 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:34:13.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:13 smithi019 ceph-mon[35186]: pgmap v525: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-16T00:34:13.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:13 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1656342418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:13.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:13 smithi001 ceph-mon[30934]: pgmap v525: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-16T00:34:13.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:13 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1656342418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:13.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:13 smithi153 ceph-mon[35538]: pgmap v525: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-16T00:34:13.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:13 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1656342418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.981+0000 7f4226065640 1 -- 172.21.15.1:0/906943243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 msgr2=0x7f4220101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.981+0000 7f4226065640 1 --2- 172.21.15.1:0/906943243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 0x7f4220101200 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4210009900 tx=0x7f421002a480 comp rx=0 tx=0).stop 2024-09-16T00:34:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.981+0000 7f4226065640 1 -- 172.21.15.1:0/906943243 shutdown_connections 2024-09-16T00:34:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.981+0000 7f4226065640 1 --2- 172.21.15.1:0/906943243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220102ef0 0x7f422010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.981+0000 7f4226065640 1 --2- 172.21.15.1:0/906943243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4220102570 0x7f42201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.981+0000 7f4226065640 1 --2- 172.21.15.1:0/906943243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 0x7f4220101200 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.981+0000 7f4226065640 1 -- 172.21.15.1:0/906943243 >> 172.21.15.1:0/906943243 conn(0x7f42200fc7d0 msgr2=0x7f42200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.981+0000 7f4226065640 1 -- 172.21.15.1:0/906943243 shutdown_connections 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.981+0000 7f4226065640 1 -- 172.21.15.1:0/906943243 wait complete. 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f4226065640 1 Processor -- start 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f4226065640 1 -- start start 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f4226065640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 0x7f42201983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f4226065640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4220102570 0x7f4220198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f4226065640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220102ef0 0x7f422019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f4226065640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4220110f80 con 0x7f4220100e20 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f4226065640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4220110e00 con 0x7f4220102570 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f4226065640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4220111100 con 0x7f4220102ef0 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f421f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 0x7f42201983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f421f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 0x7f42201983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56204/0 (socket says 172.21.15.1:56204) 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f421f7fe640 1 -- 172.21.15.1:0/11904544 learned_addr learned my addr 172.21.15.1:0/11904544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f4216ffd640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4220102570 0x7f4220198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.982+0000 7f421ffff640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220102ef0 0x7f422019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421f7fe640 1 -- 172.21.15.1:0/11904544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220102ef0 msgr2=0x7f422019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421f7fe640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220102ef0 0x7f422019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421f7fe640 1 -- 172.21.15.1:0/11904544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4220102570 msgr2=0x7f4220198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421f7fe640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4220102570 0x7f4220198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421f7fe640 1 -- 172.21.15.1:0/11904544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422019d380 con 0x7f4220100e20 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421ffff640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220102ef0 0x7f422019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421f7fe640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 0x7f42201983d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4210006010 tx=0x7f4210003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421d7fa640 1 -- 172.21.15.1:0/11904544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421003f070 con 0x7f4220100e20 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421d7fa640 1 -- 172.21.15.1:0/11904544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4210031e30 con 0x7f4220100e20 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422019d610 con 0x7f4220100e20 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f421d7fa640 1 -- 172.21.15.1:0/11904544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210035030 con 0x7f4220100e20 2024-09-16T00:34:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.983+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42201a4f50 con 0x7f4220100e20 2024-09-16T00:34:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.984+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42201021b0 con 0x7f4220100e20 2024-09-16T00:34:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.985+0000 7f421d7fa640 1 -- 172.21.15.1:0/11904544 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4210042070 con 0x7f4220100e20 2024-09-16T00:34:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.986+0000 7f421d7fa640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f420c0a3240 0x7f420c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.986+0000 7f4216ffd640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f420c0a3240 0x7f420c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.986+0000 7f421d7fa640 1 -- 172.21.15.1:0/11904544 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f42100f0cb0 con 0x7f4220100e20 2024-09-16T00:34:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.987+0000 7f4216ffd640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f420c0a3240 0x7f420c0a5700 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4208005e10 tx=0x7f4208005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:13.989+0000 7f421d7fa640 1 -- 172.21.15.1:0/11904544 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42100bd660 con 0x7f4220100e20 2024-09-16T00:34:14.077 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:14.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.207+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4220101200 con 0x7f4220100e20 2024-09-16T00:34:14.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.207+0000 7f421d7fa640 1 -- 172.21.15.1:0/11904544 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f4220101200 con 0x7f4220100e20 2024-09-16T00:34:14.208 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:34:14.208 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:34:14.208 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:34:14.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f420c0a3240 msgr2=0x7f420c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:14.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f420c0a3240 0x7f420c0a5700 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4208005e10 tx=0x7f4208005da0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 msgr2=0x7f42201983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:14.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 0x7f42201983d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4210006010 tx=0x7f4210003c80 comp rx=0 tx=0).stop 2024-09-16T00:34:14.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 shutdown_connections 2024-09-16T00:34:14.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f420c0a3240 0x7f420c0a5700 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4220102ef0 0x7f422019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4220102570 0x7f4220198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 --2- 172.21.15.1:0/11904544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4220100e20 0x7f42201983d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:14.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 >> 172.21.15.1:0/11904544 conn(0x7f42200fc7d0 msgr2=0x7f4220105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:14.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 shutdown_connections 2024-09-16T00:34:14.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:14.210+0000 7f4226065640 1 -- 172.21.15.1:0/11904544 wait complete. 2024-09-16T00:34:14.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:14 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/11904544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:14.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:14 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/11904544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:14.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:14 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/11904544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:15.067 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-16T00:34:15.068 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-16T00:34:15.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.322+0000 7fd2958e0640 1 -- 172.21.15.1:0/3176963131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd29010c170 msgr2=0x7fd29010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:15.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.322+0000 7fd2958e0640 1 --2- 172.21.15.1:0/3176963131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd29010c170 0x7fd29010e610 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd288009ca0 tx=0x7fd28802a950 comp rx=0 tx=0).stop 2024-09-16T00:34:15.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.322+0000 7fd2958e0640 1 -- 172.21.15.1:0/3176963131 shutdown_connections 2024-09-16T00:34:15.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.322+0000 7fd2958e0640 1 --2- 172.21.15.1:0/3176963131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd29010c170 0x7fd29010e610 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.322+0000 7fd2958e0640 1 --2- 172.21.15.1:0/3176963131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd290103660 0x7fd29010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.322+0000 7fd2958e0640 1 --2- 172.21.15.1:0/3176963131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd290102d40 0x7fd290103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.322+0000 7fd2958e0640 1 -- 172.21.15.1:0/3176963131 >> 172.21.15.1:0/3176963131 conn(0x7fd2900fc7a0 msgr2=0x7fd2900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:15.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.322+0000 7fd2958e0640 1 -- 172.21.15.1:0/3176963131 shutdown_connections 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.323+0000 7fd2958e0640 1 -- 172.21.15.1:0/3176963131 wait complete. 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd2958e0640 1 Processor -- start 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd2958e0640 1 -- start start 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd2958e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd290102d40 0x7fd290198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd2958e0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd290103660 0x7fd2901989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd2958e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29010c170 0x7fd29019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd2958e0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd290111fd0 con 0x7fd290102d40 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd2958e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd290111e50 con 0x7fd29010c170 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd2958e0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd290112150 con 0x7fd290103660 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd28effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd290102d40 0x7fd290198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd28effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd290102d40 0x7fd290198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56214/0 (socket says 172.21.15.1:56214) 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.324+0000 7fd28effd640 1 -- 172.21.15.1:0/1861689492 learned_addr learned my addr 172.21.15.1:0/1861689492 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29010c170 0x7fd29019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd290103660 0x7fd2901989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd28effd640 1 -- 172.21.15.1:0/1861689492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd290103660 msgr2=0x7fd2901989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd28effd640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd290103660 0x7fd2901989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd28effd640 1 -- 172.21.15.1:0/1861689492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29010c170 msgr2=0x7fd29019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd28effd640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29010c170 0x7fd29019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd28effd640 1 -- 172.21.15.1:0/1861689492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29019d440 con 0x7fd290102d40 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29010c170 0x7fd29019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd28effd640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd290102d40 0x7fd290198490 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd28000b4a0 tx=0x7fd28000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd2948de640 1 -- 172.21.15.1:0/1861689492 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280014070 con 0x7fd290102d40 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd2958e0640 1 -- 172.21.15.1:0/1861689492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29019d730 con 0x7fd290102d40 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd2948de640 1 -- 172.21.15.1:0/1861689492 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd280010af0 con 0x7fd290102d40 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd2958e0640 1 -- 172.21.15.1:0/1861689492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2901a5010 con 0x7fd290102d40 2024-09-16T00:34:15.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.325+0000 7fd2948de640 1 -- 172.21.15.1:0/1861689492 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280017d10 con 0x7fd290102d40 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.327+0000 7fd2948de640 1 -- 172.21.15.1:0/1861689492 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd280017070 con 0x7fd290102d40 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd297cb9740 1 Processor -- start 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd297cb9740 1 -- start start 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e254090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 0x55d97e254a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 0x55d97e261760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd297cb9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d97e1ba2f0 con 0x55d97e255b40 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd297cb9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d97e161cf0 con 0x55d97e25f370 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd297cb9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d97e1b62b0 con 0x55d97e2545d0 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e254090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e254090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56218/0 (socket says 172.21.15.1:56218) 2024-09-16T00:34:15.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd28f7fe640 1 -- 172.21.15.1:0/1313968578 learned_addr learned my addr 172.21.15.1:0/1313968578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd28effd640 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 0x55d97e254a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 0x55d97e261760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd2948de640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd2780a3240 0x7fd2780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd28f7fe640 1 -- 172.21.15.1:0/1313968578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 msgr2=0x55d97e254a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.328+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 0x55d97e254a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd28f7fe640 1 -- 172.21.15.1:0/1313968578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 msgr2=0x55d97e261760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 0x55d97e261760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd28f7fe640 1 -- 172.21.15.1:0/1313968578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97e1463a0 con 0x55d97e255b40 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 0x55d97e261760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd2948de640 1 -- 172.21.15.1:0/1861689492 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd2800cf1f0 con 0x7fd290102d40 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd2780a3240 0x7fd2780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e254090 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd288009a50 tx=0x7fd28802f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d72dba2a9bbdac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd275ffb640 1 -- 172.21.15.1:0/1313968578 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28804b070 con 0x55d97e255b40 2024-09-16T00:34:15.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd275ffb640 1 -- 172.21.15.1:0/1313968578 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd288004520 con 0x55d97e255b40 2024-09-16T00:34:15.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.329+0000 7fd275ffb640 1 -- 172.21.15.1:0/1313968578 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288034040 con 0x55d97e255b40 2024-09-16T00:34:15.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.330+0000 7fd297cb9740 1 -- 172.21.15.1:0/1313968578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 msgr2=0x55d97e254090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:15.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.330+0000 7fd297cb9740 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e254090 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd288009a50 tx=0x7fd28802f940 comp rx=0 tx=0).stop 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.330+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd2780a3240 0x7fd2780a5700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd288048000 tx=0x7fd288049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.331+0000 7fd297cb9740 1 -- 172.21.15.1:0/1313968578 shutdown_connections 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.331+0000 7fd297cb9740 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 0x55d97e261760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.331+0000 7fd297cb9740 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 0x55d97e254a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.331+0000 7fd297cb9740 1 --2- 172.21.15.1:0/1313968578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e254090 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.331+0000 7fd297cb9740 1 -- 172.21.15.1:0/1313968578 >> 172.21.15.1:0/1313968578 conn(0x55d97e21d310 msgr2=0x55d97e250c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.331+0000 7fd297cb9740 1 -- 172.21.15.1:0/1313968578 shutdown_connections 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.331+0000 7fd297cb9740 1 -- 172.21.15.1:0/1313968578 wait complete. 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.331+0000 7fd297cb9740 1 -- start start 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.331+0000 7fd297cb9740 1 Processor -- start 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 0x55d97e25ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e25c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 0x55d97e25c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd297cb9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d97e262070 con 0x55d97e255b40 2024-09-16T00:34:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd297cb9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d97e175060 con 0x55d97e2545d0 2024-09-16T00:34:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd297cb9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d97e2621f0 con 0x55d97e25f370 2024-09-16T00:34:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd297cb9740 10 client.0 _ll_register_callbacks cb 0x55d97dec88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e25c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e25c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56234/0 (socket says 172.21.15.1:56234) 2024-09-16T00:34:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 0x55d97e25ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28e7fc640 1 -- 172.21.15.1:0/3102260026 learned_addr learned my addr 172.21.15.1:0/3102260026 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 0x55d97e25c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28e7fc640 1 -- 172.21.15.1:0/3102260026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 msgr2=0x55d97e25c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 0x55d97e25c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28e7fc640 1 -- 172.21.15.1:0/3102260026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 msgr2=0x55d97e25ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 0x55d97e25ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28e7fc640 1 -- 172.21.15.1:0/3102260026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97df2b3c0 con 0x55d97e255b40 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e25c010 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd28400f6c0 tx=0x7fd28400fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef6a484c97e5e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.332+0000 7fd26effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd26effd640 1 -- 172.21.15.1:0/3102260026 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284016070 con 0x55d97e255b40 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd26effd640 1 -- 172.21.15.1:0/3102260026 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd28400bb00 con 0x55d97e255b40 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd297cb9740 1 -- 172.21.15.1:0/3102260026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd288004520 con 0x55d97e255b40 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd26effd640 1 -- 172.21.15.1:0/3102260026 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd28400bd00 con 0x55d97e255b40 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd297cb9740 10 client.16758 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd297cb9740 1 -- 172.21.15.1:0/3102260026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d97e26dbc0 con 0x55d97e255b40 2024-09-16T00:34:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd26effd640 1 -- 172.21.15.1:0/3102260026 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd288004520 con 0x55d97e255b40 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd26effd640 1 -- 172.21.15.1:0/3102260026 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd284004c10 con 0x55d97e255b40 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd297cb9740 10 client.16758 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd297cb9740 10 client.16758 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd297cb9740 20 client.16758 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d97e26f120 0x55d97e271580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd297cb9740 4 client.16758 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.333+0000 7fd297cb9740 1 -- 172.21.15.1:0/3102260026 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d97e1c7860 con 0x55d97e26f120 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.334+0000 7fd28effd640 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d97e26f120 0x55d97e271580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.334+0000 7fd28effd640 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d97e26f120 0x55d97e271580 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46fe8cfc47bb0443 server_cookie=92ddc0e4edd15363 in_seq=0 out_seq=0 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.334+0000 7fd26effd640 10 client.16758 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd26effd640 1 -- 172.21.15.1:0/3102260026 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d97e1c7860 con 0x55d97e26f120 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd26effd640 10 client.16758 handle_command_reply: tid=0 2024-09-16T00:34:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 client.16758 shutdown 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 -- 172.21.15.1:0/3102260026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 msgr2=0x55d97e25c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e25c010 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd28400f6c0 tx=0x7fd28400fb80 comp rx=0 tx=0).stop 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 -- 172.21.15.1:0/3102260026 shutdown_connections 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 0x55d97e25c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e25c010 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d97e26f120 0x55d97e271580 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd26effd640 0 client.16758 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3102260026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 0x55d97e25ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 -- 172.21.15.1:0/3102260026 >> 172.21.15.1:0/3102260026 conn(0x55d97e21d310 msgr2=0x55d97e256960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 -- 172.21.15.1:0/3102260026 shutdown_connections 2024-09-16T00:34:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 1 -- 172.21.15.1:0/3102260026 wait complete. 2024-09-16T00:34:15.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.335+0000 7fd297cb9740 20 client.16758 trim_cache size 0 max 16384 2024-09-16T00:34:15.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd297cb9740 1 Processor -- start 2024-09-16T00:34:15.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd297cb9740 1 -- start start 2024-09-16T00:34:15.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 0x55d97e25df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e257120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 0x55d97e257660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd297cb9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2901b1830 con 0x55d97e255b40 2024-09-16T00:34:15.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd297cb9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2901af100 con 0x55d97e2545d0 2024-09-16T00:34:15.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd297cb9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2901acf50 con 0x55d97e25f370 2024-09-16T00:34:15.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd28e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e257120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd28e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e257120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56246/0 (socket says 172.21.15.1:56246) 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd28e7fc640 1 -- 172.21.15.1:0/358293942 learned_addr learned my addr 172.21.15.1:0/358293942 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd28effd640 1 --2- 172.21.15.1:0/358293942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 0x55d97e25df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.349+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/358293942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 0x55d97e257660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.350+0000 7fd28e7fc640 1 -- 172.21.15.1:0/358293942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 msgr2=0x55d97e257660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.350+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/358293942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 0x55d97e257660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.350+0000 7fd28e7fc640 1 -- 172.21.15.1:0/358293942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 msgr2=0x55d97e25df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.350+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/358293942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 0x55d97e25df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.350+0000 7fd28e7fc640 1 -- 172.21.15.1:0/358293942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97e250510 con 0x55d97e255b40 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.350+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/358293942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e257120 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd2840105a0 tx=0x7fd284000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f652879bef43d5e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.350+0000 7fd2757fa640 1 -- 172.21.15.1:0/358293942 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284016070 con 0x55d97e255b40 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.350+0000 7fd2757fa640 1 -- 172.21.15.1:0/358293942 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd28400b6f0 con 0x55d97e255b40 2024-09-16T00:34:15.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.350+0000 7fd2757fa640 1 -- 172.21.15.1:0/358293942 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284005330 con 0x55d97e255b40 2024-09-16T00:34:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 -- 172.21.15.1:0/358293942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 msgr2=0x55d97e257120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 --2- 172.21.15.1:0/358293942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e257120 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd2840105a0 tx=0x7fd284000bc0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 -- 172.21.15.1:0/358293942 shutdown_connections 2024-09-16T00:34:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 --2- 172.21.15.1:0/358293942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e25f370 0x55d97e257660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 --2- 172.21.15.1:0/358293942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e257120 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 --2- 172.21.15.1:0/358293942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e2545d0 0x55d97e25df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 -- 172.21.15.1:0/358293942 >> 172.21.15.1:0/358293942 conn(0x55d97e21d310 msgr2=0x55d97e25d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 -- 172.21.15.1:0/358293942 shutdown_connections 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 -- 172.21.15.1:0/358293942 wait complete. 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.351+0000 7fd297cb9740 1 -- start start 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd297cb9740 1 Processor -- start 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 0x55d97e259110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e259650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd297cb9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 0x55d97e259b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd297cb9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2901ad5c0 con 0x55d97e255b40 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd297cb9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2901aea30 con 0x55d97e25f370 2024-09-16T00:34:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd297cb9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2901adc90 con 0x55d97e2545d0 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd297cb9740 10 client.0 _ll_register_callbacks cb 0x7fd29006b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd28e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e259650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd28e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e259650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56248/0 (socket says 172.21.15.1:56248) 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd28e7fc640 1 -- 172.21.15.1:0/3570340513 learned_addr learned my addr 172.21.15.1:0/3570340513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd28effd640 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 0x55d97e259110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd28f7fe640 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 0x55d97e259b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd28e7fc640 1 -- 172.21.15.1:0/3570340513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 msgr2=0x55d97e259110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 0x55d97e259110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd28e7fc640 1 -- 172.21.15.1:0/3570340513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 msgr2=0x55d97e259b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.352+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 0x55d97e259b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd28e7fc640 1 -- 172.21.15.1:0/3570340513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97e2d8ef0 con 0x55d97e255b40 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd28e7fc640 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e259650 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd284010c80 tx=0x7fd2840058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4856f5bd68e45214 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd26effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd26effd640 1 -- 172.21.15.1:0/3570340513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28403f060 con 0x55d97e255b40 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd26effd640 1 -- 172.21.15.1:0/3570340513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd284005480 con 0x55d97e255b40 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd297cb9740 1 -- 172.21.15.1:0/3570340513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd28400b6f0 con 0x55d97e255b40 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd26effd640 1 -- 172.21.15.1:0/3570340513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd284016040 con 0x55d97e255b40 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd297cb9740 10 client.16764 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd297cb9740 1 -- 172.21.15.1:0/3570340513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d97e2da1d0 con 0x55d97e255b40 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.353+0000 7fd26effd640 1 -- 172.21.15.1:0/3570340513 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd28400b6f0 con 0x55d97e255b40 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd26effd640 1 -- 172.21.15.1:0/3570340513 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd284039030 con 0x55d97e255b40 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd297cb9740 10 client.16764 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd297cb9740 10 client.16764 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd297cb9740 20 client.16764 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d97e2da8c0 0x55d97e2dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd297cb9740 4 client.16764 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd297cb9740 1 -- 172.21.15.1:0/3570340513 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d97e1c1d70 con 0x55d97e2da8c0 2024-09-16T00:34:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd28effd640 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d97e2da8c0 0x55d97e2dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd28effd640 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d97e2da8c0 0x55d97e2dcce0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29d570aa505a3d59 server_cookie=80c14b1c215159b5 in_seq=0 out_seq=0 2024-09-16T00:34:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.354+0000 7fd26effd640 10 client.16764 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd26effd640 1 -- 172.21.15.1:0/3570340513 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d97e1c1d70 con 0x55d97e2da8c0 2024-09-16T00:34:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd26effd640 10 client.16764 handle_command_reply: tid=0 2024-09-16T00:34:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 client.16764 shutdown 2024-09-16T00:34:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 -- 172.21.15.1:0/3570340513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 msgr2=0x55d97e259650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e259650 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd284010c80 tx=0x7fd2840058a0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 -- 172.21.15.1:0/3570340513 shutdown_connections 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d97e2da8c0 0x55d97e2dcce0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d97e25f370 0x55d97e259b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d97e255b40 0x55d97e259650 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 --2- 172.21.15.1:0/3570340513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d97e2545d0 0x55d97e259110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd26effd640 0 client.16764 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 -- 172.21.15.1:0/3570340513 >> 172.21.15.1:0/3570340513 conn(0x55d97e21d310 msgr2=0x55d97e25e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 -- 172.21.15.1:0/3570340513 shutdown_connections 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 1 -- 172.21.15.1:0/3570340513 wait complete. 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.355+0000 7fd297cb9740 20 client.16764 trim_cache size 0 max 16384 2024-09-16T00:34:15.360 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:34:15.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 -- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd2780a3240 msgr2=0x7fd2780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:15.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd2780a3240 0x7fd2780a5700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd288048000 tx=0x7fd288049000 comp rx=0 tx=0).stop 2024-09-16T00:34:15.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 -- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd290102d40 msgr2=0x7fd290198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:15.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd290102d40 0x7fd290198490 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd28000b4a0 tx=0x7fd28000b960 comp rx=0 tx=0).stop 2024-09-16T00:34:15.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 -- 172.21.15.1:0/1861689492 shutdown_connections 2024-09-16T00:34:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29010c170 0x7fd29019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd2780a3240 0x7fd2780a5700 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd290103660 0x7fd2901989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 --2- 172.21.15.1:0/1861689492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd290102d40 0x7fd290198490 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 -- 172.21.15.1:0/1861689492 >> 172.21.15.1:0/1861689492 conn(0x7fd2900fc7a0 msgr2=0x7fd2900fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 -- 172.21.15.1:0/1861689492 shutdown_connections 2024-09-16T00:34:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:15.356+0000 7fd2958e0640 1 -- 172.21.15.1:0/1861689492 wait complete. 2024-09-16T00:34:15.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:15 smithi001 ceph-mon[30934]: pgmap v526: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-16T00:34:15.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:15 smithi019 ceph-mon[35186]: pgmap v526: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-16T00:34:15.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:15 smithi153 ceph-mon[35538]: pgmap v526: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-16T00:34:16.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:34:16.172 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:16.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.706+0000 7fa671fe6640 1 -- 172.21.15.1:0/826950886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c100610 msgr2=0x7fa66c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:16.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.706+0000 7fa671fe6640 1 --2- 172.21.15.1:0/826950886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c100610 0x7fa66c100a50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa660009900 tx=0x7fa66002a410 comp rx=0 tx=0).stop 2024-09-16T00:34:16.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.706+0000 7fa671fe6640 1 -- 172.21.15.1:0/826950886 shutdown_connections 2024-09-16T00:34:16.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.706+0000 7fa671fe6640 1 --2- 172.21.15.1:0/826950886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa66c10e850 0x7fa66c110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.706+0000 7fa671fe6640 1 --2- 172.21.15.1:0/826950886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c100610 0x7fa66c100a50 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.706+0000 7fa671fe6640 1 --2- 172.21.15.1:0/826950886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa66c102cc0 0x7fa66c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.706+0000 7fa671fe6640 1 -- 172.21.15.1:0/826950886 >> 172.21.15.1:0/826950886 conn(0x7fa66c077210 msgr2=0x7fa66c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:16.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.706+0000 7fa671fe6640 1 -- 172.21.15.1:0/826950886 shutdown_connections 2024-09-16T00:34:16.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.706+0000 7fa671fe6640 1 -- 172.21.15.1:0/826950886 wait complete. 2024-09-16T00:34:16.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.707+0000 7fa671fe6640 1 Processor -- start 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.707+0000 7fa671fe6640 1 -- start start 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa671fe6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa66c100610 0x7fa66c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa671fe6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa66c102cc0 0x7fa66c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa671fe6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c10e850 0x7fa66c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa671fe6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1143b0 con 0x7fa66c10e850 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa671fe6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c114230 con 0x7fa66c102cc0 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa671fe6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c114530 con 0x7fa66c100610 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c10e850 0x7fa66c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c10e850 0x7fa66c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56266/0 (socket says 172.21.15.1:56266) 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66bfff640 1 -- 172.21.15.1:0/2409189461 learned_addr learned my addr 172.21.15.1:0/2409189461 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66affd640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa66c102cc0 0x7fa66c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa66c100610 0x7fa66c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66bfff640 1 -- 172.21.15.1:0/2409189461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa66c100610 msgr2=0x7fa66c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66bfff640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa66c100610 0x7fa66c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66bfff640 1 -- 172.21.15.1:0/2409189461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa66c102cc0 msgr2=0x7fa66c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66bfff640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa66c102cc0 0x7fa66c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66bfff640 1 -- 172.21.15.1:0/2409189461 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c19af70 con 0x7fa66c10e850 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa66c100610 0x7fa66c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa66bfff640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c10e850 0x7fa66c19a700 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa66400e9d0 tx=0x7fa66400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.708+0000 7fa668ff9640 1 -- 172.21.15.1:0/2409189461 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66400cd80 con 0x7fa66c10e850 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.709+0000 7fa671fe6640 1 -- 172.21.15.1:0/2409189461 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c19b260 con 0x7fa66c10e850 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.709+0000 7fa668ff9640 1 -- 172.21.15.1:0/2409189461 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa664005020 con 0x7fa66c10e850 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.709+0000 7fa668ff9640 1 -- 172.21.15.1:0/2409189461 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6640116d0 con 0x7fa66c10e850 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.709+0000 7fa671fe6640 1 -- 172.21.15.1:0/2409189461 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c1a75b0 con 0x7fa66c10e850 2024-09-16T00:34:16.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa668ff9640 1 -- 172.21.15.1:0/2409189461 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa6640051d0 con 0x7fa66c10e850 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa6743bf740 1 Processor -- start 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa6743bf740 1 -- start start 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb96c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 0x563d9fba14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 0x563d9fba3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa6743bf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563d9fafb160 con 0x563d9fb96850 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa6743bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563d9faa2720 con 0x563d9fba1a30 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa6743bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563d9faf7120 con 0x563d9fb97170 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa66affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb96c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa66affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb96c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56272/0 (socket says 172.21.15.1:56272) 2024-09-16T00:34:16.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.711+0000 7fa66affd640 1 -- 172.21.15.1:0/1531765446 learned_addr learned my addr 172.21.15.1:0/1531765446 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:16.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/1531765446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 0x563d9fba14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa66affd640 1 -- 172.21.15.1:0/1531765446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 msgr2=0x563d9fba14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:16.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa66affd640 1 --2- 172.21.15.1:0/1531765446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 0x563d9fba14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa66affd640 1 -- 172.21.15.1:0/1531765446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 msgr2=0x563d9fba3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:16.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa66affd640 1 --2- 172.21.15.1:0/1531765446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 0x563d9fba3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa66affd640 1 -- 172.21.15.1:0/1531765446 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9f86cba0 con 0x563d9fb96850 2024-09-16T00:34:16.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa668ff9640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6540a32c0 0x7fa6540a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa66affd640 1 --2- 172.21.15.1:0/1531765446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb96c30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa66002a930 tx=0x7fa660005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fa5b2fcb0ab5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:16.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6540a32c0 0x7fa6540a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa668ff9640 1 -- 172.21.15.1:0/2409189461 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa6640d01e0 con 0x7fa66c10e850 2024-09-16T00:34:16.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa651ffb640 1 -- 172.21.15.1:0/1531765446 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660035040 con 0x563d9fb96850 2024-09-16T00:34:16.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa651ffb640 1 -- 172.21.15.1:0/1531765446 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa660031930 con 0x563d9fb96850 2024-09-16T00:34:16.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.712+0000 7fa651ffb640 1 -- 172.21.15.1:0/1531765446 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6600047a0 con 0x563d9fb96850 2024-09-16T00:34:16.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 -- 172.21.15.1:0/1531765446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 msgr2=0x563d9fb96c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:16.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6540a32c0 0x7fa6540a5780 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0058c0 tx=0x7fa65c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:16.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1531765446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb96c30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa66002a930 tx=0x7fa660005c00 comp rx=0 tx=0).stop 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 -- 172.21.15.1:0/1531765446 shutdown_connections 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1531765446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 0x563d9fba3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1531765446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 0x563d9fba14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1531765446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb96c30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 -- 172.21.15.1:0/1531765446 >> 172.21.15.1:0/1531765446 conn(0x563d9fb92de0 msgr2=0x563d9fb931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 -- 172.21.15.1:0/1531765446 shutdown_connections 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 -- 172.21.15.1:0/1531765446 wait complete. 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.713+0000 7fa6743bf740 1 -- start start 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa6743bf740 1 Processor -- start 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb96850 0x563d9fb9e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb97170 0x563d9fb9edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fba1a30 0x563d9fb9f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa6743bf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563d9fba4600 con 0x563d9fb97170 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa6743bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563d9fa83900 con 0x563d9fb96850 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa6743bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563d9fba4780 con 0x563d9fba1a30 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa6743bf740 10 client.0 _ll_register_callbacks cb 0x563d9fa81780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb97170 0x563d9fb9edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb97170 0x563d9fb9edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56276/0 (socket says 172.21.15.1:56276) 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66affd640 1 -- 172.21.15.1:0/1709387433 learned_addr learned my addr 172.21.15.1:0/1709387433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66bfff640 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fba1a30 0x563d9fb9f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb96850 0x563d9fb9e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66affd640 1 -- 172.21.15.1:0/1709387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fba1a30 msgr2=0x563d9fb9f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66affd640 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fba1a30 0x563d9fb9f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66affd640 1 -- 172.21.15.1:0/1709387433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb96850 msgr2=0x563d9fb9e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66affd640 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb96850 0x563d9fb9e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66affd640 1 -- 172.21.15.1:0/1709387433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9fac3c50 con 0x563d9fb97170 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.714+0000 7fa66bfff640 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fba1a30 0x563d9fb9f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa66affd640 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb97170 0x563d9fb9edf0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa660005290 tx=0x7fa660005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c226c4a1f66f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa64affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa64affd640 1 -- 172.21.15.1:0/1709387433 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660004750 con 0x563d9fb97170 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa64affd640 1 -- 172.21.15.1:0/1709387433 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa6600048f0 con 0x563d9fb97170 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa6743bf740 1 -- 172.21.15.1:0/1709387433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa660031930 con 0x563d9fb97170 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa64affd640 1 -- 172.21.15.1:0/1709387433 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa660056050 con 0x563d9fb97170 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa6743bf740 10 client.16776 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa6743bf740 1 -- 172.21.15.1:0/1709387433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d9fbb02a0 con 0x563d9fb97170 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa64affd640 1 -- 172.21.15.1:0/1709387433 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa660031930 con 0x563d9fb97170 2024-09-16T00:34:16.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa64affd640 1 -- 172.21.15.1:0/1709387433 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa660045440 con 0x563d9fb97170 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa6743bf740 10 client.16776 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa6743bf740 10 client.16776 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa6743bf740 20 client.16776 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d9fbb15f0 0x563d9fbb39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa6743bf740 4 client.16776 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.715+0000 7fa6743bf740 1 -- 172.21.15.1:0/1709387433 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d9fb08890 con 0x563d9fbb15f0 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.716+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d9fbb15f0 0x563d9fbb39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.716+0000 7fa64affd640 10 client.16776 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.716+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d9fbb15f0 0x563d9fbb39e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638696f424282cdd server_cookie=5b20eb0c2fccbb30 in_seq=0 out_seq=0 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa64affd640 1 -- 172.21.15.1:0/1709387433 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d9fb08890 con 0x563d9fbb15f0 2024-09-16T00:34:16.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa64affd640 10 client.16776 handle_command_reply: tid=0 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 client.16776 shutdown 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 -- 172.21.15.1:0/1709387433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb97170 msgr2=0x563d9fb9edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb97170 0x563d9fb9edf0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa660005290 tx=0x7fa660005330 comp rx=0 tx=0).stop 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 -- 172.21.15.1:0/1709387433 shutdown_connections 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d9fbb15f0 0x563d9fbb39e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fba1a30 0x563d9fb9f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb97170 0x563d9fb9edf0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa64affd640 0 client.16776 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 --2- 172.21.15.1:0/1709387433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb96850 0x563d9fb9e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 -- 172.21.15.1:0/1709387433 >> 172.21.15.1:0/1709387433 conn(0x563d9fb92de0 msgr2=0x563d9fb98b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:16.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 -- 172.21.15.1:0/1709387433 shutdown_connections 2024-09-16T00:34:16.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.717+0000 7fa6743bf740 1 -- 172.21.15.1:0/1709387433 wait complete. 2024-09-16T00:34:16.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.718+0000 7fa6743bf740 20 client.16776 trim_cache size 0 max 16384 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.735+0000 7fa6743bf740 1 Processor -- start 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.735+0000 7fa6743bf740 1 -- start start 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.735+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb9f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.735+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 0x563d9fbad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.735+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 0x563d9fb9bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.735+0000 7fa6743bf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1b3880 con 0x563d9fb96850 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.735+0000 7fa6743bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1b1150 con 0x563d9fba1a30 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.735+0000 7fa6743bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1aefa0 con 0x563d9fb97170 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.735+0000 7fa66b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb9f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56278/0 (socket says 172.21.15.1:56278) 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66b7fe640 1 -- 172.21.15.1:0/3247108130 learned_addr learned my addr 172.21.15.1:0/3247108130 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66bfff640 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 0x563d9fb9bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66affd640 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 0x563d9fbad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66b7fe640 1 -- 172.21.15.1:0/3247108130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 msgr2=0x563d9fbad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 0x563d9fbad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66b7fe640 1 -- 172.21.15.1:0/3247108130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 msgr2=0x563d9fb9bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 0x563d9fb9bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66b7fe640 1 -- 172.21.15.1:0/3247108130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9fb92c10 con 0x563d9fb96850 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66bfff640 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 0x563d9fb9bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:16.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66affd640 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 0x563d9fbad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:16.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb9f180 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x563d9fba48d0 tx=0x7fa65c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73459f02727ffcb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:16.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa6517fa640 1 -- 172.21.15.1:0/3247108130 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c004020 con 0x563d9fb96850 2024-09-16T00:34:16.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa6517fa640 1 -- 172.21.15.1:0/3247108130 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c0041c0 con 0x563d9fb96850 2024-09-16T00:34:16.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.736+0000 7fa6517fa640 1 -- 172.21.15.1:0/3247108130 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c020690 con 0x563d9fb96850 2024-09-16T00:34:16.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.737+0000 7fa6743bf740 1 -- 172.21.15.1:0/3247108130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 msgr2=0x563d9fb9f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:16.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.737+0000 7fa6743bf740 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb9f180 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x563d9fba48d0 tx=0x7fa65c016000 comp rx=0 tx=0).stop 2024-09-16T00:34:16.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.737+0000 7fa6743bf740 1 -- 172.21.15.1:0/3247108130 shutdown_connections 2024-09-16T00:34:16.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.737+0000 7fa6743bf740 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fba1a30 0x563d9fb9bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.737+0000 7fa6743bf740 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fb97170 0x563d9fbad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.737+0000 7fa6743bf740 1 --2- 172.21.15.1:0/3247108130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fb96850 0x563d9fb9f180 secure :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0x563d9fba48d0 tx=0x7fa65c016000 comp rx=0 tx=0).stop 2024-09-16T00:34:16.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.738+0000 7fa6743bf740 1 -- 172.21.15.1:0/3247108130 >> 172.21.15.1:0/3247108130 conn(0x563d9fb92de0 msgr2=0x563d9fbaca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:16.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.738+0000 7fa6743bf740 1 -- 172.21.15.1:0/3247108130 shutdown_connections 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.738+0000 7fa6743bf740 1 -- 172.21.15.1:0/3247108130 wait complete. 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.738+0000 7fa6743bf740 1 -- start start 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.738+0000 7fa6743bf740 1 Processor -- start 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb97170 0x563d9fbb4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fba1a30 0x563d9fbb54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa6743bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fc14f70 0x563d9fc153f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa6743bf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1af610 con 0x563d9fba1a30 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa6743bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1b0a80 con 0x563d9fb97170 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa6743bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1afce0 con 0x563d9fc14f70 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa6743bf740 10 client.0 _ll_register_callbacks cb 0x7fa660005140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fba1a30 0x563d9fbb54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fba1a30 0x563d9fbb54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56290/0 (socket says 172.21.15.1:56290) 2024-09-16T00:34:16.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fc14f70 0x563d9fc153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66affd640 1 -- 172.21.15.1:0/2704865597 learned_addr learned my addr 172.21.15.1:0/2704865597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb97170 0x563d9fbb4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66affd640 1 -- 172.21.15.1:0/2704865597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fc14f70 msgr2=0x563d9fc153f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66affd640 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fc14f70 0x563d9fc153f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66affd640 1 -- 172.21.15.1:0/2704865597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb97170 msgr2=0x563d9fbb4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66affd640 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb97170 0x563d9fbb4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66affd640 1 -- 172.21.15.1:0/2704865597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9fb996c0 con 0x563d9fba1a30 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb97170 0x563d9fbb4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.739+0000 7fa66bfff640 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fc14f70 0x563d9fc153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa66affd640 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fba1a30 0x563d9fbb54e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa660009900 tx=0x7fa66002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561f7991004fb5da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa64affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa64affd640 1 -- 172.21.15.1:0/2704865597 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660035070 con 0x563d9fba1a30 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa64affd640 1 -- 172.21.15.1:0/2704865597 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa6600315b0 con 0x563d9fba1a30 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa6743bf740 1 -- 172.21.15.1:0/2704865597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65c0041c0 con 0x563d9fba1a30 2024-09-16T00:34:16.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa64affd640 1 -- 172.21.15.1:0/2704865597 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa660045c80 con 0x563d9fba1a30 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa6743bf740 10 client.16782 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa6743bf740 1 -- 172.21.15.1:0/2704865597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d9fb93a70 con 0x563d9fba1a30 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa64affd640 1 -- 172.21.15.1:0/2704865597 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c0041c0 con 0x563d9fba1a30 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.740+0000 7fa64affd640 1 -- 172.21.15.1:0/2704865597 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa660062030 con 0x563d9fba1a30 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.741+0000 7fa6743bf740 10 client.16782 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.741+0000 7fa6743bf740 10 client.16782 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.741+0000 7fa6743bf740 20 client.16782 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.741+0000 7fa6743bf740 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d9fc1d180 0x563d9fc1f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.741+0000 7fa6743bf740 4 client.16782 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.741+0000 7fa6743bf740 1 -- 172.21.15.1:0/2704865597 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x563d9fb02be0 con 0x563d9fc1d180 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.741+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d9fc1d180 0x563d9fc1f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.741+0000 7fa64affd640 10 client.16782 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:16.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.741+0000 7fa66b7fe640 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d9fc1d180 0x563d9fc1f5a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7729e4f7e248da2a server_cookie=d90df1ade0b7556e in_seq=0 out_seq=0 2024-09-16T00:34:16.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.752+0000 7fa64affd640 1 -- 172.21.15.1:0/2704865597 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563d9fb02be0 con 0x563d9fc1d180 2024-09-16T00:34:16.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.752+0000 7fa64affd640 10 client.16782 handle_command_reply: tid=0 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.752+0000 7fa6743bf740 1 client.16782 shutdown 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 -- 172.21.15.1:0/2704865597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fba1a30 msgr2=0x563d9fbb54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fba1a30 0x563d9fbb54e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa660009900 tx=0x7fa66002ab10 comp rx=0 tx=0).stop 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 -- 172.21.15.1:0/2704865597 shutdown_connections 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d9fc1d180 0x563d9fc1f5a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d9fc14f70 0x563d9fc153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d9fba1a30 0x563d9fbb54e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 --2- 172.21.15.1:0/2704865597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d9fb97170 0x563d9fbb4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 -- 172.21.15.1:0/2704865597 >> 172.21.15.1:0/2704865597 conn(0x563d9fb92de0 msgr2=0x563d9fbad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa64affd640 0 client.16782 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 -- 172.21.15.1:0/2704865597 shutdown_connections 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 1 -- 172.21.15.1:0/2704865597 wait complete. 2024-09-16T00:34:16.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.753+0000 7fa6743bf740 20 client.16782 trim_cache size 0 max 16384 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 22, 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "183573db": { 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 22, 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:34:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 145.3, 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:16.757 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:34:16.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 -- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6540a32c0 msgr2=0x7fa6540a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:16.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6540a32c0 0x7fa6540a5780 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0058c0 tx=0x7fa65c005850 comp rx=0 tx=0).stop 2024-09-16T00:34:16.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 -- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c10e850 msgr2=0x7fa66c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:16.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c10e850 0x7fa66c19a700 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa66400e9d0 tx=0x7fa66400ee90 comp rx=0 tx=0).stop 2024-09-16T00:34:16.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 -- 172.21.15.1:0/2409189461 shutdown_connections 2024-09-16T00:34:16.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa66c10e850 0x7fa66c19a700 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa66c102cc0 0x7fa66c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6540a32c0 0x7fa6540a5780 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 --2- 172.21.15.1:0/2409189461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa66c100610 0x7fa66c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:16.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 -- 172.21.15.1:0/2409189461 >> 172.21.15.1:0/2409189461 conn(0x7fa66c077210 msgr2=0x7fa66c10fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:16.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 -- 172.21.15.1:0/2409189461 shutdown_connections 2024-09-16T00:34:16.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:16.754+0000 7fa671fe6640 1 -- 172.21.15.1:0/2409189461 wait complete. 2024-09-16T00:34:17.469 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '183573db', quiesce took 5.1 seconds. Will release after: 20.25 2024-09-16T00:34:17.510 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:17.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:17 smithi001 ceph-mon[30934]: pgmap v527: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-16T00:34:17.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:17 smithi019 ceph-mon[35186]: pgmap v527: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-16T00:34:17.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:17 smithi153 ceph-mon[35538]: pgmap v527: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-16T00:34:18.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.004+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2754487538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 msgr2=0x7f2c5410bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:18.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.004+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2754487538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 0x7f2c5410bb60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009900 tx=0x7f2c3c02a480 comp rx=0 tx=0).stop 2024-09-16T00:34:18.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.004+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2754487538 shutdown_connections 2024-09-16T00:34:18.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.004+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2754487538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c5410c180 0x7f2c5410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.005+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2754487538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 0x7f2c5410bb60 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.005+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2754487538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c54102d50 0x7f2c54103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.005+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2754487538 >> 172.21.15.1:0/2754487538 conn(0x7f2c540fc7d0 msgr2=0x7f2c540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.005+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2754487538 shutdown_connections 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.005+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2754487538 wait complete. 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.005+0000 7f2c5a1a3640 1 Processor -- start 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.005+0000 7f2c5a1a3640 1 -- start start 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c5a1a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102d50 0x7f2c541984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c5a1a3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 0x7f2c541989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c5a1a3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c5410c180 0x7f2c5419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c5a1a3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54111fc0 con 0x7f2c54103670 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c5a1a3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54111e40 con 0x7f2c54102d50 2024-09-16T00:34:18.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c5a1a3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54112140 con 0x7f2c5410c180 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c52ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 0x7f2c541989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c52ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 0x7f2c541989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56306/0 (socket says 172.21.15.1:56306) 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c52ffd640 1 -- 172.21.15.1:0/2263918763 learned_addr learned my addr 172.21.15.1:0/2263918763 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c537fe640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102d50 0x7f2c541984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c53fff640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c5410c180 0x7f2c5419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c52ffd640 1 -- 172.21.15.1:0/2263918763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c5410c180 msgr2=0x7f2c5419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c5410c180 0x7f2c5419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c52ffd640 1 -- 172.21.15.1:0/2263918763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102d50 msgr2=0x7f2c541984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102d50 0x7f2c541984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c52ffd640 1 -- 172.21.15.1:0/2263918763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5419d450 con 0x7f2c54103670 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.006+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 0x7f2c541989e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005ab0 tx=0x7f2c3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.007+0000 7f2c50ff9640 1 -- 172.21.15.1:0/2263918763 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c03f030 con 0x7f2c54103670 2024-09-16T00:34:18.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.007+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2263918763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5419d6e0 con 0x7f2c54103670 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.007+0000 7f2c50ff9640 1 -- 172.21.15.1:0/2263918763 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3c003ce0 con 0x7f2c54103670 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.007+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2263918763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c541a5020 con 0x7f2c54103670 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.007+0000 7f2c50ff9640 1 -- 172.21.15.1:0/2263918763 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c035070 con 0x7f2c54103670 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.010+0000 7f2c50ff9640 1 -- 172.21.15.1:0/2263918763 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2c3c042030 con 0x7f2c54103670 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.010+0000 7f2c5c57c740 1 Processor -- start 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.010+0000 7f2c5c57c740 1 -- start start 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.010+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33fb40 0x55f83d33e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.010+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33e5d0 0x55f83d33ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 0x55f83d34b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f83d2a42f0 con 0x55f83d33fb40 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f83d24bcf0 con 0x55f83d33e5d0 2024-09-16T00:34:18.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f83d2a02b0 con 0x55f83d349370 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c53fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33fb40 0x55f83d33e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c53fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33fb40 0x55f83d33e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56312/0 (socket says 172.21.15.1:56312) 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c53fff640 1 -- 172.21.15.1:0/3524023216 learned_addr learned my addr 172.21.15.1:0/3524023216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c537fe640 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33e5d0 0x55f83d33ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 0x55f83d34b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c53fff640 1 -- 172.21.15.1:0/3524023216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 msgr2=0x55f83d34b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c53fff640 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 0x55f83d34b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c53fff640 1 -- 172.21.15.1:0/3524023216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33e5d0 msgr2=0x55f83d33ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c53fff640 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33e5d0 0x55f83d33ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c53fff640 1 -- 172.21.15.1:0/3524023216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83d2303a0 con 0x55f83d33fb40 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c537fe640 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33e5d0 0x55f83d33ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c53fff640 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33fb40 0x55f83d33e090 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c007f40 tx=0x7f2c4c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34785b44c1b7d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:18.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.011+0000 7f2c50ff9640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a3240 0x7f2c400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c49ffb640 1 -- 172.21.15.1:0/3524023216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c010900 con 0x55f83d33fb40 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c537fe640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a3240 0x7f2c400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c50ff9640 1 -- 172.21.15.1:0/2263918763 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2c3c0f0080 con 0x7f2c54103670 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c49ffb640 1 -- 172.21.15.1:0/3524023216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c4c0183f0 con 0x55f83d33fb40 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c49ffb640 1 -- 172.21.15.1:0/3524023216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c017600 con 0x55f83d33fb40 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c5c57c740 1 -- 172.21.15.1:0/3524023216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33fb40 msgr2=0x55f83d33e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33fb40 0x55f83d33e090 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c007f40 tx=0x7f2c4c00f850 comp rx=0 tx=0).stop 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c537fe640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a3240 0x7f2c400a5700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55f83d33bac0 tx=0x7f2c44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c5c57c740 1 -- 172.21.15.1:0/3524023216 shutdown_connections 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 0x55f83d34b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33e5d0 0x55f83d33ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/3524023216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33fb40 0x55f83d33e090 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c5c57c740 1 -- 172.21.15.1:0/3524023216 >> 172.21.15.1:0/3524023216 conn(0x55f83d307310 msgr2=0x55f83d33ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:18.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.012+0000 7f2c5c57c740 1 -- 172.21.15.1:0/3524023216 shutdown_connections 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 1 -- 172.21.15.1:0/3524023216 wait complete. 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 1 -- start start 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 1 Processor -- start 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d348d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33fb40 0x55f83d345fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 0x55f83d346570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f83d34c070 con 0x55f83d33e5d0 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f83d25f060 con 0x55f83d33fb40 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f83d34c1f0 con 0x55f83d349370 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c5c57c740 10 client.0 _ll_register_callbacks cb 0x55f83cfb28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.013+0000 7f2c537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d348d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d348d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56326/0 (socket says 172.21.15.1:56326) 2024-09-16T00:34:18.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c537fe640 1 -- 172.21.15.1:0/2704154190 learned_addr learned my addr 172.21.15.1:0/2704154190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c53fff640 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 0x55f83d346570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33fb40 0x55f83d345fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c537fe640 1 -- 172.21.15.1:0/2704154190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 msgr2=0x55f83d346570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c537fe640 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 0x55f83d346570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c537fe640 1 -- 172.21.15.1:0/2704154190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33fb40 msgr2=0x55f83d345fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c537fe640 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33fb40 0x55f83d345fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c537fe640 1 -- 172.21.15.1:0/2704154190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83d0153c0 con 0x55f83d33e5d0 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33fb40 0x55f83d345fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c537fe640 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d348d20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f2c440156f0 tx=0x7f2c440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3058a86757ce594f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c32ffd640 1 -- 172.21.15.1:0/2704154190 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44025020 con 0x55f83d33e5d0 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c32ffd640 1 -- 172.21.15.1:0/2704154190 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2c44014560 con 0x55f83d33e5d0 2024-09-16T00:34:18.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c5c57c740 1 -- 172.21.15.1:0/2704154190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c4c0183f0 con 0x55f83d33e5d0 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c32ffd640 1 -- 172.21.15.1:0/2704154190 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c44030050 con 0x55f83d33e5d0 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c5c57c740 10 client.16794 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.014+0000 7f2c5c57c740 1 -- 172.21.15.1:0/2704154190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f83d357bb0 con 0x55f83d33e5d0 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c32ffd640 1 -- 172.21.15.1:0/2704154190 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c4c0183f0 con 0x55f83d33e5d0 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c32ffd640 1 -- 172.21.15.1:0/2704154190 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2c440218f0 con 0x55f83d33e5d0 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c5c57c740 10 client.16794 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c5c57c740 10 client.16794 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c5c57c740 20 client.16794 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f83d359110 0x55f83d35b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c5c57c740 4 client.16794 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c5c57c740 1 -- 172.21.15.1:0/2704154190 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83d2b1860 con 0x55f83d359110 2024-09-16T00:34:18.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f83d359110 0x55f83d35b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f83d359110 0x55f83d35b530 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4f6b8880fc1b620 server_cookie=619ec1b8f5071535 in_seq=0 out_seq=0 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.015+0000 7f2c32ffd640 10 client.16794 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c32ffd640 1 -- 172.21.15.1:0/2704154190 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f83d2b1860 con 0x55f83d359110 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c32ffd640 10 client.16794 handle_command_reply: tid=0 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c5c57c740 1 client.16794 shutdown 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c5c57c740 1 -- 172.21.15.1:0/2704154190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 msgr2=0x55f83d348d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d348d20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f2c440156f0 tx=0x7f2c440046c0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c5c57c740 1 -- 172.21.15.1:0/2704154190 shutdown_connections 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d349370 0x55f83d346570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d33fb40 0x55f83d345fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f83d359110 0x55f83d35b530 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/2704154190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d348d20 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c32ffd640 0 client.16794 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:18.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.016+0000 7f2c5c57c740 1 -- 172.21.15.1:0/2704154190 >> 172.21.15.1:0/2704154190 conn(0x55f83d307310 msgr2=0x55f83d340930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:18.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.017+0000 7f2c5c57c740 1 -- 172.21.15.1:0/2704154190 shutdown_connections 2024-09-16T00:34:18.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.017+0000 7f2c5c57c740 1 -- 172.21.15.1:0/2704154190 wait complete. 2024-09-16T00:34:18.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.017+0000 7f2c5c57c740 20 client.16794 trim_cache size 0 max 16384 2024-09-16T00:34:18.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c5c57c740 1 Processor -- start 2024-09-16T00:34:18.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c5c57c740 1 -- start start 2024-09-16T00:34:18.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d344f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 0x55f83d3553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d349370 0x55f83d340820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c340125e0 con 0x55f83d33e5d0 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3400feb0 con 0x55f83d349370 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3400dd00 con 0x55f83d33fb40 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d344f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d344f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56342/0 (socket says 172.21.15.1:56342) 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c537fe640 1 -- 172.21.15.1:0/953366513 learned_addr learned my addr 172.21.15.1:0/953366513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c53fff640 1 --2- 172.21.15.1:0/953366513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d349370 0x55f83d340820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/953366513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 0x55f83d3553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c537fe640 1 -- 172.21.15.1:0/953366513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 msgr2=0x55f83d3553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.031+0000 7f2c537fe640 1 --2- 172.21.15.1:0/953366513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 0x55f83d3553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.032+0000 7f2c537fe640 1 -- 172.21.15.1:0/953366513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d349370 msgr2=0x55f83d340820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.032+0000 7f2c537fe640 1 --2- 172.21.15.1:0/953366513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d349370 0x55f83d340820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.032+0000 7f2c537fe640 1 -- 172.21.15.1:0/953366513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83d3468e0 con 0x55f83d33e5d0 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.032+0000 7f2c537fe640 1 --2- 172.21.15.1:0/953366513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d344f10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2c44008010 tx=0x7f2c44009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb58a44303b77e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.032+0000 7f2c497fa640 1 -- 172.21.15.1:0/953366513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44038070 con 0x55f83d33e5d0 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.032+0000 7f2c497fa640 1 -- 172.21.15.1:0/953366513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c4401a030 con 0x55f83d33e5d0 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.032+0000 7f2c497fa640 1 -- 172.21.15.1:0/953366513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44025040 con 0x55f83d33e5d0 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 -- 172.21.15.1:0/953366513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 msgr2=0x55f83d344f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/953366513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d344f10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2c44008010 tx=0x7f2c44009470 comp rx=0 tx=0).stop 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 -- 172.21.15.1:0/953366513 shutdown_connections 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/953366513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d349370 0x55f83d340820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/953366513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 0x55f83d3553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/953366513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d344f10 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 -- 172.21.15.1:0/953366513 >> 172.21.15.1:0/953366513 conn(0x55f83d307310 msgr2=0x55f83d340110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:18.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 -- 172.21.15.1:0/953366513 shutdown_connections 2024-09-16T00:34:18.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 -- 172.21.15.1:0/953366513 wait complete. 2024-09-16T00:34:18.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.033+0000 7f2c5c57c740 1 -- start start 2024-09-16T00:34:18.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c5c57c740 1 Processor -- start 2024-09-16T00:34:18.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d34d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 0x55f83d34da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c5c57c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d349370 0x55f83d34df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3400e370 con 0x55f83d33e5d0 2024-09-16T00:34:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3400f7e0 con 0x55f83d349370 2024-09-16T00:34:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c5c57c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3400ea40 con 0x55f83d33fb40 2024-09-16T00:34:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c5c57c740 10 client.0 _ll_register_callbacks cb 0x7f2c440097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d34d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d34d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56352/0 (socket says 172.21.15.1:56352) 2024-09-16T00:34:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c537fe640 1 -- 172.21.15.1:0/4205856113 learned_addr learned my addr 172.21.15.1:0/4205856113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 0x55f83d34da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c537fe640 1 -- 172.21.15.1:0/4205856113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 msgr2=0x55f83d34da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c537fe640 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 0x55f83d34da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c537fe640 1 -- 172.21.15.1:0/4205856113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d349370 msgr2=0x55f83d34df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c537fe640 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d349370 0x55f83d34df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c537fe640 1 -- 172.21.15.1:0/4205856113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83d3c3070 con 0x55f83d33e5d0 2024-09-16T00:34:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.034+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 0x55f83d34da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.035+0000 7f2c537fe640 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d34d500 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2c44011630 tx=0x7f2c44011fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3617f4dde53b1cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.035+0000 7f2c32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.035+0000 7f2c32ffd640 1 -- 172.21.15.1:0/4205856113 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44045070 con 0x55f83d33e5d0 2024-09-16T00:34:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.035+0000 7f2c32ffd640 1 -- 172.21.15.1:0/4205856113 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2c44041a70 con 0x55f83d33e5d0 2024-09-16T00:34:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.035+0000 7f2c5c57c740 1 -- 172.21.15.1:0/4205856113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c4401a030 con 0x55f83d33e5d0 2024-09-16T00:34:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.035+0000 7f2c32ffd640 1 -- 172.21.15.1:0/4205856113 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c44051050 con 0x55f83d33e5d0 2024-09-16T00:34:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.035+0000 7f2c5c57c740 10 client.16800 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.035+0000 7f2c5c57c740 1 -- 172.21.15.1:0/4205856113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f83d3c4350 con 0x55f83d33e5d0 2024-09-16T00:34:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.035+0000 7f2c32ffd640 1 -- 172.21.15.1:0/4205856113 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c44041cb0 con 0x55f83d33e5d0 2024-09-16T00:34:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c32ffd640 1 -- 172.21.15.1:0/4205856113 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2c44021930 con 0x55f83d33e5d0 2024-09-16T00:34:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c5c57c740 10 client.16800 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c5c57c740 10 client.16800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:34:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c5c57c740 20 client.16800 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f83d3c4900 0x55f83d3c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c5c57c740 4 client.16800 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c5c57c740 1 -- 172.21.15.1:0/4205856113 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2c4401a030 con 0x55f83d3c4900 2024-09-16T00:34:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f83d3c4900 0x55f83d3c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c32ffd640 10 client.16800 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.036+0000 7f2c52ffd640 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f83d3c4900 0x55f83d3c6cf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9df1362b167458a1 server_cookie=502c0199318b1a0f in_seq=0 out_seq=0 2024-09-16T00:34:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c32ffd640 1 -- 172.21.15.1:0/4205856113 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2c4401a030 con 0x55f83d3c4900 2024-09-16T00:34:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c32ffd640 10 client.16800 handle_command_reply: tid=0 2024-09-16T00:34:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 client.16800 shutdown 2024-09-16T00:34:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 -- 172.21.15.1:0/4205856113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 msgr2=0x55f83d34d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d34d500 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2c44011630 tx=0x7f2c44011fc0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 -- 172.21.15.1:0/4205856113 shutdown_connections 2024-09-16T00:34:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83d349370 0x55f83d34df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f83d33fb40 0x55f83d34da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55f83d3c4900 0x55f83d3c6cf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 --2- 172.21.15.1:0/4205856113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f83d33e5d0 0x55f83d34d500 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 -- 172.21.15.1:0/4205856113 >> 172.21.15.1:0/4205856113 conn(0x55f83d307310 msgr2=0x55f83d340930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c32ffd640 0 client.16800 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 -- 172.21.15.1:0/4205856113 shutdown_connections 2024-09-16T00:34:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 1 -- 172.21.15.1:0/4205856113 wait complete. 2024-09-16T00:34:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.037+0000 7f2c5c57c740 20 client.16800 trim_cache size 0 max 16384 2024-09-16T00:34:18.051 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a3240 msgr2=0x7f2c400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a3240 0x7f2c400a5700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55f83d33bac0 tx=0x7f2c44008040 comp rx=0 tx=0).stop 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 msgr2=0x7f2c541989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 0x7f2c541989e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005ab0 tx=0x7f2c3c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2263918763 shutdown_connections 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a3240 0x7f2c400a5700 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c5410c180 0x7f2c5419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54103670 0x7f2c541989e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 --2- 172.21.15.1:0/2263918763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102d50 0x7f2c541984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2263918763 >> 172.21.15.1:0/2263918763 conn(0x7f2c540fc7d0 msgr2=0x7f2c540fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2263918763 shutdown_connections 2024-09-16T00:34:18.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:18.038+0000 7f2c5a1a3640 1 -- 172.21.15.1:0/2263918763 wait complete. 2024-09-16T00:34:18.749 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:34:19.011 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.561+0000 7ff04f37d640 1 -- 172.21.15.1:0/2349103326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048101de0 msgr2=0x7ff0481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.561+0000 7ff04f37d640 1 --2- 172.21.15.1:0/2349103326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048101de0 0x7ff0481021c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff044009900 tx=0x7ff04402a410 comp rx=0 tx=0).stop 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.561+0000 7ff04f37d640 1 -- 172.21.15.1:0/2349103326 shutdown_connections 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.561+0000 7ff04f37d640 1 --2- 172.21.15.1:0/2349103326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff048106f10 0x7ff04810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.561+0000 7ff04f37d640 1 --2- 172.21.15.1:0/2349103326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff048102790 0x7ff0481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.561+0000 7ff04f37d640 1 --2- 172.21.15.1:0/2349103326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048101de0 0x7ff0481021c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.561+0000 7ff04f37d640 1 -- 172.21.15.1:0/2349103326 >> 172.21.15.1:0/2349103326 conn(0x7ff0480fd8c0 msgr2=0x7ff0480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.561+0000 7ff04f37d640 1 -- 172.21.15.1:0/2349103326 shutdown_connections 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.562+0000 7ff04f37d640 1 -- 172.21.15.1:0/2349103326 wait complete. 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.562+0000 7ff04f37d640 1 Processor -- start 2024-09-16T00:34:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.562+0000 7ff04f37d640 1 -- start start 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.562+0000 7ff04f37d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff048101de0 0x7ff0480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.562+0000 7ff04f37d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048102790 0x7ff048074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.562+0000 7ff04f37d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff048106f10 0x7ff048074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.562+0000 7ff04f37d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff048111f70 con 0x7ff048102790 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.562+0000 7ff04f37d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff048111df0 con 0x7ff048106f10 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.562+0000 7ff04f37d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0481120f0 con 0x7ff048101de0 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04c918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048102790 0x7ff048074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04c918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048102790 0x7ff048074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56368/0 (socket says 172.21.15.1:56368) 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04c918640 1 -- 172.21.15.1:0/3844976355 learned_addr learned my addr 172.21.15.1:0/3844976355 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04d91a640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff048106f10 0x7ff048074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04d119640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff048101de0 0x7ff0480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04c918640 1 -- 172.21.15.1:0/3844976355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff048101de0 msgr2=0x7ff0480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04c918640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff048101de0 0x7ff0480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04c918640 1 -- 172.21.15.1:0/3844976355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff048106f10 msgr2=0x7ff048074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04c918640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff048106f10 0x7ff048074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04c918640 1 -- 172.21.15.1:0/3844976355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0481a6870 con 0x7ff048102790 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04c918640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048102790 0x7ff048074a30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c920 tx=0x7ff03800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff03e7fc640 1 -- 172.21.15.1:0/3844976355 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038007c70 con 0x7ff048102790 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff03e7fc640 1 -- 172.21.15.1:0/3844976355 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff038007e10 con 0x7ff048102790 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff03e7fc640 1 -- 172.21.15.1:0/3844976355 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038018890 con 0x7ff048102790 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04f37d640 1 -- 172.21.15.1:0/3844976355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0481a6b60 con 0x7ff048102790 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.563+0000 7ff04f37d640 1 -- 172.21.15.1:0/3844976355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0481a70a0 con 0x7ff048102790 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff03e7fc640 1 -- 172.21.15.1:0/3844976355 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff038018a70 con 0x7ff048102790 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff051756740 1 Processor -- start 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff051756740 1 -- start start 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff051756740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa970090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff051756740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 0x556cfa970a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff051756740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa97d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff051756740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556cfa8d62f0 con 0x556cfa97b370 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff051756740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556cfa87dcf0 con 0x556cfa971b40 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff051756740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556cfa8d22b0 con 0x556cfa9705d0 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff04c918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa97d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff04c918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa97d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56370/0 (socket says 172.21.15.1:56370) 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.566+0000 7ff04c918640 1 -- 172.21.15.1:0/2025386261 learned_addr learned my addr 172.21.15.1:0/2025386261 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff03e7fc640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0340a3240 0x7ff0340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04d119640 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 0x556cfa970a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04d91a640 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa970090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04c918640 1 -- 172.21.15.1:0/2025386261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 msgr2=0x556cfa970a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04c918640 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 0x556cfa970a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04c918640 1 -- 172.21.15.1:0/2025386261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 msgr2=0x556cfa970090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04c918640 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa970090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04c918640 1 -- 172.21.15.1:0/2025386261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfa8623a0 con 0x556cfa97b370 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04d91a640 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa970090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04d91a640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0340a3240 0x7ff0340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff03e7fc640 1 -- 172.21.15.1:0/3844976355 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff0380cfdb0 con 0x7ff048102790 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff04c918640 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa97d760 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff038057070 tx=0x7ff038020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f500e91c49ab70a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff0337fe640 1 -- 172.21.15.1:0/2025386261 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03809bb10 con 0x556cfa97b370 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff0337fe640 1 -- 172.21.15.1:0/2025386261 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff03809c6d0 con 0x556cfa97b370 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.567+0000 7ff0337fe640 1 -- 172.21.15.1:0/2025386261 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03809b930 con 0x556cfa97b370 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff051756740 1 -- 172.21.15.1:0/2025386261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 msgr2=0x556cfa97d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff051756740 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa97d760 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff038057070 tx=0x7ff038020000 comp rx=0 tx=0).stop 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff04d91a640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0340a3240 0x7ff0340a5700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff040005fd0 tx=0x7ff040014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff051756740 1 -- 172.21.15.1:0/2025386261 shutdown_connections 2024-09-16T00:34:19.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff051756740 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa97d760 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff051756740 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 0x556cfa970a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff051756740 1 --2- 172.21.15.1:0/2025386261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa970090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff051756740 1 -- 172.21.15.1:0/2025386261 >> 172.21.15.1:0/2025386261 conn(0x556cfa939310 msgr2=0x556cfa96cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff051756740 1 -- 172.21.15.1:0/2025386261 shutdown_connections 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.568+0000 7ff051756740 1 -- 172.21.15.1:0/2025386261 wait complete. 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff051756740 1 -- start start 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff051756740 1 Processor -- start 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff051756740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 0x556cfa97ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff051756740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa977fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff051756740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa9785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff051756740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556cfa97e010 con 0x556cfa97b370 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff051756740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556cfa891060 con 0x556cfa971b40 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff051756740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556cfa97e190 con 0x556cfa9705d0 2024-09-16T00:34:19.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff051756740 10 client.0 _ll_register_callbacks cb 0x556cfa5e48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff04d91a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa9785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff04d91a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa9785d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56372/0 (socket says 172.21.15.1:56372) 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff04d91a640 1 -- 172.21.15.1:0/4248192586 learned_addr learned my addr 172.21.15.1:0/4248192586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff04d119640 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 0x556cfa97ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.569+0000 7ff04c918640 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa977fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff04d91a640 1 -- 172.21.15.1:0/4248192586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 msgr2=0x556cfa97ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff04d91a640 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 0x556cfa97ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff04d91a640 1 -- 172.21.15.1:0/4248192586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 msgr2=0x556cfa977fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff04d91a640 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa977fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff04d91a640 1 -- 172.21.15.1:0/4248192586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfa6473c0 con 0x556cfa97b370 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff04c918640 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa977fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff04d91a640 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa9785d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff040010c20 tx=0x7ff040009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe0b7cab50577d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff030ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:19.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff030ff9640 1 -- 172.21.15.1:0/4248192586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040010e60 con 0x556cfa97b370 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff030ff9640 1 -- 172.21.15.1:0/4248192586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff04001eba0 con 0x556cfa97b370 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff051756740 1 -- 172.21.15.1:0/4248192586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03809c6d0 con 0x556cfa97b370 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff030ff9640 1 -- 172.21.15.1:0/4248192586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff04001ede0 con 0x556cfa97b370 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff051756740 10 client.16812 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff051756740 1 -- 172.21.15.1:0/4248192586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cfa989ad0 con 0x556cfa97b370 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.570+0000 7ff030ff9640 1 -- 172.21.15.1:0/4248192586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff03809c6d0 con 0x556cfa97b370 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff030ff9640 1 -- 172.21.15.1:0/4248192586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff040025070 con 0x556cfa97b370 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff051756740 10 client.16812 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff051756740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff051756740 20 client.16812 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff051756740 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556cfa98af90 0x556cfa98d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff051756740 4 client.16812 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:19.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff051756740 1 -- 172.21.15.1:0/4248192586 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cfa8e3860 con 0x556cfa98af90 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff04d119640 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556cfa98af90 0x556cfa98d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff030ff9640 10 client.16812 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.571+0000 7ff04d119640 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556cfa98af90 0x556cfa98d3d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfec416b5fc7bda1 server_cookie=662d110998c07a4c in_seq=0 out_seq=0 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff030ff9640 1 -- 172.21.15.1:0/4248192586 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cfa8e3860 con 0x556cfa98af90 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff030ff9640 10 client.16812 handle_command_reply: tid=0 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 client.16812 shutdown 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 -- 172.21.15.1:0/4248192586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 msgr2=0x556cfa9785d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa9785d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff040010c20 tx=0x7ff040009500 comp rx=0 tx=0).stop 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 -- 172.21.15.1:0/4248192586 shutdown_connections 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556cfa98af90 0x556cfa98d3d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa97b370 0x556cfa9785d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff030ff9640 0 client.16812 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa971b40 0x556cfa977fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 --2- 172.21.15.1:0/4248192586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa9705d0 0x556cfa97ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 -- 172.21.15.1:0/4248192586 >> 172.21.15.1:0/4248192586 conn(0x556cfa939310 msgr2=0x556cfa972990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.572+0000 7ff051756740 1 -- 172.21.15.1:0/4248192586 shutdown_connections 2024-09-16T00:34:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.573+0000 7ff051756740 1 -- 172.21.15.1:0/4248192586 wait complete. 2024-09-16T00:34:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.573+0000 7ff051756740 20 client.16812 trim_cache size 0 max 16384 2024-09-16T00:34:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff051756740 1 Processor -- start 2024-09-16T00:34:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff051756740 1 -- start start 2024-09-16T00:34:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff051756740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa9705d0 0x556cfa967c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff051756740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa971b40 0x556cfa979f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff051756740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa97b370 0x556cfa973120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff051756740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0481b3890 con 0x556cfa9705d0 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff051756740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0481b1160 con 0x556cfa97b370 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff051756740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0481aefb0 con 0x556cfa971b40 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff04d119640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa9705d0 0x556cfa967c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff04d119640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa9705d0 0x556cfa967c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56374/0 (socket says 172.21.15.1:56374) 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.586+0000 7ff04d119640 1 -- 172.21.15.1:0/3058586734 learned_addr learned my addr 172.21.15.1:0/3058586734 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff04c918640 1 --2- 172.21.15.1:0/3058586734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa971b40 0x556cfa979f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff04d91a640 1 --2- 172.21.15.1:0/3058586734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa97b370 0x556cfa973120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff04d119640 1 -- 172.21.15.1:0/3058586734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa971b40 msgr2=0x556cfa979f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff04d119640 1 --2- 172.21.15.1:0/3058586734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa971b40 0x556cfa979f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff04d119640 1 -- 172.21.15.1:0/3058586734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa97b370 msgr2=0x556cfa973120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff04d119640 1 --2- 172.21.15.1:0/3058586734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa97b370 0x556cfa973120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff04d119640 1 -- 172.21.15.1:0/3058586734 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfa96d960 con 0x556cfa9705d0 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff04d119640 1 --2- 172.21.15.1:0/3058586734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa9705d0 0x556cfa967c60 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff044009850 tx=0x7ff044031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabd7fb9f326ed19 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff032ffd640 1 -- 172.21.15.1:0/3058586734 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044035040 con 0x556cfa9705d0 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff032ffd640 1 -- 172.21.15.1:0/3058586734 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff044005500 con 0x556cfa9705d0 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.587+0000 7ff032ffd640 1 -- 172.21.15.1:0/3058586734 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044004d00 con 0x556cfa9705d0 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 -- 172.21.15.1:0/3058586734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa9705d0 msgr2=0x556cfa967c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 --2- 172.21.15.1:0/3058586734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa9705d0 0x556cfa967c60 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff044009850 tx=0x7ff044031b90 comp rx=0 tx=0).stop 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 -- 172.21.15.1:0/3058586734 shutdown_connections 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 --2- 172.21.15.1:0/3058586734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa97b370 0x556cfa973120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 --2- 172.21.15.1:0/3058586734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa971b40 0x556cfa979f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 --2- 172.21.15.1:0/3058586734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa9705d0 0x556cfa967c60 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 -- 172.21.15.1:0/3058586734 >> 172.21.15.1:0/3058586734 conn(0x556cfa939310 msgr2=0x556cfa979840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 -- 172.21.15.1:0/3058586734 shutdown_connections 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 -- 172.21.15.1:0/3058586734 wait complete. 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 -- start start 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.588+0000 7ff051756740 1 Processor -- start 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff051756740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa9705d0 0x556cfa97f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff051756740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa971b40 0x556cfa97fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff051756740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa97b370 0x556cfa9fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff051756740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0481af620 con 0x556cfa971b40 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff051756740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0481b0a90 con 0x556cfa9705d0 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff051756740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0481afcf0 con 0x556cfa97b370 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff051756740 10 client.0 _ll_register_callbacks cb 0x7ff040009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04c918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa971b40 0x556cfa97fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04c918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa971b40 0x556cfa97fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56380/0 (socket says 172.21.15.1:56380) 2024-09-16T00:34:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04c918640 1 -- 172.21.15.1:0/2874854460 learned_addr learned my addr 172.21.15.1:0/2874854460 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04d91a640 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa97b370 0x556cfa9fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04d119640 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa9705d0 0x556cfa97f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04c918640 1 -- 172.21.15.1:0/2874854460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa97b370 msgr2=0x556cfa9fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04c918640 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa97b370 0x556cfa9fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04c918640 1 -- 172.21.15.1:0/2874854460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa9705d0 msgr2=0x556cfa97f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04c918640 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa9705d0 0x556cfa97f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.589+0000 7ff04c918640 1 -- 172.21.15.1:0/2874854460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfa9802e0 con 0x556cfa971b40 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff04c918640 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa971b40 0x556cfa97fa00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff038098950 tx=0x7ff03800ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503077817b2b28c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff030ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff030ff9640 1 -- 172.21.15.1:0/2874854460 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03809b930 con 0x556cfa971b40 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff030ff9640 1 -- 172.21.15.1:0/2874854460 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff038097ea0 con 0x556cfa971b40 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff051756740 1 -- 172.21.15.1:0/2874854460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff044005500 con 0x556cfa971b40 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff030ff9640 1 -- 172.21.15.1:0/2874854460 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff038082050 con 0x556cfa971b40 2024-09-16T00:34:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff051756740 10 client.16818 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff051756740 1 -- 172.21.15.1:0/2874854460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cfa9feaf0 con 0x556cfa971b40 2024-09-16T00:34:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff030ff9640 1 -- 172.21.15.1:0/2874854460 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff044005500 con 0x556cfa971b40 2024-09-16T00:34:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff030ff9640 1 -- 172.21.15.1:0/2874854460 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff03809c230 con 0x556cfa971b40 2024-09-16T00:34:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff051756740 10 client.16818 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff051756740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff051756740 20 client.16818 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff051756740 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556cfa98af90 0x556cfaa00e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff051756740 4 client.16818 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.590+0000 7ff051756740 1 -- 172.21.15.1:0/2874854460 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cfa8ddd70 con 0x556cfa98af90 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff04d119640 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556cfa98af90 0x556cfaa00e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff030ff9640 10 client.16818 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff04d119640 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556cfa98af90 0x556cfaa00e80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa38c1fdbfaf8792 server_cookie=43cddce0f18a4830 in_seq=0 out_seq=0 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff030ff9640 1 -- 172.21.15.1:0/2874854460 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cfa8ddd70 con 0x556cfa98af90 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff030ff9640 10 client.16818 handle_command_reply: tid=0 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff051756740 1 client.16818 shutdown 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff051756740 1 -- 172.21.15.1:0/2874854460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa971b40 msgr2=0x556cfa97fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff051756740 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa971b40 0x556cfa97fa00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff038098950 tx=0x7ff03800ff30 comp rx=0 tx=0).stop 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff051756740 1 -- 172.21.15.1:0/2874854460 shutdown_connections 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.591+0000 7ff051756740 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556cfa98af90 0x556cfaa00e80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff051756740 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556cfa97b370 0x556cfa9fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff030ff9640 0 client.16818 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff051756740 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556cfa971b40 0x556cfa97fa00 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff051756740 1 --2- 172.21.15.1:0/2874854460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556cfa9705d0 0x556cfa97f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff051756740 1 -- 172.21.15.1:0/2874854460 >> 172.21.15.1:0/2874854460 conn(0x556cfa939310 msgr2=0x556cfa96bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff051756740 1 -- 172.21.15.1:0/2874854460 shutdown_connections 2024-09-16T00:34:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff051756740 1 -- 172.21.15.1:0/2874854460 wait complete. 2024-09-16T00:34:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff051756740 20 client.16818 trim_cache size 0 max 16384 2024-09-16T00:34:19.605 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:34:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 -- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0340a3240 msgr2=0x7ff0340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0340a3240 0x7ff0340a5700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff040005fd0 tx=0x7ff040014040 comp rx=0 tx=0).stop 2024-09-16T00:34:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 -- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048102790 msgr2=0x7ff048074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048102790 0x7ff048074a30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c920 tx=0x7ff03800cde0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 -- 172.21.15.1:0/3844976355 shutdown_connections 2024-09-16T00:34:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff048106f10 0x7ff048074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff048102790 0x7ff048074a30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0340a3240 0x7ff0340a5700 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 --2- 172.21.15.1:0/3844976355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff048101de0 0x7ff0480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 -- 172.21.15.1:0/3844976355 >> 172.21.15.1:0/3844976355 conn(0x7ff0480fd8c0 msgr2=0x7ff0480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.592+0000 7ff04f37d640 1 -- 172.21.15.1:0/3844976355 shutdown_connections 2024-09-16T00:34:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:19.593+0000 7ff04f37d640 1 -- 172.21.15.1:0/3844976355 wait complete. 2024-09-16T00:34:19.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:19 smithi019 ceph-mon[35186]: pgmap v528: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 8 op/s 2024-09-16T00:34:19.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:19 smithi001 ceph-mon[30934]: pgmap v528: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 8 op/s 2024-09-16T00:34:19.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:19 smithi153 ceph-mon[35538]: pgmap v528: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 8 op/s 2024-09-16T00:34:21.372 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:34:21.372 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:34:21.644 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:21.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:21 smithi001 ceph-mon[30934]: pgmap v529: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-16T00:34:21.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:21 smithi019 ceph-mon[35186]: pgmap v529: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-16T00:34:21.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:21 smithi153 ceph-mon[35538]: pgmap v529: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-16T00:34:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.122+0000 7f91402e2640 1 -- 172.21.15.1:0/1855584012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138101400 msgr2=0x7f913810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.122+0000 7f91402e2640 1 --2- 172.21.15.1:0/1855584012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138101400 0x7f913810d930 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9130009cd0 tx=0x7f913002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.122+0000 7f91402e2640 1 -- 172.21.15.1:0/1855584012 shutdown_connections 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.122+0000 7f91402e2640 1 --2- 172.21.15.1:0/1855584012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138101400 0x7f913810d930 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.122+0000 7f91402e2640 1 --2- 172.21.15.1:0/1855584012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138100a60 0x7f9138100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.122+0000 7f91402e2640 1 --2- 172.21.15.1:0/1855584012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9138106a60 0x7f9138106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.122+0000 7f91402e2640 1 -- 172.21.15.1:0/1855584012 >> 172.21.15.1:0/1855584012 conn(0x7f91380fc780 msgr2=0x7f91380feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.122+0000 7f91402e2640 1 -- 172.21.15.1:0/1855584012 shutdown_connections 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.122+0000 7f91402e2640 1 -- 172.21.15.1:0/1855584012 wait complete. 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f91402e2640 1 Processor -- start 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f91402e2640 1 -- start start 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f91402e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138100a60 0x7f91381983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f91402e2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138101400 0x7f9138198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f91402e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9138106a60 0x7f913819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f91402e2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9138110e60 con 0x7f9138100a60 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f91402e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9138110ce0 con 0x7f9138106a60 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f91402e2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9138110fe0 con 0x7f9138101400 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913e07e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138100a60 0x7f91381983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913e07e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138100a60 0x7f91381983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45814/0 (socket says 172.21.15.1:45814) 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913e07e640 1 -- 172.21.15.1:0/2105385323 learned_addr learned my addr 172.21.15.1:0/2105385323 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913e87f640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9138106a60 0x7f913819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913d87d640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138101400 0x7f9138198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913e07e640 1 -- 172.21.15.1:0/2105385323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138101400 msgr2=0x7f9138198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913e07e640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138101400 0x7f9138198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913e07e640 1 -- 172.21.15.1:0/2105385323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9138106a60 msgr2=0x7f913819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913e07e640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9138106a60 0x7f913819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.123+0000 7f913e07e640 1 -- 172.21.15.1:0/2105385323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913819d390 con 0x7f9138100a60 2024-09-16T00:34:22.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.124+0000 7f913e07e640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138100a60 0x7f91381983e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f913400d950 tx=0x7f913400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.124+0000 7f912f7fe640 1 -- 172.21.15.1:0/2105385323 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134017070 con 0x7f9138100a60 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.124+0000 7f912f7fe640 1 -- 172.21.15.1:0/2105385323 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9134009c30 con 0x7f9138100a60 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.124+0000 7f91402e2640 1 -- 172.21.15.1:0/2105385323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913819d680 con 0x7f9138100a60 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.124+0000 7f912f7fe640 1 -- 172.21.15.1:0/2105385323 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913400f6b0 con 0x7f9138100a60 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.124+0000 7f91402e2640 1 -- 172.21.15.1:0/2105385323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91381a4f60 con 0x7f9138100a60 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.126+0000 7f912f7fe640 1 -- 172.21.15.1:0/2105385323 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f91340109d0 con 0x7f9138100a60 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.126+0000 7f91426bb740 1 Processor -- start 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f91426bb740 1 -- start start 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f91426bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43a8b60 0x5611b43a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f91426bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f91426bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43b3d40 0x5611b43b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f91426bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5611b430e280 con 0x5611b43a9480 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f91426bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611b42b5b10 con 0x5611b43a8b60 2024-09-16T00:34:22.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f91426bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5611b430a240 con 0x5611b43b3d40 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f913e07e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f912f7fe640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f91240a32c0 0x7f91240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f913e07e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45828/0 (socket says 172.21.15.1:45828) 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f913e07e640 1 -- 172.21.15.1:0/4126012397 learned_addr learned my addr 172.21.15.1:0/4126012397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f913e87f640 1 --2- 172.21.15.1:0/4126012397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43a8b60 0x5611b43a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f913d87d640 1 --2- 172.21.15.1:0/4126012397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43b3d40 0x5611b43b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f913e87f640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f91240a32c0 0x7f91240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.127+0000 7f912f7fe640 1 -- 172.21.15.1:0/2105385323 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f91340cf8e0 con 0x7f9138100a60 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f913e07e640 1 -- 172.21.15.1:0/4126012397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43b3d40 msgr2=0x5611b43b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f913e07e640 1 --2- 172.21.15.1:0/4126012397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43b3d40 0x5611b43b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f913e07e640 1 -- 172.21.15.1:0/4126012397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43a8b60 msgr2=0x5611b43a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f913e07e640 1 --2- 172.21.15.1:0/4126012397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43a8b60 0x5611b43a8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f913e07e640 1 -- 172.21.15.1:0/4126012397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b42c21f0 con 0x5611b43a9480 2024-09-16T00:34:22.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f913e07e640 1 --2- 172.21.15.1:0/4126012397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b3800 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f913409b800 tx=0x7f9134000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d364b49e238c78ff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f913e87f640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f91240a32c0 0x7f91240a5780 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9130009430 tx=0x7f9130032f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f912cdea640 1 -- 172.21.15.1:0/4126012397 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134020070 con 0x5611b43a9480 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f912cdea640 1 -- 172.21.15.1:0/4126012397 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9134084e50 con 0x5611b43a9480 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.128+0000 7f912cdea640 1 -- 172.21.15.1:0/4126012397 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134098170 con 0x5611b43a9480 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 -- 172.21.15.1:0/4126012397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 msgr2=0x5611b43b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 --2- 172.21.15.1:0/4126012397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b3800 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f913409b800 tx=0x7f9134000bc0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 -- 172.21.15.1:0/4126012397 shutdown_connections 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 --2- 172.21.15.1:0/4126012397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43b3d40 0x5611b43b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 --2- 172.21.15.1:0/4126012397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b3800 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 --2- 172.21.15.1:0/4126012397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43a8b60 0x5611b43a8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 -- 172.21.15.1:0/4126012397 >> 172.21.15.1:0/4126012397 conn(0x5611b43a50f0 msgr2=0x5611b43a54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 -- 172.21.15.1:0/4126012397 shutdown_connections 2024-09-16T00:34:22.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 -- 172.21.15.1:0/4126012397 wait complete. 2024-09-16T00:34:22.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 -- start start 2024-09-16T00:34:22.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.129+0000 7f91426bb740 1 Processor -- start 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f91426bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a8b60 0x5611b43b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f91426bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f91426bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43b3d40 0x5611b43b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f91426bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5611b43b68b0 con 0x5611b43a9480 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f91426bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611b42c8f10 con 0x5611b43b3d40 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f91426bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5611b43b6a30 con 0x5611b43a8b60 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f91426bb740 10 client.0 _ll_register_callbacks cb 0x5611b4291a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913d87d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913d87d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45838/0 (socket says 172.21.15.1:45838) 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913d87d640 1 -- 172.21.15.1:0/3325641455 learned_addr learned my addr 172.21.15.1:0/3325641455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:22.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913e07e640 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a8b60 0x5611b43b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913d87d640 1 -- 172.21.15.1:0/3325641455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a8b60 msgr2=0x5611b43b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913d87d640 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a8b60 0x5611b43b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913d87d640 1 -- 172.21.15.1:0/3325641455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43b3d40 msgr2=0x5611b43b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913d87d640 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43b3d40 0x5611b43b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913d87d640 1 -- 172.21.15.1:0/3325641455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611b407f6a0 con 0x5611b43a9480 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913e07e640 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a8b60 0x5611b43b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f913d87d640 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b1160 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f91280106e0 tx=0x7f9128010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4935f69c46cc6ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.130+0000 7f9121ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f9121ffb640 1 -- 172.21.15.1:0/3325641455 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912801d060 con 0x5611b43a9480 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f9121ffb640 1 -- 172.21.15.1:0/3325641455 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f912800eda0 con 0x5611b43a9480 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f91426bb740 1 -- 172.21.15.1:0/3325641455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9134084e50 con 0x5611b43a9480 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f9121ffb640 1 -- 172.21.15.1:0/3325641455 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9128024050 con 0x5611b43a9480 2024-09-16T00:34:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f91426bb740 10 client.16830 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f91426bb740 1 -- 172.21.15.1:0/3325641455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5611b43c2590 con 0x5611b43a9480 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f9121ffb640 1 -- 172.21.15.1:0/3325641455 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9134084e50 con 0x5611b43a9480 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f9121ffb640 1 -- 172.21.15.1:0/3325641455 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f91280164f0 con 0x5611b43a9480 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f91426bb740 10 client.16830 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f91426bb740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f91426bb740 20 client.16830 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f91426bb740 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5611b43c3aa0 0x5611b43c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f91426bb740 4 client.16830 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.131+0000 7f91426bb740 1 -- 172.21.15.1:0/3325641455 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611b431b7f0 con 0x5611b43c3aa0 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.132+0000 7f913e07e640 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5611b43c3aa0 0x5611b43c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.132+0000 7f9121ffb640 10 client.16830 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.132+0000 7f913e07e640 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5611b43c3aa0 0x5611b43c5f10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9fa8110f277cddf server_cookie=436971f4f1cc59eb in_seq=0 out_seq=0 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f9121ffb640 1 -- 172.21.15.1:0/3325641455 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611b431b7f0 con 0x5611b43c3aa0 2024-09-16T00:34:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f9121ffb640 10 client.16830 handle_command_reply: tid=0 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 client.16830 shutdown 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 -- 172.21.15.1:0/3325641455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 msgr2=0x5611b43b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b1160 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f91280106e0 tx=0x7f9128010ba0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 -- 172.21.15.1:0/3325641455 shutdown_connections 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5611b43c3aa0 0x5611b43c5f10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43b3d40 0x5611b43b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f9121ffb640 0 client.16830 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a9480 0x5611b43b1160 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 --2- 172.21.15.1:0/3325641455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a8b60 0x5611b43b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 -- 172.21.15.1:0/3325641455 >> 172.21.15.1:0/3325641455 conn(0x5611b43a50f0 msgr2=0x5611b43aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 -- 172.21.15.1:0/3325641455 shutdown_connections 2024-09-16T00:34:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 1 -- 172.21.15.1:0/3325641455 wait complete. 2024-09-16T00:34:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.133+0000 7f91426bb740 20 client.16830 trim_cache size 0 max 16384 2024-09-16T00:34:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.150+0000 7f91426bb740 1 Processor -- start 2024-09-16T00:34:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.150+0000 7f91426bb740 1 -- start start 2024-09-16T00:34:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f91426bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f91426bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43a9480 0x5611b43cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f91426bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43b3d40 0x5611b43cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f91426bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f91381b17a0 con 0x5611b43a8b60 2024-09-16T00:34:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f91426bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91381af070 con 0x5611b43a9480 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f91426bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91381acec0 con 0x5611b43b3d40 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f913e07e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f913e07e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45842/0 (socket says 172.21.15.1:45842) 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f913e07e640 1 -- 172.21.15.1:0/3434306211 learned_addr learned my addr 172.21.15.1:0/3434306211 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f913e07e640 1 -- 172.21.15.1:0/3434306211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43b3d40 msgr2=0x5611b43cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f913e07e640 1 --2- 172.21.15.1:0/3434306211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43b3d40 0x5611b43cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f913e07e640 1 -- 172.21.15.1:0/3434306211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43a9480 msgr2=0x5611b43cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f913e07e640 1 --2- 172.21.15.1:0/3434306211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43a9480 0x5611b43cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f913e07e640 1 -- 172.21.15.1:0/3434306211 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b43b34f0 con 0x5611b43a8b60 2024-09-16T00:34:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f913e07e640 1 --2- 172.21.15.1:0/3434306211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b2a80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f91340cd030 tx=0x7f9134086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d77c3d72e0d131 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:22.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f9123fff640 1 -- 172.21.15.1:0/3434306211 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91340b9070 con 0x5611b43a8b60 2024-09-16T00:34:22.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f9123fff640 1 -- 172.21.15.1:0/3434306211 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9134084e50 con 0x5611b43a8b60 2024-09-16T00:34:22.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.151+0000 7f9123fff640 1 -- 172.21.15.1:0/3434306211 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134020040 con 0x5611b43a8b60 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 -- 172.21.15.1:0/3434306211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 msgr2=0x5611b43b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 --2- 172.21.15.1:0/3434306211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b2a80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f91340cd030 tx=0x7f9134086040 comp rx=0 tx=0).stop 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 -- 172.21.15.1:0/3434306211 shutdown_connections 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 --2- 172.21.15.1:0/3434306211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43b3d40 0x5611b43cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 --2- 172.21.15.1:0/3434306211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43a9480 0x5611b43cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 --2- 172.21.15.1:0/3434306211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b2a80 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 -- 172.21.15.1:0/3434306211 >> 172.21.15.1:0/3434306211 conn(0x5611b43a50f0 msgr2=0x5611b43b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 -- 172.21.15.1:0/3434306211 shutdown_connections 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 -- 172.21.15.1:0/3434306211 wait complete. 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 -- start start 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.152+0000 7f91426bb740 1 Processor -- start 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f91426bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f91426bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a9480 0x5611b43b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f91426bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43b3d40 0x5611b44267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f91426bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ad530 con 0x5611b43a8b60 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f91426bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ae9a0 con 0x5611b43b3d40 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f91426bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91381adc00 con 0x5611b43a9480 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f91426bb740 10 client.0 _ll_register_callbacks cb 0x7f912800b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e07e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e07e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45858/0 (socket says 172.21.15.1:45858) 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e07e640 1 -- 172.21.15.1:0/851240719 learned_addr learned my addr 172.21.15.1:0/851240719 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913d87d640 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a9480 0x5611b43b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e87f640 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43b3d40 0x5611b44267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e07e640 1 -- 172.21.15.1:0/851240719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a9480 msgr2=0x5611b43b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e07e640 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a9480 0x5611b43b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e07e640 1 -- 172.21.15.1:0/851240719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43b3d40 msgr2=0x5611b44267b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e07e640 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43b3d40 0x5611b44267b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e07e640 1 -- 172.21.15.1:0/851240719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611b4426fe0 con 0x5611b43a8b60 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913d87d640 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a9480 0x5611b43b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f913e07e640 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b85e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f913400bfa0 tx=0x7f91340a4770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473374dbe9adab3a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f9121ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.153+0000 7f9121ffb640 1 -- 172.21.15.1:0/851240719 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91340b9070 con 0x5611b43a8b60 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f9121ffb640 1 -- 172.21.15.1:0/851240719 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f91340860b0 con 0x5611b43a8b60 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f91426bb740 1 -- 172.21.15.1:0/851240719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9134084e50 con 0x5611b43a8b60 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f9121ffb640 1 -- 172.21.15.1:0/851240719 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9134020040 con 0x5611b43a8b60 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f91426bb740 10 client.16836 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f91426bb740 1 -- 172.21.15.1:0/851240719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5611b442f330 con 0x5611b43a8b60 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f9121ffb640 1 -- 172.21.15.1:0/851240719 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9134084e50 con 0x5611b43a8b60 2024-09-16T00:34:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f9121ffb640 1 -- 172.21.15.1:0/851240719 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f91340a4370 con 0x5611b43a8b60 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f91426bb740 10 client.16836 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f91426bb740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f91426bb740 20 client.16836 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f91426bb740 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5611b43c3aa0 0x5611b4431560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f91426bb740 4 client.16836 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f91426bb740 1 -- 172.21.15.1:0/851240719 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5611b4315d00 con 0x5611b43c3aa0 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.154+0000 7f913d87d640 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5611b43c3aa0 0x5611b4431560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.155+0000 7f9121ffb640 10 client.16836 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.155+0000 7f913d87d640 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5611b43c3aa0 0x5611b4431560 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f148db3c08bea7 server_cookie=57188931bb4d9e64 in_seq=0 out_seq=0 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f9121ffb640 1 -- 172.21.15.1:0/851240719 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5611b4315d00 con 0x5611b43c3aa0 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f9121ffb640 10 client.16836 handle_command_reply: tid=0 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 client.16836 shutdown 2024-09-16T00:34:22.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 -- 172.21.15.1:0/851240719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 msgr2=0x5611b43b85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b85e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f913400bfa0 tx=0x7f91340a4770 comp rx=0 tx=0).stop 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 -- 172.21.15.1:0/851240719 shutdown_connections 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5611b43c3aa0 0x5611b4431560 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b43b3d40 0x5611b44267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611b43a9480 0x5611b43b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 --2- 172.21.15.1:0/851240719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611b43a8b60 0x5611b43b85e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 -- 172.21.15.1:0/851240719 >> 172.21.15.1:0/851240719 conn(0x5611b43a50f0 msgr2=0x5611b43b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f9121ffb640 0 client.16836 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 -- 172.21.15.1:0/851240719 shutdown_connections 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 1 -- 172.21.15.1:0/851240719 wait complete. 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.156+0000 7f91426bb740 20 client.16836 trim_cache size 0 max 16384 2024-09-16T00:34:22.161 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:34:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:34:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "f72cd9f6-f323-489d-b1c6-608836c90f33", 2024-09-16T00:34:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:34:22.162 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:34:22.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 -- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f91240a32c0 msgr2=0x7f91240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f91240a32c0 0x7f91240a5780 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9130009430 tx=0x7f9130032f60 comp rx=0 tx=0).stop 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 -- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138100a60 msgr2=0x7f91381983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138100a60 0x7f91381983e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f913400d950 tx=0x7f913400de10 comp rx=0 tx=0).stop 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 -- 172.21.15.1:0/2105385323 shutdown_connections 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9138106a60 0x7f913819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f91240a32c0 0x7f91240a5780 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138101400 0x7f9138198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 --2- 172.21.15.1:0/2105385323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9138100a60 0x7f91381983e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 -- 172.21.15.1:0/2105385323 >> 172.21.15.1:0/2105385323 conn(0x7f91380fc780 msgr2=0x7f913806a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 -- 172.21.15.1:0/2105385323 shutdown_connections 2024-09-16T00:34:22.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:22.157+0000 7f91402e2640 1 -- 172.21.15.1:0/2105385323 wait complete. 2024-09-16T00:34:22.872 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:34:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:23.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.623+0000 7f2007ef7640 1 -- 172.21.15.1:0/1666374858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 msgr2=0x7f2000100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.623+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1666374858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 0x7f2000100ee0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009930 tx=0x7f1ff002a420 comp rx=0 tx=0).stop 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.623+0000 7f2007ef7640 1 -- 172.21.15.1:0/1666374858 shutdown_connections 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.623+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1666374858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000101420 0x7f200010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.623+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1666374858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 0x7f2000100ee0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.623+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1666374858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2000106a80 0x7f2000106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.623+0000 7f2007ef7640 1 -- 172.21.15.1:0/1666374858 >> 172.21.15.1:0/1666374858 conn(0x7f20000fc7a0 msgr2=0x7f20000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.623+0000 7f2007ef7640 1 -- 172.21.15.1:0/1666374858 shutdown_connections 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.623+0000 7f2007ef7640 1 -- 172.21.15.1:0/1666374858 wait complete. 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2007ef7640 1 Processor -- start 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2007ef7640 1 -- start start 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2007ef7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 0x7f20000fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2007ef7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000101420 0x7f20000fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2007ef7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2000106a80 0x7f20000fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2007ef7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2000110e80 con 0x7f2000100a80 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2007ef7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2000110d00 con 0x7f2000106a80 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2007ef7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2000111000 con 0x7f2000101420 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2005c93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 0x7f20000fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2005c93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 0x7f20000fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45884/0 (socket says 172.21.15.1:45884) 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2005c93640 1 -- 172.21.15.1:0/1977273069 learned_addr learned my addr 172.21.15.1:0/1977273069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2005492640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000101420 0x7f20000fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.624+0000 7f2006494640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2000106a80 0x7f20000fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f2005c93640 1 -- 172.21.15.1:0/1977273069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000101420 msgr2=0x7f20000fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f2005c93640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000101420 0x7f20000fd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f2005c93640 1 -- 172.21.15.1:0/1977273069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2000106a80 msgr2=0x7f20000fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:23.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f2005c93640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2000106a80 0x7f20000fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f2005c93640 1 -- 172.21.15.1:0/1977273069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20001a6960 con 0x7f2000100a80 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f2005c93640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 0x7f20000fcd90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00e9f0 tx=0x7f1ffc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f1ff6ffd640 1 -- 172.21.15.1:0/1977273069 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00cd80 con 0x7f2000100a80 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f1ff6ffd640 1 -- 172.21.15.1:0/1977273069 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ffc010910 con 0x7f2000100a80 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f2007ef7640 1 -- 172.21.15.1:0/1977273069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20001a6c50 con 0x7f2000100a80 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f1ff6ffd640 1 -- 172.21.15.1:0/1977273069 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc018990 con 0x7f2000100a80 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.625+0000 7f2007ef7640 1 -- 172.21.15.1:0/1977273069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20001a71b0 con 0x7f2000100a80 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.627+0000 7f1ff6ffd640 1 -- 172.21.15.1:0/1977273069 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f1ffc010ac0 con 0x7f2000100a80 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f200a2d0740 1 Processor -- start 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f200a2d0740 1 -- start start 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c867d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c867ddc0 0x5568c8688140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c8688680 0x5568c868aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f200a2d0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5568c85e32e0 con 0x5568c8688680 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f200a2d0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568c858ace0 con 0x5568c867ddc0 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f200a2d0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5568c85df2a0 con 0x5568c867d4a0 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f2006494640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c867d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f2006494640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c867d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:59100/0 (socket says 172.21.15.1:59100) 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f2006494640 1 -- 172.21.15.1:0/2792428895 learned_addr learned my addr 172.21.15.1:0/2792428895 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.628+0000 7f2005c93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c867ddc0 0x5568c8688140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f2005492640 1 --2- 172.21.15.1:0/2792428895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c8688680 0x5568c868aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f1ff6ffd640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1fec0a32c0 0x7f1fec0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f2006494640 1 -- 172.21.15.1:0/2792428895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c867ddc0 msgr2=0x5568c8688140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f2006494640 1 --2- 172.21.15.1:0/2792428895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c867ddc0 0x5568c8688140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f2006494640 1 -- 172.21.15.1:0/2792428895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c8688680 msgr2=0x5568c868aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f2006494640 1 --2- 172.21.15.1:0/2792428895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c8688680 0x5568c868aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f2006494640 1 -- 172.21.15.1:0/2792428895 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c856f440 con 0x5568c867d4a0 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f1ff6ffd640 1 -- 172.21.15.1:0/1977273069 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1ffc0cff40 con 0x7f2000100a80 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f2006494640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1fec0a32c0 0x7f1fec0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f2006494640 1 --2- 172.21.15.1:0/2792428895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c867d880 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8011590 tx=0x7f1ff8011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eb801c80a1b8537 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f1febfff640 1 -- 172.21.15.1:0/2792428895 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8017050 con 0x5568c867d4a0 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f1febfff640 1 -- 172.21.15.1:0/2792428895 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff800fd20 con 0x5568c867d4a0 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.629+0000 7f1febfff640 1 -- 172.21.15.1:0/2792428895 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff801bab0 con 0x5568c867d4a0 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 -- 172.21.15.1:0/2792428895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 msgr2=0x5568c867d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f2006494640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1fec0a32c0 0x7f1fec0a5780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8022000 tx=0x7f1ff8023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 --2- 172.21.15.1:0/2792428895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c867d880 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8011590 tx=0x7f1ff8011a50 comp rx=0 tx=0).stop 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 -- 172.21.15.1:0/2792428895 shutdown_connections 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 --2- 172.21.15.1:0/2792428895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c8688680 0x5568c868aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 --2- 172.21.15.1:0/2792428895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c867ddc0 0x5568c8688140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 --2- 172.21.15.1:0/2792428895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c867d880 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 -- 172.21.15.1:0/2792428895 >> 172.21.15.1:0/2792428895 conn(0x5568c8679a30 msgr2=0x5568c8679e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 -- 172.21.15.1:0/2792428895 shutdown_connections 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 -- 172.21.15.1:0/2792428895 wait complete. 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.630+0000 7f200a2d0740 1 -- start start 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f200a2d0740 1 Processor -- start 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867d4a0 0x5568c8687f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867ddc0 0x5568c8685210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8688680 0x5568c8685750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f200a2d0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5568c868b250 con 0x5568c867d4a0 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f200a2d0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568c859e050 con 0x5568c8688680 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f200a2d0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5568c868b3d0 con 0x5568c867ddc0 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f200a2d0740 10 client.0 _ll_register_callbacks cb 0x5568c85de160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:23.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2005c93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867d4a0 0x5568c8687f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2005492640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867ddc0 0x5568c8685210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2006494640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8688680 0x5568c8685750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2005c93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867d4a0 0x5568c8687f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45896/0 (socket says 172.21.15.1:45896) 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2005c93640 1 -- 172.21.15.1:0/3535573374 learned_addr learned my addr 172.21.15.1:0/3535573374 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2005c93640 1 -- 172.21.15.1:0/3535573374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867ddc0 msgr2=0x5568c8685210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2005c93640 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867ddc0 0x5568c8685210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2005c93640 1 -- 172.21.15.1:0/3535573374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8688680 msgr2=0x5568c8685750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2005c93640 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8688680 0x5568c8685750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.631+0000 7f2005c93640 1 -- 172.21.15.1:0/3535573374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c83543c0 con 0x5568c867d4a0 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f2005492640 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867ddc0 0x5568c8685210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f2005c93640 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867d4a0 0x5568c8687f60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c6a0 tx=0x7f1ffc09bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426d4c4fa1a60837 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f1fe97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f1fe97fa640 1 -- 172.21.15.1:0/3535573374 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0a04b0 con 0x5568c867d4a0 2024-09-16T00:34:23.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f1fe97fa640 1 -- 172.21.15.1:0/3535573374 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1ffc0d3020 con 0x5568c867d4a0 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f200a2d0740 1 -- 172.21.15.1:0/3535573374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff800fd20 con 0x5568c867d4a0 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f1fe97fa640 1 -- 172.21.15.1:0/3535573374 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ffc0a4d70 con 0x5568c867d4a0 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f200a2d0740 10 client.16848 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f200a2d0740 1 -- 172.21.15.1:0/3535573374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5568c8696c40 con 0x5568c867d4a0 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f1fe97fa640 1 -- 172.21.15.1:0/3535573374 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff800fd20 con 0x5568c867d4a0 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f1fe97fa640 1 -- 172.21.15.1:0/3535573374 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ffc018bc0 con 0x5568c867d4a0 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f200a2d0740 10 client.16848 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f200a2d0740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f200a2d0740 20 client.16848 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f200a2d0740 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5568c8698050 0x5568c869a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f200a2d0740 4 client.16848 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.632+0000 7f200a2d0740 1 -- 172.21.15.1:0/3535573374 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568c85f0850 con 0x5568c8698050 2024-09-16T00:34:23.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.633+0000 7f2005492640 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5568c8698050 0x5568c869a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.633+0000 7f2005492640 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5568c8698050 0x5568c869a460 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee74fce4cb93f9 server_cookie=25f69eab2c7c08ab in_seq=0 out_seq=0 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.633+0000 7f1fe97fa640 10 client.16848 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f1fe97fa640 1 -- 172.21.15.1:0/3535573374 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568c85f0850 con 0x5568c8698050 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f1fe97fa640 10 client.16848 handle_command_reply: tid=0 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 client.16848 shutdown 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 -- 172.21.15.1:0/3535573374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867d4a0 msgr2=0x5568c8687f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867d4a0 0x5568c8687f60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c6a0 tx=0x7f1ffc09bcf0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 -- 172.21.15.1:0/3535573374 shutdown_connections 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5568c8698050 0x5568c869a460 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8688680 0x5568c8685750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867ddc0 0x5568c8685210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f1fe97fa640 0 client.16848 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 --2- 172.21.15.1:0/3535573374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867d4a0 0x5568c8687f60 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 -- 172.21.15.1:0/3535573374 >> 172.21.15.1:0/3535573374 conn(0x5568c8679a30 msgr2=0x5568c867f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:23.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 -- 172.21.15.1:0/3535573374 shutdown_connections 2024-09-16T00:34:23.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 1 -- 172.21.15.1:0/3535573374 wait complete. 2024-09-16T00:34:23.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.634+0000 7f200a2d0740 20 client.16848 trim_cache size 0 max 16384 2024-09-16T00:34:23.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f200a2d0740 1 Processor -- start 2024-09-16T00:34:23.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f200a2d0740 1 -- start start 2024-09-16T00:34:23.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c86944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c869fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c86a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f200a2d0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20001b38d0 con 0x5568c867ddc0 2024-09-16T00:34:23.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f200a2d0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20001b11a0 con 0x5568c8679a30 2024-09-16T00:34:23.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f200a2d0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20001aeff0 con 0x5568c867d4a0 2024-09-16T00:34:23.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f2006494640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c86a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f2006494640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c86a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45912/0 (socket says 172.21.15.1:45912) 2024-09-16T00:34:23.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.669+0000 7f2006494640 1 -- 172.21.15.1:0/153267800 learned_addr learned my addr 172.21.15.1:0/153267800 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:23.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f2005492640 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c869fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f2005c93640 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c86944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f2006494640 1 -- 172.21.15.1:0/153267800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 msgr2=0x5568c869fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f2006494640 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c869fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f2006494640 1 -- 172.21.15.1:0/153267800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 msgr2=0x5568c86944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f2006494640 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c86944e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f2006494640 1 -- 172.21.15.1:0/153267800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c86882d0 con 0x5568c867ddc0 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f2005c93640 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c86944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f2006494640 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c86a0270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8005fd0 tx=0x7f1ff801b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b06f8a2c1405a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f1feb7fe640 1 -- 172.21.15.1:0/153267800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff801b4a0 con 0x5568c867ddc0 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f1feb7fe640 1 -- 172.21.15.1:0/153267800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff8036070 con 0x5568c867ddc0 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.670+0000 7f1feb7fe640 1 -- 172.21.15.1:0/153267800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800f640 con 0x5568c867ddc0 2024-09-16T00:34:23.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 -- 172.21.15.1:0/153267800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 msgr2=0x5568c86a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c86a0270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8005fd0 tx=0x7f1ff801b650 comp rx=0 tx=0).stop 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 -- 172.21.15.1:0/153267800 shutdown_connections 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c86a0270 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c869fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 --2- 172.21.15.1:0/153267800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c86944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 -- 172.21.15.1:0/153267800 >> 172.21.15.1:0/153267800 conn(0x5568c86858b0 msgr2=0x5568c8683ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 -- 172.21.15.1:0/153267800 shutdown_connections 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 -- 172.21.15.1:0/153267800 wait complete. 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.671+0000 7f200a2d0740 1 -- start start 2024-09-16T00:34:23.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f200a2d0740 1 Processor -- start 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c8690d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c868c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f200a2d0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c868c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f200a2d0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20001af660 con 0x5568c867ddc0 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f200a2d0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20001b0ad0 con 0x5568c8679a30 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f200a2d0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20001afd30 con 0x5568c867d4a0 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f200a2d0740 10 client.0 _ll_register_callbacks cb 0x7f1ffc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f2006494640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c868c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f2005492640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c868c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f2006494640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c868c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45918/0 (socket says 172.21.15.1:45918) 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f2005492640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c868c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:59148/0 (socket says 172.21.15.1:59148) 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f2006494640 1 -- 172.21.15.1:0/4058557843 learned_addr learned my addr 172.21.15.1:0/4058557843 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.672+0000 7f2005c93640 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c8690d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.673+0000 7f2006494640 1 -- 172.21.15.1:0/4058557843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 msgr2=0x5568c868c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.673+0000 7f2006494640 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c868c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.673+0000 7f2006494640 1 -- 172.21.15.1:0/4058557843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 msgr2=0x5568c8690d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.673+0000 7f2006494640 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c8690d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.673+0000 7f2006494640 1 -- 172.21.15.1:0/4058557843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c868cd90 con 0x5568c867ddc0 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.673+0000 7f2005c93640 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c8690d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.674+0000 7f2006494640 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c868c580 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8000f80 tx=0x7f1ff8009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f76e0778d3abd07 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.674+0000 7f1fe97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.674+0000 7f1fe97fa640 1 -- 172.21.15.1:0/4058557843 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff803e020 con 0x5568c867ddc0 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.674+0000 7f200a2d0740 1 -- 172.21.15.1:0/4058557843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff8036070 con 0x5568c867ddc0 2024-09-16T00:34:23.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.674+0000 7f1fe97fa640 1 -- 172.21.15.1:0/4058557843 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1ff800f840 con 0x5568c867ddc0 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.674+0000 7f1fe97fa640 1 -- 172.21.15.1:0/4058557843 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ff8049050 con 0x5568c867ddc0 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.674+0000 7f200a2d0740 10 client.16854 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.674+0000 7f200a2d0740 1 -- 172.21.15.1:0/4058557843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5568c8702bf0 con 0x5568c867ddc0 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.675+0000 7f1fe97fa640 1 -- 172.21.15.1:0/4058557843 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff8036070 con 0x5568c867ddc0 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.675+0000 7f1fe97fa640 1 -- 172.21.15.1:0/4058557843 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ff801a070 con 0x5568c867ddc0 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.675+0000 7f200a2d0740 10 client.16854 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.675+0000 7f200a2d0740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.675+0000 7f200a2d0740 20 client.16854 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.675+0000 7f200a2d0740 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5568c8688680 0x5568c87051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.675+0000 7f200a2d0740 4 client.16854 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.675+0000 7f200a2d0740 1 -- 172.21.15.1:0/4058557843 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568c85ead60 con 0x5568c8688680 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.675+0000 7f2005c93640 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5568c8688680 0x5568c87051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f1fe97fa640 10 client.16854 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:23.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f2005c93640 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5568c8688680 0x5568c87051b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5320625b99b29cd0 server_cookie=2a4b80a65ca91250 in_seq=0 out_seq=0 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f1fe97fa640 1 -- 172.21.15.1:0/4058557843 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5568c85ead60 con 0x5568c8688680 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f1fe97fa640 10 client.16854 handle_command_reply: tid=0 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 client.16854 shutdown 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 -- 172.21.15.1:0/4058557843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 msgr2=0x5568c868c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c868c580 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8000f80 tx=0x7f1ff8009f50 comp rx=0 tx=0).stop 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 -- 172.21.15.1:0/4058557843 shutdown_connections 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5568c8688680 0x5568c87051b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568c867ddc0 0x5568c868c580 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568c867d4a0 0x5568c868c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 --2- 172.21.15.1:0/4058557843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c8679a30 0x5568c8690d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 -- 172.21.15.1:0/4058557843 >> 172.21.15.1:0/4058557843 conn(0x5568c86858b0 msgr2=0x5568c8678350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f1fe97fa640 0 client.16854 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:23.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 -- 172.21.15.1:0/4058557843 shutdown_connections 2024-09-16T00:34:23.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.676+0000 7f200a2d0740 1 -- 172.21.15.1:0/4058557843 wait complete. 2024-09-16T00:34:23.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f200a2d0740 20 client.16854 trim_cache size 0 max 16384 2024-09-16T00:34:23.681 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:34:23.681 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:34:23.681 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:34:23.681 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:34:23.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 -- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1fec0a32c0 msgr2=0x7f1fec0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:23.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1fec0a32c0 0x7f1fec0a5780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8022000 tx=0x7f1ff8023000 comp rx=0 tx=0).stop 2024-09-16T00:34:23.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 -- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 msgr2=0x7f20000fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:23.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 0x7f20000fcd90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00e9f0 tx=0x7f1ffc00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 -- 172.21.15.1:0/1977273069 shutdown_connections 2024-09-16T00:34:23.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2000106a80 0x7f20000fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1fec0a32c0 0x7f1fec0a5780 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000101420 0x7f20000fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 --2- 172.21.15.1:0/1977273069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000100a80 0x7f20000fcd90 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:23.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.677+0000 7f2007ef7640 1 -- 172.21.15.1:0/1977273069 >> 172.21.15.1:0/1977273069 conn(0x7f20000fc7a0 msgr2=0x7f200006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:23.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.678+0000 7f2007ef7640 1 -- 172.21.15.1:0/1977273069 shutdown_connections 2024-09-16T00:34:23.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:23.678+0000 7f2007ef7640 1 -- 172.21.15.1:0/1977273069 wait complete. 2024-09-16T00:34:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:23 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:34:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:23 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:23 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:34:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:23 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:23 smithi019 ceph-mon[35186]: pgmap v530: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-16T00:34:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:23 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:34:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:23 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:34:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:23 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:23.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:23 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:34:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:23 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:23 smithi001 ceph-mon[30934]: pgmap v530: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-16T00:34:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:23 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:34:23.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:23 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:34:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:23 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:23 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:34:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:23 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:23 smithi153 ceph-mon[35538]: pgmap v530: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-16T00:34:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:23 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:34:23.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:24.601 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:34:24.601 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:34:24.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:24.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:24.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:34:24.862 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:24.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:24.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:24.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:34:24.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:24.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:24.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:34:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.369+0000 7f799880d640 1 -- 172.21.15.1:0/3595841833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7994101440 msgr2=0x7f799410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.369+0000 7f799880d640 1 --2- 172.21.15.1:0/3595841833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7994101440 0x7f799410df30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7988009be0 tx=0x7f798802a6d0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.370+0000 7f799880d640 1 -- 172.21.15.1:0/3595841833 shutdown_connections 2024-09-16T00:34:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.370+0000 7f799880d640 1 --2- 172.21.15.1:0/3595841833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f799410e500 0x7f7994110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.370+0000 7f799880d640 1 --2- 172.21.15.1:0/3595841833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7994101440 0x7f799410df30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.370+0000 7f799880d640 1 --2- 172.21.15.1:0/3595841833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7994100b20 0x7f7994100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.370+0000 7f799880d640 1 -- 172.21.15.1:0/3595841833 >> 172.21.15.1:0/3595841833 conn(0x7f79940fc7d0 msgr2=0x7f79940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.370+0000 7f799880d640 1 -- 172.21.15.1:0/3595841833 shutdown_connections 2024-09-16T00:34:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.370+0000 7f799880d640 1 -- 172.21.15.1:0/3595841833 wait complete. 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.370+0000 7f799880d640 1 Processor -- start 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799880d640 1 -- start start 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799880d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7994100b20 0x7f79941a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799880d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7994101440 0x7f79941a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799880d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f799410e500 0x7f799419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799880d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7994114280 con 0x7f7994100b20 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799880d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7994114100 con 0x7f7994101440 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799880d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7994114400 con 0x7f799410e500 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7994100b20 0x7f79941a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f7991d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7994101440 0x7f79941a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7994100b20 0x7f79941a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45938/0 (socket says 172.21.15.1:45938) 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799259c640 1 -- 172.21.15.1:0/3186841115 learned_addr learned my addr 172.21.15.1:0/3186841115 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f7992d9d640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f799410e500 0x7f799419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799259c640 1 -- 172.21.15.1:0/3186841115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f799410e500 msgr2=0x7f799419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799259c640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f799410e500 0x7f799419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799259c640 1 -- 172.21.15.1:0/3186841115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7994101440 msgr2=0x7f79941a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799259c640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7994101440 0x7f79941a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f799259c640 1 -- 172.21.15.1:0/3186841115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799419aea0 con 0x7f7994100b20 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.371+0000 7f7991d9b640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7994101440 0x7f79941a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.372+0000 7f799259c640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7994100b20 0x7f79941a0560 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f798400b4a0 tx=0x7f798400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.372+0000 7f79837fe640 1 -- 172.21.15.1:0/3186841115 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984014070 con 0x7f7994100b20 2024-09-16T00:34:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.372+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799419b190 con 0x7f7994100b20 2024-09-16T00:34:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.372+0000 7f79837fe640 1 -- 172.21.15.1:0/3186841115 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7984010af0 con 0x7f7994100b20 2024-09-16T00:34:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.372+0000 7f79837fe640 1 -- 172.21.15.1:0/3186841115 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984017d10 con 0x7f7994100b20 2024-09-16T00:34:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.372+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79941a74e0 con 0x7f7994100b20 2024-09-16T00:34:25.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.373+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799406a850 con 0x7f7994100b20 2024-09-16T00:34:25.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.374+0000 7f79837fe640 1 -- 172.21.15.1:0/3186841115 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f7984017070 con 0x7f7994100b20 2024-09-16T00:34:25.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.375+0000 7f79837fe640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f797c0a32c0 0x7f797c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:25.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.375+0000 7f7991d9b640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f797c0a32c0 0x7f797c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:25.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.375+0000 7f79837fe640 1 -- 172.21.15.1:0/3186841115 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f79840cf000 con 0x7f7994100b20 2024-09-16T00:34:25.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.375+0000 7f7991d9b640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f797c0a32c0 0x7f797c0a5780 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7988009370 tx=0x7f7988000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:25.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.379+0000 7f79837fe640 1 -- 172.21.15.1:0/3186841115 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f798409b920 con 0x7f7994100b20 2024-09-16T00:34:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.589+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f799419bb60 con 0x7f7994100b20 2024-09-16T00:34:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.590+0000 7f79837fe640 1 -- 172.21.15.1:0/3186841115 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f798409b220 con 0x7f7994100b20 2024-09-16T00:34:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:34:25.590 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:34:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:34:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f797c0a32c0 msgr2=0x7f797c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f797c0a32c0 0x7f797c0a5780 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7988009370 tx=0x7f7988000f40 comp rx=0 tx=0).stop 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7994100b20 msgr2=0x7f79941a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7994100b20 0x7f79941a0560 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f798400b4a0 tx=0x7f798400b960 comp rx=0 tx=0).stop 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 shutdown_connections 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f797c0a32c0 0x7f797c0a5780 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f799410e500 0x7f799419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7994101440 0x7f79941a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 --2- 172.21.15.1:0/3186841115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7994100b20 0x7f79941a0560 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 >> 172.21.15.1:0/3186841115 conn(0x7f79940fc7d0 msgr2=0x7f79940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 shutdown_connections 2024-09-16T00:34:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:25.592+0000 7f799880d640 1 -- 172.21.15.1:0/3186841115 wait complete. 2024-09-16T00:34:25.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:25 smithi001 ceph-mon[30934]: pgmap v531: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-16T00:34:25.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:25 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3186841115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:26.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:25 smithi019 ceph-mon[35186]: pgmap v531: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-16T00:34:26.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:25 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3186841115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:25 smithi153 ceph-mon[35538]: pgmap v531: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-16T00:34:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:25 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3186841115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:26.406 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:34:26.674 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.127+0000 7f529868d640 1 -- 172.21.15.1:0/851562601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 msgr2=0x7f529010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.127+0000 7f529868d640 1 --2- 172.21.15.1:0/851562601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 0x7f529010ea90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5288009ca0 tx=0x7f528802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.127+0000 7f529868d640 1 -- 172.21.15.1:0/851562601 shutdown_connections 2024-09-16T00:34:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.127+0000 7f529868d640 1 --2- 172.21.15.1:0/851562601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 0x7f529010ea90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.127+0000 7f529868d640 1 --2- 172.21.15.1:0/851562601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5290102790 0x7f52901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.127+0000 7f529868d640 1 --2- 172.21.15.1:0/851562601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5290101de0 0x7f52901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.127+0000 7f529868d640 1 -- 172.21.15.1:0/851562601 >> 172.21.15.1:0/851562601 conn(0x7f52900fd8c0 msgr2=0x7f52900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.127+0000 7f529868d640 1 -- 172.21.15.1:0/851562601 shutdown_connections 2024-09-16T00:34:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.127+0000 7f529868d640 1 -- 172.21.15.1:0/851562601 wait complete. 2024-09-16T00:34:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f529868d640 1 Processor -- start 2024-09-16T00:34:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f529868d640 1 -- start start 2024-09-16T00:34:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f529868d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5290101de0 0x7f52901983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f529868d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5290102790 0x7f5290198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f529868d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 0x7f529019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f529868d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5290111fc0 con 0x7f5290106f10 2024-09-16T00:34:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f529868d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5290111e40 con 0x7f5290102790 2024-09-16T00:34:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f529868d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5290112140 con 0x7f5290101de0 2024-09-16T00:34:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f5296c2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 0x7f529019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f5296c2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 0x7f529019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45952/0 (socket says 172.21.15.1:45952) 2024-09-16T00:34:27.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f5296c2a640 1 -- 172.21.15.1:0/1545799360 learned_addr learned my addr 172.21.15.1:0/1545799360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.128+0000 7f5295c28640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5290102790 0x7f5290198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f5296429640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5290101de0 0x7f52901983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f5296c2a640 1 -- 172.21.15.1:0/1545799360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5290101de0 msgr2=0x7f52901983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f5296c2a640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5290101de0 0x7f52901983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f5296c2a640 1 -- 172.21.15.1:0/1545799360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5290102790 msgr2=0x7f5290198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f5296c2a640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5290102790 0x7f5290198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f5296c2a640 1 -- 172.21.15.1:0/1545799360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529019d370 con 0x7f5290106f10 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f5296429640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5290101de0 0x7f52901983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f5295c28640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5290102790 0x7f5290198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f5296c2a640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 0x7f529019cc90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5288003f40 tx=0x7f5288003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f52877fe640 1 -- 172.21.15.1:0/1545799360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52880318c0 con 0x7f5290106f10 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f52877fe640 1 -- 172.21.15.1:0/1545799360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5288004520 con 0x7f5290106f10 2024-09-16T00:34:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f529868d640 1 -- 172.21.15.1:0/1545799360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529019d600 con 0x7f5290106f10 2024-09-16T00:34:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f52877fe640 1 -- 172.21.15.1:0/1545799360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528802fe10 con 0x7f5290106f10 2024-09-16T00:34:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.129+0000 7f529868d640 1 -- 172.21.15.1:0/1545799360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52901a4f40 con 0x7f5290106f10 2024-09-16T00:34:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f52877fe640 1 -- 172.21.15.1:0/1545799360 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f52880046d0 con 0x7f5290106f10 2024-09-16T00:34:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f529aa66740 1 Processor -- start 2024-09-16T00:34:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f529aa66740 1 -- start start 2024-09-16T00:34:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f529aa66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f529aa66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f529aa66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f529aa66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55688203b2f0 con 0x5568820d55d0 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f529aa66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556881fe2cf0 con 0x5568820e0370 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f529aa66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5568820372b0 con 0x5568820d6b40 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f5296429640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f5296429640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820d5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45964/0 (socket says 172.21.15.1:45964) 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.132+0000 7f5296429640 1 -- 172.21.15.1:0/2954763335 learned_addr learned my addr 172.21.15.1:0/2954763335 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5295c28640 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f52877fe640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f527c0a32c0 0x7f527c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5295c28640 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5296429640 1 -- 172.21.15.1:0/2954763335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 msgr2=0x5568820d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5296429640 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820d5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5296429640 1 -- 172.21.15.1:0/2954763335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 msgr2=0x5568820e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5296429640 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5296429640 1 -- 172.21.15.1:0/2954763335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556881fc73a0 con 0x5568820d55d0 2024-09-16T00:34:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5295c28640 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5296c2a640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f527c0a32c0 0x7f527c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f52877fe640 1 -- 172.21.15.1:0/1545799360 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f52880f1420 con 0x7f5290106f10 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5296429640 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820d5a10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f528c00bb70 tx=0x7f528c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79118496f5eb5d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5284ff9640 1 -- 172.21.15.1:0/2954763335 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c017620 con 0x5568820d55d0 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5284ff9640 1 -- 172.21.15.1:0/2954763335 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528c017ca0 con 0x5568820d55d0 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.133+0000 7f5284ff9640 1 -- 172.21.15.1:0/2954763335 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c00fc70 con 0x5568820d55d0 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 -- 172.21.15.1:0/2954763335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 msgr2=0x5568820d5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f5296c2a640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f527c0a32c0 0x7f527c0a5780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f52880b9a40 tx=0x7f5288087f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820d5a10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f528c00bb70 tx=0x7f528c00d710 comp rx=0 tx=0).stop 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 -- 172.21.15.1:0/2954763335 shutdown_connections 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820d5a10 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 --2- 172.21.15.1:0/2954763335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 -- 172.21.15.1:0/2954763335 >> 172.21.15.1:0/2954763335 conn(0x55688209e310 msgr2=0x5568820d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 -- 172.21.15.1:0/2954763335 shutdown_connections 2024-09-16T00:34:27.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 -- 172.21.15.1:0/2954763335 wait complete. 2024-09-16T00:34:27.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.134+0000 7f529aa66740 1 -- start start 2024-09-16T00:34:27.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f529aa66740 1 Processor -- start 2024-09-16T00:34:27.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f529aa66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820d55d0 0x5568820dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f529aa66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820dcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f529aa66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820e0370 0x5568820dd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f529aa66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5568820e3070 con 0x5568820e0370 2024-09-16T00:34:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f529aa66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556881ff6060 con 0x5568820d55d0 2024-09-16T00:34:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f529aa66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5568820e31f0 con 0x5568820d6b40 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f529aa66740 10 client.0 _ll_register_callbacks cb 0x556881d498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820e0370 0x5568820dd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820e0370 0x5568820dd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45966/0 (socket says 172.21.15.1:45966) 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 -- 172.21.15.1:0/374244957 learned_addr learned my addr 172.21.15.1:0/374244957 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820d55d0 0x5568820dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5296429640 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820dcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 -- 172.21.15.1:0/374244957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 msgr2=0x5568820dcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820dcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 -- 172.21.15.1:0/374244957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820d55d0 msgr2=0x5568820dfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820d55d0 0x5568820dfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 -- 172.21.15.1:0/374244957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556881dac3c0 con 0x5568820e0370 2024-09-16T00:34:27.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5296429640 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820dcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5295c28640 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820e0370 0x5568820dd570 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f528000efd0 tx=0x7f528000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448b6ed7aa079db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.135+0000 7f5279ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f5279ffb640 1 -- 172.21.15.1:0/374244957 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280004020 con 0x5568820e0370 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f5279ffb640 1 -- 172.21.15.1:0/374244957 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f52800041c0 con 0x5568820e0370 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f529aa66740 1 -- 172.21.15.1:0/374244957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f528c017ca0 con 0x5568820e0370 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f5279ffb640 1 -- 172.21.15.1:0/374244957 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5280025050 con 0x5568820e0370 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f529aa66740 10 client.16872 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f529aa66740 1 -- 172.21.15.1:0/374244957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5568820eeb20 con 0x5568820e0370 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f5279ffb640 1 -- 172.21.15.1:0/374244957 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528c017ca0 con 0x5568820e0370 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f5279ffb640 1 -- 172.21.15.1:0/374244957 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f528000f5d0 con 0x5568820e0370 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f529aa66740 10 client.16872 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f529aa66740 10 client.16872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f529aa66740 20 client.16872 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f529aa66740 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5568820effe0 0x5568820f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f529aa66740 4 client.16872 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.136+0000 7f529aa66740 1 -- 172.21.15.1:0/374244957 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556882048860 con 0x5568820effe0 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.137+0000 7f5296429640 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5568820effe0 0x5568820f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.137+0000 7f5279ffb640 10 client.16872 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.137+0000 7f5296429640 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5568820effe0 0x5568820f2430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da25e612f3b3aabf server_cookie=67201240389db886 in_seq=0 out_seq=0 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f5279ffb640 1 -- 172.21.15.1:0/374244957 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556882048860 con 0x5568820effe0 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f5279ffb640 10 client.16872 handle_command_reply: tid=0 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 client.16872 shutdown 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 -- 172.21.15.1:0/374244957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820e0370 msgr2=0x5568820dd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820e0370 0x5568820dd570 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f528000efd0 tx=0x7f528000e3c0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 -- 172.21.15.1:0/374244957 shutdown_connections 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820e0370 0x5568820dd570 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820dcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5568820effe0 0x5568820f2430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 --2- 172.21.15.1:0/374244957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820d55d0 0x5568820dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f5279ffb640 0 client.16872 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 -- 172.21.15.1:0/374244957 >> 172.21.15.1:0/374244957 conn(0x55688209e310 msgr2=0x5568820d7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 -- 172.21.15.1:0/374244957 shutdown_connections 2024-09-16T00:34:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 1 -- 172.21.15.1:0/374244957 wait complete. 2024-09-16T00:34:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.138+0000 7f529aa66740 20 client.16872 trim_cache size 0 max 16384 2024-09-16T00:34:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.151+0000 7f529aa66740 1 Processor -- start 2024-09-16T00:34:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.151+0000 7f529aa66740 1 -- start start 2024-09-16T00:34:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.151+0000 7f529aa66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d55d0 0x5568820ec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.151+0000 7f529aa66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d6b40 0x5568820d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.151+0000 7f529aa66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820d8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.151+0000 7f529aa66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f52901b1670 con 0x5568820d6b40 2024-09-16T00:34:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.151+0000 7f529aa66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52901aef40 con 0x5568820e0370 2024-09-16T00:34:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.151+0000 7f529aa66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f52901acd90 con 0x5568820d55d0 2024-09-16T00:34:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5296429640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d6b40 0x5568820d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5296429640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d6b40 0x5568820d8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45968/0 (socket says 172.21.15.1:45968) 2024-09-16T00:34:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5296429640 1 -- 172.21.15.1:0/3346307887 learned_addr learned my addr 172.21.15.1:0/3346307887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5295c28640 1 --2- 172.21.15.1:0/3346307887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820d8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5295c28640 1 --2- 172.21.15.1:0/3346307887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d55d0 0x5568820ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5296429640 1 -- 172.21.15.1:0/3346307887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d55d0 msgr2=0x5568820ec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5296429640 1 --2- 172.21.15.1:0/3346307887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d55d0 0x5568820ec310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5296429640 1 -- 172.21.15.1:0/3346307887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 msgr2=0x5568820d8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5296429640 1 --2- 172.21.15.1:0/3346307887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820d8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5296429640 1 -- 172.21.15.1:0/3346307887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568820d1ed0 con 0x5568820d6b40 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f5296429640 1 --2- 172.21.15.1:0/3346307887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d6b40 0x5568820d8120 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f528c00f820 tx=0x7f528c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe40f336bb3f49d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f527bfff640 1 -- 172.21.15.1:0/3346307887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c02e070 con 0x5568820d6b40 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f527bfff640 1 -- 172.21.15.1:0/3346307887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528c00f420 con 0x5568820d6b40 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.152+0000 7f527bfff640 1 -- 172.21.15.1:0/3346307887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c028920 con 0x5568820d6b40 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 -- 172.21.15.1:0/3346307887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d6b40 msgr2=0x5568820d8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 --2- 172.21.15.1:0/3346307887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d6b40 0x5568820d8120 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f528c00f820 tx=0x7f528c00f850 comp rx=0 tx=0).stop 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 -- 172.21.15.1:0/3346307887 shutdown_connections 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 --2- 172.21.15.1:0/3346307887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820d8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 --2- 172.21.15.1:0/3346307887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d6b40 0x5568820d8120 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 --2- 172.21.15.1:0/3346307887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d55d0 0x5568820ec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 -- 172.21.15.1:0/3346307887 >> 172.21.15.1:0/3346307887 conn(0x55688209e310 msgr2=0x5568820dcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 -- 172.21.15.1:0/3346307887 shutdown_connections 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 -- 172.21.15.1:0/3346307887 wait complete. 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 -- start start 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 Processor -- start 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820dab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820e8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820e89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f52901ad400 con 0x5568820d55d0 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52901ae870 con 0x5568820e0370 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.153+0000 7f529aa66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f52901adad0 con 0x5568820d6b40 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f529aa66740 10 client.0 _ll_register_callbacks cb 0x7f5280013c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:27.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820dab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820dab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45982/0 (socket says 172.21.15.1:45982) 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 -- 172.21.15.1:0/114143009 learned_addr learned my addr 172.21.15.1:0/114143009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5296429640 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820e89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 -- 172.21.15.1:0/114143009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 msgr2=0x5568820e8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820e8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 -- 172.21.15.1:0/114143009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 msgr2=0x5568820e89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820e89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 -- 172.21.15.1:0/114143009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568820f7900 con 0x5568820d55d0 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5295c28640 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820dab50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5280010e50 tx=0x7f528000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6efcd7f97a6be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5279ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5279ffb640 1 -- 172.21.15.1:0/114143009 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280013210 con 0x5568820d55d0 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5279ffb640 1 -- 172.21.15.1:0/114143009 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f528000fd80 con 0x5568820d55d0 2024-09-16T00:34:27.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f529aa66740 1 -- 172.21.15.1:0/114143009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f528c00f420 con 0x5568820d55d0 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.154+0000 7f5279ffb640 1 -- 172.21.15.1:0/114143009 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5280038050 con 0x5568820d55d0 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f529aa66740 10 client.16878 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f529aa66740 1 -- 172.21.15.1:0/114143009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55688215ae20 con 0x5568820d55d0 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f5279ffb640 1 -- 172.21.15.1:0/114143009 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528c00f420 con 0x5568820d55d0 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f5279ffb640 1 -- 172.21.15.1:0/114143009 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5280011c80 con 0x5568820d55d0 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f529aa66740 10 client.16878 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f529aa66740 10 client.16878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f529aa66740 20 client.16878 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f529aa66740 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55688215b480 0x55688215d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f529aa66740 4 client.16878 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:27.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f529aa66740 1 -- 172.21.15.1:0/114143009 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556882042d70 con 0x55688215b480 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.155+0000 7f5296429640 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55688215b480 0x55688215d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f5296429640 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55688215b480 0x55688215d8a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78d7e2e5e7850caf server_cookie=41e9b12aa3373ffe in_seq=0 out_seq=0 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f5279ffb640 10 client.16878 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f5279ffb640 1 -- 172.21.15.1:0/114143009 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556882042d70 con 0x55688215b480 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f5279ffb640 10 client.16878 handle_command_reply: tid=0 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 client.16878 shutdown 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 -- 172.21.15.1:0/114143009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 msgr2=0x5568820dab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820dab50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5280010e50 tx=0x7f528000e3c0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 -- 172.21.15.1:0/114143009 shutdown_connections 2024-09-16T00:34:27.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55688215b480 0x55688215d8a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568820e0370 0x5568820e89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f5279ffb640 0 client.16878 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:34:27.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5568820d6b40 0x5568820e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 --2- 172.21.15.1:0/114143009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568820d55d0 0x5568820dab50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 -- 172.21.15.1:0/114143009 >> 172.21.15.1:0/114143009 conn(0x55688209e310 msgr2=0x5568820de830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.182 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:34:27.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 -- 172.21.15.1:0/114143009 shutdown_connections 2024-09-16T00:34:27.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 1 -- 172.21.15.1:0/114143009 wait complete. 2024-09-16T00:34:27.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.156+0000 7f529aa66740 20 client.16878 trim_cache size 0 max 16384 2024-09-16T00:34:27.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 -- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f527c0a32c0 msgr2=0x7f527c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f527c0a32c0 0x7f527c0a5780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f52880b9a40 tx=0x7f5288087f70 comp rx=0 tx=0).stop 2024-09-16T00:34:27.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 -- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 msgr2=0x7f529019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 0x7f529019cc90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5288003f40 tx=0x7f5288003f70 comp rx=0 tx=0).stop 2024-09-16T00:34:27.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 -- 172.21.15.1:0/1545799360 shutdown_connections 2024-09-16T00:34:27.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5290106f10 0x7f529019cc90 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5290102790 0x7f5290198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f527c0a32c0 0x7f527c0a5780 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 --2- 172.21.15.1:0/1545799360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5290101de0 0x7f52901983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 -- 172.21.15.1:0/1545799360 >> 172.21.15.1:0/1545799360 conn(0x7f52900fd8c0 msgr2=0x7f52900febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 -- 172.21.15.1:0/1545799360 shutdown_connections 2024-09-16T00:34:27.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:27.157+0000 7f529868d640 1 -- 172.21.15.1:0/1545799360 wait complete. 2024-09-16T00:34:27.857 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:34:28.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:27 smithi019 ceph-mon[35186]: pgmap v532: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-09-16T00:34:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:28.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:27 smithi001 ceph-mon[30934]: pgmap v532: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-09-16T00:34:28.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:27 smithi153 ceph-mon[35538]: pgmap v532: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-09-16T00:34:28.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.689+0000 7fc78b633640 1 -- 172.21.15.1:0/3904402044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784101420 msgr2=0x7fc78410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.689+0000 7fc78b633640 1 --2- 172.21.15.1:0/3904402044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784101420 0x7fc78410d950 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc774009cd0 tx=0x7fc77402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.689+0000 7fc78b633640 1 -- 172.21.15.1:0/3904402044 shutdown_connections 2024-09-16T00:34:28.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.689+0000 7fc78b633640 1 --2- 172.21.15.1:0/3904402044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784101420 0x7fc78410d950 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.689+0000 7fc78b633640 1 --2- 172.21.15.1:0/3904402044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc784100a80 0x7fc784100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.689+0000 7fc78b633640 1 --2- 172.21.15.1:0/3904402044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc784106a80 0x7fc784106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.689+0000 7fc78b633640 1 -- 172.21.15.1:0/3904402044 >> 172.21.15.1:0/3904402044 conn(0x7fc7840fc7a0 msgr2=0x7fc7840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.689+0000 7fc78b633640 1 -- 172.21.15.1:0/3904402044 shutdown_connections 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.689+0000 7fc78b633640 1 -- 172.21.15.1:0/3904402044 wait complete. 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc78b633640 1 Processor -- start 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc78b633640 1 -- start start 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc78b633640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc784100a80 0x7fc784199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc78b633640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc784101420 0x7fc784199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc78b633640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784106a80 0x7fc7841a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc78b633640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc784110e80 con 0x7fc784106a80 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc78b633640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc784110d00 con 0x7fc784101420 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc78b633640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc784111000 con 0x7fc784100a80 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc789bd0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784106a80 0x7fc7841a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc789bd0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784106a80 0x7fc7841a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46000/0 (socket says 172.21.15.1:46000) 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc789bd0640 1 -- 172.21.15.1:0/2365424185 learned_addr learned my addr 172.21.15.1:0/2365424185 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc788bce640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc784101420 0x7fc784199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.690+0000 7fc7893cf640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc784100a80 0x7fc784199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc789bd0640 1 -- 172.21.15.1:0/2365424185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc784100a80 msgr2=0x7fc784199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc789bd0640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc784100a80 0x7fc784199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc789bd0640 1 -- 172.21.15.1:0/2365424185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc784101420 msgr2=0x7fc784199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc789bd0640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc784101420 0x7fc784199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc789bd0640 1 -- 172.21.15.1:0/2365424185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7841a47f0 con 0x7fc784106a80 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc7893cf640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc784100a80 0x7fc784199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc789bd0640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784106a80 0x7fc7841a42b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc774003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc7827fc640 1 -- 172.21.15.1:0/2365424185 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774004450 con 0x7fc784106a80 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc78b633640 1 -- 172.21.15.1:0/2365424185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7841a49c0 con 0x7fc784106a80 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc7827fc640 1 -- 172.21.15.1:0/2365424185 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc774003e10 con 0x7fc784106a80 2024-09-16T00:34:28.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc7827fc640 1 -- 172.21.15.1:0/2365424185 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774046480 con 0x7fc784106a80 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.691+0000 7fc78b633640 1 -- 172.21.15.1:0/2365424185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7841a4e70 con 0x7fc784106a80 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.693+0000 7fc7827fc640 1 -- 172.21.15.1:0/2365424185 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc7740045f0 con 0x7fc784106a80 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc78da0c740 1 Processor -- start 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc78da0c740 1 -- start start 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a2b40 0x56498c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 0x56498c1a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc78da0c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56498c1072f0 con 0x56498c1ac370 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc78da0c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56498c0aecf0 con 0x56498c1a15d0 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc78da0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56498c1032b0 con 0x56498c1a2b40 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc7827fc640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc76c0a3240 0x7fc76c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc788bce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc789bd0640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc76c0a3240 0x7fc76c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc788bce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46010/0 (socket says 172.21.15.1:46010) 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc788bce640 1 -- 172.21.15.1:0/311673335 learned_addr learned my addr 172.21.15.1:0/311673335 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc7827fc640 1 -- 172.21.15.1:0/2365424185 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc77402e030 con 0x7fc784106a80 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.694+0000 7fc7893cf640 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 0x56498c1a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc788bce640 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a2b40 0x56498c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc788bce640 1 -- 172.21.15.1:0/311673335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a2b40 msgr2=0x56498c1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc788bce640 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a2b40 0x56498c1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc788bce640 1 -- 172.21.15.1:0/311673335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 msgr2=0x56498c1a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc788bce640 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 0x56498c1a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc788bce640 1 -- 172.21.15.1:0/311673335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498c0933a0 con 0x56498c1ac370 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc7893cf640 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 0x56498c1a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc789bd0640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc76c0a3240 0x7fc76c0a5700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc7740c6020 tx=0x7fc774090760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc788bce640 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1ae760 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc778011560 tx=0x7fc77800a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e5681c6e9a189b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc76b7fe640 1 -- 172.21.15.1:0/311673335 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778005220 con 0x56498c1ac370 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc76b7fe640 1 -- 172.21.15.1:0/311673335 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc778004c80 con 0x56498c1ac370 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.695+0000 7fc76b7fe640 1 -- 172.21.15.1:0/311673335 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77800fc40 con 0x56498c1ac370 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 -- 172.21.15.1:0/311673335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 msgr2=0x56498c1ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1ae760 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc778011560 tx=0x7fc77800a810 comp rx=0 tx=0).stop 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 -- 172.21.15.1:0/311673335 shutdown_connections 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1ae760 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 0x56498c1a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 --2- 172.21.15.1:0/311673335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a2b40 0x56498c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 -- 172.21.15.1:0/311673335 >> 172.21.15.1:0/311673335 conn(0x56498c16a310 msgr2=0x56498c19dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 -- 172.21.15.1:0/311673335 shutdown_connections 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 -- 172.21.15.1:0/311673335 wait complete. 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 -- start start 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.696+0000 7fc78da0c740 1 Processor -- start 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 0x56498c1abd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a2b40 0x56498c1a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc78da0c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56498c1af070 con 0x56498c1ac370 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc78da0c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56498c0c2060 con 0x56498c1a2b40 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc78da0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56498c1af1f0 con 0x56498c1a15d0 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc78da0c740 10 client.0 _ll_register_callbacks cb 0x56498be158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc788bce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc788bce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1a95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46016/0 (socket says 172.21.15.1:46016) 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc788bce640 1 -- 172.21.15.1:0/4105618579 learned_addr learned my addr 172.21.15.1:0/4105618579 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc788bce640 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 0x56498c1abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc7893cf640 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a2b40 0x56498c1a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc788bce640 1 -- 172.21.15.1:0/4105618579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 msgr2=0x56498c1abd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc788bce640 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 0x56498c1abd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc788bce640 1 -- 172.21.15.1:0/4105618579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a2b40 msgr2=0x56498c1a8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc788bce640 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a2b40 0x56498c1a8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc788bce640 1 -- 172.21.15.1:0/4105618579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498be783c0 con 0x56498c1ac370 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.697+0000 7fc7893cf640 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a2b40 0x56498c1a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc788bce640 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1a95d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc778011eb0 tx=0x7fc77800f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f938359dcdbca87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc768ff9640 1 -- 172.21.15.1:0/4105618579 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77802f060 con 0x56498c1ac370 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc78da0c740 1 -- 172.21.15.1:0/4105618579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc778004c80 con 0x56498c1ac370 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc768ff9640 1 -- 172.21.15.1:0/4105618579 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc7780046c0 con 0x56498c1ac370 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc768ff9640 1 -- 172.21.15.1:0/4105618579 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc778036050 con 0x56498c1ac370 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc78da0c740 10 client.16890 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc78da0c740 1 -- 172.21.15.1:0/4105618579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56498c1bef10 con 0x56498c1ac370 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc768ff9640 1 -- 172.21.15.1:0/4105618579 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc778004c80 con 0x56498c1ac370 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc768ff9640 1 -- 172.21.15.1:0/4105618579 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc778027070 con 0x56498c1ac370 2024-09-16T00:34:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc78da0c740 10 client.16890 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc78da0c740 10 client.16890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc78da0c740 20 client.16890 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc78da0c740 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56498c1c03d0 0x56498c1c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc78da0c740 4 client.16890 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.698+0000 7fc78da0c740 1 -- 172.21.15.1:0/4105618579 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56498c114860 con 0x56498c1c03d0 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.699+0000 7fc7893cf640 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56498c1c03d0 0x56498c1c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.699+0000 7fc7893cf640 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56498c1c03d0 0x56498c1c27c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8150ba11b09097b server_cookie=8c28cc4964588f0 in_seq=0 out_seq=0 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.699+0000 7fc768ff9640 10 client.16890 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc768ff9640 1 -- 172.21.15.1:0/4105618579 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56498c114860 con 0x56498c1c03d0 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc768ff9640 10 client.16890 handle_command_reply: tid=0 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 client.16890 shutdown 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 -- 172.21.15.1:0/4105618579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 msgr2=0x56498c1a95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1a95d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc778011eb0 tx=0x7fc77800f820 comp rx=0 tx=0).stop 2024-09-16T00:34:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 -- 172.21.15.1:0/4105618579 shutdown_connections 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56498c1c03d0 0x56498c1c27c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1ac370 0x56498c1a95d0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc768ff9640 0 client.16890 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a2b40 0x56498c1a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 --2- 172.21.15.1:0/4105618579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 0x56498c1abd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 -- 172.21.15.1:0/4105618579 >> 172.21.15.1:0/4105618579 conn(0x56498c16a310 msgr2=0x56498c1a3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 -- 172.21.15.1:0/4105618579 shutdown_connections 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 1 -- 172.21.15.1:0/4105618579 wait complete. 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.700+0000 7fc78da0c740 20 client.16890 trim_cache size 0 max 16384 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc78da0c740 1 Processor -- start 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc78da0c740 1 -- start start 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 0x56498c1bc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c19d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1ac370 0x56498c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc78da0c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841af3c0 con 0x56498c1a2b40 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc78da0c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841acc90 con 0x56498c1ac370 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc78da0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841aaae0 con 0x56498c1a15d0 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc7893cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc7893cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c19d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46020/0 (socket says 172.21.15.1:46020) 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.713+0000 7fc7893cf640 1 -- 172.21.15.1:0/451824706 learned_addr learned my addr 172.21.15.1:0/451824706 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc788bce640 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1ac370 0x56498c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc788bce640 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 0x56498c1bc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc7893cf640 1 -- 172.21.15.1:0/451824706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 msgr2=0x56498c1bc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc7893cf640 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 0x56498c1bc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc7893cf640 1 -- 172.21.15.1:0/451824706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1ac370 msgr2=0x56498c19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc7893cf640 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1ac370 0x56498c19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc7893cf640 1 -- 172.21.15.1:0/451824706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498c19e180 con 0x56498c1a2b40 2024-09-16T00:34:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc788bce640 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 0x56498c1bc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc7893cf640 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c19d700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x56498c1a8870 tx=0x7fc77c010b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de6a682cbe1dd40 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc76affd640 1 -- 172.21.15.1:0/451824706 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c010d10 con 0x56498c1a2b40 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc76affd640 1 -- 172.21.15.1:0/451824706 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77c00fe30 con 0x56498c1a2b40 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.714+0000 7fc76affd640 1 -- 172.21.15.1:0/451824706 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c01be40 con 0x56498c1a2b40 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 -- 172.21.15.1:0/451824706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 msgr2=0x56498c19d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c19d700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x56498c1a8870 tx=0x7fc77c010b20 comp rx=0 tx=0).stop 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 -- 172.21.15.1:0/451824706 shutdown_connections 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1ac370 0x56498c19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c19d700 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 --2- 172.21.15.1:0/451824706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1a15d0 0x56498c1bc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 -- 172.21.15.1:0/451824706 >> 172.21.15.1:0/451824706 conn(0x56498c16a310 msgr2=0x56498c1a8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 -- 172.21.15.1:0/451824706 shutdown_connections 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 -- 172.21.15.1:0/451824706 wait complete. 2024-09-16T00:34:28.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 -- start start 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.715+0000 7fc78da0c740 1 Processor -- start 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 0x56498c1b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c1b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc78da0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1ac370 0x56498c1a5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc78da0c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841ab150 con 0x56498c1a2b40 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc78da0c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841ac5c0 con 0x56498c1a15d0 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc78da0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841ab820 con 0x56498c1ac370 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc78da0c740 10 client.0 _ll_register_callbacks cb 0x7fc778011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc7893cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c1b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc7893cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c1b9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46032/0 (socket says 172.21.15.1:46032) 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc7893cf640 1 -- 172.21.15.1:0/41176354 learned_addr learned my addr 172.21.15.1:0/41176354 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc788bce640 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 0x56498c1b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc788bce640 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1ac370 0x56498c1a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc7893cf640 1 -- 172.21.15.1:0/41176354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1ac370 msgr2=0x56498c1a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc7893cf640 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1ac370 0x56498c1a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc7893cf640 1 -- 172.21.15.1:0/41176354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 msgr2=0x56498c1b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc7893cf640 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 0x56498c1b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc7893cf640 1 -- 172.21.15.1:0/41176354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498c1a5ed0 con 0x56498c1a2b40 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc7893cf640 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c1b9060 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc77c005cb0 tx=0x7fc77c00f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f0f5137d102631 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc768ff9640 1 -- 172.21.15.1:0/41176354 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c02c420 con 0x56498c1a2b40 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc768ff9640 1 -- 172.21.15.1:0/41176354 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc77c029040 con 0x56498c1a2b40 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.716+0000 7fc78da0c740 1 -- 172.21.15.1:0/41176354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77c00fe30 con 0x56498c1a2b40 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc768ff9640 1 -- 172.21.15.1:0/41176354 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc77c03a050 con 0x56498c1a2b40 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc78da0c740 10 client.16896 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc78da0c740 1 -- 172.21.15.1:0/41176354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56498c22a8c0 con 0x56498c1a2b40 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc768ff9640 1 -- 172.21.15.1:0/41176354 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77c00fe30 con 0x56498c1a2b40 2024-09-16T00:34:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc768ff9640 1 -- 172.21.15.1:0/41176354 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc77c0247c0 con 0x56498c1a2b40 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc78da0c740 10 client.16896 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc78da0c740 10 client.16896 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc78da0c740 20 client.16896 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc78da0c740 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56498c22afb0 0x56498c22d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc78da0c740 4 client.16896 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc78da0c740 1 -- 172.21.15.1:0/41176354 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56498c10ed70 con 0x56498c22afb0 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.717+0000 7fc788bce640 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56498c22afb0 0x56498c22d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc788bce640 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56498c22afb0 0x56498c22d3d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2549fb96232124e8 server_cookie=6aef2a874be715f4 in_seq=0 out_seq=0 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc768ff9640 10 client.16896 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc768ff9640 1 -- 172.21.15.1:0/41176354 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56498c10ed70 con 0x56498c22afb0 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc768ff9640 10 client.16896 handle_command_reply: tid=0 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 client.16896 shutdown 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 -- 172.21.15.1:0/41176354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 msgr2=0x56498c1b9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c1b9060 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc77c005cb0 tx=0x7fc77c00f950 comp rx=0 tx=0).stop 2024-09-16T00:34:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 -- 172.21.15.1:0/41176354 shutdown_connections 2024-09-16T00:34:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56498c1ac370 0x56498c1a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56498c22afb0 0x56498c22d3d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56498c1a2b40 0x56498c1b9060 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 --2- 172.21.15.1:0/41176354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498c1a15d0 0x56498c1b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc768ff9640 0 client.16896 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:34:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 -- 172.21.15.1:0/41176354 >> 172.21.15.1:0/41176354 conn(0x56498c16a310 msgr2=0x56498c1a74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 -- 172.21.15.1:0/41176354 shutdown_connections 2024-09-16T00:34:28.748 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 1 -- 172.21.15.1:0/41176354 wait complete. 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.718+0000 7fc78da0c740 20 client.16896 trim_cache size 0 max 16384 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 -- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc76c0a3240 msgr2=0x7fc76c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc76c0a3240 0x7fc76c0a5700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc7740c6020 tx=0x7fc774090760 comp rx=0 tx=0).stop 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 -- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784106a80 msgr2=0x7fc7841a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784106a80 0x7fc7841a42b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc774003c20 comp rx=0 tx=0).stop 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 -- 172.21.15.1:0/2365424185 shutdown_connections 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc784106a80 0x7fc7841a42b0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc784101420 0x7fc784199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc76c0a3240 0x7fc76c0a5700 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 --2- 172.21.15.1:0/2365424185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc784100a80 0x7fc784199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 -- 172.21.15.1:0/2365424185 >> 172.21.15.1:0/2365424185 conn(0x7fc7840fc7a0 msgr2=0x7fc7840fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 -- 172.21.15.1:0/2365424185 shutdown_connections 2024-09-16T00:34:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:28.719+0000 7fc78b633640 1 -- 172.21.15.1:0/2365424185 wait complete. 2024-09-16T00:34:29.631 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:34:29.899 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:29.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:29 smithi001 ceph-mon[30934]: pgmap v533: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-09-16T00:34:30.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:29 smithi019 ceph-mon[35186]: pgmap v533: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-09-16T00:34:30.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:29 smithi153 ceph-mon[35538]: pgmap v533: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-09-16T00:34:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.383+0000 7ff8290a3640 1 -- 172.21.15.1:0/2813058155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 msgr2=0x7ff824103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.383+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2813058155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 0x7ff824103120 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff814009900 tx=0x7ff81402a480 comp rx=0 tx=0).stop 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.383+0000 7ff8290a3640 1 -- 172.21.15.1:0/2813058155 shutdown_connections 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.383+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2813058155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff82410c170 0x7ff82410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.383+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2813058155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff824103660 0x7ff82410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.383+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2813058155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 0x7ff824103120 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.383+0000 7ff8290a3640 1 -- 172.21.15.1:0/2813058155 >> 172.21.15.1:0/2813058155 conn(0x7ff8240fc7a0 msgr2=0x7ff8240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.383+0000 7ff8290a3640 1 -- 172.21.15.1:0/2813058155 shutdown_connections 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.383+0000 7ff8290a3640 1 -- 172.21.15.1:0/2813058155 wait complete. 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.384+0000 7ff8290a3640 1 Processor -- start 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.384+0000 7ff8290a3640 1 -- start start 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff8290a3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 0x7ff8241009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff8290a3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff824103660 0x7ff824100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff8290a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff82410c170 0x7ff824101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff8290a3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff824111fb0 con 0x7ff824102d40 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff8290a3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff824111e30 con 0x7ff82410c170 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff8290a3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff824112130 con 0x7ff824103660 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 0x7ff8241009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 0x7ff8241009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49210/0 (socket says 172.21.15.1:49210) 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff822d9d640 1 -- 172.21.15.1:0/2554975255 learned_addr learned my addr 172.21.15.1:0/2554975255 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff82259c640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff824103660 0x7ff824100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff82359e640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff82410c170 0x7ff824101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff822d9d640 1 -- 172.21.15.1:0/2554975255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff824103660 msgr2=0x7ff824100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff822d9d640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff824103660 0x7ff824100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff822d9d640 1 -- 172.21.15.1:0/2554975255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff82410c170 msgr2=0x7ff824101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff822d9d640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff82410c170 0x7ff824101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff822d9d640 1 -- 172.21.15.1:0/2554975255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8241aae20 con 0x7ff824102d40 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff822d9d640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 0x7ff8241009a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff814005ab0 tx=0x7ff8140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.385+0000 7ff80bfff640 1 -- 172.21.15.1:0/2554975255 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81403f030 con 0x7ff824102d40 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.386+0000 7ff80bfff640 1 -- 172.21.15.1:0/2554975255 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff814003ce0 con 0x7ff824102d40 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.386+0000 7ff8290a3640 1 -- 172.21.15.1:0/2554975255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8241aaff0 con 0x7ff824102d40 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.386+0000 7ff80bfff640 1 -- 172.21.15.1:0/2554975255 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814035070 con 0x7ff824102d40 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.386+0000 7ff8290a3640 1 -- 172.21.15.1:0/2554975255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241ab450 con 0x7ff824102d40 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff80bfff640 1 -- 172.21.15.1:0/2554975255 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff814042030 con 0x7ff824102d40 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82b47c740 1 Processor -- start 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82b47c740 1 -- start start 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e32555d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 0x55e32555da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e32556a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82b47c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e3254c32f0 con 0x55e32555eb40 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82b47c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e32546acf0 con 0x55e325568370 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82b47c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e3254bf2b0 con 0x55e32555d5d0 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e32555d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e32555d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49222/0 (socket says 172.21.15.1:49222) 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82359e640 1 -- 172.21.15.1:0/2403852950 learned_addr learned my addr 172.21.15.1:0/2403852950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff822d9d640 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 0x55e32555da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82259c640 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e32556a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82359e640 1 -- 172.21.15.1:0/2403852950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 msgr2=0x55e32555da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82359e640 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 0x55e32555da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82359e640 1 -- 172.21.15.1:0/2403852950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 msgr2=0x55e32556a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82359e640 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e32556a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82359e640 1 -- 172.21.15.1:0/2403852950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32544f3a0 con 0x55e32555eb40 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff80bfff640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff80c0a3240 0x7ff80c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.389+0000 7ff82259c640 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e32556a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.390+0000 7ff822d9d640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff80c0a3240 0x7ff80c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.390+0000 7ff82359e640 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e32555d090 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00a430 tx=0x7ff81c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0d7eb5c3c1974a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.390+0000 7ff80bfff640 1 -- 172.21.15.1:0/2554975255 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff8140f04e0 con 0x7ff824102d40 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.390+0000 7ff8097fa640 1 -- 172.21.15.1:0/2403852950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c013790 con 0x55e32555eb40 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.390+0000 7ff8097fa640 1 -- 172.21.15.1:0/2403852950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c019070 con 0x55e32555eb40 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.390+0000 7ff8097fa640 1 -- 172.21.15.1:0/2403852950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c014e10 con 0x55e32555eb40 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 -- 172.21.15.1:0/2403852950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 msgr2=0x55e32555d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff822d9d640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff80c0a3240 0x7ff80c0a5700 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff8140edf50 tx=0x7ff814009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e32555d090 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00a430 tx=0x7ff81c009650 comp rx=0 tx=0).stop 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 -- 172.21.15.1:0/2403852950 shutdown_connections 2024-09-16T00:34:30.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e32556a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 0x55e32555da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 --2- 172.21.15.1:0/2403852950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e32555d090 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 -- 172.21.15.1:0/2403852950 >> 172.21.15.1:0/2403852950 conn(0x55e325526310 msgr2=0x55e325559c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 -- 172.21.15.1:0/2403852950 shutdown_connections 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 -- 172.21.15.1:0/2403852950 wait complete. 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 -- start start 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.391+0000 7ff82b47c740 1 Processor -- start 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 0x55e325567d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e325565040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e325565650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82b47c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e32556b070 con 0x55e32555eb40 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82b47c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e32547e060 con 0x55e325568370 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82b47c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e32556b1f0 con 0x55e32555d5d0 2024-09-16T00:34:30.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82b47c740 10 client.0 _ll_register_callbacks cb 0x55e3251d18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e325565040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e325565040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49232/0 (socket says 172.21.15.1:49232) 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82259c640 1 -- 172.21.15.1:0/3280099029 learned_addr learned my addr 172.21.15.1:0/3280099029 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82359e640 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 0x55e325567d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82259c640 1 -- 172.21.15.1:0/3280099029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 msgr2=0x55e325567d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82259c640 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 0x55e325567d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82259c640 1 -- 172.21.15.1:0/3280099029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 msgr2=0x55e325565650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82259c640 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e325565650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82259c640 1 -- 172.21.15.1:0/3280099029 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3252343c0 con 0x55e32555eb40 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82359e640 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 0x55e325567d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82259c640 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e325565040 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff818012700 tx=0x7ff818012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70de8ceb0272b8f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff806ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff806ff5640 1 -- 172.21.15.1:0/3280099029 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818010e80 con 0x55e32555eb40 2024-09-16T00:34:30.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff806ff5640 1 -- 172.21.15.1:0/3280099029 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff8180047a0 con 0x55e32555eb40 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.392+0000 7ff82b47c740 1 -- 172.21.15.1:0/3280099029 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81c019070 con 0x55e32555eb40 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff806ff5640 1 -- 172.21.15.1:0/3280099029 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff818025050 con 0x55e32555eb40 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff82b47c740 10 client.16908 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff82b47c740 1 -- 172.21.15.1:0/3280099029 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e325576ba0 con 0x55e32555eb40 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff806ff5640 1 -- 172.21.15.1:0/3280099029 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c019070 con 0x55e32555eb40 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff806ff5640 1 -- 172.21.15.1:0/3280099029 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff81800e680 con 0x55e32555eb40 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff82b47c740 10 client.16908 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff82b47c740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff82b47c740 20 client.16908 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff82b47c740 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e325578150 0x55e32557a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff82b47c740 4 client.16908 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff82b47c740 1 -- 172.21.15.1:0/3280099029 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3254d0860 con 0x55e325578150 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.393+0000 7ff82359e640 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e325578150 0x55e32557a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.394+0000 7ff82359e640 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e325578150 0x55e32557a540 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f12b5a10e5c93e6 server_cookie=6f1b01b7efb16166 in_seq=0 out_seq=0 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.394+0000 7ff806ff5640 10 client.16908 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.394+0000 7ff806ff5640 1 -- 172.21.15.1:0/3280099029 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3254d0860 con 0x55e325578150 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.394+0000 7ff806ff5640 10 client.16908 handle_command_reply: tid=0 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.394+0000 7ff82b47c740 1 client.16908 shutdown 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.394+0000 7ff82b47c740 1 -- 172.21.15.1:0/3280099029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 msgr2=0x55e325565040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.394+0000 7ff82b47c740 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e325565040 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff818012700 tx=0x7ff818012bc0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff82b47c740 1 -- 172.21.15.1:0/3280099029 shutdown_connections 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff82b47c740 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e325578150 0x55e32557a540 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff82b47c740 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e325565650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff82b47c740 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555eb40 0x55e325565040 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff806ff5640 0 client.16908 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:30.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff82b47c740 1 --2- 172.21.15.1:0/3280099029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555d5d0 0x55e325567d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff82b47c740 1 -- 172.21.15.1:0/3280099029 >> 172.21.15.1:0/3280099029 conn(0x55e325526310 msgr2=0x55e32555f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff82b47c740 1 -- 172.21.15.1:0/3280099029 shutdown_connections 2024-09-16T00:34:30.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff82b47c740 1 -- 172.21.15.1:0/3280099029 wait complete. 2024-09-16T00:34:30.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.395+0000 7ff82b47c740 20 client.16908 trim_cache size 0 max 16384 2024-09-16T00:34:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82b47c740 1 Processor -- start 2024-09-16T00:34:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82b47c740 1 -- start start 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555d5d0 0x55e32555e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 0x55e325574310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e325564130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82b47c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b7cf0 con 0x55e32555d5d0 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82b47c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b55c0 con 0x55e325568370 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82b47c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b3410 con 0x55e32555eb40 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555d5d0 0x55e32555e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555d5d0 0x55e32555e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49248/0 (socket says 172.21.15.1:49248) 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82359e640 1 -- 172.21.15.1:0/2702026466 learned_addr learned my addr 172.21.15.1:0/2702026466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82259c640 1 --2- 172.21.15.1:0/2702026466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 0x55e325574310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82359e640 1 --2- 172.21.15.1:0/2702026466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e325564130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82359e640 1 -- 172.21.15.1:0/2702026466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 msgr2=0x55e325574310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82359e640 1 --2- 172.21.15.1:0/2702026466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 0x55e325574310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82359e640 1 -- 172.21.15.1:0/2702026466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 msgr2=0x55e325564130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82359e640 1 --2- 172.21.15.1:0/2702026466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e325564130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.408+0000 7ff82359e640 1 -- 172.21.15.1:0/2702026466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e325565980 con 0x55e32555d5d0 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82359e640 1 --2- 172.21.15.1:0/2702026466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555d5d0 0x55e32555e300 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0149c0 tx=0x7ff81c014950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13893ca4dfb44c63 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff808ff9640 1 -- 172.21.15.1:0/2702026466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c013790 con 0x55e32555d5d0 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff808ff9640 1 -- 172.21.15.1:0/2702026466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c019040 con 0x55e32555d5d0 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff808ff9640 1 -- 172.21.15.1:0/2702026466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c01fa90 con 0x55e32555d5d0 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82b47c740 1 -- 172.21.15.1:0/2702026466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555d5d0 msgr2=0x55e32555e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82b47c740 1 --2- 172.21.15.1:0/2702026466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555d5d0 0x55e32555e300 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0149c0 tx=0x7ff81c014950 comp rx=0 tx=0).stop 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82b47c740 1 -- 172.21.15.1:0/2702026466 shutdown_connections 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82b47c740 1 --2- 172.21.15.1:0/2702026466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e325568370 0x55e325564130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82b47c740 1 --2- 172.21.15.1:0/2702026466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 0x55e325574310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82b47c740 1 --2- 172.21.15.1:0/2702026466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e32555d5d0 0x55e32555e300 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82b47c740 1 -- 172.21.15.1:0/2702026466 >> 172.21.15.1:0/2702026466 conn(0x55e325526310 msgr2=0x55e325554a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82b47c740 1 -- 172.21.15.1:0/2702026466 shutdown_connections 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.409+0000 7ff82b47c740 1 -- 172.21.15.1:0/2702026466 wait complete. 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82b47c740 1 -- start start 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82b47c740 1 Processor -- start 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e32555d5d0 0x55e325570480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 0x55e3255709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82b47c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e325568370 0x55e325561670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82b47c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b3a80 con 0x55e325568370 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82b47c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b4ef0 con 0x55e32555d5d0 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82b47c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b4150 con 0x55e32555eb40 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82b47c740 10 client.0 _ll_register_callbacks cb 0x7ff81800ae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e325568370 0x55e325561670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e325568370 0x55e325561670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49252/0 (socket says 172.21.15.1:49252) 2024-09-16T00:34:30.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82359e640 1 -- 172.21.15.1:0/229963022 learned_addr learned my addr 172.21.15.1:0/229963022 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.410+0000 7ff82259c640 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 0x55e3255709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82359e640 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e32555d5d0 0x55e325570480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82359e640 1 -- 172.21.15.1:0/229963022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 msgr2=0x55e3255709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82359e640 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 0x55e3255709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82359e640 1 -- 172.21.15.1:0/229963022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e32555d5d0 msgr2=0x55e325570480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82359e640 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e32555d5d0 0x55e325570480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82359e640 1 -- 172.21.15.1:0/229963022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e325561bb0 con 0x55e325568370 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82259c640 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 0x55e3255709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82359e640 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e325568370 0x55e325561670 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0070b0 tx=0x7ff81c013fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab04615e2d19fbc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff806ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff806ff5640 1 -- 172.21.15.1:0/229963022 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c009b00 con 0x55e325568370 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff806ff5640 1 -- 172.21.15.1:0/229963022 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff81c01f830 con 0x55e325568370 2024-09-16T00:34:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82b47c740 1 -- 172.21.15.1:0/229963022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81c019040 con 0x55e325568370 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff806ff5640 1 -- 172.21.15.1:0/229963022 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff81c03e050 con 0x55e325568370 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82b47c740 10 client.16914 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.411+0000 7ff82b47c740 1 -- 172.21.15.1:0/229963022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3255e2620 con 0x55e325568370 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff806ff5640 1 -- 172.21.15.1:0/229963022 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c019040 con 0x55e325568370 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff806ff5640 1 -- 172.21.15.1:0/229963022 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff81c01d3f0 con 0x55e325568370 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff82b47c740 10 client.16914 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff82b47c740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff82b47c740 20 client.16914 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff82b47c740 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e3255e2a40 0x55e3255e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff82b47c740 4 client.16914 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff82b47c740 1 -- 172.21.15.1:0/229963022 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3254cad70 con 0x55e3255e2a40 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff82259c640 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e3255e2a40 0x55e3255e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff82259c640 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e3255e2a40 0x55e3255e4e30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=966467975a0ade1a server_cookie=9352f503895e002c in_seq=0 out_seq=0 2024-09-16T00:34:30.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.412+0000 7ff806ff5640 10 client.16914 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff806ff5640 1 -- 172.21.15.1:0/229963022 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3254cad70 con 0x55e3255e2a40 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff806ff5640 10 client.16914 handle_command_reply: tid=0 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 client.16914 shutdown 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 -- 172.21.15.1:0/229963022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e325568370 msgr2=0x55e325561670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e325568370 0x55e325561670 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0070b0 tx=0x7ff81c013fa0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 -- 172.21.15.1:0/229963022 shutdown_connections 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e3255e2a40 0x55e3255e4e30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e325568370 0x55e325561670 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff806ff5640 0 client.16914 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e32555eb40 0x55e3255709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 --2- 172.21.15.1:0/229963022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e32555d5d0 0x55e325570480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 -- 172.21.15.1:0/229963022 >> 172.21.15.1:0/229963022 conn(0x55e325526310 msgr2=0x55e325554a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 -- 172.21.15.1:0/229963022 shutdown_connections 2024-09-16T00:34:30.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 1 -- 172.21.15.1:0/229963022 wait complete. 2024-09-16T00:34:30.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff82b47c740 20 client.16914 trim_cache size 0 max 16384 2024-09-16T00:34:30.420 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff8290a3640 1 -- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff80c0a3240 msgr2=0x7ff80c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff80c0a3240 0x7ff80c0a5700 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff8140edf50 tx=0x7ff814009680 comp rx=0 tx=0).stop 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff8290a3640 1 -- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 msgr2=0x7ff8241009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.413+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 0x7ff8241009a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff814005ab0 tx=0x7ff8140042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.414+0000 7ff8290a3640 1 -- 172.21.15.1:0/2554975255 shutdown_connections 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.414+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff82410c170 0x7ff824101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.414+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff80c0a3240 0x7ff80c0a5700 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.414+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff824103660 0x7ff824100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.414+0000 7ff8290a3640 1 --2- 172.21.15.1:0/2554975255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102d40 0x7ff8241009a0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.414+0000 7ff8290a3640 1 -- 172.21.15.1:0/2554975255 >> 172.21.15.1:0/2554975255 conn(0x7ff8240fc7a0 msgr2=0x7ff8240fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.414+0000 7ff8290a3640 1 -- 172.21.15.1:0/2554975255 shutdown_connections 2024-09-16T00:34:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:30.414+0000 7ff8290a3640 1 -- 172.21.15.1:0/2554975255 wait complete. 2024-09-16T00:34:32.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:31 smithi019 ceph-mon[35186]: pgmap v534: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:34:32.205 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:31 smithi001 ceph-mon[30934]: pgmap v534: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:34:32.209 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:34:32.210 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:34:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:31 smithi153 ceph-mon[35538]: pgmap v534: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:34:32.480 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.986+0000 7fd9a96d0640 1 -- 172.21.15.1:0/2616336176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 msgr2=0x7fd9a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.986+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/2616336176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 0x7fd9a4106e60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd994009930 tx=0x7fd99402a420 comp rx=0 tx=0).stop 2024-09-16T00:34:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.986+0000 7fd9a96d0640 1 -- 172.21.15.1:0/2616336176 shutdown_connections 2024-09-16T00:34:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.986+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/2616336176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9a4101420 0x7fd9a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.986+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/2616336176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9a4100a80 0x7fd9a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.986+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/2616336176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 0x7fd9a4106e60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.986+0000 7fd9a96d0640 1 -- 172.21.15.1:0/2616336176 >> 172.21.15.1:0/2616336176 conn(0x7fd9a40fc7a0 msgr2=0x7fd9a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.986+0000 7fd9a96d0640 1 -- 172.21.15.1:0/2616336176 shutdown_connections 2024-09-16T00:34:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.986+0000 7fd9a96d0640 1 -- 172.21.15.1:0/2616336176 wait complete. 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a96d0640 1 Processor -- start 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a96d0640 1 -- start start 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a96d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9a4100a80 0x7fd9a4102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a96d0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9a4101420 0x7fd9a4102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a96d0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 0x7fd9a41030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a96d0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4110e80 con 0x7fd9a4106a80 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a96d0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4110d00 con 0x7fd9a4100a80 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a96d0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4111000 con 0x7fd9a4101420 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 0x7fd9a41030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 0x7fd9a41030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49268/0 (socket says 172.21.15.1:49268) 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a37fe640 1 -- 172.21.15.1:0/4230940488 learned_addr learned my addr 172.21.15.1:0/4230940488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.987+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9a4100a80 0x7fd9a4102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9a4101420 0x7fd9a4102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a37fe640 1 -- 172.21.15.1:0/4230940488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9a4101420 msgr2=0x7fd9a4102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a37fe640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9a4101420 0x7fd9a4102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a37fe640 1 -- 172.21.15.1:0/4230940488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9a4100a80 msgr2=0x7fd9a4102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a37fe640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9a4100a80 0x7fd9a4102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a37fe640 1 -- 172.21.15.1:0/4230940488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a41a6910 con 0x7fd9a4106a80 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9a4100a80 0x7fd9a4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a37fe640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 0x7fd9a41030d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00eaa0 tx=0x7fd99c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd98bfff640 1 -- 172.21.15.1:0/4230940488 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00cdb0 con 0x7fd9a4106a80 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd98bfff640 1 -- 172.21.15.1:0/4230940488 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99c005020 con 0x7fd9a4106a80 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a96d0640 1 -- 172.21.15.1:0/4230940488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a41a6c00 con 0x7fd9a4106a80 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd98bfff640 1 -- 172.21.15.1:0/4230940488 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0116b0 con 0x7fd9a4106a80 2024-09-16T00:34:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.988+0000 7fd9a96d0640 1 -- 172.21.15.1:0/4230940488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a41a7140 con 0x7fd9a4106a80 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd98bfff640 1 -- 172.21.15.1:0/4230940488 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd99c005560 con 0x7fd9a4106a80 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9abaa9740 1 Processor -- start 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9abaa9740 1 -- start start 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6bb60 0x55ed5af6bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af76800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd98bfff640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd98c0a32c0 0x7fd98c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af76d40 0x55ed5af79130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ed5aed1280 con 0x55ed5af6c480 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ed5ae78b10 con 0x55ed5af76d40 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed5aecd240 con 0x55ed5af6bb60 2024-09-16T00:34:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af76800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9a37fe640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd98c0a32c0 0x7fd98c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af76800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49272/0 (socket says 172.21.15.1:49272) 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9a2ffd640 1 -- 172.21.15.1:0/2976906994 learned_addr learned my addr 172.21.15.1:0/2976906994 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.991+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6bb60 0x55ed5af6bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd98bfff640 1 -- 172.21.15.1:0/4230940488 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd99c0cfdf0 con 0x7fd9a4106a80 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af76d40 0x55ed5af79130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9a2ffd640 1 -- 172.21.15.1:0/2976906994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6bb60 msgr2=0x55ed5af6bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6bb60 0x55ed5af6bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9a2ffd640 1 -- 172.21.15.1:0/2976906994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af76d40 msgr2=0x55ed5af79130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af76d40 0x55ed5af79130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9a2ffd640 1 -- 172.21.15.1:0/2976906994 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed5ae851f0 con 0x55ed5af6c480 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6bb60 0x55ed5af6bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9a37fe640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd98c0a32c0 0x7fd98c0a5780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd99c09bdc0 tx=0x7fd99c066fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af76800 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd994000c00 tx=0x7fd994005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe6f86909e55191 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9897fa640 1 -- 172.21.15.1:0/2976906994 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994035040 con 0x55ed5af6c480 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9897fa640 1 -- 172.21.15.1:0/2976906994 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9940319f0 con 0x55ed5af6c480 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.992+0000 7fd9897fa640 1 -- 172.21.15.1:0/2976906994 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99403c440 con 0x55ed5af6c480 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2976906994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 msgr2=0x55ed5af76800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af76800 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd994000c00 tx=0x7fd994005e80 comp rx=0 tx=0).stop 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2976906994 shutdown_connections 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af76d40 0x55ed5af79130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af76800 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2976906994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6bb60 0x55ed5af6bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2976906994 >> 172.21.15.1:0/2976906994 conn(0x55ed5af680f0 msgr2=0x55ed5af684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2976906994 shutdown_connections 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2976906994 wait complete. 2024-09-16T00:34:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.993+0000 7fd9abaa9740 1 -- start start 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9abaa9740 1 Processor -- start 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 0x55ed5af76750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af73aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af76d40 0x55ed5af740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ed5af79910 con 0x55ed5af6c480 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ed5ae8bf10 con 0x55ed5af6bb60 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed5af79a90 con 0x55ed5af76d40 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9abaa9740 10 client.0 _ll_register_callbacks cb 0x55ed5ae54a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af73aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af73aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49286/0 (socket says 172.21.15.1:49286) 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a2ffd640 1 -- 172.21.15.1:0/2269419040 learned_addr learned my addr 172.21.15.1:0/2269419040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 0x55ed5af76750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af76d40 0x55ed5af740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a2ffd640 1 -- 172.21.15.1:0/2269419040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af76d40 msgr2=0x55ed5af740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af76d40 0x55ed5af740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a2ffd640 1 -- 172.21.15.1:0/2269419040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 msgr2=0x55ed5af76750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 0x55ed5af76750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.994+0000 7fd9a2ffd640 1 -- 172.21.15.1:0/2269419040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed5ac426a0 con 0x55ed5af6c480 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af73aa0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd99402a940 tx=0x7fd9940038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791e8e88a5c71316 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd986ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd986ff5640 1 -- 172.21.15.1:0/2269419040 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99404e070 con 0x55ed5af6c480 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2269419040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9940319f0 con 0x55ed5af6c480 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd986ff5640 1 -- 172.21.15.1:0/2269419040 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd994003d50 con 0x55ed5af6c480 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd986ff5640 1 -- 172.21.15.1:0/2269419040 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd99405a050 con 0x55ed5af6c480 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9abaa9740 10 client.16926 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2269419040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed5af853d0 con 0x55ed5af6c480 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd986ff5640 1 -- 172.21.15.1:0/2269419040 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9940319f0 con 0x55ed5af6c480 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd986ff5640 1 -- 172.21.15.1:0/2269419040 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd99403cb00 con 0x55ed5af6c480 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9abaa9740 10 client.16926 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9abaa9740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9abaa9740 20 client.16926 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed5af86720 0x55ed5af88b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9abaa9740 4 client.16926 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.995+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2269419040 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed5aede7f0 con 0x55ed5af86720 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.996+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed5af86720 0x55ed5af88b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.996+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed5af86720 0x55ed5af88b10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654dd1459a96bebf server_cookie=5545063df826e5fc in_seq=0 out_seq=0 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.996+0000 7fd986ff5640 10 client.16926 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd986ff5640 1 -- 172.21.15.1:0/2269419040 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed5aede7f0 con 0x55ed5af86720 2024-09-16T00:34:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd986ff5640 10 client.16926 handle_command_reply: tid=0 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 client.16926 shutdown 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2269419040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 msgr2=0x55ed5af73aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af73aa0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd99402a940 tx=0x7fd9940038a0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2269419040 shutdown_connections 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed5af86720 0x55ed5af88b10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af76d40 0x55ed5af740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6c480 0x55ed5af73aa0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/2269419040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 0x55ed5af76750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd986ff5640 0 client.16926 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2269419040 >> 172.21.15.1:0/2269419040 conn(0x55ed5af680f0 msgr2=0x55ed5af67270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2269419040 shutdown_connections 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 1 -- 172.21.15.1:0/2269419040 wait complete. 2024-09-16T00:34:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:32.997+0000 7fd9abaa9740 20 client.16926 trim_cache size 0 max 16384 2024-09-16T00:34:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9abaa9740 1 Processor -- start 2024-09-16T00:34:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9abaa9740 1 -- start start 2024-09-16T00:34:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6bb60 0x55ed5af82d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6c480 0x55ed5af6e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af76d40 0x55ed5af8bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b3940 con 0x55ed5af6bb60 2024-09-16T00:34:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b1210 con 0x55ed5af6c480 2024-09-16T00:34:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41af060 con 0x55ed5af76d40 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9a27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6bb60 0x55ed5af82d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9a27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6bb60 0x55ed5af82d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49296/0 (socket says 172.21.15.1:49296) 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9a27fc640 1 -- 172.21.15.1:0/3538896360 learned_addr learned my addr 172.21.15.1:0/3538896360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9a27fc640 1 -- 172.21.15.1:0/3538896360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af76d40 msgr2=0x55ed5af8bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/3538896360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af76d40 0x55ed5af8bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9a27fc640 1 -- 172.21.15.1:0/3538896360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6c480 msgr2=0x55ed5af6e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/3538896360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6c480 0x55ed5af6e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.015+0000 7fd9a27fc640 1 -- 172.21.15.1:0/3538896360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed5af67f20 con 0x55ed5af6bb60 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/3538896360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6bb60 0x55ed5af82d20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55ed5af79b70 tx=0x7fd99800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87d3a4e290c74bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd988ff9640 1 -- 172.21.15.1:0/3538896360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998025040 con 0x55ed5af6bb60 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd988ff9640 1 -- 172.21.15.1:0/3538896360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9980043d0 con 0x55ed5af6bb60 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd988ff9640 1 -- 172.21.15.1:0/3538896360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998004e00 con 0x55ed5af6bb60 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd9abaa9740 1 -- 172.21.15.1:0/3538896360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6bb60 msgr2=0x55ed5af82d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/3538896360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6bb60 0x55ed5af82d20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55ed5af79b70 tx=0x7fd99800b390 comp rx=0 tx=0).stop 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd9abaa9740 1 -- 172.21.15.1:0/3538896360 shutdown_connections 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/3538896360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af76d40 0x55ed5af8bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/3538896360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6c480 0x55ed5af6e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/3538896360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af6bb60 0x55ed5af82d20 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.016+0000 7fd9abaa9740 1 -- 172.21.15.1:0/3538896360 >> 172.21.15.1:0/3538896360 conn(0x55ed5af680f0 msgr2=0x55ed5af72b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 -- 172.21.15.1:0/3538896360 shutdown_connections 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 -- 172.21.15.1:0/3538896360 wait complete. 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 -- start start 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 Processor -- start 2024-09-16T00:34:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 0x55ed5af8c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6c480 0x55ed5af89770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af76d40 0x55ed5af89eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41af6d0 con 0x55ed5af76d40 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b0b40 con 0x55ed5af6bb60 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41afda0 con 0x55ed5af6c480 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9abaa9740 10 client.0 _ll_register_callbacks cb 0x7fd994005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9a27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af76d40 0x55ed5af89eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9a27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af76d40 0x55ed5af89eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49310/0 (socket says 172.21.15.1:49310) 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.017+0000 7fd9a27fc640 1 -- 172.21.15.1:0/1672886260 learned_addr learned my addr 172.21.15.1:0/1672886260 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 0x55ed5af8c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6c480 0x55ed5af89770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9a27fc640 1 -- 172.21.15.1:0/1672886260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6c480 msgr2=0x55ed5af89770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6c480 0x55ed5af89770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9a27fc640 1 -- 172.21.15.1:0/1672886260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 msgr2=0x55ed5af8c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 0x55ed5af8c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9a27fc640 1 -- 172.21.15.1:0/1672886260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed5aff0060 con 0x55ed5af76d40 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9a27fc640 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af76d40 0x55ed5af89eb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd99800b7b0 tx=0x7fd9980049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff5d27947c3c887 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd986ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd986ff5640 1 -- 172.21.15.1:0/1672886260 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99802f060 con 0x55ed5af76d40 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd986ff5640 1 -- 172.21.15.1:0/1672886260 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd998004ed0 con 0x55ed5af76d40 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9abaa9740 1 -- 172.21.15.1:0/1672886260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9980043d0 con 0x55ed5af76d40 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd986ff5640 1 -- 172.21.15.1:0/1672886260 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9980050d0 con 0x55ed5af76d40 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9abaa9740 10 client.16932 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.018+0000 7fd9abaa9740 1 -- 172.21.15.1:0/1672886260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed5aff1240 con 0x55ed5af76d40 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd986ff5640 1 -- 172.21.15.1:0/1672886260 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9980043d0 con 0x55ed5af76d40 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd986ff5640 1 -- 172.21.15.1:0/1672886260 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd998012cb0 con 0x55ed5af76d40 2024-09-16T00:34:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd9abaa9740 10 client.16932 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd9abaa9740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd9abaa9740 20 client.16932 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed5aff18b0 0x55ed5aff3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd9abaa9740 4 client.16932 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:34:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd9abaa9740 1 -- 172.21.15.1:0/1672886260 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ed5aed8d00 con 0x55ed5aff18b0 2024-09-16T00:34:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed5aff18b0 0x55ed5aff3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd9a2ffd640 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed5aff18b0 0x55ed5aff3ca0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e9c662c21a12d3 server_cookie=dfe36cbd98661f11 in_seq=0 out_seq=0 2024-09-16T00:34:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.019+0000 7fd986ff5640 10 client.16932 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:33.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.020+0000 7fd986ff5640 1 -- 172.21.15.1:0/1672886260 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ed5aed8d00 con 0x55ed5aff18b0 2024-09-16T00:34:33.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.020+0000 7fd986ff5640 10 client.16932 handle_command_reply: tid=0 2024-09-16T00:34:33.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.020+0000 7fd9abaa9740 1 client.16932 shutdown 2024-09-16T00:34:33.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.020+0000 7fd9abaa9740 1 -- 172.21.15.1:0/1672886260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af76d40 msgr2=0x55ed5af89eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.020+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af76d40 0x55ed5af89eb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd99800b7b0 tx=0x7fd9980049e0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9abaa9740 1 -- 172.21.15.1:0/1672886260 shutdown_connections 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ed5aff18b0 0x55ed5aff3ca0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd986ff5640 0 client.16932 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed5af76d40 0x55ed5af89eb0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed5af6c480 0x55ed5af89770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9abaa9740 1 --2- 172.21.15.1:0/1672886260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5af6bb60 0x55ed5af8c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9abaa9740 1 -- 172.21.15.1:0/1672886260 >> 172.21.15.1:0/1672886260 conn(0x55ed5af680f0 msgr2=0x55ed5af66a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9abaa9740 1 -- 172.21.15.1:0/1672886260 shutdown_connections 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9abaa9740 1 -- 172.21.15.1:0/1672886260 wait complete. 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9abaa9740 20 client.16932 trim_cache size 0 max 16384 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:34:33.027 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:34:33.028 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "538be803-403a-4d29-ba38-15e19becfddc", 2024-09-16T00:34:33.028 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:34:33.028 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:34:33.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9a96d0640 1 -- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd98c0a32c0 msgr2=0x7fd98c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd98c0a32c0 0x7fd98c0a5780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd99c09bdc0 tx=0x7fd99c066fa0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9a96d0640 1 -- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 msgr2=0x7fd9a41030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 0x7fd9a41030d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00eaa0 tx=0x7fd99c00ef60 comp rx=0 tx=0).stop 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.021+0000 7fd9a96d0640 1 -- 172.21.15.1:0/4230940488 shutdown_connections 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.022+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a4106a80 0x7fd9a41030d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.022+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd98c0a32c0 0x7fd98c0a5780 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.022+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9a4101420 0x7fd9a4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.022+0000 7fd9a96d0640 1 --2- 172.21.15.1:0/4230940488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9a4100a80 0x7fd9a4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.022+0000 7fd9a96d0640 1 -- 172.21.15.1:0/4230940488 >> 172.21.15.1:0/4230940488 conn(0x7fd9a40fc7a0 msgr2=0x7fd9a406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.022+0000 7fd9a96d0640 1 -- 172.21.15.1:0/4230940488 shutdown_connections 2024-09-16T00:34:33.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:33.022+0000 7fd9a96d0640 1 -- 172.21.15.1:0/4230940488 wait complete. 2024-09-16T00:34:33.830 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:34:33.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:33 smithi001 ceph-mon[30934]: pgmap v535: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-16T00:34:34.090 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:34.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:33 smithi019 ceph-mon[35186]: pgmap v535: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-16T00:34:34.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:33 smithi153 ceph-mon[35538]: pgmap v535: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-16T00:34:34.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.586+0000 7f2be8ee8640 1 -- 172.21.15.1:0/1357740303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 msgr2=0x7f2be41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.586+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/1357740303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 0x7f2be41067e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009900 tx=0x7f2bcc02a410 comp rx=0 tx=0).stop 2024-09-16T00:34:34.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.586+0000 7f2be8ee8640 1 -- 172.21.15.1:0/1357740303 shutdown_connections 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.586+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/1357740303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2be4106f10 0x7f2be410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.586+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/1357740303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 0x7f2be41067e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.586+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/1357740303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be4101de0 0x7f2be41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.586+0000 7f2be8ee8640 1 -- 172.21.15.1:0/1357740303 >> 172.21.15.1:0/1357740303 conn(0x7f2be40fd8c0 msgr2=0x7f2be40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.586+0000 7f2be8ee8640 1 -- 172.21.15.1:0/1357740303 shutdown_connections 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.586+0000 7f2be8ee8640 1 -- 172.21.15.1:0/1357740303 wait complete. 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be8ee8640 1 Processor -- start 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be8ee8640 1 -- start start 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be8ee8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be4101de0 0x7f2be41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be8ee8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 0x7f2be41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be8ee8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2be4106f10 0x7f2be419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be8ee8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4111f70 con 0x7f2be4102790 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be8ee8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4111df0 con 0x7f2be4101de0 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be8ee8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41120f0 con 0x7f2be4106f10 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 0x7f2be41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 0x7f2be41988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49322/0 (socket says 172.21.15.1:49322) 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be1d9b640 1 -- 172.21.15.1:0/2482207950 learned_addr learned my addr 172.21.15.1:0/2482207950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be2d9d640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2be4106f10 0x7f2be419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be259c640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be4101de0 0x7f2be41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be1d9b640 1 -- 172.21.15.1:0/2482207950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2be4106f10 msgr2=0x7f2be419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2be4106f10 0x7f2be419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be1d9b640 1 -- 172.21.15.1:0/2482207950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be4101de0 msgr2=0x7f2be41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be4101de0 0x7f2be41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.587+0000 7f2be1d9b640 1 -- 172.21.15.1:0/2482207950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be419d360 con 0x7f2be4102790 2024-09-16T00:34:34.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.588+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 0x7f2be41988f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005ab0 tx=0x7f2bcc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.588+0000 7f2bdb7fe640 1 -- 172.21.15.1:0/2482207950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc03f030 con 0x7f2be4102790 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.588+0000 7f2bdb7fe640 1 -- 172.21.15.1:0/2482207950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc003ce0 con 0x7f2be4102790 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.588+0000 7f2be8ee8640 1 -- 172.21.15.1:0/2482207950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be419d5f0 con 0x7f2be4102790 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.588+0000 7f2bdb7fe640 1 -- 172.21.15.1:0/2482207950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc035070 con 0x7f2be4102790 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.588+0000 7f2be8ee8640 1 -- 172.21.15.1:0/2482207950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be41a4f30 con 0x7f2be4102790 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.590+0000 7f2bdb7fe640 1 -- 172.21.15.1:0/2482207950 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2bcc042030 con 0x7f2be4102790 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.590+0000 7f2beb2c1740 1 Processor -- start 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2beb2c1740 1 -- start start 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 0x560c8e290760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e29f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e2a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560c8e1f62e0 con 0x560c8e29fa90 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560c8e19dce0 con 0x560c8e290380 2024-09-16T00:34:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560c8e1f22a0 con 0x560c8e290ca0 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2bdb7fe640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bd00a32c0 0x7f2bd00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e2a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e2a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49326/0 (socket says 172.21.15.1:49326) 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be1d9b640 1 -- 172.21.15.1:0/791353721 learned_addr learned my addr 172.21.15.1:0/791353721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be2d9d640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bd00a32c0 0x7f2bd00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be259c640 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e29f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be2d9d640 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 0x560c8e290760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2bdb7fe640 1 -- 172.21.15.1:0/2482207950 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2bcc0f0550 con 0x7f2be4102790 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be1d9b640 1 -- 172.21.15.1:0/791353721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 msgr2=0x560c8e29f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e29f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be1d9b640 1 -- 172.21.15.1:0/791353721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 msgr2=0x560c8e290760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.591+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 0x560c8e290760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.592+0000 7f2be1d9b640 1 -- 172.21.15.1:0/791353721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8e182440 con 0x560c8e29fa90 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.592+0000 7f2be259c640 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e29f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.592+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e2a1e80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005940 tx=0x7f2bcc03b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f67d95b984188f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.592+0000 7f2be2d9d640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bd00a32c0 0x7f2bd00a5780 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00afd0 tx=0x7f2bdc00a490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.592+0000 7f2bd8ff9640 1 -- 172.21.15.1:0/791353721 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc049070 con 0x560c8e29fa90 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.592+0000 7f2bd8ff9640 1 -- 172.21.15.1:0/791353721 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc0bc7c0 con 0x560c8e29fa90 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.592+0000 7f2bd8ff9640 1 -- 172.21.15.1:0/791353721 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc0b8e30 con 0x560c8e29fa90 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 -- 172.21.15.1:0/791353721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 msgr2=0x560c8e2a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e2a1e80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005940 tx=0x7f2bcc03b800 comp rx=0 tx=0).stop 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 -- 172.21.15.1:0/791353721 shutdown_connections 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e2a1e80 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e29f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/791353721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 0x560c8e290760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 -- 172.21.15.1:0/791353721 >> 172.21.15.1:0/791353721 conn(0x560c8e28ca30 msgr2=0x560c8e28ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 -- 172.21.15.1:0/791353721 shutdown_connections 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 -- 172.21.15.1:0/791353721 wait complete. 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 -- start start 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 Processor -- start 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.593+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 0x560c8e29f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e29c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e29cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560c8e2a2420 con 0x560c8e29fa90 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560c8e1b1050 con 0x560c8e290380 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560c8e2a25a0 con 0x560c8e290ca0 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2beb2c1740 10 client.0 _ll_register_callbacks cb 0x560c8e1f1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e29cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e29cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49336/0 (socket says 172.21.15.1:49336) 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be2d9d640 1 -- 172.21.15.1:0/1225865508 learned_addr learned my addr 172.21.15.1:0/1225865508 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be259c640 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 0x560c8e29f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e29c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be2d9d640 1 -- 172.21.15.1:0/1225865508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 msgr2=0x560c8e29c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be2d9d640 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e29c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be2d9d640 1 -- 172.21.15.1:0/1225865508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 msgr2=0x560c8e29f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be2d9d640 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 0x560c8e29f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be2d9d640 1 -- 172.21.15.1:0/1225865508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c8df673c0 con 0x560c8e29fa90 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be259c640 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 0x560c8e29f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2be2d9d640 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e29cbf0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00ead0 tx=0x7f2bdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c46450be1a6e130 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1225865508 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00fc30 con 0x560c8e29fa90 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1225865508 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2bdc00fdd0 con 0x560c8e29fa90 2024-09-16T00:34:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.594+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1225865508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bcc0bc7c0 con 0x560c8e29fa90 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1225865508 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2bdc004020 con 0x560c8e29fa90 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2beb2c1740 10 client.16944 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1225865508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560c8e2add80 con 0x560c8e29fa90 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1225865508 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc0bc7c0 con 0x560c8e29fa90 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1225865508 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2bdc028420 con 0x560c8e29fa90 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2beb2c1740 10 client.16944 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2beb2c1740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2beb2c1740 20 client.16944 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560c8e2af240 0x560c8e2b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2beb2c1740 4 client.16944 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1225865508 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c8e203850 con 0x560c8e2af240 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.595+0000 7f2be259c640 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560c8e2af240 0x560c8e2b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.596+0000 7f2be259c640 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560c8e2af240 0x560c8e2b1670 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f0659b897ea3b9 server_cookie=1d6b0c5848433ae9 in_seq=0 out_seq=0 2024-09-16T00:34:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.596+0000 7f2bc9ffb640 10 client.16944 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1225865508 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c8e203850 con 0x560c8e2af240 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2bc9ffb640 10 client.16944 handle_command_reply: tid=0 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 client.16944 shutdown 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1225865508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 msgr2=0x560c8e29cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e29cbf0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00ead0 tx=0x7f2bdc004730 comp rx=0 tx=0).stop 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1225865508 shutdown_connections 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560c8e2af240 0x560c8e2b1670 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e29fa90 0x560c8e29cbf0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e29c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1225865508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e290380 0x560c8e29f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1225865508 >> 172.21.15.1:0/1225865508 conn(0x560c8e28ca30 msgr2=0x560c8e292790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2bc9ffb640 0 client.16944 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1225865508 shutdown_connections 2024-09-16T00:34:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1225865508 wait complete. 2024-09-16T00:34:34.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.597+0000 7f2beb2c1740 20 client.16944 trim_cache size 0 max 16384 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2beb2c1740 1 Processor -- start 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2beb2c1740 1 -- start start 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290380 0x560c8e2ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290ca0 0x560c8e28ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e29fa90 0x560c8e297c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41af4b0 con 0x560c8e290ca0 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41acd80 con 0x560c8e29fa90 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41aabd0 con 0x560c8e290380 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2be1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290ca0 0x560c8e28ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2be1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290ca0 0x560c8e28ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49342/0 (socket says 172.21.15.1:49342) 2024-09-16T00:34:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.614+0000 7f2be1d9b640 1 -- 172.21.15.1:0/3498052092 learned_addr learned my addr 172.21.15.1:0/3498052092 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.615+0000 7f2be1d9b640 1 -- 172.21.15.1:0/3498052092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290380 msgr2=0x560c8e2ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.615+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/3498052092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290380 0x560c8e2ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.615+0000 7f2be1d9b640 1 -- 172.21.15.1:0/3498052092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e29fa90 msgr2=0x560c8e297c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.615+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/3498052092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e29fa90 0x560c8e297c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.615+0000 7f2be1d9b640 1 -- 172.21.15.1:0/3498052092 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8e29b640 con 0x560c8e290ca0 2024-09-16T00:34:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.615+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/3498052092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290ca0 0x560c8e28ce10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0bc750 tx=0x7f2bcc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790a0be0a0fd5671 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:34.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.615+0000 7f2bcbfff640 1 -- 172.21.15.1:0/3498052092 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc049040 con 0x560c8e290ca0 2024-09-16T00:34:34.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.615+0000 7f2bcbfff640 1 -- 172.21.15.1:0/3498052092 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc0c0ac0 con 0x560c8e290ca0 2024-09-16T00:34:34.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.615+0000 7f2bcbfff640 1 -- 172.21.15.1:0/3498052092 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc0b8e30 con 0x560c8e290ca0 2024-09-16T00:34:34.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 -- 172.21.15.1:0/3498052092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290ca0 msgr2=0x560c8e28ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/3498052092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290ca0 0x560c8e28ce10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0bc750 tx=0x7f2bcc02a410 comp rx=0 tx=0).stop 2024-09-16T00:34:34.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 -- 172.21.15.1:0/3498052092 shutdown_connections 2024-09-16T00:34:34.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/3498052092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e29fa90 0x560c8e297c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/3498052092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290ca0 0x560c8e28ce10 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/3498052092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290380 0x560c8e2ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 -- 172.21.15.1:0/3498052092 >> 172.21.15.1:0/3498052092 conn(0x560c8e28ca30 msgr2=0x560c8e2974f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 -- 172.21.15.1:0/3498052092 shutdown_connections 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 -- 172.21.15.1:0/3498052092 wait complete. 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 -- start start 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 Processor -- start 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290380 0x560c8e2a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e2a3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e29fa90 0x560c8e2a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41ab240 con 0x560c8e290380 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41ac6b0 con 0x560c8e29fa90 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.616+0000 7f2beb2c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41ab910 con 0x560c8e290ca0 2024-09-16T00:34:34.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2beb2c1740 10 client.0 _ll_register_callbacks cb 0x7f2bdc0115b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290380 0x560c8e2a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290380 0x560c8e2a7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49348/0 (socket says 172.21.15.1:49348) 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be259c640 1 -- 172.21.15.1:0/1394422444 learned_addr learned my addr 172.21.15.1:0/1394422444 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e2a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be2d9d640 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e29fa90 0x560c8e2a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be259c640 1 -- 172.21.15.1:0/1394422444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 msgr2=0x560c8e2a3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be259c640 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e2a3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be259c640 1 -- 172.21.15.1:0/1394422444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e29fa90 msgr2=0x560c8e2a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be259c640 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e29fa90 0x560c8e2a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be259c640 1 -- 172.21.15.1:0/1394422444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c8e2a3f60 con 0x560c8e290380 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e2a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2be259c640 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290380 0x560c8e2a7ef0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400a300 tx=0x7f2bd4010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c066eaac47f97f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:34.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1394422444 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd400fd60 con 0x560c8e290380 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1394422444 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2bd4003970 con 0x560c8e290380 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.617+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1394422444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bcc0c0ac0 con 0x560c8e290380 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1394422444 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2bd402f050 con 0x560c8e290380 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2beb2c1740 10 client.16950 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1394422444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560c8e319fb0 con 0x560c8e290380 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1394422444 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc0c0ac0 con 0x560c8e290380 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1394422444 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2bd401d520 con 0x560c8e290380 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2beb2c1740 10 client.16950 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2beb2c1740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2beb2c1740 20 client.16950 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560c8e31a6e0 0x560c8e31cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2beb2c1740 4 client.16950 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1394422444 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c8e1fdd60 con 0x560c8e31a6e0 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.618+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560c8e31a6e0 0x560c8e31cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2be1d9b640 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560c8e31a6e0 0x560c8e31cb00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c496a68c6ac9229b server_cookie=65e2c375fd9392dd in_seq=0 out_seq=0 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2bc9ffb640 10 client.16950 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2bc9ffb640 1 -- 172.21.15.1:0/1394422444 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560c8e1fdd60 con 0x560c8e31a6e0 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2bc9ffb640 10 client.16950 handle_command_reply: tid=0 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 client.16950 shutdown 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1394422444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290380 msgr2=0x560c8e2a7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290380 0x560c8e2a7ef0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400a300 tx=0x7f2bd4010c80 comp rx=0 tx=0).stop 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1394422444 shutdown_connections 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560c8e31a6e0 0x560c8e31cb00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c8e29fa90 0x560c8e2a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c8e290ca0 0x560c8e2a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 --2- 172.21.15.1:0/1394422444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c8e290380 0x560c8e2a7ef0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1394422444 >> 172.21.15.1:0/1394422444 conn(0x560c8e28ca30 msgr2=0x560c8e297d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2bc9ffb640 0 client.16950 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1394422444 shutdown_connections 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 1 -- 172.21.15.1:0/1394422444 wait complete. 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.619+0000 7f2beb2c1740 20 client.16950 trim_cache size 0 max 16384 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:34:34.625 INFO:teuthology.orchestra.run.smithi001.stdout: "538be803-403a-4d29-ba38-15e19becfddc": { 2024-09-16T00:34:34.626 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:34:34.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "538be803-403a-4d29-ba38-15e19becfddc", 2024-09-16T00:34:34.626 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:34:34.626 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:34.626 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:34.626 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:34:34.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 -- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bd00a32c0 msgr2=0x7f2bd00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bd00a32c0 0x7f2bd00a5780 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00afd0 tx=0x7f2bdc00a490 comp rx=0 tx=0).stop 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 -- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 msgr2=0x7f2be41988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 0x7f2be41988f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005ab0 tx=0x7f2bcc0042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 -- 172.21.15.1:0/2482207950 shutdown_connections 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bd00a32c0 0x7f2bd00a5780 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2be4106f10 0x7f2be419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be4102790 0x7f2be41988f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 --2- 172.21.15.1:0/2482207950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be4101de0 0x7f2be41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 -- 172.21.15.1:0/2482207950 >> 172.21.15.1:0/2482207950 conn(0x7f2be40fd8c0 msgr2=0x7f2be40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 -- 172.21.15.1:0/2482207950 shutdown_connections 2024-09-16T00:34:34.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:34.620+0000 7f2be8ee8640 1 -- 172.21.15.1:0/2482207950 wait complete. 2024-09-16T00:34:35.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:34 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:34:35.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:34 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:35.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:34 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:34:35.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:34 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:35.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:34 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:34:35.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:34 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:35.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:34 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:34:35.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:34 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:35.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:34 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:34:35.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:34 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:35.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:34 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:34:35.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:34 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:35.397 INFO:tasks.cephfs.filesystem:scrub status for tag:538be803-403a-4d29-ba38-15e19becfddc - {'path': '/', 'tag': '538be803-403a-4d29-ba38-15e19becfddc', 'options': 'recursive,force'} 2024-09-16T00:34:36.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:35 smithi019 ceph-mon[35186]: pgmap v536: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:34:36.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:35 smithi001 ceph-mon[30934]: pgmap v536: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:34:36.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:35 smithi153 ceph-mon[35538]: pgmap v536: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:34:37.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:34:37.992 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:38.114 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:37 smithi001 ceph-mon[30934]: pgmap v537: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-16T00:34:38.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:37 smithi153 ceph-mon[35538]: pgmap v537: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-16T00:34:38.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:37 smithi019 ceph-mon[35186]: pgmap v537: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-16T00:34:38.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.462+0000 7f98ecbab640 1 -- 172.21.15.1:0/2988188477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e810c180 msgr2=0x7f98e810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.462+0000 7f98ecbab640 1 --2- 172.21.15.1:0/2988188477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e810c180 0x7f98e810e5f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009ca0 tx=0x7f98e002a950 comp rx=0 tx=0).stop 2024-09-16T00:34:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.462+0000 7f98ecbab640 1 -- 172.21.15.1:0/2988188477 shutdown_connections 2024-09-16T00:34:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.462+0000 7f98ecbab640 1 --2- 172.21.15.1:0/2988188477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e810c180 0x7f98e810e5f0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.462+0000 7f98ecbab640 1 --2- 172.21.15.1:0/2988188477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e8103700 0x7f98e810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.462+0000 7f98ecbab640 1 --2- 172.21.15.1:0/2988188477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e8102de0 0x7f98e81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.462+0000 7f98ecbab640 1 -- 172.21.15.1:0/2988188477 >> 172.21.15.1:0/2988188477 conn(0x7f98e80fc780 msgr2=0x7f98e80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.462+0000 7f98ecbab640 1 -- 172.21.15.1:0/2988188477 shutdown_connections 2024-09-16T00:34:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.462+0000 7f98ecbab640 1 -- 172.21.15.1:0/2988188477 wait complete. 2024-09-16T00:34:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98ecbab640 1 Processor -- start 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98ecbab640 1 -- start start 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98ecbab640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e8102de0 0x7f98e81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98ecbab640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e8103700 0x7f98e8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98ecbab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e810c180 0x7f98e819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98ecbab640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8111f50 con 0x7f98e8102de0 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98ecbab640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8111dd0 con 0x7f98e810c180 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98ecbab640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81120d0 con 0x7f98e8103700 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e8102de0 0x7f98e81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e8102de0 0x7f98e81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49372/0 (socket says 172.21.15.1:49372) 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e659c640 1 -- 172.21.15.1:0/3551580155 learned_addr learned my addr 172.21.15.1:0/3551580155 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e6d9d640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e810c180 0x7f98e819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e5d9b640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e8103700 0x7f98e8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e659c640 1 -- 172.21.15.1:0/3551580155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e8103700 msgr2=0x7f98e8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e659c640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e8103700 0x7f98e8198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e659c640 1 -- 172.21.15.1:0/3551580155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e810c180 msgr2=0x7f98e819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e659c640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e810c180 0x7f98e819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.463+0000 7f98e659c640 1 -- 172.21.15.1:0/3551580155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e819d480 con 0x7f98e8102de0 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.464+0000 7f98e659c640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e8102de0 0x7f98e81984d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f98d800d950 tx=0x7f98d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.464+0000 7f98d77fe640 1 -- 172.21.15.1:0/3551580155 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8017070 con 0x7f98e8102de0 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.464+0000 7f98d77fe640 1 -- 172.21.15.1:0/3551580155 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d8009c30 con 0x7f98e8102de0 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.464+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e819d770 con 0x7f98e8102de0 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.464+0000 7f98d77fe640 1 -- 172.21.15.1:0/3551580155 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d800f6b0 con 0x7f98e8102de0 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.464+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e81a5050 con 0x7f98e8102de0 2024-09-16T00:34:38.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.465+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e81031c0 con 0x7f98e8102de0 2024-09-16T00:34:38.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.466+0000 7f98d77fe640 1 -- 172.21.15.1:0/3551580155 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f98d80109d0 con 0x7f98e8102de0 2024-09-16T00:34:38.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.467+0000 7f98d77fe640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98d00a32c0 0x7f98d00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.467+0000 7f98e5d9b640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98d00a32c0 0x7f98d00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.467+0000 7f98d77fe640 1 -- 172.21.15.1:0/3551580155 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f98d80cf960 con 0x7f98e8102de0 2024-09-16T00:34:38.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.467+0000 7f98e5d9b640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98d00a32c0 0x7f98d00a5780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f98e80feda0 tx=0x7f98dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:38.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.471+0000 7f98d77fe640 1 -- 172.21.15.1:0/3551580155 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98d8009de0 con 0x7f98e8102de0 2024-09-16T00:34:38.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.686+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98e8199830 con 0x7f98e8102de0 2024-09-16T00:34:38.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.687+0000 7f98d77fe640 1 -- 172.21.15.1:0/3551580155 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f98d809c220 con 0x7f98e8102de0 2024-09-16T00:34:38.688 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:34:38.688 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:34:38.688 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.689+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98d00a32c0 msgr2=0x7f98d00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.689+0000 7f98ecbab640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98d00a32c0 0x7f98d00a5780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f98e80feda0 tx=0x7f98dc005f90 comp rx=0 tx=0).stop 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.689+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e8102de0 msgr2=0x7f98e81984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.689+0000 7f98ecbab640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e8102de0 0x7f98e81984d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f98d800d950 tx=0x7f98d800de10 comp rx=0 tx=0).stop 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.690+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 shutdown_connections 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.690+0000 7f98ecbab640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e810c180 0x7f98e819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.690+0000 7f98ecbab640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f98d00a32c0 0x7f98d00a5780 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.690+0000 7f98ecbab640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e8103700 0x7f98e8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.690+0000 7f98ecbab640 1 --2- 172.21.15.1:0/3551580155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98e8102de0 0x7f98e81984d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.690+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 >> 172.21.15.1:0/3551580155 conn(0x7f98e80fc780 msgr2=0x7f98e80fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:38.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.690+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 shutdown_connections 2024-09-16T00:34:38.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:38.690+0000 7f98ecbab640 1 -- 172.21.15.1:0/3551580155 wait complete. 2024-09-16T00:34:38.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:38 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:34:38.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:38 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3551580155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:39.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:38 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:34:39.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:38 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3551580155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:39.258 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:38 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:34:39.258 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:38 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3551580155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:39.520 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 183573db --release --await' 2024-09-16T00:34:39.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 183573db --release --await 2024-09-16T00:34:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:40.198 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:39 smithi001 ceph-mon[30934]: pgmap v538: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-16T00:34:40.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:39 smithi153 ceph-mon[35538]: pgmap v538: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-16T00:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.288+0000 7fac22a2a640 1 -- 172.21.15.1:0/3856881471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 msgr2=0x7fac1c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.288+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3856881471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 0x7fac1c104e80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fac10009900 tx=0x7fac1002a410 comp rx=0 tx=0).stop 2024-09-16T00:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.288+0000 7fac22a2a640 1 -- 172.21.15.1:0/3856881471 shutdown_connections 2024-09-16T00:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.288+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3856881471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c1055b0 0x7fac1c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.288+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3856881471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 0x7fac1c104e80 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.288+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3856881471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac1c100480 0x7fac1c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.288+0000 7fac22a2a640 1 -- 172.21.15.1:0/3856881471 >> 172.21.15.1:0/3856881471 conn(0x7fac1c077210 msgr2=0x7fac1c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.288+0000 7fac22a2a640 1 -- 172.21.15.1:0/3856881471 shutdown_connections 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.288+0000 7fac22a2a640 1 -- 172.21.15.1:0/3856881471 wait complete. 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac22a2a640 1 Processor -- start 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac22a2a640 1 -- start start 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac22a2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c100480 0x7fac1c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac22a2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 0x7fac1c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac22a2a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac1c1055b0 0x7fac1c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac22a2a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c10fd90 con 0x7fac1c100e30 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac22a2a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c10fc10 con 0x7fac1c100480 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac22a2a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c10ff10 con 0x7fac1c1055b0 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 0x7fac1c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 0x7fac1c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50442/0 (socket says 172.21.15.1:50442) 2024-09-16T00:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.289+0000 7fac1b7fe640 1 -- 172.21.15.1:0/3308386292 learned_addr learned my addr 172.21.15.1:0/3308386292 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac1bfff640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c100480 0x7fac1c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac20fc7640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac1c1055b0 0x7fac1c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac1b7fe640 1 -- 172.21.15.1:0/3308386292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac1c1055b0 msgr2=0x7fac1c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac1c1055b0 0x7fac1c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac1b7fe640 1 -- 172.21.15.1:0/3308386292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c100480 msgr2=0x7fac1c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c100480 0x7fac1c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac1b7fe640 1 -- 172.21.15.1:0/3308386292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c19d440 con 0x7fac1c100e30 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac20fc7640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac1c1055b0 0x7fac1c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 0x7fac1c1989d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fac10006010 tx=0x7fac100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac197fa640 1 -- 172.21.15.1:0/3308386292 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1003f030 con 0x7fac1c100e30 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac22a2a640 1 -- 172.21.15.1:0/3308386292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c19d6d0 con 0x7fac1c100e30 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac197fa640 1 -- 172.21.15.1:0/3308386292 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac10003ce0 con 0x7fac1c100e30 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac197fa640 1 -- 172.21.15.1:0/3308386292 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10035070 con 0x7fac1c100e30 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.290+0000 7fac22a2a640 1 -- 172.21.15.1:0/3308386292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1a5010 con 0x7fac1c100e30 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.292+0000 7fac197fa640 1 -- 172.21.15.1:0/3308386292 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fac10042030 con 0x7fac1c100e30 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.292+0000 7fac24e03740 1 Processor -- start 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac24e03740 1 -- start start 2024-09-16T00:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac24e03740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45fe9b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac24e03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b980 0x55d45fea5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac24e03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fea8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac24e03740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d45fe00620 con 0x55d45fe9b060 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac24e03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d45fda7e50 con 0x55d45fea6240 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac24e03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d45fdfc5e0 con 0x55d45fe9b980 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac20fc7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45fe9b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac20fc7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45fe9b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50452/0 (socket says 172.21.15.1:50452) 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac20fc7640 1 -- 172.21.15.1:0/106268631 learned_addr learned my addr 172.21.15.1:0/106268631 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac20fc7640 1 -- 172.21.15.1:0/106268631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b980 msgr2=0x55d45fea5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac1bfff640 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b980 0x55d45fea5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac197fa640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fac040a3240 0x7fac040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac20fc7640 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b980 0x55d45fea5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fea8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac20fc7640 1 -- 172.21.15.1:0/106268631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 msgr2=0x55d45fea8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac20fc7640 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fea8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac20fc7640 1 -- 172.21.15.1:0/106268631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d45fd98d40 con 0x55d45fe9b060 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fea8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.293+0000 7fac197fa640 1 -- 172.21.15.1:0/3308386292 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fac100eff90 con 0x7fac1c100e30 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.294+0000 7fac20fc7640 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45fe9b440 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fac1400af20 tx=0x7fac1400f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46bcc7d30d0cf92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.294+0000 7fac20fc7640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fac040a3240 0x7fac040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.294+0000 7fac027fc640 1 -- 172.21.15.1:0/106268631 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1401c070 con 0x55d45fe9b060 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.294+0000 7fac027fc640 1 -- 172.21.15.1:0/106268631 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1400dc90 con 0x55d45fe9b060 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.294+0000 7fac027fc640 1 -- 172.21.15.1:0/106268631 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1400c850 con 0x55d45fe9b060 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.294+0000 7fac24e03740 1 -- 172.21.15.1:0/106268631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 msgr2=0x55d45fe9b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.294+0000 7fac20fc7640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fac040a3240 0x7fac040a5700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fac1400afd0 tx=0x7fac1400cf70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.294+0000 7fac24e03740 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45fe9b440 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fac1400af20 tx=0x7fac1400f8f0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 -- 172.21.15.1:0/106268631 shutdown_connections 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fea8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b980 0x55d45fea5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 --2- 172.21.15.1:0/106268631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45fe9b440 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 -- 172.21.15.1:0/106268631 >> 172.21.15.1:0/106268631 conn(0x55d45fe975f0 msgr2=0x55d45fe979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 -- 172.21.15.1:0/106268631 shutdown_connections 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 -- 172.21.15.1:0/106268631 wait complete. 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 -- start start 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 Processor -- start 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b060 0x55d45fea3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fea3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.295+0000 7fac24e03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fea3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac24e03740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d45fea8e10 con 0x55d45fe9b980 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac24e03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d45fd88f10 con 0x55d45fea6240 2024-09-16T00:34:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac24e03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d45fea8f90 con 0x55d45fe9b060 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac24e03740 10 client.0 _ll_register_callbacks cb 0x55d45fdacdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fea3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b060 0x55d45fea3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fea3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50468/0 (socket says 172.21.15.1:50468) 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1b7fe640 1 -- 172.21.15.1:0/1309807990 learned_addr learned my addr 172.21.15.1:0/1309807990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac20fc7640 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fea3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1b7fe640 1 -- 172.21.15.1:0/1309807990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b060 msgr2=0x55d45fea3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b060 0x55d45fea3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1b7fe640 1 -- 172.21.15.1:0/1309807990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 msgr2=0x55d45fea3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fea3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1b7fe640 1 -- 172.21.15.1:0/1309807990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45fb0f580 con 0x55d45fe9b980 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fea3680 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fac100c4950 tx=0x7fac10009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e4bd4002f7d020 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:40.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fabfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fabfb7fe640 1 -- 172.21.15.1:0/1309807990 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10049030 con 0x55d45fe9b980 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fabfb7fe640 1 -- 172.21.15.1:0/1309807990 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fac100ed740 con 0x55d45fe9b980 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.296+0000 7fac24e03740 1 -- 172.21.15.1:0/1309807990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1400dc90 con 0x55d45fe9b980 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fabfb7fe640 1 -- 172.21.15.1:0/1309807990 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac100bbb70 con 0x55d45fe9b980 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fac24e03740 10 client.16968 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fac24e03740 1 -- 172.21.15.1:0/1309807990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d45fbd9960 con 0x55d45fe9b980 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fabfb7fe640 1 -- 172.21.15.1:0/1309807990 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1400dc90 con 0x55d45fe9b980 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fabfb7fe640 1 -- 172.21.15.1:0/1309807990 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fac1004c060 con 0x55d45fe9b980 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fac24e03740 10 client.16968 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fac24e03740 10 client.16968 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fac24e03740 20 client.16968 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fac24e03740 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d45feba070 0x55d45febc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fac24e03740 4 client.16968 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:40.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fac24e03740 1 -- 172.21.15.1:0/1309807990 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d45fe0db90 con 0x55d45feba070 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.297+0000 7fac1bfff640 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d45feba070 0x55d45febc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.298+0000 7fabfb7fe640 10 client.16968 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.298+0000 7fac1bfff640 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d45feba070 0x55d45febc480 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74c1785ec0b27f1 server_cookie=30ac6d86cde47340 in_seq=0 out_seq=0 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.298+0000 7fabfb7fe640 1 -- 172.21.15.1:0/1309807990 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d45fe0db90 con 0x55d45feba070 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.298+0000 7fabfb7fe640 10 client.16968 handle_command_reply: tid=0 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.298+0000 7fac24e03740 1 client.16968 shutdown 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 -- 172.21.15.1:0/1309807990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 msgr2=0x55d45fea3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fea3680 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fac100c4950 tx=0x7fac10009620 comp rx=0 tx=0).stop 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 -- 172.21.15.1:0/1309807990 shutdown_connections 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d45feba070 0x55d45febc480 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fea3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fea3680 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 --2- 172.21.15.1:0/1309807990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b060 0x55d45fea3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 -- 172.21.15.1:0/1309807990 >> 172.21.15.1:0/1309807990 conn(0x55d45fe975f0 msgr2=0x55d45fe9d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:40.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fabfb7fe640 0 client.16968 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:40.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 -- 172.21.15.1:0/1309807990 shutdown_connections 2024-09-16T00:34:40.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 1 -- 172.21.15.1:0/1309807990 wait complete. 2024-09-16T00:34:40.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.299+0000 7fac24e03740 20 client.16968 trim_cache size 0 max 16384 2024-09-16T00:34:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac24e03740 1 Processor -- start 2024-09-16T00:34:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac24e03740 1 -- start start 2024-09-16T00:34:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac24e03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b060 0x55d45fe9dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac24e03740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fec1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac24e03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fec2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac24e03740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1af570 con 0x55d45fe9b980 2024-09-16T00:34:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac24e03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1ace40 con 0x55d45fea6240 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac24e03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1aac90 con 0x55d45fe9b060 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fec1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fec1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50484/0 (socket says 172.21.15.1:50484) 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac1b7fe640 1 -- 172.21.15.1:0/3836196544 learned_addr learned my addr 172.21.15.1:0/3836196544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac1b7fe640 1 -- 172.21.15.1:0/3836196544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b060 msgr2=0x55d45fe9dac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/3836196544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b060 0x55d45fe9dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac1b7fe640 1 -- 172.21.15.1:0/3836196544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 msgr2=0x55d45fec2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/3836196544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fec2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.316+0000 7fac1b7fe640 1 -- 172.21.15.1:0/3836196544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d45fea1690 con 0x55d45fe9b980 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/3836196544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fec1d50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac1003b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d54b5261330d96f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac01ffb640 1 -- 172.21.15.1:0/3836196544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10049070 con 0x55d45fe9b980 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac01ffb640 1 -- 172.21.15.1:0/3836196544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac100c4e80 con 0x55d45fe9b980 2024-09-16T00:34:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac01ffb640 1 -- 172.21.15.1:0/3836196544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac100c49f0 con 0x55d45fe9b980 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac24e03740 1 -- 172.21.15.1:0/3836196544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 msgr2=0x55d45fec1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac24e03740 1 --2- 172.21.15.1:0/3836196544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fec1d50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac1003b5d0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac24e03740 1 -- 172.21.15.1:0/3836196544 shutdown_connections 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac24e03740 1 --2- 172.21.15.1:0/3836196544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fea6240 0x55d45fec2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac24e03740 1 --2- 172.21.15.1:0/3836196544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b980 0x55d45fec1d50 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac24e03740 1 --2- 172.21.15.1:0/3836196544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fe9b060 0x55d45fe9dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.317+0000 7fac24e03740 1 -- 172.21.15.1:0/3836196544 >> 172.21.15.1:0/3836196544 conn(0x55d45fe975f0 msgr2=0x55d45fea49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 -- 172.21.15.1:0/3836196544 shutdown_connections 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 -- 172.21.15.1:0/3836196544 wait complete. 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 -- start start 2024-09-16T00:34:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 Processor -- start 2024-09-16T00:34:40.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45feae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fe9b980 0x55d45feae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fea6240 0x55d45feab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1ab300 con 0x55d45fe9b060 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1ac770 con 0x55d45fe9b980 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1ab9d0 con 0x55d45fea6240 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac24e03740 10 client.0 _ll_register_callbacks cb 0x7fac10003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.318+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45feae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45feae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50492/0 (socket says 172.21.15.1:50492) 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac1bfff640 1 -- 172.21.15.1:0/641052069 learned_addr learned my addr 172.21.15.1:0/641052069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fe9b980 0x55d45feae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac20fc7640 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fea6240 0x55d45feab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac1bfff640 1 -- 172.21.15.1:0/641052069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fea6240 msgr2=0x55d45feab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac1bfff640 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fea6240 0x55d45feab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac1bfff640 1 -- 172.21.15.1:0/641052069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fe9b980 msgr2=0x55d45feae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac1bfff640 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fe9b980 0x55d45feae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac1bfff640 1 -- 172.21.15.1:0/641052069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45feab6f0 con 0x55d45fe9b060 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac1bfff640 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45feae1e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00bf60 tx=0x7fac0c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc5dd2fbb666d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fabfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fabfb7fe640 1 -- 172.21.15.1:0/641052069 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c004440 con 0x55d45fe9b060 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fabfb7fe640 1 -- 172.21.15.1:0/641052069 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fac0c0035f0 con 0x55d45fe9b060 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac24e03740 1 -- 172.21.15.1:0/641052069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac100c4e80 con 0x55d45fe9b060 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fabfb7fe640 1 -- 172.21.15.1:0/641052069 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac0c029050 con 0x55d45fe9b060 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac24e03740 10 client.16974 fetch_fsmap learned FSMap version 25 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.319+0000 7fac24e03740 1 -- 172.21.15.1:0/641052069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d45ff255f0 con 0x55d45fe9b060 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.320+0000 7fabfb7fe640 1 -- 172.21.15.1:0/641052069 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac100c4e80 con 0x55d45fe9b060 2024-09-16T00:34:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.320+0000 7fabfb7fe640 1 -- 172.21.15.1:0/641052069 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fac0c00f040 con 0x55d45fe9b060 2024-09-16T00:34:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.320+0000 7fac24e03740 10 client.16974 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:34:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.320+0000 7fac24e03740 10 client.16974 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:34:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.320+0000 7fac24e03740 20 client.16974 populate_metadata read hostname 'smithi001' 2024-09-16T00:34:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.320+0000 7fac24e03740 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d45ff25d90 0x55d45ff281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.320+0000 7fac24e03740 4 client.16974 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "183573db", "release": true, "await": true}] 2024-09-16T00:34:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.320+0000 7fac24e03740 1 -- 172.21.15.1:0/641052069 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "183573db", "release": true, "await": true}) -- 0x55d45fe080a0 con 0x55d45ff25d90 2024-09-16T00:34:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.320+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d45ff25d90 0x55d45ff281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.321+0000 7fac1b7fe640 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d45ff25d90 0x55d45ff281b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2c5cb962467c2b server_cookie=ead25903045e06df in_seq=0 out_seq=0 2024-09-16T00:34:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.321+0000 7fabfb7fe640 10 client.16974 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:40.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fabfb7fe640 1 -- 172.21.15.1:0/641052069 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d45fe080a0 con 0x55d45ff25d90 2024-09-16T00:34:40.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fabfb7fe640 10 client.16974 handle_command_reply: tid=0 2024-09-16T00:34:40.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fac24e03740 1 client.16974 shutdown 2024-09-16T00:34:40.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fac24e03740 1 -- 172.21.15.1:0/641052069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 msgr2=0x55d45feae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:40.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fac24e03740 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45feae1e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00bf60 tx=0x7fac0c00bf90 comp rx=0 tx=0).stop 2024-09-16T00:34:40.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fac24e03740 1 -- 172.21.15.1:0/641052069 shutdown_connections 2024-09-16T00:34:40.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fac24e03740 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55d45ff25d90 0x55d45ff281b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fac24e03740 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d45fea6240 0x55d45feab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fac24e03740 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d45fe9b980 0x55d45feae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fac24e03740 1 --2- 172.21.15.1:0/641052069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d45fe9b060 0x55d45feae1e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fac24e03740 1 -- 172.21.15.1:0/641052069 >> 172.21.15.1:0/641052069 conn(0x55d45fe975f0 msgr2=0x55d45fe95f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:40.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.323+0000 7fabfb7fe640 0 client.16974 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:34:40.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac24e03740 1 -- 172.21.15.1:0/641052069 shutdown_connections 2024-09-16T00:34:40.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac24e03740 1 -- 172.21.15.1:0/641052069 wait complete. 2024-09-16T00:34:40.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac24e03740 20 client.16974 trim_cache size 0 max 16384 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 24, 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "183573db": { 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 24, 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:34:40.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 145.3, 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:40.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:40.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:40.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:40.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:34:40.330 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:34:40.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 -- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fac040a3240 msgr2=0x7fac040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:40.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fac040a3240 0x7fac040a5700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fac1400afd0 tx=0x7fac1400cf70 comp rx=0 tx=0).stop 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 -- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 msgr2=0x7fac1c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 0x7fac1c1989d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fac10006010 tx=0x7fac100042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 -- 172.21.15.1:0/3308386292 shutdown_connections 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fac040a3240 0x7fac040a5700 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac1c1055b0 0x7fac1c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac1c100e30 0x7fac1c1989d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 --2- 172.21.15.1:0/3308386292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c100480 0x7fac1c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 -- 172.21.15.1:0/3308386292 >> 172.21.15.1:0/3308386292 conn(0x7fac1c077210 msgr2=0x7fac1c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 -- 172.21.15.1:0/3308386292 shutdown_connections 2024-09-16T00:34:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:34:40.324+0000 7fac22a2a640 1 -- 172.21.15.1:0/3308386292 wait complete. 2024-09-16T00:34:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:39 smithi019 ceph-mon[35186]: pgmap v538: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-16T00:34:41.189 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '183573db', total seconds elapsed: 28.8 2024-09-16T00:34:42.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:41 smithi001 ceph-mon[30934]: pgmap v539: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:34:42.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:41 smithi153 ceph-mon[35538]: pgmap v539: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:34:42.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:41 smithi019 ceph-mon[35186]: pgmap v539: 129 pgs: 129 active+clean; 2.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:34:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:43 smithi001 ceph-mon[30934]: pgmap v540: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:34:44.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:43 smithi153 ceph-mon[35538]: pgmap v540: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:34:44.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:43 smithi019 ceph-mon[35186]: pgmap v540: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:34:46.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:45 smithi001 ceph-mon[30934]: pgmap v541: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-16T00:34:46.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:45 smithi153 ceph-mon[35538]: pgmap v541: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-16T00:34:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:45 smithi019 ceph-mon[35186]: pgmap v541: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-16T00:34:48.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:47 smithi001 ceph-mon[30934]: pgmap v542: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:34:48.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:47 smithi153 ceph-mon[35538]: pgmap v542: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:34:48.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:47 smithi019 ceph-mon[35186]: pgmap v542: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:34:50.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:49 smithi001 ceph-mon[30934]: pgmap v543: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-09-16T00:34:50.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:49 smithi153 ceph-mon[35538]: pgmap v543: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-09-16T00:34:50.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:49 smithi019 ceph-mon[35186]: pgmap v543: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-09-16T00:34:52.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:51 smithi001 ceph-mon[30934]: pgmap v544: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:34:52.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:51 smithi153 ceph-mon[35538]: pgmap v544: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:34:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:51 smithi019 ceph-mon[35186]: pgmap v544: 129 pgs: 129 active+clean; 2.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:34:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:53 smithi001 ceph-mon[30934]: pgmap v545: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:34:54.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:53 smithi153 ceph-mon[35538]: pgmap v545: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:34:54.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:53 smithi019 ceph-mon[35186]: pgmap v545: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:34:55.432 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.432 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:127/600: open fd = 3 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:128/600: open fd = 3 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:129/600: open fd = 3 2024-09-16T00:34:55.433 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:130/600: open fd = 3 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:131/600: open fd = 3 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:132/600: open fd = 3 2024-09-16T00:34:55.434 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:133/600: open fd = 3 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:134/600: open fd = 3 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:135/600: open fd = 3 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.435 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:136/600: open fd = 3 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:137/600: open fd = 3 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:138/600: open fd = 3 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:139/600: open fd = 3 2024-09-16T00:34:55.436 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:140/600: open fd = 3 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:141/600: open fd = 3 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:142/600: open fd = 3 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.437 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:143/600: open fd = 3 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:144/600: open fd = 3 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:145/600: open fd = 3 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:146/600: open fd = 3 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.438 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:147/600: open fd = 3 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:148/600: open fd = 3 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:149/600: open fd = 3 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:150/600: open fd = 3 2024-09-16T00:34:55.439 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:151/600: open fd = 3 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:152/600: open fd = 3 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.440 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.441 INFO:tasks.workunit.client.0.smithi001.stdout:153/600: open fd = 3 2024-09-16T00:34:55.441 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.441 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.441 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.441 INFO:tasks.workunit.client.0.smithi001.stdout:154/600: open fd = 3 2024-09-16T00:34:55.441 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.441 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.441 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:155/600: open fd = 3 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:156/600: open fd = 3 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:157/600: open fd = 3 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:158/600: open fd = 3 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.442 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:159/600: open fd = 3 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:160/600: open fd = 3 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:161/600: open fd = 3 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.443 INFO:tasks.workunit.client.0.smithi001.stdout:162/600: open fd = 3 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:163/600: open fd = 3 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:164/600: open fd = 3 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:165/600: open fd = 3 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.444 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:166/600: open fd = 3 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:167/600: open fd = 3 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:168/600: open fd = 3 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.445 INFO:tasks.workunit.client.0.smithi001.stdout:169/600: open fd = 3 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:170/600: open fd = 3 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:171/600: open fd = 3 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:172/600: open fd = 3 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.446 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:173/600: open fd = 3 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:174/600: open fd = 3 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:175/600: open fd = 3 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.447 INFO:tasks.workunit.client.0.smithi001.stdout:176/600: open fd = 3 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:177/600: open fd = 3 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:178/600: open fd = 3 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:179/600: open fd = 3 2024-09-16T00:34:55.448 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:180/600: open fd = 3 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:181/600: open fd = 3 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:182/600: open fd = 3 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.449 INFO:tasks.workunit.client.0.smithi001.stdout:183/600: open fd = 3 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:184/600: open fd = 3 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:185/600: open fd = 3 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:186/600: open fd = 3 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.450 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.451 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.451 INFO:tasks.workunit.client.0.smithi001.stdout:187/600: open fd = 3 2024-09-16T00:34:55.451 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.451 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.451 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:34:55.451 INFO:tasks.workunit.client.0.smithi001.stdout:188/600: open fd = 3 2024-09-16T00:34:55.451 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:55.451 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:34:56.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:55 smithi001 ceph-mon[30934]: pgmap v546: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:34:56.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:55 smithi153 ceph-mon[35538]: pgmap v546: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:34:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:55 smithi019 ceph-mon[35186]: pgmap v546: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:34:58.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:57 smithi001 ceph-mon[30934]: pgmap v547: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:34:58.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:57 smithi153 ceph-mon[35538]: pgmap v547: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:34:58.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:57 smithi019 ceph-mon[35186]: pgmap v547: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:35:00.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:34:59 smithi001 ceph-mon[30934]: pgmap v548: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:00.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:34:59 smithi153 ceph-mon[35538]: pgmap v548: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:34:59 smithi019 ceph-mon[35186]: pgmap v548: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:02.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:01 smithi001 ceph-mon[30934]: pgmap v549: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:35:02.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:01 smithi153 ceph-mon[35538]: pgmap v549: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:35:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:01 smithi019 ceph-mon[35186]: pgmap v549: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:35:04.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:03 smithi001 ceph-mon[30934]: pgmap v550: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:35:04.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:03 smithi153 ceph-mon[35538]: pgmap v550: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:35:04.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:03 smithi019 ceph-mon[35186]: pgmap v550: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:35:05.398 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:35:05.674 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:06.109 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:05 smithi001 ceph-mon[30934]: pgmap v551: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:06.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:05 smithi153 ceph-mon[35538]: pgmap v551: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:06.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.193+0000 7fd67dcc8640 1 -- 172.21.15.1:0/3168687230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101420 msgr2=0x7fd67810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:06.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.193+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/3168687230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101420 0x7fd67810d950 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd670009cd0 tx=0x7fd67002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.193+0000 7fd67dcc8640 1 -- 172.21.15.1:0/3168687230 shutdown_connections 2024-09-16T00:35:06.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.193+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/3168687230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678101420 0x7fd67810d950 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.193+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/3168687230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678100a80 0x7fd678100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.193+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/3168687230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678106a80 0x7fd678106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.193+0000 7fd67dcc8640 1 -- 172.21.15.1:0/3168687230 >> 172.21.15.1:0/3168687230 conn(0x7fd6780fc7a0 msgr2=0x7fd6780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:06.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.193+0000 7fd67dcc8640 1 -- 172.21.15.1:0/3168687230 shutdown_connections 2024-09-16T00:35:06.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.193+0000 7fd67dcc8640 1 -- 172.21.15.1:0/3168687230 wait complete. 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd67dcc8640 1 Processor -- start 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd67dcc8640 1 -- start start 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd67dcc8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678100a80 0x7fd678198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd67dcc8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678101420 0x7fd678198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd67dcc8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678106a80 0x7fd67819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd67dcc8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd678110ed0 con 0x7fd678100a80 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd67dcc8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd678110d50 con 0x7fd678106a80 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd67dcc8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd678111050 con 0x7fd678101420 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678100a80 0x7fd678198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678100a80 0x7fd678198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59370/0 (socket says 172.21.15.1:59370) 2024-09-16T00:35:06.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd6777fe640 1 -- 172.21.15.1:0/2913760440 learned_addr learned my addr 172.21.15.1:0/2913760440 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.194+0000 7fd677fff640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678106a80 0x7fd67819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd6777fe640 1 -- 172.21.15.1:0/2913760440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678101420 msgr2=0x7fd678198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd6777fe640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678101420 0x7fd678198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd6777fe640 1 -- 172.21.15.1:0/2913760440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678106a80 msgr2=0x7fd67819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd6777fe640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678106a80 0x7fd67819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd6777fe640 1 -- 172.21.15.1:0/2913760440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67819d3d0 con 0x7fd678100a80 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd677fff640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678106a80 0x7fd67819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd6777fe640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678100a80 0x7fd678198420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd66800c890 tx=0x7fd66800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd674ff9640 1 -- 172.21.15.1:0/2913760440 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668004220 con 0x7fd678100a80 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd674ff9640 1 -- 172.21.15.1:0/2913760440 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6680043c0 con 0x7fd678100a80 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd67dcc8640 1 -- 172.21.15.1:0/2913760440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67819d6c0 con 0x7fd678100a80 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd674ff9640 1 -- 172.21.15.1:0/2913760440 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680049e0 con 0x7fd678100a80 2024-09-16T00:35:06.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.195+0000 7fd67dcc8640 1 -- 172.21.15.1:0/2913760440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6781a4fa0 con 0x7fd678100a80 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.197+0000 7fd674ff9640 1 -- 172.21.15.1:0/2913760440 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd6680079b0 con 0x7fd678100a80 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd6800a1740 1 Processor -- start 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd6800a1740 1 -- start start 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fcc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353fd7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fd7680 0x564353fd9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd6800a1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564353f322e0 con 0x564353fcc4a0 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd6800a1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564353ed9ce0 con 0x564353fccdc0 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd6800a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564353f2e2a0 con 0x564353fd7680 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd674ff9640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3240 0x7fd6600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd677fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fcc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd677fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fcc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59386/0 (socket says 172.21.15.1:59386) 2024-09-16T00:35:06.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd677fff640 1 -- 172.21.15.1:0/2616878083 learned_addr learned my addr 172.21.15.1:0/2616878083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd677fff640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3240 0x7fd6600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd674ff9640 1 -- 172.21.15.1:0/2913760440 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd6680cfe20 con 0x7fd678100a80 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd6777fe640 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353fd7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.198+0000 7fd676ffd640 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fd7680 0x564353fd9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd677fff640 1 -- 172.21.15.1:0/2616878083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fd7680 msgr2=0x564353fd9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd677fff640 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fd7680 0x564353fd9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd677fff640 1 -- 172.21.15.1:0/2616878083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 msgr2=0x564353fd7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd677fff640 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353fd7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd677fff640 1 -- 172.21.15.1:0/2616878083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564353ebe440 con 0x564353fcc4a0 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd676ffd640 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fd7680 0x564353fd9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd677fff640 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fcc880 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd670009cd0 tx=0x7fd670031e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fbbbbf3fd9ab4c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd677fff640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3240 0x7fd6600a5700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd670035000 tx=0x7fd670048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd65dffb640 1 -- 172.21.15.1:0/2616878083 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670004100 con 0x564353fcc4a0 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd65dffb640 1 -- 172.21.15.1:0/2616878083 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd670036040 con 0x564353fcc4a0 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.199+0000 7fd65dffb640 1 -- 172.21.15.1:0/2616878083 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700309c0 con 0x564353fcc4a0 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 -- 172.21.15.1:0/2616878083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 msgr2=0x564353fcc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fcc880 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd670009cd0 tx=0x7fd670031e00 comp rx=0 tx=0).stop 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 -- 172.21.15.1:0/2616878083 shutdown_connections 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fd7680 0x564353fd9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353fd7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 --2- 172.21.15.1:0/2616878083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fcc880 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 -- 172.21.15.1:0/2616878083 >> 172.21.15.1:0/2616878083 conn(0x564353fc8a30 msgr2=0x564353fc8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 -- 172.21.15.1:0/2616878083 shutdown_connections 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 -- 172.21.15.1:0/2616878083 wait complete. 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 -- start start 2024-09-16T00:35:06.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.200+0000 7fd6800a1740 1 Processor -- start 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 0x564353fd7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353fd42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fd4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd6800a1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564353fda250 con 0x564353fd7680 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd6800a1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564353eed050 con 0x564353fccdc0 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd6800a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564353fda3d0 con 0x564353fcc4a0 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd6800a1740 10 client.0 _ll_register_callbacks cb 0x564353f2d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd677fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fd4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd677fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fd4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59400/0 (socket says 172.21.15.1:59400) 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd677fff640 1 -- 172.21.15.1:0/1061463978 learned_addr learned my addr 172.21.15.1:0/1061463978 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd676ffd640 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353fd42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd6777fe640 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 0x564353fd7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd677fff640 1 -- 172.21.15.1:0/1061463978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 msgr2=0x564353fd7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd677fff640 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 0x564353fd7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd677fff640 1 -- 172.21.15.1:0/1061463978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 msgr2=0x564353fd42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd677fff640 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353fd42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd677fff640 1 -- 172.21.15.1:0/1061463978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564353ca33c0 con 0x564353fd7680 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd677fff640 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fd4840 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd67002a920 tx=0x7fd670004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f667f2f66b642da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.201+0000 7fd656ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd656ffd640 1 -- 172.21.15.1:0/1061463978 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670056020 con 0x564353fd7680 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd656ffd640 1 -- 172.21.15.1:0/1061463978 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd670030940 con 0x564353fd7680 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6800a1740 1 -- 172.21.15.1:0/1061463978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd670036040 con 0x564353fd7680 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd656ffd640 1 -- 172.21.15.1:0/1061463978 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd670004730 con 0x564353fd7680 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6800a1740 10 client.16986 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6800a1740 1 -- 172.21.15.1:0/1061463978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564353fe5c40 con 0x564353fd7680 2024-09-16T00:35:06.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd656ffd640 1 -- 172.21.15.1:0/1061463978 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd670036040 con 0x564353fd7680 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd656ffd640 1 -- 172.21.15.1:0/1061463978 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd670030070 con 0x564353fd7680 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6800a1740 10 client.16986 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6800a1740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6800a1740 20 client.16986 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6800a1740 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564353fe6f90 0x564353fe9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6800a1740 4 client.16986 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6800a1740 1 -- 172.21.15.1:0/1061463978 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564353f3f850 con 0x564353fe6f90 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.202+0000 7fd6777fe640 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564353fe6f90 0x564353fe9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.203+0000 7fd6777fe640 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564353fe6f90 0x564353fe9380 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b917773602cbe2a8 server_cookie=26a166cf42444fc2 in_seq=0 out_seq=0 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.203+0000 7fd656ffd640 10 client.16986 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd656ffd640 1 -- 172.21.15.1:0/1061463978 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564353f3f850 con 0x564353fe6f90 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd656ffd640 10 client.16986 handle_command_reply: tid=0 2024-09-16T00:35:06.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 client.16986 shutdown 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 -- 172.21.15.1:0/1061463978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 msgr2=0x564353fd4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fd4840 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd67002a920 tx=0x7fd670004330 comp rx=0 tx=0).stop 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 -- 172.21.15.1:0/1061463978 shutdown_connections 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564353fe6f90 0x564353fe9380 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fd4840 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353fd42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd656ffd640 0 client.16986 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 --2- 172.21.15.1:0/1061463978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 0x564353fd7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 -- 172.21.15.1:0/1061463978 >> 172.21.15.1:0/1061463978 conn(0x564353fc8a30 msgr2=0x564353fce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 -- 172.21.15.1:0/1061463978 shutdown_connections 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 1 -- 172.21.15.1:0/1061463978 wait complete. 2024-09-16T00:35:06.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.204+0000 7fd6800a1740 20 client.16986 trim_cache size 0 max 16384 2024-09-16T00:35:06.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.221+0000 7fd6800a1740 1 Processor -- start 2024-09-16T00:35:06.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.221+0000 7fd6800a1740 1 -- start start 2024-09-16T00:35:06.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fccdc0 0x564353fec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fd7680 0x564353fd1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6800a1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781b1840 con 0x564353fcc4a0 2024-09-16T00:35:06.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6800a1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781af110 con 0x564353fd7680 2024-09-16T00:35:06.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6800a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781acf60 con 0x564353fccdc0 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fec3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59402/0 (socket says 172.21.15.1:59402) 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6777fe640 1 -- 172.21.15.1:0/3401401548 learned_addr learned my addr 172.21.15.1:0/3401401548 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd677fff640 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fd7680 0x564353fd1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd676ffd640 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fccdc0 0x564353fec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6777fe640 1 -- 172.21.15.1:0/3401401548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fccdc0 msgr2=0x564353fec8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6777fe640 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fccdc0 0x564353fec8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6777fe640 1 -- 172.21.15.1:0/3401401548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fd7680 msgr2=0x564353fd1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6777fe640 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fd7680 0x564353fd1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6777fe640 1 -- 172.21.15.1:0/3401401548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564353fc9ae0 con 0x564353fcc4a0 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd676ffd640 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fccdc0 0x564353fec8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:06.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd677fff640 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fd7680 0x564353fd1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd6777fe640 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fec3b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd668005fd0 tx=0x7fd66801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bf94d6ca8d7c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd65d7fa640 1 -- 172.21.15.1:0/3401401548 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66801d040 con 0x564353fcc4a0 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd65d7fa640 1 -- 172.21.15.1:0/3401401548 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6680bc0c0 con 0x564353fcc4a0 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.222+0000 7fd65d7fa640 1 -- 172.21.15.1:0/3401401548 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680987e0 con 0x564353fcc4a0 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 -- 172.21.15.1:0/3401401548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 msgr2=0x564353fec3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fec3b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd668005fd0 tx=0x7fd66801a040 comp rx=0 tx=0).stop 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 -- 172.21.15.1:0/3401401548 shutdown_connections 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fd7680 0x564353fd1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fccdc0 0x564353fec8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3401401548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fcc4a0 0x564353fec3b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 -- 172.21.15.1:0/3401401548 >> 172.21.15.1:0/3401401548 conn(0x564353fc8a30 msgr2=0x564353fd6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 -- 172.21.15.1:0/3401401548 shutdown_connections 2024-09-16T00:35:06.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 -- 172.21.15.1:0/3401401548 wait complete. 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.223+0000 7fd6800a1740 1 -- start start 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd6800a1740 1 Processor -- start 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 0x56435404b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353febf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd6800a1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd6800a1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781ad5d0 con 0x564353fd7680 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd6800a1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781aea40 con 0x564353fccdc0 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd6800a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781adca0 con 0x564353fcc4a0 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd6800a1740 10 client.0 _ll_register_callbacks cb 0x7fd668004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd677fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd677fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fec700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59408/0 (socket says 172.21.15.1:59408) 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd677fff640 1 -- 172.21.15.1:0/3224531794 learned_addr learned my addr 172.21.15.1:0/3224531794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd676ffd640 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353febf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd6777fe640 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 0x56435404b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd677fff640 1 -- 172.21.15.1:0/3224531794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 msgr2=0x56435404b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd677fff640 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 0x56435404b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd677fff640 1 -- 172.21.15.1:0/3224531794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 msgr2=0x564353febf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd677fff640 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353febf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd677fff640 1 -- 172.21.15.1:0/3224531794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564354048dc0 con 0x564353fd7680 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.224+0000 7fd676ffd640 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353febf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd677fff640 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fec700 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd67004abc0 tx=0x7fd6700048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfc5c0b0e0f4de5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd656ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd656ffd640 1 -- 172.21.15.1:0/3224531794 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670056030 con 0x564353fd7680 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd6800a1740 1 -- 172.21.15.1:0/3224531794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564353f39d60 con 0x564353fd7680 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd656ffd640 1 -- 172.21.15.1:0/3224531794 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd6700307b0 con 0x564353fd7680 2024-09-16T00:35:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd656ffd640 1 -- 172.21.15.1:0/3224531794 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd67006a050 con 0x564353fd7680 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd6800a1740 10 client.16992 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd6800a1740 1 -- 172.21.15.1:0/3224531794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564354055df0 con 0x564353fd7680 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd656ffd640 1 -- 172.21.15.1:0/3224531794 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564353f39d60 con 0x564353fd7680 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd656ffd640 1 -- 172.21.15.1:0/3224531794 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd67003c410 con 0x564353fd7680 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd6800a1740 10 client.16992 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd6800a1740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd6800a1740 20 client.16992 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.225+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564354056490 0x5643540588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd6800a1740 4 client.16992 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd6800a1740 1 -- 172.21.15.1:0/3224531794 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564353eb6210 con 0x564354056490 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd6777fe640 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564354056490 0x5643540588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd656ffd640 10 client.16992 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd6777fe640 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564354056490 0x5643540588b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=749404abf12f1d8b server_cookie=8bc46252e5173fe7 in_seq=0 out_seq=0 2024-09-16T00:35:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd656ffd640 1 -- 172.21.15.1:0/3224531794 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564353eb6210 con 0x564354056490 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd656ffd640 10 client.16992 handle_command_reply: tid=0 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd6800a1740 1 client.16992 shutdown 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd6800a1740 1 -- 172.21.15.1:0/3224531794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 msgr2=0x564353fec700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.226+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fec700 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd67004abc0 tx=0x7fd6700048d0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd6800a1740 1 -- 172.21.15.1:0/3224531794 shutdown_connections 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564354056490 0x5643540588b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564353fd7680 0x564353fec700 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564353fccdc0 0x564353febf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd6800a1740 1 --2- 172.21.15.1:0/3224531794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564353fcc4a0 0x56435404b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd656ffd640 0 client.16992 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd6800a1740 1 -- 172.21.15.1:0/3224531794 >> 172.21.15.1:0/3224531794 conn(0x564353fc8a30 msgr2=0x564353fc7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd6800a1740 1 -- 172.21.15.1:0/3224531794 shutdown_connections 2024-09-16T00:35:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd6800a1740 1 -- 172.21.15.1:0/3224531794 wait complete. 2024-09-16T00:35:06.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd6800a1740 20 client.16992 trim_cache size 0 max 16384 2024-09-16T00:35:06.239 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:35:06.239 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:35:06.239 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:35:06.239 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:35:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 -- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3240 msgr2=0x7fd6600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3240 0x7fd6600a5700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd670035000 tx=0x7fd670048000 comp rx=0 tx=0).stop 2024-09-16T00:35:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 -- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678100a80 msgr2=0x7fd678198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678100a80 0x7fd678198420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd66800c890 tx=0x7fd66800cd50 comp rx=0 tx=0).stop 2024-09-16T00:35:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 -- 172.21.15.1:0/2913760440 shutdown_connections 2024-09-16T00:35:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd678106a80 0x7fd67819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6600a3240 0x7fd6600a5700 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd678101420 0x7fd678198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 --2- 172.21.15.1:0/2913760440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd678100a80 0x7fd678198420 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:06.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.227+0000 7fd67dcc8640 1 -- 172.21.15.1:0/2913760440 >> 172.21.15.1:0/2913760440 conn(0x7fd6780fc7a0 msgr2=0x7fd6781047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:06.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.228+0000 7fd67dcc8640 1 -- 172.21.15.1:0/2913760440 shutdown_connections 2024-09-16T00:35:06.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:06.228+0000 7fd67dcc8640 1 -- 172.21.15.1:0/2913760440 wait complete. 2024-09-16T00:35:06.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:05 smithi019 ceph-mon[35186]: pgmap v551: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:07.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:35:07.128 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:35:07.396 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:07.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.910+0000 7faa64b81640 1 -- 172.21.15.1:0/791096703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa60102d30 msgr2=0x7faa60103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.910+0000 7faa64b81640 1 --2- 172.21.15.1:0/791096703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa60102d30 0x7faa60103110 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7faa50009930 tx=0x7faa5002a420 comp rx=0 tx=0).stop 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.910+0000 7faa64b81640 1 -- 172.21.15.1:0/791096703 shutdown_connections 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.910+0000 7faa64b81640 1 --2- 172.21.15.1:0/791096703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa601082a0 0x7faa6010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.910+0000 7faa64b81640 1 --2- 172.21.15.1:0/791096703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa601036e0 0x7faa60107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.910+0000 7faa64b81640 1 --2- 172.21.15.1:0/791096703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa60102d30 0x7faa60103110 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.910+0000 7faa64b81640 1 -- 172.21.15.1:0/791096703 >> 172.21.15.1:0/791096703 conn(0x7faa600fc7d0 msgr2=0x7faa600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.910+0000 7faa64b81640 1 -- 172.21.15.1:0/791096703 shutdown_connections 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.910+0000 7faa64b81640 1 -- 172.21.15.1:0/791096703 wait complete. 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa64b81640 1 Processor -- start 2024-09-16T00:35:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa64b81640 1 -- start start 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa64b81640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa60102d30 0x7faa60100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa64b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa601036e0 0x7faa60100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa64b81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa601082a0 0x7faa60101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa64b81640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faa601120a0 con 0x7faa601082a0 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa64b81640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa60111f20 con 0x7faa601036e0 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa64b81640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faa60112220 con 0x7faa60102d30 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa5ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa601082a0 0x7faa60101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa5ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa601082a0 0x7faa60101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59424/0 (socket says 172.21.15.1:59424) 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa5ed9d640 1 -- 172.21.15.1:0/1340269414 learned_addr learned my addr 172.21.15.1:0/1340269414 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa5e59c640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa60102d30 0x7faa60100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa57fff640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa601036e0 0x7faa60100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa5ed9d640 1 -- 172.21.15.1:0/1340269414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa60102d30 msgr2=0x7faa60100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa5ed9d640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa60102d30 0x7faa60100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa5ed9d640 1 -- 172.21.15.1:0/1340269414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa601036e0 msgr2=0x7faa60100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa5ed9d640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa601036e0 0x7faa60100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa5ed9d640 1 -- 172.21.15.1:0/1340269414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa601a6a60 con 0x7faa601082a0 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.911+0000 7faa57fff640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa601036e0 0x7faa60100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.912+0000 7faa5ed9d640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa601082a0 0x7faa60101410 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7faa5800eb10 tx=0x7faa5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.912+0000 7faa577fe640 1 -- 172.21.15.1:0/1340269414 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58004b20 con 0x7faa601082a0 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.912+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa601a6d50 con 0x7faa601082a0 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.912+0000 7faa577fe640 1 -- 172.21.15.1:0/1340269414 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa58004cc0 con 0x7faa601082a0 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.912+0000 7faa577fe640 1 -- 172.21.15.1:0/1340269414 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58010760 con 0x7faa601082a0 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.912+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa601a7240 con 0x7faa601082a0 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.913+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6006a850 con 0x7faa601082a0 2024-09-16T00:35:07.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.914+0000 7faa577fe640 1 -- 172.21.15.1:0/1340269414 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7faa58005020 con 0x7faa601082a0 2024-09-16T00:35:07.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.915+0000 7faa577fe640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa4c0a3240 0x7faa4c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:07.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.915+0000 7faa5e59c640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa4c0a3240 0x7faa4c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:07.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.915+0000 7faa577fe640 1 -- 172.21.15.1:0/1340269414 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7faa580d03f0 con 0x7faa601082a0 2024-09-16T00:35:07.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.915+0000 7faa5e59c640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa4c0a3240 0x7faa4c0a5700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faa50000c00 tx=0x7faa50005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:07.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:07.919+0000 7faa577fe640 1 -- 172.21.15.1:0/1340269414 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa5809ccb0 con 0x7faa601082a0 2024-09-16T00:35:08.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.130+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa6010a2b0 con 0x7faa601082a0 2024-09-16T00:35:08.132 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:07 smithi001 ceph-mon[30934]: pgmap v552: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:08.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.131+0000 7faa577fe640 1 -- 172.21.15.1:0/1340269414 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7faa5809c5b0 con 0x7faa601082a0 2024-09-16T00:35:08.132 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:35:08.133 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:35:08.133 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:35:08.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa4c0a3240 msgr2=0x7faa4c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa4c0a3240 0x7faa4c0a5700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faa50000c00 tx=0x7faa50005a20 comp rx=0 tx=0).stop 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa601082a0 msgr2=0x7faa60101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa601082a0 0x7faa60101410 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7faa5800eb10 tx=0x7faa5800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 shutdown_connections 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa601082a0 0x7faa60101410 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa601036e0 0x7faa60100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa4c0a3240 0x7faa4c0a5700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 --2- 172.21.15.1:0/1340269414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa60102d30 0x7faa60100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 >> 172.21.15.1:0/1340269414 conn(0x7faa600fc7d0 msgr2=0x7faa60105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 shutdown_connections 2024-09-16T00:35:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:08.133+0000 7faa64b81640 1 -- 172.21.15.1:0/1340269414 wait complete. 2024-09-16T00:35:08.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:07 smithi153 ceph-mon[35538]: pgmap v552: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:08.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:07 smithi019 ceph-mon[35186]: pgmap v552: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:08.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:35:09.115 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:09.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:08 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1340269414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:09.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:08 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1340269414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:09.258 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:08 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1340269414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:09.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.620+0000 7f861b8e1640 1 -- 172.21.15.1:0/3983920512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614101de0 msgr2=0x7f86141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:09.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.620+0000 7f861b8e1640 1 --2- 172.21.15.1:0/3983920512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614101de0 0x7f86141021c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8610009900 tx=0x7f861002a410 comp rx=0 tx=0).stop 2024-09-16T00:35:09.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.621+0000 7f861b8e1640 1 -- 172.21.15.1:0/3983920512 shutdown_connections 2024-09-16T00:35:09.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.621+0000 7f861b8e1640 1 --2- 172.21.15.1:0/3983920512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8614106f10 0x7f861410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.621+0000 7f861b8e1640 1 --2- 172.21.15.1:0/3983920512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8614102790 0x7f86141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.621+0000 7f861b8e1640 1 --2- 172.21.15.1:0/3983920512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614101de0 0x7f86141021c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.621+0000 7f861b8e1640 1 -- 172.21.15.1:0/3983920512 >> 172.21.15.1:0/3983920512 conn(0x7f86140fd8c0 msgr2=0x7f86140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:09.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.621+0000 7f861b8e1640 1 -- 172.21.15.1:0/3983920512 shutdown_connections 2024-09-16T00:35:09.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.621+0000 7f861b8e1640 1 -- 172.21.15.1:0/3983920512 wait complete. 2024-09-16T00:35:09.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f861b8e1640 1 Processor -- start 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f861b8e1640 1 -- start start 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f861b8e1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8614101de0 0x7f86141985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f861b8e1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614102790 0x7f8614198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f861b8e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8614106f10 0x7f861419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f861b8e1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8614112010 con 0x7f8614102790 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f861b8e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8614111e90 con 0x7f8614106f10 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f861b8e1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8614112190 con 0x7f8614101de0 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8618e7c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614102790 0x7f8614198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8618e7c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614102790 0x7f8614198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59428/0 (socket says 172.21.15.1:59428) 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8618e7c640 1 -- 172.21.15.1:0/1674875075 learned_addr learned my addr 172.21.15.1:0/1674875075 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f861967d640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8614101de0 0x7f86141985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8619e7e640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8614106f10 0x7f861419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8618e7c640 1 -- 172.21.15.1:0/1674875075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8614101de0 msgr2=0x7f86141985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8618e7c640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8614101de0 0x7f86141985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8618e7c640 1 -- 172.21.15.1:0/1674875075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8614106f10 msgr2=0x7f861419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8618e7c640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8614106f10 0x7f861419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8618e7c640 1 -- 172.21.15.1:0/1674875075 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861419d560 con 0x7f8614102790 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.622+0000 7f8619e7e640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8614106f10 0x7f861419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.623+0000 7f8618e7c640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614102790 0x7f8614198af0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f860800b4a0 tx=0x7f860800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.623+0000 7f86067fc640 1 -- 172.21.15.1:0/1674875075 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608014070 con 0x7f8614102790 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.623+0000 7f86067fc640 1 -- 172.21.15.1:0/1674875075 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608010af0 con 0x7f8614102790 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.623+0000 7f861b8e1640 1 -- 172.21.15.1:0/1674875075 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861419d850 con 0x7f8614102790 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.623+0000 7f86067fc640 1 -- 172.21.15.1:0/1674875075 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608017d10 con 0x7f8614102790 2024-09-16T00:35:09.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.623+0000 7f861b8e1640 1 -- 172.21.15.1:0/1674875075 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86141a5130 con 0x7f8614102790 2024-09-16T00:35:09.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.626+0000 7f86067fc640 1 -- 172.21.15.1:0/1674875075 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8608017070 con 0x7f8614102790 2024-09-16T00:35:09.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.626+0000 7f861dcba740 1 Processor -- start 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861dcba740 1 -- start start 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861dcba740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861dcba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc7906ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861dcba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79079760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861dcba740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cc78fd22f0 con 0x55cc7906db40 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861dcba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cc78f79cf0 con 0x55cc79077370 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861dcba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cc78fce2b0 con 0x55cc7906c5d0 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f86067fc640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86000a3240 0x7f86000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f8619e7e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f8619e7e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59434/0 (socket says 172.21.15.1:59434) 2024-09-16T00:35:09.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f8619e7e640 1 -- 172.21.15.1:0/2600408477 learned_addr learned my addr 172.21.15.1:0/2600408477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861967d640 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc7906ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f8619e7e640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86000a3240 0x7f86000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f8618e7c640 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79079760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f86067fc640 1 -- 172.21.15.1:0/1674875075 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f86080cf190 con 0x7f8614102790 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861967d640 1 -- 172.21.15.1:0/2600408477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 msgr2=0x55cc79079760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861967d640 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79079760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861967d640 1 -- 172.21.15.1:0/2600408477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 msgr2=0x55cc7906c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861967d640 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.627+0000 7f861967d640 1 -- 172.21.15.1:0/2600408477 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc78f5e3a0 con 0x55cc7906c5d0 2024-09-16T00:35:09.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.628+0000 7f8619e7e640 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:35:09.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.628+0000 7f8619e7e640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86000a3240 0x7f86000a5700 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f860c00ecf0 tx=0x7f860c008220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:09.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.628+0000 7f861967d640 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc7906ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8610009230 tx=0x7f8610031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d9f735c282114df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:09.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.628+0000 7f85ff7fe640 1 -- 172.21.15.1:0/2600408477 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861003f070 con 0x55cc7906c5d0 2024-09-16T00:35:09.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.628+0000 7f85ff7fe640 1 -- 172.21.15.1:0/2600408477 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610031a20 con 0x55cc7906c5d0 2024-09-16T00:35:09.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.628+0000 7f85ff7fe640 1 -- 172.21.15.1:0/2600408477 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610035070 con 0x55cc7906c5d0 2024-09-16T00:35:09.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 -- 172.21.15.1:0/2600408477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 msgr2=0x55cc7906ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:09.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc7906ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8610009230 tx=0x7f8610031630 comp rx=0 tx=0).stop 2024-09-16T00:35:09.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 -- 172.21.15.1:0/2600408477 shutdown_connections 2024-09-16T00:35:09.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79079760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc7906ca10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 --2- 172.21.15.1:0/2600408477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 -- 172.21.15.1:0/2600408477 >> 172.21.15.1:0/2600408477 conn(0x55cc79035310 msgr2=0x55cc79068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 -- 172.21.15.1:0/2600408477 shutdown_connections 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 -- 172.21.15.1:0/2600408477 wait complete. 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 -- start start 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.629+0000 7f861dcba740 1 Processor -- start 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f861dcba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc79076d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f861dcba740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc79073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f861dcba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79074560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f861dcba740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cc7907e420 con 0x55cc7906db40 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f861dcba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cc78f8d060 con 0x55cc79077370 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f861dcba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cc7907e5a0 con 0x55cc7906c5d0 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f861dcba740 10 client.0 _ll_register_callbacks cb 0x55cc78ce08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f8618e7c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc79073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f8618e7c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc79073fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59440/0 (socket says 172.21.15.1:59440) 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f8618e7c640 1 -- 172.21.15.1:0/1745997412 learned_addr learned my addr 172.21.15.1:0/1745997412 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f861967d640 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc79076d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f8619e7e640 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f8618e7c640 1 -- 172.21.15.1:0/1745997412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 msgr2=0x55cc79076d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f8618e7c640 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc79076d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f8618e7c640 1 -- 172.21.15.1:0/1745997412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 msgr2=0x55cc79074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f8618e7c640 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.630+0000 7f8618e7c640 1 -- 172.21.15.1:0/1745997412 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc78d433c0 con 0x55cc7906db40 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f8618e7c640 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc79073fc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f860800b470 tx=0x7f860800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89cc3db758caa18 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f85fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f85fcff9640 1 -- 172.21.15.1:0/1745997412 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860801e040 con 0x55cc7906db40 2024-09-16T00:35:09.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f861dcba740 1 -- 172.21.15.1:0/1745997412 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8610031a20 con 0x55cc7906db40 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f85fcff9640 1 -- 172.21.15.1:0/1745997412 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8608010ca0 con 0x55cc7906db40 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f85fcff9640 1 -- 172.21.15.1:0/1745997412 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f860809f760 con 0x55cc7906db40 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f861dcba740 10 client.17010 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f861dcba740 1 -- 172.21.15.1:0/1745997412 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cc79089f20 con 0x55cc7906db40 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f85fcff9640 1 -- 172.21.15.1:0/1745997412 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610031a20 con 0x55cc7906db40 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f85fcff9640 1 -- 172.21.15.1:0/1745997412 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f860801ecf0 con 0x55cc7906db40 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f861dcba740 10 client.17010 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f861dcba740 10 client.17010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f861dcba740 20 client.17010 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f861dcba740 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cc7908b330 0x55cc7908d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f861dcba740 4 client.17010 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.631+0000 7f861dcba740 1 -- 172.21.15.1:0/1745997412 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc78fdf860 con 0x55cc7908b330 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.632+0000 7f861967d640 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cc7908b330 0x55cc7908d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.632+0000 7f861967d640 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cc7908b330 0x55cc7908d720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb877f89ab719b81 server_cookie=eabe6eb04db3d1cc in_seq=0 out_seq=0 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.632+0000 7f85fcff9640 10 client.17010 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f85fcff9640 1 -- 172.21.15.1:0/1745997412 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc78fdf860 con 0x55cc7908b330 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f85fcff9640 10 client.17010 handle_command_reply: tid=0 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f861dcba740 1 client.17010 shutdown 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f861dcba740 1 -- 172.21.15.1:0/1745997412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 msgr2=0x55cc79073fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f861dcba740 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc79073fc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f860800b470 tx=0x7f860800c9e0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f861dcba740 1 -- 172.21.15.1:0/1745997412 shutdown_connections 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f861dcba740 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cc7908b330 0x55cc7908d720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f85fcff9640 0 client.17010 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f861dcba740 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f861dcba740 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc79073fc0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f861dcba740 1 --2- 172.21.15.1:0/1745997412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc79076d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.633+0000 7f861dcba740 1 -- 172.21.15.1:0/1745997412 >> 172.21.15.1:0/1745997412 conn(0x55cc79035310 msgr2=0x55cc7906e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.634+0000 7f861dcba740 1 -- 172.21.15.1:0/1745997412 shutdown_connections 2024-09-16T00:35:09.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.634+0000 7f861dcba740 1 -- 172.21.15.1:0/1745997412 wait complete. 2024-09-16T00:35:09.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.634+0000 7f861dcba740 20 client.17010 trim_cache size 0 max 16384 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861dcba740 1 Processor -- start 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861dcba740 1 -- start start 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861dcba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc790876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861dcba740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861dcba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79068700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861dcba740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86141af540 con 0x55cc7906db40 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861dcba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86141ace10 con 0x55cc79077370 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861dcba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86141aac60 con 0x55cc7906c5d0 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8618e7c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8618e7c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59444/0 (socket says 172.21.15.1:59444) 2024-09-16T00:35:09.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8618e7c640 1 -- 172.21.15.1:0/3676222160 learned_addr learned my addr 172.21.15.1:0/3676222160 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861967d640 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc790876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8619e7e640 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79068700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8618e7c640 1 -- 172.21.15.1:0/3676222160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 msgr2=0x55cc790876c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8618e7c640 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc790876c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8618e7c640 1 -- 172.21.15.1:0/3676222160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 msgr2=0x55cc79068700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8618e7c640 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79068700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8618e7c640 1 -- 172.21.15.1:0/3676222160 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7906f120 con 0x55cc7906db40 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f861967d640 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc790876c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.647+0000 7f8618e7c640 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906d300 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f860809b340 tx=0x7f86080998c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76dabfb84b82221 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f85feffd640 1 -- 172.21.15.1:0/3676222160 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860801e070 con 0x55cc7906db40 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f85feffd640 1 -- 172.21.15.1:0/3676222160 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860801c120 con 0x55cc7906db40 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f85feffd640 1 -- 172.21.15.1:0/3676222160 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608010ca0 con 0x55cc7906db40 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 -- 172.21.15.1:0/3676222160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 msgr2=0x55cc7906d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:09.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906d300 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f860809b340 tx=0x7f86080998c0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 -- 172.21.15.1:0/3676222160 shutdown_connections 2024-09-16T00:35:09.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc79077370 0x55cc79068700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc7906db40 0x55cc7906d300 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 --2- 172.21.15.1:0/3676222160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906c5d0 0x55cc790876c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 -- 172.21.15.1:0/3676222160 >> 172.21.15.1:0/3676222160 conn(0x55cc79035310 msgr2=0x55cc79063a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:09.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 -- 172.21.15.1:0/3676222160 shutdown_connections 2024-09-16T00:35:09.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 -- 172.21.15.1:0/3676222160 wait complete. 2024-09-16T00:35:09.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.648+0000 7f861dcba740 1 -- start start 2024-09-16T00:35:09.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f861dcba740 1 Processor -- start 2024-09-16T00:35:09.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f861dcba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc7906c5d0 0x55cc79084050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f861dcba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906db40 0x55cc79070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f861dcba740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc79077370 0x55cc79071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f861dcba740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86141ab2d0 con 0x55cc79077370 2024-09-16T00:35:09.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f861dcba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86141ac740 con 0x55cc7906c5d0 2024-09-16T00:35:09.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f861dcba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86141ab9a0 con 0x55cc7906db40 2024-09-16T00:35:09.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f861dcba740 10 client.0 _ll_register_callbacks cb 0x7f8608097690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:09.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f8619e7e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc79077370 0x55cc79071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f8619e7e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc79077370 0x55cc79071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59450/0 (socket says 172.21.15.1:59450) 2024-09-16T00:35:09.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f8619e7e640 1 -- 172.21.15.1:0/811928654 learned_addr learned my addr 172.21.15.1:0/811928654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f861967d640 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc7906c5d0 0x55cc79084050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f8619e7e640 1 -- 172.21.15.1:0/811928654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906db40 msgr2=0x55cc79070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f8618e7c640 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906db40 0x55cc79070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.649+0000 7f8619e7e640 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906db40 0x55cc79070e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f8619e7e640 1 -- 172.21.15.1:0/811928654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc7906c5d0 msgr2=0x55cc79084050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f8619e7e640 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc7906c5d0 0x55cc79084050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f8619e7e640 1 -- 172.21.15.1:0/811928654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc79071c60 con 0x55cc79077370 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f8619e7e640 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc79077370 0x55cc79071460 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f860c00f010 tx=0x7f860c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f852ea84b5fdaa4d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f85fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:09.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f85fcff9640 1 -- 172.21.15.1:0/811928654 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c025440 con 0x55cc79077370 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f861dcba740 1 -- 172.21.15.1:0/811928654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f860801c120 con 0x55cc79077370 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f85fcff9640 1 -- 172.21.15.1:0/811928654 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f860c017070 con 0x55cc79077370 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f85fcff9640 1 -- 172.21.15.1:0/811928654 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f860c033050 con 0x55cc79077370 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f861dcba740 10 client.17016 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f861dcba740 1 -- 172.21.15.1:0/811928654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cc790f6120 con 0x55cc79077370 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.650+0000 7f85fcff9640 1 -- 172.21.15.1:0/811928654 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860801c120 con 0x55cc79077370 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f85fcff9640 1 -- 172.21.15.1:0/811928654 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f860c0235b0 con 0x55cc79077370 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f861dcba740 10 client.17016 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f861dcba740 10 client.17016 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f861dcba740 20 client.17016 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:09.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f861dcba740 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cc7908b330 0x55cc790f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f861dcba740 4 client.17016 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f861dcba740 1 -- 172.21.15.1:0/811928654 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc78fd9d70 con 0x55cc7908b330 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f861967d640 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cc7908b330 0x55cc790f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f861967d640 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cc7908b330 0x55cc790f8910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4628a6ef1568d7d server_cookie=92ebf61e37292291 in_seq=0 out_seq=0 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f85fcff9640 10 client.17016 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f85fcff9640 1 -- 172.21.15.1:0/811928654 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc78fd9d70 con 0x55cc7908b330 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f85fcff9640 10 client.17016 handle_command_reply: tid=0 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.651+0000 7f861dcba740 1 client.17016 shutdown 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 -- 172.21.15.1:0/811928654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc79077370 msgr2=0x55cc79071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc79077370 0x55cc79071460 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f860c00f010 tx=0x7f860c004b50 comp rx=0 tx=0).stop 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 -- 172.21.15.1:0/811928654 shutdown_connections 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc79077370 0x55cc79071460 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc7906db40 0x55cc79070e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55cc7908b330 0x55cc790f8910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 --2- 172.21.15.1:0/811928654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cc7906c5d0 0x55cc79084050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 -- 172.21.15.1:0/811928654 >> 172.21.15.1:0/811928654 conn(0x55cc79035310 msgr2=0x55cc79068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:09.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f85fcff9640 0 client.17016 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:35:09.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 -- 172.21.15.1:0/811928654 shutdown_connections 2024-09-16T00:35:09.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 1 -- 172.21.15.1:0/811928654 wait complete. 2024-09-16T00:35:09.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861dcba740 20 client.17016 trim_cache size 0 max 16384 2024-09-16T00:35:09.660 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:35:09.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861b8e1640 1 -- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86000a3240 msgr2=0x7f86000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:09.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861b8e1640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86000a3240 0x7f86000a5700 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f860c00ecf0 tx=0x7f860c008220 comp rx=0 tx=0).stop 2024-09-16T00:35:09.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861b8e1640 1 -- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614102790 msgr2=0x7f8614198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:09.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.652+0000 7f861b8e1640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614102790 0x7f8614198af0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f860800b4a0 tx=0x7f860800b960 comp rx=0 tx=0).stop 2024-09-16T00:35:09.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.653+0000 7f861b8e1640 1 -- 172.21.15.1:0/1674875075 shutdown_connections 2024-09-16T00:35:09.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.653+0000 7f861b8e1640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8614106f10 0x7f861419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.653+0000 7f861b8e1640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8614102790 0x7f8614198af0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.653+0000 7f861b8e1640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86000a3240 0x7f86000a5700 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.653+0000 7f861b8e1640 1 --2- 172.21.15.1:0/1674875075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8614101de0 0x7f86141985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:09.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.653+0000 7f861b8e1640 1 -- 172.21.15.1:0/1674875075 >> 172.21.15.1:0/1674875075 conn(0x7f86140fd8c0 msgr2=0x7f86140fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:09.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.653+0000 7f861b8e1640 1 -- 172.21.15.1:0/1674875075 shutdown_connections 2024-09-16T00:35:09.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:09.653+0000 7f861b8e1640 1 -- 172.21.15.1:0/1674875075 wait complete. 2024-09-16T00:35:10.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:09 smithi001 ceph-mon[30934]: pgmap v553: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:10.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:09 smithi153 ceph-mon[35538]: pgmap v553: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:09 smithi019 ceph-mon[35186]: pgmap v553: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:10.416 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:35:10.675 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:11.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.189+0000 7f3c05e3e640 1 -- 172.21.15.1:0/381865448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00100a80 msgr2=0x7f3c00100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.189+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/381865448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00100a80 0x7f3c00100ee0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009930 tx=0x7f3bf402a420 comp rx=0 tx=0).stop 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.189+0000 7f3c05e3e640 1 -- 172.21.15.1:0/381865448 shutdown_connections 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.189+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/381865448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00101420 0x7f3c0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.189+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/381865448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00100a80 0x7f3c00100ee0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.189+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/381865448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c00106a80 0x7f3c00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.189+0000 7f3c05e3e640 1 -- 172.21.15.1:0/381865448 >> 172.21.15.1:0/381865448 conn(0x7f3c000fc7a0 msgr2=0x7f3c000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.189+0000 7f3c05e3e640 1 -- 172.21.15.1:0/381865448 shutdown_connections 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.189+0000 7f3c05e3e640 1 -- 172.21.15.1:0/381865448 wait complete. 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3c05e3e640 1 Processor -- start 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3c05e3e640 1 -- start start 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3c05e3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00100a80 0x7f3c00198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3c05e3e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00101420 0x7f3c00198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3c05e3e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c00106a80 0x7f3c0019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3c05e3e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00111fe0 con 0x7f3c00101420 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3c05e3e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00111e60 con 0x7f3c00100a80 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3c05e3e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00112160 con 0x7f3c00106a80 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3bfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00101420 0x7f3c00198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3bfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00101420 0x7f3c00198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45614/0 (socket says 172.21.15.1:45614) 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3bfeffd640 1 -- 172.21.15.1:0/2776086902 learned_addr learned my addr 172.21.15.1:0/2776086902 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.190+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00100a80 0x7f3c00198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfffff640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c00106a80 0x7f3c0019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfeffd640 1 -- 172.21.15.1:0/2776086902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c00106a80 msgr2=0x7f3c0019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c00106a80 0x7f3c0019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfeffd640 1 -- 172.21.15.1:0/2776086902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00100a80 msgr2=0x7f3c00198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:11.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00100a80 0x7f3c00198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfeffd640 1 -- 172.21.15.1:0/2776086902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0019d3f0 con 0x7f3c00101420 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfffff640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c00106a80 0x7f3c0019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00100a80 0x7f3c00198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00101420 0x7f3c00198980 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3bf402a940 tx=0x7f3bf4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfcff9640 1 -- 172.21.15.1:0/2776086902 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf403f070 con 0x7f3c00101420 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfcff9640 1 -- 172.21.15.1:0/2776086902 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf4031c90 con 0x7f3c00101420 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3bfcff9640 1 -- 172.21.15.1:0/2776086902 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4035030 con 0x7f3c00101420 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3c05e3e640 1 -- 172.21.15.1:0/2776086902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0019d680 con 0x7f3c00101420 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.191+0000 7f3c05e3e640 1 -- 172.21.15.1:0/2776086902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0019db10 con 0x7f3c00101420 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.193+0000 7f3bfcff9640 1 -- 172.21.15.1:0/2776086902 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3bf4042070 con 0x7f3c00101420 2024-09-16T00:35:11.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3c08217740 1 Processor -- start 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3c08217740 1 -- start start 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3c08217740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 0x555c1bff8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3c08217740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff85d0 0x555c1bff8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3c08217740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c005760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3c08217740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555c1bf5e2f0 con 0x555c1c003370 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3c08217740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c1bf05cf0 con 0x555c1bff85d0 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3c08217740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555c1bf5a2b0 con 0x555c1bff9b40 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3bfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c005760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3bfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c005760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45628/0 (socket says 172.21.15.1:45628) 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3bfeffd640 1 -- 172.21.15.1:0/575386853 learned_addr learned my addr 172.21.15.1:0/575386853 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3bfffff640 1 --2- 172.21.15.1:0/575386853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 0x555c1bff8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3bfcff9640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3be80a32c0 0x7f3be80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/575386853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff85d0 0x555c1bff8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3bfffff640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3be80a32c0 0x7f3be80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.194+0000 7f3bfeffd640 1 -- 172.21.15.1:0/575386853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 msgr2=0x555c1bff8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/575386853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 0x555c1bff8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3bfeffd640 1 -- 172.21.15.1:0/575386853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff85d0 msgr2=0x555c1bff8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/575386853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff85d0 0x555c1bff8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3bfcff9640 1 -- 172.21.15.1:0/2776086902 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3bf40f0460 con 0x7f3c00101420 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3bfeffd640 1 -- 172.21.15.1:0/575386853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1beea3a0 con 0x555c1c003370 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/575386853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c005760 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40bc5b0 tx=0x7f3bf4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2884ae93a0f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3bfffff640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3be80a32c0 0x7f3be80a5780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8005fd0 tx=0x7f3bf8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3be5ffb640 1 -- 172.21.15.1:0/575386853 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4049070 con 0x555c1c003370 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3be5ffb640 1 -- 172.21.15.1:0/575386853 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf40c3d50 con 0x555c1c003370 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.195+0000 7f3be5ffb640 1 -- 172.21.15.1:0/575386853 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40bc620 con 0x555c1c003370 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 -- 172.21.15.1:0/575386853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 msgr2=0x555c1c005760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 --2- 172.21.15.1:0/575386853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c005760 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40bc5b0 tx=0x7f3bf4009260 comp rx=0 tx=0).stop 2024-09-16T00:35:11.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 -- 172.21.15.1:0/575386853 shutdown_connections 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 --2- 172.21.15.1:0/575386853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c005760 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 --2- 172.21.15.1:0/575386853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff85d0 0x555c1bff8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 --2- 172.21.15.1:0/575386853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 0x555c1bff8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 -- 172.21.15.1:0/575386853 >> 172.21.15.1:0/575386853 conn(0x555c1bfc1310 msgr2=0x555c1bff4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 -- 172.21.15.1:0/575386853 shutdown_connections 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 -- 172.21.15.1:0/575386853 wait complete. 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 -- start start 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.196+0000 7f3c08217740 1 Processor -- start 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3c08217740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c002d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3c08217740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 0x555c1bfffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3c08217740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1c003370 0x555c1c0005e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3c08217740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555c1c005e30 con 0x555c1bff85d0 2024-09-16T00:35:11.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3c08217740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c1bf19060 con 0x555c1bff9b40 2024-09-16T00:35:11.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3c08217740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555c1c005fb0 con 0x555c1c003370 2024-09-16T00:35:11.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3c08217740 10 client.0 _ll_register_callbacks cb 0x555c1bc6c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bff7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c002d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bff7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c002d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45632/0 (socket says 172.21.15.1:45632) 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bff7fe640 1 -- 172.21.15.1:0/108571270 learned_addr learned my addr 172.21.15.1:0/108571270 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 0x555c1bfffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bfffff640 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1c003370 0x555c1c0005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bff7fe640 1 -- 172.21.15.1:0/108571270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1c003370 msgr2=0x555c1c0005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1c003370 0x555c1c0005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bff7fe640 1 -- 172.21.15.1:0/108571270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 msgr2=0x555c1bfffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 0x555c1bfffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bff7fe640 1 -- 172.21.15.1:0/108571270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1bccf3c0 con 0x555c1bff85d0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 0x555c1bfffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c002d40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0004330 tx=0x7f3bf00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8570cf4b4cb1a119 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bdeffd640 1 -- 172.21.15.1:0/108571270 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0005220 con 0x555c1bff85d0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3bdeffd640 1 -- 172.21.15.1:0/108571270 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3bf00053c0 con 0x555c1bff85d0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.197+0000 7f3c08217740 1 -- 172.21.15.1:0/108571270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c1bf6b860 con 0x555c1bff85d0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3bdeffd640 1 -- 172.21.15.1:0/108571270 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3bf0023050 con 0x555c1bff85d0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3c08217740 10 client.17028 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3c08217740 1 -- 172.21.15.1:0/108571270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c1c0117f0 con 0x555c1bff85d0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3bdeffd640 1 -- 172.21.15.1:0/108571270 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555c1bf6b860 con 0x555c1bff85d0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3bdeffd640 1 -- 172.21.15.1:0/108571270 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3bf0019030 con 0x555c1bff85d0 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3c08217740 10 client.17028 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:11.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3c08217740 10 client.17028 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3c08217740 20 client.17028 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3c08217740 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555c1c012cb0 0x555c1c0150c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3c08217740 4 client.17028 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3c08217740 1 -- 172.21.15.1:0/108571270 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c1bf65d70 con 0x555c1c012cb0 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.198+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555c1c012cb0 0x555c1c0150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.199+0000 7f3bdeffd640 10 client.17028 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.199+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555c1c012cb0 0x555c1c0150c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65a7506b8085f217 server_cookie=5e66c2c7df620d3e in_seq=0 out_seq=0 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.199+0000 7f3bdeffd640 1 -- 172.21.15.1:0/108571270 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c1bf65d70 con 0x555c1c012cb0 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.199+0000 7f3bdeffd640 10 client.17028 handle_command_reply: tid=0 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.199+0000 7f3c08217740 1 client.17028 shutdown 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 -- 172.21.15.1:0/108571270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 msgr2=0x555c1c002d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c002d40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0004330 tx=0x7f3bf00039c0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 -- 172.21.15.1:0/108571270 shutdown_connections 2024-09-16T00:35:11.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1c003370 0x555c1c0005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 0x555c1bfffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555c1c012cb0 0x555c1c0150c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 --2- 172.21.15.1:0/108571270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c002d40 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3bdeffd640 0 client.17028 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:35:11.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 -- 172.21.15.1:0/108571270 >> 172.21.15.1:0/108571270 conn(0x555c1bfc1310 msgr2=0x555c1bffa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:11.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 -- 172.21.15.1:0/108571270 shutdown_connections 2024-09-16T00:35:11.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 1 -- 172.21.15.1:0/108571270 wait complete. 2024-09-16T00:35:11.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.200+0000 7f3c08217740 20 client.17028 trim_cache size 0 max 16384 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3c08217740 1 Processor -- start 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3c08217740 1 -- start start 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3c08217740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c001f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3c08217740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 0x555c1bffb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3c08217740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1c003370 0x555c1bffb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3c08217740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001b1760 con 0x555c1bff85d0 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3c08217740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001af030 con 0x555c1c003370 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3c08217740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001ace80 con 0x555c1bff9b40 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3bff7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c001f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3bff7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c001f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45636/0 (socket says 172.21.15.1:45636) 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3bff7fe640 1 -- 172.21.15.1:0/3637555184 learned_addr learned my addr 172.21.15.1:0/3637555184 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 0x555c1bffb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.213+0000 7f3bfffff640 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1c003370 0x555c1bffb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3bff7fe640 1 -- 172.21.15.1:0/3637555184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 msgr2=0x555c1bffb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 0x555c1bffb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3bff7fe640 1 -- 172.21.15.1:0/3637555184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1c003370 msgr2=0x555c1bffb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1c003370 0x555c1bffb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3bff7fe640 1 -- 172.21.15.1:0/3637555184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1bff5960 con 0x555c1bff85d0 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3bfffff640 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1c003370 0x555c1bffb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c001f60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000be60 tx=0x7f3bf000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480683e7a762e7e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3be57fa640 1 -- 172.21.15.1:0/3637555184 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0003ec0 con 0x555c1bff85d0 2024-09-16T00:35:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3be57fa640 1 -- 172.21.15.1:0/3637555184 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf0005520 con 0x555c1bff85d0 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.214+0000 7f3be57fa640 1 -- 172.21.15.1:0/3637555184 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0004b30 con 0x555c1bff85d0 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 -- 172.21.15.1:0/3637555184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 msgr2=0x555c1c001f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c001f60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000be60 tx=0x7f3bf000bac0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 -- 172.21.15.1:0/3637555184 shutdown_connections 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1c003370 0x555c1bffb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff9b40 0x555c1bffb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 --2- 172.21.15.1:0/3637555184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1bff85d0 0x555c1c001f60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 -- 172.21.15.1:0/3637555184 >> 172.21.15.1:0/3637555184 conn(0x555c1bfc1310 msgr2=0x555c1c001850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 -- 172.21.15.1:0/3637555184 shutdown_connections 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 -- 172.21.15.1:0/3637555184 wait complete. 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 -- start start 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 Processor -- start 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff85d0 0x555c1c00b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 0x555c1c006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.215+0000 7f3c08217740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c007020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3c08217740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001ad4f0 con 0x555c1c003370 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3c08217740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001ae960 con 0x555c1bff9b40 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3c08217740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001adbc0 con 0x555c1bff85d0 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3c08217740 10 client.0 _ll_register_callbacks cb 0x7f3bf000bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c007020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c007020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45648/0 (socket says 172.21.15.1:45648) 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfffff640 1 -- 172.21.15.1:0/1695908854 learned_addr learned my addr 172.21.15.1:0/1695908854 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff85d0 0x555c1c00b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfeffd640 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 0x555c1c006ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfffff640 1 -- 172.21.15.1:0/1695908854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff85d0 msgr2=0x555c1c00b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfffff640 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff85d0 0x555c1c00b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfffff640 1 -- 172.21.15.1:0/1695908854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 msgr2=0x555c1c006ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfffff640 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 0x555c1c006ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfffff640 1 -- 172.21.15.1:0/1695908854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1c007850 con 0x555c1c003370 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bfffff640 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c007020 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8011e80 tx=0x7f3bf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf674b7daf801df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bdeffd640 1 -- 172.21.15.1:0/1695908854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8025070 con 0x555c1c003370 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3bdeffd640 1 -- 172.21.15.1:0/1695908854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3bf8004d20 con 0x555c1c003370 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.216+0000 7f3c08217740 1 -- 172.21.15.1:0/1695908854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bf0005520 con 0x555c1c003370 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3bdeffd640 1 -- 172.21.15.1:0/1695908854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3bf8030050 con 0x555c1c003370 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3c08217740 10 client.17034 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3c08217740 1 -- 172.21.15.1:0/1695908854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c1c07d8a0 con 0x555c1c003370 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3bdeffd640 1 -- 172.21.15.1:0/1695908854 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf0005520 con 0x555c1c003370 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3bdeffd640 1 -- 172.21.15.1:0/1695908854 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3bf80051c0 con 0x555c1c003370 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3c08217740 10 client.17034 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:11.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3c08217740 10 client.17034 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3c08217740 20 client.17034 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3c08217740 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555c1c07e020 0x555c1c080440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3c08217740 4 client.17034 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3c08217740 1 -- 172.21.15.1:0/1695908854 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c1bee2170 con 0x555c1c07e020 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.217+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555c1c07e020 0x555c1c080440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3bff7fe640 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555c1c07e020 0x555c1c080440 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2aa2daada92caf3 server_cookie=d5b55db015619093 in_seq=0 out_seq=0 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3bdeffd640 10 client.17034 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3bdeffd640 1 -- 172.21.15.1:0/1695908854 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c1bee2170 con 0x555c1c07e020 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3bdeffd640 10 client.17034 handle_command_reply: tid=0 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 client.17034 shutdown 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 -- 172.21.15.1:0/1695908854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 msgr2=0x555c1c007020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c007020 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8011e80 tx=0x7f3bf8004a40 comp rx=0 tx=0).stop 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 -- 172.21.15.1:0/1695908854 shutdown_connections 2024-09-16T00:35:11.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555c1c07e020 0x555c1c080440 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c1c003370 0x555c1c007020 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1bff9b40 0x555c1c006ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 --2- 172.21.15.1:0/1695908854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1bff85d0 0x555c1c00b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 -- 172.21.15.1:0/1695908854 >> 172.21.15.1:0/1695908854 conn(0x555c1bfc1310 msgr2=0x555c1bff3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3bdeffd640 0 client.17034 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:35:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 -- 172.21.15.1:0/1695908854 shutdown_connections 2024-09-16T00:35:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 1 -- 172.21.15.1:0/1695908854 wait complete. 2024-09-16T00:35:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.218+0000 7f3c08217740 20 client.17034 trim_cache size 0 max 16384 2024-09-16T00:35:11.228 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:35:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 -- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3be80a32c0 msgr2=0x7f3be80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3be80a32c0 0x7f3be80a5780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8005fd0 tx=0x7f3bf8004ab0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 -- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00101420 msgr2=0x7f3c00198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00101420 0x7f3c00198980 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3bf402a940 tx=0x7f3bf4003b90 comp rx=0 tx=0).stop 2024-09-16T00:35:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 -- 172.21.15.1:0/2776086902 shutdown_connections 2024-09-16T00:35:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3be80a32c0 0x7f3be80a5780 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c00106a80 0x7f3c0019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c00101420 0x7f3c00198980 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 --2- 172.21.15.1:0/2776086902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00100a80 0x7f3c00198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 -- 172.21.15.1:0/2776086902 >> 172.21.15.1:0/2776086902 conn(0x7f3c000fc7a0 msgr2=0x7f3c000fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 -- 172.21.15.1:0/2776086902 shutdown_connections 2024-09-16T00:35:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:11.219+0000 7f3c05e3e640 1 -- 172.21.15.1:0/2776086902 wait complete. 2024-09-16T00:35:11.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:35:12.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:11 smithi001 ceph-mon[30934]: pgmap v554: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:12.208 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:12.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:11 smithi153 ceph-mon[35538]: pgmap v554: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:12.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:11 smithi019 ceph-mon[35186]: pgmap v554: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.691+0000 7f3e5d555640 1 -- 172.21.15.1:0/4199808512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58106a80 msgr2=0x7f3e58106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.691+0000 7f3e5d555640 1 --2- 172.21.15.1:0/4199808512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58106a80 0x7f3e58106e60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009930 tx=0x7f3e4802a420 comp rx=0 tx=0).stop 2024-09-16T00:35:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.691+0000 7f3e5d555640 1 -- 172.21.15.1:0/4199808512 shutdown_connections 2024-09-16T00:35:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.691+0000 7f3e5d555640 1 --2- 172.21.15.1:0/4199808512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58101420 0x7f3e5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.691+0000 7f3e5d555640 1 --2- 172.21.15.1:0/4199808512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58100a80 0x7f3e58100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.691+0000 7f3e5d555640 1 --2- 172.21.15.1:0/4199808512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58106a80 0x7f3e58106e60 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.691+0000 7f3e5d555640 1 -- 172.21.15.1:0/4199808512 >> 172.21.15.1:0/4199808512 conn(0x7f3e580fc7a0 msgr2=0x7f3e580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.691+0000 7f3e5d555640 1 -- 172.21.15.1:0/4199808512 shutdown_connections 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.691+0000 7f3e5d555640 1 -- 172.21.15.1:0/4199808512 wait complete. 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e5d555640 1 Processor -- start 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e5d555640 1 -- start start 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e5d555640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58100a80 0x7f3e58198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e5d555640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58101420 0x7f3e58198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e5d555640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58106a80 0x7f3e5819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e5d555640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58110ef0 con 0x7f3e58100a80 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e5d555640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58110d70 con 0x7f3e58106a80 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e5d555640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58111070 con 0x7f3e58101420 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e56ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58100a80 0x7f3e58198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e56ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58100a80 0x7f3e58198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45664/0 (socket says 172.21.15.1:45664) 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e56ffd640 1 -- 172.21.15.1:0/3961638726 learned_addr learned my addr 172.21.15.1:0/3961638726 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58101420 0x7f3e58198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e577fe640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58106a80 0x7f3e5819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.692+0000 7f3e56ffd640 1 -- 172.21.15.1:0/3961638726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58101420 msgr2=0x7f3e58198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58101420 0x7f3e58198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e56ffd640 1 -- 172.21.15.1:0/3961638726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58106a80 msgr2=0x7f3e5819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58106a80 0x7f3e5819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e56ffd640 1 -- 172.21.15.1:0/3961638726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5819d3f0 con 0x7f3e58100a80 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e577fe640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58106a80 0x7f3e5819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58100a80 0x7f3e58198440 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3e4802a940 tx=0x7f3e48003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e54ff9640 1 -- 172.21.15.1:0/3961638726 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4803f070 con 0x7f3e58100a80 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e54ff9640 1 -- 172.21.15.1:0/3961638726 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e48031c90 con 0x7f3e58100a80 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e5d555640 1 -- 172.21.15.1:0/3961638726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5819d680 con 0x7f3e58100a80 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e54ff9640 1 -- 172.21.15.1:0/3961638726 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48035030 con 0x7f3e58100a80 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.693+0000 7f3e5d555640 1 -- 172.21.15.1:0/3961638726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e581a4fc0 con 0x7f3e58100a80 2024-09-16T00:35:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.695+0000 7f3e54ff9640 1 -- 172.21.15.1:0/3961638726 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3e48042070 con 0x7f3e58100a80 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e5f92e740 1 Processor -- start 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e5f92e740 1 -- start start 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941aab40 0x5578941a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 0x5578941b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55789410f2f0 con 0x5578941a95d0 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578940b6cf0 con 0x5578941b4370 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55789410b2b0 con 0x5578941aab40 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e56ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e54ff9640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3e440a3240 0x7f3e440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e56ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45674/0 (socket says 172.21.15.1:45674) 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e56ffd640 1 -- 172.21.15.1:0/1518619541 learned_addr learned my addr 172.21.15.1:0/1518619541 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.696+0000 7f3e577fe640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3e440a3240 0x7f3e440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 0x5578941b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e577fe640 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941aab40 0x5578941a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e54ff9640 1 -- 172.21.15.1:0/3961638726 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3e480f0580 con 0x7f3e58100a80 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e56ffd640 1 -- 172.21.15.1:0/1518619541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941aab40 msgr2=0x5578941a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941aab40 0x5578941a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e56ffd640 1 -- 172.21.15.1:0/1518619541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 msgr2=0x5578941b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 0x5578941b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e56ffd640 1 -- 172.21.15.1:0/1518619541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789409b3a0 con 0x5578941a95d0 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e577fe640 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941aab40 0x5578941a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 0x5578941b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941a9a10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3e48031fd0 tx=0x7f3e4802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb90be7d6954aa13 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e577fe640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3e440a3240 0x7f3e440a5700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009440 tx=0x7f3e50009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e4d7fa640 1 -- 172.21.15.1:0/1518619541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48049070 con 0x5578941a95d0 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e4d7fa640 1 -- 172.21.15.1:0/1518619541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e480bcea0 con 0x5578941a95d0 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.697+0000 7f3e4d7fa640 1 -- 172.21.15.1:0/1518619541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e480f0240 con 0x5578941a95d0 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1518619541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 msgr2=0x5578941a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941a9a10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3e48031fd0 tx=0x7f3e4802a420 comp rx=0 tx=0).stop 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1518619541 shutdown_connections 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 0x5578941b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941a9a10 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1518619541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941aab40 0x5578941a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1518619541 >> 172.21.15.1:0/1518619541 conn(0x557894172310 msgr2=0x5578941a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1518619541 shutdown_connections 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1518619541 wait complete. 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.698+0000 7f3e5f92e740 1 -- start start 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e5f92e740 1 Processor -- start 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941a95d0 0x5578941b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 0x5578941b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941b4370 0x5578941b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5578941b6e30 con 0x5578941b4370 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578940ca060 con 0x5578941aab40 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578941b6fb0 con 0x5578941a95d0 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e5f92e740 10 client.0 _ll_register_callbacks cb 0x557893e1d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941b4370 0x5578941b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941b4370 0x5578941b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45684/0 (socket says 172.21.15.1:45684) 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e577fe640 1 -- 172.21.15.1:0/1602257488 learned_addr learned my addr 172.21.15.1:0/1602257488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941a95d0 0x5578941b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 0x5578941b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e577fe640 1 -- 172.21.15.1:0/1602257488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941a95d0 msgr2=0x5578941b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e577fe640 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941a95d0 0x5578941b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e577fe640 1 -- 172.21.15.1:0/1602257488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 msgr2=0x5578941b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e577fe640 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 0x5578941b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e577fe640 1 -- 172.21.15.1:0/1602257488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557893e803c0 con 0x5578941b4370 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e577fe640 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941b4370 0x5578941b1860 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009c90 tx=0x7f3e50004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45a4b6a76b4a62f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.699+0000 7f3e3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/1602257488 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50028060 con 0x5578941b4370 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/1602257488 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3e5001a930 con 0x5578941b4370 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1602257488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e480bcea0 con 0x5578941b4370 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/1602257488 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e50013040 con 0x5578941b4370 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e5f92e740 10 client.17046 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1602257488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578941c2860 con 0x5578941b4370 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/1602257488 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e480bcea0 con 0x5578941b4370 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/1602257488 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3e50023b00 con 0x5578941b4370 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e5f92e740 10 client.17046 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e5f92e740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e5f92e740 20 client.17046 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5578941c3d70 0x5578941c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e5f92e740 4 client.17046 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.700+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1602257488 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55789411c860 con 0x5578941c3d70 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.701+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5578941c3d70 0x5578941c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.701+0000 7f3e3e7fc640 10 client.17046 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.701+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5578941c3d70 0x5578941c61d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4f18754e614645d server_cookie=10fad294c2437124 in_seq=0 out_seq=0 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/1602257488 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55789411c860 con 0x5578941c3d70 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e3e7fc640 10 client.17046 handle_command_reply: tid=0 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 client.17046 shutdown 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1602257488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941b4370 msgr2=0x5578941b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941b4370 0x5578941b1860 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009c90 tx=0x7f3e50004b50 comp rx=0 tx=0).stop 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1602257488 shutdown_connections 2024-09-16T00:35:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5578941c3d70 0x5578941c61d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941b4370 0x5578941b1860 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 0x5578941b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e3e7fc640 0 client.17046 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1602257488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941a95d0 0x5578941b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1602257488 >> 172.21.15.1:0/1602257488 conn(0x557894172310 msgr2=0x5578941ab990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1602257488 shutdown_connections 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1602257488 wait complete. 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.702+0000 7f3e5f92e740 20 client.17046 trim_cache size 0 max 16384 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e5f92e740 1 Processor -- start 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e5f92e740 1 -- start start 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941a95d0 0x5578941b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941aab40 0x5578941ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 0x5578941ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581b1700 con 0x5578941aab40 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581aefd0 con 0x5578941b4370 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581ace20 con 0x5578941a95d0 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e4e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941aab40 0x5578941ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e4e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941aab40 0x5578941ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45686/0 (socket says 172.21.15.1:45686) 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.715+0000 7f3e4e7fc640 1 -- 172.21.15.1:0/1946522486 learned_addr learned my addr 172.21.15.1:0/1946522486 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e4e7fc640 1 -- 172.21.15.1:0/1946522486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941a95d0 msgr2=0x5578941b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/1946522486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941a95d0 0x5578941b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e577fe640 1 --2- 172.21.15.1:0/1946522486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 0x5578941ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e4e7fc640 1 -- 172.21.15.1:0/1946522486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 msgr2=0x5578941ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/1946522486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 0x5578941ac6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e4e7fc640 1 -- 172.21.15.1:0/1946522486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578941c6850 con 0x5578941aab40 2024-09-16T00:35:12.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/1946522486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941aab40 0x5578941ac180 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000ac10 tx=0x7f3e40013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c0170ef7572279 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e4cff9640 1 -- 172.21.15.1:0/1946522486 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000f650 con 0x5578941aab40 2024-09-16T00:35:12.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e4cff9640 1 -- 172.21.15.1:0/1946522486 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e4000fcd0 con 0x5578941aab40 2024-09-16T00:35:12.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.716+0000 7f3e4cff9640 1 -- 172.21.15.1:0/1946522486 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40011e80 con 0x5578941aab40 2024-09-16T00:35:12.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1946522486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941aab40 msgr2=0x5578941ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1946522486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941aab40 0x5578941ac180 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000ac10 tx=0x7f3e40013830 comp rx=0 tx=0).stop 2024-09-16T00:35:12.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1946522486 shutdown_connections 2024-09-16T00:35:12.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1946522486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941b4370 0x5578941ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1946522486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941aab40 0x5578941ac180 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/1946522486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941a95d0 0x5578941b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1946522486 >> 172.21.15.1:0/1946522486 conn(0x557894172310 msgr2=0x5578941b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1946522486 shutdown_connections 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 -- 172.21.15.1:0/1946522486 wait complete. 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 -- start start 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 Processor -- start 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 0x5578941b8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941b4370 0x55789422da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581ad490 con 0x5578941a95d0 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581ae900 con 0x5578941aab40 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.717+0000 7f3e5f92e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581adb60 con 0x5578941b4370 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e5f92e740 10 client.0 _ll_register_callbacks cb 0x7f3e5000af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e56ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e56ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941b81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45692/0 (socket says 172.21.15.1:45692) 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e56ffd640 1 -- 172.21.15.1:0/4090247253 learned_addr learned my addr 172.21.15.1:0/4090247253 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 0x5578941b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e577fe640 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941b4370 0x55789422da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e56ffd640 1 -- 172.21.15.1:0/4090247253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941b4370 msgr2=0x55789422da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941b4370 0x55789422da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e56ffd640 1 -- 172.21.15.1:0/4090247253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 msgr2=0x5578941b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 0x5578941b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e56ffd640 1 -- 172.21.15.1:0/4090247253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55789422dfb0 con 0x5578941a95d0 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e56ffd640 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941b81c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3e480bc730 tx=0x7f3e48009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d978733783bd48 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/4090247253 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48049040 con 0x5578941a95d0 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/4090247253 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3e480baaa0 con 0x5578941a95d0 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e5f92e740 1 -- 172.21.15.1:0/4090247253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e4000fcd0 con 0x5578941a95d0 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/4090247253 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e480bcea0 con 0x5578941a95d0 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e5f92e740 10 client.17052 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.718+0000 7f3e5f92e740 1 -- 172.21.15.1:0/4090247253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55789422f170 con 0x5578941a95d0 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/4090247253 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e4000fcd0 con 0x5578941a95d0 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/4090247253 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3e480b8e10 con 0x5578941a95d0 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e5f92e740 10 client.17052 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e5f92e740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e5f92e740 20 client.17052 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:12.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55789422f570 0x557894231960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e5f92e740 4 client.17052 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e5f92e740 1 -- 172.21.15.1:0/4090247253 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557894116d70 con 0x55789422f570 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55789422f570 0x557894231960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e3e7fc640 10 client.17052 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.719+0000 7f3e4e7fc640 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55789422f570 0x557894231960 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cde8fac7bdb8f19 server_cookie=cfcbee29ebd5bbc7 in_seq=0 out_seq=0 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/4090247253 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557894116d70 con 0x55789422f570 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e3e7fc640 10 client.17052 handle_command_reply: tid=0 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 client.17052 shutdown 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 -- 172.21.15.1:0/4090247253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 msgr2=0x5578941b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941b81c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3e480bc730 tx=0x7f3e48009560 comp rx=0 tx=0).stop 2024-09-16T00:35:12.740 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:35:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 -- 172.21.15.1:0/4090247253 shutdown_connections 2024-09-16T00:35:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55789422f570 0x557894231960 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578941b4370 0x55789422da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578941aab40 0x5578941b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e3e7fc640 0 client.17052 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 --2- 172.21.15.1:0/4090247253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578941a95d0 0x5578941b81c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 -- 172.21.15.1:0/4090247253 >> 172.21.15.1:0/4090247253 conn(0x557894172310 msgr2=0x5578941b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 -- 172.21.15.1:0/4090247253 shutdown_connections 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 1 -- 172.21.15.1:0/4090247253 wait complete. 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5f92e740 20 client.17052 trim_cache size 0 max 16384 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5d555640 1 -- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3e440a3240 msgr2=0x7f3e440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5d555640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3e440a3240 0x7f3e440a5700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009440 tx=0x7f3e50009610 comp rx=0 tx=0).stop 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5d555640 1 -- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58100a80 msgr2=0x7f3e58198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5d555640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58100a80 0x7f3e58198440 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3e4802a940 tx=0x7f3e48003c00 comp rx=0 tx=0).stop 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5d555640 1 -- 172.21.15.1:0/3961638726 shutdown_connections 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5d555640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58106a80 0x7f3e5819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5d555640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3e440a3240 0x7f3e440a5700 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5d555640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58101420 0x7f3e58198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.720+0000 7f3e5d555640 1 --2- 172.21.15.1:0/3961638726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e58100a80 0x7f3e58198440 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.721+0000 7f3e5d555640 1 -- 172.21.15.1:0/3961638726 >> 172.21.15.1:0/3961638726 conn(0x7f3e580fc7a0 msgr2=0x7f3e5806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.721+0000 7f3e5d555640 1 -- 172.21.15.1:0/3961638726 shutdown_connections 2024-09-16T00:35:12.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:12.721+0000 7f3e5d555640 1 -- 172.21.15.1:0/3961638726 wait complete. 2024-09-16T00:35:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:13 smithi001 ceph-mon[30934]: pgmap v555: 129 pgs: 129 active+clean; 3.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:14.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:13 smithi153 ceph-mon[35538]: pgmap v555: 129 pgs: 129 active+clean; 3.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:14.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:13 smithi019 ceph-mon[35186]: pgmap v555: 129 pgs: 129 active+clean; 3.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:14.537 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:35:14.538 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:35:14.809 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.270+0000 7fb8e3327640 1 -- 172.21.15.1:0/1074807366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 msgr2=0x7fb8dc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.270+0000 7fb8e3327640 1 --2- 172.21.15.1:0/1074807366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 0x7fb8dc10e610 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009ca0 tx=0x7fb8d402a950 comp rx=0 tx=0).stop 2024-09-16T00:35:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.270+0000 7fb8e3327640 1 -- 172.21.15.1:0/1074807366 shutdown_connections 2024-09-16T00:35:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.270+0000 7fb8e3327640 1 --2- 172.21.15.1:0/1074807366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 0x7fb8dc10e610 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.270+0000 7fb8e3327640 1 --2- 172.21.15.1:0/1074807366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8dc103660 0x7fb8dc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.270+0000 7fb8e3327640 1 --2- 172.21.15.1:0/1074807366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102d40 0x7fb8dc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.270+0000 7fb8e3327640 1 -- 172.21.15.1:0/1074807366 >> 172.21.15.1:0/1074807366 conn(0x7fb8dc0fc7a0 msgr2=0x7fb8dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.270+0000 7fb8e3327640 1 -- 172.21.15.1:0/1074807366 shutdown_connections 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.270+0000 7fb8e3327640 1 -- 172.21.15.1:0/1074807366 wait complete. 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e3327640 1 Processor -- start 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e3327640 1 -- start start 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e3327640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102d40 0x7fb8dc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e3327640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8dc103660 0x7fb8dc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e3327640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 0x7fb8dc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e3327640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc111fb0 con 0x7fb8dc10c170 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e3327640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc111e30 con 0x7fb8dc102d40 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e3327640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc112130 con 0x7fb8dc103660 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e18c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 0x7fb8dc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e18c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 0x7fb8dc19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45702/0 (socket says 172.21.15.1:45702) 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e18c4640 1 -- 172.21.15.1:0/3806127986 learned_addr learned my addr 172.21.15.1:0/3806127986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102d40 0x7fb8dc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8dc103660 0x7fb8dc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e18c4640 1 -- 172.21.15.1:0/3806127986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8dc103660 msgr2=0x7fb8dc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e18c4640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8dc103660 0x7fb8dc19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e18c4640 1 -- 172.21.15.1:0/3806127986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102d40 msgr2=0x7fb8dc19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e18c4640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102d40 0x7fb8dc19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.271+0000 7fb8e18c4640 1 -- 172.21.15.1:0/3806127986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc19f5d0 con 0x7fb8dc10c170 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.272+0000 7fb8e18c4640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 0x7fb8dc19eef0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4006010 tx=0x7fb8d402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.272+0000 7fb8d27fc640 1 -- 172.21.15.1:0/3806127986 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4034040 con 0x7fb8dc10c170 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.272+0000 7fb8d27fc640 1 -- 172.21.15.1:0/3806127986 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d4003e00 con 0x7fb8dc10c170 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.272+0000 7fb8e3327640 1 -- 172.21.15.1:0/3806127986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc19f860 con 0x7fb8dc10c170 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.272+0000 7fb8d27fc640 1 -- 172.21.15.1:0/3806127986 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40456e0 con 0x7fb8dc10c170 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.272+0000 7fb8e3327640 1 -- 172.21.15.1:0/3806127986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc1a71a0 con 0x7fb8dc10c170 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e5700740 1 Processor -- start 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8d27fc640 1 -- 172.21.15.1:0/3806127986 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fb8d4037020 con 0x7fb8dc10c170 2024-09-16T00:35:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e5700740 1 -- start start 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x560583473f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x560583481130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e5700740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5605833d9280 con 0x560583474480 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e5700740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560583380b10 con 0x56058347ed40 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e5700740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5605833d5240 con 0x560583473b60 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e10c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e10c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45708/0 (socket says 172.21.15.1:45708) 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.275+0000 7fb8e10c3640 1 -- 172.21.15.1:0/3547011772 learned_addr learned my addr 172.21.15.1:0/3547011772 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/3547011772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x560583473f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8d27fc640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8c80a3240 0x7fb8c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8e10c3640 1 -- 172.21.15.1:0/3547011772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 msgr2=0x560583473f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/3547011772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x560583473f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8e10c3640 1 -- 172.21.15.1:0/3547011772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 msgr2=0x560583481130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/3547011772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x560583481130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8e10c3640 1 -- 172.21.15.1:0/3547011772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56058338d1f0 con 0x560583474480 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/3547011772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x560583473f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8e18c4640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8c80a3240 0x7fb8c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/3547011772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347e800 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800cbf0 tx=0x7fb8d800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f32495b8013cb12 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8d27fc640 1 -- 172.21.15.1:0/3806127986 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb8d40f0ee0 con 0x7fb8dc10c170 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8c77fe640 1 -- 172.21.15.1:0/3547011772 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8010630 con 0x560583474480 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8c77fe640 1 -- 172.21.15.1:0/3547011772 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d8010cb0 con 0x560583474480 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.276+0000 7fb8c77fe640 1 -- 172.21.15.1:0/3547011772 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8019c90 con 0x560583474480 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e5700740 1 -- 172.21.15.1:0/3547011772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 msgr2=0x56058347e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e5700740 1 --2- 172.21.15.1:0/3547011772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347e800 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800cbf0 tx=0x7fb8d800e720 comp rx=0 tx=0).stop 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e18c4640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8c80a3240 0x7fb8c80a5700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009430 tx=0x7fb8d4004530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e5700740 1 -- 172.21.15.1:0/3547011772 shutdown_connections 2024-09-16T00:35:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e5700740 1 --2- 172.21.15.1:0/3547011772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x560583481130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e5700740 1 --2- 172.21.15.1:0/3547011772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347e800 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e5700740 1 --2- 172.21.15.1:0/3547011772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x560583473f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e5700740 1 -- 172.21.15.1:0/3547011772 >> 172.21.15.1:0/3547011772 conn(0x5605834700f0 msgr2=0x5605834704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e5700740 1 -- 172.21.15.1:0/3547011772 shutdown_connections 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.277+0000 7fb8e5700740 1 -- 172.21.15.1:0/3547011772 wait complete. 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e5700740 1 -- start start 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e5700740 1 Processor -- start 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x56058347bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x56058347c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e5700740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560583481910 con 0x560583474480 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e5700740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560583393f10 con 0x56058347ed40 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e5700740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560583481a90 con 0x560583473b60 2024-09-16T00:35:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e5700740 10 client.0 _ll_register_callbacks cb 0x56058335ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e10c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e10c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45716/0 (socket says 172.21.15.1:45716) 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e10c3640 1 -- 172.21.15.1:0/1374990009 learned_addr learned my addr 172.21.15.1:0/1374990009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x56058347bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.278+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x56058347c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8e10c3640 1 -- 172.21.15.1:0/1374990009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 msgr2=0x56058347bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x56058347bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8e10c3640 1 -- 172.21.15.1:0/1374990009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 msgr2=0x56058347c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x56058347c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8e10c3640 1 -- 172.21.15.1:0/1374990009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56058314a6a0 con 0x560583474480 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347c140 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8019810 tx=0x7fb8d800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c377af50bfbf35 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/1374990009 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8010e60 con 0x560583474480 2024-09-16T00:35:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8e5700740 1 -- 172.21.15.1:0/1374990009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d8010cb0 con 0x560583474480 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/1374990009 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb8d8005250 con 0x560583474480 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/1374990009 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8d802e050 con 0x560583474480 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8e5700740 10 client.17064 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8e5700740 1 -- 172.21.15.1:0/1374990009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56058348d460 con 0x560583474480 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/1374990009 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d8010cb0 con 0x560583474480 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.279+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/1374990009 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb8d80046c0 con 0x560583474480 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.280+0000 7fb8e5700740 10 client.17064 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.280+0000 7fb8e5700740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.280+0000 7fb8e5700740 20 client.17064 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.280+0000 7fb8e5700740 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56058348e870 0x560583490c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.280+0000 7fb8e5700740 4 client.17064 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.280+0000 7fb8e5700740 1 -- 172.21.15.1:0/1374990009 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605833e67f0 con 0x56058348e870 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.280+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56058348e870 0x560583490c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.280+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56058348e870 0x560583490c60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f572cdbeb4aacd server_cookie=4e86995f1365499b in_seq=0 out_seq=0 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.280+0000 7fb8c4ff9640 10 client.17064 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/1374990009 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605833e67f0 con 0x56058348e870 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8c4ff9640 10 client.17064 handle_command_reply: tid=0 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 client.17064 shutdown 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 -- 172.21.15.1:0/1374990009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 msgr2=0x56058347c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347c140 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8019810 tx=0x7fb8d800c960 comp rx=0 tx=0).stop 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 -- 172.21.15.1:0/1374990009 shutdown_connections 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56058348e870 0x560583490c60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x56058347c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x56058347c140 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 --2- 172.21.15.1:0/1374990009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x56058347bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8c4ff9640 0 client.17064 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 -- 172.21.15.1:0/1374990009 >> 172.21.15.1:0/1374990009 conn(0x5605834700f0 msgr2=0x56058346f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 -- 172.21.15.1:0/1374990009 shutdown_connections 2024-09-16T00:35:15.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 1 -- 172.21.15.1:0/1374990009 wait complete. 2024-09-16T00:35:15.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.281+0000 7fb8e5700740 20 client.17064 trim_cache size 0 max 16384 2024-09-16T00:35:15.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e5700740 1 Processor -- start 2024-09-16T00:35:15.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e5700740 1 -- start start 2024-09-16T00:35:15.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560583473b60 0x560583496280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058347ed40 0x56058346ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e5700740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1b38d0 con 0x560583474480 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e5700740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1b11a0 con 0x560583473b60 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e5700740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1aeff0 con 0x56058347ed40 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e10c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e10c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45730/0 (socket says 172.21.15.1:45730) 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e10c3640 1 -- 172.21.15.1:0/545860263 learned_addr learned my addr 172.21.15.1:0/545860263 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/545860263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560583473b60 0x560583496280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e10c3640 1 -- 172.21.15.1:0/545860263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058347ed40 msgr2=0x56058346ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/545860263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058347ed40 0x56058346ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e10c3640 1 -- 172.21.15.1:0/545860263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560583473b60 msgr2=0x560583496280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/545860263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560583473b60 0x560583496280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e10c3640 1 -- 172.21.15.1:0/545860263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56058347e4d0 con 0x560583474480 2024-09-16T00:35:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.299+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/545860263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560583473b60 0x560583496280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/545860263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834967c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8004c30 tx=0x7fb8d8004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365c366fe683c76f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8c6ffd640 1 -- 172.21.15.1:0/545860263 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8037070 con 0x560583474480 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8c6ffd640 1 -- 172.21.15.1:0/545860263 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d8004e30 con 0x560583474480 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8c6ffd640 1 -- 172.21.15.1:0/545860263 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d801e030 con 0x560583474480 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8e5700740 1 -- 172.21.15.1:0/545860263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 msgr2=0x5605834967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8e5700740 1 --2- 172.21.15.1:0/545860263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834967c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8004c30 tx=0x7fb8d8004d30 comp rx=0 tx=0).stop 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8e5700740 1 -- 172.21.15.1:0/545860263 shutdown_connections 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8e5700740 1 --2- 172.21.15.1:0/545860263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058347ed40 0x56058346ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8e5700740 1 --2- 172.21.15.1:0/545860263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834967c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8e5700740 1 --2- 172.21.15.1:0/545860263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560583473b60 0x560583496280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.300+0000 7fb8e5700740 1 -- 172.21.15.1:0/545860263 >> 172.21.15.1:0/545860263 conn(0x5605834700f0 msgr2=0x56058347db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 -- 172.21.15.1:0/545860263 shutdown_connections 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 -- 172.21.15.1:0/545860263 wait complete. 2024-09-16T00:35:15.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 -- start start 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 Processor -- start 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x560583487140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x560583478c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1af660 con 0x560583474480 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1b0ad0 con 0x56058347ed40 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1afd30 con 0x560583473b60 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e5700740 10 client.0 _ll_register_callbacks cb 0x7fb8d8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e10c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e10c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834786f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45732/0 (socket says 172.21.15.1:45732) 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.301+0000 7fb8e10c3640 1 -- 172.21.15.1:0/2452373183 learned_addr learned my addr 172.21.15.1:0/2452373183 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x560583487140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x560583478c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e10c3640 1 -- 172.21.15.1:0/2452373183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 msgr2=0x560583487140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x560583487140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e10c3640 1 -- 172.21.15.1:0/2452373183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 msgr2=0x560583478c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x560583478c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e10c3640 1 -- 172.21.15.1:0/2452373183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560583479170 con 0x560583474480 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x560583478c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e10c3640 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834786f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80058c0 tx=0x7fb8d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf3d76a45b93aa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/2452373183 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8037070 con 0x560583474480 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/2452373183 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb8d80195c0 con 0x560583474480 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e5700740 1 -- 172.21.15.1:0/2452373183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d8004e30 con 0x560583474480 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/2452373183 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8d8042050 con 0x560583474480 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e5700740 10 client.17070 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.302+0000 7fb8e5700740 1 -- 172.21.15.1:0/2452373183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5605834f8fa0 con 0x560583474480 2024-09-16T00:35:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/2452373183 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d8004e30 con 0x560583474480 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/2452373183 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb8d8025470 con 0x560583474480 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8e5700740 10 client.17070 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8e5700740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8e5700740 20 client.17070 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8e5700740 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5605834f95b0 0x5605834fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8e5700740 4 client.17070 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8e5700740 1 -- 172.21.15.1:0/2452373183 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5605833e0d00 con 0x5605834f95b0 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5605834f95b0 0x5605834fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8e08c2640 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5605834f95b0 0x5605834fb9a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4f305b6aa63e65 server_cookie=21653bd7f4e33858 in_seq=0 out_seq=0 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.303+0000 7fb8c4ff9640 10 client.17070 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.304+0000 7fb8c4ff9640 1 -- 172.21.15.1:0/2452373183 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5605833e0d00 con 0x5605834f95b0 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.304+0000 7fb8c4ff9640 10 client.17070 handle_command_reply: tid=0 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.304+0000 7fb8e5700740 1 client.17070 shutdown 2024-09-16T00:35:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 -- 172.21.15.1:0/2452373183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 msgr2=0x5605834786f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834786f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80058c0 tx=0x7fb8d800c3a0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 -- 172.21.15.1:0/2452373183 shutdown_connections 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5605834f95b0 0x5605834fb9a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56058347ed40 0x560583478c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560583474480 0x5605834786f0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8c4ff9640 0 client.17070 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 --2- 172.21.15.1:0/2452373183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560583473b60 0x560583487140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 -- 172.21.15.1:0/2452373183 >> 172.21.15.1:0/2452373183 conn(0x5605834700f0 msgr2=0x56058346ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 -- 172.21.15.1:0/2452373183 shutdown_connections 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 1 -- 172.21.15.1:0/2452373183 wait complete. 2024-09-16T00:35:15.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.305+0000 7fb8e5700740 20 client.17070 trim_cache size 0 max 16384 2024-09-16T00:35:15.313 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:35:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:35:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "64923857-c344-49ad-a59c-d0b8ee525ef3", 2024-09-16T00:35:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:35:15.313 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 -- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8c80a3240 msgr2=0x7fb8c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8c80a3240 0x7fb8c80a5700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009430 tx=0x7fb8d4004530 comp rx=0 tx=0).stop 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 -- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 msgr2=0x7fb8dc19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 0x7fb8dc19eef0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4006010 tx=0x7fb8d402f710 comp rx=0 tx=0).stop 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 -- 172.21.15.1:0/3806127986 shutdown_connections 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8dc10c170 0x7fb8dc19eef0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8c80a3240 0x7fb8c80a5700 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8dc103660 0x7fb8dc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 --2- 172.21.15.1:0/3806127986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102d40 0x7fb8dc19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 -- 172.21.15.1:0/3806127986 >> 172.21.15.1:0/3806127986 conn(0x7fb8dc0fc7a0 msgr2=0x7fb8dc0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 -- 172.21.15.1:0/3806127986 shutdown_connections 2024-09-16T00:35:15.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:15.306+0000 7fb8e3327640 1 -- 172.21.15.1:0/3806127986 wait complete. 2024-09-16T00:35:16.105 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:35:16.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:15 smithi001 ceph-mon[30934]: pgmap v556: 129 pgs: 129 active+clean; 3.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:16.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:15 smithi153 ceph-mon[35538]: pgmap v556: 129 pgs: 129 active+clean; 3.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:15 smithi019 ceph-mon[35186]: pgmap v556: 129 pgs: 129 active+clean; 3.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:16.367 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:16.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.886+0000 7fa6050a6640 1 -- 172.21.15.1:0/192774478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6001049a0 msgr2=0x7fa6001089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.886+0000 7fa6050a6640 1 --2- 172.21.15.1:0/192774478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6001049a0 0x7fa6001089f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009900 tx=0x7fa5f402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.886+0000 7fa6050a6640 1 -- 172.21.15.1:0/192774478 shutdown_connections 2024-09-16T00:35:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.886+0000 7fa6050a6640 1 --2- 172.21.15.1:0/192774478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa600109120 0x7fa600110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.886+0000 7fa6050a6640 1 --2- 172.21.15.1:0/192774478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6001049a0 0x7fa6001089f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.886+0000 7fa6050a6640 1 --2- 172.21.15.1:0/192774478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa600103ff0 0x7fa6001043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.886+0000 7fa6050a6640 1 -- 172.21.15.1:0/192774478 >> 172.21.15.1:0/192774478 conn(0x7fa6000ffa90 msgr2=0x7fa600101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.886+0000 7fa6050a6640 1 -- 172.21.15.1:0/192774478 shutdown_connections 2024-09-16T00:35:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.886+0000 7fa6050a6640 1 -- 172.21.15.1:0/192774478 wait complete. 2024-09-16T00:35:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.887+0000 7fa6050a6640 1 Processor -- start 2024-09-16T00:35:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.887+0000 7fa6050a6640 1 -- start start 2024-09-16T00:35:16.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa6050a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa600103ff0 0x7fa60019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa6050a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001049a0 0x7fa60019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa6050a6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa600109120 0x7fa60019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa6050a6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa600114170 con 0x7fa600103ff0 2024-09-16T00:35:16.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa6050a6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa600113ff0 con 0x7fa6001049a0 2024-09-16T00:35:16.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa6050a6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001142f0 con 0x7fa600109120 2024-09-16T00:35:16.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa600103ff0 0x7fa60019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa600103ff0 0x7fa60019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45760/0 (socket says 172.21.15.1:45760) 2024-09-16T00:35:16.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fed9d640 1 -- 172.21.15.1:0/105138021 learned_addr learned my addr 172.21.15.1:0/105138021 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5ff59e640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa600109120 0x7fa60019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fe59c640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001049a0 0x7fa60019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fed9d640 1 -- 172.21.15.1:0/105138021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa600109120 msgr2=0x7fa60019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa600109120 0x7fa60019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fed9d640 1 -- 172.21.15.1:0/105138021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001049a0 msgr2=0x7fa60019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001049a0 0x7fa60019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fed9d640 1 -- 172.21.15.1:0/105138021 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60019f4c0 con 0x7fa600103ff0 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fe59c640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001049a0 0x7fa60019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa600103ff0 0x7fa60019a510 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c920 tx=0x7fa5f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.888+0000 7fa5e7fff640 1 -- 172.21.15.1:0/105138021 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0007c70 con 0x7fa600103ff0 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.889+0000 7fa6050a6640 1 -- 172.21.15.1:0/105138021 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60019f7b0 con 0x7fa600103ff0 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.889+0000 7fa5e7fff640 1 -- 172.21.15.1:0/105138021 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f0007e10 con 0x7fa600103ff0 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.889+0000 7fa5e7fff640 1 -- 172.21.15.1:0/105138021 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0018890 con 0x7fa600103ff0 2024-09-16T00:35:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.889+0000 7fa6050a6640 1 -- 172.21.15.1:0/105138021 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6001a7090 con 0x7fa600103ff0 2024-09-16T00:35:16.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa5e7fff640 1 -- 172.21.15.1:0/105138021 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa5f0018b10 con 0x7fa600103ff0 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa60747f740 1 Processor -- start 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa60747f740 1 -- start start 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa60747f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5f4a0 0x55f337c5f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa60747f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c6a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa60747f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c6a680 0x55f337c6ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa60747f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f337bc52e0 con 0x55f337c5fdc0 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa60747f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f337b6cce0 con 0x55f337c5f4a0 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa60747f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f337bc12a0 con 0x55f337c6a680 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa5fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c6a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa5fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c6a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45774/0 (socket says 172.21.15.1:45774) 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.892+0000 7fa5fed9d640 1 -- 172.21.15.1:0/2129740904 learned_addr learned my addr 172.21.15.1:0/2129740904 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5ff59e640 1 --2- 172.21.15.1:0/2129740904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5f4a0 0x55f337c5f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5fe59c640 1 --2- 172.21.15.1:0/2129740904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c6a680 0x55f337c6ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5e7fff640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5e80a3240 0x7fa5e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5fed9d640 1 -- 172.21.15.1:0/2129740904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c6a680 msgr2=0x55f337c6ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/2129740904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c6a680 0x55f337c6ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5fed9d640 1 -- 172.21.15.1:0/2129740904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5f4a0 msgr2=0x55f337c5f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/2129740904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5f4a0 0x55f337c5f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5ff59e640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5e80a3240 0x7fa5e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5fed9d640 1 -- 172.21.15.1:0/2129740904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f337b51440 con 0x55f337c5fdc0 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5e7fff640 1 -- 172.21.15.1:0/105138021 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa5f00cfe60 con 0x7fa600103ff0 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/2129740904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c6a140 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0049dd0 tx=0x7fa5f0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b5561df37dfc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5e57fa640 1 -- 172.21.15.1:0/2129740904 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f009b9e0 con 0x55f337c5fdc0 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5ff59e640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5e80a3240 0x7fa5e80a5700 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8014610 tx=0x7fa5f8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5e57fa640 1 -- 172.21.15.1:0/2129740904 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f0074070 con 0x55f337c5fdc0 2024-09-16T00:35:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.893+0000 7fa5e57fa640 1 -- 172.21.15.1:0/2129740904 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00475f0 con 0x55f337c5fdc0 2024-09-16T00:35:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.894+0000 7fa60747f740 1 -- 172.21.15.1:0/2129740904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 msgr2=0x55f337c6a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.894+0000 7fa60747f740 1 --2- 172.21.15.1:0/2129740904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c6a140 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0049dd0 tx=0x7fa5f0020000 comp rx=0 tx=0).stop 2024-09-16T00:35:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.894+0000 7fa60747f740 1 -- 172.21.15.1:0/2129740904 shutdown_connections 2024-09-16T00:35:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.894+0000 7fa60747f740 1 --2- 172.21.15.1:0/2129740904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c6a680 0x55f337c6ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.894+0000 7fa60747f740 1 --2- 172.21.15.1:0/2129740904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c6a140 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.894+0000 7fa60747f740 1 --2- 172.21.15.1:0/2129740904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5f4a0 0x55f337c5f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.894+0000 7fa60747f740 1 -- 172.21.15.1:0/2129740904 >> 172.21.15.1:0/2129740904 conn(0x55f337c5ba30 msgr2=0x55f337c5be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.894+0000 7fa60747f740 1 -- 172.21.15.1:0/2129740904 shutdown_connections 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.894+0000 7fa60747f740 1 -- 172.21.15.1:0/2129740904 wait complete. 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa60747f740 1 -- start start 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa60747f740 1 Processor -- start 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa60747f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5f4a0 0x55f337c69f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa60747f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c67260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa60747f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 0x55f337c677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa60747f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f337c6d250 con 0x55f337c5fdc0 2024-09-16T00:35:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa60747f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f337b80050 con 0x55f337c6a680 2024-09-16T00:35:16.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa60747f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f337c6d3d0 con 0x55f337c5f4a0 2024-09-16T00:35:16.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa60747f740 10 client.0 _ll_register_callbacks cb 0x55f337bc0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:16.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c67260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c67260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45790/0 (socket says 172.21.15.1:45790) 2024-09-16T00:35:16.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fe59c640 1 -- 172.21.15.1:0/3012507482 learned_addr learned my addr 172.21.15.1:0/3012507482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:16.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5ff59e640 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 0x55f337c677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5f4a0 0x55f337c69f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fe59c640 1 -- 172.21.15.1:0/3012507482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5f4a0 msgr2=0x55f337c69f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fe59c640 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5f4a0 0x55f337c69f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fe59c640 1 -- 172.21.15.1:0/3012507482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 msgr2=0x55f337c677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fe59c640 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 0x55f337c677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fe59c640 1 -- 172.21.15.1:0/3012507482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3379363c0 con 0x55f337c5fdc0 2024-09-16T00:35:16.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.895+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5f4a0 0x55f337c69f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:16.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa5fe59c640 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c67260 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009230 tx=0x7fa5f40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91f41856192fbc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:16.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa5e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:16.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/3012507482 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4035070 con 0x55f337c5fdc0 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa60747f740 1 -- 172.21.15.1:0/3012507482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f0074070 con 0x55f337c5fdc0 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/3012507482 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa5f4003c90 con 0x55f337c5fdc0 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/3012507482 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa5f4005d50 con 0x55f337c5fdc0 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa60747f740 10 client.17082 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa60747f740 1 -- 172.21.15.1:0/3012507482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f337c78c50 con 0x55f337c5fdc0 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/3012507482 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f0074070 con 0x55f337c5fdc0 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/3012507482 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa5f403b6b0 con 0x55f337c5fdc0 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa60747f740 10 client.17082 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa60747f740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.896+0000 7fa60747f740 20 client.17082 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.897+0000 7fa60747f740 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f337c7a060 0x55f337c7c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.897+0000 7fa60747f740 4 client.17082 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.897+0000 7fa60747f740 1 -- 172.21.15.1:0/3012507482 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f337bd2850 con 0x55f337c7a060 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.897+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f337c7a060 0x55f337c7c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.897+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f337c7a060 0x55f337c7c520 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97bf2456ad15a39 server_cookie=9b6b67d8e3327ab1 in_seq=0 out_seq=0 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.897+0000 7fa5e2ff5640 10 client.17082 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/3012507482 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f337bd2850 con 0x55f337c7a060 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa5e2ff5640 10 client.17082 handle_command_reply: tid=0 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 client.17082 shutdown 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 -- 172.21.15.1:0/3012507482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 msgr2=0x55f337c67260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c67260 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009230 tx=0x7fa5f40038a0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 -- 172.21.15.1:0/3012507482 shutdown_connections 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f337c7a060 0x55f337c7c520 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 0x55f337c677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5fdc0 0x55f337c67260 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 --2- 172.21.15.1:0/3012507482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5f4a0 0x55f337c69f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 -- 172.21.15.1:0/3012507482 >> 172.21.15.1:0/3012507482 conn(0x55f337c5ba30 msgr2=0x55f337c61710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa5e2ff5640 0 client.17082 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 -- 172.21.15.1:0/3012507482 shutdown_connections 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 1 -- 172.21.15.1:0/3012507482 wait complete. 2024-09-16T00:35:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.898+0000 7fa60747f740 20 client.17082 trim_cache size 0 max 16384 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.915+0000 7fa60747f740 1 Processor -- start 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.915+0000 7fa60747f740 1 -- start start 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa60747f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c764b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa60747f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5fdc0 0x55f337c67450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa60747f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 0x55f337c66240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa60747f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b37d0 con 0x55f337c5f4a0 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa60747f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b10a0 con 0x55f337c6a680 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa60747f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001aeef0 con 0x55f337c5fdc0 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c764b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45796/0 (socket says 172.21.15.1:45796) 2024-09-16T00:35:16.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5fed9d640 1 -- 172.21.15.1:0/515180722 learned_addr learned my addr 172.21.15.1:0/515180722 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5fed9d640 1 -- 172.21.15.1:0/515180722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5fdc0 msgr2=0x55f337c67450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/515180722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5fdc0 0x55f337c67450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5ff59e640 1 --2- 172.21.15.1:0/515180722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 0x55f337c66240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5fed9d640 1 -- 172.21.15.1:0/515180722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 msgr2=0x55f337c66240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/515180722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 0x55f337c66240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5fed9d640 1 -- 172.21.15.1:0/515180722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f337c6cdd0 con 0x55f337c5f4a0 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5ff59e640 1 --2- 172.21.15.1:0/515180722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 0x55f337c66240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/515180722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c764b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00cd5b0 tx=0x7fa5f00bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12171a2e8d13b126 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5e4ff9640 1 -- 172.21.15.1:0/515180722 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f009c780 con 0x55f337c5f4a0 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5e4ff9640 1 -- 172.21.15.1:0/515180722 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f009c080 con 0x55f337c5f4a0 2024-09-16T00:35:16.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.916+0000 7fa5e4ff9640 1 -- 172.21.15.1:0/515180722 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f009a380 con 0x55f337c5f4a0 2024-09-16T00:35:16.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 -- 172.21.15.1:0/515180722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 msgr2=0x55f337c764b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 --2- 172.21.15.1:0/515180722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c764b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00cd5b0 tx=0x7fa5f00bc040 comp rx=0 tx=0).stop 2024-09-16T00:35:16.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 -- 172.21.15.1:0/515180722 shutdown_connections 2024-09-16T00:35:16.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 --2- 172.21.15.1:0/515180722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c6a680 0x55f337c66240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 --2- 172.21.15.1:0/515180722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c5fdc0 0x55f337c67450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 --2- 172.21.15.1:0/515180722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c764b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 -- 172.21.15.1:0/515180722 >> 172.21.15.1:0/515180722 conn(0x55f337c5ba30 msgr2=0x55f337c65b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:16.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 -- 172.21.15.1:0/515180722 shutdown_connections 2024-09-16T00:35:16.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 -- 172.21.15.1:0/515180722 wait complete. 2024-09-16T00:35:16.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.917+0000 7fa60747f740 1 -- start start 2024-09-16T00:35:16.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa60747f740 1 Processor -- start 2024-09-16T00:35:16.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa60747f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c72db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa60747f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5fdc0 0x55f337c6e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa60747f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c6a680 0x55f337c6e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa60747f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001af560 con 0x55f337c5f4a0 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa60747f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b09d0 con 0x55f337c5fdc0 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa60747f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001afc30 con 0x55f337c6a680 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa60747f740 10 client.0 _ll_register_callbacks cb 0x7fa5f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c72db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c72db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45800/0 (socket says 172.21.15.1:45800) 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fed9d640 1 -- 172.21.15.1:0/781921491 learned_addr learned my addr 172.21.15.1:0/781921491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fed9d640 1 -- 172.21.15.1:0/781921491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c6a680 msgr2=0x55f337c6e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c6a680 0x55f337c6e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fed9d640 1 -- 172.21.15.1:0/781921491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5fdc0 msgr2=0x55f337c6e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fe59c640 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5fdc0 0x55f337c6e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5fdc0 0x55f337c6e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fed9d640 1 -- 172.21.15.1:0/781921491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f337c6ee70 con 0x55f337c5f4a0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5fed9d640 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c72db0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c8f0 tx=0x7fa5f009bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52a85f1c0d8a70a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.918+0000 7fa5e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/781921491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f009c780 con 0x55f337c5f4a0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/781921491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa5f00bc0b0 con 0x55f337c5f4a0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa60747f740 1 -- 172.21.15.1:0/781921491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f009c080 con 0x55f337c5f4a0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/781921491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa5f009a120 con 0x55f337c5f4a0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa60747f740 10 client.17088 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa60747f740 1 -- 172.21.15.1:0/781921491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f337ce4d70 con 0x55f337c5f4a0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/781921491 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f009c080 con 0x55f337c5f4a0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/781921491 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa5f00986f0 con 0x55f337c5f4a0 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa60747f740 10 client.17088 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa60747f740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa60747f740 20 client.17088 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa60747f740 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f337ce5050 0x55f337ce7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa60747f740 4 client.17088 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.919+0000 7fa60747f740 1 -- 172.21.15.1:0/781921491 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f337c67b30 con 0x55f337ce5050 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa5fe59c640 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f337ce5050 0x55f337ce7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa5e2ff5640 10 client.17088 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa5fe59c640 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f337ce5050 0x55f337ce7440 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b8fc56f89e5f8 server_cookie=4be2433ab7cfb2d8 in_seq=0 out_seq=0 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa5e2ff5640 1 -- 172.21.15.1:0/781921491 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f337c67b30 con 0x55f337ce5050 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa5e2ff5640 10 client.17088 handle_command_reply: tid=0 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa60747f740 1 client.17088 shutdown 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa60747f740 1 -- 172.21.15.1:0/781921491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 msgr2=0x55f337c72db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa60747f740 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c72db0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c8f0 tx=0x7fa5f009bbc0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa60747f740 1 -- 172.21.15.1:0/781921491 shutdown_connections 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa60747f740 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f337ce5050 0x55f337ce7440 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa60747f740 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f337c6a680 0x55f337c6e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa60747f740 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f337c5fdc0 0x55f337c6e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa5e2ff5640 0 client.17088 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa60747f740 1 --2- 172.21.15.1:0/781921491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f337c5f4a0 0x55f337c72db0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.920+0000 7fa60747f740 1 -- 172.21.15.1:0/781921491 >> 172.21.15.1:0/781921491 conn(0x55f337c5ba30 msgr2=0x55f337c66350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa60747f740 1 -- 172.21.15.1:0/781921491 shutdown_connections 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa60747f740 1 -- 172.21.15.1:0/781921491 wait complete. 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa60747f740 20 client.17088 trim_cache size 0 max 16384 2024-09-16T00:35:16.928 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:35:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:35:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:35:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "64923857-c344-49ad-a59c-d0b8ee525ef3": { 2024-09-16T00:35:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:35:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "64923857-c344-49ad-a59c-d0b8ee525ef3", 2024-09-16T00:35:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:35:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:35:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:35:16.929 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 -- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5e80a3240 msgr2=0x7fa5e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5e80a3240 0x7fa5e80a5700 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8014610 tx=0x7fa5f8004ab0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 -- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa600103ff0 msgr2=0x7fa60019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa600103ff0 0x7fa60019a510 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c920 tx=0x7fa5f000cde0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 -- 172.21.15.1:0/105138021 shutdown_connections 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5e80a3240 0x7fa5e80a5700 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa600109120 0x7fa60019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001049a0 0x7fa60019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 --2- 172.21.15.1:0/105138021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa600103ff0 0x7fa60019a510 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 -- 172.21.15.1:0/105138021 >> 172.21.15.1:0/105138021 conn(0x7fa6000ffa90 msgr2=0x7fa600100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 -- 172.21.15.1:0/105138021 shutdown_connections 2024-09-16T00:35:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:16.921+0000 7fa6050a6640 1 -- 172.21.15.1:0/105138021 wait complete. 2024-09-16T00:35:17.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:16 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:35:17.265 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:16 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:17.265 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:16 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:35:17.265 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:16 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:17.266 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:16 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:35:17.266 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:16 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:17.266 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:16 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:35:17.266 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:16 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:17.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:16 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:35:17.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:16 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:17.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:16 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:35:17.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:16 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:17.818 INFO:tasks.cephfs.filesystem:scrub status for tag:64923857-c344-49ad-a59c-d0b8ee525ef3 - {'path': '/', 'tag': '64923857-c344-49ad-a59c-d0b8ee525ef3', 'options': 'recursive,force'} 2024-09-16T00:35:18.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:17 smithi001 ceph-mon[30934]: pgmap v557: 129 pgs: 129 active+clean; 3.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:18.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:17 smithi153 ceph-mon[35538]: pgmap v557: 129 pgs: 129 active+clean; 3.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:17 smithi019 ceph-mon[35186]: pgmap v557: 129 pgs: 129 active+clean; 3.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:19.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:18 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:35:19.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:18 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:35:19.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:18 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:35:20.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:19 smithi001 ceph-mon[30934]: pgmap v558: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:35:20.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:19 smithi153 ceph-mon[35538]: pgmap v558: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:35:20.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:19 smithi019 ceph-mon[35186]: pgmap v558: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:35:22.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:21 smithi001 ceph-mon[30934]: pgmap v559: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:22.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:21 smithi153 ceph-mon[35538]: pgmap v559: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:22.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:21 smithi019 ceph-mon[35186]: pgmap v559: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:24.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:23 smithi001 ceph-mon[30934]: pgmap v560: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:24.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:23 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:24.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:23 smithi153 ceph-mon[35538]: pgmap v560: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:24.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:23 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:24.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:23 smithi019 ceph-mon[35186]: pgmap v560: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:24.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:23 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:25.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.203 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.203 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.203 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:25.203 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:25.203 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:24 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:35:25.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:25.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:24 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:35:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:25.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:25.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:25.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:24 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:35:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:25 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-16T00:35:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:25 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi019 to 177.0M 2024-09-16T00:35:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:25 smithi001 ceph-mon[30934]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-16T00:35:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:25 smithi001 ceph-mon[30934]: Unable to set osd_memory_target on smithi019 to 185700659: error parsing value: Value '185700659' is below minimum 939524096 2024-09-16T00:35:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:25 smithi001 ceph-mon[30934]: pgmap v561: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:26.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:25 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-16T00:35:26.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:25 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi019 to 177.0M 2024-09-16T00:35:26.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:25 smithi153 ceph-mon[35538]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-16T00:35:26.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:25 smithi153 ceph-mon[35538]: Unable to set osd_memory_target on smithi019 to 185700659: error parsing value: Value '185700659' is below minimum 939524096 2024-09-16T00:35:26.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:25 smithi153 ceph-mon[35538]: pgmap v561: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:26.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:25 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-16T00:35:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:25 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi019 to 177.0M 2024-09-16T00:35:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:25 smithi019 ceph-mon[35186]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-16T00:35:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:25 smithi019 ceph-mon[35186]: Unable to set osd_memory_target on smithi019 to 185700659: error parsing value: Value '185700659' is below minimum 939524096 2024-09-16T00:35:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:25 smithi019 ceph-mon[35186]: pgmap v561: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:28.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:27 smithi001 ceph-mon[30934]: pgmap v562: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:28.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:27 smithi153 ceph-mon[35538]: pgmap v562: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:28.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:27 smithi019 ceph-mon[35186]: pgmap v562: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:30.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:29 smithi001 ceph-mon[30934]: pgmap v563: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:30.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:29 smithi153 ceph-mon[35538]: pgmap v563: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:30.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:29 smithi019 ceph-mon[35186]: pgmap v563: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:31 smithi153 ceph-mon[35538]: pgmap v564: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:31 smithi019 ceph-mon[35186]: pgmap v564: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:31 smithi001 ceph-mon[30934]: pgmap v564: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:34.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:33 smithi153 ceph-mon[35538]: pgmap v565: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:33 smithi019 ceph-mon[35186]: pgmap v565: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:33 smithi001 ceph-mon[30934]: pgmap v565: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:36.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:35 smithi153 ceph-mon[35538]: pgmap v566: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:35 smithi019 ceph-mon[35186]: pgmap v566: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:35 smithi001 ceph-mon[30934]: pgmap v566: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:38.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:37 smithi019 ceph-mon[35186]: pgmap v567: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:38.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:37 smithi001 ceph-mon[30934]: pgmap v567: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:37 smithi153 ceph-mon[35538]: pgmap v567: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:40.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:39 smithi019 ceph-mon[35186]: pgmap v568: 129 pgs: 129 active+clean; 3.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:40.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:39 smithi001 ceph-mon[30934]: pgmap v568: 129 pgs: 129 active+clean; 3.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:39 smithi153 ceph-mon[35538]: pgmap v568: 129 pgs: 129 active+clean; 3.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:35:42.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:41 smithi019 ceph-mon[35186]: pgmap v569: 129 pgs: 129 active+clean; 3.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:42.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:41 smithi001 ceph-mon[30934]: pgmap v569: 129 pgs: 129 active+clean; 3.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:42.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:41 smithi153 ceph-mon[35538]: pgmap v569: 129 pgs: 129 active+clean; 3.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:44.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:43 smithi019 ceph-mon[35186]: pgmap v570: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:44.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:43 smithi001 ceph-mon[30934]: pgmap v570: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:43 smithi153 ceph-mon[35538]: pgmap v570: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:35:46.254 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:45 smithi001 ceph-mon[30934]: pgmap v571: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:46.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:45 smithi019 ceph-mon[35186]: pgmap v571: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:45 smithi153 ceph-mon[35538]: pgmap v571: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:47.819 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:35:48.090 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:48.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:47 smithi001 ceph-mon[30934]: pgmap v572: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:48.362 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:47 smithi019 ceph-mon[35186]: pgmap v572: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:48.471 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:47 smithi153 ceph-mon[35538]: pgmap v572: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:48.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.619+0000 7f7025bf3640 1 -- 172.21.15.1:0/2034271512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020105a90 msgr2=0x7f7020105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:48.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.619+0000 7f7025bf3640 1 --2- 172.21.15.1:0/2034271512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020105a90 0x7f7020105e70 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7010009900 tx=0x7f701002a410 comp rx=0 tx=0).stop 2024-09-16T00:35:48.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.619+0000 7f7025bf3640 1 -- 172.21.15.1:0/2034271512 shutdown_connections 2024-09-16T00:35:48.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.619+0000 7f7025bf3640 1 --2- 172.21.15.1:0/2034271512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7020101bc0 0x7f702010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.619+0000 7f7025bf3640 1 --2- 172.21.15.1:0/2034271512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7020101240 0x7f7020101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.619+0000 7f7025bf3640 1 --2- 172.21.15.1:0/2034271512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020105a90 0x7f7020105e70 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.619+0000 7f7025bf3640 1 -- 172.21.15.1:0/2034271512 >> 172.21.15.1:0/2034271512 conn(0x7f7020077210 msgr2=0x7f70200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.619+0000 7f7025bf3640 1 -- 172.21.15.1:0/2034271512 shutdown_connections 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.619+0000 7f7025bf3640 1 -- 172.21.15.1:0/2034271512 wait complete. 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.620+0000 7f7025bf3640 1 Processor -- start 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.620+0000 7f7025bf3640 1 -- start start 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f7025bf3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7020101240 0x7f70201993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f7025bf3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020101bc0 0x7f7020199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f7025bf3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7020105a90 0x7f70201a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f7025bf3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7020110f70 con 0x7f7020101bc0 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f7025bf3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7020110df0 con 0x7f7020105a90 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f7025bf3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f70201110f0 con 0x7f7020101240 2024-09-16T00:35:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020101bc0 0x7f7020199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020101bc0 0x7f7020199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53888/0 (socket says 172.21.15.1:53888) 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701effd640 1 -- 172.21.15.1:0/4177454543 learned_addr learned my addr 172.21.15.1:0/4177454543 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701f7fe640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7020101240 0x7f70201993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701ffff640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7020105a90 0x7f70201a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701effd640 1 -- 172.21.15.1:0/4177454543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7020101240 msgr2=0x7f70201993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701effd640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7020101240 0x7f70201993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701effd640 1 -- 172.21.15.1:0/4177454543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7020105a90 msgr2=0x7f70201a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701effd640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7020105a90 0x7f70201a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701effd640 1 -- 172.21.15.1:0/4177454543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70201a4980 con 0x7f7020101bc0 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701ffff640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7020105a90 0x7f70201a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701effd640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020101bc0 0x7f7020199900 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f701400d950 tx=0x7f701400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701cff9640 1 -- 172.21.15.1:0/4177454543 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014017070 con 0x7f7020101bc0 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701cff9640 1 -- 172.21.15.1:0/4177454543 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7014009c30 con 0x7f7020101bc0 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f7025bf3640 1 -- 172.21.15.1:0/4177454543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70201a4b50 con 0x7f7020101bc0 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.621+0000 7f701cff9640 1 -- 172.21.15.1:0/4177454543 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701400f6b0 con 0x7f7020101bc0 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.622+0000 7f7025bf3640 1 -- 172.21.15.1:0/4177454543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70201a5090 con 0x7f7020101bc0 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.624+0000 7f701cff9640 1 -- 172.21.15.1:0/4177454543 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f70140109d0 con 0x7f7020101bc0 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.624+0000 7f7027fcc740 1 Processor -- start 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.624+0000 7f7027fcc740 1 -- start start 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.624+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f365734880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.624+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f36573f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.624+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 0x55f365741a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.624+0000 7f7027fcc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f36569a2e0 con 0x55f365734dc0 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.624+0000 7f7027fcc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f365641ce0 con 0x55f36573f680 2024-09-16T00:35:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.624+0000 7f7027fcc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f3656962a0 con 0x55f3657344a0 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f36573f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f36573f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53894/0 (socket says 172.21.15.1:53894) 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701f7fe640 1 -- 172.21.15.1:0/1132267136 learned_addr learned my addr 172.21.15.1:0/1132267136 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701effd640 1 --2- 172.21.15.1:0/1132267136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 0x55f365741a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701ffff640 1 --2- 172.21.15.1:0/1132267136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f365734880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701cff9640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70080a32c0 0x7f70080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701f7fe640 1 -- 172.21.15.1:0/1132267136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 msgr2=0x55f365734880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701f7fe640 1 --2- 172.21.15.1:0/1132267136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f365734880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701f7fe640 1 -- 172.21.15.1:0/1132267136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 msgr2=0x55f365741a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701f7fe640 1 --2- 172.21.15.1:0/1132267136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 0x55f365741a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701f7fe640 1 -- 172.21.15.1:0/1132267136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f365626440 con 0x55f365734dc0 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701ffff640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70080a32c0 0x7f70080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701cff9640 1 -- 172.21.15.1:0/4177454543 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f70140cf8a0 con 0x7f7020101bc0 2024-09-16T00:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f701f7fe640 1 --2- 172.21.15.1:0/1132267136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f36573f140 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f701002a930 tx=0x7f70100040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c968c77ed67ba51 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f7005ffb640 1 -- 172.21.15.1:0/1132267136 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010035030 con 0x55f365734dc0 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f7005ffb640 1 -- 172.21.15.1:0/1132267136 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7010040070 con 0x55f365734dc0 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.625+0000 7f7005ffb640 1 -- 172.21.15.1:0/1132267136 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70100319d0 con 0x55f365734dc0 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 -- 172.21.15.1:0/1132267136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 msgr2=0x55f36573f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f701ffff640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70080a32c0 0x7f70080a5780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7018005fd0 tx=0x7f7018004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1132267136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f36573f140 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f701002a930 tx=0x7f70100040c0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 -- 172.21.15.1:0/1132267136 shutdown_connections 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1132267136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 0x55f365741a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1132267136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f36573f140 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1132267136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f365734880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 -- 172.21.15.1:0/1132267136 >> 172.21.15.1:0/1132267136 conn(0x55f365730a30 msgr2=0x55f365730e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 -- 172.21.15.1:0/1132267136 shutdown_connections 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 -- 172.21.15.1:0/1132267136 wait complete. 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.626+0000 7f7027fcc740 1 -- start start 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f7027fcc740 1 Processor -- start 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f36573ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f365734dc0 0x55f36573c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f36573c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f7027fcc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f365742250 con 0x55f36573f680 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f7027fcc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f365655050 con 0x55f365734dc0 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f7027fcc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f3657423d0 con 0x55f3657344a0 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f7027fcc740 10 client.0 _ll_register_callbacks cb 0x55f365695160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f701ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f36573c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f701ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f36573c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53904/0 (socket says 172.21.15.1:53904) 2024-09-16T00:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f701ffff640 1 -- 172.21.15.1:0/1137421156 learned_addr learned my addr 172.21.15.1:0/1137421156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f701effd640 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f365734dc0 0x55f36573c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.627+0000 7f701f7fe640 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f36573ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f701ffff640 1 -- 172.21.15.1:0/1137421156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 msgr2=0x55f36573ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f701ffff640 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f36573ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f701ffff640 1 -- 172.21.15.1:0/1137421156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f365734dc0 msgr2=0x55f36573c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f701ffff640 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f365734dc0 0x55f36573c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f701ffff640 1 -- 172.21.15.1:0/1137421156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f36540b3c0 con 0x55f36573f680 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f701ffff640 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f36573c740 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7018012850 tx=0x7f7018004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8038db4828e2436 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f6ffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f6ffeffd640 1 -- 172.21.15.1:0/1137421156 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701801cb60 con 0x55f36573f680 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f6ffeffd640 1 -- 172.21.15.1:0/1137421156 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f701801cd00 con 0x55f36573f680 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f7027fcc740 1 -- 172.21.15.1:0/1137421156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7010040070 con 0x55f36573f680 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f6ffeffd640 1 -- 172.21.15.1:0/1137421156 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7018030050 con 0x55f36573f680 2024-09-16T00:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f7027fcc740 10 client.17100 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f7027fcc740 1 -- 172.21.15.1:0/1137421156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f36574dc00 con 0x55f36573f680 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.628+0000 7f6ffeffd640 1 -- 172.21.15.1:0/1137421156 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7010040070 con 0x55f36573f680 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f6ffeffd640 1 -- 172.21.15.1:0/1137421156 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f701801a5d0 con 0x55f36573f680 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f7027fcc740 10 client.17100 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f7027fcc740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f7027fcc740 20 client.17100 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f36574f0c0 0x55f365751560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f7027fcc740 4 client.17100 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f7027fcc740 1 -- 172.21.15.1:0/1137421156 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3656a7850 con 0x55f36574f0c0 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f701f7fe640 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f36574f0c0 0x55f365751560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f6ffeffd640 10 client.17100 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.629+0000 7f701f7fe640 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f36574f0c0 0x55f365751560 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229ba43e2609dab8 server_cookie=dff89152f03af138 in_seq=0 out_seq=0 2024-09-16T00:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f6ffeffd640 1 -- 172.21.15.1:0/1137421156 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3656a7850 con 0x55f36574f0c0 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f6ffeffd640 10 client.17100 handle_command_reply: tid=0 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 client.17100 shutdown 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 -- 172.21.15.1:0/1137421156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 msgr2=0x55f36573c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f36573c740 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7018012850 tx=0x7f7018004ab0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 -- 172.21.15.1:0/1137421156 shutdown_connections 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f36574f0c0 0x55f365751560 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f36573c740 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f365734dc0 0x55f36573c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1137421156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f36573ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f6ffeffd640 0 client.17100 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 -- 172.21.15.1:0/1137421156 >> 172.21.15.1:0/1137421156 conn(0x55f365730a30 msgr2=0x55f365736710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 -- 172.21.15.1:0/1137421156 shutdown_connections 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 1 -- 172.21.15.1:0/1137421156 wait complete. 2024-09-16T00:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.630+0000 7f7027fcc740 20 client.17100 trim_cache size 0 max 16384 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f7027fcc740 1 Processor -- start 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f7027fcc740 1 -- start start 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f36573aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f365756e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 0x55f3657573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f7027fcc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70201b17d0 con 0x55f365734dc0 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f7027fcc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70201af0a0 con 0x55f36573f680 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f7027fcc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f70201acef0 con 0x55f3657344a0 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f365756e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f365756e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53908/0 (socket says 172.21.15.1:53908) 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701effd640 1 -- 172.21.15.1:0/1587352500 learned_addr learned my addr 172.21.15.1:0/1587352500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701f7fe640 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f36573aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701ffff640 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 0x55f3657573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701effd640 1 -- 172.21.15.1:0/1587352500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 msgr2=0x55f36573aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701effd640 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f36573aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701effd640 1 -- 172.21.15.1:0/1587352500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 msgr2=0x55f3657573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701effd640 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 0x55f3657573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701effd640 1 -- 172.21.15.1:0/1587352500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f36573b1b0 con 0x55f365734dc0 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701ffff640 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 0x55f3657573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.648+0000 7f701effd640 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f365756e80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f7014010fd0 tx=0x7f7014097e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b28e66ec7e9a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f70057fa640 1 -- 172.21.15.1:0/1587352500 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014020040 con 0x55f365734dc0 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f70057fa640 1 -- 172.21.15.1:0/1587352500 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701409bac0 con 0x55f365734dc0 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f70057fa640 1 -- 172.21.15.1:0/1587352500 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140860b0 con 0x55f365734dc0 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f7027fcc740 1 -- 172.21.15.1:0/1587352500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 msgr2=0x55f365756e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f365756e80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f7014010fd0 tx=0x7f7014097e90 comp rx=0 tx=0).stop 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f7027fcc740 1 -- 172.21.15.1:0/1587352500 shutdown_connections 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f36573f680 0x55f3657573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f365734dc0 0x55f365756e80 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f7027fcc740 1 --2- 172.21.15.1:0/1587352500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3657344a0 0x55f36573aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f7027fcc740 1 -- 172.21.15.1:0/1587352500 >> 172.21.15.1:0/1587352500 conn(0x55f365730a30 msgr2=0x55f36573e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f7027fcc740 1 -- 172.21.15.1:0/1587352500 shutdown_connections 2024-09-16T00:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.649+0000 7f7027fcc740 1 -- 172.21.15.1:0/1587352500 wait complete. 2024-09-16T00:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f7027fcc740 1 -- start start 2024-09-16T00:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f7027fcc740 1 Processor -- start 2024-09-16T00:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3657344a0 0x55f3657455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f365734dc0 0x55f365745b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f7027fcc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f365746060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f7027fcc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70201ad560 con 0x55f36573f680 2024-09-16T00:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f7027fcc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70201ae9d0 con 0x55f3657344a0 2024-09-16T00:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f7027fcc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f70201adc30 con 0x55f365734dc0 2024-09-16T00:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f7027fcc740 10 client.0 _ll_register_callbacks cb 0x7f701800df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f701ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f365746060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f701ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f365746060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53922/0 (socket says 172.21.15.1:53922) 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.650+0000 7f701ffff640 1 -- 172.21.15.1:0/3052787314 learned_addr learned my addr 172.21.15.1:0/3052787314 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f701f7fe640 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3657344a0 0x55f3657455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f701effd640 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f365734dc0 0x55f365745b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f701ffff640 1 -- 172.21.15.1:0/3052787314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f365734dc0 msgr2=0x55f365745b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f701ffff640 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f365734dc0 0x55f365745b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f701ffff640 1 -- 172.21.15.1:0/3052787314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3657344a0 msgr2=0x55f3657455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f701ffff640 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3657344a0 0x55f3657455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f701ffff640 1 -- 172.21.15.1:0/3052787314 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3657b9060 con 0x55f36573f680 2024-09-16T00:35:48.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f701effd640 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f365734dc0 0x55f365745b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:48.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f701ffff640 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f365746060 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f701800dbe0 tx=0x7f701801ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a212c64274eaf6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:48.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f6ffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:48.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f6ffeffd640 1 -- 172.21.15.1:0/3052787314 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701803b030 con 0x55f36573f680 2024-09-16T00:35:48.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f6ffeffd640 1 -- 172.21.15.1:0/3052787314 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f701801a650 con 0x55f36573f680 2024-09-16T00:35:48.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f7027fcc740 1 -- 172.21.15.1:0/3052787314 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f701409bac0 con 0x55f36573f680 2024-09-16T00:35:48.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f6ffeffd640 1 -- 172.21.15.1:0/3052787314 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7018045050 con 0x55f36573f680 2024-09-16T00:35:48.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f7027fcc740 10 client.17106 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:48.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.651+0000 7f7027fcc740 1 -- 172.21.15.1:0/3052787314 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f3657ba2a0 con 0x55f36573f680 2024-09-16T00:35:48.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f6ffeffd640 1 -- 172.21.15.1:0/3052787314 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701409bac0 con 0x55f36573f680 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f6ffeffd640 1 -- 172.21.15.1:0/3052787314 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f701801d720 con 0x55f36573f680 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f7027fcc740 10 client.17106 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f7027fcc740 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f7027fcc740 20 client.17106 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f7027fcc740 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f3657baa10 0x55f3657bce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f7027fcc740 4 client.17106 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f7027fcc740 1 -- 172.21.15.1:0/3052787314 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3656a1d60 con 0x55f3657baa10 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f701f7fe640 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f3657baa10 0x55f3657bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f701f7fe640 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f3657baa10 0x55f3657bce30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5883dd96108330c0 server_cookie=2a7e246bd51a3875 in_seq=0 out_seq=0 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.652+0000 7f6ffeffd640 10 client.17106 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f6ffeffd640 1 -- 172.21.15.1:0/3052787314 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f3656a1d60 con 0x55f3657baa10 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f6ffeffd640 10 client.17106 handle_command_reply: tid=0 2024-09-16T00:35:48.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 client.17106 shutdown 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 -- 172.21.15.1:0/3052787314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 msgr2=0x55f365746060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f365746060 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f701800dbe0 tx=0x7f701801ae90 comp rx=0 tx=0).stop 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 -- 172.21.15.1:0/3052787314 shutdown_connections 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f3657baa10 0x55f3657bce30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f36573f680 0x55f365746060 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f365734dc0 0x55f365745b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 --2- 172.21.15.1:0/3052787314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3657344a0 0x55f3657455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 -- 172.21.15.1:0/3052787314 >> 172.21.15.1:0/3052787314 conn(0x55f365730a30 msgr2=0x55f36572f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f6ffeffd640 0 client.17106 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 -- 172.21.15.1:0/3052787314 shutdown_connections 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 1 -- 172.21.15.1:0/3052787314 wait complete. 2024-09-16T00:35:48.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.653+0000 7f7027fcc740 20 client.17106 trim_cache size 0 max 16384 2024-09-16T00:35:48.665 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:35:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:35:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:35:48.665 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:35:48.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 -- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70080a32c0 msgr2=0x7f70080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70080a32c0 0x7f70080a5780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7018005fd0 tx=0x7f7018004a40 comp rx=0 tx=0).stop 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 -- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020101bc0 msgr2=0x7f7020199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020101bc0 0x7f7020199900 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f701400d950 tx=0x7f701400de10 comp rx=0 tx=0).stop 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 -- 172.21.15.1:0/4177454543 shutdown_connections 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7020105a90 0x7f70201a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7020101bc0 0x7f7020199900 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70080a32c0 0x7f70080a5780 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 --2- 172.21.15.1:0/4177454543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7020101240 0x7f70201993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 -- 172.21.15.1:0/4177454543 >> 172.21.15.1:0/4177454543 conn(0x7f7020077210 msgr2=0x7f7020103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 -- 172.21.15.1:0/4177454543 shutdown_connections 2024-09-16T00:35:48.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:48.654+0000 7f7025bf3640 1 -- 172.21.15.1:0/4177454543 wait complete. 2024-09-16T00:35:49.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:35:49.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:35:49.810 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:50.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:49 smithi001 ceph-mon[30934]: pgmap v573: 129 pgs: 129 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:35:50.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.319+0000 7fae745c1640 1 -- 172.21.15.1:0/970381938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 msgr2=0x7fae6c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:50.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.319+0000 7fae745c1640 1 --2- 172.21.15.1:0/970381938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 0x7fae6c10d130 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fae6400adb0 tx=0x7fae6402b7a0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.319+0000 7fae745c1640 1 -- 172.21.15.1:0/970381938 shutdown_connections 2024-09-16T00:35:50.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.319+0000 7fae745c1640 1 --2- 172.21.15.1:0/970381938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 0x7fae6c10d130 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.319+0000 7fae745c1640 1 --2- 172.21.15.1:0/970381938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae6c100e30 0x7fae6c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.319+0000 7fae745c1640 1 --2- 172.21.15.1:0/970381938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae6c100480 0x7fae6c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.319+0000 7fae745c1640 1 -- 172.21.15.1:0/970381938 >> 172.21.15.1:0/970381938 conn(0x7fae6c077210 msgr2=0x7fae6c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:50.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.319+0000 7fae745c1640 1 -- 172.21.15.1:0/970381938 shutdown_connections 2024-09-16T00:35:50.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.319+0000 7fae745c1640 1 -- 172.21.15.1:0/970381938 wait complete. 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae745c1640 1 Processor -- start 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae745c1640 1 -- start start 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae745c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae6c100480 0x7fae6c0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae745c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae6c100e30 0x7fae6c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae745c1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 0x7fae6c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae745c1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c10fd90 con 0x7fae6c1055b0 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae745c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c10fc10 con 0x7fae6c100480 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae745c1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c10ff10 con 0x7fae6c100e30 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae72b5e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 0x7fae6c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.320+0000 7fae72b5e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 0x7fae6c0fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43640/0 (socket says 172.21.15.1:43640) 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae72b5e640 1 -- 172.21.15.1:0/245420577 learned_addr learned my addr 172.21.15.1:0/245420577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:50.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae7235d640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae6c100480 0x7fae6c0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae71b5c640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae6c100e30 0x7fae6c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae72b5e640 1 -- 172.21.15.1:0/245420577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae6c100e30 msgr2=0x7fae6c0fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae72b5e640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae6c100e30 0x7fae6c0fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae72b5e640 1 -- 172.21.15.1:0/245420577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae6c100480 msgr2=0x7fae6c0fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae72b5e640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae6c100480 0x7fae6c0fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae72b5e640 1 -- 172.21.15.1:0/245420577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c1a6890 con 0x7fae6c1055b0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae72b5e640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 0x7fae6c0fd110 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fae64005cc0 tx=0x7fae64003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae637fe640 1 -- 172.21.15.1:0/245420577 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64040070 con 0x7fae6c1055b0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae637fe640 1 -- 172.21.15.1:0/245420577 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae640309f0 con 0x7fae6c1055b0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c1a6b20 con 0x7fae6c1055b0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae637fe640 1 -- 172.21.15.1:0/245420577 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64004600 con 0x7fae6c1055b0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.321+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c1a7000 con 0x7fae6c1055b0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.322+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6c100860 con 0x7fae6c1055b0 2024-09-16T00:35:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.323+0000 7fae637fe640 1 -- 172.21.15.1:0/245420577 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fae640328a0 con 0x7fae6c1055b0 2024-09-16T00:35:50.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.324+0000 7fae637fe640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fae580a3240 0x7fae580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:50.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.324+0000 7fae7235d640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fae580a3240 0x7fae580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:50.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.325+0000 7fae637fe640 1 -- 172.21.15.1:0/245420577 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fae640f1120 con 0x7fae6c1055b0 2024-09-16T00:35:50.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.325+0000 7fae7235d640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fae580a3240 0x7fae580a5700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fae6800bbe0 tx=0x7fae6800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:50.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.329+0000 7fae637fe640 1 -- 172.21.15.1:0/245420577 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae640bda40 con 0x7fae6c1055b0 2024-09-16T00:35:50.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:49 smithi019 ceph-mon[35186]: pgmap v573: 129 pgs: 129 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:35:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:49 smithi153 ceph-mon[35538]: pgmap v573: 129 pgs: 129 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:35:50.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.542+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae6c10a7a0 con 0x7fae6c1055b0 2024-09-16T00:35:50.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.543+0000 7fae637fe640 1 -- 172.21.15.1:0/245420577 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fae640bd340 con 0x7fae6c1055b0 2024-09-16T00:35:50.544 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:35:50.544 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:35:50.544 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:35:50.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fae580a3240 msgr2=0x7fae580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:50.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fae580a3240 0x7fae580a5700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fae6800bbe0 tx=0x7fae6800b390 comp rx=0 tx=0).stop 2024-09-16T00:35:50.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 msgr2=0x7fae6c0fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:50.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 0x7fae6c0fd110 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fae64005cc0 tx=0x7fae64003f90 comp rx=0 tx=0).stop 2024-09-16T00:35:50.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 shutdown_connections 2024-09-16T00:35:50.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae6c1055b0 0x7fae6c0fd110 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fae580a3240 0x7fae580a5700 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae6c100e30 0x7fae6c0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 --2- 172.21.15.1:0/245420577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae6c100480 0x7fae6c0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:50.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 >> 172.21.15.1:0/245420577 conn(0x7fae6c077210 msgr2=0x7fae6c103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:50.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 shutdown_connections 2024-09-16T00:35:50.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:50.546+0000 7fae745c1640 1 -- 172.21.15.1:0/245420577 wait complete. 2024-09-16T00:35:51.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:50 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/245420577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:51.268 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:35:51.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:50 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/245420577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:51.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:50 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/245420577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:51.531 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.057+0000 7f88b9fce640 1 -- 172.21.15.1:0/1732576401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 msgr2=0x7f88b410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.057+0000 7f88b9fce640 1 --2- 172.21.15.1:0/1732576401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 0x7f88b410e610 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009ca0 tx=0x7f88ac02a950 comp rx=0 tx=0).stop 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.057+0000 7f88b9fce640 1 -- 172.21.15.1:0/1732576401 shutdown_connections 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.057+0000 7f88b9fce640 1 --2- 172.21.15.1:0/1732576401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 0x7f88b410e610 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.057+0000 7f88b9fce640 1 --2- 172.21.15.1:0/1732576401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4103700 0x7f88b410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.057+0000 7f88b9fce640 1 --2- 172.21.15.1:0/1732576401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88b4102de0 0x7f88b41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.057+0000 7f88b9fce640 1 -- 172.21.15.1:0/1732576401 >> 172.21.15.1:0/1732576401 conn(0x7f88b40fc7a0 msgr2=0x7f88b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 -- 172.21.15.1:0/1732576401 shutdown_connections 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 -- 172.21.15.1:0/1732576401 wait complete. 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 Processor -- start 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 -- start start 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88b4102de0 0x7f88b4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4103700 0x7f88b4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 0x7f88b419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4111f70 con 0x7f88b410c180 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4111df0 con 0x7f88b4103700 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.058+0000 7f88b9fce640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41120f0 con 0x7f88b4102de0 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 0x7f88b419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 0x7f88b419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43660/0 (socket says 172.21.15.1:43660) 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b3fff640 1 -- 172.21.15.1:0/844943649 learned_addr learned my addr 172.21.15.1:0/844943649 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b37fe640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88b4102de0 0x7f88b4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88abfff640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4103700 0x7f88b4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b3fff640 1 -- 172.21.15.1:0/844943649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88b4102de0 msgr2=0x7f88b4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b3fff640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88b4102de0 0x7f88b4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b3fff640 1 -- 172.21.15.1:0/844943649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4103700 msgr2=0x7f88b4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b3fff640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4103700 0x7f88b4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b3fff640 1 -- 172.21.15.1:0/844943649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b419d3c0 con 0x7f88b410c180 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b3fff640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 0x7f88b419cce0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f88ac02ae70 tx=0x7f88ac02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b17fa640 1 -- 172.21.15.1:0/844943649 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac034040 con 0x7f88b410c180 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b9fce640 1 -- 172.21.15.1:0/844943649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b419d650 con 0x7f88b410c180 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b17fa640 1 -- 172.21.15.1:0/844943649 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ac030920 con 0x7f88b410c180 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b17fa640 1 -- 172.21.15.1:0/844943649 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac045480 con 0x7f88b410c180 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.059+0000 7f88b9fce640 1 -- 172.21.15.1:0/844943649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b41a4f90 con 0x7f88b410c180 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88b17fa640 1 -- 172.21.15.1:0/844943649 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f88ac037020 con 0x7f88b410c180 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88bc3a7740 1 Processor -- start 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88bc3a7740 1 -- start start 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 0x55d86ae20a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae2b370 0x55d86ae2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d86ad862f0 con 0x55d86ae21b40 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d86ad2dcf0 con 0x55d86ae2b370 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d86ad822b0 con 0x55d86ae205d0 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88b17fa640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88a00a3240 0x7f88a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae20090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43664/0 (socket says 172.21.15.1:43664) 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88abfff640 1 -- 172.21.15.1:0/813676887 learned_addr learned my addr 172.21.15.1:0/813676887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.062+0000 7f88b3fff640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88a00a3240 0x7f88a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88b17fa640 1 -- 172.21.15.1:0/844943649 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f88ac0f0b80 con 0x7f88b410c180 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88abfff640 1 --2- 172.21.15.1:0/813676887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae2b370 0x55d86ae2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88b37fe640 1 --2- 172.21.15.1:0/813676887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 0x55d86ae20a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88abfff640 1 -- 172.21.15.1:0/813676887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 msgr2=0x55d86ae20a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88abfff640 1 --2- 172.21.15.1:0/813676887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 0x55d86ae20a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88abfff640 1 -- 172.21.15.1:0/813676887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae2b370 msgr2=0x55d86ae2d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88abfff640 1 --2- 172.21.15.1:0/813676887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae2b370 0x55d86ae2d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88abfff640 1 -- 172.21.15.1:0/813676887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86ad123a0 con 0x55d86ae21b40 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88abfff640 1 --2- 172.21.15.1:0/813676887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae20090 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x55d86ae21070 tx=0x7f889c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a1a0fc341bd489 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88b3fff640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88a00a3240 0x7f88a00a5700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0bcd30 tx=0x7f88ac0877e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88a9ffb640 1 -- 172.21.15.1:0/813676887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c01e640 con 0x55d86ae21b40 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88a9ffb640 1 -- 172.21.15.1:0/813676887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f889c010910 con 0x55d86ae21b40 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.063+0000 7f88a9ffb640 1 -- 172.21.15.1:0/813676887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c01d7b0 con 0x55d86ae21b40 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 -- 172.21.15.1:0/813676887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 msgr2=0x55d86ae20090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/813676887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae20090 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x55d86ae21070 tx=0x7f889c00b7c0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 -- 172.21.15.1:0/813676887 shutdown_connections 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/813676887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae2b370 0x55d86ae2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/813676887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 0x55d86ae20a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/813676887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae20090 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 -- 172.21.15.1:0/813676887 >> 172.21.15.1:0/813676887 conn(0x55d86ade9310 msgr2=0x55d86ae1cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 -- 172.21.15.1:0/813676887 shutdown_connections 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 -- 172.21.15.1:0/813676887 wait complete. 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 -- start start 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.064+0000 7f88bc3a7740 1 Processor -- start 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae205d0 0x55d86ae2ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae28030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae2b370 0x55d86ae28640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d86ae2e070 con 0x55d86ae21b40 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d86ad41060 con 0x55d86ae205d0 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d86ae2e1f0 con 0x55d86ae2b370 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88bc3a7740 10 client.0 _ll_register_callbacks cb 0x55d86aa948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae28030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae205d0 0x55d86ae2ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae28030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43678/0 (socket says 172.21.15.1:43678) 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88b37fe640 1 -- 172.21.15.1:0/3883151714 learned_addr learned my addr 172.21.15.1:0/3883151714 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88abfff640 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae2b370 0x55d86ae28640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88b37fe640 1 -- 172.21.15.1:0/3883151714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae2b370 msgr2=0x55d86ae28640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88b37fe640 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae2b370 0x55d86ae28640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88b37fe640 1 -- 172.21.15.1:0/3883151714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae205d0 msgr2=0x55d86ae2ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88b37fe640 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae205d0 0x55d86ae2ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88b37fe640 1 -- 172.21.15.1:0/3883151714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86aaf73c0 con 0x55d86ae21b40 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f88b37fe640 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae28030 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f88a4012770 tx=0x7f88a4012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafd83cba7bbf7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.065+0000 7f8892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f8892ffd640 1 -- 172.21.15.1:0/3883151714 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a401f060 con 0x55d86ae21b40 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f8892ffd640 1 -- 172.21.15.1:0/3883151714 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f88a4010ce0 con 0x55d86ae21b40 2024-09-16T00:35:52.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f88bc3a7740 1 -- 172.21.15.1:0/3883151714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889c010910 con 0x55d86ae21b40 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f8892ffd640 1 -- 172.21.15.1:0/3883151714 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88a4026050 con 0x55d86ae21b40 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f88bc3a7740 10 client.17124 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f88bc3a7740 1 -- 172.21.15.1:0/3883151714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d86ae39ba0 con 0x55d86ae21b40 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f8892ffd640 1 -- 172.21.15.1:0/3883151714 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f889c010910 con 0x55d86ae21b40 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f8892ffd640 1 -- 172.21.15.1:0/3883151714 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f88a400d4d0 con 0x55d86ae21b40 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f88bc3a7740 10 client.17124 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f88bc3a7740 10 client.17124 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f88bc3a7740 20 client.17124 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d86ae3b060 0x55d86ae3d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f88bc3a7740 4 client.17124 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.066+0000 7f88bc3a7740 1 -- 172.21.15.1:0/3883151714 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d86ad93860 con 0x55d86ae3b060 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.067+0000 7f88abfff640 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d86ae3b060 0x55d86ae3d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.067+0000 7f88abfff640 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d86ae3b060 0x55d86ae3d470 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96a49ff9c976daa server_cookie=fb8db6f5bb9a9947 in_seq=0 out_seq=0 2024-09-16T00:35:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.067+0000 7f8892ffd640 10 client.17124 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f8892ffd640 1 -- 172.21.15.1:0/3883151714 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d86ad93860 con 0x55d86ae3b060 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f8892ffd640 10 client.17124 handle_command_reply: tid=0 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 client.17124 shutdown 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 -- 172.21.15.1:0/3883151714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 msgr2=0x55d86ae28030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae28030 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f88a4012770 tx=0x7f88a4012c30 comp rx=0 tx=0).stop 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 -- 172.21.15.1:0/3883151714 shutdown_connections 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae2b370 0x55d86ae28640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae21b40 0x55d86ae28030 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d86ae3b060 0x55d86ae3d470 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/3883151714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae205d0 0x55d86ae2ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f8892ffd640 0 client.17124 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 -- 172.21.15.1:0/3883151714 >> 172.21.15.1:0/3883151714 conn(0x55d86ade9310 msgr2=0x55d86ae22990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 -- 172.21.15.1:0/3883151714 shutdown_connections 2024-09-16T00:35:52.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 1 -- 172.21.15.1:0/3883151714 wait complete. 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.068+0000 7f88bc3a7740 20 client.17124 trim_cache size 0 max 16384 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88bc3a7740 1 Processor -- start 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88bc3a7740 1 -- start start 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 0x55d86ae29fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 0x55d86ae23120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae2b370 0x55d86ae23660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41b1680 con 0x55d86ae2b370 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41aef50 con 0x55d86ae21b40 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41acda0 con 0x55d86ae205d0 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae2b370 0x55d86ae23660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae2b370 0x55d86ae23660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43686/0 (socket says 172.21.15.1:43686) 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.081+0000 7f88abfff640 1 -- 172.21.15.1:0/1979768731 learned_addr learned my addr 172.21.15.1:0/1979768731 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:52.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88b37fe640 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 0x55d86ae23120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88abfff640 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 0x55d86ae29fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88abfff640 1 -- 172.21.15.1:0/1979768731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 msgr2=0x55d86ae29fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88abfff640 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 0x55d86ae29fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88abfff640 1 -- 172.21.15.1:0/1979768731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 msgr2=0x55d86ae23120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88abfff640 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 0x55d86ae23120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88abfff640 1 -- 172.21.15.1:0/1979768731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86ae1d960 con 0x55d86ae2b370 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88b37fe640 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 0x55d86ae23120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88abfff640 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae2b370 0x55d86ae23660 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x55d86ae21070 tx=0x7f889c00b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1d913c41e16343 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88a97fa640 1 -- 172.21.15.1:0/1979768731 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c010690 con 0x55d86ae2b370 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88a97fa640 1 -- 172.21.15.1:0/1979768731 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f889c010d10 con 0x55d86ae2b370 2024-09-16T00:35:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.082+0000 7f88a97fa640 1 -- 172.21.15.1:0/1979768731 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c036d50 con 0x55d86ae2b370 2024-09-16T00:35:52.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 -- 172.21.15.1:0/1979768731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae2b370 msgr2=0x55d86ae23660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae2b370 0x55d86ae23660 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x55d86ae21070 tx=0x7f889c00b570 comp rx=0 tx=0).stop 2024-09-16T00:35:52.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 -- 172.21.15.1:0/1979768731 shutdown_connections 2024-09-16T00:35:52.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae2b370 0x55d86ae23660 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 0x55d86ae23120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/1979768731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae205d0 0x55d86ae29fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 -- 172.21.15.1:0/1979768731 >> 172.21.15.1:0/1979768731 conn(0x55d86ade9310 msgr2=0x55d86ae29890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:52.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 -- 172.21.15.1:0/1979768731 shutdown_connections 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 -- 172.21.15.1:0/1979768731 wait complete. 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 -- start start 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.083+0000 7f88bc3a7740 1 Processor -- start 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae205d0 0x55d86ae33d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 0x55d86ae2f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88bc3a7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae2b370 0x55d86ae2f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41ad410 con 0x55d86ae205d0 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41ae880 con 0x55d86ae21b40 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88bc3a7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41adae0 con 0x55d86ae2b370 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88bc3a7740 10 client.0 _ll_register_callbacks cb 0x7f88a400a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae205d0 0x55d86ae33d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae205d0 0x55d86ae33d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43700/0 (socket says 172.21.15.1:43700) 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88abfff640 1 -- 172.21.15.1:0/125540110 learned_addr learned my addr 172.21.15.1:0/125540110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:52.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88b37fe640 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 0x55d86ae2f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88abfff640 1 -- 172.21.15.1:0/125540110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae2b370 msgr2=0x55d86ae2f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88abfff640 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae2b370 0x55d86ae2f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88abfff640 1 -- 172.21.15.1:0/125540110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 msgr2=0x55d86ae2f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88abfff640 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 0x55d86ae2f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88abfff640 1 -- 172.21.15.1:0/125540110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86aea4d10 con 0x55d86ae205d0 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88abfff640 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae205d0 0x55d86ae33d80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f889c000c00 tx=0x7f889c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6686838c78349d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f8892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f8892ffd640 1 -- 172.21.15.1:0/125540110 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c03b030 con 0x55d86ae205d0 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f8892ffd640 1 -- 172.21.15.1:0/125540110 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f889c036640 con 0x55d86ae205d0 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.084+0000 7f88bc3a7740 1 -- 172.21.15.1:0/125540110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889c010d10 con 0x55d86ae205d0 2024-09-16T00:35:52.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f8892ffd640 1 -- 172.21.15.1:0/125540110 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f889c023040 con 0x55d86ae205d0 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f88bc3a7740 10 client.17130 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f88bc3a7740 1 -- 172.21.15.1:0/125540110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d86aea5ed0 con 0x55d86ae205d0 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f8892ffd640 1 -- 172.21.15.1:0/125540110 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f889c010d10 con 0x55d86ae205d0 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f8892ffd640 1 -- 172.21.15.1:0/125540110 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f889c029040 con 0x55d86ae205d0 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f88bc3a7740 10 client.17130 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f88bc3a7740 10 client.17130 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f88bc3a7740 20 client.17130 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d86aea63a0 0x55d86aea8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f88bc3a7740 4 client.17130 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f88bc3a7740 1 -- 172.21.15.1:0/125540110 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d86ad8dd70 con 0x55d86aea63a0 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.085+0000 7f88b37fe640 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d86aea63a0 0x55d86aea8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f8892ffd640 10 client.17130 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88b37fe640 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d86aea63a0 0x55d86aea8790 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=269b4a159fcd159d server_cookie=f34ab3288d494ee6 in_seq=0 out_seq=0 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f8892ffd640 1 -- 172.21.15.1:0/125540110 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d86ad8dd70 con 0x55d86aea63a0 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f8892ffd640 10 client.17130 handle_command_reply: tid=0 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 client.17130 shutdown 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 -- 172.21.15.1:0/125540110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae205d0 msgr2=0x55d86ae33d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae205d0 0x55d86ae33d80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f889c000c00 tx=0x7f889c00bd40 comp rx=0 tx=0).stop 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 -- 172.21.15.1:0/125540110 shutdown_connections 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86ae2b370 0x55d86ae2f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55d86aea63a0 0x55d86aea8790 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d86ae21b40 0x55d86ae2f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 --2- 172.21.15.1:0/125540110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d86ae205d0 0x55d86ae33d80 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 -- 172.21.15.1:0/125540110 >> 172.21.15.1:0/125540110 conn(0x55d86ade9310 msgr2=0x55d86ae2a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:52.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f8892ffd640 0 client.17130 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:35:52.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 -- 172.21.15.1:0/125540110 shutdown_connections 2024-09-16T00:35:52.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 1 -- 172.21.15.1:0/125540110 wait complete. 2024-09-16T00:35:52.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.086+0000 7f88bc3a7740 20 client.17130 trim_cache size 0 max 16384 2024-09-16T00:35:52.102 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:35:52.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 -- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88a00a3240 msgr2=0x7f88a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88a00a3240 0x7f88a00a5700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0bcd30 tx=0x7f88ac0877e0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 -- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 msgr2=0x7f88b419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 0x7f88b419cce0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f88ac02ae70 tx=0x7f88ac02f5b0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 -- 172.21.15.1:0/844943649 shutdown_connections 2024-09-16T00:35:52.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410c180 0x7f88b419cce0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4103700 0x7f88b4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88a00a3240 0x7f88a00a5700 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 --2- 172.21.15.1:0/844943649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88b4102de0 0x7f88b4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 -- 172.21.15.1:0/844943649 >> 172.21.15.1:0/844943649 conn(0x7f88b40fc7a0 msgr2=0x7f88b40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:52.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 -- 172.21.15.1:0/844943649 shutdown_connections 2024-09-16T00:35:52.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:52.087+0000 7f88b9fce640 1 -- 172.21.15.1:0/844943649 wait complete. 2024-09-16T00:35:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:51 smithi019 ceph-mon[35186]: pgmap v574: 129 pgs: 129 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:52.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:51 smithi001 ceph-mon[30934]: pgmap v574: 129 pgs: 129 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:51 smithi153 ceph-mon[35538]: pgmap v574: 129 pgs: 129 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:52.948 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:35:53.206 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 -- 172.21.15.1:0/3337610501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0101410 msgr2=0x7f95b010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/3337610501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0101410 0x7f95b010d940 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009cd0 tx=0x7f95a802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 -- 172.21.15.1:0/3337610501 shutdown_connections 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/3337610501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0101410 0x7f95b010d940 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/3337610501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95b0100a70 0x7f95b0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/3337610501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95b0106a70 0x7f95b0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 -- 172.21.15.1:0/3337610501 >> 172.21.15.1:0/3337610501 conn(0x7f95b00fc790 msgr2=0x7f95b00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 -- 172.21.15.1:0/3337610501 shutdown_connections 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 -- 172.21.15.1:0/3337610501 wait complete. 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 Processor -- start 2024-09-16T00:35:53.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.742+0000 7f95b6ad3640 1 -- start start 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b6ad3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95b0100a70 0x7f95b0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b6ad3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95b0101410 0x7f95b0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b6ad3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0106a70 0x7f95b019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b6ad3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0110e70 con 0x7f95b0106a70 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b6ad3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0110cf0 con 0x7f95b0101410 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b6ad3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0110ff0 con 0x7f95b0100a70 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b5070640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0106a70 0x7f95b019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b5070640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0106a70 0x7f95b019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43718/0 (socket says 172.21.15.1:43718) 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b5070640 1 -- 172.21.15.1:0/2243147843 learned_addr learned my addr 172.21.15.1:0/2243147843 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b486f640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95b0100a70 0x7f95b0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95affff640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95b0101410 0x7f95b0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b5070640 1 -- 172.21.15.1:0/2243147843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95b0100a70 msgr2=0x7f95b0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b5070640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95b0100a70 0x7f95b0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b5070640 1 -- 172.21.15.1:0/2243147843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95b0101410 msgr2=0x7f95b0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b5070640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95b0101410 0x7f95b0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b5070640 1 -- 172.21.15.1:0/2243147843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b019d3b0 con 0x7f95b0106a70 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95b5070640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0106a70 0x7f95b019ccd0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005cc0 tx=0x7f95a8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.743+0000 7f95adffb640 1 -- 172.21.15.1:0/2243147843 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004040 con 0x7f95b0106a70 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.744+0000 7f95adffb640 1 -- 172.21.15.1:0/2243147843 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a80046c0 con 0x7f95b0106a70 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.744+0000 7f95b6ad3640 1 -- 172.21.15.1:0/2243147843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b019d640 con 0x7f95b0106a70 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.744+0000 7f95adffb640 1 -- 172.21.15.1:0/2243147843 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8046480 con 0x7f95b0106a70 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.744+0000 7f95b6ad3640 1 -- 172.21.15.1:0/2243147843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b01a4f80 con 0x7f95b0106a70 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.746+0000 7f95adffb640 1 -- 172.21.15.1:0/2243147843 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f95a80305a0 con 0x7f95b0106a70 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.746+0000 7f95b8eac740 1 Processor -- start 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.746+0000 7f95b8eac740 1 -- start start 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.746+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b0a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0a5d0 0x55e998b0aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b15370 0x55e998b17760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95b8eac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e998a702f0 con 0x55e998b0bb40 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95b8eac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e998a17cf0 con 0x55e998b15370 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95b8eac740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e998a6c2b0 con 0x55e998b0a5d0 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b0a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b0a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43728/0 (socket says 172.21.15.1:43728) 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 -- 172.21.15.1:0/2778769455 learned_addr learned my addr 172.21.15.1:0/2778769455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95adffb640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95a00a32c0 0x7f95a00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 --2- 172.21.15.1:0/2778769455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b15370 0x55e998b17760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95b486f640 1 --2- 172.21.15.1:0/2778769455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0a5d0 0x55e998b0aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95adffb640 1 -- 172.21.15.1:0/2243147843 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f95a802e030 con 0x7f95b0106a70 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95b5070640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95a00a32c0 0x7f95a00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 -- 172.21.15.1:0/2778769455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0a5d0 msgr2=0x55e998b0aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 --2- 172.21.15.1:0/2778769455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0a5d0 0x55e998b0aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 -- 172.21.15.1:0/2778769455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b15370 msgr2=0x55e998b17760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 --2- 172.21.15.1:0/2778769455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b15370 0x55e998b17760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 -- 172.21.15.1:0/2778769455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9989fc3a0 con 0x55e998b0bb40 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95affff640 1 --2- 172.21.15.1:0/2778769455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b0a090 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x55e998b0b070 tx=0x7f959c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49a8a79698b4c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f95b5070640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95a00a32c0 0x7f95a00a5780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f95a80f1570 tx=0x7f95a80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.747+0000 7f959affd640 1 -- 172.21.15.1:0/2778769455 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c012ce0 con 0x55e998b0bb40 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f959affd640 1 -- 172.21.15.1:0/2778769455 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f959c010c20 con 0x55e998b0bb40 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f959affd640 1 -- 172.21.15.1:0/2778769455 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c01eab0 con 0x55e998b0bb40 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f95b8eac740 1 -- 172.21.15.1:0/2778769455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 msgr2=0x55e998b0a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f95b8eac740 1 --2- 172.21.15.1:0/2778769455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b0a090 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x55e998b0b070 tx=0x7f959c00b790 comp rx=0 tx=0).stop 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f95b8eac740 1 -- 172.21.15.1:0/2778769455 shutdown_connections 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f95b8eac740 1 --2- 172.21.15.1:0/2778769455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b15370 0x55e998b17760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f95b8eac740 1 --2- 172.21.15.1:0/2778769455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0a5d0 0x55e998b0aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f95b8eac740 1 --2- 172.21.15.1:0/2778769455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b0a090 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f95b8eac740 1 -- 172.21.15.1:0/2778769455 >> 172.21.15.1:0/2778769455 conn(0x55e998ad3310 msgr2=0x55e998b06c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f95b8eac740 1 -- 172.21.15.1:0/2778769455 shutdown_connections 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.748+0000 7f95b8eac740 1 -- 172.21.15.1:0/2778769455 wait complete. 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95b8eac740 1 -- start start 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95b8eac740 1 Processor -- start 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b14d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0bb40 0x55e998b12000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 0x55e998b125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95b8eac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e998b18070 con 0x55e998b0a5d0 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95b8eac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e998a2b060 con 0x55e998b0bb40 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95b8eac740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e998b181f0 con 0x55e998b15370 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95b8eac740 10 client.0 _ll_register_callbacks cb 0x55e99877e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b14d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b14d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43734/0 (socket says 172.21.15.1:43734) 2024-09-16T00:35:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95affff640 1 -- 172.21.15.1:0/1803416494 learned_addr learned my addr 172.21.15.1:0/1803416494 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95affff640 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 0x55e998b125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.749+0000 7f95affff640 1 -- 172.21.15.1:0/1803416494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 msgr2=0x55e998b125d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95affff640 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 0x55e998b125d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95affff640 1 -- 172.21.15.1:0/1803416494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0bb40 msgr2=0x55e998b12000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95b486f640 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0bb40 0x55e998b12000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95affff640 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0bb40 0x55e998b12000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95affff640 1 -- 172.21.15.1:0/1803416494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9987e13c0 con 0x55e998b0a5d0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95b486f640 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0bb40 0x55e998b12000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95affff640 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b14d50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f959c011000 tx=0x7f959c00b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4abd3fd8729bc79 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f958ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f958ffff640 1 -- 172.21.15.1:0/1803416494 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c024040 con 0x55e998b0a5d0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f958ffff640 1 -- 172.21.15.1:0/1803416494 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f959c010940 con 0x55e998b0a5d0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95b8eac740 1 -- 172.21.15.1:0/1803416494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f959c010c20 con 0x55e998b0a5d0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f958ffff640 1 -- 172.21.15.1:0/1803416494 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f959c004040 con 0x55e998b0a5d0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95b8eac740 10 client.17142 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f95b8eac740 1 -- 172.21.15.1:0/1803416494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e998b23af0 con 0x55e998b0a5d0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f958ffff640 1 -- 172.21.15.1:0/1803416494 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f959c010c20 con 0x55e998b0a5d0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.750+0000 7f958ffff640 1 -- 172.21.15.1:0/1803416494 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f959c0035f0 con 0x55e998b0a5d0 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.751+0000 7f95b8eac740 10 client.17142 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.751+0000 7f95b8eac740 10 client.17142 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:35:53.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.751+0000 7f95b8eac740 20 client.17142 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.751+0000 7f95b8eac740 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e998b24e40 0x55e998b27230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.751+0000 7f95b8eac740 4 client.17142 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.751+0000 7f95b8eac740 1 -- 172.21.15.1:0/1803416494 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e998a7d860 con 0x55e998b24e40 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.751+0000 7f95b486f640 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e998b24e40 0x55e998b27230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.751+0000 7f95b486f640 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e998b24e40 0x55e998b27230 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f478066095f9064b server_cookie=3f8fba966c07b780 in_seq=0 out_seq=0 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.751+0000 7f958ffff640 10 client.17142 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f958ffff640 1 -- 172.21.15.1:0/1803416494 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e998a7d860 con 0x55e998b24e40 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f958ffff640 10 client.17142 handle_command_reply: tid=0 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f95b8eac740 1 client.17142 shutdown 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f95b8eac740 1 -- 172.21.15.1:0/1803416494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 msgr2=0x55e998b14d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f95b8eac740 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b14d50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f959c011000 tx=0x7f959c00b590 comp rx=0 tx=0).stop 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f95b8eac740 1 -- 172.21.15.1:0/1803416494 shutdown_connections 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f95b8eac740 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 0x55e998b125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f95b8eac740 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0bb40 0x55e998b12000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f95b8eac740 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e998b24e40 0x55e998b27230 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f95b8eac740 1 --2- 172.21.15.1:0/1803416494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b14d50 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f958ffff640 0 client.17142 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:35:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.752+0000 7f95b8eac740 1 -- 172.21.15.1:0/1803416494 >> 172.21.15.1:0/1803416494 conn(0x55e998ad3310 msgr2=0x55e998b0c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.753+0000 7f95b8eac740 1 -- 172.21.15.1:0/1803416494 shutdown_connections 2024-09-16T00:35:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.753+0000 7f95b8eac740 1 -- 172.21.15.1:0/1803416494 wait complete. 2024-09-16T00:35:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.753+0000 7f95b8eac740 20 client.17142 trim_cache size 0 max 16384 2024-09-16T00:35:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b8eac740 1 Processor -- start 2024-09-16T00:35:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b8eac740 1 -- start start 2024-09-16T00:35:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0a5d0 0x55e998b21310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 0x55e998b0d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b8eac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01b1650 con 0x55e998b0bb40 2024-09-16T00:35:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b8eac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01aef20 con 0x55e998b0a5d0 2024-09-16T00:35:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b8eac740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01acd70 con 0x55e998b15370 2024-09-16T00:35:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b486f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b486f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b118c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43740/0 (socket says 172.21.15.1:43740) 2024-09-16T00:35:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b486f640 1 -- 172.21.15.1:0/3054221468 learned_addr learned my addr 172.21.15.1:0/3054221468 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95affff640 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0a5d0 0x55e998b21310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95affff640 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 0x55e998b0d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b486f640 1 -- 172.21.15.1:0/3054221468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 msgr2=0x55e998b0d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b486f640 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 0x55e998b0d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b486f640 1 -- 172.21.15.1:0/3054221468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0a5d0 msgr2=0x55e998b21310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b486f640 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0a5d0 0x55e998b21310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b486f640 1 -- 172.21.15.1:0/3054221468 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e998b150c0 con 0x55e998b0bb40 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95affff640 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0a5d0 0x55e998b21310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.766+0000 7f95b486f640 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b118c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f95a400abe0 tx=0x7f95a4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3723348ff4139d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f959a7fc640 1 -- 172.21.15.1:0/3054221468 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4004020 con 0x55e998b0bb40 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f959a7fc640 1 -- 172.21.15.1:0/3054221468 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a40041c0 con 0x55e998b0bb40 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f959a7fc640 1 -- 172.21.15.1:0/3054221468 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a401c7f0 con 0x55e998b0bb40 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f95b8eac740 1 -- 172.21.15.1:0/3054221468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 msgr2=0x55e998b118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f95b8eac740 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b118c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f95a400abe0 tx=0x7f95a4010800 comp rx=0 tx=0).stop 2024-09-16T00:35:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f95b8eac740 1 -- 172.21.15.1:0/3054221468 shutdown_connections 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f95b8eac740 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b15370 0x55e998b0d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f95b8eac740 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0bb40 0x55e998b118c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f95b8eac740 1 --2- 172.21.15.1:0/3054221468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b0a5d0 0x55e998b21310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f95b8eac740 1 -- 172.21.15.1:0/3054221468 >> 172.21.15.1:0/3054221468 conn(0x55e998ad3310 msgr2=0x55e998b110f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f95b8eac740 1 -- 172.21.15.1:0/3054221468 shutdown_connections 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.767+0000 7f95b8eac740 1 -- 172.21.15.1:0/3054221468 wait complete. 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b8eac740 1 -- start start 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b8eac740 1 Processor -- start 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b87b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0bb40 0x55e998b88070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b8eac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b15370 0x55e998b1d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b8eac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01ad3e0 con 0x55e998b0a5d0 2024-09-16T00:35:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b8eac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01ae850 con 0x55e998b15370 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b8eac740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01adab0 con 0x55e998b0bb40 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b8eac740 10 client.0 _ll_register_callbacks cb 0x7f959c00be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b87b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b87b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43746/0 (socket says 172.21.15.1:43746) 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95affff640 1 -- 172.21.15.1:0/153478131 learned_addr learned my addr 172.21.15.1:0/153478131 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.768+0000 7f95b486f640 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0bb40 0x55e998b88070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f95affff640 1 -- 172.21.15.1:0/153478131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0bb40 msgr2=0x55e998b88070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f95affff640 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0bb40 0x55e998b88070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f95affff640 1 -- 172.21.15.1:0/153478131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b15370 msgr2=0x55e998b1d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f95affff640 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b15370 0x55e998b1d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f95affff640 1 -- 172.21.15.1:0/153478131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e998b12940 con 0x55e998b0a5d0 2024-09-16T00:35:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f95affff640 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b87b30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f959c01e620 tx=0x7f959c00b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dc15aebeb78701 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f958ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f958ffff640 1 -- 172.21.15.1:0/153478131 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00baa0 con 0x55e998b0a5d0 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f958ffff640 1 -- 172.21.15.1:0/153478131 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f959c0035f0 con 0x55e998b0a5d0 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f95b8eac740 1 -- 172.21.15.1:0/153478131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a40041c0 con 0x55e998b0a5d0 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f958ffff640 1 -- 172.21.15.1:0/153478131 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f959c044050 con 0x55e998b0a5d0 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f95b8eac740 10 client.17148 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f95b8eac740 1 -- 172.21.15.1:0/153478131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e998b8f190 con 0x55e998b0a5d0 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.769+0000 7f958ffff640 1 -- 172.21.15.1:0/153478131 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a40041c0 con 0x55e998b0a5d0 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f958ffff640 1 -- 172.21.15.1:0/153478131 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f959c01e7f0 con 0x55e998b0a5d0 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f95b8eac740 10 client.17148 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f95b8eac740 10 client.17148 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f95b8eac740 20 client.17148 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f95b8eac740 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e998b8f7d0 0x55e998b91bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f95b8eac740 4 client.17148 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f95b8eac740 1 -- 172.21.15.1:0/153478131 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e998a77d70 con 0x55e998b8f7d0 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f95b486f640 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e998b8f7d0 0x55e998b91bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f95b486f640 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e998b8f7d0 0x55e998b91bf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c17981199a32b4d1 server_cookie=948675aa3d16a6ae in_seq=0 out_seq=0 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.770+0000 7f958ffff640 10 client.17148 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f958ffff640 1 -- 172.21.15.1:0/153478131 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e998a77d70 con 0x55e998b8f7d0 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f958ffff640 10 client.17148 handle_command_reply: tid=0 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 client.17148 shutdown 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 -- 172.21.15.1:0/153478131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 msgr2=0x55e998b87b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b87b30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f959c01e620 tx=0x7f959c00b400 comp rx=0 tx=0).stop 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 -- 172.21.15.1:0/153478131 shutdown_connections 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e998b15370 0x55e998b1d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e998b0bb40 0x55e998b88070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55e998b8f7d0 0x55e998b91bf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 --2- 172.21.15.1:0/153478131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e998b0a5d0 0x55e998b87b30 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f958ffff640 0 client.17148 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 -- 172.21.15.1:0/153478131 >> 172.21.15.1:0/153478131 conn(0x55e998ad3310 msgr2=0x55e998b05310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 -- 172.21.15.1:0/153478131 shutdown_connections 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 1 -- 172.21.15.1:0/153478131 wait complete. 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.771+0000 7f95b8eac740 20 client.17148 trim_cache size 0 max 16384 2024-09-16T00:35:53.786 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:35:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 -- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95a00a32c0 msgr2=0x7f95a00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95a00a32c0 0x7f95a00a5780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f95a80f1570 tx=0x7f95a80095c0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 -- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0106a70 msgr2=0x7f95b019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0106a70 0x7f95b019ccd0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005cc0 tx=0x7f95a8003970 comp rx=0 tx=0).stop 2024-09-16T00:35:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 -- 172.21.15.1:0/2243147843 shutdown_connections 2024-09-16T00:35:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95b0106a70 0x7f95b019ccd0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95b0101410 0x7f95b0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f95a00a32c0 0x7f95a00a5780 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 --2- 172.21.15.1:0/2243147843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95b0100a70 0x7f95b0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 -- 172.21.15.1:0/2243147843 >> 172.21.15.1:0/2243147843 conn(0x7f95b00fc790 msgr2=0x7f95b006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 -- 172.21.15.1:0/2243147843 shutdown_connections 2024-09-16T00:35:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:53.772+0000 7f95b6ad3640 1 -- 172.21.15.1:0/2243147843 wait complete. 2024-09-16T00:35:54.356 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:53 smithi001 ceph-mon[30934]: pgmap v575: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-16T00:35:54.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:53 smithi019 ceph-mon[35186]: pgmap v575: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-16T00:35:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:53 smithi153 ceph-mon[35538]: pgmap v575: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-16T00:35:54.485 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:35:54.747 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:55.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.269+0000 7f05ac4a4640 1 -- 172.21.15.1:0/1488525153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4106a80 msgr2=0x7f05a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.269+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/1488525153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4106a80 0x7f05a4106e60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009930 tx=0x7f05a002a420 comp rx=0 tx=0).stop 2024-09-16T00:35:55.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.269+0000 7f05ac4a4640 1 -- 172.21.15.1:0/1488525153 shutdown_connections 2024-09-16T00:35:55.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.269+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/1488525153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a4101420 0x7f05a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.269+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/1488525153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05a4100a80 0x7f05a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.269+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/1488525153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4106a80 0x7f05a4106e60 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.269+0000 7f05ac4a4640 1 -- 172.21.15.1:0/1488525153 >> 172.21.15.1:0/1488525153 conn(0x7f05a40fc7a0 msgr2=0x7f05a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.269+0000 7f05ac4a4640 1 -- 172.21.15.1:0/1488525153 shutdown_connections 2024-09-16T00:35:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.269+0000 7f05ac4a4640 1 -- 172.21.15.1:0/1488525153 wait complete. 2024-09-16T00:35:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.270+0000 7f05ac4a4640 1 Processor -- start 2024-09-16T00:35:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.270+0000 7f05ac4a4640 1 -- start start 2024-09-16T00:35:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.270+0000 7f05ac4a4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4100a80 0x7f05a4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.270+0000 7f05ac4a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a4101420 0x7f05a4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.270+0000 7f05ac4a4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05a4106a80 0x7f05a41a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.270+0000 7f05ac4a4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4110e80 con 0x7f05a4100a80 2024-09-16T00:35:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.270+0000 7f05ac4a4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4110d00 con 0x7f05a4101420 2024-09-16T00:35:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.270+0000 7f05ac4a4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4111000 con 0x7f05a4106a80 2024-09-16T00:35:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.270+0000 7f05aa240640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4100a80 0x7f05a4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05aa240640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4100a80 0x7f05a4199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43762/0 (socket says 172.21.15.1:43762) 2024-09-16T00:35:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05aa240640 1 -- 172.21.15.1:0/583950005 learned_addr learned my addr 172.21.15.1:0/583950005 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a4101420 0x7f05a4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05aaa41640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05a4106a80 0x7f05a41a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05aa240640 1 -- 172.21.15.1:0/583950005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05a4106a80 msgr2=0x7f05a41a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05aa240640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05a4106a80 0x7f05a41a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05aa240640 1 -- 172.21.15.1:0/583950005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a4101420 msgr2=0x7f05a4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05aa240640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a4101420 0x7f05a4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05aa240640 1 -- 172.21.15.1:0/583950005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a41a48d0 con 0x7f05a4100a80 2024-09-16T00:35:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05aa240640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4100a80 0x7f05a4199310 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009230 tx=0x7f05a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f059b7fe640 1 -- 172.21.15.1:0/583950005 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a003f030 con 0x7f05a4100a80 2024-09-16T00:35:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f059b7fe640 1 -- 172.21.15.1:0/583950005 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a0003ad0 con 0x7f05a4100a80 2024-09-16T00:35:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05ac4a4640 1 -- 172.21.15.1:0/583950005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a41a4aa0 con 0x7f05a4100a80 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f059b7fe640 1 -- 172.21.15.1:0/583950005 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0035070 con 0x7f05a4100a80 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.271+0000 7f05ac4a4640 1 -- 172.21.15.1:0/583950005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a41a4f50 con 0x7f05a4100a80 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.274+0000 7f059b7fe640 1 -- 172.21.15.1:0/583950005 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f05a0042030 con 0x7f05a4100a80 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.274+0000 7f05ae87d740 1 Processor -- start 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05ae87d740 1 -- start start 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca2b40 0x55fa00ca1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 0x55fa00ca1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 0x55fa00cae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05ae87d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fa00c072f0 con 0x55fa00ca2b40 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05ae87d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fa00baecf0 con 0x55fa00cac370 2024-09-16T00:35:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05ae87d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fa00c032b0 con 0x55fa00ca15d0 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aaa41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca2b40 0x55fa00ca1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aaa41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca2b40 0x55fa00ca1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43778/0 (socket says 172.21.15.1:43778) 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aaa41640 1 -- 172.21.15.1:0/2415098934 learned_addr learned my addr 172.21.15.1:0/2415098934 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aa240640 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 0x55fa00ca1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 0x55fa00cae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aaa41640 1 -- 172.21.15.1:0/2415098934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 msgr2=0x55fa00ca1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aaa41640 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 0x55fa00ca1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aaa41640 1 -- 172.21.15.1:0/2415098934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 msgr2=0x55fa00cae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aaa41640 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 0x55fa00cae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aaa41640 1 -- 172.21.15.1:0/2415098934 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa00b933a0 con 0x55fa00ca2b40 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 0x55fa00cae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f05aa240640 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 0x55fa00ca1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.275+0000 7f059b7fe640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f05900a32c0 0x7f05900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.276+0000 7f05aa240640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f05900a32c0 0x7f05900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.276+0000 7f05aaa41640 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca2b40 0x55fa00ca1090 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f059c009f70 tx=0x7f059c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fd66d46ee142e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.276+0000 7f059b7fe640 1 -- 172.21.15.1:0/583950005 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f05a00f00f0 con 0x7f05a4100a80 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.276+0000 7f0598ff9640 1 -- 172.21.15.1:0/2415098934 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c016070 con 0x55fa00ca2b40 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.276+0000 7f0598ff9640 1 -- 172.21.15.1:0/2415098934 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059c004d20 con 0x55fa00ca2b40 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.276+0000 7f0598ff9640 1 -- 172.21.15.1:0/2415098934 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c012910 con 0x55fa00ca2b40 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 -- 172.21.15.1:0/2415098934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca2b40 msgr2=0x55fa00ca1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05aa240640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f05900a32c0 0x7f05900a5780 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f05a00edb60 tx=0x7f05a0040fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca2b40 0x55fa00ca1090 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f059c009f70 tx=0x7f059c010800 comp rx=0 tx=0).stop 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 -- 172.21.15.1:0/2415098934 shutdown_connections 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 0x55fa00cae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 0x55fa00ca1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 --2- 172.21.15.1:0/2415098934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca2b40 0x55fa00ca1090 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 -- 172.21.15.1:0/2415098934 >> 172.21.15.1:0/2415098934 conn(0x55fa00c6a310 msgr2=0x55fa00c9dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:55.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 -- 172.21.15.1:0/2415098934 shutdown_connections 2024-09-16T00:35:55.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 -- 172.21.15.1:0/2415098934 wait complete. 2024-09-16T00:35:55.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 -- start start 2024-09-16T00:35:55.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 Processor -- start 2024-09-16T00:35:55.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 0x55fa00cabd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 0x55fa00ca8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00cac370 0x55fa00ca9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fa00caf070 con 0x55fa00cac370 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.277+0000 7f05ae87d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fa00bc2060 con 0x55fa00ca2b40 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05ae87d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fa00caf1f0 con 0x55fa00ca15d0 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05ae87d740 10 client.0 _ll_register_callbacks cb 0x55fa009158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00cac370 0x55fa00ca9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00cac370 0x55fa00ca9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43780/0 (socket says 172.21.15.1:43780) 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 -- 172.21.15.1:0/4061182179 learned_addr learned my addr 172.21.15.1:0/4061182179 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 0x55fa00ca8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 0x55fa00cabd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 -- 172.21.15.1:0/4061182179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 msgr2=0x55fa00cabd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 0x55fa00cabd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 -- 172.21.15.1:0/4061182179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 msgr2=0x55fa00ca8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:55.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 0x55fa00ca8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 -- 172.21.15.1:0/4061182179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa009783c0 con 0x55fa00cac370 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05aaa41640 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00cac370 0x55fa00ca9570 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f059c012530 tx=0x7f059c012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7591a63d8463a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f058dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f058dffb640 1 -- 172.21.15.1:0/4061182179 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c016070 con 0x55fa00cac370 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f058dffb640 1 -- 172.21.15.1:0/4061182179 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f059c003970 con 0x55fa00cac370 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.278+0000 7f05ae87d740 1 -- 172.21.15.1:0/4061182179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059c004d20 con 0x55fa00cac370 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f058dffb640 1 -- 172.21.15.1:0/4061182179 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f059c02f050 con 0x55fa00cac370 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f05ae87d740 10 client.17160 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f05ae87d740 1 -- 172.21.15.1:0/4061182179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa00cbaaf0 con 0x55fa00cac370 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f058dffb640 1 -- 172.21.15.1:0/4061182179 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059c004d20 con 0x55fa00cac370 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f058dffb640 1 -- 172.21.15.1:0/4061182179 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f059c00f040 con 0x55fa00cac370 2024-09-16T00:35:55.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f05ae87d740 10 client.17160 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f05ae87d740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f05ae87d740 20 client.17160 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f05ae87d740 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fa00cbbef0 0x55fa00cbe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f05ae87d740 4 client.17160 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f05ae87d740 1 -- 172.21.15.1:0/4061182179 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa00c14860 con 0x55fa00cbbef0 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.279+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fa00cbbef0 0x55fa00cbe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.280+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fa00cbbef0 0x55fa00cbe2e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247264d012a83ccf server_cookie=5f10db009af23967 in_seq=0 out_seq=0 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.280+0000 7f058dffb640 10 client.17160 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.280+0000 7f058dffb640 1 -- 172.21.15.1:0/4061182179 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa00c14860 con 0x55fa00cbbef0 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.280+0000 7f058dffb640 10 client.17160 handle_command_reply: tid=0 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.280+0000 7f05ae87d740 1 client.17160 shutdown 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 -- 172.21.15.1:0/4061182179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00cac370 msgr2=0x55fa00ca9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00cac370 0x55fa00ca9570 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f059c012530 tx=0x7f059c012590 comp rx=0 tx=0).stop 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 -- 172.21.15.1:0/4061182179 shutdown_connections 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fa00cbbef0 0x55fa00cbe2e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00cac370 0x55fa00ca9570 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f058dffb640 0 client.17160 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 0x55fa00ca8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 --2- 172.21.15.1:0/4061182179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca15d0 0x55fa00cabd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 -- 172.21.15.1:0/4061182179 >> 172.21.15.1:0/4061182179 conn(0x55fa00c6a310 msgr2=0x55fa00ca3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 -- 172.21.15.1:0/4061182179 shutdown_connections 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 1 -- 172.21.15.1:0/4061182179 wait complete. 2024-09-16T00:35:55.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.281+0000 7f05ae87d740 20 client.17160 trim_cache size 0 max 16384 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05ae87d740 1 Processor -- start 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05ae87d740 1 -- start start 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca2b40 0x55fa00cc3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 0x55fa00cc3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05ae87d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b1770 con 0x55fa00ca15d0 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05ae87d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41af040 con 0x55fa00cac370 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05ae87d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41ace90 con 0x55fa00ca2b40 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05aaa41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05aaa41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43788/0 (socket says 172.21.15.1:43788) 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.294+0000 7f05aaa41640 1 -- 172.21.15.1:0/3937627773 learned_addr learned my addr 172.21.15.1:0/3937627773 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:55.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.295+0000 7f05aaa41640 1 -- 172.21.15.1:0/3937627773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca2b40 msgr2=0x55fa00cc3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:55.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.295+0000 7f05aaa41640 1 --2- 172.21.15.1:0/3937627773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca2b40 0x55fa00cc3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.295+0000 7f05aaa41640 1 -- 172.21.15.1:0/3937627773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 msgr2=0x55fa00cc3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:55.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.295+0000 7f05aaa41640 1 --2- 172.21.15.1:0/3937627773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 0x55fa00cc3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.295+0000 7f05aaa41640 1 -- 172.21.15.1:0/3937627773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa00c9de30 con 0x55fa00ca15d0 2024-09-16T00:35:55.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.295+0000 7f05aaa41640 1 --2- 172.21.15.1:0/3937627773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb8340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f059c002230 tx=0x7f059c003fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b71209a929d310 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:55.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.295+0000 7f058ffff640 1 -- 172.21.15.1:0/3937627773 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c016040 con 0x55fa00ca15d0 2024-09-16T00:35:55.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.295+0000 7f058ffff640 1 -- 172.21.15.1:0/3937627773 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059c025040 con 0x55fa00ca15d0 2024-09-16T00:35:55.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.295+0000 7f058ffff640 1 -- 172.21.15.1:0/3937627773 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c02a420 con 0x55fa00ca15d0 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 -- 172.21.15.1:0/3937627773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 msgr2=0x55fa00cb8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3937627773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb8340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f059c002230 tx=0x7f059c003fc0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 -- 172.21.15.1:0/3937627773 shutdown_connections 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3937627773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00cac370 0x55fa00cc3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3937627773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00ca2b40 0x55fa00cc3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3937627773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb8340 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 -- 172.21.15.1:0/3937627773 >> 172.21.15.1:0/3937627773 conn(0x55fa00c6a310 msgr2=0x55fa00ca80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 -- 172.21.15.1:0/3937627773 shutdown_connections 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 -- 172.21.15.1:0/3937627773 wait complete. 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 -- start start 2024-09-16T00:35:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 Processor -- start 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 0x55fa00caff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00cac370 0x55fa00cb0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41ad500 con 0x55fa00ca15d0 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41ae970 con 0x55fa00ca2b40 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41adbd0 con 0x55fa00cac370 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.296+0000 7f05ae87d740 10 client.0 _ll_register_callbacks cb 0x7f059c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43798/0 (socket says 172.21.15.1:43798) 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 -- 172.21.15.1:0/3367564081 learned_addr learned my addr 172.21.15.1:0/3367564081 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 0x55fa00caff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00cac370 0x55fa00cb0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 -- 172.21.15.1:0/3367564081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00cac370 msgr2=0x55fa00cb0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:55.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00cac370 0x55fa00cb0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 -- 172.21.15.1:0/3367564081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 msgr2=0x55fa00caff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 0x55fa00caff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 -- 172.21.15.1:0/3367564081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa00cb0c50 con 0x55fa00ca15d0 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 0x55fa00caff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05aaa41640 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb4bb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f059c000f80 tx=0x7f059c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ff119f01c125d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f058dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f058dffb640 1 -- 172.21.15.1:0/3367564081 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c016070 con 0x55fa00ca15d0 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f058dffb640 1 -- 172.21.15.1:0/3367564081 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f059c005050 con 0x55fa00ca15d0 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f05ae87d740 1 -- 172.21.15.1:0/3367564081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059c025040 con 0x55fa00ca15d0 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.297+0000 7f058dffb640 1 -- 172.21.15.1:0/3367564081 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f059c047050 con 0x55fa00ca15d0 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f05ae87d740 10 client.17166 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f05ae87d740 1 -- 172.21.15.1:0/3367564081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa00d26890 con 0x55fa00ca15d0 2024-09-16T00:35:55.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f058dffb640 1 -- 172.21.15.1:0/3367564081 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059c025040 con 0x55fa00ca15d0 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f058dffb640 1 -- 172.21.15.1:0/3367564081 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f059c004780 con 0x55fa00ca15d0 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f05ae87d740 10 client.17166 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f05ae87d740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f05ae87d740 20 client.17166 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fa00d26ef0 0x55fa00d292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f05ae87d740 4 client.17166 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f05ae87d740 1 -- 172.21.15.1:0/3367564081 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa00c0ed70 con 0x55fa00d26ef0 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.298+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fa00d26ef0 0x55fa00d292e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05a9a3f640 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fa00d26ef0 0x55fa00d292e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e419b6f718130482 server_cookie=9f69b841addb7824 in_seq=0 out_seq=0 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f058dffb640 10 client.17166 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f058dffb640 1 -- 172.21.15.1:0/3367564081 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa00c0ed70 con 0x55fa00d26ef0 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f058dffb640 10 client.17166 handle_command_reply: tid=0 2024-09-16T00:35:55.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 client.17166 shutdown 2024-09-16T00:35:55.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 -- 172.21.15.1:0/3367564081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 msgr2=0x55fa00cb4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb4bb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f059c000f80 tx=0x7f059c003970 comp rx=0 tx=0).stop 2024-09-16T00:35:55.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 -- 172.21.15.1:0/3367564081 shutdown_connections 2024-09-16T00:35:55.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55fa00d26ef0 0x55fa00d292e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa00cac370 0x55fa00cb0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa00ca2b40 0x55fa00caff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f058dffb640 0 client.17166 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:55.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 --2- 172.21.15.1:0/3367564081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa00ca15d0 0x55fa00cb4bb0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 -- 172.21.15.1:0/3367564081 >> 172.21.15.1:0/3367564081 conn(0x55fa00c6a310 msgr2=0x55fa00caa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:55.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 -- 172.21.15.1:0/3367564081 shutdown_connections 2024-09-16T00:35:55.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 1 -- 172.21.15.1:0/3367564081 wait complete. 2024-09-16T00:35:55.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.299+0000 7f05ae87d740 20 client.17166 trim_cache size 0 max 16384 2024-09-16T00:35:55.306 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:35:55.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 -- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f05900a32c0 msgr2=0x7f05900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f05900a32c0 0x7f05900a5780 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f05a00edb60 tx=0x7f05a0040fb0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 -- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4100a80 msgr2=0x7f05a4199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4100a80 0x7f05a4199310 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009230 tx=0x7f05a0004270 comp rx=0 tx=0).stop 2024-09-16T00:35:55.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 -- 172.21.15.1:0/583950005 shutdown_connections 2024-09-16T00:35:55.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f05900a32c0 0x7f05900a5780 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05a4106a80 0x7f05a41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a4101420 0x7f05a4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 --2- 172.21.15.1:0/583950005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05a4100a80 0x7f05a4199310 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 -- 172.21.15.1:0/583950005 >> 172.21.15.1:0/583950005 conn(0x7f05a40fc7a0 msgr2=0x7f05a40fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:55.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 -- 172.21.15.1:0/583950005 shutdown_connections 2024-09-16T00:35:55.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:55.300+0000 7f05ac4a4640 1 -- 172.21.15.1:0/583950005 wait complete. 2024-09-16T00:35:56.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:55 smithi001 ceph-mon[30934]: pgmap v576: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:55 smithi019 ceph-mon[35186]: pgmap v576: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:55 smithi153 ceph-mon[35538]: pgmap v576: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:35:56.960 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:35:56.961 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:35:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.727+0000 7f1c514d5640 1 -- 172.21.15.1:0/1606877489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 msgr2=0x7f1c4c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.727+0000 7f1c514d5640 1 --2- 172.21.15.1:0/1606877489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 0x7f1c4c1021c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009900 tx=0x7f1c3c02a410 comp rx=0 tx=0).stop 2024-09-16T00:35:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.727+0000 7f1c514d5640 1 -- 172.21.15.1:0/1606877489 shutdown_connections 2024-09-16T00:35:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.727+0000 7f1c514d5640 1 --2- 172.21.15.1:0/1606877489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c106f10 0x7f1c4c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.727+0000 7f1c514d5640 1 --2- 172.21.15.1:0/1606877489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c4c102790 0x7f1c4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.727+0000 7f1c514d5640 1 --2- 172.21.15.1:0/1606877489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 0x7f1c4c1021c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.727+0000 7f1c514d5640 1 -- 172.21.15.1:0/1606877489 >> 172.21.15.1:0/1606877489 conn(0x7f1c4c0fd8c0 msgr2=0x7f1c4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.727+0000 7f1c514d5640 1 -- 172.21.15.1:0/1606877489 shutdown_connections 2024-09-16T00:35:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.727+0000 7f1c514d5640 1 -- 172.21.15.1:0/1606877489 wait complete. 2024-09-16T00:35:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.728+0000 7f1c514d5640 1 Processor -- start 2024-09-16T00:35:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.728+0000 7f1c514d5640 1 -- start start 2024-09-16T00:35:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.728+0000 7f1c514d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 0x7f1c4c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.728+0000 7f1c514d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c4c102790 0x7f1c4c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.728+0000 7f1c514d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c106f10 0x7f1c4c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.728+0000 7f1c514d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c111f90 con 0x7f1c4c101de0 2024-09-16T00:35:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.728+0000 7f1c514d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c111e10 con 0x7f1c4c106f10 2024-09-16T00:35:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.728+0000 7f1c514d5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c112110 con 0x7f1c4c102790 2024-09-16T00:35:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 0x7f1c4c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 0x7f1c4c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43816/0 (socket says 172.21.15.1:43816) 2024-09-16T00:35:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4affd640 1 -- 172.21.15.1:0/55328532 learned_addr learned my addr 172.21.15.1:0/55328532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c4c102790 0x7f1c4c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c106f10 0x7f1c4c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4affd640 1 -- 172.21.15.1:0/55328532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c4c102790 msgr2=0x7f1c4c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4affd640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c4c102790 0x7f1c4c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4affd640 1 -- 172.21.15.1:0/55328532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c106f10 msgr2=0x7f1c4c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4affd640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c106f10 0x7f1c4c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4affd640 1 -- 172.21.15.1:0/55328532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c1a6990 con 0x7f1c4c101de0 2024-09-16T00:35:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c4affd640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 0x7f1c4c0744f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006010 tx=0x7f1c3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c43fff640 1 -- 172.21.15.1:0/55328532 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c03f030 con 0x7f1c4c101de0 2024-09-16T00:35:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c43fff640 1 -- 172.21.15.1:0/55328532 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c3c003ce0 con 0x7f1c4c101de0 2024-09-16T00:35:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c514d5640 1 -- 172.21.15.1:0/55328532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c1a6c20 con 0x7f1c4c101de0 2024-09-16T00:35:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c43fff640 1 -- 172.21.15.1:0/55328532 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c035070 con 0x7f1c4c101de0 2024-09-16T00:35:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.729+0000 7f1c514d5640 1 -- 172.21.15.1:0/55328532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c1a7100 con 0x7f1c4c101de0 2024-09-16T00:35:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c43fff640 1 -- 172.21.15.1:0/55328532 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f1c3c042030 con 0x7f1c4c101de0 2024-09-16T00:35:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c538ae740 1 Processor -- start 2024-09-16T00:35:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c538ae740 1 -- start start 2024-09-16T00:35:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59ce5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 0x55ad59cf0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59cf0d40 0x55ad59cf3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c538ae740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad59c4b280 con 0x55ad59ce5b60 2024-09-16T00:35:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c538ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad59bf2b10 con 0x55ad59cf0d40 2024-09-16T00:35:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c538ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad59c47240 con 0x55ad59ce6480 2024-09-16T00:35:57.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.732+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59ce5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59ce5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43828/0 (socket says 172.21.15.1:43828) 2024-09-16T00:35:57.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4b7fe640 1 -- 172.21.15.1:0/1177821366 learned_addr learned my addr 172.21.15.1:0/1177821366 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:57.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c43fff640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c380a3240 0x7f1c380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59cf0d40 0x55ad59cf3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4affd640 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 0x55ad59cf0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c380a3240 0x7f1c380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c43fff640 1 -- 172.21.15.1:0/55328532 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1c3c0f02d0 con 0x7f1c4c101de0 2024-09-16T00:35:57.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4b7fe640 1 -- 172.21.15.1:0/1177821366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 msgr2=0x55ad59cf0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:57.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 0x55ad59cf0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4b7fe640 1 -- 172.21.15.1:0/1177821366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59cf0d40 msgr2=0x55ad59cf3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:57.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59cf0d40 0x55ad59cf3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4b7fe640 1 -- 172.21.15.1:0/1177821366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad59bff1f0 con 0x55ad59ce5b60 2024-09-16T00:35:57.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4affd640 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 0x55ad59cf0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:57.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59cf0d40 0x55ad59cf3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:35:57.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59ce5f40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1c44013a70 tx=0x7f1c44013f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2158559bd757f6f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:57.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c417fa640 1 -- 172.21.15.1:0/1177821366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44013540 con 0x55ad59ce5b60 2024-09-16T00:35:57.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c417fa640 1 -- 172.21.15.1:0/1177821366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c44022070 con 0x55ad59ce5b60 2024-09-16T00:35:57.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.733+0000 7f1c417fa640 1 -- 172.21.15.1:0/1177821366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4401cda0 con 0x55ad59ce5b60 2024-09-16T00:35:57.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c538ae740 1 -- 172.21.15.1:0/1177821366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 msgr2=0x55ad59ce5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c380a3240 0x7f1c380a5700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c44021000 tx=0x7f1c4401f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:57.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c538ae740 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59ce5f40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1c44013a70 tx=0x7f1c44013f30 comp rx=0 tx=0).stop 2024-09-16T00:35:57.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c538ae740 1 -- 172.21.15.1:0/1177821366 shutdown_connections 2024-09-16T00:35:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c538ae740 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59cf0d40 0x55ad59cf3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c538ae740 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 0x55ad59cf0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c538ae740 1 --2- 172.21.15.1:0/1177821366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59ce5f40 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c538ae740 1 -- 172.21.15.1:0/1177821366 >> 172.21.15.1:0/1177821366 conn(0x55ad59ce20f0 msgr2=0x55ad59ce24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c538ae740 1 -- 172.21.15.1:0/1177821366 shutdown_connections 2024-09-16T00:35:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.734+0000 7f1c538ae740 1 -- 172.21.15.1:0/1177821366 wait complete. 2024-09-16T00:35:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c538ae740 1 -- start start 2024-09-16T00:35:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c538ae740 1 Processor -- start 2024-09-16T00:35:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 0x55ad59cedc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 0x55ad59cee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59cf0d40 0x55ad59cee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c538ae740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad59cf3910 con 0x55ad59cf0d40 2024-09-16T00:35:57.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c538ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad59c05f10 con 0x55ad59ce5b60 2024-09-16T00:35:57.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c538ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad59cf3a90 con 0x55ad59ce6480 2024-09-16T00:35:57.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c538ae740 10 client.0 _ll_register_callbacks cb 0x55ad59bcea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:57.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59cf0d40 0x55ad59cee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59cf0d40 0x55ad59cee6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43840/0 (socket says 172.21.15.1:43840) 2024-09-16T00:35:57.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c4b7fe640 1 -- 172.21.15.1:0/501251696 learned_addr learned my addr 172.21.15.1:0/501251696 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:57.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c4affd640 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 0x55ad59cedc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.735+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 0x55ad59cee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c4b7fe640 1 -- 172.21.15.1:0/501251696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 msgr2=0x55ad59cee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:57.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 0x55ad59cee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c4b7fe640 1 -- 172.21.15.1:0/501251696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 msgr2=0x55ad59cedc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 0x55ad59cedc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c4b7fe640 1 -- 172.21.15.1:0/501251696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad599bc6a0 con 0x55ad59cf0d40 2024-09-16T00:35:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59cf0d40 0x55ad59cee6a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1c4401cac0 tx=0x7f1c44009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace2431bc1c136f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c327fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c327fc640 1 -- 172.21.15.1:0/501251696 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44035060 con 0x55ad59cf0d40 2024-09-16T00:35:57.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c327fc640 1 -- 172.21.15.1:0/501251696 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1c4401c350 con 0x55ad59cf0d40 2024-09-16T00:35:57.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c538ae740 1 -- 172.21.15.1:0/501251696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c44022070 con 0x55ad59cf0d40 2024-09-16T00:35:57.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c327fc640 1 -- 172.21.15.1:0/501251696 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c4403c050 con 0x55ad59cf0d40 2024-09-16T00:35:57.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c538ae740 10 client.17178 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:57.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c538ae740 1 -- 172.21.15.1:0/501251696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad59cff540 con 0x55ad59cf0d40 2024-09-16T00:35:57.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c327fc640 1 -- 172.21.15.1:0/501251696 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c44022070 con 0x55ad59cf0d40 2024-09-16T00:35:57.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.736+0000 7f1c327fc640 1 -- 172.21.15.1:0/501251696 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1c44005110 con 0x55ad59cf0d40 2024-09-16T00:35:57.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.737+0000 7f1c538ae740 10 client.17178 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:57.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.737+0000 7f1c538ae740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:57.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.737+0000 7f1c538ae740 20 client.17178 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:57.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.737+0000 7f1c538ae740 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad59d008f0 0x55ad59d02ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.737+0000 7f1c538ae740 4 client.17178 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:57.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.737+0000 7f1c538ae740 1 -- 172.21.15.1:0/501251696 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad59c587f0 con 0x55ad59d008f0 2024-09-16T00:35:57.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.737+0000 7f1c4affd640 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad59d008f0 0x55ad59d02ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.737+0000 7f1c327fc640 10 client.17178 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:57.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.737+0000 7f1c4affd640 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad59d008f0 0x55ad59d02ce0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a0e35f40db939f9 server_cookie=d6a4d6ad65844350 in_seq=0 out_seq=0 2024-09-16T00:35:57.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c327fc640 1 -- 172.21.15.1:0/501251696 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad59c587f0 con 0x55ad59d008f0 2024-09-16T00:35:57.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c327fc640 10 client.17178 handle_command_reply: tid=0 2024-09-16T00:35:57.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 client.17178 shutdown 2024-09-16T00:35:57.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 -- 172.21.15.1:0/501251696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59cf0d40 msgr2=0x55ad59cee6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59cf0d40 0x55ad59cee6a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1c4401cac0 tx=0x7f1c44009940 comp rx=0 tx=0).stop 2024-09-16T00:35:57.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 -- 172.21.15.1:0/501251696 shutdown_connections 2024-09-16T00:35:57.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad59d008f0 0x55ad59d02ce0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59cf0d40 0x55ad59cee6a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c327fc640 0 client.17178 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:57.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59ce6480 0x55ad59cee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 --2- 172.21.15.1:0/501251696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 0x55ad59cedc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 -- 172.21.15.1:0/501251696 >> 172.21.15.1:0/501251696 conn(0x55ad59ce20f0 msgr2=0x55ad59ce1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:57.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 -- 172.21.15.1:0/501251696 shutdown_connections 2024-09-16T00:35:57.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 1 -- 172.21.15.1:0/501251696 wait complete. 2024-09-16T00:35:57.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.738+0000 7f1c538ae740 20 client.17178 trim_cache size 0 max 16384 2024-09-16T00:35:57.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c538ae740 1 Processor -- start 2024-09-16T00:35:57.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c538ae740 1 -- start start 2024-09-16T00:35:57.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 0x55ad59ce85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce6480 0x55ad59d08390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 0x55ad59d088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c538ae740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b3940 con 0x55ad59ce6480 2024-09-16T00:35:57.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c538ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b1210 con 0x55ad59ce5b60 2024-09-16T00:35:57.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c538ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1af060 con 0x55ad59cf0d40 2024-09-16T00:35:57.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c4a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce6480 0x55ad59d08390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c4a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce6480 0x55ad59d08390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43856/0 (socket says 172.21.15.1:43856) 2024-09-16T00:35:57.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.756+0000 7f1c4a7fc640 1 -- 172.21.15.1:0/1076493884 learned_addr learned my addr 172.21.15.1:0/1076493884 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:57.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c4affd640 1 --2- 172.21.15.1:0/1076493884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 0x55ad59ce85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/1076493884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 0x55ad59d088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c4a7fc640 1 -- 172.21.15.1:0/1076493884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 msgr2=0x55ad59d088d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:57.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/1076493884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 0x55ad59d088d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c4a7fc640 1 -- 172.21.15.1:0/1076493884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 msgr2=0x55ad59ce85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:57.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/1076493884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 0x55ad59ce85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c4a7fc640 1 -- 172.21.15.1:0/1076493884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad59cf0220 con 0x55ad59ce6480 2024-09-16T00:35:57.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/1076493884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce6480 0x55ad59d08390 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400ba30 tx=0x7f1c34008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75af3a49af7c785a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:57.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c40ff9640 1 -- 172.21.15.1:0/1076493884 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400ecc0 con 0x55ad59ce6480 2024-09-16T00:35:57.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c40ff9640 1 -- 172.21.15.1:0/1076493884 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c3401c070 con 0x55ad59ce6480 2024-09-16T00:35:57.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.757+0000 7f1c40ff9640 1 -- 172.21.15.1:0/1076493884 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c340167b0 con 0x55ad59ce6480 2024-09-16T00:35:57.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 -- 172.21.15.1:0/1076493884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce6480 msgr2=0x55ad59d08390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 --2- 172.21.15.1:0/1076493884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce6480 0x55ad59d08390 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400ba30 tx=0x7f1c34008040 comp rx=0 tx=0).stop 2024-09-16T00:35:57.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 -- 172.21.15.1:0/1076493884 shutdown_connections 2024-09-16T00:35:57.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 --2- 172.21.15.1:0/1076493884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 0x55ad59d088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 --2- 172.21.15.1:0/1076493884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce6480 0x55ad59d08390 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 --2- 172.21.15.1:0/1076493884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce5b60 0x55ad59ce85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 -- 172.21.15.1:0/1076493884 >> 172.21.15.1:0/1076493884 conn(0x55ad59ce20f0 msgr2=0x55ad59cecc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:57.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 -- 172.21.15.1:0/1076493884 shutdown_connections 2024-09-16T00:35:57.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 -- 172.21.15.1:0/1076493884 wait complete. 2024-09-16T00:35:57.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 -- start start 2024-09-16T00:35:57.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 Processor -- start 2024-09-16T00:35:57.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59cf9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.758+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce6480 0x55ad59cf5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c538ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 0x55ad59cf6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c538ae740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1af6d0 con 0x55ad59ce5b60 2024-09-16T00:35:57.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c538ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b0b40 con 0x55ad59ce6480 2024-09-16T00:35:57.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c538ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1afda0 con 0x55ad59cf0d40 2024-09-16T00:35:57.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c538ae740 10 client.0 _ll_register_callbacks cb 0x7f1c3400b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:57.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59cf9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59cf9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43860/0 (socket says 172.21.15.1:43860) 2024-09-16T00:35:57.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4affd640 1 -- 172.21.15.1:0/607257084 learned_addr learned my addr 172.21.15.1:0/607257084 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:57.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 0x55ad59cf6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce6480 0x55ad59cf5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4affd640 1 -- 172.21.15.1:0/607257084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 msgr2=0x55ad59cf6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:57.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4affd640 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 0x55ad59cf6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4affd640 1 -- 172.21.15.1:0/607257084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce6480 msgr2=0x55ad59cf5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:57.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4affd640 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce6480 0x55ad59cf5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4affd640 1 -- 172.21.15.1:0/607257084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad59ce62b0 con 0x55ad59ce5b60 2024-09-16T00:35:57.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4b7fe640 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 0x55ad59cf6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:57.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce6480 0x55ad59cf5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:57.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c4affd640 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59cf9200 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0bc2b0 tx=0x7f1c3c0f04d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89db1992f79ff7f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:57.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c327fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:57.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c327fc640 1 -- 172.21.15.1:0/607257084 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c049040 con 0x55ad59ce5b60 2024-09-16T00:35:57.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c327fc640 1 -- 172.21.15.1:0/607257084 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1c3c0b8c20 con 0x55ad59ce5b60 2024-09-16T00:35:57.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.759+0000 7f1c538ae740 1 -- 172.21.15.1:0/607257084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c3401c070 con 0x55ad59ce5b60 2024-09-16T00:35:57.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c327fc640 1 -- 172.21.15.1:0/607257084 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c3c03bcb0 con 0x55ad59ce5b60 2024-09-16T00:35:57.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c538ae740 10 client.17184 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c538ae740 1 -- 172.21.15.1:0/607257084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad59d6c010 con 0x55ad59ce5b60 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c327fc640 1 -- 172.21.15.1:0/607257084 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c3c03b580 con 0x55ad59ce5b60 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c327fc640 1 -- 172.21.15.1:0/607257084 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1c3c0bc810 con 0x55ad59ce5b60 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c538ae740 10 client.17184 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c538ae740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c538ae740 20 client.17184 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c538ae740 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad59d6c6c0 0x55ad59d6eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c538ae740 4 client.17184 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c538ae740 1 -- 172.21.15.1:0/607257084 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f1c3401c070 con 0x55ad59d6c6c0 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.760+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad59d6c6c0 0x55ad59d6eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:57.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.761+0000 7f1c327fc640 10 client.17184 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:57.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.761+0000 7f1c4a7fc640 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad59d6c6c0 0x55ad59d6eae0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724f9bff47d893db server_cookie=fdc30dd22a788cc8 in_seq=0 out_seq=0 2024-09-16T00:35:57.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c327fc640 1 -- 172.21.15.1:0/607257084 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f1c3401c070 con 0x55ad59d6c6c0 2024-09-16T00:35:57.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c327fc640 10 client.17184 handle_command_reply: tid=0 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 client.17184 shutdown 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 -- 172.21.15.1:0/607257084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 msgr2=0x55ad59cf9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59cf9200 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0bc2b0 tx=0x7f1c3c0f04d0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 -- 172.21.15.1:0/607257084 shutdown_connections 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad59d6c6c0 0x55ad59d6eae0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad59cf0d40 0x55ad59cf6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad59ce6480 0x55ad59cf5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c327fc640 0 client.17184 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 --2- 172.21.15.1:0/607257084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad59ce5b60 0x55ad59cf9200 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 -- 172.21.15.1:0/607257084 >> 172.21.15.1:0/607257084 conn(0x55ad59ce20f0 msgr2=0x55ad59cec400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:57.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 -- 172.21.15.1:0/607257084 shutdown_connections 2024-09-16T00:35:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 1 -- 172.21.15.1:0/607257084 wait complete. 2024-09-16T00:35:57.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.762+0000 7f1c538ae740 20 client.17184 trim_cache size 0 max 16384 2024-09-16T00:35:57.795 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:35:57.795 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:35:57.795 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "3a6605ba-80dd-45ea-869d-ec1da0f17e82", 2024-09-16T00:35:57.796 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:35:57.796 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:35:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 -- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c380a3240 msgr2=0x7f1c380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c380a3240 0x7f1c380a5700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c44021000 tx=0x7f1c4401f000 comp rx=0 tx=0).stop 2024-09-16T00:35:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 -- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 msgr2=0x7f1c4c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 0x7f1c4c0744f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006010 tx=0x7f1c3c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 -- 172.21.15.1:0/55328532 shutdown_connections 2024-09-16T00:35:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c106f10 0x7f1c4c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1c380a3240 0x7f1c380a5700 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c4c102790 0x7f1c4c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 --2- 172.21.15.1:0/55328532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101de0 0x7f1c4c0744f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 -- 172.21.15.1:0/55328532 >> 172.21.15.1:0/55328532 conn(0x7f1c4c0fd8c0 msgr2=0x7f1c4c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:57.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 -- 172.21.15.1:0/55328532 shutdown_connections 2024-09-16T00:35:57.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:57.763+0000 7f1c514d5640 1 -- 172.21.15.1:0/55328532 wait complete. 2024-09-16T00:35:57.888 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:189/600: open fd = 3 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:190/600: open fd = 3 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:191/600: open fd = 3 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.889 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:192/600: open fd = 3 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:193/600: open fd = 3 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:194/600: open fd = 3 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:195/600: open fd = 3 2024-09-16T00:35:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:196/600: open fd = 3 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:197/600: open fd = 3 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:198/600: open fd = 3 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.892 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.892 INFO:tasks.workunit.client.0.smithi001.stdout:199/600: open fd = 3 2024-09-16T00:35:57.892 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.892 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.892 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.892 INFO:tasks.workunit.client.0.smithi001.stdout:200/600: open fd = 3 2024-09-16T00:35:57.892 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.892 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.892 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:201/600: open fd = 3 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:202/600: open fd = 3 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:203/600: open fd = 3 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:204/600: open fd = 3 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:205/600: open fd = 3 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:206/600: open fd = 3 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:207/600: open fd = 3 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:208/600: open fd = 3 2024-09-16T00:35:57.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:209/600: open fd = 3 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:210/600: open fd = 3 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:211/600: open fd = 3 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.895 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:212/600: open fd = 3 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:213/600: open fd = 3 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:214/600: open fd = 3 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:215/600: open fd = 3 2024-09-16T00:35:57.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:216/600: open fd = 3 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:217/600: open fd = 3 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:218/600: open fd = 3 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.897 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:219/600: open fd = 3 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:220/600: open fd = 3 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:221/600: open fd = 3 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:222/600: open fd = 3 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:223/600: open fd = 3 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:224/600: open fd = 3 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:225/600: open fd = 3 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:226/600: open fd = 3 2024-09-16T00:35:57.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:227/600: open fd = 3 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:228/600: open fd = 3 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:229/600: open fd = 3 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.900 INFO:tasks.workunit.client.0.smithi001.stdout:230/600: open fd = 3 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:231/600: open fd = 3 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:232/600: open fd = 3 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:233/600: open fd = 3 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:234/600: open fd = 3 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:235/600: open fd = 3 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:236/600: open fd = 3 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:237/600: open fd = 3 2024-09-16T00:35:57.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:238/600: open fd = 3 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:239/600: open fd = 3 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:240/600: open fd = 3 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.903 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:241/600: open fd = 3 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:242/600: open fd = 3 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:243/600: open fd = 3 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:244/600: open fd = 3 2024-09-16T00:35:57.904 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:245/600: open fd = 3 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:246/600: open fd = 3 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.905 INFO:tasks.workunit.client.0.smithi001.stdout:247/600: open fd = 3 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:248/600: open fd = 3 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:249/600: open fd = 3 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:250/600: open fd = 3 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:57.906 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:35:58.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:57 smithi019 ceph-mon[35186]: pgmap v577: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:58.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:57 smithi001 ceph-mon[30934]: pgmap v577: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:57 smithi153 ceph-mon[35538]: pgmap v577: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:35:58.545 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:35:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:59.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:59 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:35:59.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:59 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:59.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:59 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:35:59.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:59 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:59.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:35:59 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:35:59.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:59 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:35:59.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:59 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:59.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:59 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:35:59.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:59 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:59.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:35:59 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:35:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.349+0000 7efd47b43640 1 -- 172.21.15.1:0/399712301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 msgr2=0x7efd401089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.349+0000 7efd47b43640 1 --2- 172.21.15.1:0/399712301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 0x7efd401089f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efd30009900 tx=0x7efd3002a410 comp rx=0 tx=0).stop 2024-09-16T00:35:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.349+0000 7efd47b43640 1 -- 172.21.15.1:0/399712301 shutdown_connections 2024-09-16T00:35:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.349+0000 7efd47b43640 1 --2- 172.21.15.1:0/399712301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40109120 0x7efd40110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.349+0000 7efd47b43640 1 --2- 172.21.15.1:0/399712301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 0x7efd401089f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.349+0000 7efd47b43640 1 --2- 172.21.15.1:0/399712301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd40103ff0 0x7efd401043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.349+0000 7efd47b43640 1 -- 172.21.15.1:0/399712301 >> 172.21.15.1:0/399712301 conn(0x7efd400ffab0 msgr2=0x7efd40101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.349+0000 7efd47b43640 1 -- 172.21.15.1:0/399712301 shutdown_connections 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.349+0000 7efd47b43640 1 -- 172.21.15.1:0/399712301 wait complete. 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd47b43640 1 Processor -- start 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd47b43640 1 -- start start 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd47b43640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd40103ff0 0x7efd4019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd47b43640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 0x7efd4019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd47b43640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40109120 0x7efd4019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd47b43640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efd40114140 con 0x7efd401049a0 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd47b43640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd40113fc0 con 0x7efd40103ff0 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd47b43640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd401142c0 con 0x7efd40109120 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd450de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 0x7efd4019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd450de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 0x7efd4019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43878/0 (socket says 172.21.15.1:43878) 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd450de640 1 -- 172.21.15.1:0/3014956648 learned_addr learned my addr 172.21.15.1:0/3014956648 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd458df640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd40103ff0 0x7efd4019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd460e0640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40109120 0x7efd4019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd450de640 1 -- 172.21.15.1:0/3014956648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40109120 msgr2=0x7efd4019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd450de640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40109120 0x7efd4019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd450de640 1 -- 172.21.15.1:0/3014956648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd40103ff0 msgr2=0x7efd4019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.350+0000 7efd450de640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd40103ff0 0x7efd4019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.351+0000 7efd450de640 1 -- 172.21.15.1:0/3014956648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4019f5d0 con 0x7efd401049a0 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.351+0000 7efd450de640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 0x7efd4019ab60 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efd30006010 tx=0x7efd300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.351+0000 7efd36ffd640 1 -- 172.21.15.1:0/3014956648 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3003f030 con 0x7efd401049a0 2024-09-16T00:35:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.351+0000 7efd47b43640 1 -- 172.21.15.1:0/3014956648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4019f860 con 0x7efd401049a0 2024-09-16T00:35:59.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.351+0000 7efd36ffd640 1 -- 172.21.15.1:0/3014956648 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd30003ce0 con 0x7efd401049a0 2024-09-16T00:35:59.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.351+0000 7efd36ffd640 1 -- 172.21.15.1:0/3014956648 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30035070 con 0x7efd401049a0 2024-09-16T00:35:59.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.351+0000 7efd47b43640 1 -- 172.21.15.1:0/3014956648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd401a71a0 con 0x7efd401049a0 2024-09-16T00:35:59.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.353+0000 7efd36ffd640 1 -- 172.21.15.1:0/3014956648 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7efd30042030 con 0x7efd401049a0 2024-09-16T00:35:59.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd49f1c740 1 Processor -- start 2024-09-16T00:35:59.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd49f1c740 1 -- start start 2024-09-16T00:35:59.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b85b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b866140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b868a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd49f1c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d9b7c12e0 con 0x564d9b85b4a0 2024-09-16T00:35:59.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd49f1c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564d9b768ce0 con 0x564d9b85bdc0 2024-09-16T00:35:59.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd49f1c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564d9b7bd2a0 con 0x564d9b866680 2024-09-16T00:35:59.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd460e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b85b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd460e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b85b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43890/0 (socket says 172.21.15.1:43890) 2024-09-16T00:35:59.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd460e0640 1 -- 172.21.15.1:0/3094557090 learned_addr learned my addr 172.21.15.1:0/3094557090 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:59.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd458df640 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b866140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd36ffd640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efd2c0a3240 0x7efd2c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.354+0000 7efd450de640 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b868a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd460e0640 1 -- 172.21.15.1:0/3094557090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 msgr2=0x564d9b868a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:59.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd460e0640 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b868a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd460e0640 1 -- 172.21.15.1:0/3094557090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 msgr2=0x564d9b866140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:59.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd460e0640 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b866140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd460e0640 1 -- 172.21.15.1:0/3094557090 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9b74d440 con 0x564d9b85b4a0 2024-09-16T00:35:59.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd36ffd640 1 -- 172.21.15.1:0/3014956648 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efd300efe80 con 0x7efd401049a0 2024-09-16T00:35:59.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd458df640 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b866140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:35:59.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd450de640 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b868a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:59.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd460e0640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efd2c0a3240 0x7efd2c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd460e0640 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b85b880 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efd38009fc0 tx=0x7efd38010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9954b9f6add35bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:59.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd2bfff640 1 -- 172.21.15.1:0/3094557090 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38021070 con 0x564d9b85b4a0 2024-09-16T00:35:59.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd2bfff640 1 -- 172.21.15.1:0/3094557090 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd3801b3c0 con 0x564d9b85b4a0 2024-09-16T00:35:59.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.355+0000 7efd2bfff640 1 -- 172.21.15.1:0/3094557090 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3800c580 con 0x564d9b85b4a0 2024-09-16T00:35:59.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 -- 172.21.15.1:0/3094557090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 msgr2=0x564d9b85b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b85b880 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efd38009fc0 tx=0x7efd38010800 comp rx=0 tx=0).stop 2024-09-16T00:35:59.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd460e0640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efd2c0a3240 0x7efd2c0a5700 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd3801e000 tx=0x7efd3801c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:59.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 -- 172.21.15.1:0/3094557090 shutdown_connections 2024-09-16T00:35:59.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b868a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b866140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3094557090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b85b880 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 -- 172.21.15.1:0/3094557090 >> 172.21.15.1:0/3094557090 conn(0x564d9b857a30 msgr2=0x564d9b857e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:59.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 -- 172.21.15.1:0/3094557090 shutdown_connections 2024-09-16T00:35:59.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 -- 172.21.15.1:0/3094557090 wait complete. 2024-09-16T00:35:59.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 -- start start 2024-09-16T00:35:59.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.356+0000 7efd49f1c740 1 Processor -- start 2024-09-16T00:35:59.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b865ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b8632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b863800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd49f1c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d9b869250 con 0x564d9b85b4a0 2024-09-16T00:35:59.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd49f1c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564d9b77c050 con 0x564d9b85bdc0 2024-09-16T00:35:59.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd49f1c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564d9b8693d0 con 0x564d9b866680 2024-09-16T00:35:59.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd49f1c740 10 client.0 _ll_register_callbacks cb 0x564d9b7bc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:59.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd458df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b865ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd458df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b865ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43896/0 (socket says 172.21.15.1:43896) 2024-09-16T00:35:59.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd458df640 1 -- 172.21.15.1:0/3712389115 learned_addr learned my addr 172.21.15.1:0/3712389115 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:59.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd450de640 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b8632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd460e0640 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b863800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd458df640 1 -- 172.21.15.1:0/3712389115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 msgr2=0x564d9b863800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:59.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd458df640 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b863800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd458df640 1 -- 172.21.15.1:0/3712389115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 msgr2=0x564d9b8632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:59.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd458df640 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b8632a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd458df640 1 -- 172.21.15.1:0/3712389115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9b5323c0 con 0x564d9b85b4a0 2024-09-16T00:35:59.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd460e0640 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b863800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:59.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.357+0000 7efd450de640 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b8632a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:59.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd458df640 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b865ff0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0126e0 tx=0x7efd3c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d2dfdff984fdc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:59.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd297fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:59.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd297fa640 1 -- 172.21.15.1:0/3712389115 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0121d0 con 0x564d9b85b4a0 2024-09-16T00:35:59.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd297fa640 1 -- 172.21.15.1:0/3712389115 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efd3c004030 con 0x564d9b85b4a0 2024-09-16T00:35:59.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd49f1c740 1 -- 172.21.15.1:0/3712389115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd3801b3c0 con 0x564d9b85b4a0 2024-09-16T00:35:59.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd297fa640 1 -- 172.21.15.1:0/3712389115 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd3c004230 con 0x564d9b85b4a0 2024-09-16T00:35:59.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd49f1c740 10 client.17196 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:59.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd49f1c740 1 -- 172.21.15.1:0/3712389115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d9b874bf0 con 0x564d9b85b4a0 2024-09-16T00:35:59.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd297fa640 1 -- 172.21.15.1:0/3712389115 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd3801b3c0 con 0x564d9b85b4a0 2024-09-16T00:35:59.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd297fa640 1 -- 172.21.15.1:0/3712389115 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efd3c0039f0 con 0x564d9b85b4a0 2024-09-16T00:35:59.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd49f1c740 10 client.17196 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:59.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd49f1c740 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:59.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd49f1c740 20 client.17196 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:59.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564d9b8760b0 0x564d9b8784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd49f1c740 4 client.17196 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:59.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.358+0000 7efd49f1c740 1 -- 172.21.15.1:0/3712389115 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d9b7ce850 con 0x564d9b8760b0 2024-09-16T00:35:59.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.359+0000 7efd450de640 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564d9b8760b0 0x564d9b8784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.359+0000 7efd450de640 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564d9b8760b0 0x564d9b8784e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=400ad8270fbfb890 server_cookie=cca4a05365a4337 in_seq=0 out_seq=0 2024-09-16T00:35:59.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.359+0000 7efd297fa640 10 client.17196 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:59.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd297fa640 1 -- 172.21.15.1:0/3712389115 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d9b7ce850 con 0x564d9b8760b0 2024-09-16T00:35:59.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd297fa640 10 client.17196 handle_command_reply: tid=0 2024-09-16T00:35:59.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 client.17196 shutdown 2024-09-16T00:35:59.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 -- 172.21.15.1:0/3712389115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 msgr2=0x564d9b865ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b865ff0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0126e0 tx=0x7efd3c012ba0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 -- 172.21.15.1:0/3712389115 shutdown_connections 2024-09-16T00:35:59.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564d9b8760b0 0x564d9b8784e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b863800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b8632a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 --2- 172.21.15.1:0/3712389115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b865ff0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd297fa640 0 client.17196 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:59.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 -- 172.21.15.1:0/3712389115 >> 172.21.15.1:0/3712389115 conn(0x564d9b857a30 msgr2=0x564d9b85d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:59.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 -- 172.21.15.1:0/3712389115 shutdown_connections 2024-09-16T00:35:59.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 1 -- 172.21.15.1:0/3712389115 wait complete. 2024-09-16T00:35:59.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.360+0000 7efd49f1c740 20 client.17196 trim_cache size 0 max 16384 2024-09-16T00:35:59.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.377+0000 7efd49f1c740 1 Processor -- start 2024-09-16T00:35:59.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.377+0000 7efd49f1c740 1 -- start start 2024-09-16T00:35:59.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b861b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b87ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b87e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd49f1c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efd401b3980 con 0x564d9b85b4a0 2024-09-16T00:35:59.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd49f1c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd401b1250 con 0x564d9b85bdc0 2024-09-16T00:35:59.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd49f1c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd401af0a0 con 0x564d9b866680 2024-09-16T00:35:59.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd458df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b861b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd458df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b861b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43908/0 (socket says 172.21.15.1:43908) 2024-09-16T00:35:59.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd458df640 1 -- 172.21.15.1:0/136734601 learned_addr learned my addr 172.21.15.1:0/136734601 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:59.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd450de640 1 --2- 172.21.15.1:0/136734601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b87ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd460e0640 1 --2- 172.21.15.1:0/136734601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b87e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd458df640 1 -- 172.21.15.1:0/136734601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 msgr2=0x564d9b87e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:59.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd458df640 1 --2- 172.21.15.1:0/136734601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b87e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd458df640 1 -- 172.21.15.1:0/136734601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 msgr2=0x564d9b87ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:59.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd458df640 1 --2- 172.21.15.1:0/136734601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b87ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd458df640 1 -- 172.21.15.1:0/136734601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9b862250 con 0x564d9b85b4a0 2024-09-16T00:35:59.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd458df640 1 --2- 172.21.15.1:0/136734601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b861b70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7efd3c001eb0 tx=0x7efd3c009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4a8023d91e3246 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:59.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd2b7fe640 1 -- 172.21.15.1:0/136734601 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c02f070 con 0x564d9b85b4a0 2024-09-16T00:35:59.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd2b7fe640 1 -- 172.21.15.1:0/136734601 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd3c00fe00 con 0x564d9b85b4a0 2024-09-16T00:35:59.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.378+0000 7efd2b7fe640 1 -- 172.21.15.1:0/136734601 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0035f0 con 0x564d9b85b4a0 2024-09-16T00:35:59.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.379+0000 7efd49f1c740 1 -- 172.21.15.1:0/136734601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 msgr2=0x564d9b861b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.379+0000 7efd49f1c740 1 --2- 172.21.15.1:0/136734601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b861b70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7efd3c001eb0 tx=0x7efd3c009fa0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.379+0000 7efd49f1c740 1 -- 172.21.15.1:0/136734601 shutdown_connections 2024-09-16T00:35:59.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.379+0000 7efd49f1c740 1 --2- 172.21.15.1:0/136734601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b866680 0x564d9b87e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.379+0000 7efd49f1c740 1 --2- 172.21.15.1:0/136734601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b85bdc0 0x564d9b87ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.379+0000 7efd49f1c740 1 --2- 172.21.15.1:0/136734601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85b4a0 0x564d9b861b70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.379+0000 7efd49f1c740 1 -- 172.21.15.1:0/136734601 >> 172.21.15.1:0/136734601 conn(0x564d9b857a30 msgr2=0x564d9b856b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.379+0000 7efd49f1c740 1 -- 172.21.15.1:0/136734601 shutdown_connections 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.379+0000 7efd49f1c740 1 -- 172.21.15.1:0/136734601 wait complete. 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd49f1c740 1 -- start start 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd49f1c740 1 Processor -- start 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b85b4a0 0x564d9b86c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85bdc0 0x564d9b86cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd49f1c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b866680 0x564d9b86d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd49f1c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efd401af710 con 0x564d9b85bdc0 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd49f1c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd401b0b80 con 0x564d9b866680 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd49f1c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd401afde0 con 0x564d9b85b4a0 2024-09-16T00:35:59.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd49f1c740 10 client.0 _ll_register_callbacks cb 0x7efd3c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd450de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85bdc0 0x564d9b86cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd450de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85bdc0 0x564d9b86cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43916/0 (socket says 172.21.15.1:43916) 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd450de640 1 -- 172.21.15.1:0/903972777 learned_addr learned my addr 172.21.15.1:0/903972777 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd460e0640 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b866680 0x564d9b86d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.380+0000 7efd458df640 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b85b4a0 0x564d9b86c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd450de640 1 -- 172.21.15.1:0/903972777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b85b4a0 msgr2=0x564d9b86c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd450de640 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b85b4a0 0x564d9b86c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd450de640 1 -- 172.21.15.1:0/903972777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b866680 msgr2=0x564d9b86d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd450de640 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b866680 0x564d9b86d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd450de640 1 -- 172.21.15.1:0/903972777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9b866360 con 0x564d9b85bdc0 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd458df640 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b85b4a0 0x564d9b86c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd450de640 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85bdc0 0x564d9b86cbd0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7efd30000c00 tx=0x7efd3008af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6729769009ddb66 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd297fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:35:59.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd297fa640 1 -- 172.21.15.1:0/903972777 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30088070 con 0x564d9b85bdc0 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd297fa640 1 -- 172.21.15.1:0/903972777 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efd300ed600 con 0x564d9b85bdc0 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd49f1c740 1 -- 172.21.15.1:0/903972777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd3c00fe00 con 0x564d9b85bdc0 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd297fa640 1 -- 172.21.15.1:0/903972777 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd30049040 con 0x564d9b85bdc0 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd49f1c740 10 client.17202 fetch_fsmap learned FSMap version 25 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd49f1c740 1 -- 172.21.15.1:0/903972777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d9b8e11a0 con 0x564d9b85bdc0 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.381+0000 7efd297fa640 1 -- 172.21.15.1:0/903972777 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd3c00fe00 con 0x564d9b85bdc0 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd297fa640 1 -- 172.21.15.1:0/903972777 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efd300d7080 con 0x564d9b85bdc0 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd49f1c740 10 client.17202 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd49f1c740 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd49f1c740 20 client.17202 populate_metadata read hostname 'smithi001' 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd49f1c740 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564d9b8e18d0 0x564d9b8e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd49f1c740 4 client.17202 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd49f1c740 1 -- 172.21.15.1:0/903972777 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d9b7c8d60 con 0x564d9b8e18d0 2024-09-16T00:35:59.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd458df640 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564d9b8e18d0 0x564d9b8e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd458df640 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564d9b8e18d0 0x564d9b8e3cf0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c485be78bf241267 server_cookie=44cd02a6cb9f44cc in_seq=0 out_seq=0 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd297fa640 10 client.17202 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd297fa640 1 -- 172.21.15.1:0/903972777 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564d9b7c8d60 con 0x564d9b8e18d0 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd297fa640 10 client.17202 handle_command_reply: tid=0 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.382+0000 7efd49f1c740 1 client.17202 shutdown 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 -- 172.21.15.1:0/903972777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85bdc0 msgr2=0x564d9b86cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85bdc0 0x564d9b86cbd0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7efd30000c00 tx=0x7efd3008af60 comp rx=0 tx=0).stop 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 -- 172.21.15.1:0/903972777 shutdown_connections 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564d9b8e18d0 0x564d9b8e3cf0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d9b866680 0x564d9b86d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d9b85bdc0 0x564d9b86cbd0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 --2- 172.21.15.1:0/903972777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d9b85b4a0 0x564d9b86c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd297fa640 0 client.17202 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:35:59.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 -- 172.21.15.1:0/903972777 >> 172.21.15.1:0/903972777 conn(0x564d9b857a30 msgr2=0x564d9b864b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:59.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 -- 172.21.15.1:0/903972777 shutdown_connections 2024-09-16T00:35:59.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 1 -- 172.21.15.1:0/903972777 wait complete. 2024-09-16T00:35:59.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd49f1c740 20 client.17202 trim_cache size 0 max 16384 2024-09-16T00:35:59.426 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:35:59.426 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:35:59.426 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:35:59.426 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd47b43640 1 -- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efd2c0a3240 msgr2=0x7efd2c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.383+0000 7efd47b43640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efd2c0a3240 0x7efd2c0a5700 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd3801e000 tx=0x7efd3801c000 comp rx=0 tx=0).stop 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 -- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 msgr2=0x7efd4019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 0x7efd4019ab60 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efd30006010 tx=0x7efd300042f0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 -- 172.21.15.1:0/3014956648 shutdown_connections 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efd2c0a3240 0x7efd2c0a5700 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40109120 0x7efd4019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd401049a0 0x7efd4019ab60 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 --2- 172.21.15.1:0/3014956648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd40103ff0 0x7efd4019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 -- 172.21.15.1:0/3014956648 >> 172.21.15.1:0/3014956648 conn(0x7efd400ffab0 msgr2=0x7efd40100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 -- 172.21.15.1:0/3014956648 shutdown_connections 2024-09-16T00:35:59.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:35:59.384+0000 7efd47b43640 1 -- 172.21.15.1:0/3014956648 wait complete. 2024-09-16T00:35:59.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:59 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:35:59.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:59 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:59.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:59 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:35:59.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:59 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:35:59.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:35:59 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:36:00.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:36:00.155 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:36:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:00 smithi019 ceph-mon[35186]: pgmap v578: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:36:00.415 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:00 smithi001 ceph-mon[30934]: pgmap v578: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:36:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:00 smithi153 ceph-mon[35538]: pgmap v578: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 -- 172.21.15.1:0/1030800343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 msgr2=0x7f875810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 --2- 172.21.15.1:0/1030800343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 0x7f875810e610 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8750009be0 tx=0x7f875002a750 comp rx=0 tx=0).stop 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 -- 172.21.15.1:0/1030800343 shutdown_connections 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 --2- 172.21.15.1:0/1030800343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 0x7f875810e610 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 --2- 172.21.15.1:0/1030800343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758103660 0x7f875810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 --2- 172.21.15.1:0/1030800343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8758102d40 0x7f8758103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 -- 172.21.15.1:0/1030800343 >> 172.21.15.1:0/1030800343 conn(0x7f87580fc7a0 msgr2=0x7f87580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 -- 172.21.15.1:0/1030800343 shutdown_connections 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 -- 172.21.15.1:0/1030800343 wait complete. 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 Processor -- start 2024-09-16T00:36:00.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.955+0000 7f875ef00640 1 -- start start 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875ef00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758102d40 0x7f87581983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875ef00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8758103660 0x7f8758198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875ef00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 0x7f875819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875ef00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8758111f20 con 0x7f875810c170 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875ef00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8758111da0 con 0x7f8758102d40 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875ef00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87581120a0 con 0x7f8758103660 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875d49d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 0x7f875819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875cc9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758102d40 0x7f87581983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875d49d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 0x7f875819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58642/0 (socket says 172.21.15.1:58642) 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875d49d640 1 -- 172.21.15.1:0/3153357648 learned_addr learned my addr 172.21.15.1:0/3153357648 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f8757fff640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8758103660 0x7f8758198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875d49d640 1 -- 172.21.15.1:0/3153357648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8758103660 msgr2=0x7f8758198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875d49d640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8758103660 0x7f8758198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875d49d640 1 -- 172.21.15.1:0/3153357648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758102d40 msgr2=0x7f87581983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875d49d640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758102d40 0x7f87581983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875d49d640 1 -- 172.21.15.1:0/3153357648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875819d390 con 0x7f875810c170 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875cc9c640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758102d40 0x7f87581983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f875d49d640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 0x7f875819ccb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8750009d00 tx=0x7f8750030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.956+0000 7f8755ffb640 1 -- 172.21.15.1:0/3153357648 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750041020 con 0x7f875810c170 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.957+0000 7f8755ffb640 1 -- 172.21.15.1:0/3153357648 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8750037040 con 0x7f875810c170 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.957+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875819d620 con 0x7f875810c170 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.957+0000 7f8755ffb640 1 -- 172.21.15.1:0/3153357648 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750036070 con 0x7f875810c170 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.957+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87581a4f60 con 0x7f875810c170 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.958+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8758103120 con 0x7f875810c170 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.959+0000 7f8755ffb640 1 -- 172.21.15.1:0/3153357648 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8750044020 con 0x7f875810c170 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.960+0000 7f8755ffb640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f87400a32c0 0x7f87400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.960+0000 7f8755ffb640 1 -- 172.21.15.1:0/3153357648 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f87500f0bf0 con 0x7f875810c170 2024-09-16T00:36:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.960+0000 7f875cc9c640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f87400a32c0 0x7f87400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:00.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.960+0000 7f875cc9c640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f87400a32c0 0x7f87400a5780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8748005fd0 tx=0x7f874800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:00.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:00.963+0000 7f8755ffb640 1 -- 172.21.15.1:0/3153357648 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f875004b020 con 0x7f875810c170 2024-09-16T00:36:01.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.180+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f875802d7c0 con 0x7f875810c170 2024-09-16T00:36:01.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.181+0000 7f8755ffb640 1 -- 172.21.15.1:0/3153357648 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f87500bd630 con 0x7f875810c170 2024-09-16T00:36:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:36:01.181 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:36:01.182 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:36:01.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f87400a32c0 msgr2=0x7f87400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f87400a32c0 0x7f87400a5780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8748005fd0 tx=0x7f874800b390 comp rx=0 tx=0).stop 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 msgr2=0x7f875819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 0x7f875819ccb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8750009d00 tx=0x7f8750030650 comp rx=0 tx=0).stop 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 shutdown_connections 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f875810c170 0x7f875819ccb0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f87400a32c0 0x7f87400a5780 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8758103660 0x7f8758198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 --2- 172.21.15.1:0/3153357648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758102d40 0x7f87581983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 >> 172.21.15.1:0/3153357648 conn(0x7f87580fc7a0 msgr2=0x7f87580fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 shutdown_connections 2024-09-16T00:36:01.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:01.183+0000 7f875ef00640 1 -- 172.21.15.1:0/3153357648 wait complete. 2024-09-16T00:36:01.881 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:36:02.138 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:02.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:02 smithi001 ceph-mon[30934]: pgmap v579: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:02.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:02 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3153357648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:02.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:02 smithi019 ceph-mon[35186]: pgmap v579: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:02 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3153357648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:02 smithi153 ceph-mon[35538]: pgmap v579: 129 pgs: 129 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:02 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3153357648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:02.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.665+0000 7fa323ecd640 1 -- 172.21.15.1:0/3514142922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102570 msgr2=0x7fa31c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.665+0000 7fa323ecd640 1 --2- 172.21.15.1:0/3514142922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102570 0x7fa31c1029b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009900 tx=0x7fa30c02a480 comp rx=0 tx=0).stop 2024-09-16T00:36:02.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.665+0000 7fa323ecd640 1 -- 172.21.15.1:0/3514142922 shutdown_connections 2024-09-16T00:36:02.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.665+0000 7fa323ecd640 1 --2- 172.21.15.1:0/3514142922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa31c102ef0 0x7fa31c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.665+0000 7fa323ecd640 1 --2- 172.21.15.1:0/3514142922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102570 0x7fa31c1029b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.665+0000 7fa323ecd640 1 --2- 172.21.15.1:0/3514142922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa31c100e20 0x7fa31c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.665+0000 7fa323ecd640 1 -- 172.21.15.1:0/3514142922 >> 172.21.15.1:0/3514142922 conn(0x7fa31c0fc7d0 msgr2=0x7fa31c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.665+0000 7fa323ecd640 1 -- 172.21.15.1:0/3514142922 shutdown_connections 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.665+0000 7fa323ecd640 1 -- 172.21.15.1:0/3514142922 wait complete. 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa323ecd640 1 Processor -- start 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa323ecd640 1 -- start start 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa323ecd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa31c100e20 0x7fa31c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa323ecd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa31c102570 0x7fa31c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa323ecd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102ef0 0x7fa31c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa323ecd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c110f80 con 0x7fa31c102ef0 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa323ecd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c110e00 con 0x7fa31c102570 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa323ecd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c111100 con 0x7fa31c100e20 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa32246a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102ef0 0x7fa31c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa32246a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102ef0 0x7fa31c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58652/0 (socket says 172.21.15.1:58652) 2024-09-16T00:36:02.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa32246a640 1 -- 172.21.15.1:0/1037504516 learned_addr learned my addr 172.21.15.1:0/1037504516 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa321c69640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa31c100e20 0x7fa31c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa321468640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa31c102570 0x7fa31c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa32246a640 1 -- 172.21.15.1:0/1037504516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa31c100e20 msgr2=0x7fa31c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa32246a640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa31c100e20 0x7fa31c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa32246a640 1 -- 172.21.15.1:0/1037504516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa31c102570 msgr2=0x7fa31c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa32246a640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa31c102570 0x7fa31c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa32246a640 1 -- 172.21.15.1:0/1037504516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c1a6990 con 0x7fa31c102ef0 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.666+0000 7fa321468640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa31c102570 0x7fa31c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.667+0000 7fa32246a640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102ef0 0x7fa31c0fd700 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa31400eaa0 tx=0x7fa31400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.667+0000 7fa312ffd640 1 -- 172.21.15.1:0/1037504516 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31400cdb0 con 0x7fa31c102ef0 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.667+0000 7fa323ecd640 1 -- 172.21.15.1:0/1037504516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c1a6c80 con 0x7fa31c102ef0 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.667+0000 7fa312ffd640 1 -- 172.21.15.1:0/1037504516 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314005020 con 0x7fa31c102ef0 2024-09-16T00:36:02.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.667+0000 7fa312ffd640 1 -- 172.21.15.1:0/1037504516 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140116d0 con 0x7fa31c102ef0 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.667+0000 7fa323ecd640 1 -- 172.21.15.1:0/1037504516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c1a71c0 con 0x7fa31c102ef0 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa3262a6740 1 Processor -- start 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa312ffd640 1 -- 172.21.15.1:0/1037504516 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa3140054d0 con 0x7fa31c102ef0 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa3262a6740 1 -- start start 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a512b40 0x55e00a511090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a5115d0 0x55e00a511a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a51e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa3262a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e00a4772f0 con 0x55e00a512b40 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa3262a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e00a41ecf0 con 0x55e00a5115d0 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa3262a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e00a4732b0 con 0x55e00a51c370 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa321468640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a512b40 0x55e00a511090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa321468640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a512b40 0x55e00a511090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58656/0 (socket says 172.21.15.1:58656) 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.670+0000 7fa321468640 1 -- 172.21.15.1:0/1690823718 learned_addr learned my addr 172.21.15.1:0/1690823718 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa321c69640 1 --2- 172.21.15.1:0/1690823718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a5115d0 0x55e00a511a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa321468640 1 -- 172.21.15.1:0/1690823718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 msgr2=0x55e00a51e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa321468640 1 --2- 172.21.15.1:0/1690823718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a51e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa321468640 1 -- 172.21.15.1:0/1690823718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a5115d0 msgr2=0x55e00a511a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa321468640 1 --2- 172.21.15.1:0/1690823718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a5115d0 0x55e00a511a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa321468640 1 -- 172.21.15.1:0/1690823718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e00a4033a0 con 0x55e00a512b40 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa312ffd640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3080a3240 0x7fa3080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa321468640 1 --2- 172.21.15.1:0/1690823718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a512b40 0x55e00a511090 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa30c006010 tx=0x7fa30c004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eff5892a02334f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa321c69640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3080a3240 0x7fa3080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa312ffd640 1 -- 172.21.15.1:0/1037504516 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa3140cffa0 con 0x7fa31c102ef0 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa307fff640 1 -- 172.21.15.1:0/1690823718 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c040020 con 0x55e00a512b40 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa307fff640 1 -- 172.21.15.1:0/1690823718 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c005520 con 0x55e00a512b40 2024-09-16T00:36:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.671+0000 7fa307fff640 1 -- 172.21.15.1:0/1690823718 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c035070 con 0x55e00a512b40 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa321c69640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3080a3240 0x7fa3080a5700 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55e00a50eac0 tx=0x7fa31800b4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 -- 172.21.15.1:0/1690823718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a512b40 msgr2=0x55e00a511090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1690823718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a512b40 0x55e00a511090 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa30c006010 tx=0x7fa30c004180 comp rx=0 tx=0).stop 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 -- 172.21.15.1:0/1690823718 shutdown_connections 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1690823718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a51e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1690823718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a5115d0 0x55e00a511a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1690823718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a512b40 0x55e00a511090 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 -- 172.21.15.1:0/1690823718 >> 172.21.15.1:0/1690823718 conn(0x55e00a4da310 msgr2=0x55e00a50dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 -- 172.21.15.1:0/1690823718 shutdown_connections 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 -- 172.21.15.1:0/1690823718 wait complete. 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.672+0000 7fa3262a6740 1 -- start start 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa3262a6740 1 Processor -- start 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a51bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 0x55e00a519080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a519810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa3262a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e00a51f010 con 0x55e00a5115d0 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa3262a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e00a432060 con 0x55e00a512b40 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa3262a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e00a51f190 con 0x55e00a51c370 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa3262a6740 10 client.0 _ll_register_callbacks cb 0x55e00a1858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa321c69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a51bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa321c69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a51bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58666/0 (socket says 172.21.15.1:58666) 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa321c69640 1 -- 172.21.15.1:0/3561259721 learned_addr learned my addr 172.21.15.1:0/3561259721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa321468640 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 0x55e00a519080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa321c69640 1 -- 172.21.15.1:0/3561259721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 msgr2=0x55e00a519810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa321c69640 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a519810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa321c69640 1 -- 172.21.15.1:0/3561259721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 msgr2=0x55e00a519080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa321c69640 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 0x55e00a519080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.673+0000 7fa321c69640 1 -- 172.21.15.1:0/3561259721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e00a1e83c0 con 0x55e00a5115d0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa321c69640 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a51bdd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa31801d6f0 tx=0x7fa3180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1527fa2f92c1132 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3057fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3057fa640 1 -- 172.21.15.1:0/3561259721 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31801de60 con 0x55e00a5115d0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3057fa640 1 -- 172.21.15.1:0/3561259721 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa31801e820 con 0x55e00a5115d0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3262a6740 1 -- 172.21.15.1:0/3561259721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa30c005520 con 0x55e00a5115d0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3057fa640 1 -- 172.21.15.1:0/3561259721 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa31801ea20 con 0x55e00a5115d0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3262a6740 10 client.17220 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3262a6740 1 -- 172.21.15.1:0/3561259721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e00a52abc0 con 0x55e00a5115d0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3057fa640 1 -- 172.21.15.1:0/3561259721 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c005520 con 0x55e00a5115d0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3057fa640 1 -- 172.21.15.1:0/3561259721 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa31801b390 con 0x55e00a5115d0 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3262a6740 10 client.17220 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3262a6740 10 client.17220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3262a6740 20 client.17220 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3262a6740 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e00a52c0d0 0x55e00a52e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3262a6740 4 client.17220 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.674+0000 7fa3262a6740 1 -- 172.21.15.1:0/3561259721 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e00a484860 con 0x55e00a52c0d0 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.675+0000 7fa321468640 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e00a52c0d0 0x55e00a52e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.675+0000 7fa3057fa640 10 client.17220 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.675+0000 7fa321468640 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e00a52c0d0 0x55e00a52e560 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ed1ac6013d24b32 server_cookie=95c2c39df468cad5 in_seq=0 out_seq=0 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3057fa640 1 -- 172.21.15.1:0/3561259721 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e00a484860 con 0x55e00a52c0d0 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3057fa640 10 client.17220 handle_command_reply: tid=0 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 client.17220 shutdown 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 -- 172.21.15.1:0/3561259721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 msgr2=0x55e00a51bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a51bdd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa31801d6f0 tx=0x7fa3180046c0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 -- 172.21.15.1:0/3561259721 shutdown_connections 2024-09-16T00:36:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a519810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e00a52c0d0 0x55e00a52e560 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 0x55e00a519080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 --2- 172.21.15.1:0/3561259721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a51bdd0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3057fa640 0 client.17220 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 -- 172.21.15.1:0/3561259721 >> 172.21.15.1:0/3561259721 conn(0x55e00a4da310 msgr2=0x55e00a513930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 -- 172.21.15.1:0/3561259721 shutdown_connections 2024-09-16T00:36:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 1 -- 172.21.15.1:0/3561259721 wait complete. 2024-09-16T00:36:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.676+0000 7fa3262a6740 20 client.17220 trim_cache size 0 max 16384 2024-09-16T00:36:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.689+0000 7fa3262a6740 1 Processor -- start 2024-09-16T00:36:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.689+0000 7fa3262a6740 1 -- start start 2024-09-16T00:36:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a510fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a512b40 0x55e00a517950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a51c370 0x55e00a517e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa3262a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1b3970 con 0x55e00a5115d0 2024-09-16T00:36:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa3262a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1b1240 con 0x55e00a51c370 2024-09-16T00:36:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa3262a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1af090 con 0x55e00a512b40 2024-09-16T00:36:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321c69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a510fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321c69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a510fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58676/0 (socket says 172.21.15.1:58676) 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321c69640 1 -- 172.21.15.1:0/2851696428 learned_addr learned my addr 172.21.15.1:0/2851696428 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321468640 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a512b40 0x55e00a517950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa32246a640 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a51c370 0x55e00a517e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321c69640 1 -- 172.21.15.1:0/2851696428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a512b40 msgr2=0x55e00a517950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321c69640 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a512b40 0x55e00a517950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321c69640 1 -- 172.21.15.1:0/2851696428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a51c370 msgr2=0x55e00a517e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321c69640 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a51c370 0x55e00a517e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321c69640 1 -- 172.21.15.1:0/2851696428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e00a51f180 con 0x55e00a5115d0 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa32246a640 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a51c370 0x55e00a517e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa321c69640 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a510fb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa31801b830 tx=0x7fa31800b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e57b50da73e2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa3077fe640 1 -- 172.21.15.1:0/2851696428 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31801b590 con 0x55e00a5115d0 2024-09-16T00:36:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa3077fe640 1 -- 172.21.15.1:0/2851696428 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31803d070 con 0x55e00a5115d0 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.690+0000 7fa3077fe640 1 -- 172.21.15.1:0/2851696428 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318005040 con 0x55e00a5115d0 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 -- 172.21.15.1:0/2851696428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 msgr2=0x55e00a510fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a510fb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa31801b830 tx=0x7fa31800b5f0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 -- 172.21.15.1:0/2851696428 shutdown_connections 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a51c370 0x55e00a517e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a512b40 0x55e00a517950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 --2- 172.21.15.1:0/2851696428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a510fb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 -- 172.21.15.1:0/2851696428 >> 172.21.15.1:0/2851696428 conn(0x55e00a4da310 msgr2=0x55e00a508a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 -- 172.21.15.1:0/2851696428 shutdown_connections 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 -- 172.21.15.1:0/2851696428 wait complete. 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.691+0000 7fa3262a6740 1 -- start start 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa3262a6740 1 Processor -- start 2024-09-16T00:36:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a524730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 0x55e00a524c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa3262a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a515aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa3262a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1af700 con 0x55e00a5115d0 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa3262a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1b0b70 con 0x55e00a512b40 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa3262a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1afdd0 con 0x55e00a51c370 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa3262a6740 10 client.0 _ll_register_callbacks cb 0x7fa31800b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa321c69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a524730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa321c69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a524730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58684/0 (socket says 172.21.15.1:58684) 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa321c69640 1 -- 172.21.15.1:0/1605745337 learned_addr learned my addr 172.21.15.1:0/1605745337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa321468640 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 0x55e00a524c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.692+0000 7fa32246a640 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a515aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa321c69640 1 -- 172.21.15.1:0/1605745337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 msgr2=0x55e00a515aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa321c69640 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a515aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa321c69640 1 -- 172.21.15.1:0/1605745337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 msgr2=0x55e00a524c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa321c69640 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 0x55e00a524c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa321c69640 1 -- 172.21.15.1:0/1605745337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e00a5162d0 con 0x55e00a5115d0 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa321c69640 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a524730 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa318013fd0 tx=0x7fa318012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9df09e83cb1375 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa3057fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa3057fa640 1 -- 172.21.15.1:0/1605745337 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318045020 con 0x55e00a5115d0 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa3057fa640 1 -- 172.21.15.1:0/1605745337 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa31801bdb0 con 0x55e00a5115d0 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa3262a6740 1 -- 172.21.15.1:0/1605745337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa31803d070 con 0x55e00a5115d0 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa3057fa640 1 -- 172.21.15.1:0/1605745337 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa318050050 con 0x55e00a5115d0 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa3262a6740 10 client.17226 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa3262a6740 1 -- 172.21.15.1:0/1605745337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e00a596a00 con 0x55e00a5115d0 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.693+0000 7fa3057fa640 1 -- 172.21.15.1:0/1605745337 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31803d070 con 0x55e00a5115d0 2024-09-16T00:36:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa3057fa640 1 -- 172.21.15.1:0/1605745337 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa318004c30 con 0x55e00a5115d0 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa3262a6740 10 client.17226 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa3262a6740 10 client.17226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa3262a6740 20 client.17226 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e00a597060 0x55e00a599450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa3262a6740 4 client.17226 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa3262a6740 1 -- 172.21.15.1:0/1605745337 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e00a47ed70 con 0x55e00a597060 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa321468640 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e00a597060 0x55e00a599450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa321468640 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e00a597060 0x55e00a599450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bee8a75e3cf28e2 server_cookie=6723cf4ba36f405d in_seq=0 out_seq=0 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.694+0000 7fa3057fa640 10 client.17226 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3057fa640 1 -- 172.21.15.1:0/1605745337 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e00a47ed70 con 0x55e00a597060 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3057fa640 10 client.17226 handle_command_reply: tid=0 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 client.17226 shutdown 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 -- 172.21.15.1:0/1605745337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 msgr2=0x55e00a524730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a524730 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa318013fd0 tx=0x7fa318012e60 comp rx=0 tx=0).stop 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 -- 172.21.15.1:0/1605745337 shutdown_connections 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e00a51c370 0x55e00a515aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e00a597060 0x55e00a599450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e00a512b40 0x55e00a524c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 --2- 172.21.15.1:0/1605745337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e00a5115d0 0x55e00a524730 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 -- 172.21.15.1:0/1605745337 >> 172.21.15.1:0/1605745337 conn(0x55e00a4da310 msgr2=0x55e00a5178c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3057fa640 0 client.17226 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 -- 172.21.15.1:0/1605745337 shutdown_connections 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 1 -- 172.21.15.1:0/1605745337 wait complete. 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa3262a6740 20 client.17226 trim_cache size 0 max 16384 2024-09-16T00:36:02.702 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:36:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.695+0000 7fa323ecd640 1 -- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3080a3240 msgr2=0x7fa3080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3080a3240 0x7fa3080a5700 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55e00a50eac0 tx=0x7fa31800b4d0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 -- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102ef0 msgr2=0x7fa31c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102ef0 0x7fa31c0fd700 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa31400eaa0 tx=0x7fa31400ef60 comp rx=0 tx=0).stop 2024-09-16T00:36:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 -- 172.21.15.1:0/1037504516 shutdown_connections 2024-09-16T00:36:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c102ef0 0x7fa31c0fd700 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa31c102570 0x7fa31c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3080a3240 0x7fa3080a5700 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 --2- 172.21.15.1:0/1037504516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa31c100e20 0x7fa31c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 -- 172.21.15.1:0/1037504516 >> 172.21.15.1:0/1037504516 conn(0x7fa31c0fc7d0 msgr2=0x7fa31c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 -- 172.21.15.1:0/1037504516 shutdown_connections 2024-09-16T00:36:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:02.696+0000 7fa323ecd640 1 -- 172.21.15.1:0/1037504516 wait complete. 2024-09-16T00:36:03.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:36:03.712 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:04.150 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:04 smithi001 ceph-mon[30934]: pgmap v580: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:36:04.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.233+0000 7f2cce6a8640 1 -- 172.21.15.1:0/428173949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8101420 msgr2=0x7f2cc810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.233+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/428173949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8101420 0x7f2cc810d950 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009cd0 tx=0x7f2cc002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.233+0000 7f2cce6a8640 1 -- 172.21.15.1:0/428173949 shutdown_connections 2024-09-16T00:36:04.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.233+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/428173949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8101420 0x7f2cc810d950 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.233+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/428173949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8100a80 0x7f2cc8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.233+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/428173949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8106a80 0x7f2cc8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.233+0000 7f2cce6a8640 1 -- 172.21.15.1:0/428173949 >> 172.21.15.1:0/428173949 conn(0x7f2cc80fc7a0 msgr2=0x7f2cc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.233+0000 7f2cce6a8640 1 -- 172.21.15.1:0/428173949 shutdown_connections 2024-09-16T00:36:04.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.233+0000 7f2cce6a8640 1 -- 172.21.15.1:0/428173949 wait complete. 2024-09-16T00:36:04.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cce6a8640 1 Processor -- start 2024-09-16T00:36:04.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cce6a8640 1 -- start start 2024-09-16T00:36:04.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cce6a8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8100a80 0x7f2cc8197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cce6a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8101420 0x7f2cc81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cce6a8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8106a80 0x7f2cc8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cce6a8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8110e80 con 0x7f2cc8100a80 2024-09-16T00:36:04.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cce6a8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8110d00 con 0x7f2cc8101420 2024-09-16T00:36:04.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cce6a8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8111000 con 0x7f2cc8106a80 2024-09-16T00:36:04.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8100a80 0x7f2cc8197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8100a80 0x7f2cc8197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58696/0 (socket says 172.21.15.1:58696) 2024-09-16T00:36:04.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cc7fff640 1 -- 172.21.15.1:0/2386262824 learned_addr learned my addr 172.21.15.1:0/2386262824 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:04.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cccc45640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8106a80 0x7f2cc8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.234+0000 7f2cc77fe640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8101420 0x7f2cc81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cc7fff640 1 -- 172.21.15.1:0/2386262824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8106a80 msgr2=0x7f2cc8197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8106a80 0x7f2cc8197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cc7fff640 1 -- 172.21.15.1:0/2386262824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8101420 msgr2=0x7f2cc81975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8101420 0x7f2cc81975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cc7fff640 1 -- 172.21.15.1:0/2386262824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc81a68f0 con 0x7f2cc8100a80 2024-09-16T00:36:04.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8100a80 0x7f2cc8197080 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800d770 tx=0x7f2cb800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cc57fa640 1 -- 172.21.15.1:0/2386262824 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8017070 con 0x7f2cc8100a80 2024-09-16T00:36:04.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cc57fa640 1 -- 172.21.15.1:0/2386262824 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb8005500 con 0x7f2cc8100a80 2024-09-16T00:36:04.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cce6a8640 1 -- 172.21.15.1:0/2386262824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc81a6be0 con 0x7f2cc8100a80 2024-09-16T00:36:04.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cc57fa640 1 -- 172.21.15.1:0/2386262824 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8004e00 con 0x7f2cc8100a80 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.235+0000 7f2cce6a8640 1 -- 172.21.15.1:0/2386262824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc81a70d0 con 0x7f2cc8100a80 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.237+0000 7f2cc57fa640 1 -- 172.21.15.1:0/2386262824 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2cb8004fa0 con 0x7f2cc8100a80 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cd0a81740 1 Processor -- start 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cd0a81740 1 -- start start 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b4e35142f0 con 0x55b4e35afb40 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b4e34bbcf0 con 0x55b4e35ae5d0 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b4e35102b0 con 0x55b4e35b9370 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cccc45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cccc45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58708/0 (socket says 172.21.15.1:58708) 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cccc45640 1 -- 172.21.15.1:0/1850204683 learned_addr learned my addr 172.21.15.1:0/1850204683 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cc57fa640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb00a3240 0x7f2cb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cc77fe640 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cccc45640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb00a3240 0x7f2cb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cc57fa640 1 -- 172.21.15.1:0/2386262824 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2cb80cf6e0 con 0x7f2cc8100a80 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cccc45640 1 -- 172.21.15.1:0/1850204683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 msgr2=0x55b4e35bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.238+0000 7f2cccc45640 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.239+0000 7f2cccc45640 1 -- 172.21.15.1:0/1850204683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 msgr2=0x55b4e35aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.239+0000 7f2cccc45640 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.239+0000 7f2cccc45640 1 -- 172.21.15.1:0/1850204683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e34a03a0 con 0x55b4e35afb40 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.239+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.239+0000 7f2cccc45640 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35ae090 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0004ab0 tx=0x7f2cc00316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e52101c020491c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.239+0000 7f2cae7fc640 1 -- 172.21.15.1:0/1850204683 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0004490 con 0x55b4e35afb40 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.239+0000 7f2cae7fc640 1 -- 172.21.15.1:0/1850204683 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc004d070 con 0x55b4e35afb40 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.239+0000 7f2cae7fc640 1 -- 172.21.15.1:0/1850204683 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0030e40 con 0x55b4e35afb40 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.239+0000 7f2cccc45640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb00a3240 0x7f2cb00a5700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2cc004c000 tx=0x7f2cc004a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1850204683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 msgr2=0x55b4e35ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35ae090 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0004ab0 tx=0x7f2cc00316e0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1850204683 shutdown_connections 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/1850204683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35ae090 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1850204683 >> 172.21.15.1:0/1850204683 conn(0x55b4e3577310 msgr2=0x55b4e35aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1850204683 shutdown_connections 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1850204683 wait complete. 2024-09-16T00:36:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 -- start start 2024-09-16T00:36:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.240+0000 7f2cd0a81740 1 Processor -- start 2024-09-16T00:36:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b4e35bc070 con 0x55b4e35afb40 2024-09-16T00:36:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b4e34cf060 con 0x55b4e35ae5d0 2024-09-16T00:36:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b4e35bc1f0 con 0x55b4e35b9370 2024-09-16T00:36:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58724/0 (socket says 172.21.15.1:58724) 2024-09-16T00:36:04.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cc77fe640 1 -- 172.21.15.1:0/2343639924 learned_addr learned my addr 172.21.15.1:0/2343639924 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:04.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cd0a81740 10 client.0 _ll_register_callbacks cb 0x55b4e32228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:04.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cccc45640 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cc77fe640 1 -- 172.21.15.1:0/2343639924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 msgr2=0x55b4e35b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:04.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cc77fe640 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cc77fe640 1 -- 172.21.15.1:0/2343639924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 msgr2=0x55b4e35b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:04.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cc77fe640 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.241+0000 7f2cc77fe640 1 -- 172.21.15.1:0/2343639924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e32853c0 con 0x55b4e35afb40 2024-09-16T00:36:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cc77fe640 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35b5ff0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00beb0 tx=0x7f2cbc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cae80a536464fd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2ca77fe640 1 -- 172.21.15.1:0/2343639924 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc004230 con 0x55b4e35afb40 2024-09-16T00:36:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2ca77fe640 1 -- 172.21.15.1:0/2343639924 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2cbc0043d0 con 0x55b4e35afb40 2024-09-16T00:36:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cd0a81740 1 -- 172.21.15.1:0/2343639924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc004d070 con 0x55b4e35afb40 2024-09-16T00:36:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2ca77fe640 1 -- 172.21.15.1:0/2343639924 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cbc0106a0 con 0x55b4e35afb40 2024-09-16T00:36:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cd0a81740 10 client.17238 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cd0a81740 1 -- 172.21.15.1:0/2343639924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b4e35c7ba0 con 0x55b4e35afb40 2024-09-16T00:36:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2ca77fe640 1 -- 172.21.15.1:0/2343639924 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc004d070 con 0x55b4e35afb40 2024-09-16T00:36:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2ca77fe640 1 -- 172.21.15.1:0/2343639924 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2cbc019070 con 0x55b4e35afb40 2024-09-16T00:36:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cd0a81740 10 client.17238 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cd0a81740 10 client.17238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:36:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cd0a81740 20 client.17238 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b4e35c90b0 0x55b4e35cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cd0a81740 4 client.17238 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:04.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.242+0000 7f2cd0a81740 1 -- 172.21.15.1:0/2343639924 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4e3521860 con 0x55b4e35c90b0 2024-09-16T00:36:04.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.243+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b4e35c90b0 0x55b4e35cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.243+0000 7f2ca77fe640 10 client.17238 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:04.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.243+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b4e35c90b0 0x55b4e35cb550 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c36c531aebd9ee1a server_cookie=8ca6a1896cd66a10 in_seq=0 out_seq=0 2024-09-16T00:36:04.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2ca77fe640 1 -- 172.21.15.1:0/2343639924 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4e3521860 con 0x55b4e35c90b0 2024-09-16T00:36:04.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2ca77fe640 10 client.17238 handle_command_reply: tid=0 2024-09-16T00:36:04.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 client.17238 shutdown 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 -- 172.21.15.1:0/2343639924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 msgr2=0x55b4e35b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35b5ff0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00beb0 tx=0x7f2cbc012820 comp rx=0 tx=0).stop 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 -- 172.21.15.1:0/2343639924 shutdown_connections 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b4e35c90b0 0x55b4e35cb550 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2ca77fe640 0 client.17238 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35afb40 0x55b4e35b5ff0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/2343639924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 -- 172.21.15.1:0/2343639924 >> 172.21.15.1:0/2343639924 conn(0x55b4e3577310 msgr2=0x55b4e35b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 -- 172.21.15.1:0/2343639924 shutdown_connections 2024-09-16T00:36:04.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 1 -- 172.21.15.1:0/2343639924 wait complete. 2024-09-16T00:36:04.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.244+0000 7f2cd0a81740 20 client.17238 trim_cache size 0 max 16384 2024-09-16T00:36:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.257+0000 7f2cd0a81740 1 Processor -- start 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.257+0000 7f2cd0a81740 1 -- start start 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.257+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.257+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35afb40 0x55b4e35b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.257+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35b9370 0x55b4e35b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.257+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81b3900 con 0x55b4e35b9370 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.257+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81b11d0 con 0x55b4e35ae5d0 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.257+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81af020 con 0x55b4e35afb40 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cccc45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35b9370 0x55b4e35b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cccc45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35b9370 0x55b4e35b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58740/0 (socket says 172.21.15.1:58740) 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cccc45640 1 -- 172.21.15.1:0/1323595589 learned_addr learned my addr 172.21.15.1:0/1323595589 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cc77fe640 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35afb40 0x55b4e35b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cccc45640 1 -- 172.21.15.1:0/1323595589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35afb40 msgr2=0x55b4e35b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cccc45640 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35afb40 0x55b4e35b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cccc45640 1 -- 172.21.15.1:0/1323595589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 msgr2=0x55b4e35c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cccc45640 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cccc45640 1 -- 172.21.15.1:0/1323595589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e35aaed0 con 0x55b4e35b9370 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cccc45640 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35b9370 0x55b4e35b1660 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009cd0 tx=0x7f2cc0009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8d3efcfc1209e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cadffb640 1 -- 172.21.15.1:0/1323595589 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0031ca0 con 0x55b4e35b9370 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cadffb640 1 -- 172.21.15.1:0/1323595589 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc004d040 con 0x55b4e35b9370 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.258+0000 7f2cadffb640 1 -- 172.21.15.1:0/1323595589 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0048990 con 0x55b4e35b9370 2024-09-16T00:36:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1323595589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35b9370 msgr2=0x55b4e35b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35b9370 0x55b4e35b1660 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009cd0 tx=0x7f2cc0009990 comp rx=0 tx=0).stop 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1323595589 shutdown_connections 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35b9370 0x55b4e35b1660 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35afb40 0x55b4e35b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/1323595589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1323595589 >> 172.21.15.1:0/1323595589 conn(0x55b4e3577310 msgr2=0x55b4e35b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1323595589 shutdown_connections 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 -- 172.21.15.1:0/1323595589 wait complete. 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.259+0000 7f2cd0a81740 1 -- start start 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cd0a81740 1 Processor -- start 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35afb40 0x55b4e35bce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cd0a81740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81af690 con 0x55b4e35ae5d0 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81b0b00 con 0x55b4e35afb40 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cd0a81740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81afd60 con 0x55b4e35b9370 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cd0a81740 10 client.0 _ll_register_callbacks cb 0x7f2cbc00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58742/0 (socket says 172.21.15.1:58742) 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc7fff640 1 -- 172.21.15.1:0/318235484 learned_addr learned my addr 172.21.15.1:0/318235484 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc7fff640 1 -- 172.21.15.1:0/318235484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 msgr2=0x55b4e35bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc77fe640 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35afb40 0x55b4e35bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cccc45640 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35bd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc7fff640 1 -- 172.21.15.1:0/318235484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35afb40 msgr2=0x55b4e35bce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35afb40 0x55b4e35bce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc7fff640 1 -- 172.21.15.1:0/318235484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e35bdba0 con 0x55b4e35ae5d0 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2cc7fff640 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c1b00 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8097d60 tx=0x7f2cb80c66d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf0ecdf52dda40d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.260+0000 7f2ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2ca77fe640 1 -- 172.21.15.1:0/318235484 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8020040 con 0x55b4e35ae5d0 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2ca77fe640 1 -- 172.21.15.1:0/318235484 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2cb8005290 con 0x55b4e35ae5d0 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2cd0a81740 1 -- 172.21.15.1:0/318235484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc004d040 con 0x55b4e35ae5d0 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2ca77fe640 1 -- 172.21.15.1:0/318235484 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cb809c000 con 0x55b4e35ae5d0 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2cd0a81740 10 client.17244 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2cd0a81740 1 -- 172.21.15.1:0/318235484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b4e3633d30 con 0x55b4e35ae5d0 2024-09-16T00:36:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2ca77fe640 1 -- 172.21.15.1:0/318235484 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc004d040 con 0x55b4e35ae5d0 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2ca77fe640 1 -- 172.21.15.1:0/318235484 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2cb80cf3b0 con 0x55b4e35ae5d0 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2cd0a81740 10 client.17244 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2cd0a81740 10 client.17244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2cd0a81740 20 client.17244 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b4e35c90b0 0x55b4e3636460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2cd0a81740 4 client.17244 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.261+0000 7f2cd0a81740 1 -- 172.21.15.1:0/318235484 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4e351bd70 con 0x55b4e35c90b0 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cc77fe640 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b4e35c90b0 0x55b4e3636460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2ca77fe640 10 client.17244 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cc77fe640 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b4e35c90b0 0x55b4e3636460 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36c7930375ec0500 server_cookie=73db79d54ccb374d in_seq=0 out_seq=0 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2ca77fe640 1 -- 172.21.15.1:0/318235484 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4e351bd70 con 0x55b4e35c90b0 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2ca77fe640 10 client.17244 handle_command_reply: tid=0 2024-09-16T00:36:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 client.17244 shutdown 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 -- 172.21.15.1:0/318235484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35ae5d0 msgr2=0x55b4e35c1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c1b00 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8097d60 tx=0x7f2cb80c66d0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 -- 172.21.15.1:0/318235484 shutdown_connections 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4e35b9370 0x55b4e35bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4e35afb40 0x55b4e35bce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55b4e35c90b0 0x55b4e3636460 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 --2- 172.21.15.1:0/318235484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4e35ae5d0 0x55b4e35c1b00 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 -- 172.21.15.1:0/318235484 >> 172.21.15.1:0/318235484 conn(0x55b4e3577310 msgr2=0x55b4e35b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2ca77fe640 0 client.17244 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.262+0000 7f2cd0a81740 1 -- 172.21.15.1:0/318235484 shutdown_connections 2024-09-16T00:36:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cd0a81740 1 -- 172.21.15.1:0/318235484 wait complete. 2024-09-16T00:36:04.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cd0a81740 20 client.17244 trim_cache size 0 max 16384 2024-09-16T00:36:04.274 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:36:04.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 -- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb00a3240 msgr2=0x7f2cb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb00a3240 0x7f2cb00a5700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2cc004c000 tx=0x7f2cc004a000 comp rx=0 tx=0).stop 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 -- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8100a80 msgr2=0x7f2cc8197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8100a80 0x7f2cc8197080 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800d770 tx=0x7f2cb800dc30 comp rx=0 tx=0).stop 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 -- 172.21.15.1:0/2386262824 shutdown_connections 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2cb00a3240 0x7f2cb00a5700 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8106a80 0x7f2cc8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8101420 0x7f2cc81975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 --2- 172.21.15.1:0/2386262824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8100a80 0x7f2cc8197080 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 -- 172.21.15.1:0/2386262824 >> 172.21.15.1:0/2386262824 conn(0x7f2cc80fc7a0 msgr2=0x7f2cc80fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 -- 172.21.15.1:0/2386262824 shutdown_connections 2024-09-16T00:36:04.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:04.263+0000 7f2cce6a8640 1 -- 172.21.15.1:0/2386262824 wait complete. 2024-09-16T00:36:04.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:04 smithi019 ceph-mon[35186]: pgmap v580: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:36:04.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:04 smithi153 ceph-mon[35538]: pgmap v580: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:36:05.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:36:05.330 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.851+0000 7fc62861b640 1 -- 172.21.15.1:0/1985408956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 msgr2=0x7fc6201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.851+0000 7fc62861b640 1 --2- 172.21.15.1:0/1985408956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 0x7fc6201029b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc610009900 tx=0x7fc61002a480 comp rx=0 tx=0).stop 2024-09-16T00:36:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.851+0000 7fc62861b640 1 -- 172.21.15.1:0/1985408956 shutdown_connections 2024-09-16T00:36:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.851+0000 7fc62861b640 1 --2- 172.21.15.1:0/1985408956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc620102ef0 0x7fc62010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.851+0000 7fc62861b640 1 --2- 172.21.15.1:0/1985408956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 0x7fc6201029b0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.851+0000 7fc62861b640 1 --2- 172.21.15.1:0/1985408956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc620100e20 0x7fc620101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.851+0000 7fc62861b640 1 -- 172.21.15.1:0/1985408956 >> 172.21.15.1:0/1985408956 conn(0x7fc6200fc7d0 msgr2=0x7fc6200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.851+0000 7fc62861b640 1 -- 172.21.15.1:0/1985408956 shutdown_connections 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.851+0000 7fc62861b640 1 -- 172.21.15.1:0/1985408956 wait complete. 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc62861b640 1 Processor -- start 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc62861b640 1 -- start start 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc62861b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc620100e20 0x7fc620199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc62861b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 0x7fc6201998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc62861b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc620102ef0 0x7fc6201a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc62861b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc620110f80 con 0x7fc620102570 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc62861b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc620110e00 con 0x7fc620100e20 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc62861b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc620111100 con 0x7fc620102ef0 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc625bb6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 0x7fc6201998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc625bb6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 0x7fc6201998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58756/0 (socket says 172.21.15.1:58756) 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc625bb6640 1 -- 172.21.15.1:0/1332446142 learned_addr learned my addr 172.21.15.1:0/1332446142 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.852+0000 7fc6263b7640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc620100e20 0x7fc620199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc626bb8640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc620102ef0 0x7fc6201a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc625bb6640 1 -- 172.21.15.1:0/1332446142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc620102ef0 msgr2=0x7fc6201a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc625bb6640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc620102ef0 0x7fc6201a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc625bb6640 1 -- 172.21.15.1:0/1332446142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc620100e20 msgr2=0x7fc620199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc625bb6640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc620100e20 0x7fc620199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc625bb6640 1 -- 172.21.15.1:0/1332446142 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6201a4920 con 0x7fc620102570 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc626bb8640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc620102ef0 0x7fc6201a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc625bb6640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 0x7fc6201998a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc610006010 tx=0x7fc610003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc6177fe640 1 -- 172.21.15.1:0/1332446142 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61003f070 con 0x7fc620102570 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc6177fe640 1 -- 172.21.15.1:0/1332446142 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc610031c30 con 0x7fc620102570 2024-09-16T00:36:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc6177fe640 1 -- 172.21.15.1:0/1332446142 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610035030 con 0x7fc620102570 2024-09-16T00:36:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc62861b640 1 -- 172.21.15.1:0/1332446142 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201a4af0 con 0x7fc620102570 2024-09-16T00:36:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.853+0000 7fc62861b640 1 -- 172.21.15.1:0/1332446142 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201a4fa0 con 0x7fc620102570 2024-09-16T00:36:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc6177fe640 1 -- 172.21.15.1:0/1332446142 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc610042070 con 0x7fc620102570 2024-09-16T00:36:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc62a9f4740 1 Processor -- start 2024-09-16T00:36:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc62a9f4740 1 -- start start 2024-09-16T00:36:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3058090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce30585d0 0x564ce3058a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce3063370 0x564ce3065760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ce2fbe2f0 con 0x564ce3059b40 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ce2f65cf0 con 0x564ce3063370 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ce2fba2b0 con 0x564ce30585d0 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc626bb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3058090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc626bb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3058090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58772/0 (socket says 172.21.15.1:58772) 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc626bb8640 1 -- 172.21.15.1:0/2090858159 learned_addr learned my addr 172.21.15.1:0/2090858159 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.856+0000 7fc625bb6640 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce3063370 0x564ce3065760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc6263b7640 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce30585d0 0x564ce3058a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc6177fe640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc60c0a32c0 0x7fc60c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc626bb8640 1 -- 172.21.15.1:0/2090858159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce30585d0 msgr2=0x564ce3058a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc626bb8640 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce30585d0 0x564ce3058a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc626bb8640 1 -- 172.21.15.1:0/2090858159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce3063370 msgr2=0x564ce3065760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc626bb8640 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce3063370 0x564ce3065760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc626bb8640 1 -- 172.21.15.1:0/2090858159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce2f4a3a0 con 0x564ce3059b40 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc6263b7640 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce30585d0 0x564ce3058a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc626bb8640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc60c0a32c0 0x7fc60c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc6177fe640 1 -- 172.21.15.1:0/1332446142 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc6100f04f0 con 0x7fc620102570 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc626bb8640 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3058090 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc61800a2b0 tx=0x7fc61800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd4e2806572f286 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc614ff9640 1 -- 172.21.15.1:0/2090858159 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618022070 con 0x564ce3059b40 2024-09-16T00:36:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc614ff9640 1 -- 172.21.15.1:0/2090858159 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc618018070 con 0x564ce3059b40 2024-09-16T00:36:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.857+0000 7fc614ff9640 1 -- 172.21.15.1:0/2090858159 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61801cd30 con 0x564ce3059b40 2024-09-16T00:36:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc62a9f4740 1 -- 172.21.15.1:0/2090858159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 msgr2=0x564ce3058090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc626bb8640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc60c0a32c0 0x7fc60c0a5780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc61801f000 tx=0x7fc618020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3058090 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc61800a2b0 tx=0x7fc61800e7e0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc62a9f4740 1 -- 172.21.15.1:0/2090858159 shutdown_connections 2024-09-16T00:36:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce3063370 0x564ce3065760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce30585d0 0x564ce3058a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/2090858159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3058090 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc62a9f4740 1 -- 172.21.15.1:0/2090858159 >> 172.21.15.1:0/2090858159 conn(0x564ce3021310 msgr2=0x564ce3054c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc62a9f4740 1 -- 172.21.15.1:0/2090858159 shutdown_connections 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.858+0000 7fc62a9f4740 1 -- 172.21.15.1:0/2090858159 wait complete. 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc62a9f4740 1 -- start start 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc62a9f4740 1 Processor -- start 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 0x564ce3062d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 0x564ce3060030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3063370 0x564ce3060640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ce3066070 con 0x564ce3063370 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ce2f79060 con 0x564ce30585d0 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ce30661f0 con 0x564ce3059b40 2024-09-16T00:36:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc62a9f4740 10 client.0 _ll_register_callbacks cb 0x564ce2ccc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:05.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc626bb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3063370 0x564ce3060640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc626bb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3063370 0x564ce3060640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58778/0 (socket says 172.21.15.1:58778) 2024-09-16T00:36:05.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc626bb8640 1 -- 172.21.15.1:0/728546276 learned_addr learned my addr 172.21.15.1:0/728546276 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:05.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc6263b7640 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 0x564ce3062d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc625bb6640 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 0x564ce3060030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc626bb8640 1 -- 172.21.15.1:0/728546276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 msgr2=0x564ce3060030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc626bb8640 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 0x564ce3060030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc626bb8640 1 -- 172.21.15.1:0/728546276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 msgr2=0x564ce3062d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc626bb8640 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 0x564ce3062d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc626bb8640 1 -- 172.21.15.1:0/728546276 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce2d2f3c0 con 0x564ce3063370 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.859+0000 7fc625bb6640 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 0x564ce3060030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc626bb8640 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3063370 0x564ce3060640 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc61800f010 tx=0x7fc618011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1ae9e1b177524b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc609ffb640 1 -- 172.21.15.1:0/728546276 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618022050 con 0x564ce3063370 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc609ffb640 1 -- 172.21.15.1:0/728546276 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc61801c640 con 0x564ce3063370 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc62a9f4740 1 -- 172.21.15.1:0/728546276 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc618018070 con 0x564ce3063370 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc609ffb640 1 -- 172.21.15.1:0/728546276 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc61803c050 con 0x564ce3063370 2024-09-16T00:36:05.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc62a9f4740 10 client.17256 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc62a9f4740 1 -- 172.21.15.1:0/728546276 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ce3071b70 con 0x564ce3063370 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc609ffb640 1 -- 172.21.15.1:0/728546276 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc618018070 con 0x564ce3063370 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.860+0000 7fc609ffb640 1 -- 172.21.15.1:0/728546276 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc618037030 con 0x564ce3063370 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.861+0000 7fc62a9f4740 10 client.17256 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.861+0000 7fc62a9f4740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.861+0000 7fc62a9f4740 20 client.17256 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.861+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564ce3072fd0 0x564ce30753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.861+0000 7fc62a9f4740 4 client.17256 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.861+0000 7fc62a9f4740 1 -- 172.21.15.1:0/728546276 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ce2fcb860 con 0x564ce3072fd0 2024-09-16T00:36:05.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.861+0000 7fc6263b7640 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564ce3072fd0 0x564ce30753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.861+0000 7fc6263b7640 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564ce3072fd0 0x564ce30753c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6395bd476925b9ea server_cookie=1a0072d43ee44b40 in_seq=0 out_seq=0 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.861+0000 7fc609ffb640 10 client.17256 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc609ffb640 1 -- 172.21.15.1:0/728546276 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ce2fcb860 con 0x564ce3072fd0 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc609ffb640 10 client.17256 handle_command_reply: tid=0 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 client.17256 shutdown 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 -- 172.21.15.1:0/728546276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3063370 msgr2=0x564ce3060640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3063370 0x564ce3060640 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc61800f010 tx=0x7fc618011df0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 -- 172.21.15.1:0/728546276 shutdown_connections 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564ce3072fd0 0x564ce30753c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3063370 0x564ce3060640 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 0x564ce3060030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/728546276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 0x564ce3062d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 -- 172.21.15.1:0/728546276 >> 172.21.15.1:0/728546276 conn(0x564ce3021310 msgr2=0x564ce305a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:05.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc609ffb640 0 client.17256 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:05.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 -- 172.21.15.1:0/728546276 shutdown_connections 2024-09-16T00:36:05.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 1 -- 172.21.15.1:0/728546276 wait complete. 2024-09-16T00:36:05.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.862+0000 7fc62a9f4740 20 client.17256 trim_cache size 0 max 16384 2024-09-16T00:36:05.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc62a9f4740 1 Processor -- start 2024-09-16T00:36:05.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc62a9f4740 1 -- start start 2024-09-16T00:36:05.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce30585d0 0x564ce306f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 0x564ce307aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce3063370 0x564ce307af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201b16d0 con 0x564ce30585d0 2024-09-16T00:36:05.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201aefa0 con 0x564ce3063370 2024-09-16T00:36:05.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201acdf0 con 0x564ce3059b40 2024-09-16T00:36:05.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc6263b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce30585d0 0x564ce306f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc6263b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce30585d0 0x564ce306f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58780/0 (socket says 172.21.15.1:58780) 2024-09-16T00:36:05.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc6263b7640 1 -- 172.21.15.1:0/69786981 learned_addr learned my addr 172.21.15.1:0/69786981 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:05.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc6263b7640 1 -- 172.21.15.1:0/69786981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 msgr2=0x564ce307aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:05.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc6263b7640 1 --2- 172.21.15.1:0/69786981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 0x564ce307aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc6263b7640 1 -- 172.21.15.1:0/69786981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce3063370 msgr2=0x564ce307af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:05.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc6263b7640 1 --2- 172.21.15.1:0/69786981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce3063370 0x564ce307af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.876+0000 7fc6263b7640 1 -- 172.21.15.1:0/69786981 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce30630f0 con 0x564ce30585d0 2024-09-16T00:36:05.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc6263b7640 1 --2- 172.21.15.1:0/69786981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce30585d0 0x564ce306f340 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00b620 tx=0x7fc61c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb38f0c8c1c6986 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:05.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc60bfff640 1 -- 172.21.15.1:0/69786981 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c003d20 con 0x564ce30585d0 2024-09-16T00:36:05.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc60bfff640 1 -- 172.21.15.1:0/69786981 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c005500 con 0x564ce30585d0 2024-09-16T00:36:05.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc60bfff640 1 -- 172.21.15.1:0/69786981 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0116c0 con 0x564ce30585d0 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc62a9f4740 1 -- 172.21.15.1:0/69786981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce30585d0 msgr2=0x564ce306f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/69786981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce30585d0 0x564ce306f340 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00b620 tx=0x7fc61c005f00 comp rx=0 tx=0).stop 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc62a9f4740 1 -- 172.21.15.1:0/69786981 shutdown_connections 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/69786981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce3063370 0x564ce307af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/69786981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3059b40 0x564ce307aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/69786981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce30585d0 0x564ce306f340 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.877+0000 7fc62a9f4740 1 -- 172.21.15.1:0/69786981 >> 172.21.15.1:0/69786981 conn(0x564ce3021310 msgr2=0x564ce305f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 -- 172.21.15.1:0/69786981 shutdown_connections 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 -- 172.21.15.1:0/69786981 wait complete. 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 -- start start 2024-09-16T00:36:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 Processor -- start 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 0x564ce306ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3066d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3063370 0x564ce3067280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ad460 con 0x564ce3059b40 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ae8d0 con 0x564ce30585d0 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201adb30 con 0x564ce3063370 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc62a9f4740 10 client.0 _ll_register_callbacks cb 0x7fc618020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc625bb6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3066d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc625bb6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3066d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58796/0 (socket says 172.21.15.1:58796) 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc625bb6640 1 -- 172.21.15.1:0/1567430562 learned_addr learned my addr 172.21.15.1:0/1567430562 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.878+0000 7fc6263b7640 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 0x564ce306ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc626bb8640 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3063370 0x564ce3067280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc625bb6640 1 -- 172.21.15.1:0/1567430562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3063370 msgr2=0x564ce3067280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc625bb6640 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3063370 0x564ce3067280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc625bb6640 1 -- 172.21.15.1:0/1567430562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 msgr2=0x564ce306ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc625bb6640 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 0x564ce306ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc625bb6640 1 -- 172.21.15.1:0/1567430562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce30678c0 con 0x564ce3059b40 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc625bb6640 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3066d40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc6100bc6a0 tx=0x7fc61002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaac7235144de52c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc609ffb640 1 -- 172.21.15.1:0/1567430562 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610049040 con 0x564ce3059b40 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc609ffb640 1 -- 172.21.15.1:0/1567430562 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc6100bc970 con 0x564ce3059b40 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc62a9f4740 1 -- 172.21.15.1:0/1567430562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61c005500 con 0x564ce3059b40 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc609ffb640 1 -- 172.21.15.1:0/1567430562 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc6100bc250 con 0x564ce3059b40 2024-09-16T00:36:05.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc62a9f4740 10 client.17262 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc62a9f4740 1 -- 172.21.15.1:0/1567430562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ce30dd800 con 0x564ce3059b40 2024-09-16T00:36:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.879+0000 7fc609ffb640 1 -- 172.21.15.1:0/1567430562 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c005500 con 0x564ce3059b40 2024-09-16T00:36:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc609ffb640 1 -- 172.21.15.1:0/1567430562 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc6100c4ef0 con 0x564ce3059b40 2024-09-16T00:36:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc62a9f4740 10 client.17262 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc62a9f4740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc62a9f4740 20 client.17262 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564ce30ddf30 0x564ce30e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc62a9f4740 4 client.17262 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:36:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc62a9f4740 1 -- 172.21.15.1:0/1567430562 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ce2fc5d70 con 0x564ce30ddf30 2024-09-16T00:36:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc6263b7640 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564ce30ddf30 0x564ce30e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc6263b7640 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564ce30ddf30 0x564ce30e0350 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f68f0ff87a96e58 server_cookie=a17118688a313a7f in_seq=0 out_seq=0 2024-09-16T00:36:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc609ffb640 10 client.17262 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc609ffb640 1 -- 172.21.15.1:0/1567430562 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ce2fc5d70 con 0x564ce30ddf30 2024-09-16T00:36:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc609ffb640 10 client.17262 handle_command_reply: tid=0 2024-09-16T00:36:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.880+0000 7fc62a9f4740 1 client.17262 shutdown 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 -- 172.21.15.1:0/1567430562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 msgr2=0x564ce3066d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3066d40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc6100bc6a0 tx=0x7fc61002a480 comp rx=0 tx=0).stop 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 -- 172.21.15.1:0/1567430562 shutdown_connections 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564ce30ddf30 0x564ce30e0350 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce3063370 0x564ce3067280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce3059b40 0x564ce3066d40 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 --2- 172.21.15.1:0/1567430562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ce30585d0 0x564ce306ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 -- 172.21.15.1:0/1567430562 >> 172.21.15.1:0/1567430562 conn(0x564ce3021310 msgr2=0x564ce3061890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc609ffb640 0 client.17262 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 -- 172.21.15.1:0/1567430562 shutdown_connections 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 1 -- 172.21.15.1:0/1567430562 wait complete. 2024-09-16T00:36:05.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62a9f4740 20 client.17262 trim_cache size 0 max 16384 2024-09-16T00:36:05.895 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:36:05.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62861b640 1 -- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc60c0a32c0 msgr2=0x7fc60c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:05.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62861b640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc60c0a32c0 0x7fc60c0a5780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc61801f000 tx=0x7fc618020000 comp rx=0 tx=0).stop 2024-09-16T00:36:05.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62861b640 1 -- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 msgr2=0x7fc6201998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:05.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.881+0000 7fc62861b640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 0x7fc6201998a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc610006010 tx=0x7fc610003c10 comp rx=0 tx=0).stop 2024-09-16T00:36:05.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.882+0000 7fc62861b640 1 -- 172.21.15.1:0/1332446142 shutdown_connections 2024-09-16T00:36:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.882+0000 7fc62861b640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc60c0a32c0 0x7fc60c0a5780 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.882+0000 7fc62861b640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc620102ef0 0x7fc6201a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.882+0000 7fc62861b640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc620102570 0x7fc6201998a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.882+0000 7fc62861b640 1 --2- 172.21.15.1:0/1332446142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc620100e20 0x7fc620199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.882+0000 7fc62861b640 1 -- 172.21.15.1:0/1332446142 >> 172.21.15.1:0/1332446142 conn(0x7fc6200fc7d0 msgr2=0x7fc6200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.882+0000 7fc62861b640 1 -- 172.21.15.1:0/1332446142 shutdown_connections 2024-09-16T00:36:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:05.882+0000 7fc62861b640 1 -- 172.21.15.1:0/1332446142 wait complete. 2024-09-16T00:36:06.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:06 smithi001 ceph-mon[30934]: pgmap v581: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:36:06.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:06 smithi019 ceph-mon[35186]: pgmap v581: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:36:06.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:06 smithi153 ceph-mon[35538]: pgmap v581: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:36:07.671 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:36:07.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:36:07.943 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:08.264 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:08 smithi153 ceph-mon[35538]: pgmap v582: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:08.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:08 smithi019 ceph-mon[35186]: pgmap v582: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:08.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:08 smithi001 ceph-mon[30934]: pgmap v582: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.465+0000 7f5ba2861640 1 -- 172.21.15.1:0/456494914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c102d30 msgr2=0x7f5b9c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.465+0000 7f5ba2861640 1 --2- 172.21.15.1:0/456494914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c102d30 0x7f5b9c103110 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009930 tx=0x7f5b8c02a420 comp rx=0 tx=0).stop 2024-09-16T00:36:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.465+0000 7f5ba2861640 1 -- 172.21.15.1:0/456494914 shutdown_connections 2024-09-16T00:36:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.465+0000 7f5ba2861640 1 --2- 172.21.15.1:0/456494914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9c1082a0 0x7f5b9c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.465+0000 7f5ba2861640 1 --2- 172.21.15.1:0/456494914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c1036e0 0x7f5b9c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.465+0000 7f5ba2861640 1 --2- 172.21.15.1:0/456494914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c102d30 0x7f5b9c103110 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.465+0000 7f5ba2861640 1 -- 172.21.15.1:0/456494914 >> 172.21.15.1:0/456494914 conn(0x7f5b9c0fc7d0 msgr2=0x7f5b9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.465+0000 7f5ba2861640 1 -- 172.21.15.1:0/456494914 shutdown_connections 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.465+0000 7f5ba2861640 1 -- 172.21.15.1:0/456494914 wait complete. 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba2861640 1 Processor -- start 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba2861640 1 -- start start 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba2861640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c102d30 0x7f5b9c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba2861640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9c1036e0 0x7f5b9c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba2861640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c1082a0 0x7f5b9c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba2861640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1120a0 con 0x7f5b9c1082a0 2024-09-16T00:36:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba2861640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c111f20 con 0x7f5b9c1036e0 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba2861640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c112220 con 0x7f5b9c102d30 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba0dfe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c1082a0 0x7f5b9c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba0dfe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c1082a0 0x7f5b9c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58808/0 (socket says 172.21.15.1:58808) 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba0dfe640 1 -- 172.21.15.1:0/2184658232 learned_addr learned my addr 172.21.15.1:0/2184658232 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5b93fff640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9c1036e0 0x7f5b9c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c102d30 0x7f5b9c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba0dfe640 1 -- 172.21.15.1:0/2184658232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c102d30 msgr2=0x7f5b9c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba0dfe640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c102d30 0x7f5b9c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba0dfe640 1 -- 172.21.15.1:0/2184658232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9c1036e0 msgr2=0x7f5b9c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba0dfe640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9c1036e0 0x7f5b9c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.466+0000 7f5ba0dfe640 1 -- 172.21.15.1:0/2184658232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c19d570 con 0x7f5b9c1082a0 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.467+0000 7f5ba0dfe640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c1082a0 0x7f5b9c19ce90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400eaa0 tx=0x7f5b9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.467+0000 7f5b99ffb640 1 -- 172.21.15.1:0/2184658232 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400cdb0 con 0x7f5b9c1082a0 2024-09-16T00:36:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.467+0000 7f5b99ffb640 1 -- 172.21.15.1:0/2184658232 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b94005020 con 0x7f5b9c1082a0 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.467+0000 7f5b99ffb640 1 -- 172.21.15.1:0/2184658232 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b940116b0 con 0x7f5b9c1082a0 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.467+0000 7f5ba2861640 1 -- 172.21.15.1:0/2184658232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c19d860 con 0x7f5b9c1082a0 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.467+0000 7f5ba2861640 1 -- 172.21.15.1:0/2184658232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1a5140 con 0x7f5b9c1082a0 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.469+0000 7f5b99ffb640 1 -- 172.21.15.1:0/2184658232 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f5b94005560 con 0x7f5b9c1082a0 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5ba4c3a740 1 Processor -- start 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5ba4c3a740 1 -- start start 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f89f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 0x5586c1f94800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f94d40 0x5586c1f97130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5586c1eef280 con 0x5586c1f89b60 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5586c1e96b10 con 0x5586c1f8a480 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5586c1eeb240 con 0x5586c1f94d40 2024-09-16T00:36:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5b93fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f89f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5b99ffb640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5b880a32c0 0x7f5b880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5b93fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f89f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58818/0 (socket says 172.21.15.1:58818) 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5b93fff640 1 -- 172.21.15.1:0/3857719490 learned_addr learned my addr 172.21.15.1:0/3857719490 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.470+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/3857719490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 0x5586c1f94800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5ba0dfe640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5b880a32c0 0x7f5b880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b93fff640 1 -- 172.21.15.1:0/3857719490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f94d40 msgr2=0x5586c1f97130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b93fff640 1 --2- 172.21.15.1:0/3857719490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f94d40 0x5586c1f97130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b93fff640 1 -- 172.21.15.1:0/3857719490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 msgr2=0x5586c1f94800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b93fff640 1 --2- 172.21.15.1:0/3857719490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 0x5586c1f94800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b93fff640 1 -- 172.21.15.1:0/3857719490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c1ea31f0 con 0x5586c1f89b60 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/3857719490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 0x5586c1f94800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b99ffb640 1 -- 172.21.15.1:0/2184658232 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5b940cfe40 con 0x7f5b9c1082a0 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b93fff640 1 --2- 172.21.15.1:0/3857719490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f89f40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5b840058a0 tx=0x7f5b840127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fccb672f289f883 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:08.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5ba0dfe640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5b880a32c0 0x7f5b880a5780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b9409c040 tx=0x7f5b940bc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b927fc640 1 -- 172.21.15.1:0/3857719490 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b840120d0 con 0x5586c1f89b60 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b927fc640 1 -- 172.21.15.1:0/3857719490 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b84016070 con 0x5586c1f89b60 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.471+0000 7f5b927fc640 1 -- 172.21.15.1:0/3857719490 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84004a00 con 0x5586c1f89b60 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3857719490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 msgr2=0x5586c1f89f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3857719490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f89f40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5b840058a0 tx=0x7f5b840127c0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3857719490 shutdown_connections 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3857719490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f94d40 0x5586c1f97130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3857719490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 0x5586c1f94800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3857719490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f89f40 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3857719490 >> 172.21.15.1:0/3857719490 conn(0x5586c1f860f0 msgr2=0x5586c1f864d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3857719490 shutdown_connections 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3857719490 wait complete. 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.472+0000 7f5ba4c3a740 1 -- start start 2024-09-16T00:36:08.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5ba4c3a740 1 Processor -- start 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 0x5586c1f91c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f92170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f94d40 0x5586c1f926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5586c1f97910 con 0x5586c1f8a480 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5586c1ea9f10 con 0x5586c1f94d40 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5586c1f97a90 con 0x5586c1f89b60 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5ba4c3a740 10 client.0 _ll_register_callbacks cb 0x5586c1e72a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f92170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f92170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58828/0 (socket says 172.21.15.1:58828) 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b9bfff640 1 -- 172.21.15.1:0/4011178882 learned_addr learned my addr 172.21.15.1:0/4011178882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b93fff640 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 0x5586c1f91c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b9bfff640 1 -- 172.21.15.1:0/4011178882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 msgr2=0x5586c1f91c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 0x5586c1f91c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b9bfff640 1 -- 172.21.15.1:0/4011178882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f94d40 msgr2=0x5586c1f926b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f94d40 0x5586c1f926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b9bfff640 1 -- 172.21.15.1:0/4011178882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c1c606a0 con 0x5586c1f8a480 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f92170 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009930 tx=0x7f5b8c031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b9fc30bfd20a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.473+0000 7f5b7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/4011178882 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c035040 con 0x5586c1f8a480 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/4011178882 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5b8c004460 con 0x5586c1f8a480 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/4011178882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b84016070 con 0x5586c1f8a480 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/4011178882 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b8c005020 con 0x5586c1f8a480 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5ba4c3a740 10 client.17274 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/4011178882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586c1fa3520 con 0x5586c1f8a480 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/4011178882 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b84016070 con 0x5586c1f8a480 2024-09-16T00:36:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/4011178882 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5b8c0035f0 con 0x5586c1f8a480 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5ba4c3a740 10 client.17274 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5ba4c3a740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5ba4c3a740 20 client.17274 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5586c1fa49e0 0x5586c1fa6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5ba4c3a740 4 client.17274 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.474+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/4011178882 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586c1efc7f0 con 0x5586c1fa49e0 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.475+0000 7f5b93fff640 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5586c1fa49e0 0x5586c1fa6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.475+0000 7f5b7b7fe640 10 client.17274 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.475+0000 7f5b93fff640 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5586c1fa49e0 0x5586c1fa6e20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b3ec5943f57ad4 server_cookie=b4d295f807a0085b in_seq=0 out_seq=0 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/4011178882 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586c1efc7f0 con 0x5586c1fa49e0 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5b7b7fe640 10 client.17274 handle_command_reply: tid=0 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 client.17274 shutdown 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/4011178882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 msgr2=0x5586c1f92170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f92170 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009930 tx=0x7f5b8c031860 comp rx=0 tx=0).stop 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/4011178882 shutdown_connections 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5586c1fa49e0 0x5586c1fa6e20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f94d40 0x5586c1f926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f92170 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/4011178882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 0x5586c1f91c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5b7b7fe640 0 client.17274 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/4011178882 >> 172.21.15.1:0/4011178882 conn(0x5586c1f860f0 msgr2=0x5586c1f8be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/4011178882 shutdown_connections 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/4011178882 wait complete. 2024-09-16T00:36:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.476+0000 7f5ba4c3a740 20 client.17274 trim_cache size 0 max 16384 2024-09-16T00:36:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5ba4c3a740 1 Processor -- start 2024-09-16T00:36:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5ba4c3a740 1 -- start start 2024-09-16T00:36:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f92500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 0x5586c1fac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f94d40 0x5586c1facc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1b1730 con 0x5586c1f89b60 2024-09-16T00:36:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1af000 con 0x5586c1f8a480 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1ace50 con 0x5586c1f94d40 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5b93fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f92500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5b93fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f92500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58836/0 (socket says 172.21.15.1:58836) 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5b93fff640 1 -- 172.21.15.1:0/2585279595 learned_addr learned my addr 172.21.15.1:0/2585279595 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.494+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/2585279595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 0x5586c1fac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5b93fff640 1 -- 172.21.15.1:0/2585279595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f94d40 msgr2=0x5586c1facc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5b93fff640 1 --2- 172.21.15.1:0/2585279595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f94d40 0x5586c1facc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5b93fff640 1 -- 172.21.15.1:0/2585279595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 msgr2=0x5586c1fac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5b93fff640 1 --2- 172.21.15.1:0/2585279595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 0x5586c1fac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5b93fff640 1 -- 172.21.15.1:0/2585279595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c1fa74a0 con 0x5586c1f89b60 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5b93fff640 1 --2- 172.21.15.1:0/2585279595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f92500 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f5b84005510 tx=0x7f5b84005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4f69acfe09d8b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5b91ffb640 1 -- 172.21.15.1:0/2585279595 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8400b970 con 0x5586c1f89b60 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5b91ffb640 1 -- 172.21.15.1:0/2585279595 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b84016040 con 0x5586c1f89b60 2024-09-16T00:36:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5b91ffb640 1 -- 172.21.15.1:0/2585279595 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84010790 con 0x5586c1f89b60 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/2585279595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 msgr2=0x5586c1f92500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.495+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/2585279595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f92500 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f5b84005510 tx=0x7f5b84005c70 comp rx=0 tx=0).stop 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/2585279595 shutdown_connections 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/2585279595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f94d40 0x5586c1facc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/2585279595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f8a480 0x5586c1fac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/2585279595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f89b60 0x5586c1f92500 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/2585279595 >> 172.21.15.1:0/2585279595 conn(0x5586c1f860f0 msgr2=0x5586c1f93ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/2585279595 shutdown_connections 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/2585279595 wait complete. 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 -- start start 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 Processor -- start 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 0x5586c1f9cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f9d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f94d40 0x5586c1f99cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1ad4c0 con 0x5586c1f8a480 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1ae930 con 0x5586c1f94d40 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1adb90 con 0x5586c1f89b60 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.496+0000 7f5ba4c3a740 10 client.0 _ll_register_callbacks cb 0x7f5b8c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f9d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f9d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58852/0 (socket says 172.21.15.1:58852) 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b9bfff640 1 -- 172.21.15.1:0/3370686613 learned_addr learned my addr 172.21.15.1:0/3370686613 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b93fff640 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f94d40 0x5586c1f99cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b93fff640 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 0x5586c1f9cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b9bfff640 1 -- 172.21.15.1:0/3370686613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 msgr2=0x5586c1f9cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 0x5586c1f9cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b9bfff640 1 -- 172.21.15.1:0/3370686613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f94d40 msgr2=0x5586c1f99cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f94d40 0x5586c1f99cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b9bfff640 1 -- 172.21.15.1:0/3370686613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c1f9a350 con 0x5586c1f8a480 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b93fff640 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 0x5586c1f9cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b9bfff640 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f9d400 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c003a40 tx=0x7f5b8c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f515ed1f10ca98b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/3370686613 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0051c0 con 0x5586c1f8a480 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/3370686613 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5b8c005360 con 0x5586c1f8a480 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3370686613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b84016040 con 0x5586c1f8a480 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.497+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/3370686613 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b8c062050 con 0x5586c1f8a480 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5ba4c3a740 10 client.17280 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3370686613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586c2010090 con 0x5586c1f8a480 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/3370686613 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b84016040 con 0x5586c1f8a480 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/3370686613 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5b8c03c440 con 0x5586c1f8a480 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5ba4c3a740 10 client.17280 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:08.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5ba4c3a740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5ba4c3a740 20 client.17280 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5586c20107b0 0x5586c2012bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5ba4c3a740 4 client.17280 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3370686613 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5586c1ef6d00 con 0x5586c20107b0 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.498+0000 7f5b93fff640 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5586c20107b0 0x5586c2012bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.499+0000 7f5b93fff640 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5586c20107b0 0x5586c2012bd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f787be3b83993fbd server_cookie=c30bdc53f32fc331 in_seq=0 out_seq=0 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.499+0000 7f5b7b7fe640 10 client.17280 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5b7b7fe640 1 -- 172.21.15.1:0/3370686613 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5586c1ef6d00 con 0x5586c20107b0 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5b7b7fe640 10 client.17280 handle_command_reply: tid=0 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 client.17280 shutdown 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3370686613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 msgr2=0x5586c1f9d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f9d400 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c003a40 tx=0x7f5b8c005fd0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3370686613 shutdown_connections 2024-09-16T00:36:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5586c20107b0 0x5586c2012bd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586c1f94d40 0x5586c1f99cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586c1f8a480 0x5586c1f9d400 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5b7b7fe640 0 client.17280 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 --2- 172.21.15.1:0/3370686613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586c1f89b60 0x5586c1f9cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3370686613 >> 172.21.15.1:0/3370686613 conn(0x5586c1f860f0 msgr2=0x5586c1f84a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3370686613 shutdown_connections 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 1 -- 172.21.15.1:0/3370686613 wait complete. 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.500+0000 7f5ba4c3a740 20 client.17280 trim_cache size 0 max 16384 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:36:08.507 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "140cbba9-4c2f-40f6-8064-359158ca51b3", 2024-09-16T00:36:08.508 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:36:08.508 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:36:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 -- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5b880a32c0 msgr2=0x7f5b880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5b880a32c0 0x7f5b880a5780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b9409c040 tx=0x7f5b940bc040 comp rx=0 tx=0).stop 2024-09-16T00:36:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 -- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c1082a0 msgr2=0x7f5b9c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c1082a0 0x7f5b9c19ce90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400eaa0 tx=0x7f5b9400ef60 comp rx=0 tx=0).stop 2024-09-16T00:36:08.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 -- 172.21.15.1:0/2184658232 shutdown_connections 2024-09-16T00:36:08.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b9c1082a0 0x7f5b9c19ce90 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9c1036e0 0x7f5b9c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5b880a32c0 0x7f5b880a5780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 --2- 172.21.15.1:0/2184658232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c102d30 0x7f5b9c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:08.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 -- 172.21.15.1:0/2184658232 >> 172.21.15.1:0/2184658232 conn(0x7f5b9c0fc7d0 msgr2=0x7f5b9c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:08.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 -- 172.21.15.1:0/2184658232 shutdown_connections 2024-09-16T00:36:08.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:08.501+0000 7f5ba2861640 1 -- 172.21.15.1:0/2184658232 wait complete. 2024-09-16T00:36:09.292 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:36:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:10.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.064+0000 7fa6ed281640 1 -- 172.21.15.1:0/4252271868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8101de0 msgr2=0x7fa6e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:10.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.064+0000 7fa6ed281640 1 --2- 172.21.15.1:0/4252271868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8101de0 0x7fa6e81021c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009900 tx=0x7fa6d802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.064+0000 7fa6ed281640 1 -- 172.21.15.1:0/4252271868 shutdown_connections 2024-09-16T00:36:10.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.064+0000 7fa6ed281640 1 --2- 172.21.15.1:0/4252271868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e8106f10 0x7fa6e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.064+0000 7fa6ed281640 1 --2- 172.21.15.1:0/4252271868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6e8102790 0x7fa6e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.064+0000 7fa6ed281640 1 --2- 172.21.15.1:0/4252271868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8101de0 0x7fa6e81021c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.064+0000 7fa6ed281640 1 -- 172.21.15.1:0/4252271868 >> 172.21.15.1:0/4252271868 conn(0x7fa6e80fd8c0 msgr2=0x7fa6e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.064+0000 7fa6ed281640 1 -- 172.21.15.1:0/4252271868 shutdown_connections 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.064+0000 7fa6ed281640 1 -- 172.21.15.1:0/4252271868 wait complete. 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6ed281640 1 Processor -- start 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6ed281640 1 -- start start 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6ed281640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6e8101de0 0x7fa6e81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6ed281640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e8102790 0x7fa6e8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6ed281640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8106f10 0x7fa6e819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6ed281640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e8111f70 con 0x7fa6e8106f10 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6ed281640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e8111df0 con 0x7fa6e8102790 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6ed281640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81120f0 con 0x7fa6e8101de0 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8106f10 0x7fa6e819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8106f10 0x7fa6e819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58876/0 (socket says 172.21.15.1:58876) 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e77fe640 1 -- 172.21.15.1:0/1270834759 learned_addr learned my addr 172.21.15.1:0/1270834759 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e8102790 0x7fa6e8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6e8101de0 0x7fa6e81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e77fe640 1 -- 172.21.15.1:0/1270834759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6e8101de0 msgr2=0x7fa6e81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e77fe640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6e8101de0 0x7fa6e81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e77fe640 1 -- 172.21.15.1:0/1270834759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e8102790 msgr2=0x7fa6e8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e77fe640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e8102790 0x7fa6e8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e77fe640 1 -- 172.21.15.1:0/1270834759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e819d380 con 0x7fa6e8106f10 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.065+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6e8101de0 0x7fa6e81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.066+0000 7fa6e77fe640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8106f10 0x7fa6e819cca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000cce0 tx=0x7fa6e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.066+0000 7fa6cffff640 1 -- 172.21.15.1:0/1270834759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e000f450 con 0x7fa6e8106f10 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.066+0000 7fa6ed281640 1 -- 172.21.15.1:0/1270834759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e819d670 con 0x7fa6e8106f10 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.066+0000 7fa6cffff640 1 -- 172.21.15.1:0/1270834759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e000fad0 con 0x7fa6e8106f10 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.066+0000 7fa6cffff640 1 -- 172.21.15.1:0/1270834759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0018c10 con 0x7fa6e8106f10 2024-09-16T00:36:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.066+0000 7fa6ed281640 1 -- 172.21.15.1:0/1270834759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e81a4f50 con 0x7fa6e8106f10 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.068+0000 7fa6cffff640 1 -- 172.21.15.1:0/1270834759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa6e000f5f0 con 0x7fa6e8106f10 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6ef65a740 1 Processor -- start 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6ef65a740 1 -- start start 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42743f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743fca0 0x55b42744e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42744ea90 0x55b427450e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b4273a52e0 con 0x55b42743f380 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6cffff640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6d00a3240 0x7fa6d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b42734cce0 con 0x55b42744ea90 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b4273a12a0 con 0x55b42743fca0 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42743f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6e77fe640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6d00a3240 0x7fa6d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42743f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58890/0 (socket says 172.21.15.1:58890) 2024-09-16T00:36:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6e67fc640 1 -- 172.21.15.1:0/1532895376 learned_addr learned my addr 172.21.15.1:0/1532895376 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/1532895376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743fca0 0x55b42744e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6cffff640 1 -- 172.21.15.1:0/1270834759 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa6e00d0390 con 0x7fa6e8106f10 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.069+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/1532895376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42744ea90 0x55b427450e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6e67fc640 1 -- 172.21.15.1:0/1532895376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743fca0 msgr2=0x55b42744e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/1532895376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743fca0 0x55b42744e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6e67fc640 1 -- 172.21.15.1:0/1532895376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42744ea90 msgr2=0x55b427450e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/1532895376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42744ea90 0x55b427450e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6e67fc640 1 -- 172.21.15.1:0/1532895376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b427331440 con 0x55b42743f380 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/1532895376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42743f760 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc013550 tx=0x7fa6dc00a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79691927e07cb89 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6e77fe640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6d00a3240 0x7fa6d00a5700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa6e001f000 tx=0x7fa6e006b2f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6cd7fa640 1 -- 172.21.15.1:0/1532895376 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc019030 con 0x55b42743f380 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6cd7fa640 1 -- 172.21.15.1:0/1532895376 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6dc005500 con 0x55b42743f380 2024-09-16T00:36:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.070+0000 7fa6cd7fa640 1 -- 172.21.15.1:0/1532895376 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc00f890 con 0x55b42743f380 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 -- 172.21.15.1:0/1532895376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 msgr2=0x55b42743f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/1532895376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42743f760 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc013550 tx=0x7fa6dc00a760 comp rx=0 tx=0).stop 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 -- 172.21.15.1:0/1532895376 shutdown_connections 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/1532895376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42744ea90 0x55b427450e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/1532895376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743fca0 0x55b42744e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/1532895376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42743f760 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 -- 172.21.15.1:0/1532895376 >> 172.21.15.1:0/1532895376 conn(0x55b42743ba30 msgr2=0x55b42743be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 -- 172.21.15.1:0/1532895376 shutdown_connections 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 -- 172.21.15.1:0/1532895376 wait complete. 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 -- start start 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 Processor -- start 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42744e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.071+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 0x55b42744b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42744ea90 0x55b42744bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b427451660 con 0x55b42743f380 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b427360050 con 0x55b42743fca0 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b4274517e0 con 0x55b42744ea90 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6ef65a740 10 client.0 _ll_register_callbacks cb 0x55b4273a0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42744e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42744e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58906/0 (socket says 172.21.15.1:58906) 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 -- 172.21.15.1:0/4118236665 learned_addr learned my addr 172.21.15.1:0/4118236665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 0x55b42744b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42744ea90 0x55b42744bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 -- 172.21.15.1:0/4118236665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42744ea90 msgr2=0x55b42744bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42744ea90 0x55b42744bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 -- 172.21.15.1:0/4118236665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 msgr2=0x55b42744b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 0x55b42744b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 -- 172.21.15.1:0/4118236665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4271163c0 con 0x55b42743f380 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 0x55b42744b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42744e350 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0057c0 tx=0x7fa6dc00abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb25c1250aee398 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6caff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6caff5640 1 -- 172.21.15.1:0/4118236665 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc019040 con 0x55b42743f380 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6caff5640 1 -- 172.21.15.1:0/4118236665 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa6dc00f630 con 0x55b42743f380 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.072+0000 7fa6ef65a740 1 -- 172.21.15.1:0/4118236665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6dc005500 con 0x55b42743f380 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6caff5640 1 -- 172.21.15.1:0/4118236665 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa6dc02f050 con 0x55b42743f380 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6ef65a740 10 client.17292 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6ef65a740 1 -- 172.21.15.1:0/4118236665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b42745cff0 con 0x55b42743f380 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6caff5640 1 -- 172.21.15.1:0/4118236665 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6dc005500 con 0x55b42743f380 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6caff5640 1 -- 172.21.15.1:0/4118236665 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa6dc01e8e0 con 0x55b42743f380 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6ef65a740 10 client.17292 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6ef65a740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6ef65a740 20 client.17292 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b42745e450 0x55b427460840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6ef65a740 4 client.17292 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6ef65a740 1 -- 172.21.15.1:0/4118236665 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4273b2850 con 0x55b42745e450 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.073+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b42745e450 0x55b427460840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.074+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b42745e450 0x55b427460840 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c3e5a29d22b55d4 server_cookie=93bd849b2fa608e4 in_seq=0 out_seq=0 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.074+0000 7fa6caff5640 10 client.17292 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.074+0000 7fa6caff5640 1 -- 172.21.15.1:0/4118236665 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4273b2850 con 0x55b42745e450 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.074+0000 7fa6caff5640 10 client.17292 handle_command_reply: tid=0 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.074+0000 7fa6ef65a740 1 client.17292 shutdown 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 -- 172.21.15.1:0/4118236665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 msgr2=0x55b42744e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42744e350 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0057c0 tx=0x7fa6dc00abc0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 -- 172.21.15.1:0/4118236665 shutdown_connections 2024-09-16T00:36:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b42745e450 0x55b427460840 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42744ea90 0x55b42744bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 0x55b42744b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/4118236665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743f380 0x55b42744e350 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6caff5640 0 client.17292 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 -- 172.21.15.1:0/4118236665 >> 172.21.15.1:0/4118236665 conn(0x55b42743ba30 msgr2=0x55b427441790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 -- 172.21.15.1:0/4118236665 shutdown_connections 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 1 -- 172.21.15.1:0/4118236665 wait complete. 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.075+0000 7fa6ef65a740 20 client.17292 trim_cache size 0 max 16384 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6ef65a740 1 Processor -- start 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6ef65a740 1 -- start start 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b42745a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743f380 0x55b427446d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 0x55b42744a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81af470 con 0x55b42743ba30 2024-09-16T00:36:10.116 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:36:10.116 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:36:10.116 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:36:10.116 INFO:teuthology.orchestra.run.smithi001.stdout: "140cbba9-4c2f-40f6-8064-359158ca51b3": { 2024-09-16T00:36:10.116 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:36:10.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "140cbba9-4c2f-40f6-8064-359158ca51b3", 2024-09-16T00:36:10.116 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:36:10.116 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:10.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:10.117 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:36:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81acd40 con 0x55b42743fca0 2024-09-16T00:36:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81aab90 con 0x55b42743f380 2024-09-16T00:36:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b42745a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b42745a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58914/0 (socket says 172.21.15.1:58914) 2024-09-16T00:36:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.092+0000 7fa6e67fc640 1 -- 172.21.15.1:0/2288058471 learned_addr learned my addr 172.21.15.1:0/2288058471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6e67fc640 1 -- 172.21.15.1:0/2288058471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743f380 msgr2=0x55b427446d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/2288058471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743f380 0x55b427446d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6e67fc640 1 -- 172.21.15.1:0/2288058471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 msgr2=0x55b42744a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/2288058471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 0x55b42744a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6e67fc640 1 -- 172.21.15.1:0/2288058471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4274511c0 con 0x55b42743ba30 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/2288058471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b42745a900 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00ac70 tx=0x7fa6dc005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559b39992d599148 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6ccff9640 1 -- 172.21.15.1:0/2288058471 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc019040 con 0x55b42743ba30 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6ccff9640 1 -- 172.21.15.1:0/2288058471 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6dc00ad20 con 0x55b42743ba30 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6ccff9640 1 -- 172.21.15.1:0/2288058471 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc01e460 con 0x55b42743ba30 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6ef65a740 1 -- 172.21.15.1:0/2288058471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 msgr2=0x55b42745a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.093+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/2288058471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b42745a900 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00ac70 tx=0x7fa6dc005ac0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 -- 172.21.15.1:0/2288058471 shutdown_connections 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/2288058471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743fca0 0x55b42744a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/2288058471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743f380 0x55b427446d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/2288058471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b42745a900 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 -- 172.21.15.1:0/2288058471 >> 172.21.15.1:0/2288058471 conn(0x55b42744bca0 msgr2=0x55b427449ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 -- 172.21.15.1:0/2288058471 shutdown_connections 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 -- 172.21.15.1:0/2288058471 wait complete. 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 -- start start 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 Processor -- start 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b427451fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743f380 0x55b427452510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743fca0 0x55b4274c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81ab200 con 0x55b42743ba30 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81ac670 con 0x55b42743f380 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81ab8d0 con 0x55b42743fca0 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6ef65a740 10 client.0 _ll_register_callbacks cb 0x7fa6d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b427451fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b427451fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58928/0 (socket says 172.21.15.1:58928) 2024-09-16T00:36:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.094+0000 7fa6e67fc640 1 -- 172.21.15.1:0/3537760612 learned_addr learned my addr 172.21.15.1:0/3537760612 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:10.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743f380 0x55b427452510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6e67fc640 1 -- 172.21.15.1:0/3537760612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743fca0 msgr2=0x55b4274c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:10.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743fca0 0x55b4274c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6e67fc640 1 -- 172.21.15.1:0/3537760612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743f380 msgr2=0x55b427452510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:10.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743f380 0x55b427452510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6e67fc640 1 -- 172.21.15.1:0/3537760612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4274c7430 con 0x55b42743ba30 2024-09-16T00:36:10.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6e67fc640 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b427451fd0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00a980 tx=0x7fa6dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb6f6a7411eda54 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:10.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6caff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6caff5640 1 -- 172.21.15.1:0/3537760612 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc019070 con 0x55b42743ba30 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6caff5640 1 -- 172.21.15.1:0/3537760612 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa6dc038040 con 0x55b42743ba30 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6ef65a740 1 -- 172.21.15.1:0/3537760612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6dc00ad20 con 0x55b42743ba30 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6caff5640 1 -- 172.21.15.1:0/3537760612 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa6dc00a2d0 con 0x55b42743ba30 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6ef65a740 10 client.17298 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6ef65a740 1 -- 172.21.15.1:0/3537760612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b4274c8740 con 0x55b42743ba30 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.095+0000 7fa6caff5640 1 -- 172.21.15.1:0/3537760612 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6dc00ad20 con 0x55b42743ba30 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6caff5640 1 -- 172.21.15.1:0/3537760612 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa6dc0130d0 con 0x55b42743ba30 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6ef65a740 10 client.17298 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6ef65a740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6ef65a740 20 client.17298 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b42744ea90 0x55b4274cad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6ef65a740 4 client.17298 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6ef65a740 1 -- 172.21.15.1:0/3537760612 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b4273acd60 con 0x55b42744ea90 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b42744ea90 0x55b4274cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6caff5640 10 client.17298 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6e6ffd640 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b42744ea90 0x55b4274cad50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7920e242574b3337 server_cookie=bdaf8c412986df32 in_seq=0 out_seq=0 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6caff5640 1 -- 172.21.15.1:0/3537760612 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b4273acd60 con 0x55b42744ea90 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6caff5640 10 client.17298 handle_command_reply: tid=0 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6ef65a740 1 client.17298 shutdown 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6ef65a740 1 -- 172.21.15.1:0/3537760612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 msgr2=0x55b427451fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.096+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b427451fd0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00a980 tx=0x7fa6dc004a40 comp rx=0 tx=0).stop 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ef65a740 1 -- 172.21.15.1:0/3537760612 shutdown_connections 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b42744ea90 0x55b4274cad50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b42743fca0 0x55b4274c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b42743f380 0x55b427452510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ef65a740 1 --2- 172.21.15.1:0/3537760612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b42743ba30 0x55b427451fd0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6caff5640 0 client.17298 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ef65a740 1 -- 172.21.15.1:0/3537760612 >> 172.21.15.1:0/3537760612 conn(0x55b42744bca0 msgr2=0x55b42744a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ef65a740 1 -- 172.21.15.1:0/3537760612 shutdown_connections 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ef65a740 1 -- 172.21.15.1:0/3537760612 wait complete. 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ef65a740 20 client.17298 trim_cache size 0 max 16384 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 -- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6d00a3240 msgr2=0x7fa6d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6d00a3240 0x7fa6d00a5700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa6e001f000 tx=0x7fa6e006b2f0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 -- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8106f10 msgr2=0x7fa6e819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8106f10 0x7fa6e819cca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000cce0 tx=0x7fa6e000a620 comp rx=0 tx=0).stop 2024-09-16T00:36:10.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 -- 172.21.15.1:0/1270834759 shutdown_connections 2024-09-16T00:36:10.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6e8106f10 0x7fa6e819cca0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e8102790 0x7fa6e8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa6d00a3240 0x7fa6d00a5700 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 --2- 172.21.15.1:0/1270834759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6e8101de0 0x7fa6e81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:10.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 -- 172.21.15.1:0/1270834759 >> 172.21.15.1:0/1270834759 conn(0x7fa6e80fd8c0 msgr2=0x7fa6e80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:10.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.097+0000 7fa6ed281640 1 -- 172.21.15.1:0/1270834759 shutdown_connections 2024-09-16T00:36:10.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:10.098+0000 7fa6ed281640 1 -- 172.21.15.1:0/1270834759 wait complete. 2024-09-16T00:36:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:10 smithi019 ceph-mon[35186]: pgmap v583: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:36:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:10 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:36:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:10 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:10 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:36:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:10 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:10.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:10 smithi001 ceph-mon[30934]: pgmap v583: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:36:10.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:10 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:36:10.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:10 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:10.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:10 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:36:10.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:10 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:10.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:10 smithi153 ceph-mon[35538]: pgmap v583: 129 pgs: 129 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:36:10.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:10 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:36:10.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:10 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:10.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:10 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:36:10.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:10 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:10.890 INFO:tasks.cephfs.filesystem:scrub status for tag:140cbba9-4c2f-40f6-8064-359158ca51b3 - {'path': '/', 'tag': '140cbba9-4c2f-40f6-8064-359158ca51b3', 'options': 'recursive,force'} 2024-09-16T00:36:12.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:12 smithi019 ceph-mon[35186]: pgmap v584: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:12 smithi001 ceph-mon[30934]: pgmap v584: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:12.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:12 smithi153 ceph-mon[35538]: pgmap v584: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:13.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:13 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:36:13.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:13 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:36:13.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:13 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:36:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:14 smithi019 ceph-mon[35186]: pgmap v585: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:36:14.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:14 smithi001 ceph-mon[30934]: pgmap v585: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:36:14.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:14 smithi153 ceph-mon[35538]: pgmap v585: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:36:15.347 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.7 2024-09-16T00:36:15.347 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:36:15.618 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:16.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.100+0000 7f9a255bd640 1 -- 172.21.15.1:0/287687004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a20102de0 msgr2=0x7f9a201031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.100+0000 7f9a255bd640 1 --2- 172.21.15.1:0/287687004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a20102de0 0x7f9a201031c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009900 tx=0x7f9a1002a480 comp rx=0 tx=0).stop 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.100+0000 7f9a255bd640 1 -- 172.21.15.1:0/287687004 shutdown_connections 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.100+0000 7f9a255bd640 1 --2- 172.21.15.1:0/287687004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a2010c180 0x7f9a2010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.100+0000 7f9a255bd640 1 --2- 172.21.15.1:0/287687004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a20103700 0x7f9a2010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.100+0000 7f9a255bd640 1 --2- 172.21.15.1:0/287687004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a20102de0 0x7f9a201031c0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.100+0000 7f9a255bd640 1 -- 172.21.15.1:0/287687004 >> 172.21.15.1:0/287687004 conn(0x7f9a200fc7a0 msgr2=0x7f9a200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.100+0000 7f9a255bd640 1 -- 172.21.15.1:0/287687004 shutdown_connections 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.100+0000 7f9a255bd640 1 -- 172.21.15.1:0/287687004 wait complete. 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a255bd640 1 Processor -- start 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a255bd640 1 -- start start 2024-09-16T00:36:16.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a255bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a20102de0 0x7f9a20198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a255bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a20103700 0x7f9a20198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a255bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a2010c180 0x7f9a2019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a255bd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20111f30 con 0x7f9a2010c180 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a255bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20111db0 con 0x7f9a20102de0 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a255bd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a201120b0 con 0x7f9a20103700 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a2010c180 0x7f9a2019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a2010c180 0x7f9a2019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55682/0 (socket says 172.21.15.1:55682) 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a1f7fe640 1 -- 172.21.15.1:0/3849773732 learned_addr learned my addr 172.21.15.1:0/3849773732 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a1effd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a20102de0 0x7f9a20198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.101+0000 7f9a17fff640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a20103700 0x7f9a20198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1f7fe640 1 -- 172.21.15.1:0/3849773732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a20103700 msgr2=0x7f9a20198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1f7fe640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a20103700 0x7f9a20198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1f7fe640 1 -- 172.21.15.1:0/3849773732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a20102de0 msgr2=0x7f9a20198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:16.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1f7fe640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a20102de0 0x7f9a20198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1f7fe640 1 -- 172.21.15.1:0/3849773732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2019d3f0 con 0x7f9a2010c180 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1effd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a20102de0 0x7f9a20198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1f7fe640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a2010c180 0x7f9a2019cd10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800eb10 tx=0x7f9a1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1cff9640 1 -- 172.21.15.1:0/3849773732 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18004b20 con 0x7f9a2010c180 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2019d6e0 con 0x7f9a2010c180 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1cff9640 1 -- 172.21.15.1:0/3849773732 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a18004cc0 con 0x7f9a2010c180 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a1cff9640 1 -- 172.21.15.1:0/3849773732 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18010760 con 0x7f9a2010c180 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.102+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201a4fc0 con 0x7f9a2010c180 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.103+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a201031c0 con 0x7f9a2010c180 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.104+0000 7f9a1cff9640 1 -- 172.21.15.1:0/3849773732 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9a18005020 con 0x7f9a2010c180 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.105+0000 7f9a1cff9640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a0c0a3240 0x7f9a0c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.105+0000 7f9a1effd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a0c0a3240 0x7f9a0c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.106+0000 7f9a1cff9640 1 -- 172.21.15.1:0/3849773732 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9a180d0260 con 0x7f9a2010c180 2024-09-16T00:36:16.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.106+0000 7f9a1effd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a0c0a3240 0x7f9a0c0a5700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009230 tx=0x7f9a100058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:16.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.109+0000 7f9a1cff9640 1 -- 172.21.15.1:0/3849773732 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a1809cb20 con 0x7f9a2010c180 2024-09-16T00:36:16.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:16 smithi001 ceph-mon[30934]: pgmap v586: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:36:16.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.324+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a201072c0 con 0x7f9a2010c180 2024-09-16T00:36:16.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.325+0000 7f9a1cff9640 1 -- 172.21.15.1:0/3849773732 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f9a1809c420 con 0x7f9a2010c180 2024-09-16T00:36:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:36:16.326 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:36:16.326 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:36:16.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a0c0a3240 msgr2=0x7f9a0c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:16.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a0c0a3240 0x7f9a0c0a5700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009230 tx=0x7f9a100058d0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a2010c180 msgr2=0x7f9a2019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:16.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a2010c180 0x7f9a2019cd10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800eb10 tx=0x7f9a1800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 shutdown_connections 2024-09-16T00:36:16.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a2010c180 0x7f9a2019cd10 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a0c0a3240 0x7f9a0c0a5700 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a20103700 0x7f9a20198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 --2- 172.21.15.1:0/3849773732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a20102de0 0x7f9a20198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:16.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.327+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 >> 172.21.15.1:0/3849773732 conn(0x7f9a200fc7a0 msgr2=0x7f9a200fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:16.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.328+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 shutdown_connections 2024-09-16T00:36:16.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:16.328+0000 7f9a255bd640 1 -- 172.21.15.1:0/3849773732 wait complete. 2024-09-16T00:36:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:16 smithi019 ceph-mon[35186]: pgmap v586: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:36:16.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:16 smithi153 ceph-mon[35538]: pgmap v586: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:36:17.235 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0' 2024-09-16T00:36:17.235 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0 2024-09-16T00:36:17.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:17 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3849773732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:17.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:17 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3849773732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:17.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:17 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3849773732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:18.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 -- 172.21.15.1:0/2858621104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08381027a0 msgr2=0x7f08381067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 --2- 172.21.15.1:0/2858621104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08381027a0 0x7f08381067f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f082c009900 tx=0x7f082c02a410 comp rx=0 tx=0).stop 2024-09-16T00:36:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 -- 172.21.15.1:0/2858621104 shutdown_connections 2024-09-16T00:36:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 --2- 172.21.15.1:0/2858621104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0838106f20 0x7f083810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 --2- 172.21.15.1:0/2858621104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08381027a0 0x7f08381067f0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 --2- 172.21.15.1:0/2858621104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0838101df0 0x7f08381021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 -- 172.21.15.1:0/2858621104 >> 172.21.15.1:0/2858621104 conn(0x7f08380fd8f0 msgr2=0x7f08380ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 -- 172.21.15.1:0/2858621104 shutdown_connections 2024-09-16T00:36:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 -- 172.21.15.1:0/2858621104 wait complete. 2024-09-16T00:36:18.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.038+0000 7f083d438640 1 Processor -- start 2024-09-16T00:36:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f083d438640 1 -- start start 2024-09-16T00:36:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f083d438640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0838101df0 0x7f0838198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f083d438640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08381027a0 0x7f08381989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f083d438640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0838106f20 0x7f083819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f083d438640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0838111f80 con 0x7f0838106f20 2024-09-16T00:36:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f083d438640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0838111e00 con 0x7f08381027a0 2024-09-16T00:36:18.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f083d438640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0838112100 con 0x7f0838101df0 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f08377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0838106f20 0x7f083819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f0836ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0838101df0 0x7f0838198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f08377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0838106f20 0x7f083819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55698/0 (socket says 172.21.15.1:55698) 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f08377fe640 1 -- 172.21.15.1:0/2693326162 learned_addr learned my addr 172.21.15.1:0/2693326162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f08367fc640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08381027a0 0x7f08381989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f08377fe640 1 -- 172.21.15.1:0/2693326162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0838101df0 msgr2=0x7f0838198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f08377fe640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0838101df0 0x7f0838198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f08377fe640 1 -- 172.21.15.1:0/2693326162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08381027a0 msgr2=0x7f08381989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f08377fe640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08381027a0 0x7f08381989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.039+0000 7f08377fe640 1 -- 172.21.15.1:0/2693326162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083819d440 con 0x7f0838106f20 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.040+0000 7f08377fe640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0838106f20 0x7f083819cd60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0830009fd0 tx=0x7f083000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:18.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.040+0000 7f081ffff640 1 -- 172.21.15.1:0/2693326162 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830017070 con 0x7f0838106f20 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.040+0000 7f083d438640 1 -- 172.21.15.1:0/2693326162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083819d730 con 0x7f0838106f20 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.040+0000 7f081ffff640 1 -- 172.21.15.1:0/2693326162 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0830003e50 con 0x7f0838106f20 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.040+0000 7f081ffff640 1 -- 172.21.15.1:0/2693326162 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300106c0 con 0x7f0838106f20 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.040+0000 7f083d438640 1 -- 172.21.15.1:0/2693326162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08381a5010 con 0x7f0838106f20 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.042+0000 7f081ffff640 1 -- 172.21.15.1:0/2693326162 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f0830003970 con 0x7f0838106f20 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f083f811740 1 Processor -- start 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f083f811740 1 -- start start 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f083f811740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 0x55872e2d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f083f811740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f083f811740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2e4d50 0x55872e2e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f083f811740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55872e23b2a0 con 0x55872e2d5f60 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f083f811740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55872e1e2b10 con 0x55872e2d5640 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f083f811740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55872e237260 con 0x55872e2e4d50 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f0836ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f0836ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55714/0 (socket says 172.21.15.1:55714) 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.043+0000 7f0836ffd640 1 -- 172.21.15.1:0/2882623934 learned_addr learned my addr 172.21.15.1:0/2882623934 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f081ffff640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08200a3240 0x7f08200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f08367fc640 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 0x55872e2d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f08367fc640 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2e4d50 0x55872e2e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f0836ffd640 1 -- 172.21.15.1:0/2882623934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2e4d50 msgr2=0x55872e2e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f0836ffd640 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2e4d50 0x55872e2e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f0836ffd640 1 -- 172.21.15.1:0/2882623934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 msgr2=0x55872e2d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f0836ffd640 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 0x55872e2d5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f0836ffd640 1 -- 172.21.15.1:0/2882623934 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872dfacb90 con 0x55872e2d5f60 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f08367fc640 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2e4d50 0x55872e2e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f081ffff640 1 -- 172.21.15.1:0/2693326162 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f08300cf9f0 con 0x7f0838106f20 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f08377fe640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08200a3240 0x7f08200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f0836ffd640 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e4810 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0828009cf0 tx=0x7f0828011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3512934bd8f81fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f081d7fa640 1 -- 172.21.15.1:0/2882623934 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828004230 con 0x55872e2d5f60 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f081d7fa640 1 -- 172.21.15.1:0/2882623934 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08280043d0 con 0x55872e2d5f60 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.044+0000 7f081d7fa640 1 -- 172.21.15.1:0/2882623934 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08280198c0 con 0x55872e2d5f60 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 -- 172.21.15.1:0/2882623934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 msgr2=0x55872e2e4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f08377fe640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08200a3240 0x7f08200a5700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f083000b760 tx=0x7f0830004020 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e4810 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0828009cf0 tx=0x7f0828011810 comp rx=0 tx=0).stop 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 -- 172.21.15.1:0/2882623934 shutdown_connections 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2e4d50 0x55872e2e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e4810 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 --2- 172.21.15.1:0/2882623934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 0x55872e2d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 -- 172.21.15.1:0/2882623934 >> 172.21.15.1:0/2882623934 conn(0x55872e2d1cf0 msgr2=0x55872e2d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 -- 172.21.15.1:0/2882623934 shutdown_connections 2024-09-16T00:36:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 -- 172.21.15.1:0/2882623934 wait complete. 2024-09-16T00:36:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.045+0000 7f083f811740 1 -- start start 2024-09-16T00:36:18.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f083f811740 1 Processor -- start 2024-09-16T00:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f083f811740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 0x55872e2e1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f083f811740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f083f811740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2e4d50 0x55872e2e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f083f811740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55872e2e7920 con 0x55872e2d5f60 2024-09-16T00:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f083f811740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55872e1c36e0 con 0x55872e2e4d50 2024-09-16T00:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f083f811740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55872e2e7aa0 con 0x55872e2d5640 2024-09-16T00:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f083f811740 10 client.0 _ll_register_callbacks cb 0x55872e1e7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f0836ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f0836ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55728/0 (socket says 172.21.15.1:55728) 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f0836ffd640 1 -- 172.21.15.1:0/2995230548 learned_addr learned my addr 172.21.15.1:0/2995230548 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f08367fc640 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 0x55872e2e1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f08367fc640 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2e4d50 0x55872e2e26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f0836ffd640 1 -- 172.21.15.1:0/2995230548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 msgr2=0x55872e2e1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f0836ffd640 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 0x55872e2e1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f0836ffd640 1 -- 172.21.15.1:0/2995230548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2e4d50 msgr2=0x55872e2e26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f0836ffd640 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2e4d50 0x55872e2e26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.046+0000 7f0836ffd640 1 -- 172.21.15.1:0/2995230548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55872e2f0e80 con 0x55872e2d5f60 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f0836ffd640 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e2190 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0828009730 tx=0x7f08280052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2d3a31c9dac702 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f081aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f081aff5640 1 -- 172.21.15.1:0/2995230548 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828027070 con 0x55872e2d5f60 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f081aff5640 1 -- 172.21.15.1:0/2995230548 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f08280037f0 con 0x55872e2d5f60 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f083f811740 1 -- 172.21.15.1:0/2995230548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08280043d0 con 0x55872e2d5f60 2024-09-16T00:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f081aff5640 1 -- 172.21.15.1:0/2995230548 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0828033050 con 0x55872e2d5f60 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f083f811740 10 client.17316 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f083f811740 1 -- 172.21.15.1:0/2995230548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55872e2f3760 con 0x55872e2d5f60 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f081aff5640 1 -- 172.21.15.1:0/2995230548 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08280043d0 con 0x55872e2d5f60 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f081aff5640 1 -- 172.21.15.1:0/2995230548 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0828004ce0 con 0x55872e2d5f60 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f083f811740 10 client.17316 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f083f811740 10 client.17316 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f083f811740 20 client.17316 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f083f811740 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55872e2f4b10 0x55872e2f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f083f811740 4 client.17316 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.047+0000 7f083f811740 1 -- 172.21.15.1:0/2995230548 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55872e2489d0 con 0x55872e2f4b10 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.048+0000 7f08367fc640 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55872e2f4b10 0x55872e2f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.048+0000 7f081aff5640 10 client.17316 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.048+0000 7f08367fc640 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55872e2f4b10 0x55872e2f6f00 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b8d033e5f93e83 server_cookie=210ef33db4cb6d7c in_seq=0 out_seq=0 2024-09-16T00:36:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f081aff5640 1 -- 172.21.15.1:0/2995230548 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55872e2489d0 con 0x55872e2f4b10 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f081aff5640 10 client.17316 handle_command_reply: tid=0 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 client.17316 shutdown 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 -- 172.21.15.1:0/2995230548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 msgr2=0x55872e2e2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e2190 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0828009730 tx=0x7f08280052f0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 -- 172.21.15.1:0/2995230548 shutdown_connections 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55872e2f4b10 0x55872e2f6f00 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2e4d50 0x55872e2e26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2d5f60 0x55872e2e2190 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 --2- 172.21.15.1:0/2995230548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 0x55872e2e1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 -- 172.21.15.1:0/2995230548 >> 172.21.15.1:0/2995230548 conn(0x55872e2d1cf0 msgr2=0x55872e2d7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f081aff5640 0 client.17316 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 -- 172.21.15.1:0/2995230548 shutdown_connections 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 1 -- 172.21.15.1:0/2995230548 wait complete. 2024-09-16T00:36:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.049+0000 7f083f811740 20 client.17316 trim_cache size 0 max 16384 2024-09-16T00:36:18.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f083f811740 1 Processor -- start 2024-09-16T00:36:18.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f083f811740 1 -- start start 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f083f811740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 0x55872e2e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f083f811740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5f60 0x55872e2fc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f083f811740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e2fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f083f811740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f08381af630 con 0x55872e2e4d50 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f083f811740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08381acf00 con 0x55872e2d5640 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f083f811740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f08381aad50 con 0x55872e2d5f60 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f08367fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e2fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f08367fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e2fcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55736/0 (socket says 172.21.15.1:55736) 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.066+0000 7f08367fc640 1 -- 172.21.15.1:0/165980876 learned_addr learned my addr 172.21.15.1:0/165980876 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f08367fc640 1 --2- 172.21.15.1:0/165980876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 0x55872e2e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f0836ffd640 1 --2- 172.21.15.1:0/165980876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5f60 0x55872e2fc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f08367fc640 1 -- 172.21.15.1:0/165980876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5f60 msgr2=0x55872e2fc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:18.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f08367fc640 1 --2- 172.21.15.1:0/165980876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5f60 0x55872e2fc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f08367fc640 1 -- 172.21.15.1:0/165980876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 msgr2=0x55872e2e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:18.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f08367fc640 1 --2- 172.21.15.1:0/165980876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 0x55872e2e0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f08367fc640 1 -- 172.21.15.1:0/165980876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872e2d1b20 con 0x55872e2e4d50 2024-09-16T00:36:18.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f08367fc640 1 --2- 172.21.15.1:0/165980876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e2fcc50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f082c000c00 tx=0x7f082c0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e683b53e09436 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:18.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f081cff9640 1 -- 172.21.15.1:0/165980876 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c035040 con 0x55872e2e4d50 2024-09-16T00:36:18.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f081cff9640 1 -- 172.21.15.1:0/165980876 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f082c0035f0 con 0x55872e2e4d50 2024-09-16T00:36:18.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.067+0000 7f081cff9640 1 -- 172.21.15.1:0/165980876 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c04a5b0 con 0x55872e2e4d50 2024-09-16T00:36:18.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 -- 172.21.15.1:0/165980876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 msgr2=0x55872e2fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:18.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 --2- 172.21.15.1:0/165980876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e2fcc50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f082c000c00 tx=0x7f082c0314e0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 -- 172.21.15.1:0/165980876 shutdown_connections 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 --2- 172.21.15.1:0/165980876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e2fcc50 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 --2- 172.21.15.1:0/165980876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5f60 0x55872e2fc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 --2- 172.21.15.1:0/165980876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5640 0x55872e2e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 -- 172.21.15.1:0/165980876 >> 172.21.15.1:0/165980876 conn(0x55872e2d1cf0 msgr2=0x55872e2e03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 -- 172.21.15.1:0/165980876 shutdown_connections 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 -- 172.21.15.1:0/165980876 wait complete. 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 -- start start 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.068+0000 7f083f811740 1 Processor -- start 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f083f811740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 0x55872e2e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f083f811740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5f60 0x55872e2e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f083f811740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e35e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f083f811740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f08381ab3c0 con 0x55872e2e4d50 2024-09-16T00:36:18.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f083f811740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08381ac830 con 0x55872e2d5f60 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f083f811740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f08381aba90 con 0x55872e2d5640 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f083f811740 10 client.0 _ll_register_callbacks cb 0x7f0828005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e35e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e35e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55742/0 (socket says 172.21.15.1:55742) 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 -- 172.21.15.1:0/1668558756 learned_addr learned my addr 172.21.15.1:0/1668558756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f0836ffd640 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5f60 0x55872e2e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 0x55872e2e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 -- 172.21.15.1:0/1668558756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 msgr2=0x55872e2e8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 0x55872e2e8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 -- 172.21.15.1:0/1668558756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5f60 msgr2=0x55872e2e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5f60 0x55872e2e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 -- 172.21.15.1:0/1668558756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55872e2cd260 con 0x55872e2e4d50 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f08367fc640 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e35e6e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f082c005e90 tx=0x7f082c04c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97bb9c04dd3ac89 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:18.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f081aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f081aff5640 1 -- 172.21.15.1:0/1668558756 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c035040 con 0x55872e2e4d50 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f081aff5640 1 -- 172.21.15.1:0/1668558756 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f082c0039a0 con 0x55872e2e4d50 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.069+0000 7f083f811740 1 -- 172.21.15.1:0/1668558756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f082c0035f0 con 0x55872e2e4d50 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f081aff5640 1 -- 172.21.15.1:0/1668558756 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f082c003ba0 con 0x55872e2e4d50 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f083f811740 10 client.17322 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f083f811740 1 -- 172.21.15.1:0/1668558756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55872e35fc10 con 0x55872e2e4d50 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f081aff5640 1 -- 172.21.15.1:0/1668558756 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f082c0035f0 con 0x55872e2e4d50 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f081aff5640 1 -- 172.21.15.1:0/1668558756 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f082c04a030 con 0x55872e2e4d50 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f083f811740 10 client.17322 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f083f811740 10 client.17322 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f083f811740 20 client.17322 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f083f811740 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55872e360240 0x55872e362630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f083f811740 4 client.17322 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}] 2024-09-16T00:36:18.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.070+0000 7f083f811740 1 -- 172.21.15.1:0/1668558756 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}) -- 0x55872e242d20 con 0x55872e360240 2024-09-16T00:36:18.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.071+0000 7f0836ffd640 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55872e360240 0x55872e362630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:18.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.071+0000 7f0836ffd640 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55872e360240 0x55872e362630 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303edf759e866d40 server_cookie=3bc2546bb7606bd1 in_seq=0 out_seq=0 2024-09-16T00:36:18.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.071+0000 7f081aff5640 10 client.17322 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f081aff5640 1 -- 172.21.15.1:0/1668558756 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55872e242d20 con 0x55872e360240 2024-09-16T00:36:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f081aff5640 10 client.17322 handle_command_reply: tid=0 2024-09-16T00:36:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 client.17322 shutdown 2024-09-16T00:36:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 -- 172.21.15.1:0/1668558756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 msgr2=0x55872e35e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e35e6e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f082c005e90 tx=0x7f082c04c040 comp rx=0 tx=0).stop 2024-09-16T00:36:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 -- 172.21.15.1:0/1668558756 shutdown_connections 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55872e360240 0x55872e362630 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55872e2e4d50 0x55872e35e6e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55872e2d5f60 0x55872e2e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 --2- 172.21.15.1:0/1668558756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55872e2d5640 0x55872e2e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 -- 172.21.15.1:0/1668558756 >> 172.21.15.1:0/1668558756 conn(0x55872e2d1cf0 msgr2=0x55872e2e0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f081aff5640 0 client.17322 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 -- 172.21.15.1:0/1668558756 shutdown_connections 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 1 -- 172.21.15.1:0/1668558756 wait complete. 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.082+0000 7f083f811740 20 client.17322 trim_cache size 0 max 16384 2024-09-16T00:36:18.084 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 26, 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "86ae24ec": { 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 26, 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:36:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 143.7, 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:18.086 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:36:18.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 -- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08200a3240 msgr2=0x7f08200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:18.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08200a3240 0x7f08200a5700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f083000b760 tx=0x7f0830004020 comp rx=0 tx=0).stop 2024-09-16T00:36:18.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 -- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0838106f20 msgr2=0x7f083819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:18.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0838106f20 0x7f083819cd60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0830009fd0 tx=0x7f083000b260 comp rx=0 tx=0).stop 2024-09-16T00:36:18.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 -- 172.21.15.1:0/2693326162 shutdown_connections 2024-09-16T00:36:18.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0838106f20 0x7f083819cd60 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08381027a0 0x7f08381989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08200a3240 0x7f08200a5700 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 --2- 172.21.15.1:0/2693326162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0838101df0 0x7f0838198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:18.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 -- 172.21.15.1:0/2693326162 >> 172.21.15.1:0/2693326162 conn(0x7f08380fd8f0 msgr2=0x7f08380fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:18.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 -- 172.21.15.1:0/2693326162 shutdown_connections 2024-09-16T00:36:18.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:18.083+0000 7f083d438640 1 -- 172.21.15.1:0/2693326162 wait complete. 2024-09-16T00:36:18.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:18 smithi001 ceph-mon[30934]: pgmap v587: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:18 smithi019 ceph-mon[35186]: pgmap v587: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:18 smithi153 ceph-mon[35538]: pgmap v587: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:18.954 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '86ae24ec', quiesce took 3.6 seconds. Will release after: 20.099999999999998 2024-09-16T00:36:20.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:20 smithi019 ceph-mon[35186]: pgmap v588: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:36:20.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:20 smithi001 ceph-mon[30934]: pgmap v588: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:36:20.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:20 smithi153 ceph-mon[35538]: pgmap v588: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:36:22.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:22 smithi019 ceph-mon[35186]: pgmap v589: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:22.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:22 smithi001 ceph-mon[30934]: pgmap v589: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:22 smithi153 ceph-mon[35538]: pgmap v589: 129 pgs: 129 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:36:24.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:24 smithi019 ceph-mon[35186]: pgmap v590: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:36:24.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:24 smithi001 ceph-mon[30934]: pgmap v590: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:36:24.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:24 smithi153 ceph-mon[35538]: pgmap v590: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:36:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:25.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:36:25.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:25.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:36:25.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:25.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:25.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:25.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:25.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:36:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:26 smithi019 ceph-mon[35186]: pgmap v591: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 8 op/s 2024-09-16T00:36:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:26 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-16T00:36:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:26 smithi019 ceph-mon[35186]: Unable to set osd_memory_target on smithi153 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-16T00:36:26.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:26 smithi001 ceph-mon[30934]: pgmap v591: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 8 op/s 2024-09-16T00:36:26.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:26 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-16T00:36:26.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:26 smithi001 ceph-mon[30934]: Unable to set osd_memory_target on smithi153 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-16T00:36:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:26 smithi153 ceph-mon[35538]: pgmap v591: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 8 op/s 2024-09-16T00:36:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:26 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-16T00:36:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:26 smithi153 ceph-mon[35538]: Unable to set osd_memory_target on smithi153 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-16T00:36:28.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:28 smithi001 ceph-mon[30934]: pgmap v592: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-16T00:36:28.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:28 smithi019 ceph-mon[35186]: pgmap v592: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-16T00:36:28.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:28 smithi153 ceph-mon[35538]: pgmap v592: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-16T00:36:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:30 smithi019 ceph-mon[35186]: pgmap v593: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-16T00:36:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:30 smithi001 ceph-mon[30934]: pgmap v593: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-16T00:36:30.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:30 smithi153 ceph-mon[35538]: pgmap v593: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-16T00:36:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:32 smithi019 ceph-mon[35186]: pgmap v594: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:36:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:32 smithi001 ceph-mon[30934]: pgmap v594: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:36:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:32 smithi153 ceph-mon[35538]: pgmap v594: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:36:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:34 smithi019 ceph-mon[35186]: pgmap v595: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-16T00:36:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:34 smithi001 ceph-mon[30934]: pgmap v595: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-16T00:36:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:34 smithi153 ceph-mon[35538]: pgmap v595: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-16T00:36:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:36 smithi019 ceph-mon[35186]: pgmap v596: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-16T00:36:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:36 smithi001 ceph-mon[30934]: pgmap v596: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-16T00:36:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:36 smithi153 ceph-mon[35538]: pgmap v596: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-16T00:36:38.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:38 smithi019 ceph-mon[35186]: pgmap v597: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:36:38.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:38 smithi001 ceph-mon[30934]: pgmap v597: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:36:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:38 smithi153 ceph-mon[35538]: pgmap v597: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:36:39.055 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:36:39.328 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:39.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.822+0000 7f262b2da640 1 -- 172.21.15.1:0/1217401112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624102dd0 msgr2=0x7f26241031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:39.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.822+0000 7f262b2da640 1 --2- 172.21.15.1:0/1217401112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624102dd0 0x7f26241031b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2618009930 tx=0x7f261802a420 comp rx=0 tx=0).stop 2024-09-16T00:36:39.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.822+0000 7f262b2da640 1 -- 172.21.15.1:0/1217401112 shutdown_connections 2024-09-16T00:36:39.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.822+0000 7f262b2da640 1 --2- 172.21.15.1:0/1217401112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624108300 0x7f262410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:39.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.822+0000 7f262b2da640 1 --2- 172.21.15.1:0/1217401112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624103780 0x7f2624107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:39.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.822+0000 7f262b2da640 1 --2- 172.21.15.1:0/1217401112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624102dd0 0x7f26241031b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.822+0000 7f262b2da640 1 -- 172.21.15.1:0/1217401112 >> 172.21.15.1:0/1217401112 conn(0x7f26240fc7d0 msgr2=0x7f26240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.822+0000 7f262b2da640 1 -- 172.21.15.1:0/1217401112 shutdown_connections 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.822+0000 7f262b2da640 1 -- 172.21.15.1:0/1217401112 wait complete. 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f262b2da640 1 Processor -- start 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f262b2da640 1 -- start start 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f262b2da640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624102dd0 0x7f26241a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f262b2da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624103780 0x7f26241a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f262b2da640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624108300 0x7f262419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f262b2da640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f26241120b0 con 0x7f2624108300 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f262b2da640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2624111f30 con 0x7f2624103780 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f262b2da640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2624112230 con 0x7f2624102dd0 2024-09-16T00:36:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f2629877640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624108300 0x7f262419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f2629877640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624108300 0x7f262419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55848/0 (socket says 172.21.15.1:55848) 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f2629877640 1 -- 172.21.15.1:0/3640682985 learned_addr learned my addr 172.21.15.1:0/3640682985 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.823+0000 7f2628875640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624103780 0x7f26241a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f2629076640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624102dd0 0x7f26241a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f2629877640 1 -- 172.21.15.1:0/3640682985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624102dd0 msgr2=0x7f26241a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f2629877640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624102dd0 0x7f26241a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f2629877640 1 -- 172.21.15.1:0/3640682985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624103780 msgr2=0x7f26241a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f2629877640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624103780 0x7f26241a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f2629877640 1 -- 172.21.15.1:0/3640682985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262419aef0 con 0x7f2624108300 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f2629076640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624102dd0 0x7f26241a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f2628875640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624103780 0x7f26241a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f2629877640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624108300 0x7f262419a680 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f261c00eaa0 tx=0x7f261c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f26167fc640 1 -- 172.21.15.1:0/3640682985 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c00cdb0 con 0x7f2624108300 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f26167fc640 1 -- 172.21.15.1:0/3640682985 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f261c005020 con 0x7f2624108300 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262419b1e0 con 0x7f2624108300 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.824+0000 7f26167fc640 1 -- 172.21.15.1:0/3640682985 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c0116b0 con 0x7f2624108300 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.825+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26241a7530 con 0x7f2624108300 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.826+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262406a7b0 con 0x7f2624108300 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.826+0000 7f26167fc640 1 -- 172.21.15.1:0/3640682985 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f261c005560 con 0x7f2624108300 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.827+0000 7f26167fc640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26100a32c0 0x7f26100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.827+0000 7f2629076640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26100a32c0 0x7f26100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.828+0000 7f26167fc640 1 -- 172.21.15.1:0/3640682985 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f261c0cfb10 con 0x7f2624108300 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.828+0000 7f2629076640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26100a32c0 0x7f26100a5780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2618000c00 tx=0x7f2618005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:39.832+0000 7f26167fc640 1 -- 172.21.15.1:0/3640682985 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f261c016030 con 0x7f2624108300 2024-09-16T00:36:40.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.046+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f262419bbb0 con 0x7f2624108300 2024-09-16T00:36:40.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.047+0000 7f26167fc640 1 -- 172.21.15.1:0/3640682985 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f261c09c3d0 con 0x7f2624108300 2024-09-16T00:36:40.048 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:36:40.048 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:36:40.048 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:36:40.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.050+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26100a32c0 msgr2=0x7f26100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:40.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.050+0000 7f262b2da640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26100a32c0 0x7f26100a5780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2618000c00 tx=0x7f2618005a20 comp rx=0 tx=0).stop 2024-09-16T00:36:40.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.050+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624108300 msgr2=0x7f262419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:40.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.050+0000 7f262b2da640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624108300 0x7f262419a680 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f261c00eaa0 tx=0x7f261c00ef60 comp rx=0 tx=0).stop 2024-09-16T00:36:40.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.050+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 shutdown_connections 2024-09-16T00:36:40.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.050+0000 7f262b2da640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2624108300 0x7f262419a680 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:40.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.050+0000 7f262b2da640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624103780 0x7f26241a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:40.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.050+0000 7f262b2da640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26100a32c0 0x7f26100a5780 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:40.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.050+0000 7f262b2da640 1 --2- 172.21.15.1:0/3640682985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624102dd0 0x7f26241a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:40.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.051+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 >> 172.21.15.1:0/3640682985 conn(0x7f26240fc7d0 msgr2=0x7f2624100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:40.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.051+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 shutdown_connections 2024-09-16T00:36:40.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:40.051+0000 7f262b2da640 1 -- 172.21.15.1:0/3640682985 wait complete. 2024-09-16T00:36:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:40 smithi019 ceph-mon[35186]: pgmap v598: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:36:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:40 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3640682985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:40.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:40 smithi001 ceph-mon[30934]: pgmap v598: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:36:40.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:40 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3640682985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:40 smithi153 ceph-mon[35538]: pgmap v598: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:36:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:40 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3640682985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:40.891 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:36:40.907 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 86ae24ec --release --await' 2024-09-16T00:36:40.908 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 86ae24ec --release --await 2024-09-16T00:36:41.175 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:41.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 -- 172.21.15.1:0/1754498956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5681008d0 msgr2=0x7fa568100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:41.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1754498956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5681008d0 0x7fa568100cb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa564009900 tx=0x7fa56402a410 comp rx=0 tx=0).stop 2024-09-16T00:36:41.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 -- 172.21.15.1:0/1754498956 shutdown_connections 2024-09-16T00:36:41.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1754498956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa56810e480 0x7fa568110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1754498956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa568101280 0x7fa56810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1754498956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5681008d0 0x7fa568100cb0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 -- 172.21.15.1:0/1754498956 >> 172.21.15.1:0/1754498956 conn(0x7fa5680fc7a0 msgr2=0x7fa5680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:41.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 -- 172.21.15.1:0/1754498956 shutdown_connections 2024-09-16T00:36:41.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 -- 172.21.15.1:0/1754498956 wait complete. 2024-09-16T00:36:41.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 Processor -- start 2024-09-16T00:36:41.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.689+0000 7fa5700d5640 1 -- start start 2024-09-16T00:36:41.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa5700d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5681008d0 0x7fa56819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa5700d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa568101280 0x7fa56819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa5700d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa56810e480 0x7fa56819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa5700d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681141b0 con 0x7fa56810e480 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa5700d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa568114030 con 0x7fa5681008d0 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa5700d5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa568114330 con 0x7fa568101280 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56e672640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa56810e480 0x7fa56819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56e672640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa56810e480 0x7fa56819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54924/0 (socket says 172.21.15.1:54924) 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56e672640 1 -- 172.21.15.1:0/1299548861 learned_addr learned my addr 172.21.15.1:0/1299548861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56de71640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5681008d0 0x7fa56819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56d670640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa568101280 0x7fa56819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56e672640 1 -- 172.21.15.1:0/1299548861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa568101280 msgr2=0x7fa56819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56e672640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa568101280 0x7fa56819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56e672640 1 -- 172.21.15.1:0/1299548861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5681008d0 msgr2=0x7fa56819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56e672640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5681008d0 0x7fa56819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56e672640 1 -- 172.21.15.1:0/1299548861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56819f680 con 0x7fa56810e480 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa56e672640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa56810e480 0x7fa56819efa0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fa560009fd0 tx=0x7fa56000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa55effd640 1 -- 172.21.15.1:0/1299548861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560004d20 con 0x7fa56810e480 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa55effd640 1 -- 172.21.15.1:0/1299548861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa560005020 con 0x7fa56810e480 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa5700d5640 1 -- 172.21.15.1:0/1299548861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56819f970 con 0x7fa56810e480 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.690+0000 7fa55effd640 1 -- 172.21.15.1:0/1299548861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5600116d0 con 0x7fa56810e480 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.691+0000 7fa5700d5640 1 -- 172.21.15.1:0/1299548861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5681a7250 con 0x7fa56810e480 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.693+0000 7fa55effd640 1 -- 172.21.15.1:0/1299548861 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa5600051d0 con 0x7fa56810e480 2024-09-16T00:36:41.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa5724ae740 1 Processor -- start 2024-09-16T00:36:41.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa5724ae740 1 -- start start 2024-09-16T00:36:41.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 0x5644c890c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 0x5644c8917140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8919a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa5724ae740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5644c88722e0 con 0x5644c8917680 2024-09-16T00:36:41.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa5724ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5644c8819ce0 con 0x5644c890c4a0 2024-09-16T00:36:41.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa5724ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5644c886e2a0 con 0x5644c890cdc0 2024-09-16T00:36:41.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa56d670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8919a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa56d670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8919a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54932/0 (socket says 172.21.15.1:54932) 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa56d670640 1 -- 172.21.15.1:0/380442065 learned_addr learned my addr 172.21.15.1:0/380442065 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa56de71640 1 --2- 172.21.15.1:0/380442065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 0x5644c8917140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa55effd640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5540a32c0 0x7fa5540a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.694+0000 7fa56d670640 1 --2- 172.21.15.1:0/380442065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 0x5644c890c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa56e672640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5540a32c0 0x7fa5540a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa56d670640 1 -- 172.21.15.1:0/380442065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 msgr2=0x5644c8917140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa56d670640 1 --2- 172.21.15.1:0/380442065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 0x5644c8917140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa56d670640 1 -- 172.21.15.1:0/380442065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 msgr2=0x5644c890c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa56d670640 1 --2- 172.21.15.1:0/380442065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 0x5644c890c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa56d670640 1 -- 172.21.15.1:0/380442065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c87fe440 con 0x5644c8917680 2024-09-16T00:36:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa55effd640 1 -- 172.21.15.1:0/1299548861 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa5600d00d0 con 0x7fa56810e480 2024-09-16T00:36:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa56d670640 1 --2- 172.21.15.1:0/380442065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8919a70 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa5580125c0 tx=0x7fa55800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb3330a68eddfe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa553fff640 1 -- 172.21.15.1:0/380442065 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5580106b0 con 0x5644c8917680 2024-09-16T00:36:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa553fff640 1 -- 172.21.15.1:0/380442065 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa558010d30 con 0x5644c8917680 2024-09-16T00:36:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa56e672640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5540a32c0 0x7fa5540a5780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa56009c330 tx=0x7fa560091880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.695+0000 7fa553fff640 1 -- 172.21.15.1:0/380442065 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55801dd20 con 0x5644c8917680 2024-09-16T00:36:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 -- 172.21.15.1:0/380442065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 msgr2=0x5644c8919a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 --2- 172.21.15.1:0/380442065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8919a70 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa5580125c0 tx=0x7fa55800b870 comp rx=0 tx=0).stop 2024-09-16T00:36:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 -- 172.21.15.1:0/380442065 shutdown_connections 2024-09-16T00:36:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 --2- 172.21.15.1:0/380442065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8919a70 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 --2- 172.21.15.1:0/380442065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 0x5644c8917140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 --2- 172.21.15.1:0/380442065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 0x5644c890c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 -- 172.21.15.1:0/380442065 >> 172.21.15.1:0/380442065 conn(0x5644c8908a30 msgr2=0x5644c8908e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:41.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 -- 172.21.15.1:0/380442065 shutdown_connections 2024-09-16T00:36:41.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 -- 172.21.15.1:0/380442065 wait complete. 2024-09-16T00:36:41.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 -- start start 2024-09-16T00:36:41.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.696+0000 7fa5724ae740 1 Processor -- start 2024-09-16T00:36:41.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 0x5644c8917010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 0x5644c89142c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8914840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa5724ae740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5644c891a250 con 0x5644c8917680 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa5724ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5644c882d050 con 0x5644c890c4a0 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa5724ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5644c891a3d0 con 0x5644c890cdc0 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa5724ae740 10 client.0 _ll_register_callbacks cb 0x5644c886d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8914840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8914840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54942/0 (socket says 172.21.15.1:54942) 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 -- 172.21.15.1:0/1169982501 learned_addr learned my addr 172.21.15.1:0/1169982501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 0x5644c8917010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56de71640 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 0x5644c89142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 -- 172.21.15.1:0/1169982501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 msgr2=0x5644c89142c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 0x5644c89142c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 -- 172.21.15.1:0/1169982501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 msgr2=0x5644c8917010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 0x5644c8917010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 -- 172.21.15.1:0/1169982501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c85e33c0 con 0x5644c8917680 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56de71640 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 0x5644c89142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.697+0000 7fa56d670640 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8914840 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa558001eb0 tx=0x7fa55801d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd9bcc8b88c2dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5517fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5517fa640 1 -- 172.21.15.1:0/1169982501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558031060 con 0x5644c8917680 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5517fa640 1 -- 172.21.15.1:0/1169982501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa558005050 con 0x5644c8917680 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5724ae740 1 -- 172.21.15.1:0/1169982501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa558010d30 con 0x5644c8917680 2024-09-16T00:36:41.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5517fa640 1 -- 172.21.15.1:0/1169982501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa558038050 con 0x5644c8917680 2024-09-16T00:36:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5724ae740 10 client.17340 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5724ae740 1 -- 172.21.15.1:0/1169982501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644c8925c40 con 0x5644c8917680 2024-09-16T00:36:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5517fa640 1 -- 172.21.15.1:0/1169982501 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa558010d30 con 0x5644c8917680 2024-09-16T00:36:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5517fa640 1 -- 172.21.15.1:0/1169982501 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa55801c070 con 0x5644c8917680 2024-09-16T00:36:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5724ae740 10 client.17340 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5724ae740 10 client.17340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5724ae740 20 client.17340 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5724ae740 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5644c8927100 0x5644c89294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5724ae740 4 client.17340 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.698+0000 7fa5724ae740 1 -- 172.21.15.1:0/1169982501 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644c887f850 con 0x5644c8927100 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.699+0000 7fa56de71640 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5644c8927100 0x5644c89294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.699+0000 7fa5517fa640 10 client.17340 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.699+0000 7fa56de71640 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5644c8927100 0x5644c89294f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33fe7ca8415afad7 server_cookie=ffbc3d3d3fc404bc in_seq=0 out_seq=0 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5517fa640 1 -- 172.21.15.1:0/1169982501 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644c887f850 con 0x5644c8927100 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5517fa640 10 client.17340 handle_command_reply: tid=0 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 client.17340 shutdown 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 -- 172.21.15.1:0/1169982501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 msgr2=0x5644c8914840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8914840 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa558001eb0 tx=0x7fa55801d8c0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 -- 172.21.15.1:0/1169982501 shutdown_connections 2024-09-16T00:36:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5644c8927100 0x5644c89294f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c8914840 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890cdc0 0x5644c89142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5517fa640 0 client.17340 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 --2- 172.21.15.1:0/1169982501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890c4a0 0x5644c8917010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 -- 172.21.15.1:0/1169982501 >> 172.21.15.1:0/1169982501 conn(0x5644c8908a30 msgr2=0x5644c890e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 -- 172.21.15.1:0/1169982501 shutdown_connections 2024-09-16T00:36:41.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 1 -- 172.21.15.1:0/1169982501 wait complete. 2024-09-16T00:36:41.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.700+0000 7fa5724ae740 20 client.17340 trim_cache size 0 max 16384 2024-09-16T00:36:41.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.717+0000 7fa5724ae740 1 Processor -- start 2024-09-16T00:36:41.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa5724ae740 1 -- start start 2024-09-16T00:36:41.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890c4a0 0x5644c89234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 0x5644c892ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c892f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa5724ae740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681b39e0 con 0x5644c8917680 2024-09-16T00:36:41.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa5724ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681b12b0 con 0x5644c890cdc0 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa5724ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681af100 con 0x5644c890c4a0 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c892f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c892f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54948/0 (socket says 172.21.15.1:54948) 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 -- 172.21.15.1:0/4192696455 learned_addr learned my addr 172.21.15.1:0/4192696455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56de71640 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 0x5644c892ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890c4a0 0x5644c89234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 -- 172.21.15.1:0/4192696455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890c4a0 msgr2=0x5644c89234b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890c4a0 0x5644c89234b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 -- 172.21.15.1:0/4192696455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 msgr2=0x5644c892ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 0x5644c892ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 -- 172.21.15.1:0/4192696455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c89096c0 con 0x5644c8917680 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56de71640 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 0x5644c892ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:41.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.718+0000 7fa56d670640 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c892f150 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa558010bd0 tx=0x7fa5580047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ca0f3d3c37a8f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5537fe640 1 -- 172.21.15.1:0/4192696455 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558023040 con 0x5644c8917680 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5537fe640 1 -- 172.21.15.1:0/4192696455 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa55800b410 con 0x5644c8917680 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5537fe640 1 -- 172.21.15.1:0/4192696455 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558010640 con 0x5644c8917680 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5724ae740 1 -- 172.21.15.1:0/4192696455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 msgr2=0x5644c892f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5724ae740 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c892f150 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa558010bd0 tx=0x7fa5580047d0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5724ae740 1 -- 172.21.15.1:0/4192696455 shutdown_connections 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5724ae740 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c8917680 0x5644c892f150 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5724ae740 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 0x5644c892ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5724ae740 1 --2- 172.21.15.1:0/4192696455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c890c4a0 0x5644c89234b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5724ae740 1 -- 172.21.15.1:0/4192696455 >> 172.21.15.1:0/4192696455 conn(0x5644c8908a30 msgr2=0x5644c8916340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5724ae740 1 -- 172.21.15.1:0/4192696455 shutdown_connections 2024-09-16T00:36:41.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.719+0000 7fa5724ae740 1 -- 172.21.15.1:0/4192696455 wait complete. 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa5724ae740 1 -- start start 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa5724ae740 1 Processor -- start 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c890c4a0 0x5644c891fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 0x5644c891b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa5724ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c8917680 0x5644c891b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa5724ae740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681af770 con 0x5644c890c4a0 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa5724ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681b0be0 con 0x5644c890cdc0 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa5724ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681afe40 con 0x5644c8917680 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa5724ae740 10 client.0 _ll_register_callbacks cb 0x7fa564005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa56d670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c890c4a0 0x5644c891fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa56d670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c890c4a0 0x5644c891fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54950/0 (socket says 172.21.15.1:54950) 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.720+0000 7fa56d670640 1 -- 172.21.15.1:0/3128908111 learned_addr learned my addr 172.21.15.1:0/3128908111 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa56de71640 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 0x5644c891b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa56d670640 1 -- 172.21.15.1:0/3128908111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c8917680 msgr2=0x5644c891b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa56d670640 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c8917680 0x5644c891b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa56d670640 1 -- 172.21.15.1:0/3128908111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 msgr2=0x5644c891b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa56d670640 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 0x5644c891b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa56d670640 1 -- 172.21.15.1:0/3128908111 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c891bd60 con 0x5644c890c4a0 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa56d670640 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c890c4a0 0x5644c891fd10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa558010e70 tx=0x7fa5580124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b2c67b694201fe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa5517fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa5517fa640 1 -- 172.21.15.1:0/3128908111 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558042070 con 0x5644c890c4a0 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa5517fa640 1 -- 172.21.15.1:0/3128908111 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa558010800 con 0x5644c890c4a0 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa5724ae740 1 -- 172.21.15.1:0/3128908111 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa55800b410 con 0x5644c890c4a0 2024-09-16T00:36:41.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa5517fa640 1 -- 172.21.15.1:0/3128908111 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa55804e050 con 0x5644c890c4a0 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa5724ae740 10 client.17346 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.721+0000 7fa5724ae740 1 -- 172.21.15.1:0/3128908111 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644c8991ba0 con 0x5644c890c4a0 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa5517fa640 1 -- 172.21.15.1:0/3128908111 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa55800b410 con 0x5644c890c4a0 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa5517fa640 1 -- 172.21.15.1:0/3128908111 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa55801c070 con 0x5644c890c4a0 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa5724ae740 10 client.17346 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa5724ae740 10 client.17346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa5724ae740 20 client.17346 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa5724ae740 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5644c8992200 0x5644c89945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa5724ae740 4 client.17346 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa5724ae740 1 -- 172.21.15.1:0/3128908111 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644c8879d60 con 0x5644c8992200 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa56de71640 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5644c8992200 0x5644c89945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa56de71640 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5644c8992200 0x5644c89945f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56552f7b0bed09b5 server_cookie=ebda182dc76e09b6 in_seq=0 out_seq=0 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.722+0000 7fa5517fa640 10 client.17346 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5517fa640 1 -- 172.21.15.1:0/3128908111 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5644c8879d60 con 0x5644c8992200 2024-09-16T00:36:41.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5517fa640 10 client.17346 handle_command_reply: tid=0 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 client.17346 shutdown 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 -- 172.21.15.1:0/3128908111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c890c4a0 msgr2=0x5644c891fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c890c4a0 0x5644c891fd10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa558010e70 tx=0x7fa5580124f0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 -- 172.21.15.1:0/3128908111 shutdown_connections 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5644c8992200 0x5644c89945f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644c8917680 0x5644c891b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5517fa640 0 client.17346 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644c890cdc0 0x5644c891b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 --2- 172.21.15.1:0/3128908111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644c890c4a0 0x5644c891fd10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 -- 172.21.15.1:0/3128908111 >> 172.21.15.1:0/3128908111 conn(0x5644c8908a30 msgr2=0x5644c8912a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 -- 172.21.15.1:0/3128908111 shutdown_connections 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 1 -- 172.21.15.1:0/3128908111 wait complete. 2024-09-16T00:36:41.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.723+0000 7fa5724ae740 20 client.17346 trim_cache size 0 max 16384 2024-09-16T00:36:41.733 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:36:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:36:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:36:41.733 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 -- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5540a32c0 msgr2=0x7fa5540a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5540a32c0 0x7fa5540a5780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa56009c330 tx=0x7fa560091880 comp rx=0 tx=0).stop 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 -- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa56810e480 msgr2=0x7fa56819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa56810e480 0x7fa56819efa0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fa560009fd0 tx=0x7fa56000ee60 comp rx=0 tx=0).stop 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 -- 172.21.15.1:0/1299548861 shutdown_connections 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa56810e480 0x7fa56819efa0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5540a32c0 0x7fa5540a5780 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa568101280 0x7fa56819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 --2- 172.21.15.1:0/1299548861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5681008d0 0x7fa56819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 -- 172.21.15.1:0/1299548861 >> 172.21.15.1:0/1299548861 conn(0x7fa5680fc7a0 msgr2=0x7fa5680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:41.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 -- 172.21.15.1:0/1299548861 shutdown_connections 2024-09-16T00:36:41.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:41.724+0000 7fa5700d5640 1 -- 172.21.15.1:0/1299548861 wait complete. 2024-09-16T00:36:42.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:42 smithi001 ceph-mon[30934]: pgmap v599: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:36:42.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:42 smithi019 ceph-mon[35186]: pgmap v599: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:36:42.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:42 smithi153 ceph-mon[35538]: pgmap v599: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T00:36:42.749 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:36:42.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:42.999+0000 7f36e1da0640 1 -- 172.21.15.1:0/3040949540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 msgr2=0x7f36dc106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:42.999+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3040949540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 0x7f36dc106cb0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009930 tx=0x7f36cc02a420 comp rx=0 tx=0).stop 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:42.999+0000 7f36e1da0640 1 -- 172.21.15.1:0/3040949540 shutdown_connections 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:42.999+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3040949540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36dc101270 0x7f36dc10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:42.999+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3040949540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36dc1008d0 0x7f36dc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:42.999+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3040949540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 0x7f36dc106cb0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:42.999+0000 7f36e1da0640 1 -- 172.21.15.1:0/3040949540 >> 172.21.15.1:0/3040949540 conn(0x7f36dc0fc630 msgr2=0x7f36dc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:42.999+0000 7f36e1da0640 1 -- 172.21.15.1:0/3040949540 shutdown_connections 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:42.999+0000 7f36e1da0640 1 -- 172.21.15.1:0/3040949540 wait complete. 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e1da0640 1 Processor -- start 2024-09-16T00:36:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e1da0640 1 -- start start 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e1da0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36dc1008d0 0x7f36dc198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e1da0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36dc101270 0x7f36dc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e1da0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 0x7f36dc19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e1da0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc111db0 con 0x7f36dc1068d0 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e1da0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc111c30 con 0x7f36dc1008d0 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e1da0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc111f30 con 0x7f36dc101270 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e159f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 0x7f36dc19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e159f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 0x7f36dc19cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54976/0 (socket says 172.21.15.1:54976) 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e159f640 1 -- 172.21.15.1:0/3471317663 learned_addr learned my addr 172.21.15.1:0/3471317663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36e0d9e640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36dc1008d0 0x7f36dc198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.000+0000 7f36dbfff640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36dc101270 0x7f36dc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36e159f640 1 -- 172.21.15.1:0/3471317663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36dc101270 msgr2=0x7f36dc198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36e159f640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36dc101270 0x7f36dc198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36e159f640 1 -- 172.21.15.1:0/3471317663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36dc1008d0 msgr2=0x7f36dc198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36e159f640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36dc1008d0 0x7f36dc198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36e159f640 1 -- 172.21.15.1:0/3471317663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc19d200 con 0x7f36dc1068d0 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36e159f640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 0x7f36dc19cb20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f36d400eaa0 tx=0x7f36d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36d9ffb640 1 -- 172.21.15.1:0/3471317663 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d400cdb0 con 0x7f36dc1068d0 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36d9ffb640 1 -- 172.21.15.1:0/3471317663 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d4005020 con 0x7f36dc1068d0 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36d9ffb640 1 -- 172.21.15.1:0/3471317663 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40116b0 con 0x7f36dc1068d0 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36e1da0640 1 -- 172.21.15.1:0/3471317663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36dc19d4f0 con 0x7f36dc1068d0 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.001+0000 7f36e1da0640 1 -- 172.21.15.1:0/3471317663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36dc1a4dd0 con 0x7f36dc1068d0 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.003+0000 7f36d9ffb640 1 -- 172.21.15.1:0/3471317663 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f36d4005560 con 0x7f36dc1068d0 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36e53db740 1 Processor -- start 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36e53db740 1 -- start start 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36d9ffb640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c40a3240 0x7f36c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36e53db740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212210 0x557e5d2125f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36e53db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212b30 0x557e5d21ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36e53db740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d21f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36e53db740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557e5d177620 con 0x557e5d21d3f0 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36e53db740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e5d11ee50 con 0x557e5d212210 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36e53db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e5d1735e0 con 0x557e5d212b30 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.004+0000 7f36dbfff640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c40a3240 0x7f36c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e159f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d21f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e159f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d21f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54990/0 (socket says 172.21.15.1:54990) 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e159f640 1 -- 172.21.15.1:0/3030528693 learned_addr learned my addr 172.21.15.1:0/3030528693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36d9ffb640 1 -- 172.21.15.1:0/3471317663 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f36d40cff60 con 0x7f36dc1068d0 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e0d9e640 1 --2- 172.21.15.1:0/3030528693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212210 0x557e5d2125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36dbfff640 1 --2- 172.21.15.1:0/3030528693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212b30 0x557e5d21ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e159f640 1 -- 172.21.15.1:0/3030528693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212b30 msgr2=0x557e5d21ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e159f640 1 --2- 172.21.15.1:0/3030528693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212b30 0x557e5d21ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e159f640 1 -- 172.21.15.1:0/3030528693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212210 msgr2=0x557e5d2125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e159f640 1 --2- 172.21.15.1:0/3030528693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212210 0x557e5d2125f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e159f640 1 -- 172.21.15.1:0/3030528693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e5d10fd40 con 0x557e5d21d3f0 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36dbfff640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c40a3240 0x7f36c40a5700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0fec20 tx=0x7f36d00127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36e159f640 1 --2- 172.21.15.1:0/3030528693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d21f7e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f36d40092f0 tx=0x7f36d409bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efcb8e3a7ac3a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36c2ffd640 1 -- 172.21.15.1:0/3030528693 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d401f070 con 0x557e5d21d3f0 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36c2ffd640 1 -- 172.21.15.1:0/3030528693 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d409c120 con 0x557e5d21d3f0 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.005+0000 7f36c2ffd640 1 -- 172.21.15.1:0/3030528693 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40a0160 con 0x557e5d21d3f0 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 -- 172.21.15.1:0/3030528693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 msgr2=0x557e5d21f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 --2- 172.21.15.1:0/3030528693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d21f7e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f36d40092f0 tx=0x7f36d409bb00 comp rx=0 tx=0).stop 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 -- 172.21.15.1:0/3030528693 shutdown_connections 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 --2- 172.21.15.1:0/3030528693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d21f7e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 --2- 172.21.15.1:0/3030528693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212b30 0x557e5d21ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 --2- 172.21.15.1:0/3030528693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212210 0x557e5d2125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 -- 172.21.15.1:0/3030528693 >> 172.21.15.1:0/3030528693 conn(0x557e5d20e7a0 msgr2=0x557e5d20eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 -- 172.21.15.1:0/3030528693 shutdown_connections 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 -- 172.21.15.1:0/3030528693 wait complete. 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.006+0000 7f36e53db740 1 -- start start 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e53db740 1 Processor -- start 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e53db740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d21a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e53db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212b30 0x557e5d21a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e53db740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d21d3f0 0x557e5d21ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e53db740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557e5d21ffc0 con 0x557e5d212210 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e53db740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e5d0fff10 con 0x557e5d21d3f0 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e53db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e5d220140 con 0x557e5d212b30 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e53db740 10 client.0 _ll_register_callbacks cb 0x557e5d123db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e0d9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d21a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e0d9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d21a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54996/0 (socket says 172.21.15.1:54996) 2024-09-16T00:36:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e0d9e640 1 -- 172.21.15.1:0/2596695504 learned_addr learned my addr 172.21.15.1:0/2596695504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e0d9e640 1 -- 172.21.15.1:0/2596695504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212b30 msgr2=0x557e5d21a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e0d9e640 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212b30 0x557e5d21a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e0d9e640 1 -- 172.21.15.1:0/2596695504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d21d3f0 msgr2=0x557e5d21ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e0d9e640 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d21d3f0 0x557e5d21ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.007+0000 7f36e0d9e640 1 -- 172.21.15.1:0/2596695504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e5ce86580 con 0x557e5d212210 2024-09-16T00:36:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36e0d9e640 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d21a290 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f36cc02a940 tx=0x7f36cc0319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3faf4e9f218f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36bbfff640 1 -- 172.21.15.1:0/2596695504 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc035040 con 0x557e5d212210 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36bbfff640 1 -- 172.21.15.1:0/2596695504 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f36cc005020 con 0x557e5d212210 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36e53db740 1 -- 172.21.15.1:0/2596695504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d409c120 con 0x557e5d212210 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36bbfff640 1 -- 172.21.15.1:0/2596695504 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36cc04e050 con 0x557e5d212210 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36e53db740 10 client.17358 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36e53db740 1 -- 172.21.15.1:0/2596695504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e5cf50960 con 0x557e5d212210 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36bbfff640 1 -- 172.21.15.1:0/2596695504 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d409c120 con 0x557e5d212210 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36bbfff640 1 -- 172.21.15.1:0/2596695504 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f36cc0035f0 con 0x557e5d212210 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36e53db740 10 client.17358 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.008+0000 7f36e53db740 10 client.17358 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.009+0000 7f36e53db740 20 client.17358 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.009+0000 7f36e53db740 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557e5d22cf60 0x557e5d22f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.009+0000 7f36e53db740 4 client.17358 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.009+0000 7f36e53db740 1 -- 172.21.15.1:0/2596695504 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e5d184b90 con 0x557e5d22cf60 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.009+0000 7f36dbfff640 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557e5d22cf60 0x557e5d22f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.009+0000 7f36bbfff640 10 client.17358 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.009+0000 7f36dbfff640 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557e5d22cf60 0x557e5d22f420 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3be186f9b76d3e1 server_cookie=1777375fc9cc7b79 in_seq=0 out_seq=0 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36bbfff640 1 -- 172.21.15.1:0/2596695504 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e5d184b90 con 0x557e5d22cf60 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36bbfff640 10 client.17358 handle_command_reply: tid=0 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36e53db740 1 client.17358 shutdown 2024-09-16T00:36:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36e53db740 1 -- 172.21.15.1:0/2596695504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 msgr2=0x557e5d21a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36e53db740 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d21a290 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f36cc02a940 tx=0x7f36cc0319b0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36e53db740 1 -- 172.21.15.1:0/2596695504 shutdown_connections 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36e53db740 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557e5d22cf60 0x557e5d22f420 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36e53db740 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d21d3f0 0x557e5d21ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36e53db740 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212b30 0x557e5d21a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36e53db740 1 --2- 172.21.15.1:0/2596695504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d21a290 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36e53db740 1 -- 172.21.15.1:0/2596695504 >> 172.21.15.1:0/2596695504 conn(0x557e5d20e7a0 msgr2=0x557e5d2144d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.010+0000 7f36bbfff640 0 client.17358 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.011+0000 7f36e53db740 1 -- 172.21.15.1:0/2596695504 shutdown_connections 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.011+0000 7f36e53db740 1 -- 172.21.15.1:0/2596695504 wait complete. 2024-09-16T00:36:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.011+0000 7f36e53db740 20 client.17358 trim_cache size 0 max 16384 2024-09-16T00:36:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e53db740 1 Processor -- start 2024-09-16T00:36:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e53db740 1 -- start start 2024-09-16T00:36:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e53db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212210 0x557e5d214370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e53db740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 0x557e5d234cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e53db740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d235230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e53db740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1b15a0 con 0x557e5d21d3f0 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e53db740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1aee70 con 0x557e5d212b30 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e53db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1accc0 con 0x557e5d212210 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e159f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d235230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e159f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d235230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55004/0 (socket says 172.21.15.1:55004) 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e159f640 1 -- 172.21.15.1:0/442898751 learned_addr learned my addr 172.21.15.1:0/442898751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36e0d9e640 1 --2- 172.21.15.1:0/442898751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212210 0x557e5d214370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.028+0000 7f36dbfff640 1 --2- 172.21.15.1:0/442898751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 0x557e5d234cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.029+0000 7f36e159f640 1 -- 172.21.15.1:0/442898751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212210 msgr2=0x557e5d214370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.029+0000 7f36e159f640 1 --2- 172.21.15.1:0/442898751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212210 0x557e5d214370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.029+0000 7f36e159f640 1 -- 172.21.15.1:0/442898751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 msgr2=0x557e5d234cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.029+0000 7f36e159f640 1 --2- 172.21.15.1:0/442898751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 0x557e5d234cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.029+0000 7f36e159f640 1 -- 172.21.15.1:0/442898751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e5d20e5d0 con 0x557e5d21d3f0 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.029+0000 7f36e159f640 1 --2- 172.21.15.1:0/442898751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d235230 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f36d4098af0 tx=0x7f36d409c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639e1b263399f969 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.029+0000 7f36c27fc640 1 -- 172.21.15.1:0/442898751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d401f040 con 0x557e5d21d3f0 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.029+0000 7f36c27fc640 1 -- 172.21.15.1:0/442898751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d40b5070 con 0x557e5d21d3f0 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.029+0000 7f36c27fc640 1 -- 172.21.15.1:0/442898751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4098790 con 0x557e5d21d3f0 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 -- 172.21.15.1:0/442898751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 msgr2=0x557e5d235230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 --2- 172.21.15.1:0/442898751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d235230 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f36d4098af0 tx=0x7f36d409c8e0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 -- 172.21.15.1:0/442898751 shutdown_connections 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 --2- 172.21.15.1:0/442898751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d21d3f0 0x557e5d235230 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 --2- 172.21.15.1:0/442898751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 0x557e5d234cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 --2- 172.21.15.1:0/442898751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d212210 0x557e5d214370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 -- 172.21.15.1:0/442898751 >> 172.21.15.1:0/442898751 conn(0x557e5d20e7a0 msgr2=0x557e5d213c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 -- 172.21.15.1:0/442898751 shutdown_connections 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 -- 172.21.15.1:0/442898751 wait complete. 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.030+0000 7f36e53db740 1 -- start start 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e53db740 1 Processor -- start 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e53db740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d225390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e53db740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 0x557e5d2258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e53db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d21d3f0 0x557e5d222360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e53db740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1ad330 con 0x557e5d212210 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e53db740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1ae7a0 con 0x557e5d212b30 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e53db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1ada00 con 0x557e5d21d3f0 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e53db740 10 client.0 _ll_register_callbacks cb 0x7f36cc031640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e0d9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d225390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e0d9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d225390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55012/0 (socket says 172.21.15.1:55012) 2024-09-16T00:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e0d9e640 1 -- 172.21.15.1:0/3540523858 learned_addr learned my addr 172.21.15.1:0/3540523858 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36dbfff640 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 0x557e5d2258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e0d9e640 1 -- 172.21.15.1:0/3540523858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d21d3f0 msgr2=0x557e5d222360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e0d9e640 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d21d3f0 0x557e5d222360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e0d9e640 1 -- 172.21.15.1:0/3540523858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 msgr2=0x557e5d2258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e0d9e640 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 0x557e5d2258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.031+0000 7f36e0d9e640 1 -- 172.21.15.1:0/3540523858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e5d226010 con 0x557e5d212210 2024-09-16T00:36:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.032+0000 7f36e0d9e640 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d225390 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f36cc005940 tx=0x7f36cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bfc412c8c5c820 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.032+0000 7f36bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.032+0000 7f36bbfff640 1 -- 172.21.15.1:0/3540523858 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc035050 con 0x557e5d212210 2024-09-16T00:36:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.032+0000 7f36bbfff640 1 -- 172.21.15.1:0/3540523858 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f36cc0035f0 con 0x557e5d212210 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.032+0000 7f36e53db740 1 -- 172.21.15.1:0/3540523858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d40b5070 con 0x557e5d212210 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.032+0000 7f36bbfff640 1 -- 172.21.15.1:0/3540523858 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36cc031a20 con 0x557e5d212210 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.032+0000 7f36e53db740 10 client.17364 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.032+0000 7f36e53db740 1 -- 172.21.15.1:0/3540523858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e5d298350 con 0x557e5d212210 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.032+0000 7f36bbfff640 1 -- 172.21.15.1:0/3540523858 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d40b5070 con 0x557e5d212210 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36bbfff640 1 -- 172.21.15.1:0/3540523858 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f36cc055a80 con 0x557e5d212210 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36e53db740 10 client.17364 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36e53db740 10 client.17364 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36e53db740 20 client.17364 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36e53db740 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557e5d298a70 0x557e5d29aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36e53db740 4 client.17364 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "86ae24ec", "release": true, "await": true}] 2024-09-16T00:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36e53db740 1 -- 172.21.15.1:0/3540523858 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "86ae24ec", "release": true, "await": true}) -- 0x557e5d17f0a0 con 0x557e5d298a70 2024-09-16T00:36:43.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36dbfff640 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557e5d298a70 0x557e5d29aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:43.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36bbfff640 10 client.17364 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:43.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.033+0000 7f36dbfff640 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557e5d298a70 0x557e5d29aed0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f28965626baf69 server_cookie=d4f95c6faa6531ad in_seq=0 out_seq=0 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36bbfff640 1 -- 172.21.15.1:0/3540523858 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557e5d17f0a0 con 0x557e5d298a70 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36bbfff640 10 client.17364 handle_command_reply: tid=0 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36e53db740 1 client.17364 shutdown 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36e53db740 1 -- 172.21.15.1:0/3540523858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 msgr2=0x557e5d225390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36e53db740 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d225390 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f36cc005940 tx=0x7f36cc005fd0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36e53db740 1 -- 172.21.15.1:0/3540523858 shutdown_connections 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36e53db740 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557e5d298a70 0x557e5d29aed0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36e53db740 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e5d21d3f0 0x557e5d222360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36e53db740 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e5d212b30 0x557e5d2258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36e53db740 1 --2- 172.21.15.1:0/3540523858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e5d212210 0x557e5d225390 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36bbfff640 0 client.17364 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.036+0000 7f36e53db740 1 -- 172.21.15.1:0/3540523858 >> 172.21.15.1:0/3540523858 conn(0x557e5d20e7a0 msgr2=0x557e5d21aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:43.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e53db740 1 -- 172.21.15.1:0/3540523858 shutdown_connections 2024-09-16T00:36:43.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e53db740 1 -- 172.21.15.1:0/3540523858 wait complete. 2024-09-16T00:36:43.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e53db740 20 client.17364 trim_cache size 0 max 16384 2024-09-16T00:36:43.044 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:36:43.044 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:36:43.044 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:36:43.044 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 28, 2024-09-16T00:36:43.044 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:36:43.044 INFO:teuthology.orchestra.run.smithi001.stdout: "86ae24ec": { 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 28, 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 143.7, 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:36:43.045 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:36:43.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 -- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c40a3240 msgr2=0x7f36c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:43.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c40a3240 0x7f36c40a5700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0fec20 tx=0x7f36d00127c0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 -- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 msgr2=0x7f36dc19cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:43.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 0x7f36dc19cb20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f36d400eaa0 tx=0x7f36d400ef60 comp rx=0 tx=0).stop 2024-09-16T00:36:43.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 -- 172.21.15.1:0/3471317663 shutdown_connections 2024-09-16T00:36:43.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36dc1068d0 0x7f36dc19cb20 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c40a3240 0x7f36c40a5700 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36dc101270 0x7f36dc198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 --2- 172.21.15.1:0/3471317663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36dc1008d0 0x7f36dc198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:43.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.037+0000 7f36e1da0640 1 -- 172.21.15.1:0/3471317663 >> 172.21.15.1:0/3471317663 conn(0x7f36dc0fc630 msgr2=0x7f36dc0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:43.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.038+0000 7f36e1da0640 1 -- 172.21.15.1:0/3471317663 shutdown_connections 2024-09-16T00:36:43.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:43.038+0000 7f36e1da0640 1 -- 172.21.15.1:0/3471317663 wait complete. 2024-09-16T00:36:43.735 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '86ae24ec', total seconds elapsed: 28.4 2024-09-16T00:36:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:44.253 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:44 smithi001 ceph-mon[30934]: pgmap v600: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-16T00:36:44.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2587197348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804102790 msgr2=0x7fe8041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:44.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2587197348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804102790 0x7fe8041067e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009900 tx=0x7fe7f802a410 comp rx=0 tx=0).stop 2024-09-16T00:36:44.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2587197348 shutdown_connections 2024-09-16T00:36:44.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2587197348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe804106f10 0x7fe80410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2587197348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804102790 0x7fe8041067e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2587197348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe804101de0 0x7fe8041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2587197348 >> 172.21.15.1:0/2587197348 conn(0x7fe8040fd8c0 msgr2=0x7fe8040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2587197348 shutdown_connections 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2587197348 wait complete. 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 Processor -- start 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.335+0000 7fe80a9f1640 1 -- start start 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe80a9f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe804101de0 0x7fe804198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe80a9f1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe804102790 0x7fe804198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe80a9f1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804106f10 0x7fe80419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe80a9f1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe804111f70 con 0x7fe804106f10 2024-09-16T00:36:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe80a9f1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe804111df0 con 0x7fe804102790 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe80a9f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041120f0 con 0x7fe804101de0 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe808f8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804106f10 0x7fe80419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe808f8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804106f10 0x7fe80419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55032/0 (socket says 172.21.15.1:55032) 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe808f8e640 1 -- 172.21.15.1:0/2446415515 learned_addr learned my addr 172.21.15.1:0/2446415515 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe8037fe640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe804102790 0x7fe804198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe803fff640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe804101de0 0x7fe804198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe808f8e640 1 -- 172.21.15.1:0/2446415515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe804101de0 msgr2=0x7fe804198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe808f8e640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe804101de0 0x7fe804198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe808f8e640 1 -- 172.21.15.1:0/2446415515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe804102790 msgr2=0x7fe804198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe808f8e640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe804102790 0x7fe804198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe808f8e640 1 -- 172.21.15.1:0/2446415515 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80419d3e0 con 0x7fe804106f10 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe803fff640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe804101de0 0x7fe804198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.336+0000 7fe808f8e640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804106f10 0x7fe80419cd00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00eaa0 tx=0x7fe7fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.337+0000 7fe8017fa640 1 -- 172.21.15.1:0/2446415515 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00cdb0 con 0x7fe804106f10 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.337+0000 7fe8017fa640 1 -- 172.21.15.1:0/2446415515 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc005020 con 0x7fe804106f10 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.337+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80419d6d0 con 0x7fe804106f10 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.337+0000 7fe8017fa640 1 -- 172.21.15.1:0/2446415515 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc0116b0 con 0x7fe804106f10 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.337+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8041a4fb0 con 0x7fe804106f10 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.338+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8041021c0 con 0x7fe804106f10 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.339+0000 7fe8017fa640 1 -- 172.21.15.1:0/2446415515 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe7fc005560 con 0x7fe804106f10 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.340+0000 7fe8017fa640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7ec0a3240 0x7fe7ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.340+0000 7fe803fff640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7ec0a3240 0x7fe7ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.340+0000 7fe8017fa640 1 -- 172.21.15.1:0/2446415515 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe7fc0d0310 con 0x7fe804106f10 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.341+0000 7fe803fff640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7ec0a3240 0x7fe7ec0a5700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40058c0 tx=0x7fe7f4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.344+0000 7fe8017fa640 1 -- 172.21.15.1:0/2446415515 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7fc016030 con 0x7fe804106f10 2024-09-16T00:36:44.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:44 smithi019 ceph-mon[35186]: pgmap v600: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-16T00:36:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:44 smithi153 ceph-mon[35538]: pgmap v600: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-16T00:36:44.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.555+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe80410a150 con 0x7fe804106f10 2024-09-16T00:36:44.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.556+0000 7fe8017fa640 1 -- 172.21.15.1:0/2446415515 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fe7fc09cbd0 con 0x7fe804106f10 2024-09-16T00:36:44.556 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:36:44.557 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:36:44.557 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:36:44.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7ec0a3240 msgr2=0x7fe7ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7ec0a3240 0x7fe7ec0a5700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40058c0 tx=0x7fe7f4005d30 comp rx=0 tx=0).stop 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804106f10 msgr2=0x7fe80419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804106f10 0x7fe80419cd00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00eaa0 tx=0x7fe7fc00ef60 comp rx=0 tx=0).stop 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 shutdown_connections 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe804106f10 0x7fe80419cd00 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe804102790 0x7fe804198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7ec0a3240 0x7fe7ec0a5700 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 --2- 172.21.15.1:0/2446415515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe804101de0 0x7fe804198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 >> 172.21.15.1:0/2446415515 conn(0x7fe8040fd8c0 msgr2=0x7fe8040fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 shutdown_connections 2024-09-16T00:36:44.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:44.558+0000 7fe80a9f1640 1 -- 172.21.15.1:0/2446415515 wait complete. 2024-09-16T00:36:45.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:45 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2446415515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:45.336 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:36:45.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:45 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2446415515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:45.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:45 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2446415515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:45.601 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.099+0000 7f7ea8e32640 1 -- 172.21.15.1:0/4223105900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4102790 msgr2=0x7f7ea41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.099+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/4223105900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4102790 0x7f7ea41067e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009900 tx=0x7f7e8c02a410 comp rx=0 tx=0).stop 2024-09-16T00:36:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.099+0000 7f7ea8e32640 1 -- 172.21.15.1:0/4223105900 shutdown_connections 2024-09-16T00:36:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.099+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/4223105900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea4106f10 0x7f7ea410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.099+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/4223105900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4102790 0x7f7ea41067e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.099+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/4223105900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea4101de0 0x7f7ea41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.099+0000 7f7ea8e32640 1 -- 172.21.15.1:0/4223105900 >> 172.21.15.1:0/4223105900 conn(0x7f7ea40fd8c0 msgr2=0x7f7ea40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.099+0000 7f7ea8e32640 1 -- 172.21.15.1:0/4223105900 shutdown_connections 2024-09-16T00:36:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.099+0000 7f7ea8e32640 1 -- 172.21.15.1:0/4223105900 wait complete. 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea8e32640 1 Processor -- start 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea8e32640 1 -- start start 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea8e32640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4101de0 0x7f7ea40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea8e32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea4102790 0x7f7ea4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea8e32640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea4106f10 0x7f7ea4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea8e32640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4111fe0 con 0x7f7ea4101de0 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea8e32640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4111e60 con 0x7f7ea4102790 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea8e32640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4112160 con 0x7f7ea4106f10 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4101de0 0x7f7ea40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea4102790 0x7f7ea4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4101de0 0x7f7ea40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55046/0 (socket says 172.21.15.1:55046) 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.100+0000 7f7ea259c640 1 -- 172.21.15.1:0/815817602 learned_addr learned my addr 172.21.15.1:0/815817602 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea4106f10 0x7f7ea4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7e9bfff640 1 -- 172.21.15.1:0/815817602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea4106f10 msgr2=0x7f7ea4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea4106f10 0x7f7ea4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7e9bfff640 1 -- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4101de0 msgr2=0x7f7ea40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4101de0 0x7f7ea40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7e9bfff640 1 -- 172.21.15.1:0/815817602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea41a69d0 con 0x7f7ea4102790 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7ea259c640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4101de0 0x7f7ea40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea4106f10 0x7f7ea4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea4102790 0x7f7ea4074a30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c02a930 tx=0x7f7e8c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7e9b7fe640 1 -- 172.21.15.1:0/815817602 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c03f070 con 0x7f7ea4102790 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7ea8e32640 1 -- 172.21.15.1:0/815817602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea41a6c60 con 0x7f7ea4102790 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7e9b7fe640 1 -- 172.21.15.1:0/815817602 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c031c90 con 0x7f7ea4102790 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7e9b7fe640 1 -- 172.21.15.1:0/815817602 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c035030 con 0x7f7ea4102790 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.101+0000 7f7ea8e32640 1 -- 172.21.15.1:0/815817602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea41a7190 con 0x7f7ea4102790 2024-09-16T00:36:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.103+0000 7f7e9b7fe640 1 -- 172.21.15.1:0/815817602 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f7e8c042070 con 0x7f7ea4102790 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7eab20b740 1 Processor -- start 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7eab20b740 1 -- start start 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e1b40 0x55e8940e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e05d0 0x55e8940e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940eb370 0x55e8940ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7eab20b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e8940462f0 con 0x55e8940e05d0 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7eab20b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e893fedcf0 con 0x55e8940eb370 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7eab20b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8940422b0 con 0x55e8940e1b40 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7ea259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e05d0 0x55e8940e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7ea259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e05d0 0x55e8940e0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55060/0 (socket says 172.21.15.1:55060) 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7ea259c640 1 -- 172.21.15.1:0/3383859235 learned_addr learned my addr 172.21.15.1:0/3383859235 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7e9b7fe640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7e900a3240 0x7f7e900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/3383859235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940eb370 0x55e8940ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.104+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/3383859235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e1b40 0x55e8940e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7e900a3240 0x7f7e900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7ea259c640 1 -- 172.21.15.1:0/3383859235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e1b40 msgr2=0x55e8940e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7ea259c640 1 --2- 172.21.15.1:0/3383859235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e1b40 0x55e8940e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7ea259c640 1 -- 172.21.15.1:0/3383859235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940eb370 msgr2=0x55e8940ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7ea259c640 1 --2- 172.21.15.1:0/3383859235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940eb370 0x55e8940ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7ea259c640 1 -- 172.21.15.1:0/3383859235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e893fd23a0 con 0x55e8940e05d0 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7e9b7fe640 1 -- 172.21.15.1:0/815817602 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7e8c0f01a0 con 0x7f7ea4102790 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7ea259c640 1 --2- 172.21.15.1:0/3383859235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e05d0 0x55e8940e0a10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7e94010580 tx=0x7f7e94010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1f5af72a9f7138 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7e98ff9640 1 -- 172.21.15.1:0/3383859235 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400e890 con 0x55e8940e05d0 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7e900a3240 0x7f7e900a5700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c4f0 tx=0x7f7e9c008220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7e98ff9640 1 -- 172.21.15.1:0/3383859235 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9401c480 con 0x55e8940e05d0 2024-09-16T00:36:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.105+0000 7f7e98ff9640 1 -- 172.21.15.1:0/3383859235 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94011680 con 0x55e8940e05d0 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 -- 172.21.15.1:0/3383859235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e05d0 msgr2=0x55e8940e0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3383859235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e05d0 0x55e8940e0a10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7e94010580 tx=0x7f7e94010a40 comp rx=0 tx=0).stop 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 -- 172.21.15.1:0/3383859235 shutdown_connections 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3383859235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940eb370 0x55e8940ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3383859235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e05d0 0x55e8940e0a10 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3383859235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e1b40 0x55e8940e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 -- 172.21.15.1:0/3383859235 >> 172.21.15.1:0/3383859235 conn(0x55e8940a9310 msgr2=0x55e8940dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 -- 172.21.15.1:0/3383859235 shutdown_connections 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 -- 172.21.15.1:0/3383859235 wait complete. 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 -- start start 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.106+0000 7f7eab20b740 1 Processor -- start 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940ead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 0x55e8940e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7eab20b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e8940ee070 con 0x55e8940e1b40 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7eab20b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e894001060 con 0x55e8940e05d0 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7eab20b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8940ee1f0 con 0x55e8940eb370 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7eab20b740 10 client.0 _ll_register_callbacks cb 0x55e893d548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940e8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55068/0 (socket says 172.21.15.1:55068) 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e9bfff640 1 -- 172.21.15.1:0/2605230772 learned_addr learned my addr 172.21.15.1:0/2605230772 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7ea259c640 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 0x55e8940e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e9bfff640 1 -- 172.21.15.1:0/2605230772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 msgr2=0x55e8940e8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 0x55e8940e8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e9bfff640 1 -- 172.21.15.1:0/2605230772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 msgr2=0x55e8940ead80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940ead80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e9bfff640 1 -- 172.21.15.1:0/2605230772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e893db73c0 con 0x55e8940e1b40 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7ea259c640 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940ead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940e8030 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0ed840 tx=0x7f7e8c08b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ecc4dc7f312929 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.107+0000 7f7e89ffb640 1 -- 172.21.15.1:0/2605230772 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0bca60 con 0x55e8940e1b40 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7e89ffb640 1 -- 172.21.15.1:0/2605230772 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7e8c0c04a0 con 0x55e8940e1b40 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7eab20b740 1 -- 172.21.15.1:0/2605230772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9401c480 con 0x55e8940e1b40 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7e89ffb640 1 -- 172.21.15.1:0/2605230772 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e8c0bc360 con 0x55e8940e1b40 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7eab20b740 10 client.17382 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7eab20b740 1 -- 172.21.15.1:0/2605230772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8940f9b40 con 0x55e8940e1b40 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7e89ffb640 1 -- 172.21.15.1:0/2605230772 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9401c480 con 0x55e8940e1b40 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7e89ffb640 1 -- 172.21.15.1:0/2605230772 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7e8c04c060 con 0x55e8940e1b40 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7eab20b740 10 client.17382 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7eab20b740 10 client.17382 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:36:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7eab20b740 20 client.17382 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e8940faf80 0x55e8940fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7eab20b740 4 client.17382 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7eab20b740 1 -- 172.21.15.1:0/2605230772 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e894053860 con 0x55e8940faf80 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.108+0000 7f7ea259c640 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e8940faf80 0x55e8940fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.109+0000 7f7e89ffb640 10 client.17382 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.109+0000 7f7ea259c640 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e8940faf80 0x55e8940fd3b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3074924e2890e98c server_cookie=fb53fd0aa0c79a1e in_seq=0 out_seq=0 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7e89ffb640 1 -- 172.21.15.1:0/2605230772 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e894053860 con 0x55e8940faf80 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7e89ffb640 10 client.17382 handle_command_reply: tid=0 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 client.17382 shutdown 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 -- 172.21.15.1:0/2605230772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 msgr2=0x55e8940e8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940e8030 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0ed840 tx=0x7f7e8c08b0a0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 -- 172.21.15.1:0/2605230772 shutdown_connections 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 0x55e8940e8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940e8030 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e8940faf80 0x55e8940fd3b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2605230772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940ead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7e89ffb640 0 client.17382 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 -- 172.21.15.1:0/2605230772 >> 172.21.15.1:0/2605230772 conn(0x55e8940a9310 msgr2=0x55e8940e2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 -- 172.21.15.1:0/2605230772 shutdown_connections 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 1 -- 172.21.15.1:0/2605230772 wait complete. 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.110+0000 7f7eab20b740 20 client.17382 trim_cache size 0 max 16384 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7eab20b740 1 Processor -- start 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7eab20b740 1 -- start start 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e05d0 0x55e8940ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e1b40 0x55e8940f7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940eb370 0x55e8940dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7eab20b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41b39b0 con 0x55e8940eb370 2024-09-16T00:36:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7eab20b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41b1280 con 0x55e8940e1b40 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7eab20b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41af0d0 con 0x55e8940e05d0 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7ea2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940eb370 0x55e8940dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7ea2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940eb370 0x55e8940dc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55078/0 (socket says 172.21.15.1:55078) 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7ea2d9d640 1 -- 172.21.15.1:0/2868701465 learned_addr learned my addr 172.21.15.1:0/2868701465 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.123+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e1b40 0x55e8940f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7ea259c640 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e05d0 0x55e8940ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7ea2d9d640 1 -- 172.21.15.1:0/2868701465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e05d0 msgr2=0x55e8940ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e05d0 0x55e8940ed760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7ea2d9d640 1 -- 172.21.15.1:0/2868701465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e1b40 msgr2=0x55e8940f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e1b40 0x55e8940f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7ea2d9d640 1 -- 172.21.15.1:0/2868701465 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8940eb0b0 con 0x55e8940eb370 2024-09-16T00:36:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e1b40 0x55e8940f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940eb370 0x55e8940dc700 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e810 tx=0x7f7e9c010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3b2034c092dc51 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7e8bfff640 1 -- 172.21.15.1:0/2868701465 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c028070 con 0x55e8940eb370 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7e8bfff640 1 -- 172.21.15.1:0/2868701465 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9c01a900 con 0x55e8940eb370 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.124+0000 7f7e8bfff640 1 -- 172.21.15.1:0/2868701465 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c023990 con 0x55e8940eb370 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 -- 172.21.15.1:0/2868701465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940eb370 msgr2=0x55e8940dc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940eb370 0x55e8940dc700 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e810 tx=0x7f7e9c010040 comp rx=0 tx=0).stop 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 -- 172.21.15.1:0/2868701465 shutdown_connections 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940eb370 0x55e8940dc700 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e1b40 0x55e8940f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 --2- 172.21.15.1:0/2868701465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940e05d0 0x55e8940ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 -- 172.21.15.1:0/2868701465 >> 172.21.15.1:0/2868701465 conn(0x55e8940a9310 msgr2=0x55e8940d7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 -- 172.21.15.1:0/2868701465 shutdown_connections 2024-09-16T00:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 -- 172.21.15.1:0/2868701465 wait complete. 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 -- start start 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 Processor -- start 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940ef020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 0x55e8940ef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41af740 con 0x55e8940e1b40 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41b0bb0 con 0x55e8940e05d0 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41afe10 con 0x55e8940eb370 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7eab20b740 10 client.0 _ll_register_callbacks cb 0x7f7e8c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.125+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940ef020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55094/0 (socket says 172.21.15.1:55094) 2024-09-16T00:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e9bfff640 1 -- 172.21.15.1:0/3694246425 learned_addr learned my addr 172.21.15.1:0/3694246425 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7ea259c640 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7ea2d9d640 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 0x55e8940ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e9bfff640 1 -- 172.21.15.1:0/3694246425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 msgr2=0x55e8940ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 0x55e8940ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e9bfff640 1 -- 172.21.15.1:0/3694246425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 msgr2=0x55e8940f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e9bfff640 1 -- 172.21.15.1:0/3694246425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8940efe40 con 0x55e8940e1b40 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7ea259c640 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e9bfff640 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940ef020 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0b87c0 tx=0x7f7e8c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c216f4a5ddd112 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e89ffb640 1 -- 172.21.15.1:0/3694246425 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0d5060 con 0x55e8940e1b40 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e89ffb640 1 -- 172.21.15.1:0/3694246425 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7e8c0bc5c0 con 0x55e8940e1b40 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7eab20b740 1 -- 172.21.15.1:0/3694246425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9c01a900 con 0x55e8940e1b40 2024-09-16T00:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e89ffb640 1 -- 172.21.15.1:0/3694246425 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e8c049070 con 0x55e8940e1b40 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7eab20b740 10 client.17388 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7eab20b740 1 -- 172.21.15.1:0/3694246425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e894165da0 con 0x55e8940e1b40 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.126+0000 7f7e89ffb640 1 -- 172.21.15.1:0/3694246425 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9c01a900 con 0x55e8940e1b40 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7e89ffb640 1 -- 172.21.15.1:0/3694246425 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7e8c0c4b40 con 0x55e8940e1b40 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7eab20b740 10 client.17388 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7eab20b740 10 client.17388 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7eab20b740 20 client.17388 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e894166340 0x55e894168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7eab20b740 4 client.17388 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7eab20b740 1 -- 172.21.15.1:0/3694246425 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e89404dd70 con 0x55e894166340 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7ea259c640 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e894166340 0x55e894168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7ea259c640 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e894166340 0x55e894168760 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=330eca2f53e63f server_cookie=4cebbc3e5d79bb11 in_seq=0 out_seq=0 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7e89ffb640 10 client.17388 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7e89ffb640 1 -- 172.21.15.1:0/3694246425 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e89404dd70 con 0x55e894166340 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7e89ffb640 10 client.17388 handle_command_reply: tid=0 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.127+0000 7f7eab20b740 1 client.17388 shutdown 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 -- 172.21.15.1:0/3694246425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 msgr2=0x55e8940ef020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940ef020 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0b87c0 tx=0x7f7e8c003f90 comp rx=0 tx=0).stop 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 -- 172.21.15.1:0/3694246425 shutdown_connections 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8940eb370 0x55e8940ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8940e1b40 0x55e8940ef020 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55e894166340 0x55e894168760 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 --2- 172.21.15.1:0/3694246425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8940e05d0 0x55e8940f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 -- 172.21.15.1:0/3694246425 >> 172.21.15.1:0/3694246425 conn(0x55e8940a9310 msgr2=0x55e8940e9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7e89ffb640 0 client.17388 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 -- 172.21.15.1:0/3694246425 shutdown_connections 2024-09-16T00:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 1 -- 172.21.15.1:0/3694246425 wait complete. 2024-09-16T00:36:46.157 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:36:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7eab20b740 20 client.17388 trim_cache size 0 max 16384 2024-09-16T00:36:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7ea8e32640 1 -- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7e900a3240 msgr2=0x7f7e900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7e900a3240 0x7f7e900a5700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c4f0 tx=0x7f7e9c008220 comp rx=0 tx=0).stop 2024-09-16T00:36:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7ea8e32640 1 -- 172.21.15.1:0/815817602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea4102790 msgr2=0x7f7ea4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea4102790 0x7f7ea4074a30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c02a930 tx=0x7f7e8c003b90 comp rx=0 tx=0).stop 2024-09-16T00:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.128+0000 7f7ea8e32640 1 -- 172.21.15.1:0/815817602 shutdown_connections 2024-09-16T00:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.129+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7e900a3240 0x7f7e900a5700 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.129+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea4106f10 0x7f7ea4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.129+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea4102790 0x7f7ea4074a30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.129+0000 7f7ea8e32640 1 --2- 172.21.15.1:0/815817602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ea4101de0 0x7f7ea40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.129+0000 7f7ea8e32640 1 -- 172.21.15.1:0/815817602 >> 172.21.15.1:0/815817602 conn(0x7f7ea40fd8c0 msgr2=0x7f7ea40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.129+0000 7f7ea8e32640 1 -- 172.21.15.1:0/815817602 shutdown_connections 2024-09-16T00:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:46.129+0000 7f7ea8e32640 1 -- 172.21.15.1:0/815817602 wait complete. 2024-09-16T00:36:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:46 smithi019 ceph-mon[35186]: pgmap v601: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-16T00:36:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:46 smithi001 ceph-mon[30934]: pgmap v601: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-16T00:36:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:46 smithi153 ceph-mon[35538]: pgmap v601: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-16T00:36:46.940 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:36:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:47.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.701+0000 7f4357e57640 1 -- 172.21.15.1:0/1208429970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501027a0 msgr2=0x7f43501067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.701+0000 7f4357e57640 1 --2- 172.21.15.1:0/1208429970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501027a0 0x7f43501067f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4340009900 tx=0x7f434002a410 comp rx=0 tx=0).stop 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.701+0000 7f4357e57640 1 -- 172.21.15.1:0/1208429970 shutdown_connections 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.701+0000 7f4357e57640 1 --2- 172.21.15.1:0/1208429970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350106f20 0x7f435010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.701+0000 7f4357e57640 1 --2- 172.21.15.1:0/1208429970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501027a0 0x7f43501067f0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.701+0000 7f4357e57640 1 --2- 172.21.15.1:0/1208429970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4350101df0 0x7f43501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.701+0000 7f4357e57640 1 -- 172.21.15.1:0/1208429970 >> 172.21.15.1:0/1208429970 conn(0x7f43500fd8f0 msgr2=0x7f43500ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.701+0000 7f4357e57640 1 -- 172.21.15.1:0/1208429970 shutdown_connections 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.701+0000 7f4357e57640 1 -- 172.21.15.1:0/1208429970 wait complete. 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f4357e57640 1 Processor -- start 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f4357e57640 1 -- start start 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f4357e57640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350101df0 0x7f43501983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f4357e57640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43501027a0 0x7f4350198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f4357e57640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4350106f20 0x7f435019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f4357e57640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4350111f80 con 0x7f4350106f20 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f4357e57640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4350111e00 con 0x7f4350101df0 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f4357e57640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4350112100 con 0x7f43501027a0 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f43563f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4350106f20 0x7f435019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f43563f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4350106f20 0x7f435019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55104/0 (socket says 172.21.15.1:55104) 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f43563f4640 1 -- 172.21.15.1:0/1069241782 learned_addr learned my addr 172.21.15.1:0/1069241782 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.702+0000 7f4355bf3640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350101df0 0x7f43501983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f43553f2640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43501027a0 0x7f4350198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f43563f4640 1 -- 172.21.15.1:0/1069241782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43501027a0 msgr2=0x7f4350198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f43563f4640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43501027a0 0x7f4350198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f43563f4640 1 -- 172.21.15.1:0/1069241782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350101df0 msgr2=0x7f43501983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f43563f4640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350101df0 0x7f43501983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f43563f4640 1 -- 172.21.15.1:0/1069241782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435019d370 con 0x7f4350106f20 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f43553f2640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43501027a0 0x7f4350198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f43563f4640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4350106f20 0x7f435019cc90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f434800cce0 tx=0x7f434800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f4346ffd640 1 -- 172.21.15.1:0/1069241782 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434800f450 con 0x7f4350106f20 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f4357e57640 1 -- 172.21.15.1:0/1069241782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435019d660 con 0x7f4350106f20 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f4346ffd640 1 -- 172.21.15.1:0/1069241782 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434800fad0 con 0x7f4350106f20 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f4346ffd640 1 -- 172.21.15.1:0/1069241782 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348018c10 con 0x7f4350106f20 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.703+0000 7f4357e57640 1 -- 172.21.15.1:0/1069241782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43501a4f40 con 0x7f4350106f20 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.705+0000 7f4346ffd640 1 -- 172.21.15.1:0/1069241782 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f434800f5f0 con 0x7f4350106f20 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f435a230740 1 Processor -- start 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f435a230740 1 -- start start 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f435a230740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57323b40 0x562a57322090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f435a230740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a573225d0 0x562a57322a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f435a230740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 0x562a57333b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f435a230740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562a572882f0 con 0x562a573225d0 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f435a230740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562a5722fcf0 con 0x562a57331780 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f435a230740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a572842b0 con 0x562a57323b40 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f4355bf3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a573225d0 0x562a57322a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f4355bf3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a573225d0 0x562a57322a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55108/0 (socket says 172.21.15.1:55108) 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f4346ffd640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f4355bf3640 1 -- 172.21.15.1:0/2787359775 learned_addr learned my addr 172.21.15.1:0/2787359775 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f43553f2640 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 0x562a57333b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f43553f2640 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57323b40 0x562a57322090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f43563f4640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.706+0000 7f4346ffd640 1 -- 172.21.15.1:0/1069241782 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f43480d0400 con 0x7f4350106f20 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f4355bf3640 1 -- 172.21.15.1:0/2787359775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57323b40 msgr2=0x562a57322090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f4355bf3640 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57323b40 0x562a57322090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f4355bf3640 1 -- 172.21.15.1:0/2787359775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 msgr2=0x562a57333b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f4355bf3640 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 0x562a57333b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f4355bf3640 1 -- 172.21.15.1:0/2787359775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a572143a0 con 0x562a573225d0 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f43553f2640 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57323b40 0x562a57322090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f4355bf3640 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a573225d0 0x562a57322a10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f434c00bcf0 tx=0x7f434c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b593d19f213170 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f433bfff640 1 -- 172.21.15.1:0/2787359775 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c015070 con 0x562a573225d0 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f43563f4640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f43480186f0 tx=0x7f4348091b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f433bfff640 1 -- 172.21.15.1:0/2787359775 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434c01a940 con 0x562a573225d0 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.707+0000 7f433bfff640 1 -- 172.21.15.1:0/2787359775 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c0119d0 con 0x562a573225d0 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 -- 172.21.15.1:0/2787359775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a573225d0 msgr2=0x562a57322a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a573225d0 0x562a57322a10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f434c00bcf0 tx=0x7f434c00f730 comp rx=0 tx=0).stop 2024-09-16T00:36:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 -- 172.21.15.1:0/2787359775 shutdown_connections 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 0x562a57333b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a573225d0 0x562a57322a10 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 --2- 172.21.15.1:0/2787359775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57323b40 0x562a57322090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 -- 172.21.15.1:0/2787359775 >> 172.21.15.1:0/2787359775 conn(0x562a572eb310 msgr2=0x562a5731ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 -- 172.21.15.1:0/2787359775 shutdown_connections 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 -- 172.21.15.1:0/2787359775 wait complete. 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 -- start start 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.708+0000 7f435a230740 1 Processor -- start 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f435a230740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 0x562a57331190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f435a230740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57323b40 0x562a5732e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f435a230740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57331780 0x562a5732ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f435a230740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562a57334480 con 0x562a57331780 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f435a230740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562a57243060 con 0x562a57323b40 2024-09-16T00:36:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f435a230740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a57334600 con 0x562a573225d0 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f435a230740 10 client.0 _ll_register_callbacks cb 0x562a56f968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57331780 0x562a5732ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57331780 0x562a5732ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55118/0 (socket says 172.21.15.1:55118) 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 -- 172.21.15.1:0/3426383953 learned_addr learned my addr 172.21.15.1:0/3426383953 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 0x562a57331190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f4355bf3640 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57323b40 0x562a5732e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 -- 172.21.15.1:0/3426383953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 msgr2=0x562a57331190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 0x562a57331190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 -- 172.21.15.1:0/3426383953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57323b40 msgr2=0x562a5732e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57323b40 0x562a5732e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 -- 172.21.15.1:0/3426383953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a56ff93c0 con 0x562a57331780 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f4355bf3640 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57323b40 0x562a5732e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43553f2640 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57331780 0x562a5732ea50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4340005c40 tx=0x7f434002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfa60e2a9f78e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:47.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43397fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.709+0000 7f43397fa640 1 -- 172.21.15.1:0/3426383953 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340035040 con 0x562a57331780 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f43397fa640 1 -- 172.21.15.1:0/3426383953 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f43400037f0 con 0x562a57331780 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f435a230740 1 -- 172.21.15.1:0/3426383953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c01a940 con 0x562a57331780 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f43397fa640 1 -- 172.21.15.1:0/3426383953 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4340052050 con 0x562a57331780 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f435a230740 10 client.17400 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f435a230740 1 -- 172.21.15.1:0/3426383953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a5733fec0 con 0x562a57331780 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f43397fa640 1 -- 172.21.15.1:0/3426383953 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434c01a940 con 0x562a57331780 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f43397fa640 1 -- 172.21.15.1:0/3426383953 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f434003a050 con 0x562a57331780 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f435a230740 10 client.17400 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f435a230740 10 client.17400 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f435a230740 20 client.17400 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f435a230740 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x562a573412d0 0x562a573436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f435a230740 4 client.17400 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f435a230740 1 -- 172.21.15.1:0/3426383953 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a57295860 con 0x562a573412d0 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.710+0000 7f4355bf3640 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x562a573412d0 0x562a573436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.711+0000 7f4355bf3640 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x562a573412d0 0x562a573436c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e2986ddd805ed0 server_cookie=699ee08654d32d51 in_seq=0 out_seq=0 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.711+0000 7f43397fa640 10 client.17400 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.711+0000 7f43397fa640 1 -- 172.21.15.1:0/3426383953 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a57295860 con 0x562a573412d0 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.711+0000 7f43397fa640 10 client.17400 handle_command_reply: tid=0 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.711+0000 7f435a230740 1 client.17400 shutdown 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 -- 172.21.15.1:0/3426383953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57331780 msgr2=0x562a5732ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57331780 0x562a5732ea50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4340005c40 tx=0x7f434002a410 comp rx=0 tx=0).stop 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 -- 172.21.15.1:0/3426383953 shutdown_connections 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x562a573412d0 0x562a573436c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57331780 0x562a5732ea50 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57323b40 0x562a5732e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f43397fa640 0 client.17400 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 --2- 172.21.15.1:0/3426383953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 0x562a57331190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 -- 172.21.15.1:0/3426383953 >> 172.21.15.1:0/3426383953 conn(0x562a572eb310 msgr2=0x562a57324960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 -- 172.21.15.1:0/3426383953 shutdown_connections 2024-09-16T00:36:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 1 -- 172.21.15.1:0/3426383953 wait complete. 2024-09-16T00:36:47.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.712+0000 7f435a230740 20 client.17400 trim_cache size 0 max 16384 2024-09-16T00:36:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f435a230740 1 Processor -- start 2024-09-16T00:36:47.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f435a230740 1 -- start start 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f435a230740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a573225d0 0x562a5731ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f435a230740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5733d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f435a230740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57331780 0x562a57348f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f435a230740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43501af4c0 con 0x562a57323b40 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f435a230740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43501acd90 con 0x562a573225d0 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f435a230740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43501aabe0 con 0x562a57331780 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f4355bf3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5733d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f4355bf3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5733d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55126/0 (socket says 172.21.15.1:55126) 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f4355bf3640 1 -- 172.21.15.1:0/3609429168 learned_addr learned my addr 172.21.15.1:0/3609429168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f43553f2640 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a573225d0 0x562a5731ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.725+0000 7f43553f2640 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57331780 0x562a57348f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f4355bf3640 1 -- 172.21.15.1:0/3609429168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57331780 msgr2=0x562a57348f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f4355bf3640 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57331780 0x562a57348f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f4355bf3640 1 -- 172.21.15.1:0/3609429168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a573225d0 msgr2=0x562a5731ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f4355bf3640 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a573225d0 0x562a5731ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f4355bf3640 1 -- 172.21.15.1:0/3609429168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a57343d20 con 0x562a57323b40 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f43553f2640 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57331780 0x562a57348f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f4355bf3640 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5733d720 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f434c00b870 tx=0x7f434c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e6174f0a72dd18 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f433b7fe640 1 -- 172.21.15.1:0/3609429168 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c02c070 con 0x562a57323b40 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f433b7fe640 1 -- 172.21.15.1:0/3609429168 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434c011740 con 0x562a57323b40 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.726+0000 7f433b7fe640 1 -- 172.21.15.1:0/3609429168 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c015040 con 0x562a57323b40 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 -- 172.21.15.1:0/3609429168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 msgr2=0x562a5733d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5733d720 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f434c00b870 tx=0x7f434c005850 comp rx=0 tx=0).stop 2024-09-16T00:36:47.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 -- 172.21.15.1:0/3609429168 shutdown_connections 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a57331780 0x562a57348f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5733d720 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 --2- 172.21.15.1:0/3609429168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a573225d0 0x562a5731ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 -- 172.21.15.1:0/3609429168 >> 172.21.15.1:0/3609429168 conn(0x562a572eb310 msgr2=0x562a5731e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 -- 172.21.15.1:0/3609429168 shutdown_connections 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 -- 172.21.15.1:0/3609429168 wait complete. 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 -- start start 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 Processor -- start 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 0x562a5732b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5732ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 0x562a5732bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43501ab250 con 0x562a57323b40 2024-09-16T00:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43501ac6c0 con 0x562a57331780 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.727+0000 7f435a230740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43501ab920 con 0x562a573225d0 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f4355bf3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5732ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f435a230740 10 client.0 _ll_register_callbacks cb 0x7f4340005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f4355bf3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5732ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55132/0 (socket says 172.21.15.1:55132) 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f4355bf3640 1 -- 172.21.15.1:0/1918043893 learned_addr learned my addr 172.21.15.1:0/1918043893 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f43553f2640 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 0x562a5732b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f43553f2640 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 0x562a5732bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f4355bf3640 1 -- 172.21.15.1:0/1918043893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 msgr2=0x562a5732b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f4355bf3640 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 0x562a5732b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f4355bf3640 1 -- 172.21.15.1:0/1918043893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 msgr2=0x562a5732bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f4355bf3640 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 0x562a5732bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f4355bf3640 1 -- 172.21.15.1:0/1918043893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a573aaf90 con 0x562a57323b40 2024-09-16T00:36:47.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f43553f2640 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 0x562a5732bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f4355bf3640 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5732ba20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f434c0058c0 tx=0x7f434c00b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f03e773169173c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f43397fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f43397fa640 1 -- 172.21.15.1:0/1918043893 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c02c070 con 0x562a57323b40 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f43397fa640 1 -- 172.21.15.1:0/1918043893 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f434c011d00 con 0x562a57323b40 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f435a230740 1 -- 172.21.15.1:0/1918043893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c011740 con 0x562a57323b40 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f43397fa640 1 -- 172.21.15.1:0/1918043893 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f434c005040 con 0x562a57323b40 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f435a230740 10 client.17406 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.728+0000 7f435a230740 1 -- 172.21.15.1:0/1918043893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a573ac290 con 0x562a57323b40 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f43397fa640 1 -- 172.21.15.1:0/1918043893 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434c011740 con 0x562a57323b40 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f43397fa640 1 -- 172.21.15.1:0/1918043893 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f434c025470 con 0x562a57323b40 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f435a230740 10 client.17406 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f435a230740 10 client.17406 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:36:47.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f435a230740 20 client.17406 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f435a230740 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x562a573ac750 0x562a573aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f435a230740 4 client.17406 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f435a230740 1 -- 172.21.15.1:0/1918043893 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a5728fd70 con 0x562a573ac750 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f43553f2640 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x562a573ac750 0x562a573aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f43397fa640 10 client.17406 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.729+0000 7f43553f2640 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x562a573ac750 0x562a573aeb40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f4ab120043c9fb6 server_cookie=a995865308f80822 in_seq=0 out_seq=0 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f43397fa640 1 -- 172.21.15.1:0/1918043893 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a5728fd70 con 0x562a573ac750 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f43397fa640 10 client.17406 handle_command_reply: tid=0 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 client.17406 shutdown 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 -- 172.21.15.1:0/1918043893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 msgr2=0x562a5732ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5732ba20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f434c0058c0 tx=0x7f434c00b3a0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 -- 172.21.15.1:0/1918043893 shutdown_connections 2024-09-16T00:36:47.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a57331780 0x562a5732bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x562a573ac750 0x562a573aeb40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a57323b40 0x562a5732ba20 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 --2- 172.21.15.1:0/1918043893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a573225d0 0x562a5732b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f43397fa640 0 client.17406 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:47.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 -- 172.21.15.1:0/1918043893 >> 172.21.15.1:0/1918043893 conn(0x562a572eb310 msgr2=0x562a573304c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:47.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 -- 172.21.15.1:0/1918043893 shutdown_connections 2024-09-16T00:36:47.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 1 -- 172.21.15.1:0/1918043893 wait complete. 2024-09-16T00:36:47.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f435a230740 20 client.17406 trim_cache size 0 max 16384 2024-09-16T00:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:36:47.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.730+0000 7f4357e57640 1 -- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 msgr2=0x7f433c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:47.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f43480186f0 tx=0x7f4348091b00 comp rx=0 tx=0).stop 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 -- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4350106f20 msgr2=0x7f435019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4350106f20 0x7f435019cc90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f434800cce0 tx=0x7f434800a620 comp rx=0 tx=0).stop 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 -- 172.21.15.1:0/1069241782 shutdown_connections 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4350106f20 0x7f435019cc90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43501027a0 0x7f4350198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 --2- 172.21.15.1:0/1069241782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350101df0 0x7f43501983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 -- 172.21.15.1:0/1069241782 >> 172.21.15.1:0/1069241782 conn(0x7f43500fd8f0 msgr2=0x7f43500fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 -- 172.21.15.1:0/1069241782 shutdown_connections 2024-09-16T00:36:47.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:47.731+0000 7f4357e57640 1 -- 172.21.15.1:0/1069241782 wait complete. 2024-09-16T00:36:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:48 smithi019 ceph-mon[35186]: pgmap v602: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T00:36:48.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:48 smithi001 ceph-mon[30934]: pgmap v602: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T00:36:48.460 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:36:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:48 smithi153 ceph-mon[35538]: pgmap v602: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T00:36:48.721 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:49.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.234+0000 7f61e73b4640 1 -- 172.21.15.1:0/3977404205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 msgr2=0x7f61e01009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:49.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.234+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3977404205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 0x7f61e01009c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009900 tx=0x7f61d002a410 comp rx=0 tx=0).stop 2024-09-16T00:36:49.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.234+0000 7f61e73b4640 1 -- 172.21.15.1:0/3977404205 shutdown_connections 2024-09-16T00:36:49.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.234+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3977404205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e0104e80 0x7f61e0105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.234+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3977404205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 0x7f61e01009c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.234+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3977404205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e0069f00 0x7f61e0100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.234+0000 7f61e73b4640 1 -- 172.21.15.1:0/3977404205 >> 172.21.15.1:0/3977404205 conn(0x7f61e0077210 msgr2=0x7f61e00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:49.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.234+0000 7f61e73b4640 1 -- 172.21.15.1:0/3977404205 shutdown_connections 2024-09-16T00:36:49.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.234+0000 7f61e73b4640 1 -- 172.21.15.1:0/3977404205 wait complete. 2024-09-16T00:36:49.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e73b4640 1 Processor -- start 2024-09-16T00:36:49.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e73b4640 1 -- start start 2024-09-16T00:36:49.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e73b4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e0069f00 0x7f61e0102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e73b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 0x7f61e0103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e73b4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e0104e80 0x7f61e0103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e73b4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0110eb0 con 0x7f61e006a370 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e73b4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0110d30 con 0x7f61e0104e80 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e73b4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0111030 con 0x7f61e0069f00 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e494f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 0x7f61e0103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e494f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 0x7f61e0103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55162/0 (socket says 172.21.15.1:55162) 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e494f640 1 -- 172.21.15.1:0/3713471320 learned_addr learned my addr 172.21.15.1:0/3713471320 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.235+0000 7f61e5150640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e0069f00 0x7f61e0102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61e5951640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e0104e80 0x7f61e0103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61e494f640 1 -- 172.21.15.1:0/3713471320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e0069f00 msgr2=0x7f61e0102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61e494f640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e0069f00 0x7f61e0102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61e494f640 1 -- 172.21.15.1:0/3713471320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e0104e80 msgr2=0x7f61e0103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:49.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61e494f640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e0104e80 0x7f61e0103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61e494f640 1 -- 172.21.15.1:0/3713471320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e01a6930 con 0x7f61e006a370 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61e494f640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 0x7f61e0103020 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f61d0006010 tx=0x7f61d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61d67fc640 1 -- 172.21.15.1:0/3713471320 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d003f030 con 0x7f61e006a370 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61d67fc640 1 -- 172.21.15.1:0/3713471320 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d0003ce0 con 0x7f61e006a370 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61e73b4640 1 -- 172.21.15.1:0/3713471320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e01a6bc0 con 0x7f61e006a370 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61d67fc640 1 -- 172.21.15.1:0/3713471320 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0035070 con 0x7f61e006a370 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.236+0000 7f61e73b4640 1 -- 172.21.15.1:0/3713471320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01a70f0 con 0x7f61e006a370 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61d67fc640 1 -- 172.21.15.1:0/3713471320 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f61d0042030 con 0x7f61e006a370 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61e978d740 1 Processor -- start 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61e978d740 1 -- start start 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61e978d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa86b40 0x56035aa85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61e978d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa85a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61e978d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa90370 0x56035aa92760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61e978d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56035a9eb2f0 con 0x56035aa86b40 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61e978d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56035a992cf0 con 0x56035aa855d0 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61e978d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56035a9e72b0 con 0x56035aa90370 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.239+0000 7f61e5951640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa86b40 0x56035aa85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5951640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa86b40 0x56035aa85090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55178/0 (socket says 172.21.15.1:55178) 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5951640 1 -- 172.21.15.1:0/1505650523 learned_addr learned my addr 172.21.15.1:0/1505650523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5150640 1 --2- 172.21.15.1:0/1505650523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa85a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e494f640 1 --2- 172.21.15.1:0/1505650523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa90370 0x56035aa92760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61d67fc640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61cc0a32c0 0x7f61cc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5951640 1 -- 172.21.15.1:0/1505650523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa90370 msgr2=0x56035aa92760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5951640 1 --2- 172.21.15.1:0/1505650523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa90370 0x56035aa92760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5951640 1 -- 172.21.15.1:0/1505650523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 msgr2=0x56035aa85a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5951640 1 --2- 172.21.15.1:0/1505650523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa85a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5951640 1 -- 172.21.15.1:0/1505650523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035a9773a0 con 0x56035aa86b40 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61d67fc640 1 -- 172.21.15.1:0/3713471320 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f61d00efd30 con 0x7f61e006a370 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5951640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61cc0a32c0 0x7f61cc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61e5951640 1 --2- 172.21.15.1:0/1505650523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa86b40 0x56035aa85090 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009f50 tx=0x7f61d80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578cd43c5f51af64 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.240+0000 7f61cb7fe640 1 -- 172.21.15.1:0/1505650523 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8004250 con 0x56035aa86b40 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61cb7fe640 1 -- 172.21.15.1:0/1505650523 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d800fe80 con 0x56035aa86b40 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61cb7fe640 1 -- 172.21.15.1:0/1505650523 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d801bb20 con 0x56035aa86b40 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e978d740 1 -- 172.21.15.1:0/1505650523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa86b40 msgr2=0x56035aa85090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e978d740 1 --2- 172.21.15.1:0/1505650523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa86b40 0x56035aa85090 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009f50 tx=0x7f61d80118d0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e5951640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61cc0a32c0 0x7f61cc0a5780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f61d8012fd0 tx=0x7f61d8021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e978d740 1 -- 172.21.15.1:0/1505650523 shutdown_connections 2024-09-16T00:36:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e978d740 1 --2- 172.21.15.1:0/1505650523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa90370 0x56035aa92760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e978d740 1 --2- 172.21.15.1:0/1505650523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa85a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e978d740 1 --2- 172.21.15.1:0/1505650523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa86b40 0x56035aa85090 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e978d740 1 -- 172.21.15.1:0/1505650523 >> 172.21.15.1:0/1505650523 conn(0x56035aa4e310 msgr2=0x56035aa81c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e978d740 1 -- 172.21.15.1:0/1505650523 shutdown_connections 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.241+0000 7f61e978d740 1 -- 172.21.15.1:0/1505650523 wait complete. 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.242+0000 7f61e978d740 1 -- start start 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.242+0000 7f61e978d740 1 Processor -- start 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.242+0000 7f61e978d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa8fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.242+0000 7f61e978d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa8cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.242+0000 7f61e978d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa8d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.242+0000 7f61e978d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56035aa93070 con 0x56035aa90370 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.242+0000 7f61e978d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56035a9a6060 con 0x56035aa855d0 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.242+0000 7f61e978d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56035aa931f0 con 0x56035aa86b40 2024-09-16T00:36:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.242+0000 7f61e978d740 10 client.0 _ll_register_callbacks cb 0x56035a6f98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5951640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa8d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5951640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa8d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55190/0 (socket says 172.21.15.1:55190) 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5951640 1 -- 172.21.15.1:0/2122749079 learned_addr learned my addr 172.21.15.1:0/2122749079 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e494f640 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa8cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5150640 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa8fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5951640 1 -- 172.21.15.1:0/2122749079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 msgr2=0x56035aa8cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5951640 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa8cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5951640 1 -- 172.21.15.1:0/2122749079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 msgr2=0x56035aa8fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5951640 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa8fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5951640 1 -- 172.21.15.1:0/2122749079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035a75c3c0 con 0x56035aa90370 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5150640 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa8fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e494f640 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa8cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61e5951640 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa8d520 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009730 tx=0x7f61d801b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7888fe1cf1cb4b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.243+0000 7f61c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61c8ff9640 1 -- 172.21.15.1:0/2122749079 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8033020 con 0x56035aa90370 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61c8ff9640 1 -- 172.21.15.1:0/2122749079 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f61d800f920 con 0x56035aa90370 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61e978d740 1 -- 172.21.15.1:0/2122749079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56035a9f8860 con 0x56035aa90370 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61c8ff9640 1 -- 172.21.15.1:0/2122749079 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61d803e050 con 0x56035aa90370 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61e978d740 10 client.17418 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61e978d740 1 -- 172.21.15.1:0/2122749079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56035aa9eb00 con 0x56035aa90370 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61c8ff9640 1 -- 172.21.15.1:0/2122749079 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56035a9f8860 con 0x56035aa90370 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61c8ff9640 1 -- 172.21.15.1:0/2122749079 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f61d8025c40 con 0x56035aa90370 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61e978d740 10 client.17418 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61e978d740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61e978d740 20 client.17418 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61e978d740 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56035aa9ff00 0x56035aaa22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61e978d740 4 client.17418 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:49.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.244+0000 7f61e978d740 1 -- 172.21.15.1:0/2122749079 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56035a9f2d70 con 0x56035aa9ff00 2024-09-16T00:36:49.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.245+0000 7f61e5150640 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56035aa9ff00 0x56035aaa22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.245+0000 7f61e5150640 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56035aa9ff00 0x56035aaa22f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb95f9db1838ab9f server_cookie=f9b1c62ba5cf9efb in_seq=0 out_seq=0 2024-09-16T00:36:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.245+0000 7f61c8ff9640 10 client.17418 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61c8ff9640 1 -- 172.21.15.1:0/2122749079 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56035a9f2d70 con 0x56035aa9ff00 2024-09-16T00:36:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61c8ff9640 10 client.17418 handle_command_reply: tid=0 2024-09-16T00:36:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 client.17418 shutdown 2024-09-16T00:36:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 -- 172.21.15.1:0/2122749079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 msgr2=0x56035aa8d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa8d520 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009730 tx=0x7f61d801b430 comp rx=0 tx=0).stop 2024-09-16T00:36:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 -- 172.21.15.1:0/2122749079 shutdown_connections 2024-09-16T00:36:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56035aa9ff00 0x56035aaa22f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa8d520 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa8cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 --2- 172.21.15.1:0/2122749079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa8fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 -- 172.21.15.1:0/2122749079 >> 172.21.15.1:0/2122749079 conn(0x56035aa4e310 msgr2=0x56035aa87930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61c8ff9640 0 client.17418 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 -- 172.21.15.1:0/2122749079 shutdown_connections 2024-09-16T00:36:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 1 -- 172.21.15.1:0/2122749079 wait complete. 2024-09-16T00:36:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.246+0000 7f61e978d740 20 client.17418 trim_cache size 0 max 16384 2024-09-16T00:36:49.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.259+0000 7f61e978d740 1 Processor -- start 2024-09-16T00:36:49.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.259+0000 7f61e978d740 1 -- start start 2024-09-16T00:36:49.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.259+0000 7f61e978d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa81e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.259+0000 7f61e978d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa9c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.259+0000 7f61e978d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa88120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.259+0000 7f61e978d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01b38e0 con 0x56035aa90370 2024-09-16T00:36:49.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.259+0000 7f61e978d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01b11b0 con 0x56035aa855d0 2024-09-16T00:36:49.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.259+0000 7f61e978d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01af000 con 0x56035aa86b40 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5951640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa88120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5951640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa88120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55202/0 (socket says 172.21.15.1:55202) 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5951640 1 -- 172.21.15.1:0/474490425 learned_addr learned my addr 172.21.15.1:0/474490425 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5150640 1 --2- 172.21.15.1:0/474490425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa81e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e494f640 1 --2- 172.21.15.1:0/474490425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa9c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5951640 1 -- 172.21.15.1:0/474490425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 msgr2=0x56035aa9c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5951640 1 --2- 172.21.15.1:0/474490425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa9c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5951640 1 -- 172.21.15.1:0/474490425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 msgr2=0x56035aa81e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5951640 1 --2- 172.21.15.1:0/474490425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa81e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5951640 1 -- 172.21.15.1:0/474490425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035aa8b4f0 con 0x56035aa90370 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61e5951640 1 --2- 172.21.15.1:0/474490425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa88120 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f61d802f4f0 tx=0x7f61d80094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49b286f9a8fd41 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61caffd640 1 -- 172.21.15.1:0/474490425 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8033070 con 0x56035aa90370 2024-09-16T00:36:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61caffd640 1 -- 172.21.15.1:0/474490425 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d8009b30 con 0x56035aa90370 2024-09-16T00:36:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.260+0000 7f61caffd640 1 -- 172.21.15.1:0/474490425 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d801be60 con 0x56035aa90370 2024-09-16T00:36:49.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 -- 172.21.15.1:0/474490425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 msgr2=0x56035aa88120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:49.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 --2- 172.21.15.1:0/474490425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa88120 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f61d802f4f0 tx=0x7f61d80094e0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 -- 172.21.15.1:0/474490425 shutdown_connections 2024-09-16T00:36:49.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 --2- 172.21.15.1:0/474490425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa90370 0x56035aa88120 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 --2- 172.21.15.1:0/474490425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa9c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 --2- 172.21.15.1:0/474490425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa855d0 0x56035aa81e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 -- 172.21.15.1:0/474490425 >> 172.21.15.1:0/474490425 conn(0x56035aa4e310 msgr2=0x56035aa81700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 -- 172.21.15.1:0/474490425 shutdown_connections 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 -- 172.21.15.1:0/474490425 wait complete. 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 -- start start 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.261+0000 7f61e978d740 1 Processor -- start 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e978d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa855d0 0x56035aa934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e978d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa939e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e978d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa90370 0x56035aa93f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e978d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01af670 con 0x56035aa855d0 2024-09-16T00:36:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e978d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01b0ae0 con 0x56035aa90370 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e978d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01afd40 con 0x56035aa86b40 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e978d740 10 client.0 _ll_register_callbacks cb 0x7f61dc00a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e5150640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa855d0 0x56035aa934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e5150640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa855d0 0x56035aa934a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55214/0 (socket says 172.21.15.1:55214) 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e5150640 1 -- 172.21.15.1:0/1990612799 learned_addr learned my addr 172.21.15.1:0/1990612799 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e494f640 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.262+0000 7f61e5951640 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa90370 0x56035aa93f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e5150640 1 -- 172.21.15.1:0/1990612799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 msgr2=0x56035aa939e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e5150640 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa939e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e5150640 1 -- 172.21.15.1:0/1990612799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa90370 msgr2=0x56035aa93f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e5150640 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa90370 0x56035aa93f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e5150640 1 -- 172.21.15.1:0/1990612799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035ab08a70 con 0x56035aa855d0 2024-09-16T00:36:49.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e5951640 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa90370 0x56035aa93f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:49.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e5150640 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa855d0 0x56035aa934a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f61dc010810 tx=0x7f61dc010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc7aca8151334e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:49.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:49.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61c8ff9640 1 -- 172.21.15.1:0/1990612799 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00fb90 con 0x56035aa855d0 2024-09-16T00:36:49.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61c8ff9640 1 -- 172.21.15.1:0/1990612799 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f61dc00fd30 con 0x56035aa855d0 2024-09-16T00:36:49.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e978d740 1 -- 172.21.15.1:0/1990612799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d8009b30 con 0x56035aa855d0 2024-09-16T00:36:49.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61c8ff9640 1 -- 172.21.15.1:0/1990612799 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61dc031050 con 0x56035aa855d0 2024-09-16T00:36:49.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e978d740 10 client.17424 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:49.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.263+0000 7f61e978d740 1 -- 172.21.15.1:0/1990612799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56035ab09d10 con 0x56035aa855d0 2024-09-16T00:36:49.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.264+0000 7f61c8ff9640 1 -- 172.21.15.1:0/1990612799 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d8009b30 con 0x56035aa855d0 2024-09-16T00:36:49.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.264+0000 7f61c8ff9640 1 -- 172.21.15.1:0/1990612799 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f61dc01e500 con 0x56035aa855d0 2024-09-16T00:36:49.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.264+0000 7f61e978d740 10 client.17424 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:49.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.264+0000 7f61e978d740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:49.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.264+0000 7f61e978d740 20 client.17424 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:49.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.264+0000 7f61e978d740 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56035ab0a460 0x56035ab0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:49.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.264+0000 7f61e978d740 4 client.17424 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:36:49.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.264+0000 7f61e978d740 1 -- 172.21.15.1:0/1990612799 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56035a96f170 con 0x56035ab0a460 2024-09-16T00:36:49.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.264+0000 7f61e494f640 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56035ab0a460 0x56035ab0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:49.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61c8ff9640 10 client.17424 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:49.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e494f640 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56035ab0a460 0x56035ab0c880 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b6b2c18c4b6bb5 server_cookie=7b74f4f28928ac7a in_seq=0 out_seq=0 2024-09-16T00:36:49.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61c8ff9640 1 -- 172.21.15.1:0/1990612799 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56035a96f170 con 0x56035ab0a460 2024-09-16T00:36:49.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61c8ff9640 10 client.17424 handle_command_reply: tid=0 2024-09-16T00:36:49.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 client.17424 shutdown 2024-09-16T00:36:49.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 -- 172.21.15.1:0/1990612799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa855d0 msgr2=0x56035aa934a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:49.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa855d0 0x56035aa934a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f61dc010810 tx=0x7f61dc010c10 comp rx=0 tx=0).stop 2024-09-16T00:36:49.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 -- 172.21.15.1:0/1990612799 shutdown_connections 2024-09-16T00:36:49.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56035ab0a460 0x56035ab0c880 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035aa90370 0x56035aa93f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61c8ff9640 0 client.17424 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:49.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56035aa86b40 0x56035aa939e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 --2- 172.21.15.1:0/1990612799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56035aa855d0 0x56035aa934a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 -- 172.21.15.1:0/1990612799 >> 172.21.15.1:0/1990612799 conn(0x56035aa4e310 msgr2=0x56035aa8b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:49.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 -- 172.21.15.1:0/1990612799 shutdown_connections 2024-09-16T00:36:49.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 1 -- 172.21.15.1:0/1990612799 wait complete. 2024-09-16T00:36:49.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.265+0000 7f61e978d740 20 client.17424 trim_cache size 0 max 16384 2024-09-16T00:36:49.274 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:36:49.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 -- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61cc0a32c0 msgr2=0x7f61cc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:49.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61cc0a32c0 0x7f61cc0a5780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f61d8012fd0 tx=0x7f61d8021000 comp rx=0 tx=0).stop 2024-09-16T00:36:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 -- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 msgr2=0x7f61e0103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 0x7f61e0103020 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f61d0006010 tx=0x7f61d00042f0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 -- 172.21.15.1:0/3713471320 shutdown_connections 2024-09-16T00:36:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e0104e80 0x7f61e0103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e006a370 0x7f61e0103020 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61cc0a32c0 0x7f61cc0a5780 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 --2- 172.21.15.1:0/3713471320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e0069f00 0x7f61e0102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 -- 172.21.15.1:0/3713471320 >> 172.21.15.1:0/3713471320 conn(0x7f61e0077210 msgr2=0x7f61e00fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 -- 172.21.15.1:0/3713471320 shutdown_connections 2024-09-16T00:36:49.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:49.266+0000 7f61e73b4640 1 -- 172.21.15.1:0/3713471320 wait complete. 2024-09-16T00:36:50.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:50 smithi001 ceph-mon[30934]: pgmap v603: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-09-16T00:36:50.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:50 smithi019 ceph-mon[35186]: pgmap v603: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-09-16T00:36:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:50 smithi153 ceph-mon[35538]: pgmap v603: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-09-16T00:36:50.992 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:36:50.992 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:36:51.265 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:51.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.789+0000 7f43581d6640 1 -- 172.21.15.1:0/1511297854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f435010c180 msgr2=0x7f435010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:51.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.789+0000 7f43581d6640 1 --2- 172.21.15.1:0/1511297854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f435010c180 0x7f435010e600 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4348009ca0 tx=0x7f434802a950 comp rx=0 tx=0).stop 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.789+0000 7f43581d6640 1 -- 172.21.15.1:0/1511297854 shutdown_connections 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.789+0000 7f43581d6640 1 --2- 172.21.15.1:0/1511297854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f435010c180 0x7f435010e600 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.789+0000 7f43581d6640 1 --2- 172.21.15.1:0/1511297854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43501036c0 0x7f435010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.789+0000 7f43581d6640 1 --2- 172.21.15.1:0/1511297854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4350102da0 0x7f4350103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.789+0000 7f43581d6640 1 -- 172.21.15.1:0/1511297854 >> 172.21.15.1:0/1511297854 conn(0x7f43500fc7a0 msgr2=0x7f43500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.789+0000 7f43581d6640 1 -- 172.21.15.1:0/1511297854 shutdown_connections 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.789+0000 7f43581d6640 1 -- 172.21.15.1:0/1511297854 wait complete. 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f43581d6640 1 Processor -- start 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f43581d6640 1 -- start start 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f43581d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350102da0 0x7f4350100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f43581d6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501036c0 0x7f4350100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f43581d6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435010c180 0x7f4350101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f43581d6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4350111f60 con 0x7f43501036c0 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f43581d6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4350111de0 con 0x7f4350102da0 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f43581d6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43501120e0 con 0x7f435010c180 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f4355771640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501036c0 0x7f4350100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f4355771640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501036c0 0x7f4350100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40398/0 (socket says 172.21.15.1:40398) 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f4355771640 1 -- 172.21.15.1:0/844222297 learned_addr learned my addr 172.21.15.1:0/844222297 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f4355f72640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350102da0 0x7f4350100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f4356773640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435010c180 0x7f4350101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f4355771640 1 -- 172.21.15.1:0/844222297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435010c180 msgr2=0x7f4350101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f4355771640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435010c180 0x7f4350101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f4355771640 1 -- 172.21.15.1:0/844222297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350102da0 msgr2=0x7f4350100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:51.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.790+0000 7f4355771640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350102da0 0x7f4350100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.791+0000 7f4355771640 1 -- 172.21.15.1:0/844222297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43501a6900 con 0x7f43501036c0 2024-09-16T00:36:51.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.791+0000 7f4356773640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435010c180 0x7f4350101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:51.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.791+0000 7f4355f72640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350102da0 0x7f4350100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:51.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.791+0000 7f4355771640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501036c0 0x7f4350100f40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f434000e9f0 tx=0x7f434000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:51.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.791+0000 7f4346ffd640 1 -- 172.21.15.1:0/844222297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434000cd80 con 0x7f43501036c0 2024-09-16T00:36:51.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.791+0000 7f4346ffd640 1 -- 172.21.15.1:0/844222297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4340010910 con 0x7f43501036c0 2024-09-16T00:36:51.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.791+0000 7f43581d6640 1 -- 172.21.15.1:0/844222297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43501a6bf0 con 0x7f43501036c0 2024-09-16T00:36:51.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.791+0000 7f4346ffd640 1 -- 172.21.15.1:0/844222297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340018990 con 0x7f43501036c0 2024-09-16T00:36:51.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.791+0000 7f43581d6640 1 -- 172.21.15.1:0/844222297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43501a7130 con 0x7f43501036c0 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.793+0000 7f4346ffd640 1 -- 172.21.15.1:0/844222297 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4340010ac0 con 0x7f43501036c0 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.793+0000 7f435a5af740 1 Processor -- start 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f435a5af740 1 -- start start 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f435a5af740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb789f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f435a5af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 0x557abb794800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f435a5af740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 0x557abb797130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f435a5af740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557abb6ef280 con 0x557abb789b60 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f435a5af740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557abb696b10 con 0x557abb794d40 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f435a5af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557abb6eb240 con 0x557abb78a480 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f4346ffd640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f4356773640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb789f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f4356773640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb789f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40412/0 (socket says 172.21.15.1:40412) 2024-09-16T00:36:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f4356773640 1 -- 172.21.15.1:0/107172575 learned_addr learned my addr 172.21.15.1:0/107172575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f4356773640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f4355771640 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 0x557abb797130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f4346ffd640 1 -- 172.21.15.1:0/844222297 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f43400d0160 con 0x7f43501036c0 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.794+0000 7f4355f72640 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 0x557abb794800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f4356773640 1 -- 172.21.15.1:0/107172575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 msgr2=0x557abb794800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f4356773640 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 0x557abb794800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f4356773640 1 -- 172.21.15.1:0/107172575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 msgr2=0x557abb797130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f4356773640 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 0x557abb797130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f4356773640 1 -- 172.21.15.1:0/107172575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557abb6a31f0 con 0x557abb789b60 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f4355771640 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 0x557abb797130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f4356773640 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb789f40 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f434802f4d0 tx=0x7f434802f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ad73c37c63cae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:51.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f4356773640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f434802f570 tx=0x7f4348041000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f433bfff640 1 -- 172.21.15.1:0/107172575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434804c070 con 0x557abb789b60 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f433bfff640 1 -- 172.21.15.1:0/107172575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4348003d70 con 0x557abb789b60 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.795+0000 7f433bfff640 1 -- 172.21.15.1:0/107172575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348034040 con 0x557abb789b60 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 -- 172.21.15.1:0/107172575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 msgr2=0x557abb789f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb789f40 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f434802f4d0 tx=0x7f434802f500 comp rx=0 tx=0).stop 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 -- 172.21.15.1:0/107172575 shutdown_connections 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 0x557abb797130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 0x557abb794800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 --2- 172.21.15.1:0/107172575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb789f40 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 -- 172.21.15.1:0/107172575 >> 172.21.15.1:0/107172575 conn(0x557abb7860f0 msgr2=0x557abb7864d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 -- 172.21.15.1:0/107172575 shutdown_connections 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 -- 172.21.15.1:0/107172575 wait complete. 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 -- start start 2024-09-16T00:36:51.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.796+0000 7f435a5af740 1 Processor -- start 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f435a5af740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb791c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f435a5af740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 0x557abb792190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f435a5af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb794d40 0x557abb7926d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f435a5af740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557abb797910 con 0x557abb789b60 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f435a5af740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557abb6a9f10 con 0x557abb78a480 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f435a5af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557abb797a90 con 0x557abb794d40 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f435a5af740 10 client.0 _ll_register_callbacks cb 0x557abb672a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4355f72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb791c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4355f72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb791c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40428/0 (socket says 172.21.15.1:40428) 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4355f72640 1 -- 172.21.15.1:0/2712053813 learned_addr learned my addr 172.21.15.1:0/2712053813 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4355771640 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 0x557abb792190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4356773640 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb794d40 0x557abb7926d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4355f72640 1 -- 172.21.15.1:0/2712053813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb794d40 msgr2=0x557abb7926d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:51.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4355f72640 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb794d40 0x557abb7926d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4355f72640 1 -- 172.21.15.1:0/2712053813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 msgr2=0x557abb792190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4355f72640 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 0x557abb792190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4355f72640 1 -- 172.21.15.1:0/2712053813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557abb4606a0 con 0x557abb789b60 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.797+0000 7f4356773640 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb794d40 0x557abb7926d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f4355f72640 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb791c50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f434c011410 tx=0x7f434c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa4dbc2e21eae41 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f43397fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f43397fa640 1 -- 172.21.15.1:0/2712053813 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c012920 con 0x557abb789b60 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f43397fa640 1 -- 172.21.15.1:0/2712053813 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f434c012ac0 con 0x557abb789b60 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f435a5af740 1 -- 172.21.15.1:0/2712053813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4348003d70 con 0x557abb789b60 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f43397fa640 1 -- 172.21.15.1:0/2712053813 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f434c025050 con 0x557abb789b60 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f435a5af740 10 client.17436 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f435a5af740 1 -- 172.21.15.1:0/2712053813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557abb7a35f0 con 0x557abb789b60 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f43397fa640 1 -- 172.21.15.1:0/2712053813 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4348003d70 con 0x557abb789b60 2024-09-16T00:36:51.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f43397fa640 1 -- 172.21.15.1:0/2712053813 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f434c00f040 con 0x557abb789b60 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f435a5af740 10 client.17436 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f435a5af740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f435a5af740 20 client.17436 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f435a5af740 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557abb7a4ab0 0x557abb7a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f435a5af740 4 client.17436 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.798+0000 7f435a5af740 1 -- 172.21.15.1:0/2712053813 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557abb6fc7f0 con 0x557abb7a4ab0 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.799+0000 7f4355771640 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557abb7a4ab0 0x557abb7a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.799+0000 7f4355771640 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557abb7a4ab0 0x557abb7a6f30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba47ca2a9536fc0d server_cookie=93241f4d84781760 in_seq=0 out_seq=0 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.799+0000 7f43397fa640 10 client.17436 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f43397fa640 1 -- 172.21.15.1:0/2712053813 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557abb6fc7f0 con 0x557abb7a4ab0 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f43397fa640 10 client.17436 handle_command_reply: tid=0 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 client.17436 shutdown 2024-09-16T00:36:51.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 -- 172.21.15.1:0/2712053813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 msgr2=0x557abb791c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb791c50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f434c011410 tx=0x7f434c011810 comp rx=0 tx=0).stop 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 -- 172.21.15.1:0/2712053813 shutdown_connections 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557abb7a4ab0 0x557abb7a6f30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb794d40 0x557abb7926d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 0x557abb792190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f43397fa640 0 client.17436 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 --2- 172.21.15.1:0/2712053813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb791c50 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 -- 172.21.15.1:0/2712053813 >> 172.21.15.1:0/2712053813 conn(0x557abb7860f0 msgr2=0x557abb785270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 -- 172.21.15.1:0/2712053813 shutdown_connections 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 1 -- 172.21.15.1:0/2712053813 wait complete. 2024-09-16T00:36:51.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.800+0000 7f435a5af740 20 client.17436 trim_cache size 0 max 16384 2024-09-16T00:36:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.817+0000 7f435a5af740 1 Processor -- start 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.817+0000 7f435a5af740 1 -- start start 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.817+0000 7f435a5af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb789b60 0x557abb78c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.817+0000 7f435a5af740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 0x557abb7ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.817+0000 7f435a5af740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb794d40 0x557abb7acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.817+0000 7f435a5af740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b3930 con 0x557abb794d40 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.817+0000 7f435a5af740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b1200 con 0x557abb78a480 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.817+0000 7f435a5af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43501af050 con 0x557abb789b60 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4356773640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb794d40 0x557abb7acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4356773640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb794d40 0x557abb7acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40442/0 (socket says 172.21.15.1:40442) 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4356773640 1 -- 172.21.15.1:0/1821181155 learned_addr learned my addr 172.21.15.1:0/1821181155 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4355771640 1 --2- 172.21.15.1:0/1821181155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 0x557abb7ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4355f72640 1 --2- 172.21.15.1:0/1821181155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb789b60 0x557abb78c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4356773640 1 -- 172.21.15.1:0/1821181155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb789b60 msgr2=0x557abb78c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4356773640 1 --2- 172.21.15.1:0/1821181155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb789b60 0x557abb78c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4356773640 1 -- 172.21.15.1:0/1821181155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 msgr2=0x557abb7ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4356773640 1 --2- 172.21.15.1:0/1821181155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 0x557abb7ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4356773640 1 -- 172.21.15.1:0/1821181155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557abb794520 con 0x557abb794d40 2024-09-16T00:36:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f4356773640 1 --2- 172.21.15.1:0/1821181155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb794d40 0x557abb7acd90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4348003ed0 tx=0x7f4348003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a85dcfbaac8384c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f433b7fe640 1 -- 172.21.15.1:0/1821181155 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434804c070 con 0x557abb794d40 2024-09-16T00:36:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f433b7fe640 1 -- 172.21.15.1:0/1821181155 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4348056070 con 0x557abb794d40 2024-09-16T00:36:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.818+0000 7f433b7fe640 1 -- 172.21.15.1:0/1821181155 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348034040 con 0x557abb794d40 2024-09-16T00:36:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 -- 172.21.15.1:0/1821181155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb794d40 msgr2=0x557abb7acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 --2- 172.21.15.1:0/1821181155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb794d40 0x557abb7acd90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4348003ed0 tx=0x7f4348003f00 comp rx=0 tx=0).stop 2024-09-16T00:36:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 -- 172.21.15.1:0/1821181155 shutdown_connections 2024-09-16T00:36:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 --2- 172.21.15.1:0/1821181155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb794d40 0x557abb7acd90 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 --2- 172.21.15.1:0/1821181155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb78a480 0x557abb7ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 --2- 172.21.15.1:0/1821181155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb789b60 0x557abb78c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 -- 172.21.15.1:0/1821181155 >> 172.21.15.1:0/1821181155 conn(0x557abb7860f0 msgr2=0x557abb793bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 -- 172.21.15.1:0/1821181155 shutdown_connections 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 -- 172.21.15.1:0/1821181155 wait complete. 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 -- start start 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.819+0000 7f435a5af740 1 Processor -- start 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f435a5af740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb7991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f435a5af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 0x557abb799700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f435a5af740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 0x557abb80e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f435a5af740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43501af6c0 con 0x557abb789b60 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f435a5af740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b0b30 con 0x557abb794d40 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f435a5af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43501afd90 con 0x557abb78a480 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f435a5af740 10 client.0 _ll_register_callbacks cb 0x7f434c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4355f72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb7991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4355f72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb7991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40448/0 (socket says 172.21.15.1:40448) 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4355f72640 1 -- 172.21.15.1:0/4054636191 learned_addr learned my addr 172.21.15.1:0/4054636191 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4355771640 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 0x557abb799700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4356773640 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 0x557abb80e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4355f72640 1 -- 172.21.15.1:0/4054636191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 msgr2=0x557abb799700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4355f72640 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 0x557abb799700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4355f72640 1 -- 172.21.15.1:0/4054636191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 msgr2=0x557abb80e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4355f72640 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 0x557abb80e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.820+0000 7f4355f72640 1 -- 172.21.15.1:0/4054636191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557abb80ed60 con 0x557abb789b60 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f4355f72640 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb7991c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f434c011d90 tx=0x7f434c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980d7d625b120d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f43397fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f43397fa640 1 -- 172.21.15.1:0/4054636191 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c012c30 con 0x557abb789b60 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f43397fa640 1 -- 172.21.15.1:0/4054636191 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f434c012dd0 con 0x557abb789b60 2024-09-16T00:36:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f435a5af740 1 -- 172.21.15.1:0/4054636191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4348056070 con 0x557abb789b60 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f43397fa640 1 -- 172.21.15.1:0/4054636191 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f434c01a3f0 con 0x557abb789b60 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f435a5af740 10 client.17442 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f435a5af740 1 -- 172.21.15.1:0/4054636191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557abb8100a0 con 0x557abb789b60 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f43397fa640 1 -- 172.21.15.1:0/4054636191 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4348056070 con 0x557abb789b60 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f43397fa640 1 -- 172.21.15.1:0/4054636191 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f434c019940 con 0x557abb789b60 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f435a5af740 10 client.17442 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f435a5af740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f435a5af740 20 client.17442 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f435a5af740 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557abb810810 0x557abb812c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f435a5af740 4 client.17442 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.821+0000 7f435a5af740 1 -- 172.21.15.1:0/4054636191 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557abb6f6d00 con 0x557abb810810 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.822+0000 7f4355771640 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557abb810810 0x557abb812c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.822+0000 7f4355771640 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557abb810810 0x557abb812c00 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7696189215f79216 server_cookie=30be337beb540b24 in_seq=0 out_seq=0 2024-09-16T00:36:51.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.822+0000 7f43397fa640 10 client.17442 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:51.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f43397fa640 1 -- 172.21.15.1:0/4054636191 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557abb6f6d00 con 0x557abb810810 2024-09-16T00:36:51.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f43397fa640 10 client.17442 handle_command_reply: tid=0 2024-09-16T00:36:51.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 client.17442 shutdown 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 -- 172.21.15.1:0/4054636191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 msgr2=0x557abb7991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb7991c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f434c011d90 tx=0x7f434c0099b0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 -- 172.21.15.1:0/4054636191 shutdown_connections 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557abb810810 0x557abb812c00 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557abb794d40 0x557abb80e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557abb78a480 0x557abb799700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 --2- 172.21.15.1:0/4054636191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557abb789b60 0x557abb7991c0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f43397fa640 0 client.17442 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 -- 172.21.15.1:0/4054636191 >> 172.21.15.1:0/4054636191 conn(0x557abb7860f0 msgr2=0x557abb79fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 -- 172.21.15.1:0/4054636191 shutdown_connections 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.823+0000 7f435a5af740 1 -- 172.21.15.1:0/4054636191 wait complete. 2024-09-16T00:36:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f435a5af740 20 client.17442 trim_cache size 0 max 16384 2024-09-16T00:36:51.829 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:36:51.829 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:36:51.829 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "54f53b81-2a99-4263-b609-46a417a825cd", 2024-09-16T00:36:51.829 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:36:51.829 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:36:51.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 -- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 msgr2=0x7f433c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:51.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f434802f570 tx=0x7f4348041000 comp rx=0 tx=0).stop 2024-09-16T00:36:51.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 -- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501036c0 msgr2=0x7f4350100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:51.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501036c0 0x7f4350100f40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f434000e9f0 tx=0x7f434000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 -- 172.21.15.1:0/844222297 shutdown_connections 2024-09-16T00:36:51.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f433c0a3240 0x7f433c0a5700 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435010c180 0x7f4350101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43501036c0 0x7f4350100f40 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 --2- 172.21.15.1:0/844222297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350102da0 0x7f4350100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:51.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 -- 172.21.15.1:0/844222297 >> 172.21.15.1:0/844222297 conn(0x7f43500fc7a0 msgr2=0x7f43500fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:51.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 -- 172.21.15.1:0/844222297 shutdown_connections 2024-09-16T00:36:51.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:51.824+0000 7f43581d6640 1 -- 172.21.15.1:0/844222297 wait complete. 2024-09-16T00:36:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:52 smithi019 ceph-mon[35186]: pgmap v604: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-09-16T00:36:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:52 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:36:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:52 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:52 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:36:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:52 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:52.419 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:52 smithi001 ceph-mon[30934]: pgmap v604: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-09-16T00:36:52.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:52 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:36:52.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:52 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:52.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:52 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:36:52.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:52 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:52 smithi153 ceph-mon[35538]: pgmap v604: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-09-16T00:36:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:52 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:36:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:52 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:52 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:36:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:52 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:36:52.589 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:36:52.847 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:53.246 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:53 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:36:53.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.330+0000 7f469dbd4640 1 -- 172.21.15.1:0/2412564657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4698102d30 msgr2=0x7f4698103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:53.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.330+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2412564657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4698102d30 0x7f4698103110 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4688009930 tx=0x7f468802a420 comp rx=0 tx=0).stop 2024-09-16T00:36:53.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.330+0000 7f469dbd4640 1 -- 172.21.15.1:0/2412564657 shutdown_connections 2024-09-16T00:36:53.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.330+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2412564657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46981082a0 0x7f469810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.330+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2412564657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981036e0 0x7f4698107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.330+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2412564657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4698102d30 0x7f4698103110 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.330+0000 7f469dbd4640 1 -- 172.21.15.1:0/2412564657 >> 172.21.15.1:0/2412564657 conn(0x7f46980fc7d0 msgr2=0x7f46980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.330+0000 7f469dbd4640 1 -- 172.21.15.1:0/2412564657 shutdown_connections 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.330+0000 7f469dbd4640 1 -- 172.21.15.1:0/2412564657 wait complete. 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f469dbd4640 1 Processor -- start 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f469dbd4640 1 -- start start 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f469dbd4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102d30 0x7f4698100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f469dbd4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f469dbd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981082a0 0x7f4698101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f469dbd4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46981120a0 con 0x7f46981036e0 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f469dbd4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4698111f20 con 0x7f46981082a0 2024-09-16T00:36:53.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f469dbd4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4698112220 con 0x7f4698102d30 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f4696ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f4696ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40470/0 (socket says 172.21.15.1:40470) 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f4696ffd640 1 -- 172.21.15.1:0/2764045816 learned_addr learned my addr 172.21.15.1:0/2764045816 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f4697fff640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981082a0 0x7f4698101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f46977fe640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102d30 0x7f4698100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f4696ffd640 1 -- 172.21.15.1:0/2764045816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102d30 msgr2=0x7f4698100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f4696ffd640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102d30 0x7f4698100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f4696ffd640 1 -- 172.21.15.1:0/2764045816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981082a0 msgr2=0x7f4698101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f4696ffd640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981082a0 0x7f4698101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f4696ffd640 1 -- 172.21.15.1:0/2764045816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46981a6a80 con 0x7f46981036e0 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.331+0000 7f46977fe640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102d30 0x7f4698100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.332+0000 7f4696ffd640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f468c00d740 tx=0x7f468c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.332+0000 7f4694ff9640 1 -- 172.21.15.1:0/2764045816 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c017070 con 0x7f46981036e0 2024-09-16T00:36:53.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.332+0000 7f469dbd4640 1 -- 172.21.15.1:0/2764045816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46981a6d70 con 0x7f46981036e0 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.332+0000 7f4694ff9640 1 -- 172.21.15.1:0/2764045816 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c00de50 con 0x7f46981036e0 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.332+0000 7f4694ff9640 1 -- 172.21.15.1:0/2764045816 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c00f6b0 con 0x7f46981036e0 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.332+0000 7f469dbd4640 1 -- 172.21.15.1:0/2764045816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46981a7270 con 0x7f46981036e0 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.334+0000 7f4694ff9640 1 -- 172.21.15.1:0/2764045816 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f468c01a070 con 0x7f46981036e0 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.334+0000 7f469ffad740 1 Processor -- start 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.334+0000 7f469ffad740 1 -- start start 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f469ffad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 0x55a961e64880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f469ffad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 0x55a961e6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f469ffad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e6f680 0x55a961e71a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f469ffad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a961dca2e0 con 0x55a961e6f680 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f469ffad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a961d71ce0 con 0x55a961e644a0 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f469ffad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a961dc62a0 con 0x55a961e64dc0 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4696ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e6f680 0x55a961e71a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4696ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e6f680 0x55a961e71a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40478/0 (socket says 172.21.15.1:40478) 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4697fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 0x55a961e64880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4696ffd640 1 -- 172.21.15.1:0/385787652 learned_addr learned my addr 172.21.15.1:0/385787652 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f46977fe640 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 0x55a961e6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4694ff9640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46800a32c0 0x7f46800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4697fff640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46800a32c0 0x7f46800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4694ff9640 1 -- 172.21.15.1:0/2764045816 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f468c0cfa30 con 0x7f46981036e0 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4696ffd640 1 -- 172.21.15.1:0/385787652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 msgr2=0x55a961e6f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4696ffd640 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 0x55a961e6f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4696ffd640 1 -- 172.21.15.1:0/385787652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 msgr2=0x55a961e64880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4696ffd640 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 0x55a961e64880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4696ffd640 1 -- 172.21.15.1:0/385787652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a961d56440 con 0x55a961e6f680 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f46977fe640 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 0x55a961e6f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.335+0000 7f4697fff640 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 0x55a961e64880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:53.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.336+0000 7f4696ffd640 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e6f680 0x55a961e71a70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f468c09b950 tx=0x7f468c005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9f49c106ca6d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.336+0000 7f467dffb640 1 -- 172.21.15.1:0/385787652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c021070 con 0x55a961e6f680 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.336+0000 7f4697fff640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46800a32c0 0x7f46800a5780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4690013a30 tx=0x7f4690004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.336+0000 7f467dffb640 1 -- 172.21.15.1:0/385787652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c09bc50 con 0x55a961e6f680 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.336+0000 7f467dffb640 1 -- 172.21.15.1:0/385787652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c0a6330 con 0x55a961e6f680 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.336+0000 7f469ffad740 1 -- 172.21.15.1:0/385787652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e6f680 msgr2=0x55a961e71a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.336+0000 7f469ffad740 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e6f680 0x55a961e71a70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f468c09b950 tx=0x7f468c005cc0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.336+0000 7f469ffad740 1 -- 172.21.15.1:0/385787652 shutdown_connections 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e6f680 0x55a961e71a70 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 0x55a961e6f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 --2- 172.21.15.1:0/385787652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 0x55a961e64880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 -- 172.21.15.1:0/385787652 >> 172.21.15.1:0/385787652 conn(0x55a961e60a30 msgr2=0x55a961e60e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 -- 172.21.15.1:0/385787652 shutdown_connections 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 -- 172.21.15.1:0/385787652 wait complete. 2024-09-16T00:36:53.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 -- start start 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 Processor -- start 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e6efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 0x55a961e6c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6f680 0x55a961e6c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a961e72250 con 0x55a961e644a0 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a961d85050 con 0x55a961e6f680 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.337+0000 7f469ffad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a961e723d0 con 0x55a961e64dc0 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f469ffad740 10 client.0 _ll_register_callbacks cb 0x55a961dc5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f46977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e6efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f46977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e6efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40480/0 (socket says 172.21.15.1:40480) 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f46977fe640 1 -- 172.21.15.1:0/2813845132 learned_addr learned my addr 172.21.15.1:0/2813845132 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f4696ffd640 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 0x55a961e6c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f4697fff640 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6f680 0x55a961e6c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f46977fe640 1 -- 172.21.15.1:0/2813845132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 msgr2=0x55a961e6c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f46977fe640 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 0x55a961e6c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f46977fe640 1 -- 172.21.15.1:0/2813845132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6f680 msgr2=0x55a961e6c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f46977fe640 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6f680 0x55a961e6c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f46977fe640 1 -- 172.21.15.1:0/2813845132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a961b3b3c0 con 0x55a961e644a0 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f46977fe640 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e6efc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4688009900 tx=0x7f4688031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ed2306db796d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f4676ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f4676ffd640 1 -- 172.21.15.1:0/2813845132 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688035040 con 0x55a961e644a0 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f469ffad740 1 -- 172.21.15.1:0/2813845132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468c09bc50 con 0x55a961e644a0 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f4676ffd640 1 -- 172.21.15.1:0/2813845132 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4688004240 con 0x55a961e644a0 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f4676ffd640 1 -- 172.21.15.1:0/2813845132 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f468804c050 con 0x55a961e644a0 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f469ffad740 10 client.17454 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.338+0000 7f469ffad740 1 -- 172.21.15.1:0/2813845132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a961e7dc00 con 0x55a961e644a0 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.339+0000 7f4676ffd640 1 -- 172.21.15.1:0/2813845132 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c09bc50 con 0x55a961e644a0 2024-09-16T00:36:53.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.339+0000 7f4676ffd640 1 -- 172.21.15.1:0/2813845132 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4688004b10 con 0x55a961e644a0 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.339+0000 7f469ffad740 10 client.17454 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.339+0000 7f469ffad740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.339+0000 7f469ffad740 20 client.17454 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.339+0000 7f469ffad740 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a961e7f0c0 0x55a961e81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.339+0000 7f469ffad740 4 client.17454 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.339+0000 7f469ffad740 1 -- 172.21.15.1:0/2813845132 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a961dd7850 con 0x55a961e7f0c0 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.339+0000 7f4696ffd640 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a961e7f0c0 0x55a961e81510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.340+0000 7f4676ffd640 10 client.17454 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.340+0000 7f4696ffd640 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a961e7f0c0 0x55a961e81510 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b942ca043ab17e06 server_cookie=af01394db6aff99a in_seq=0 out_seq=0 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.340+0000 7f4676ffd640 1 -- 172.21.15.1:0/2813845132 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a961dd7850 con 0x55a961e7f0c0 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.340+0000 7f4676ffd640 10 client.17454 handle_command_reply: tid=0 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.340+0000 7f469ffad740 1 client.17454 shutdown 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.340+0000 7f469ffad740 1 -- 172.21.15.1:0/2813845132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 msgr2=0x55a961e6efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:53.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.340+0000 7f469ffad740 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e6efc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4688009900 tx=0x7f4688031850 comp rx=0 tx=0).stop 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f469ffad740 1 -- 172.21.15.1:0/2813845132 shutdown_connections 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f469ffad740 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a961e7f0c0 0x55a961e81510 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f469ffad740 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6f680 0x55a961e6c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f469ffad740 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e64dc0 0x55a961e6c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f469ffad740 1 --2- 172.21.15.1:0/2813845132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e6efc0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f469ffad740 1 -- 172.21.15.1:0/2813845132 >> 172.21.15.1:0/2813845132 conn(0x55a961e60a30 msgr2=0x55a961e66710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f4676ffd640 0 client.17454 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f469ffad740 1 -- 172.21.15.1:0/2813845132 shutdown_connections 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f469ffad740 1 -- 172.21.15.1:0/2813845132 wait complete. 2024-09-16T00:36:53.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.341+0000 7f469ffad740 20 client.17454 trim_cache size 0 max 16384 2024-09-16T00:36:53.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:53 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:36:53.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.358+0000 7f469ffad740 1 Processor -- start 2024-09-16T00:36:53.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.358+0000 7f469ffad740 1 -- start start 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.358+0000 7f469ffad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e6c910 0x55a961e7b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.358+0000 7f469ffad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e60a30 0x55a961e86e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.358+0000 7f469ffad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 0x55a961e87370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.358+0000 7f469ffad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46981b3a40 con 0x55a961e60a30 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.358+0000 7f469ffad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46981b1310 con 0x55a961e644a0 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.358+0000 7f469ffad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46981af160 con 0x55a961e6c910 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4696ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e60a30 0x55a961e86e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4696ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e60a30 0x55a961e86e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40482/0 (socket says 172.21.15.1:40482) 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4696ffd640 1 -- 172.21.15.1:0/606768386 learned_addr learned my addr 172.21.15.1:0/606768386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4697fff640 1 --2- 172.21.15.1:0/606768386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 0x55a961e87370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f46977fe640 1 --2- 172.21.15.1:0/606768386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e6c910 0x55a961e7b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4696ffd640 1 -- 172.21.15.1:0/606768386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e6c910 msgr2=0x55a961e7b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:53.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4696ffd640 1 --2- 172.21.15.1:0/606768386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e6c910 0x55a961e7b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4696ffd640 1 -- 172.21.15.1:0/606768386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 msgr2=0x55a961e87370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:53.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4696ffd640 1 --2- 172.21.15.1:0/606768386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 0x55a961e87370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4696ffd640 1 -- 172.21.15.1:0/606768386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a961e6f2f0 con 0x55a961e60a30 2024-09-16T00:36:53.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f4696ffd640 1 --2- 172.21.15.1:0/606768386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e60a30 0x55a961e86e30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f468c0980b0 tx=0x7f468c010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefc9b9958a7478e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:53.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f467d7fa640 1 -- 172.21.15.1:0/606768386 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c0b1070 con 0x55a961e60a30 2024-09-16T00:36:53.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f467d7fa640 1 -- 172.21.15.1:0/606768386 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c0a48c0 con 0x55a961e60a30 2024-09-16T00:36:53.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.359+0000 7f467d7fa640 1 -- 172.21.15.1:0/606768386 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c021040 con 0x55a961e60a30 2024-09-16T00:36:53.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 -- 172.21.15.1:0/606768386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e60a30 msgr2=0x55a961e86e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:53.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 --2- 172.21.15.1:0/606768386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e60a30 0x55a961e86e30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f468c0980b0 tx=0x7f468c010d40 comp rx=0 tx=0).stop 2024-09-16T00:36:53.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 -- 172.21.15.1:0/606768386 shutdown_connections 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 --2- 172.21.15.1:0/606768386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e644a0 0x55a961e87370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 --2- 172.21.15.1:0/606768386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e60a30 0x55a961e86e30 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 --2- 172.21.15.1:0/606768386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e6c910 0x55a961e7b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 -- 172.21.15.1:0/606768386 >> 172.21.15.1:0/606768386 conn(0x55a961e67cb0 msgr2=0x55a961e6ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 -- 172.21.15.1:0/606768386 shutdown_connections 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 -- 172.21.15.1:0/606768386 wait complete. 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.360+0000 7f469ffad740 1 -- start start 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f469ffad740 1 Processor -- start 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f469ffad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e60a30 0x55a961e73610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f469ffad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e73b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f469ffad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6c910 0x55a961e74090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f469ffad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46981af7d0 con 0x55a961e644a0 2024-09-16T00:36:53.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f469ffad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46981b0c40 con 0x55a961e6c910 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f469ffad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46981afea0 con 0x55a961e60a30 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f469ffad740 10 client.0 _ll_register_callbacks cb 0x7f4688009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f4696ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e73b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f4696ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e73b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40492/0 (socket says 172.21.15.1:40492) 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f4696ffd640 1 -- 172.21.15.1:0/49958380 learned_addr learned my addr 172.21.15.1:0/49958380 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f46977fe640 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e60a30 0x55a961e73610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f4697fff640 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6c910 0x55a961e74090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f4696ffd640 1 -- 172.21.15.1:0/49958380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e60a30 msgr2=0x55a961e73610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.361+0000 7f4696ffd640 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e60a30 0x55a961e73610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4696ffd640 1 -- 172.21.15.1:0/49958380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6c910 msgr2=0x55a961e74090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4696ffd640 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6c910 0x55a961e74090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4696ffd640 1 -- 172.21.15.1:0/49958380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a961ee90a0 con 0x55a961e644a0 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4696ffd640 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e73b50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f468c09b950 tx=0x7f468c09c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c945ea36f58b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:53.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4676ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4676ffd640 1 -- 172.21.15.1:0/49958380 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c0b1070 con 0x55a961e644a0 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4676ffd640 1 -- 172.21.15.1:0/49958380 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f468c0a6330 con 0x55a961e644a0 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f469ffad740 1 -- 172.21.15.1:0/49958380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468c0a48c0 con 0x55a961e644a0 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4676ffd640 1 -- 172.21.15.1:0/49958380 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f468c021070 con 0x55a961e644a0 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f469ffad740 10 client.17460 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f469ffad740 1 -- 172.21.15.1:0/49958380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a961eea390 con 0x55a961e644a0 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4676ffd640 1 -- 172.21.15.1:0/49958380 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c0a48c0 con 0x55a961e644a0 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f4676ffd640 1 -- 172.21.15.1:0/49958380 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f468c0a4430 con 0x55a961e644a0 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f469ffad740 10 client.17460 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f469ffad740 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f469ffad740 20 client.17460 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f469ffad740 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a961e64dc0 0x55a961eec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f469ffad740 4 client.17460 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:36:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.362+0000 7f469ffad740 1 -- 172.21.15.1:0/49958380 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a961dd1d60 con 0x55a961e64dc0 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f46977fe640 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a961e64dc0 0x55a961eec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f46977fe640 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a961e64dc0 0x55a961eec620 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ea6d5c4fb9f641a server_cookie=b80ae320456212de in_seq=0 out_seq=0 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f4676ffd640 10 client.17460 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f4676ffd640 1 -- 172.21.15.1:0/49958380 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a961dd1d60 con 0x55a961e64dc0 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f4676ffd640 10 client.17460 handle_command_reply: tid=0 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f469ffad740 1 client.17460 shutdown 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f469ffad740 1 -- 172.21.15.1:0/49958380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 msgr2=0x55a961e73b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f469ffad740 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e73b50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f468c09b950 tx=0x7f468c09c490 comp rx=0 tx=0).stop 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f469ffad740 1 -- 172.21.15.1:0/49958380 shutdown_connections 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f469ffad740 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a961e64dc0 0x55a961eec620 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.363+0000 7f469ffad740 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a961e6c910 0x55a961e74090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469ffad740 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a961e644a0 0x55a961e73b50 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f4676ffd640 0 client.17460 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:36:53.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469ffad740 1 --2- 172.21.15.1:0/49958380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a961e60a30 0x55a961e73610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469ffad740 1 -- 172.21.15.1:0/49958380 >> 172.21.15.1:0/49958380 conn(0x55a961e67cb0 msgr2=0x55a961e5f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:53.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469ffad740 1 -- 172.21.15.1:0/49958380 shutdown_connections 2024-09-16T00:36:53.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469ffad740 1 -- 172.21.15.1:0/49958380 wait complete. 2024-09-16T00:36:53.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469ffad740 20 client.17460 trim_cache size 0 max 16384 2024-09-16T00:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:36:53.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 -- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46800a32c0 msgr2=0x7f46800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:53.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46800a32c0 0x7f46800a5780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4690013a30 tx=0x7f4690004ab0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 -- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46981036e0 msgr2=0x7f4698100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:53.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f468c00d740 tx=0x7f468c00dc00 comp rx=0 tx=0).stop 2024-09-16T00:36:53.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 -- 172.21.15.1:0/2764045816 shutdown_connections 2024-09-16T00:36:53.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981082a0 0x7f4698101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46800a32c0 0x7f46800a5780 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 --2- 172.21.15.1:0/2764045816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102d30 0x7f4698100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:53.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 -- 172.21.15.1:0/2764045816 >> 172.21.15.1:0/2764045816 conn(0x7f46980fc7d0 msgr2=0x7f4698105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:53.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 -- 172.21.15.1:0/2764045816 shutdown_connections 2024-09-16T00:36:53.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:53.364+0000 7f469dbd4640 1 -- 172.21.15.1:0/2764045816 wait complete. 2024-09-16T00:36:53.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:53 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:36:54.266 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:36:54.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:36:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:54 smithi019 ceph-mon[35186]: pgmap v605: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:36:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:54 smithi001 ceph-mon[30934]: pgmap v605: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:36:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:54 smithi153 ceph-mon[35538]: pgmap v605: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:36:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.031+0000 7fc7a5f75640 1 -- 172.21.15.1:0/973870636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0100e20 msgr2=0x7fc7a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.031+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/973870636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0100e20 0x7fc7a0101200 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc790009900 tx=0x7fc79002a480 comp rx=0 tx=0).stop 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.032+0000 7fc7a5f75640 1 -- 172.21.15.1:0/973870636 shutdown_connections 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.032+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/973870636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7a0102ef0 0x7fc7a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.032+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/973870636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a0102570 0x7fc7a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.032+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/973870636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0100e20 0x7fc7a0101200 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.032+0000 7fc7a5f75640 1 -- 172.21.15.1:0/973870636 >> 172.21.15.1:0/973870636 conn(0x7fc7a00fc7d0 msgr2=0x7fc7a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.032+0000 7fc7a5f75640 1 -- 172.21.15.1:0/973870636 shutdown_connections 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.032+0000 7fc7a5f75640 1 -- 172.21.15.1:0/973870636 wait complete. 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.032+0000 7fc7a5f75640 1 Processor -- start 2024-09-16T00:36:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.032+0000 7fc7a5f75640 1 -- start start 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc7a5f75640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a0100e20 0x7fc7a0103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc7a5f75640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7a0102570 0x7fc7a0103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc7a5f75640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0102ef0 0x7fc7a01041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc7a5f75640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0110f80 con 0x7fc7a0102ef0 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc7a5f75640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0110e00 con 0x7fc7a0102570 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc7a5f75640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0111100 con 0x7fc7a0100e20 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0102ef0 0x7fc7a01041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0102ef0 0x7fc7a01041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40520/0 (socket says 172.21.15.1:40520) 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79ffff640 1 -- 172.21.15.1:0/4060062718 learned_addr learned my addr 172.21.15.1:0/4060062718 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79effd640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7a0102570 0x7fc7a0103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79f7fe640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a0100e20 0x7fc7a0103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79ffff640 1 -- 172.21.15.1:0/4060062718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a0100e20 msgr2=0x7fc7a0103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79ffff640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a0100e20 0x7fc7a0103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79ffff640 1 -- 172.21.15.1:0/4060062718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7a0102570 msgr2=0x7fc7a0103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79ffff640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7a0102570 0x7fc7a0103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79ffff640 1 -- 172.21.15.1:0/4060062718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a01a6a30 con 0x7fc7a0102ef0 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79effd640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7a0102570 0x7fc7a0103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79ffff640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0102ef0 0x7fc7a01041d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc79800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.033+0000 7fc79cff9640 1 -- 172.21.15.1:0/4060062718 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798017070 con 0x7fc7a0102ef0 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.034+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a01a6d20 con 0x7fc7a0102ef0 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.034+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a01a7260 con 0x7fc7a0102ef0 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.034+0000 7fc79cff9640 1 -- 172.21.15.1:0/4060062718 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc798003e50 con 0x7fc7a0102ef0 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.034+0000 7fc79cff9640 1 -- 172.21.15.1:0/4060062718 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798020060 con 0x7fc7a0102ef0 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.035+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc780005290 con 0x7fc7a0102ef0 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.037+0000 7fc79cff9640 1 -- 172.21.15.1:0/4060062718 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc798003970 con 0x7fc7a0102ef0 2024-09-16T00:36:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.037+0000 7fc79cff9640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc78c0a3240 0x7fc78c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.037+0000 7fc79cff9640 1 -- 172.21.15.1:0/4060062718 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc7980cf880 con 0x7fc7a0102ef0 2024-09-16T00:36:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.038+0000 7fc79f7fe640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc78c0a3240 0x7fc78c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.038+0000 7fc79f7fe640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc78c0a3240 0x7fc78c0a5700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc790009230 tx=0x7fc7900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.041+0000 7fc79cff9640 1 -- 172.21.15.1:0/4060062718 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc79809bb50 con 0x7fc7a0102ef0 2024-09-16T00:36:55.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.253+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc780005580 con 0x7fc7a0102ef0 2024-09-16T00:36:55.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.254+0000 7fc79cff9640 1 -- 172.21.15.1:0/4060062718 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fc79809fe50 con 0x7fc7a0102ef0 2024-09-16T00:36:55.255 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:36:55.255 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:36:55.255 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:36:55.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc78c0a3240 msgr2=0x7fc78c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:55.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc78c0a3240 0x7fc78c0a5700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc790009230 tx=0x7fc7900058d0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0102ef0 msgr2=0x7fc7a01041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:55.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0102ef0 0x7fc7a01041d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc79800b260 comp rx=0 tx=0).stop 2024-09-16T00:36:55.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 shutdown_connections 2024-09-16T00:36:55.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0102ef0 0x7fc7a01041d0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7a0102570 0x7fc7a0103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc78c0a3240 0x7fc78c0a5700 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 --2- 172.21.15.1:0/4060062718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a0100e20 0x7fc7a0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:55.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 >> 172.21.15.1:0/4060062718 conn(0x7fc7a00fc7d0 msgr2=0x7fc7a006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:55.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 shutdown_connections 2024-09-16T00:36:55.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:55.256+0000 7fc7a5f75640 1 -- 172.21.15.1:0/4060062718 wait complete. 2024-09-16T00:36:56.049 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:36:56.319 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:56 smithi019 ceph-mon[35186]: pgmap v606: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-09-16T00:36:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:56 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4060062718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:56 smithi001 ceph-mon[30934]: pgmap v606: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-09-16T00:36:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:56 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4060062718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:56 smithi153 ceph-mon[35538]: pgmap v606: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-09-16T00:36:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:56 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4060062718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:36:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.828+0000 7fd9e77a9640 1 -- 172.21.15.1:0/3399050905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 msgr2=0x7fd9e01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.828+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/3399050905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 0x7fd9e01067e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009900 tx=0x7fd9d402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.828+0000 7fd9e77a9640 1 -- 172.21.15.1:0/3399050905 shutdown_connections 2024-09-16T00:36:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.828+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/3399050905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9e0106f10 0x7fd9e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.828+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/3399050905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 0x7fd9e01067e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.828+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/3399050905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e0101de0 0x7fd9e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.828+0000 7fd9e77a9640 1 -- 172.21.15.1:0/3399050905 >> 172.21.15.1:0/3399050905 conn(0x7fd9e00fd8c0 msgr2=0x7fd9e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 -- 172.21.15.1:0/3399050905 shutdown_connections 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 -- 172.21.15.1:0/3399050905 wait complete. 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 Processor -- start 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 -- start start 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9e0101de0 0x7fd9e00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 0x7fd9e0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e0106f10 0x7fd9e0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0111f30 con 0x7fd9e0102790 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0111db0 con 0x7fd9e0101de0 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.829+0000 7fd9e77a9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01120b0 con 0x7fd9e0106f10 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e4d44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 0x7fd9e0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e4d44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 0x7fd9e0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40530/0 (socket says 172.21.15.1:40530) 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e4d44640 1 -- 172.21.15.1:0/1923639499 learned_addr learned my addr 172.21.15.1:0/1923639499 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e5545640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9e0101de0 0x7fd9e00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e0106f10 0x7fd9e0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e4d44640 1 -- 172.21.15.1:0/1923639499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e0106f10 msgr2=0x7fd9e0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e0106f10 0x7fd9e0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e4d44640 1 -- 172.21.15.1:0/1923639499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9e0101de0 msgr2=0x7fd9e00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9e0101de0 0x7fd9e00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e4d44640 1 -- 172.21.15.1:0/1923639499 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e01a67d0 con 0x7fd9e0102790 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e0106f10 0x7fd9e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 0x7fd9e0074a30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005ab0 tx=0x7fd9d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9de7fc640 1 -- 172.21.15.1:0/1923639499 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d403f030 con 0x7fd9e0102790 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9de7fc640 1 -- 172.21.15.1:0/1923639499 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d4003ce0 con 0x7fd9e0102790 2024-09-16T00:36:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e77a9640 1 -- 172.21.15.1:0/1923639499 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e01a6a60 con 0x7fd9e0102790 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9de7fc640 1 -- 172.21.15.1:0/1923639499 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4035070 con 0x7fd9e0102790 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.830+0000 7fd9e77a9640 1 -- 172.21.15.1:0/1923639499 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e01a6f40 con 0x7fd9e0102790 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.833+0000 7fd9de7fc640 1 -- 172.21.15.1:0/1923639499 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd9d4042030 con 0x7fd9e0102790 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e9b82740 1 Processor -- start 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e9b82740 1 -- start start 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbf7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 0x55bf0fbf7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 0x55bf0fc04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bf0fb5d2f0 con 0x55bf0fbf8b40 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bf0fb04cf0 con 0x55bf0fc02370 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf0fb592b0 con 0x55bf0fbf75d0 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e5d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbf7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e5d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbf7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40540/0 (socket says 172.21.15.1:40540) 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e5d46640 1 -- 172.21.15.1:0/621774200 learned_addr learned my addr 172.21.15.1:0/621774200 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 0x55bf0fc04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e5545640 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 0x55bf0fbf7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9de7fc640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9c80a3240 0x7fd9c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.834+0000 7fd9e5d46640 1 -- 172.21.15.1:0/621774200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 msgr2=0x55bf0fbf7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 0x55bf0fbf7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9e5d46640 1 -- 172.21.15.1:0/621774200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 msgr2=0x55bf0fc04760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9de7fc640 1 -- 172.21.15.1:0/1923639499 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd9d40f06b0 con 0x7fd9e0102790 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 0x55bf0fc04760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9e5d46640 1 -- 172.21.15.1:0/621774200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0fae93a0 con 0x55bf0fbf8b40 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 0x55bf0fc04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9c80a3240 0x7fd9c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbf7090 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800bc80 tx=0x7fd9d800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedb50ccc5c07f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9c77fe640 1 -- 172.21.15.1:0/621774200 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8019410 con 0x55bf0fbf8b40 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9c77fe640 1 -- 172.21.15.1:0/621774200 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d8019a90 con 0x55bf0fbf8b40 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.835+0000 7fd9c77fe640 1 -- 172.21.15.1:0/621774200 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800e8b0 con 0x55bf0fbf8b40 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 -- 172.21.15.1:0/621774200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 msgr2=0x55bf0fbf7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbf7090 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800bc80 tx=0x7fd9d800d710 comp rx=0 tx=0).stop 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9c80a3240 0x7fd9c80a5700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd9d801f000 tx=0x7fd9d8020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 -- 172.21.15.1:0/621774200 shutdown_connections 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 0x55bf0fc04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 0x55bf0fbf7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/621774200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbf7090 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 -- 172.21.15.1:0/621774200 >> 172.21.15.1:0/621774200 conn(0x55bf0fbc0310 msgr2=0x55bf0fbf3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 -- 172.21.15.1:0/621774200 shutdown_connections 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 -- 172.21.15.1:0/621774200 wait complete. 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 -- start start 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.836+0000 7fd9e9b82740 1 Processor -- start 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc01d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fc02370 0x55bf0fbff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bf0fc05070 con 0x55bf0fbf8b40 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bf0fb18060 con 0x55bf0fbf75d0 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf0fc051f0 con 0x55bf0fc02370 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e9b82740 10 client.0 _ll_register_callbacks cb 0x55bf0f86b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e4d44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e4d44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbff020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40552/0 (socket says 172.21.15.1:40552) 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e4d44640 1 -- 172.21.15.1:0/980848267 learned_addr learned my addr 172.21.15.1:0/980848267 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e5545640 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc01d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fc02370 0x55bf0fbff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e4d44640 1 -- 172.21.15.1:0/980848267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fc02370 msgr2=0x55bf0fbff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fc02370 0x55bf0fbff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e4d44640 1 -- 172.21.15.1:0/980848267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf75d0 msgr2=0x55bf0fc01d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc01d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e4d44640 1 -- 172.21.15.1:0/980848267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0f8ce3c0 con 0x55bf0fbf8b40 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbff020 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005940 tx=0x7fd9d403b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a800f4990d9615b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.837+0000 7fd9c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/980848267 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4049070 con 0x55bf0fbf8b40 2024-09-16T00:36:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/980848267 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd9d40b8f40 con 0x55bf0fbf8b40 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e9b82740 1 -- 172.21.15.1:0/980848267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d8019a90 con 0x55bf0fbf8b40 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/980848267 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9d40bc230 con 0x55bf0fbf8b40 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e9b82740 10 client.17478 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e9b82740 1 -- 172.21.15.1:0/980848267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf0fc10b10 con 0x55bf0fbf8b40 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/980848267 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d8019a90 con 0x55bf0fbf8b40 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/980848267 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd9d403b460 con 0x55bf0fbf8b40 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e9b82740 10 client.17478 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e9b82740 10 client.17478 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e9b82740 20 client.17478 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bf0fc11e60 0x55bf0fc14290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e9b82740 4 client.17478 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e9b82740 1 -- 172.21.15.1:0/980848267 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf0fb6a860 con 0x55bf0fc11e60 2024-09-16T00:36:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.838+0000 7fd9e5545640 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bf0fc11e60 0x55bf0fc14290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.839+0000 7fd9c4ff9640 10 client.17478 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.839+0000 7fd9e5545640 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bf0fc11e60 0x55bf0fc14290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a5d558a81c1484a server_cookie=128896c2c0a1b208 in_seq=0 out_seq=0 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/980848267 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf0fb6a860 con 0x55bf0fc11e60 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9c4ff9640 10 client.17478 handle_command_reply: tid=0 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 client.17478 shutdown 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 -- 172.21.15.1:0/980848267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 msgr2=0x55bf0fbff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbff020 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005940 tx=0x7fd9d403b800 comp rx=0 tx=0).stop 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 -- 172.21.15.1:0/980848267 shutdown_connections 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fc02370 0x55bf0fbff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbff020 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bf0fc11e60 0x55bf0fc14290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/980848267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc01d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9c4ff9640 0 client.17478 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 -- 172.21.15.1:0/980848267 >> 172.21.15.1:0/980848267 conn(0x55bf0fbc0310 msgr2=0x55bf0fbf9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 -- 172.21.15.1:0/980848267 shutdown_connections 2024-09-16T00:36:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 1 -- 172.21.15.1:0/980848267 wait complete. 2024-09-16T00:36:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.840+0000 7fd9e9b82740 20 client.17478 trim_cache size 0 max 16384 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.853+0000 7fd9e9b82740 1 Processor -- start 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.853+0000 7fd9e9b82740 1 -- start start 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 0x55bf0fbf8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf8b40 0x55bf0fc0e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fc02370 0x55bf0fbfa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01b3740 con 0x55bf0fc02370 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01b1010 con 0x55bf0fbf8b40 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01aee60 con 0x55bf0fbf75d0 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fc02370 0x55bf0fbfa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fc02370 0x55bf0fbfa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40560/0 (socket says 172.21.15.1:40560) 2024-09-16T00:36:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5d46640 1 -- 172.21.15.1:0/2694923954 learned_addr learned my addr 172.21.15.1:0/2694923954 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5545640 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 0x55bf0fbf8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5d46640 1 -- 172.21.15.1:0/2694923954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 msgr2=0x55bf0fbf8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 0x55bf0fbf8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5d46640 1 -- 172.21.15.1:0/2694923954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf8b40 msgr2=0x55bf0fc0e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf8b40 0x55bf0fc0e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf8b40 0x55bf0fc0e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5d46640 1 -- 172.21.15.1:0/2694923954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0fc04f50 con 0x55bf0fc02370 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5545640 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 0x55bf0fbf8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fc02370 0x55bf0fbfa120 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c010 tx=0x7fd9d800bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29519904b30c795 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9c6ffd640 1 -- 172.21.15.1:0/2694923954 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800eb70 con 0x55bf0fc02370 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9c6ffd640 1 -- 172.21.15.1:0/2694923954 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d800e890 con 0x55bf0fc02370 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.854+0000 7fd9c6ffd640 1 -- 172.21.15.1:0/2694923954 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8004680 con 0x55bf0fc02370 2024-09-16T00:36:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 -- 172.21.15.1:0/2694923954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fc02370 msgr2=0x55bf0fbfa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fc02370 0x55bf0fbfa120 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c010 tx=0x7fd9d800bbd0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 -- 172.21.15.1:0/2694923954 shutdown_connections 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fc02370 0x55bf0fbfa120 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fbf8b40 0x55bf0fc0e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/2694923954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf75d0 0x55bf0fbf8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 -- 172.21.15.1:0/2694923954 >> 172.21.15.1:0/2694923954 conn(0x55bf0fbc0310 msgr2=0x55bf0fbeea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 -- 172.21.15.1:0/2694923954 shutdown_connections 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 -- 172.21.15.1:0/2694923954 wait complete. 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.855+0000 7fd9e9b82740 1 -- start start 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e9b82740 1 Processor -- start 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc0ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbfbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e9b82740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 0x55bf0fbfc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01af4d0 con 0x55bf0fbf75d0 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01b0940 con 0x55bf0fc02370 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e9b82740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01afba0 con 0x55bf0fbf8b40 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e9b82740 10 client.0 _ll_register_callbacks cb 0x7fd9d40b8450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e5545640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc0ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e5545640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc0ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40564/0 (socket says 172.21.15.1:40564) 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e5545640 1 -- 172.21.15.1:0/567033714 learned_addr learned my addr 172.21.15.1:0/567033714 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e5d46640 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 0x55bf0fbfc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbfbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e5545640 1 -- 172.21.15.1:0/567033714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf8b40 msgr2=0x55bf0fbfbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e5545640 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbfbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e5545640 1 -- 172.21.15.1:0/567033714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 msgr2=0x55bf0fbfc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.856+0000 7fd9e5545640 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 0x55bf0fbfc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9e5545640 1 -- 172.21.15.1:0/567033714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0fbfca50 con 0x55bf0fbf75d0 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9e5545640 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc0ab30 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x55bf0fc14aa0 tx=0x7fd9d0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dff77b5c63ca4a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/567033714 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0004d20 con 0x55bf0fbf75d0 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/567033714 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd9d001e640 con 0x55bf0fbf75d0 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9e9b82740 1 -- 172.21.15.1:0/567033714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf0fb64d70 con 0x55bf0fbf75d0 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/567033714 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9d002c050 con 0x55bf0fbf75d0 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9e9b82740 10 client.17484 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9e9b82740 1 -- 172.21.15.1:0/567033714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf0fc7cb00 con 0x55bf0fbf75d0 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/567033714 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bf0fb64d70 con 0x55bf0fbf75d0 2024-09-16T00:36:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.857+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/567033714 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd9d0025060 con 0x55bf0fbf75d0 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.858+0000 7fd9e9b82740 10 client.17484 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.858+0000 7fd9e9b82740 10 client.17484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.858+0000 7fd9e9b82740 20 client.17484 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.858+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bf0fc7d2b0 0x55bf0fc7f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.858+0000 7fd9e9b82740 4 client.17484 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.858+0000 7fd9e9b82740 1 -- 172.21.15.1:0/567033714 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf0fae1170 con 0x55bf0fc7d2b0 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.858+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bf0fc7d2b0 0x55bf0fc7f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.858+0000 7fd9e4d44640 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bf0fc7d2b0 0x55bf0fc7f6d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69867dd953482036 server_cookie=ded4af11bf144643 in_seq=0 out_seq=0 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.858+0000 7fd9c4ff9640 10 client.17484 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9c4ff9640 1 -- 172.21.15.1:0/567033714 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf0fae1170 con 0x55bf0fc7d2b0 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9c4ff9640 10 client.17484 handle_command_reply: tid=0 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 client.17484 shutdown 2024-09-16T00:36:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 -- 172.21.15.1:0/567033714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf75d0 msgr2=0x55bf0fc0ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc0ab30 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x55bf0fc14aa0 tx=0x7fd9d0004410 comp rx=0 tx=0).stop 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 -- 172.21.15.1:0/567033714 shutdown_connections 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bf0fc7d2b0 0x55bf0fc7f6d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf0fc02370 0x55bf0fbfc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf0fbf8b40 0x55bf0fbfbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 --2- 172.21.15.1:0/567033714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf0fbf75d0 0x55bf0fc0ab30 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 -- 172.21.15.1:0/567033714 >> 172.21.15.1:0/567033714 conn(0x55bf0fbc0310 msgr2=0x55bf0fc00880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9c4ff9640 0 client.17484 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 -- 172.21.15.1:0/567033714 shutdown_connections 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 1 -- 172.21.15.1:0/567033714 wait complete. 2024-09-16T00:36:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e9b82740 20 client.17484 trim_cache size 0 max 16384 2024-09-16T00:36:56.868 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:36:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e77a9640 1 -- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9c80a3240 msgr2=0x7fd9c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9c80a3240 0x7fd9c80a5700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd9d801f000 tx=0x7fd9d8020000 comp rx=0 tx=0).stop 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e77a9640 1 -- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 msgr2=0x7fd9e0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.859+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 0x7fd9e0074a30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005ab0 tx=0x7fd9d40042f0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.860+0000 7fd9e77a9640 1 -- 172.21.15.1:0/1923639499 shutdown_connections 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.860+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd9c80a3240 0x7fd9c80a5700 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.860+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e0106f10 0x7fd9e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.860+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e0102790 0x7fd9e0074a30 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.860+0000 7fd9e77a9640 1 --2- 172.21.15.1:0/1923639499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9e0101de0 0x7fd9e00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.860+0000 7fd9e77a9640 1 -- 172.21.15.1:0/1923639499 >> 172.21.15.1:0/1923639499 conn(0x7fd9e00fd8c0 msgr2=0x7fd9e00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.860+0000 7fd9e77a9640 1 -- 172.21.15.1:0/1923639499 shutdown_connections 2024-09-16T00:36:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:56.860+0000 7fd9e77a9640 1 -- 172.21.15.1:0/1923639499 wait complete. 2024-09-16T00:36:57.721 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:36:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:58.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:36:58 smithi019 ceph-mon[35186]: pgmap v607: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:36:58.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:36:58 smithi001 ceph-mon[30934]: pgmap v607: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:36:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:36:58 smithi153 ceph-mon[35538]: pgmap v607: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:36:58.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.523+0000 7fc7403d3640 1 -- 172.21.15.1:0/3471661617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 msgr2=0x7fc738104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.523+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3471661617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 0x7fc738104e80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc728009900 tx=0x7fc72802a410 comp rx=0 tx=0).stop 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.523+0000 7fc7403d3640 1 -- 172.21.15.1:0/3471661617 shutdown_connections 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.523+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3471661617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7381055b0 0x7fc73810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.523+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3471661617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 0x7fc738104e80 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.523+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3471661617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc738100480 0x7fc738100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.523+0000 7fc7403d3640 1 -- 172.21.15.1:0/3471661617 >> 172.21.15.1:0/3471661617 conn(0x7fc738077210 msgr2=0x7fc7380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.523+0000 7fc7403d3640 1 -- 172.21.15.1:0/3471661617 shutdown_connections 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.523+0000 7fc7403d3640 1 -- 172.21.15.1:0/3471661617 wait complete. 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.524+0000 7fc7403d3640 1 Processor -- start 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.524+0000 7fc7403d3640 1 -- start start 2024-09-16T00:36:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.524+0000 7fc7403d3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc738100480 0x7fc73810e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.524+0000 7fc7403d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 0x7fc73810ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.524+0000 7fc7403d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7381055b0 0x7fc73810f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.524+0000 7fc7403d3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc738111f90 con 0x7fc738100e30 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.524+0000 7fc7403d3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc738111e10 con 0x7fc7381055b0 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.524+0000 7fc7403d3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc738112110 con 0x7fc738100480 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 0x7fc73810ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 0x7fc73810ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40586/0 (socket says 172.21.15.1:40586) 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73d96e640 1 -- 172.21.15.1:0/3480152879 learned_addr learned my addr 172.21.15.1:0/3480152879 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73e970640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7381055b0 0x7fc73810f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73e16f640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc738100480 0x7fc73810e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73d96e640 1 -- 172.21.15.1:0/3480152879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc738100480 msgr2=0x7fc73810e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73d96e640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc738100480 0x7fc73810e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73d96e640 1 -- 172.21.15.1:0/3480152879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7381055b0 msgr2=0x7fc73810f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73d96e640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7381055b0 0x7fc73810f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73d96e640 1 -- 172.21.15.1:0/3480152879 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7381a6910 con 0x7fc738100e30 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73e970640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7381055b0 0x7fc73810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc73d96e640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 0x7fc73810ed90 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc728006010 tx=0x7fc7280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc72f7fe640 1 -- 172.21.15.1:0/3480152879 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72803f030 con 0x7fc738100e30 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc72f7fe640 1 -- 172.21.15.1:0/3480152879 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc728003ce0 con 0x7fc738100e30 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc7403d3640 1 -- 172.21.15.1:0/3480152879 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7381a6ba0 con 0x7fc738100e30 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc72f7fe640 1 -- 172.21.15.1:0/3480152879 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728035070 con 0x7fc738100e30 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.525+0000 7fc7403d3640 1 -- 172.21.15.1:0/3480152879 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7381a7080 con 0x7fc738100e30 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.528+0000 7fc72f7fe640 1 -- 172.21.15.1:0/3480152879 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc728042030 con 0x7fc738100e30 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.528+0000 7fc7427ac740 1 Processor -- start 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.528+0000 7fc7427ac740 1 -- start start 2024-09-16T00:36:58.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.528+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74ba1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.528+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74ba1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.528+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.528+0000 7fc7427ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cf74b072f0 con 0x55cf74bac370 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.528+0000 7fc7427ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cf74aaecf0 con 0x55cf74ba2b40 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.528+0000 7fc7427ac740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cf74b032b0 con 0x55cf74ba15d0 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40600/0 (socket says 172.21.15.1:40600) 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc72f7fe640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc7240a3240 0x7fc7240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73d96e640 1 -- 172.21.15.1:0/1295229946 learned_addr learned my addr 172.21.15.1:0/1295229946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73e16f640 1 --2- 172.21.15.1:0/1295229946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74ba1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73e970640 1 --2- 172.21.15.1:0/1295229946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74ba1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73e970640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc7240a3240 0x7fc7240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc72f7fe640 1 -- 172.21.15.1:0/3480152879 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc7280f02d0 con 0x7fc738100e30 2024-09-16T00:36:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73d96e640 1 -- 172.21.15.1:0/1295229946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 msgr2=0x55cf74ba1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73d96e640 1 --2- 172.21.15.1:0/1295229946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74ba1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73d96e640 1 -- 172.21.15.1:0/1295229946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 msgr2=0x55cf74ba1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73d96e640 1 --2- 172.21.15.1:0/1295229946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74ba1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73d96e640 1 -- 172.21.15.1:0/1295229946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf74a933a0 con 0x55cf74bac370 2024-09-16T00:36:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73d96e640 1 --2- 172.21.15.1:0/1295229946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bae760 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc7280bc540 tx=0x7fc728047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9001a00e75620ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc73e970640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc7240a3240 0x7fc7240a5700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc730013b30 tx=0x7fc730005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.529+0000 7fc72cdea640 1 -- 172.21.15.1:0/1295229946 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728049070 con 0x55cf74bac370 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc72cdea640 1 -- 172.21.15.1:0/1295229946 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7280bccb0 con 0x55cf74bac370 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc72cdea640 1 -- 172.21.15.1:0/1295229946 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7280c5220 con 0x55cf74bac370 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc7427ac740 1 -- 172.21.15.1:0/1295229946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 msgr2=0x55cf74bae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc7427ac740 1 --2- 172.21.15.1:0/1295229946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bae760 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc7280bc540 tx=0x7fc728047000 comp rx=0 tx=0).stop 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc7427ac740 1 -- 172.21.15.1:0/1295229946 shutdown_connections 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc7427ac740 1 --2- 172.21.15.1:0/1295229946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bae760 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc7427ac740 1 --2- 172.21.15.1:0/1295229946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74ba1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc7427ac740 1 --2- 172.21.15.1:0/1295229946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74ba1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc7427ac740 1 -- 172.21.15.1:0/1295229946 >> 172.21.15.1:0/1295229946 conn(0x55cf74b6a310 msgr2=0x55cf74b9dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc7427ac740 1 -- 172.21.15.1:0/1295229946 shutdown_connections 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.530+0000 7fc7427ac740 1 -- 172.21.15.1:0/1295229946 wait complete. 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc7427ac740 1 -- start start 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc7427ac740 1 Processor -- start 2024-09-16T00:36:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba15d0 0x55cf74babd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74ba9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74bac370 0x55cf74ba9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc7427ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cf74b072f0 con 0x55cf74ba2b40 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc7427ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cf74ac2060 con 0x55cf74ba15d0 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc7427ac740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cf74baee30 con 0x55cf74bac370 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc7427ac740 10 client.0 _ll_register_callbacks cb 0x55cf748158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74ba9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74ba9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40606/0 (socket says 172.21.15.1:40606) 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc73d96e640 1 -- 172.21.15.1:0/231782782 learned_addr learned my addr 172.21.15.1:0/231782782 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc73e16f640 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba15d0 0x55cf74babd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc73e970640 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74bac370 0x55cf74ba9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc73d96e640 1 -- 172.21.15.1:0/231782782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74bac370 msgr2=0x55cf74ba9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.531+0000 7fc73d96e640 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74bac370 0x55cf74ba9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc73d96e640 1 -- 172.21.15.1:0/231782782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba15d0 msgr2=0x55cf74babd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc73d96e640 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba15d0 0x55cf74babd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc73d96e640 1 -- 172.21.15.1:0/231782782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf748783c0 con 0x55cf74ba2b40 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc73d96e640 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74ba9010 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc7280c4cf0 tx=0x7fc7280b8250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b88734a0870f1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc721ffb640 1 -- 172.21.15.1:0/231782782 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728049040 con 0x55cf74ba2b40 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc721ffb640 1 -- 172.21.15.1:0/231782782 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc7280bc810 con 0x55cf74ba2b40 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc7427ac740 1 -- 172.21.15.1:0/231782782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7280bccb0 con 0x55cf74ba2b40 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc721ffb640 1 -- 172.21.15.1:0/231782782 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7280bbf10 con 0x55cf74ba2b40 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc7427ac740 10 client.17496 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc7427ac740 1 -- 172.21.15.1:0/231782782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf74bba780 con 0x55cf74ba2b40 2024-09-16T00:36:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc721ffb640 1 -- 172.21.15.1:0/231782782 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7280bccb0 con 0x55cf74ba2b40 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc721ffb640 1 -- 172.21.15.1:0/231782782 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc7280c4d90 con 0x55cf74ba2b40 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc7427ac740 10 client.17496 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc7427ac740 10 client.17496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc7427ac740 20 client.17496 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc7427ac740 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55cf74bbbad0 0x55cf74bbdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc7427ac740 4 client.17496 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.532+0000 7fc7427ac740 1 -- 172.21.15.1:0/231782782 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf74b14860 con 0x55cf74bbbad0 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.533+0000 7fc73e16f640 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55cf74bbbad0 0x55cf74bbdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.533+0000 7fc73e16f640 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55cf74bbbad0 0x55cf74bbdec0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3375791c7cef4fd2 server_cookie=5e55d144aee6344f in_seq=0 out_seq=0 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.533+0000 7fc721ffb640 10 client.17496 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc721ffb640 1 -- 172.21.15.1:0/231782782 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf74b14860 con 0x55cf74bbbad0 2024-09-16T00:36:58.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc721ffb640 10 client.17496 handle_command_reply: tid=0 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 client.17496 shutdown 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 -- 172.21.15.1:0/231782782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 msgr2=0x55cf74ba9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74ba9010 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc7280c4cf0 tx=0x7fc7280b8250 comp rx=0 tx=0).stop 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 -- 172.21.15.1:0/231782782 shutdown_connections 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74bac370 0x55cf74ba9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55cf74bbbad0 0x55cf74bbdec0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74ba9010 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 --2- 172.21.15.1:0/231782782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba15d0 0x55cf74babd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 -- 172.21.15.1:0/231782782 >> 172.21.15.1:0/231782782 conn(0x55cf74b6a310 msgr2=0x55cf74ba3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc721ffb640 0 client.17496 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 -- 172.21.15.1:0/231782782 shutdown_connections 2024-09-16T00:36:58.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 1 -- 172.21.15.1:0/231782782 wait complete. 2024-09-16T00:36:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.534+0000 7fc7427ac740 20 client.17496 trim_cache size 0 max 16384 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.548+0000 7fc7427ac740 1 Processor -- start 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.548+0000 7fc7427ac740 1 -- start start 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.548+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74b98c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.548+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74b9de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.548+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bb7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.548+0000 7fc7427ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381b3950 con 0x55cf74bac370 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.548+0000 7fc7427ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381b1220 con 0x55cf74ba2b40 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.548+0000 7fc7427ac740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381af070 con 0x55cf74ba15d0 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74b9de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73e970640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bb7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74b9de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:44616/0 (socket says 172.21.15.1:44616) 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73d96e640 1 -- 172.21.15.1:0/1679355577 learned_addr learned my addr 172.21.15.1:0/1679355577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73e16f640 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74b98c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73e970640 1 -- 172.21.15.1:0/1679355577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 msgr2=0x55cf74b98c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73e970640 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74b98c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73e970640 1 -- 172.21.15.1:0/1679355577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 msgr2=0x55cf74b9de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73e970640 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74b9de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73e970640 1 -- 172.21.15.1:0/1679355577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf74ba7fc0 con 0x55cf74bac370 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73e16f640 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74b98c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73d96e640 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74b9de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc73e970640 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bb7f80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc73000a420 tx=0x7fc73000a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b6815896f516bc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc723fff640 1 -- 172.21.15.1:0/1679355577 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730012650 con 0x55cf74bac370 2024-09-16T00:36:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc723fff640 1 -- 172.21.15.1:0/1679355577 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc730012cd0 con 0x55cf74bac370 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.549+0000 7fc723fff640 1 -- 172.21.15.1:0/1679355577 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73000edf0 con 0x55cf74bac370 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 -- 172.21.15.1:0/1679355577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 msgr2=0x55cf74bb7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bb7f80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc73000a420 tx=0x7fc73000a590 comp rx=0 tx=0).stop 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 -- 172.21.15.1:0/1679355577 shutdown_connections 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74bac370 0x55cf74bb7f80 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74ba2b40 0x55cf74b9de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 --2- 172.21.15.1:0/1679355577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74b98c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 -- 172.21.15.1:0/1679355577 >> 172.21.15.1:0/1679355577 conn(0x55cf74b6a310 msgr2=0x55cf74b9d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 -- 172.21.15.1:0/1679355577 shutdown_connections 2024-09-16T00:36:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 -- 172.21.15.1:0/1679355577 wait complete. 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 -- start start 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.550+0000 7fc7427ac740 1 Processor -- start 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74bb40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74bb4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc7427ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74bac370 0x55cf74bb10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc7427ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381af6e0 con 0x55cf74ba2b40 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc7427ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381b0b50 con 0x55cf74bac370 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc7427ac740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381afdb0 con 0x55cf74ba15d0 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc7427ac740 10 client.0 _ll_register_callbacks cb 0x7fc73000ad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74bb4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73d96e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74bb4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40622/0 (socket says 172.21.15.1:40622) 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73d96e640 1 -- 172.21.15.1:0/2283498950 learned_addr learned my addr 172.21.15.1:0/2283498950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:36:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73e16f640 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74bb40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73e970640 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74bac370 0x55cf74bb10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73d96e640 1 -- 172.21.15.1:0/2283498950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 msgr2=0x55cf74bb40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73d96e640 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74bb40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73d96e640 1 -- 172.21.15.1:0/2283498950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74bac370 msgr2=0x55cf74bb10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73d96e640 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74bac370 0x55cf74bb10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73d96e640 1 -- 172.21.15.1:0/2283498950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf74ba9970 con 0x55cf74ba2b40 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73e970640 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74bac370 0x55cf74bb10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc73d96e640 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74bb4630 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc7280c4cf0 tx=0x7fc7280044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac82ba18c30712b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.551+0000 7fc721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc721ffb640 1 -- 172.21.15.1:0/2283498950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728049040 con 0x55cf74ba2b40 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc721ffb640 1 -- 172.21.15.1:0/2283498950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc7280bc810 con 0x55cf74ba2b40 2024-09-16T00:36:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc7427ac740 1 -- 172.21.15.1:0/2283498950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc730012cd0 con 0x55cf74ba2b40 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc721ffb640 1 -- 172.21.15.1:0/2283498950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7280aa050 con 0x55cf74ba2b40 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc7427ac740 10 client.17502 fetch_fsmap learned FSMap version 25 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc7427ac740 1 -- 172.21.15.1:0/2283498950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf74ba9970 con 0x55cf74ba2b40 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc721ffb640 1 -- 172.21.15.1:0/2283498950 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc730012cd0 con 0x55cf74ba2b40 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc721ffb640 1 -- 172.21.15.1:0/2283498950 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc7280bbaa0 con 0x55cf74ba2b40 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc7427ac740 10 client.17502 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc7427ac740 10 client.17502 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc7427ac740 20 client.17502 populate_metadata read hostname 'smithi001' 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc7427ac740 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55cf74c26ee0 0x55cf74c29300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc7427ac740 4 client.17502 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.552+0000 7fc7427ac740 1 -- 172.21.15.1:0/2283498950 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf74b0ed70 con 0x55cf74c26ee0 2024-09-16T00:36:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc73e16f640 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55cf74c26ee0 0x55cf74c29300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc73e16f640 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55cf74c26ee0 0x55cf74c29300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9322b64f4b55c3f3 server_cookie=e062d4326f2aa4c0 in_seq=0 out_seq=0 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc721ffb640 10 client.17502 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc721ffb640 1 -- 172.21.15.1:0/2283498950 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf74b0ed70 con 0x55cf74c26ee0 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc721ffb640 10 client.17502 handle_command_reply: tid=0 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc7427ac740 1 client.17502 shutdown 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc7427ac740 1 -- 172.21.15.1:0/2283498950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 msgr2=0x55cf74bb4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc7427ac740 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74bb4630 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc7280c4cf0 tx=0x7fc7280044e0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc7427ac740 1 -- 172.21.15.1:0/2283498950 shutdown_connections 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc7427ac740 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf74bac370 0x55cf74bb10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc7427ac740 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55cf74c26ee0 0x55cf74c29300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc7427ac740 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf74ba2b40 0x55cf74bb4630 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc7427ac740 1 --2- 172.21.15.1:0/2283498950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf74ba15d0 0x55cf74bb40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc7427ac740 1 -- 172.21.15.1:0/2283498950 >> 172.21.15.1:0/2283498950 conn(0x55cf74b6a310 msgr2=0x55cf74bab090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.553+0000 7fc721ffb640 0 client.17502 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:36:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7427ac740 1 -- 172.21.15.1:0/2283498950 shutdown_connections 2024-09-16T00:36:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7427ac740 1 -- 172.21.15.1:0/2283498950 wait complete. 2024-09-16T00:36:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7427ac740 20 client.17502 trim_cache size 0 max 16384 2024-09-16T00:36:58.562 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:36:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 -- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc7240a3240 msgr2=0x7fc7240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc7240a3240 0x7fc7240a5700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc730013b30 tx=0x7fc730005c60 comp rx=0 tx=0).stop 2024-09-16T00:36:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 -- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 msgr2=0x7fc73810ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 0x7fc73810ed90 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc728006010 tx=0x7fc7280042f0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 -- 172.21.15.1:0/3480152879 shutdown_connections 2024-09-16T00:36:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7381055b0 0x7fc73810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc738100e30 0x7fc73810ed90 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc7240a3240 0x7fc7240a5700 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 --2- 172.21.15.1:0/3480152879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc738100480 0x7fc73810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:58.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 -- 172.21.15.1:0/3480152879 >> 172.21.15.1:0/3480152879 conn(0x7fc738077210 msgr2=0x7fc738103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:58.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 -- 172.21.15.1:0/3480152879 shutdown_connections 2024-09-16T00:36:58.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:36:58.554+0000 7fc7403d3640 1 -- 172.21.15.1:0/3480152879 wait complete. 2024-09-16T00:36:59.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:36:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.183+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/1424487617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 msgr2=0x7f5f3410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.183+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/1424487617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 0x7f5f3410d950 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009cd0 tx=0x7f5f2c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.183+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/1424487617 shutdown_connections 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.183+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/1424487617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 0x7f5f3410d950 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.183+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/1424487617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f34100a80 0x7f5f34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.183+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/1424487617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f34106a80 0x7f5f34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.183+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/1424487617 >> 172.21.15.1:0/1424487617 conn(0x7f5f340fc7a0 msgr2=0x7f5f340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.183+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/1424487617 shutdown_connections 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.183+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/1424487617 wait complete. 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f3b3f5640 1 Processor -- start 2024-09-16T00:37:00.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f3b3f5640 1 -- start start 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f3b3f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f34100a80 0x7f5f34102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f3b3f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 0x7f5f34102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f3b3f5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f34106a80 0x7f5f341030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f3b3f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34110e80 con 0x7f5f34101420 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f3b3f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34110d00 con 0x7f5f34100a80 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f3b3f5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34111000 con 0x7f5f34106a80 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f38990640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 0x7f5f34102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f38990640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 0x7f5f34102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40646/0 (socket says 172.21.15.1:40646) 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.184+0000 7f5f38990640 1 -- 172.21.15.1:0/2207813942 learned_addr learned my addr 172.21.15.1:0/2207813942 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f39992640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f34106a80 0x7f5f341030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f39191640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f34100a80 0x7f5f34102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f38990640 1 -- 172.21.15.1:0/2207813942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f34106a80 msgr2=0x7f5f341030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:00.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f38990640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f34106a80 0x7f5f341030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f38990640 1 -- 172.21.15.1:0/2207813942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f34100a80 msgr2=0x7f5f34102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f38990640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f34100a80 0x7f5f34102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f38990640 1 -- 172.21.15.1:0/2207813942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341a68f0 con 0x7f5f34101420 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f38990640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 0x7f5f34102b90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400e9f0 tx=0x7f5f2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f2a7fc640 1 -- 172.21.15.1:0/2207813942 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400cd80 con 0x7f5f34101420 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/2207813942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f341a6be0 con 0x7f5f34101420 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f2a7fc640 1 -- 172.21.15.1:0/2207813942 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f24010910 con 0x7f5f34101420 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f2a7fc640 1 -- 172.21.15.1:0/2207813942 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240188f0 con 0x7f5f34101420 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.185+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/2207813942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f341a7170 con 0x7f5f34101420 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.187+0000 7f5f2a7fc640 1 -- 172.21.15.1:0/2207813942 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f5f24010ac0 con 0x7f5f34101420 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.188+0000 7f5f3d7ce740 1 Processor -- start 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.188+0000 7f5f3d7ce740 1 -- start start 2024-09-16T00:37:00.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.188+0000 7f5f2a7fc640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5f200a3240 0x7f5f200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.188+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 0x560b30c07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.188+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.188+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c12370 0x560b30c14760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.188+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560b30b6d2f0 con 0x560b30c12370 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.188+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560b30b14cf0 con 0x560b30c075d0 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.188+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560b30b692b0 con 0x560b30c08b40 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39992640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5f200a3240 0x7f5f200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f2a7fc640 1 -- 172.21.15.1:0/2207813942 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5f240d00f0 con 0x7f5f34101420 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f38990640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39992640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c12370 0x560b30c14760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f38990640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c07a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:44648/0 (socket says 172.21.15.1:44648) 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f38990640 1 -- 172.21.15.1:0/1043883032 learned_addr learned my addr 172.21.15.1:0/1043883032 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39191640 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 0x560b30c07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39191640 1 -- 172.21.15.1:0/1043883032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 msgr2=0x560b30c07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39191640 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c07a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39191640 1 -- 172.21.15.1:0/1043883032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c12370 msgr2=0x560b30c14760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39191640 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c12370 0x560b30c14760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39191640 1 -- 172.21.15.1:0/1043883032 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b30af93a0 con 0x560b30c08b40 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f38990640 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39992640 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c12370 0x560b30c14760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39992640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5f200a3240 0x7f5f200a5700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009430 tx=0x7f5f2c0098f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.189+0000 7f5f39191640 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 0x560b30c07090 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f300045b0 tx=0x7f5f3000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8564e04131df8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f1f7fe640 1 -- 172.21.15.1:0/1043883032 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30003bd0 con 0x560b30c08b40 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f1f7fe640 1 -- 172.21.15.1:0/1043883032 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f30015070 con 0x560b30c08b40 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f1f7fe640 1 -- 172.21.15.1:0/1043883032 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30004cc0 con 0x560b30c08b40 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/1043883032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 msgr2=0x560b30c07090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:00.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 0x560b30c07090 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f300045b0 tx=0x7f5f3000bb20 comp rx=0 tx=0).stop 2024-09-16T00:37:00.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/1043883032 shutdown_connections 2024-09-16T00:37:00.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c12370 0x560b30c14760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/1043883032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 0x560b30c07090 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/1043883032 >> 172.21.15.1:0/1043883032 conn(0x560b30bd0310 msgr2=0x560b30c03c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:00.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/1043883032 shutdown_connections 2024-09-16T00:37:00.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.190+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/1043883032 wait complete. 2024-09-16T00:37:00.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.191+0000 7f5f3d7ce740 1 -- start start 2024-09-16T00:37:00.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.191+0000 7f5f3d7ce740 1 Processor -- start 2024-09-16T00:37:00.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.191+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c075d0 0x560b30c11cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.191+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 0x560b30c0ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.191+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 0x560b30c0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.191+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560b30c15070 con 0x560b30c075d0 2024-09-16T00:37:00.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.191+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560b30b28060 con 0x560b30c12370 2024-09-16T00:37:00.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.191+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560b30c151f0 con 0x560b30c08b40 2024-09-16T00:37:00.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.191+0000 7f5f3d7ce740 10 client.0 _ll_register_callbacks cb 0x560b3087b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:00.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39191640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c075d0 0x560b30c11cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f38990640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 0x560b30c0ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39191640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c075d0 0x560b30c11cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40666/0 (socket says 172.21.15.1:40666) 2024-09-16T00:37:00.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39191640 1 -- 172.21.15.1:0/2517355407 learned_addr learned my addr 172.21.15.1:0/2517355407 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:00.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39992640 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 0x560b30c0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39191640 1 -- 172.21.15.1:0/2517355407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 msgr2=0x560b30c0ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:00.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39191640 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 0x560b30c0ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39191640 1 -- 172.21.15.1:0/2517355407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 msgr2=0x560b30c0f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39191640 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 0x560b30c0f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39191640 1 -- 172.21.15.1:0/2517355407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b308de3c0 con 0x560b30c075d0 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39992640 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 0x560b30c0f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f39191640 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c075d0 0x560b30c11cd0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5f30006010 tx=0x7f5f30004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd163a07a824d1b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2517355407 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30028060 con 0x560b30c075d0 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2517355407 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5f30011640 con 0x560b30c075d0 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2517355407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f30015070 con 0x560b30c075d0 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2517355407 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f3002f050 con 0x560b30c075d0 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f3d7ce740 10 client.17514 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.192+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2517355407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b30c20a80 con 0x560b30c075d0 2024-09-16T00:37:00.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.193+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2517355407 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f30015070 con 0x560b30c075d0 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.193+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2517355407 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5f30022070 con 0x560b30c075d0 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.193+0000 7f5f3d7ce740 10 client.17514 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.193+0000 7f5f3d7ce740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.193+0000 7f5f3d7ce740 20 client.17514 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.193+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560b30c21e80 0x560b30c24270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.193+0000 7f5f3d7ce740 4 client.17514 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.193+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2517355407 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b30b7a860 con 0x560b30c21e80 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.193+0000 7f5f38990640 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560b30c21e80 0x560b30c24270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.194+0000 7f5f38990640 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560b30c21e80 0x560b30c24270 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea156e87a1c50bea server_cookie=cbf034b3de6ab9b6 in_seq=0 out_seq=0 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.194+0000 7f5f1cff9640 10 client.17514 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:00.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.194+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2517355407 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b30b7a860 con 0x560b30c21e80 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.194+0000 7f5f1cff9640 10 client.17514 handle_command_reply: tid=0 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.194+0000 7f5f3d7ce740 1 client.17514 shutdown 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.194+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2517355407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c075d0 msgr2=0x560b30c11cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.194+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c075d0 0x560b30c11cd0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5f30006010 tx=0x7f5f30004220 comp rx=0 tx=0).stop 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2517355407 shutdown_connections 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560b30c21e80 0x560b30c24270 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f1cff9640 0 client.17514 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 0x560b30c0f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c08b40 0x560b30c0ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2517355407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c075d0 0x560b30c11cd0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2517355407 >> 172.21.15.1:0/2517355407 conn(0x560b30bd0310 msgr2=0x560b30c09960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2517355407 shutdown_connections 2024-09-16T00:37:00.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2517355407 wait complete. 2024-09-16T00:37:00.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.195+0000 7f5f3d7ce740 20 client.17514 trim_cache size 0 max 16384 2024-09-16T00:37:00.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f3d7ce740 1 Processor -- start 2024-09-16T00:37:00.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f3d7ce740 1 -- start start 2024-09-16T00:37:00.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c075d0 0x560b30c1e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c03700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 0x560b30c29810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b3a00 con 0x560b30c08b40 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b12d0 con 0x560b30c12370 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341af120 con 0x560b30c075d0 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f38990640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c03700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f38990640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c03700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40676/0 (socket says 172.21.15.1:40676) 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f38990640 1 -- 172.21.15.1:0/255181275 learned_addr learned my addr 172.21.15.1:0/255181275 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f39191640 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c075d0 0x560b30c1e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f39992640 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 0x560b30c29810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f38990640 1 -- 172.21.15.1:0/255181275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c075d0 msgr2=0x560b30c1e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f38990640 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c075d0 0x560b30c1e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f38990640 1 -- 172.21.15.1:0/255181275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 msgr2=0x560b30c29810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:00.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f38990640 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 0x560b30c29810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f38990640 1 -- 172.21.15.1:0/255181275 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b30c04960 con 0x560b30c08b40 2024-09-16T00:37:00.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.208+0000 7f5f39191640 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c075d0 0x560b30c1e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:00.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f38990640 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c03700 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5f240cd840 tx=0x7f5f24098660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1771a971f090aaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:00.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f1effd640 1 -- 172.21.15.1:0/255181275 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240a4f80 con 0x560b30c08b40 2024-09-16T00:37:00.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f1effd640 1 -- 172.21.15.1:0/255181275 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f240ae0d0 con 0x560b30c08b40 2024-09-16T00:37:00.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f1effd640 1 -- 172.21.15.1:0/255181275 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2409b810 con 0x560b30c08b40 2024-09-16T00:37:00.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/255181275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 msgr2=0x560b30c03700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:00.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c03700 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5f240cd840 tx=0x7f5f24098660 comp rx=0 tx=0).stop 2024-09-16T00:37:00.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/255181275 shutdown_connections 2024-09-16T00:37:00.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c12370 0x560b30c29810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c03700 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/255181275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c075d0 0x560b30c1e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/255181275 >> 172.21.15.1:0/255181275 conn(0x560b30bd0310 msgr2=0x560b30c0e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:00.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.209+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/255181275 shutdown_connections 2024-09-16T00:37:00.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/255181275 wait complete. 2024-09-16T00:37:00.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 1 -- start start 2024-09-16T00:37:00.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 1 Processor -- start 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c16240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c16780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c12370 0x560b30c16cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341af790 con 0x560b30c08b40 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b0c00 con 0x560b30c075d0 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341afe60 con 0x560b30c12370 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f3d7ce740 10 client.0 _ll_register_callbacks cb 0x7f5f2c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.210+0000 7f5f38990640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c16780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f38990640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c16780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40690/0 (socket says 172.21.15.1:40690) 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f39191640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c16240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f38990640 1 -- 172.21.15.1:0/2058279673 learned_addr learned my addr 172.21.15.1:0/2058279673 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f39992640 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c12370 0x560b30c16cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f38990640 1 -- 172.21.15.1:0/2058279673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c12370 msgr2=0x560b30c16cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f38990640 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c12370 0x560b30c16cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f38990640 1 -- 172.21.15.1:0/2058279673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 msgr2=0x560b30c16240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f38990640 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c16240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f38990640 1 -- 172.21.15.1:0/2058279673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b30c8b860 con 0x560b30c08b40 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f39191640 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c16240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f38990640 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c16780 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5f240a4a50 tx=0x7f5f24005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee900e82ad2a4a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2058279673 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240a4f80 con 0x560b30c08b40 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2058279673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f240ae0d0 con 0x560b30c08b40 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2058279673 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f5f240b5230 con 0x560b30c08b40 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2058279673 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f24098aa0 con 0x560b30c08b40 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f3d7ce740 10 client.17520 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:00.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.211+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2058279673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b30c8cb20 con 0x560b30c08b40 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2058279673 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f240ae0d0 con 0x560b30c08b40 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2058279673 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5f240b03a0 con 0x560b30c08b40 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f3d7ce740 10 client.17520 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f3d7ce740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f3d7ce740 20 client.17520 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560b30c21e80 0x560b30c8f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f3d7ce740 4 client.17520 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2058279673 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b30b74d70 con 0x560b30c21e80 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f39191640 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560b30c21e80 0x560b30c8f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f39191640 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560b30c21e80 0x560b30c8f1a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd6c282fba1185a server_cookie=ec60c9255aec19f1 in_seq=0 out_seq=0 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f1cff9640 10 client.17520 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.212+0000 7f5f1cff9640 1 -- 172.21.15.1:0/2058279673 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b30b74d70 con 0x560b30c21e80 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f1cff9640 10 client.17520 handle_command_reply: tid=0 2024-09-16T00:37:00.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 client.17520 shutdown 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2058279673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 msgr2=0x560b30c16780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c16780 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5f240a4a50 tx=0x7f5f24005c80 comp rx=0 tx=0).stop 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2058279673 shutdown_connections 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560b30c21e80 0x560b30c8f1a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b30c12370 0x560b30c16cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560b30c08b40 0x560b30c16780 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 --2- 172.21.15.1:0/2058279673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b30c075d0 0x560b30c16240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2058279673 >> 172.21.15.1:0/2058279673 conn(0x560b30bd0310 msgr2=0x560b30c024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f1cff9640 0 client.17520 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2058279673 shutdown_connections 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 1 -- 172.21.15.1:0/2058279673 wait complete. 2024-09-16T00:37:00.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3d7ce740 20 client.17520 trim_cache size 0 max 16384 2024-09-16T00:37:00.221 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:37:00.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5f200a3240 msgr2=0x7f5f200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:00.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5f200a3240 0x7f5f200a5700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009430 tx=0x7f5f2c0098f0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 msgr2=0x7f5f34102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.213+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 0x7f5f34102b90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400e9f0 tx=0x7f5f2400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.214+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/2207813942 shutdown_connections 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.214+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5f200a3240 0x7f5f200a5700 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.214+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f34106a80 0x7f5f341030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.214+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f34101420 0x7f5f34102b90 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.214+0000 7f5f3b3f5640 1 --2- 172.21.15.1:0/2207813942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f34100a80 0x7f5f34102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.214+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/2207813942 >> 172.21.15.1:0/2207813942 conn(0x7f5f340fc7a0 msgr2=0x7f5f3406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.214+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/2207813942 shutdown_connections 2024-09-16T00:37:00.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:00.214+0000 7f5f3b3f5640 1 -- 172.21.15.1:0/2207813942 wait complete. 2024-09-16T00:37:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:00 smithi019 ceph-mon[35186]: pgmap v608: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-16T00:37:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:00 smithi001 ceph-mon[30934]: pgmap v608: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-16T00:37:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:00 smithi153 ceph-mon[35538]: pgmap v608: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-16T00:37:02.127 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:37:02.127 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:37:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:02 smithi019 ceph-mon[35186]: pgmap v609: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:37:02.401 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:02 smithi001 ceph-mon[30934]: pgmap v609: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:37:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:02 smithi153 ceph-mon[35538]: pgmap v609: 129 pgs: 129 active+clean; 3.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:37:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.895+0000 7f3d40dfd640 1 -- 172.21.15.1:0/1131431883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c101de0 msgr2=0x7f3d3c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.895+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/1131431883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c101de0 0x7f3d3c1021c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009900 tx=0x7f3d2c02a410 comp rx=0 tx=0).stop 2024-09-16T00:37:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.895+0000 7f3d40dfd640 1 -- 172.21.15.1:0/1131431883 shutdown_connections 2024-09-16T00:37:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.895+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/1131431883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3c106f10 0x7f3d3c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.895+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/1131431883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d3c102790 0x7f3d3c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.895+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/1131431883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c101de0 0x7f3d3c1021c0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.895+0000 7f3d40dfd640 1 -- 172.21.15.1:0/1131431883 >> 172.21.15.1:0/1131431883 conn(0x7f3d3c0fd8c0 msgr2=0x7f3d3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.895+0000 7f3d40dfd640 1 -- 172.21.15.1:0/1131431883 shutdown_connections 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.895+0000 7f3d40dfd640 1 -- 172.21.15.1:0/1131431883 wait complete. 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.896+0000 7f3d40dfd640 1 Processor -- start 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.896+0000 7f3d40dfd640 1 -- start start 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.896+0000 7f3d40dfd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d3c101de0 0x7f3d3c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.896+0000 7f3d40dfd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c102790 0x7f3d3c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.896+0000 7f3d40dfd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3c106f10 0x7f3d3c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.896+0000 7f3d40dfd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c111f90 con 0x7f3d3c102790 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.896+0000 7f3d40dfd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c111e10 con 0x7f3d3c106f10 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.896+0000 7f3d40dfd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c112110 con 0x7f3d3c101de0 2024-09-16T00:37:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.896+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c102790 0x7f3d3c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c102790 0x7f3d3c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51018/0 (socket says 172.21.15.1:51018) 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d33fff640 1 -- 172.21.15.1:0/4262191322 learned_addr learned my addr 172.21.15.1:0/4262191322 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d3c101de0 0x7f3d3c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d3ad9d640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3c106f10 0x7f3d3c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d33fff640 1 -- 172.21.15.1:0/4262191322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d3c101de0 msgr2=0x7f3d3c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d33fff640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d3c101de0 0x7f3d3c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d33fff640 1 -- 172.21.15.1:0/4262191322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3c106f10 msgr2=0x7f3d3c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d33fff640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3c106f10 0x7f3d3c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d33fff640 1 -- 172.21.15.1:0/4262191322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c1a6940 con 0x7f3d3c102790 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.897+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d3c101de0 0x7f3d3c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.898+0000 7f3d33fff640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c102790 0x7f3d3c074a30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400c890 tx=0x7f3d2400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.898+0000 7f3d337fe640 1 -- 172.21.15.1:0/4262191322 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24004220 con 0x7f3d3c102790 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.898+0000 7f3d337fe640 1 -- 172.21.15.1:0/4262191322 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d240043c0 con 0x7f3d3c102790 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.898+0000 7f3d40dfd640 1 -- 172.21.15.1:0/4262191322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c1a6c30 con 0x7f3d3c102790 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.898+0000 7f3d337fe640 1 -- 172.21.15.1:0/4262191322 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d240049e0 con 0x7f3d3c102790 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.898+0000 7f3d40dfd640 1 -- 172.21.15.1:0/4262191322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c1a71c0 con 0x7f3d3c102790 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.900+0000 7f3d337fe640 1 -- 172.21.15.1:0/4262191322 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3d240079b0 con 0x7f3d3c102790 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.901+0000 7f3d431d6740 1 Processor -- start 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.901+0000 7f3d431d6740 1 -- start start 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.901+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c44bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.901+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c456800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.901+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c459130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.901+0000 7f3d431d6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a96c3b1280 con 0x55a96c44c480 2024-09-16T00:37:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.901+0000 7f3d431d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a96c358b10 con 0x55a96c44bb60 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.901+0000 7f3d431d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a96c3ad240 con 0x55a96c456d40 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c456800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d337fe640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3d280a32c0 0x7f3d280a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c44bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c456800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51028/0 (socket says 172.21.15.1:51028) 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c44bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:56402/0 (socket says 172.21.15.1:56402) 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3a59c640 1 -- 172.21.15.1:0/1280243215 learned_addr learned my addr 172.21.15.1:0/1280243215 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d33fff640 1 --2- 172.21.15.1:0/1280243215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c459130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3ad9d640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3d280a32c0 0x7f3d280a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d337fe640 1 -- 172.21.15.1:0/4262191322 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3d240cfda0 con 0x7f3d3c102790 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3a59c640 1 -- 172.21.15.1:0/1280243215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 msgr2=0x55a96c459130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/1280243215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c459130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3a59c640 1 -- 172.21.15.1:0/1280243215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 msgr2=0x55a96c44bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/1280243215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c44bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3a59c640 1 -- 172.21.15.1:0/1280243215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a96c3651f0 con 0x55a96c44c480 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3ad9d640 1 --2- 172.21.15.1:0/1280243215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c44bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/1280243215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c456800 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009900 tx=0x7f3d2c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e2a9826e448b77 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d3ad9d640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3d280a32c0 0x7f3d280a5780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009fd0 tx=0x7f3d34004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.902+0000 7f3d30ff9640 1 -- 172.21.15.1:0/1280243215 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c03f070 con 0x55a96c44c480 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.903+0000 7f3d30ff9640 1 -- 172.21.15.1:0/1280243215 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2c0035f0 con 0x55a96c44c480 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.903+0000 7f3d30ff9640 1 -- 172.21.15.1:0/1280243215 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c035070 con 0x55a96c44c480 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 -- 172.21.15.1:0/1280243215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 msgr2=0x55a96c456800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 --2- 172.21.15.1:0/1280243215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c456800 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009900 tx=0x7f3d2c0316e0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 -- 172.21.15.1:0/1280243215 shutdown_connections 2024-09-16T00:37:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 --2- 172.21.15.1:0/1280243215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c459130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 --2- 172.21.15.1:0/1280243215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c456800 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 --2- 172.21.15.1:0/1280243215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c44bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 -- 172.21.15.1:0/1280243215 >> 172.21.15.1:0/1280243215 conn(0x55a96c4480f0 msgr2=0x55a96c4484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 -- 172.21.15.1:0/1280243215 shutdown_connections 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 -- 172.21.15.1:0/1280243215 wait complete. 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 -- start start 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 Processor -- start 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c453c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c454190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c4546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.904+0000 7f3d431d6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a96c459910 con 0x55a96c44c480 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d431d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a96c36bf10 con 0x55a96c44bb60 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d431d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a96c459a90 con 0x55a96c456d40 2024-09-16T00:37:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d431d6740 10 client.0 _ll_register_callbacks cb 0x55a96c334a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c454190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c454190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51036/0 (socket says 172.21.15.1:51036) 2024-09-16T00:37:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d33fff640 1 -- 172.21.15.1:0/3107128717 learned_addr learned my addr 172.21.15.1:0/3107128717 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d3a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c453c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d3ad9d640 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c4546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d33fff640 1 -- 172.21.15.1:0/3107128717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 msgr2=0x55a96c4546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d33fff640 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c4546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d33fff640 1 -- 172.21.15.1:0/3107128717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 msgr2=0x55a96c453c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d33fff640 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c453c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d33fff640 1 -- 172.21.15.1:0/3107128717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a96c1226a0 con 0x55a96c44c480 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c453c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d33fff640 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c454190 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000c00 tx=0x7f3d2406bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed6cc3d549cf0c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d21ffb640 1 -- 172.21.15.1:0/3107128717 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2401d070 con 0x55a96c44c480 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d21ffb640 1 -- 172.21.15.1:0/3107128717 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3d2409c220 con 0x55a96c44c480 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.905+0000 7f3d431d6740 1 -- 172.21.15.1:0/3107128717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d2c0035f0 con 0x55a96c44c480 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d21ffb640 1 -- 172.21.15.1:0/3107128717 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d2409c6c0 con 0x55a96c44c480 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d431d6740 10 client.17532 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d431d6740 1 -- 172.21.15.1:0/3107128717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a96c4655a0 con 0x55a96c44c480 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d21ffb640 1 -- 172.21.15.1:0/3107128717 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2c0035f0 con 0x55a96c44c480 2024-09-16T00:37:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d21ffb640 1 -- 172.21.15.1:0/3107128717 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3d2409e030 con 0x55a96c44c480 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d431d6740 10 client.17532 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d431d6740 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d431d6740 20 client.17532 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d431d6740 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a96c466a60 0x55a96c468e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d431d6740 4 client.17532 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d431d6740 1 -- 172.21.15.1:0/3107128717 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a96c3be7f0 con 0x55a96c466a60 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.906+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a96c466a60 0x55a96c468e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.907+0000 7f3d21ffb640 10 client.17532 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.907+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a96c466a60 0x55a96c468e90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce377c8a00f9c1c server_cookie=c58c31e0d80fa784 in_seq=0 out_seq=0 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.907+0000 7f3d21ffb640 1 -- 172.21.15.1:0/3107128717 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a96c3be7f0 con 0x55a96c466a60 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.907+0000 7f3d21ffb640 10 client.17532 handle_command_reply: tid=0 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.907+0000 7f3d431d6740 1 client.17532 shutdown 2024-09-16T00:37:02.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 -- 172.21.15.1:0/3107128717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 msgr2=0x55a96c454190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c454190 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000c00 tx=0x7f3d2406bc30 comp rx=0 tx=0).stop 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 -- 172.21.15.1:0/3107128717 shutdown_connections 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a96c466a60 0x55a96c468e90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c4546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c454190 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 --2- 172.21.15.1:0/3107128717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c453c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 -- 172.21.15.1:0/3107128717 >> 172.21.15.1:0/3107128717 conn(0x55a96c4480f0 msgr2=0x55a96c447270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d21ffb640 0 client.17532 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 -- 172.21.15.1:0/3107128717 shutdown_connections 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 1 -- 172.21.15.1:0/3107128717 wait complete. 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.908+0000 7f3d431d6740 20 client.17532 trim_cache size 0 max 16384 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.925+0000 7f3d431d6740 1 Processor -- start 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.925+0000 7f3d431d6740 1 -- start start 2024-09-16T00:37:02.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.925+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c44bb60 0x55a96c44dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.925+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c46e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.925+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c456d40 0x55a96c46eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.925+0000 7f3d431d6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c1b38f0 con 0x55a96c44c480 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.925+0000 7f3d431d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c1b11c0 con 0x55a96c456d40 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.925+0000 7f3d431d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c1af010 con 0x55a96c44bb60 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.925+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c46e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c46e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51042/0 (socket says 172.21.15.1:51042) 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d33fff640 1 -- 172.21.15.1:0/2621089821 learned_addr learned my addr 172.21.15.1:0/2621089821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c44bb60 0x55a96c44dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d33fff640 1 -- 172.21.15.1:0/2621089821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c44bb60 msgr2=0x55a96c44dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d3ad9d640 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c456d40 0x55a96c46eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d33fff640 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c44bb60 0x55a96c44dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d33fff640 1 -- 172.21.15.1:0/2621089821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c456d40 msgr2=0x55a96c46eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d33fff640 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c456d40 0x55a96c46eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d33fff640 1 -- 172.21.15.1:0/2621089821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a96c443830 con 0x55a96c44c480 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d3ad9d640 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c456d40 0x55a96c46eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d33fff640 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c46e760 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3d2409a270 tx=0x7f3d2409a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0c6b01c25b0c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d23fff640 1 -- 172.21.15.1:0/2621089821 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2401d070 con 0x55a96c44c480 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d23fff640 1 -- 172.21.15.1:0/2621089821 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2409bfc0 con 0x55a96c44c480 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.926+0000 7f3d23fff640 1 -- 172.21.15.1:0/2621089821 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24096b30 con 0x55a96c44c480 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.927+0000 7f3d431d6740 1 -- 172.21.15.1:0/2621089821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 msgr2=0x55a96c46e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.927+0000 7f3d431d6740 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c46e760 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3d2409a270 tx=0x7f3d2409a4d0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.927+0000 7f3d431d6740 1 -- 172.21.15.1:0/2621089821 shutdown_connections 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.927+0000 7f3d431d6740 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c456d40 0x55a96c46eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.927+0000 7f3d431d6740 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c46e760 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.927+0000 7f3d431d6740 1 --2- 172.21.15.1:0/2621089821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c44bb60 0x55a96c44dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.927+0000 7f3d431d6740 1 -- 172.21.15.1:0/2621089821 >> 172.21.15.1:0/2621089821 conn(0x55a96c4480f0 msgr2=0x55a96c44d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.928+0000 7f3d431d6740 1 -- 172.21.15.1:0/2621089821 shutdown_connections 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.928+0000 7f3d431d6740 1 -- 172.21.15.1:0/2621089821 wait complete. 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d431d6740 1 -- start start 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d431d6740 1 Processor -- start 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c45f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c45a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d431d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c45ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d431d6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c1af680 con 0x55a96c44c480 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d431d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c1b0af0 con 0x55a96c44bb60 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d431d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c1afd50 con 0x55a96c456d40 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c45a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c45a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51050/0 (socket says 172.21.15.1:51050) 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d431d6740 10 client.0 _ll_register_callbacks cb 0x7f3d240053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d33fff640 1 -- 172.21.15.1:0/402902997 learned_addr learned my addr 172.21.15.1:0/402902997 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.929+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c45f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d3ad9d640 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c45ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d33fff640 1 -- 172.21.15.1:0/402902997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 msgr2=0x55a96c45ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d33fff640 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c45ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d33fff640 1 -- 172.21.15.1:0/402902997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 msgr2=0x55a96c45f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d33fff640 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c45f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d33fff640 1 -- 172.21.15.1:0/402902997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a96c45b150 con 0x55a96c44c480 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c45f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d3ad9d640 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c45ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d33fff640 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c45a5d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000c00 tx=0x7f3d2409bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592df9d84e256428 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d21ffb640 1 -- 172.21.15.1:0/402902997 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2401d040 con 0x55a96c44c480 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d431d6740 1 -- 172.21.15.1:0/402902997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d2409bfc0 con 0x55a96c44c480 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d21ffb640 1 -- 172.21.15.1:0/402902997 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3d2409b610 con 0x55a96c44c480 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d21ffb640 1 -- 172.21.15.1:0/402902997 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d24083050 con 0x55a96c44c480 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d431d6740 10 client.17538 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.930+0000 7f3d431d6740 1 -- 172.21.15.1:0/402902997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a96c4d14f0 con 0x55a96c44c480 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.931+0000 7f3d21ffb640 1 -- 172.21.15.1:0/402902997 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2409bfc0 con 0x55a96c44c480 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.931+0000 7f3d21ffb640 1 -- 172.21.15.1:0/402902997 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3d2409e070 con 0x55a96c44c480 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.931+0000 7f3d431d6740 10 client.17538 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.931+0000 7f3d431d6740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.931+0000 7f3d431d6740 20 client.17538 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.931+0000 7f3d431d6740 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a96c4d1970 0x55a96c4d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.931+0000 7f3d431d6740 4 client.17538 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:37:02.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.931+0000 7f3d431d6740 1 -- 172.21.15.1:0/402902997 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a96c3b8d00 con 0x55a96c4d1970 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.931+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a96c4d1970 0x55a96c4d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.932+0000 7f3d3a59c640 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a96c4d1970 0x55a96c4d3d60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1005baee90506687 server_cookie=b5af9c23e1ac55f6 in_seq=0 out_seq=0 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.932+0000 7f3d21ffb640 10 client.17538 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.932+0000 7f3d21ffb640 1 -- 172.21.15.1:0/402902997 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a96c3b8d00 con 0x55a96c4d1970 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.932+0000 7f3d21ffb640 10 client.17538 handle_command_reply: tid=0 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 client.17538 shutdown 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 -- 172.21.15.1:0/402902997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 msgr2=0x55a96c45a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c45a5d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000c00 tx=0x7f3d2409bd90 comp rx=0 tx=0).stop 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 -- 172.21.15.1:0/402902997 shutdown_connections 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a96c4d1970 0x55a96c4d3d60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a96c456d40 0x55a96c45ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a96c44c480 0x55a96c45a5d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 --2- 172.21.15.1:0/402902997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a96c44bb60 0x55a96c45f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 -- 172.21.15.1:0/402902997 >> 172.21.15.1:0/402902997 conn(0x55a96c4480f0 msgr2=0x55a96c454430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:02.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d21ffb640 0 client.17538 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:02.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 -- 172.21.15.1:0/402902997 shutdown_connections 2024-09-16T00:37:02.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 1 -- 172.21.15.1:0/402902997 wait complete. 2024-09-16T00:37:02.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.933+0000 7f3d431d6740 20 client.17538 trim_cache size 0 max 16384 2024-09-16T00:37:02.948 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:37:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:37:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "8b6efff3-d324-40d3-9cda-36f5baf50833", 2024-09-16T00:37:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:37:02.948 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:37:02.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 -- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3d280a32c0 msgr2=0x7f3d280a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:02.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3d280a32c0 0x7f3d280a5780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009fd0 tx=0x7f3d34004a40 comp rx=0 tx=0).stop 2024-09-16T00:37:02.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 -- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c102790 msgr2=0x7f3d3c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:02.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c102790 0x7f3d3c074a30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400c890 tx=0x7f3d2400cd50 comp rx=0 tx=0).stop 2024-09-16T00:37:02.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 -- 172.21.15.1:0/4262191322 shutdown_connections 2024-09-16T00:37:02.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3c106f10 0x7f3d3c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d3c102790 0x7f3d3c074a30 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3d280a32c0 0x7f3d280a5780 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 --2- 172.21.15.1:0/4262191322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d3c101de0 0x7f3d3c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:02.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 -- 172.21.15.1:0/4262191322 >> 172.21.15.1:0/4262191322 conn(0x7f3d3c0fd8c0 msgr2=0x7f3d3c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:02.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 -- 172.21.15.1:0/4262191322 shutdown_connections 2024-09-16T00:37:02.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:02.934+0000 7f3d40dfd640 1 -- 172.21.15.1:0/4262191322 wait complete. 2024-09-16T00:37:03.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:03 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:37:03.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:03 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:03.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:03 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:37:03.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:03 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:03.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:03 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:37:03.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:03 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:37:03.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:03 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:03.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:03 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:37:03.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:03 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:03.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:03 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:37:03.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:03 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:37:03.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:03 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:03.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:03 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:37:03.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:03 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:03.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:03 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:37:03.754 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:37:04.017 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:04.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:04 smithi001 ceph-mon[30934]: pgmap v610: 129 pgs: 129 active+clean; 4.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-16T00:37:04.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:04 smithi153 ceph-mon[35538]: pgmap v610: 129 pgs: 129 active+clean; 4.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-16T00:37:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.514+0000 7f5673d1b640 1 -- 172.21.15.1:0/1546947931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 msgr2=0x7f566c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.514+0000 7f5673d1b640 1 --2- 172.21.15.1:0/1546947931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 0x7f566c1021c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5668009900 tx=0x7f566802a410 comp rx=0 tx=0).stop 2024-09-16T00:37:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.514+0000 7f5673d1b640 1 -- 172.21.15.1:0/1546947931 shutdown_connections 2024-09-16T00:37:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.514+0000 7f5673d1b640 1 --2- 172.21.15.1:0/1546947931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f566c106f10 0x7f566c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.514+0000 7f5673d1b640 1 --2- 172.21.15.1:0/1546947931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c102790 0x7f566c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.514+0000 7f5673d1b640 1 --2- 172.21.15.1:0/1546947931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 0x7f566c1021c0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.514+0000 7f5673d1b640 1 -- 172.21.15.1:0/1546947931 >> 172.21.15.1:0/1546947931 conn(0x7f566c0fd8c0 msgr2=0x7f566c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.514+0000 7f5673d1b640 1 -- 172.21.15.1:0/1546947931 shutdown_connections 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.514+0000 7f5673d1b640 1 -- 172.21.15.1:0/1546947931 wait complete. 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5673d1b640 1 Processor -- start 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5673d1b640 1 -- start start 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5673d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 0x7f566c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5673d1b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f566c102790 0x7f566c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5673d1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c106f10 0x7f566c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5673d1b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f566c112010 con 0x7f566c101de0 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5673d1b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f566c111e90 con 0x7f566c106f10 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5673d1b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f566c112190 con 0x7f566c102790 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5671ab7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 0x7f566c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5671ab7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 0x7f566c1a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51068/0 (socket says 172.21.15.1:51068) 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5671ab7640 1 -- 172.21.15.1:0/2192665112 learned_addr learned my addr 172.21.15.1:0/2192665112 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f56712b6640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f566c102790 0x7f566c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f56722b8640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c106f10 0x7f566c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5671ab7640 1 -- 172.21.15.1:0/2192665112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f566c102790 msgr2=0x7f566c1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5671ab7640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f566c102790 0x7f566c1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5671ab7640 1 -- 172.21.15.1:0/2192665112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c106f10 msgr2=0x7f566c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5671ab7640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c106f10 0x7f566c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f5671ab7640 1 -- 172.21.15.1:0/2192665112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c19ad50 con 0x7f566c101de0 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.515+0000 7f56722b8640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c106f10 0x7f566c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.516+0000 7f5671ab7640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 0x7f566c1a0460 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5668005ab0 tx=0x7f56680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.516+0000 7f5662ffd640 1 -- 172.21.15.1:0/2192665112 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566803f030 con 0x7f566c101de0 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.516+0000 7f5673d1b640 1 -- 172.21.15.1:0/2192665112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c19afe0 con 0x7f566c101de0 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.516+0000 7f5662ffd640 1 -- 172.21.15.1:0/2192665112 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5668003ce0 con 0x7f566c101de0 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.516+0000 7f5662ffd640 1 -- 172.21.15.1:0/2192665112 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668035070 con 0x7f566c101de0 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.516+0000 7f5673d1b640 1 -- 172.21.15.1:0/2192665112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c1a7330 con 0x7f566c101de0 2024-09-16T00:37:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.518+0000 7f5662ffd640 1 -- 172.21.15.1:0/2192665112 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f5668042030 con 0x7f566c101de0 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.518+0000 7f56760f4740 1 Processor -- start 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f56760f4740 1 -- start start 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f56760f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3eda8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f56760f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f56760f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3edb3680 0x560e3edb5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f56760f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560e3ed0e2e0 con 0x560e3eda8dc0 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f56760f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560e3ecb5ce0 con 0x560e3eda84a0 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f56760f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e3ed0a2a0 con 0x560e3edb3680 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f5671ab7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f5671ab7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51076/0 (socket says 172.21.15.1:51076) 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f5671ab7640 1 -- 172.21.15.1:0/1806866459 learned_addr learned my addr 172.21.15.1:0/1806866459 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f56722b8640 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3eda8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f5662ffd640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56580a3240 0x7f56580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.519+0000 7f56712b6640 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3edb3680 0x560e3edb5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5671ab7640 1 -- 172.21.15.1:0/1806866459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3edb3680 msgr2=0x560e3edb5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5671ab7640 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3edb3680 0x560e3edb5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5671ab7640 1 -- 172.21.15.1:0/1806866459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 msgr2=0x560e3eda8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5671ab7640 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3eda8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5671ab7640 1 -- 172.21.15.1:0/1806866459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3ec9a440 con 0x560e3eda8dc0 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f56722b8640 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3eda8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f56722b8640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56580a3240 0x7f56580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5662ffd640 1 -- 172.21.15.1:0/2192665112 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f56680f0710 con 0x7f566c101de0 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5671ab7640 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb3140 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f56680a54b0 tx=0x7f5668087260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277c70f997c9006c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5657fff640 1 -- 172.21.15.1:0/1806866459 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566808c930 con 0x560e3eda8dc0 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5657fff640 1 -- 172.21.15.1:0/1806866459 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56680a5c50 con 0x560e3eda8dc0 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.520+0000 7f5657fff640 1 -- 172.21.15.1:0/1806866459 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566808c5b0 con 0x560e3eda8dc0 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56760f4740 1 -- 172.21.15.1:0/1806866459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 msgr2=0x560e3edb3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56722b8640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56580a3240 0x7f56580a5700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f566400a3c0 tx=0x7f5664004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56760f4740 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb3140 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f56680a54b0 tx=0x7f5668087260 comp rx=0 tx=0).stop 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56760f4740 1 -- 172.21.15.1:0/1806866459 shutdown_connections 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56760f4740 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3edb3680 0x560e3edb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56760f4740 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb3140 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56760f4740 1 --2- 172.21.15.1:0/1806866459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3eda8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56760f4740 1 -- 172.21.15.1:0/1806866459 >> 172.21.15.1:0/1806866459 conn(0x560e3eda4a30 msgr2=0x560e3eda4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56760f4740 1 -- 172.21.15.1:0/1806866459 shutdown_connections 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.521+0000 7f56760f4740 1 -- 172.21.15.1:0/1806866459 wait complete. 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56760f4740 1 -- start start 2024-09-16T00:37:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56760f4740 1 Processor -- start 2024-09-16T00:37:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56760f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3edb2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56760f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda8dc0 0x560e3edb0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56760f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3edb3680 0x560e3edb0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56760f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560e3edb6010 con 0x560e3edb3680 2024-09-16T00:37:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56760f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560e3ecc9050 con 0x560e3eda84a0 2024-09-16T00:37:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56760f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e3edb6190 con 0x560e3eda8dc0 2024-09-16T00:37:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56760f4740 10 client.0 _ll_register_callbacks cb 0x560e3ed09160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56722b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3edb3680 0x560e3edb0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56722b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3edb3680 0x560e3edb0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51078/0 (socket says 172.21.15.1:51078) 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56722b8640 1 -- 172.21.15.1:0/4023737776 learned_addr learned my addr 172.21.15.1:0/4023737776 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f56712b6640 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda8dc0 0x560e3edb0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.522+0000 7f5671ab7640 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3edb2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56722b8640 1 -- 172.21.15.1:0/4023737776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda8dc0 msgr2=0x560e3edb0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56722b8640 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda8dc0 0x560e3edb0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56722b8640 1 -- 172.21.15.1:0/4023737776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 msgr2=0x560e3edb2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56722b8640 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3edb2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56722b8640 1 -- 172.21.15.1:0/4023737776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3ea7f3c0 con 0x560e3edb3680 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f5671ab7640 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3edb2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56722b8640 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3edb3680 0x560e3edb0760 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5664011b70 tx=0x7f5664004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a058f67a5fda5eee server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56557fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56557fa640 1 -- 172.21.15.1:0/4023737776 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664022020 con 0x560e3edb3680 2024-09-16T00:37:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56557fa640 1 -- 172.21.15.1:0/4023737776 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f566401e450 con 0x560e3edb3680 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56760f4740 1 -- 172.21.15.1:0/4023737776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e3ed1b850 con 0x560e3edb3680 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56557fa640 1 -- 172.21.15.1:0/4023737776 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f566402d050 con 0x560e3edb3680 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56760f4740 10 client.17550 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.523+0000 7f56760f4740 1 -- 172.21.15.1:0/4023737776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e3edc1910 con 0x560e3edb3680 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f56557fa640 1 -- 172.21.15.1:0/4023737776 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560e3ed1b850 con 0x560e3edb3680 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f56557fa640 1 -- 172.21.15.1:0/4023737776 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f566401d820 con 0x560e3edb3680 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f56760f4740 10 client.17550 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f56760f4740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f56760f4740 20 client.17550 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f56760f4740 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560e3edc2dd0 0x560e3edc5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f56760f4740 4 client.17550 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f56760f4740 1 -- 172.21.15.1:0/4023737776 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e3ed15d60 con 0x560e3edc2dd0 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f5671ab7640 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560e3edc2dd0 0x560e3edc5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f56557fa640 10 client.17550 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.524+0000 7f5671ab7640 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560e3edc2dd0 0x560e3edc5230 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f062f948224997 server_cookie=7041c0f618c16480 in_seq=0 out_seq=0 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56557fa640 1 -- 172.21.15.1:0/4023737776 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e3ed15d60 con 0x560e3edc2dd0 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56557fa640 10 client.17550 handle_command_reply: tid=0 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56760f4740 1 client.17550 shutdown 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56760f4740 1 -- 172.21.15.1:0/4023737776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3edb3680 msgr2=0x560e3edb0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56760f4740 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3edb3680 0x560e3edb0760 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5664011b70 tx=0x7f5664004a40 comp rx=0 tx=0).stop 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56760f4740 1 -- 172.21.15.1:0/4023737776 shutdown_connections 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56760f4740 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560e3edc2dd0 0x560e3edc5230 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56760f4740 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3edb3680 0x560e3edb0760 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56760f4740 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda8dc0 0x560e3edb0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56557fa640 0 client.17550 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56760f4740 1 --2- 172.21.15.1:0/4023737776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3edb2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.525+0000 7f56760f4740 1 -- 172.21.15.1:0/4023737776 >> 172.21.15.1:0/4023737776 conn(0x560e3eda4a30 msgr2=0x560e3eda3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.526+0000 7f56760f4740 1 -- 172.21.15.1:0/4023737776 shutdown_connections 2024-09-16T00:37:04.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.526+0000 7f56760f4740 1 -- 172.21.15.1:0/4023737776 wait complete. 2024-09-16T00:37:04.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.526+0000 7f56760f4740 20 client.17550 trim_cache size 0 max 16384 2024-09-16T00:37:04.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56760f4740 1 Processor -- start 2024-09-16T00:37:04.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56760f4740 1 -- start start 2024-09-16T00:37:04.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56760f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3edbf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56760f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edcab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56760f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3edb3680 0x560e3edcb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56760f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1b3ac0 con 0x560e3eda8dc0 2024-09-16T00:37:04.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56760f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1b1390 con 0x560e3eda84a0 2024-09-16T00:37:04.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56760f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1af1e0 con 0x560e3edb3680 2024-09-16T00:37:04.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56712b6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edcab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56712b6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edcab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51088/0 (socket says 172.21.15.1:51088) 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56712b6640 1 -- 172.21.15.1:0/3993110190 learned_addr learned my addr 172.21.15.1:0/3993110190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f5671ab7640 1 --2- 172.21.15.1:0/3993110190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3edbf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56712b6640 1 -- 172.21.15.1:0/3993110190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3edb3680 msgr2=0x560e3edcb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56712b6640 1 --2- 172.21.15.1:0/3993110190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3edb3680 0x560e3edcb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56712b6640 1 -- 172.21.15.1:0/3993110190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 msgr2=0x560e3edbf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56712b6640 1 --2- 172.21.15.1:0/3993110190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3edbf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.543+0000 7f56712b6640 1 -- 172.21.15.1:0/3993110190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3edc58b0 con 0x560e3eda8dc0 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56712b6640 1 --2- 172.21.15.1:0/3993110190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edcab00 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f566c19b760 tx=0x7f565c00bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334f10ec4a84a1cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56577fe640 1 -- 172.21.15.1:0/3993110190 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c019030 con 0x560e3eda8dc0 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56577fe640 1 -- 172.21.15.1:0/3993110190 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f565c012d90 con 0x560e3eda8dc0 2024-09-16T00:37:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56577fe640 1 -- 172.21.15.1:0/3993110190 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c015cd0 con 0x560e3eda8dc0 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56760f4740 1 -- 172.21.15.1:0/3993110190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 msgr2=0x560e3edcab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56760f4740 1 --2- 172.21.15.1:0/3993110190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edcab00 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f566c19b760 tx=0x7f565c00bd10 comp rx=0 tx=0).stop 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56760f4740 1 -- 172.21.15.1:0/3993110190 shutdown_connections 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56760f4740 1 --2- 172.21.15.1:0/3993110190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3edb3680 0x560e3edcb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56760f4740 1 --2- 172.21.15.1:0/3993110190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edcab00 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56760f4740 1 --2- 172.21.15.1:0/3993110190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3eda84a0 0x560e3edbf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56760f4740 1 -- 172.21.15.1:0/3993110190 >> 172.21.15.1:0/3993110190 conn(0x560e3eda4a30 msgr2=0x560e3edaeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56760f4740 1 -- 172.21.15.1:0/3993110190 shutdown_connections 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.544+0000 7f56760f4740 1 -- 172.21.15.1:0/3993110190 wait complete. 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56760f4740 1 -- start start 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56760f4740 1 Processor -- start 2024-09-16T00:37:04.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56760f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda84a0 0x560e3edb70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56760f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56760f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3edb3680 0x560e3edb7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56760f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1af850 con 0x560e3eda8dc0 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56760f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1b0cc0 con 0x560e3edb3680 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56760f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1aff20 con 0x560e3eda84a0 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56760f4740 10 client.0 _ll_register_callbacks cb 0x7f5664007cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56712b6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56712b6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51090/0 (socket says 172.21.15.1:51090) 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.545+0000 7f56712b6640 1 -- 172.21.15.1:0/2880077325 learned_addr learned my addr 172.21.15.1:0/2880077325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56722b8640 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3edb3680 0x560e3edb7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f5671ab7640 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda84a0 0x560e3edb70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56712b6640 1 -- 172.21.15.1:0/2880077325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda84a0 msgr2=0x560e3edb70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56712b6640 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda84a0 0x560e3edb70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56712b6640 1 -- 172.21.15.1:0/2880077325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3edb3680 msgr2=0x560e3edb7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56712b6640 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3edb3680 0x560e3edb7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56712b6640 1 -- 172.21.15.1:0/2880077325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3ee2cb20 con 0x560e3eda8dc0 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56722b8640 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3edb3680 0x560e3edb7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56712b6640 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb7620 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f565c004700 tx=0x7f565c016f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a70e2ce68e8fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56557fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56557fa640 1 -- 172.21.15.1:0/2880077325 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c019030 con 0x560e3eda8dc0 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56760f4740 1 -- 172.21.15.1:0/2880077325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565c012d90 con 0x560e3eda8dc0 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56557fa640 1 -- 172.21.15.1:0/2880077325 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f565c012ac0 con 0x560e3eda8dc0 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56557fa640 1 -- 172.21.15.1:0/2880077325 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f565c02f050 con 0x560e3eda8dc0 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56760f4740 10 client.17556 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.546+0000 7f56760f4740 1 -- 172.21.15.1:0/2880077325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e3ee2ddf0 con 0x560e3eda8dc0 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.547+0000 7f56557fa640 1 -- 172.21.15.1:0/2880077325 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f565c012d90 con 0x560e3eda8dc0 2024-09-16T00:37:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.547+0000 7f56557fa640 1 -- 172.21.15.1:0/2880077325 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f565c026410 con 0x560e3eda8dc0 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.547+0000 7f56760f4740 10 client.17556 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.547+0000 7f56760f4740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.547+0000 7f56760f4740 20 client.17556 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.547+0000 7f56760f4740 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560e3ee2e460 0x560e3ee30880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.547+0000 7f56760f4740 4 client.17556 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.547+0000 7f56760f4740 1 -- 172.21.15.1:0/2880077325 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e3edb0af0 con 0x560e3ee2e460 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.547+0000 7f5671ab7640 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560e3ee2e460 0x560e3ee30880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f5671ab7640 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560e3ee2e460 0x560e3ee30880 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cccc1870a3840494 server_cookie=efb4aa4a3d60df67 in_seq=0 out_seq=0 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56557fa640 10 client.17556 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56557fa640 1 -- 172.21.15.1:0/2880077325 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e3edb0af0 con 0x560e3ee2e460 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56557fa640 10 client.17556 handle_command_reply: tid=0 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 client.17556 shutdown 2024-09-16T00:37:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 -- 172.21.15.1:0/2880077325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 msgr2=0x560e3edb7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb7620 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f565c004700 tx=0x7f565c016f10 comp rx=0 tx=0).stop 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 -- 172.21.15.1:0/2880077325 shutdown_connections 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560e3ee2e460 0x560e3ee30880 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e3edb3680 0x560e3edb7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e3eda8dc0 0x560e3edb7620 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56557fa640 0 client.17556 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 --2- 172.21.15.1:0/2880077325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3eda84a0 0x560e3edb70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 -- 172.21.15.1:0/2880077325 >> 172.21.15.1:0/2880077325 conn(0x560e3eda4a30 msgr2=0x560e3edb1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 -- 172.21.15.1:0/2880077325 shutdown_connections 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 1 -- 172.21.15.1:0/2880077325 wait complete. 2024-09-16T00:37:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.548+0000 7f56760f4740 20 client.17556 trim_cache size 0 max 16384 2024-09-16T00:37:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:37:04.554 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:37:04.554 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:37:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 -- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56580a3240 msgr2=0x7f56580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56580a3240 0x7f56580a5700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f566400a3c0 tx=0x7f5664004b50 comp rx=0 tx=0).stop 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 -- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 msgr2=0x7f566c1a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 0x7f566c1a0460 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5668005ab0 tx=0x7f56680042f0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 -- 172.21.15.1:0/2192665112 shutdown_connections 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c106f10 0x7f566c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56580a3240 0x7f56580a5700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f566c102790 0x7f566c1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 --2- 172.21.15.1:0/2192665112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f566c101de0 0x7f566c1a0460 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 -- 172.21.15.1:0/2192665112 >> 172.21.15.1:0/2192665112 conn(0x7f566c0fd8c0 msgr2=0x7f566c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 -- 172.21.15.1:0/2192665112 shutdown_connections 2024-09-16T00:37:04.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:04.549+0000 7f5673d1b640 1 -- 172.21.15.1:0/2192665112 wait complete. 2024-09-16T00:37:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:04 smithi019 ceph-mon[35186]: pgmap v610: 129 pgs: 129 active+clean; 4.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-16T00:37:05.502 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:37:05.502 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:37:05.764 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:06.199 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:06 smithi001 ceph-mon[30934]: pgmap v611: 129 pgs: 129 active+clean; 4.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:37:06.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.282+0000 7fca9fd89640 1 -- 172.21.15.1:0/840472707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 msgr2=0x7fca98100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.282+0000 7fca9fd89640 1 --2- 172.21.15.1:0/840472707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 0x7fca98100cb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fca94009900 tx=0x7fca9402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.282+0000 7fca9fd89640 1 -- 172.21.15.1:0/840472707 shutdown_connections 2024-09-16T00:37:06.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.282+0000 7fca9fd89640 1 --2- 172.21.15.1:0/840472707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9810e480 0x7fca98110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.282+0000 7fca9fd89640 1 --2- 172.21.15.1:0/840472707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca98101280 0x7fca9810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.282+0000 7fca9fd89640 1 --2- 172.21.15.1:0/840472707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 0x7fca98100cb0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.282+0000 7fca9fd89640 1 -- 172.21.15.1:0/840472707 >> 172.21.15.1:0/840472707 conn(0x7fca980fc7a0 msgr2=0x7fca980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 -- 172.21.15.1:0/840472707 shutdown_connections 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 -- 172.21.15.1:0/840472707 wait complete. 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 Processor -- start 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 -- start start 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 0x7fca9819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca98101280 0x7fca9819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9810e480 0x7fca9819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fca98114190 con 0x7fca981008d0 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca98114010 con 0x7fca98101280 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.283+0000 7fca9fd89640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca98114310 con 0x7fca9810e480 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9db25640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 0x7fca9819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9db25640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 0x7fca9819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51114/0 (socket says 172.21.15.1:51114) 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9db25640 1 -- 172.21.15.1:0/4274334818 learned_addr learned my addr 172.21.15.1:0/4274334818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:06.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9d324640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca98101280 0x7fca9819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9e326640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9810e480 0x7fca9819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9db25640 1 -- 172.21.15.1:0/4274334818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9810e480 msgr2=0x7fca9819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9db25640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9810e480 0x7fca9819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9db25640 1 -- 172.21.15.1:0/4274334818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca98101280 msgr2=0x7fca9819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9db25640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca98101280 0x7fca9819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9db25640 1 -- 172.21.15.1:0/4274334818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9819f620 con 0x7fca981008d0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9db25640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 0x7fca9819a670 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fca94006010 tx=0x7fca940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca8effd640 1 -- 172.21.15.1:0/4274334818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9403f030 con 0x7fca981008d0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9819f8b0 con 0x7fca981008d0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca8effd640 1 -- 172.21.15.1:0/4274334818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca94003ce0 con 0x7fca981008d0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.284+0000 7fca8effd640 1 -- 172.21.15.1:0/4274334818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94035070 con 0x7fca981008d0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.285+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca981a71f0 con 0x7fca981008d0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.286+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9806b070 con 0x7fca981008d0 2024-09-16T00:37:06.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.287+0000 7fca8effd640 1 -- 172.21.15.1:0/4274334818 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fca94042030 con 0x7fca981008d0 2024-09-16T00:37:06.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.287+0000 7fca8effd640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca840a3240 0x7fca840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.288+0000 7fca8effd640 1 -- 172.21.15.1:0/4274334818 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fca940f0410 con 0x7fca981008d0 2024-09-16T00:37:06.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.288+0000 7fca9d324640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca840a3240 0x7fca840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.288+0000 7fca9d324640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca840a3240 0x7fca840a5700 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fca9810ed80 tx=0x7fca8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:06.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.291+0000 7fca8effd640 1 -- 172.21.15.1:0/4274334818 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca940bc7a0 con 0x7fca981008d0 2024-09-16T00:37:06.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:06 smithi153 ceph-mon[35538]: pgmap v611: 129 pgs: 129 active+clean; 4.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:37:06.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.504+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca98100cb0 con 0x7fca981008d0 2024-09-16T00:37:06.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.505+0000 7fca8effd640 1 -- 172.21.15.1:0/4274334818 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fca98100cb0 con 0x7fca981008d0 2024-09-16T00:37:06.506 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:37:06.506 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:37:06.506 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:37:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca840a3240 msgr2=0x7fca840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca840a3240 0x7fca840a5700 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fca9810ed80 tx=0x7fca8800b390 comp rx=0 tx=0).stop 2024-09-16T00:37:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 msgr2=0x7fca9819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 0x7fca9819a670 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fca94006010 tx=0x7fca940042f0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 shutdown_connections 2024-09-16T00:37:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fca840a3240 0x7fca840a5700 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9810e480 0x7fca9819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca98101280 0x7fca9819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 --2- 172.21.15.1:0/4274334818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca981008d0 0x7fca9819a670 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 >> 172.21.15.1:0/4274334818 conn(0x7fca980fc7a0 msgr2=0x7fca980feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 shutdown_connections 2024-09-16T00:37:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:06.507+0000 7fca9fd89640 1 -- 172.21.15.1:0/4274334818 wait complete. 2024-09-16T00:37:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:06 smithi019 ceph-mon[35186]: pgmap v611: 129 pgs: 129 active+clean; 4.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:37:07.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:07 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4274334818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:07.358 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:37:07.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:07 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4274334818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:07.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:07 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4274334818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:08.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.116+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3458896156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0106f10 msgr2=0x7fe8a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.116+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3458896156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0106f10 0x7fe8a010ea90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe898009ca0 tx=0x7fe89802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.116+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3458896156 shutdown_connections 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.116+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3458896156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0106f10 0x7fe8a010ea90 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.116+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3458896156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a0102790 0x7fe8a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.116+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3458896156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8a0101de0 0x7fe8a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.116+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3458896156 >> 172.21.15.1:0/3458896156 conn(0x7fe8a00fd8c0 msgr2=0x7fe8a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.116+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3458896156 shutdown_connections 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.116+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3458896156 wait complete. 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a86c7640 1 Processor -- start 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a86c7640 1 -- start start 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a86c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0101de0 0x7fe8a00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a86c7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8a0102790 0x7fe8a0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a86c7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a0106f10 0x7fe8a0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a86c7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0111f70 con 0x7fe8a0101de0 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a86c7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0111df0 con 0x7fe8a0106f10 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a86c7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01120f0 con 0x7fe8a0102790 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a6463640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0101de0 0x7fe8a00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.117+0000 7fe8a6463640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0101de0 0x7fe8a00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51134/0 (socket says 172.21.15.1:51134) 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a6463640 1 -- 172.21.15.1:0/3139958321 learned_addr learned my addr 172.21.15.1:0/3139958321 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a0106f10 0x7fe8a0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8a0102790 0x7fe8a0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a6463640 1 -- 172.21.15.1:0/3139958321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8a0102790 msgr2=0x7fe8a0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a6463640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8a0102790 0x7fe8a0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a6463640 1 -- 172.21.15.1:0/3139958321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a0106f10 msgr2=0x7fe8a0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a6463640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a0106f10 0x7fe8a0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a6463640 1 -- 172.21.15.1:0/3139958321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a01a6890 con 0x7fe8a0101de0 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a6463640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0101de0 0x7fe8a00744f0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00d770 tx=0x7fe89c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8977fe640 1 -- 172.21.15.1:0/3139958321 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c017070 con 0x7fe8a0101de0 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8977fe640 1 -- 172.21.15.1:0/3139958321 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe89c005500 con 0x7fe8a0101de0 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3139958321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a01a6b80 con 0x7fe8a0101de0 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8977fe640 1 -- 172.21.15.1:0/3139958321 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c004e00 con 0x7fe8a0101de0 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.118+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3139958321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a01a7070 con 0x7fe8a0101de0 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.121+0000 7fe8977fe640 1 -- 172.21.15.1:0/3139958321 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe89c004fa0 con 0x7fe8a0101de0 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.121+0000 7fe8aaaa0740 1 Processor -- start 2024-09-16T00:37:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.121+0000 7fe8aaaa0740 1 -- start start 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.121+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a1e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.121+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 0x5595b3a1ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.121+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.121+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5595b39842f0 con 0x5595b3a29370 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.121+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5595b392bcf0 con 0x5595b3a1fb40 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.121+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595b39802b0 con 0x5595b3a1e5d0 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a5c62640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a5c62640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51146/0 (socket says 172.21.15.1:51146) 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a5c62640 1 -- 172.21.15.1:0/2384701462 learned_addr learned my addr 172.21.15.1:0/2384701462 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8977fe640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe88c0a3240 0x7fe88c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a6463640 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 0x5595b3a1ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a1e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe88c0a3240 0x7fe88c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a5c62640 1 -- 172.21.15.1:0/2384701462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 msgr2=0x5595b3a1ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 0x5595b3a1ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a5c62640 1 -- 172.21.15.1:0/2384701462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 msgr2=0x5595b3a1e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a1e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a5c62640 1 -- 172.21.15.1:0/2384701462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b39103a0 con 0x5595b3a29370 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8977fe640 1 -- 172.21.15.1:0/3139958321 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe89c0cf740 con 0x7fe8a0101de0 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a1e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2b760 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe89000bc30 tx=0x7fe8900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894d01969814d162 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe894ff9640 1 -- 172.21.15.1:0/2384701462 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89001c480 con 0x5595b3a29370 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.122+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe88c0a3240 0x7fe88c0a5700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe898004af0 tx=0x7fe898004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe894ff9640 1 -- 172.21.15.1:0/2384701462 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe89001cb00 con 0x5595b3a29370 2024-09-16T00:37:08.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe894ff9640 1 -- 172.21.15.1:0/2384701462 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890011b50 con 0x5595b3a29370 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2384701462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 msgr2=0x5595b3a2b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2b760 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe89000bc30 tx=0x7fe8900107a0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2384701462 shutdown_connections 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2b760 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 0x5595b3a1ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2384701462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a1e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2384701462 >> 172.21.15.1:0/2384701462 conn(0x5595b39e7310 msgr2=0x5595b3a1ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2384701462 shutdown_connections 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.123+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2384701462 wait complete. 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8aaaa0740 1 -- start start 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8aaaa0740 1 Processor -- start 2024-09-16T00:37:08.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a1e5d0 0x5595b3a28d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a25fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a29370 0x5595b3a26570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5595b3a2c070 con 0x5595b3a1e5d0 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5595b393f060 con 0x5595b3a1fb40 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595b3a2c1f0 con 0x5595b3a29370 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8aaaa0740 10 client.0 _ll_register_callbacks cb 0x5595b36928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8a6463640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a1e5d0 0x5595b3a28d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8a6463640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a1e5d0 0x5595b3a28d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51152/0 (socket says 172.21.15.1:51152) 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8a6463640 1 -- 172.21.15.1:0/2623834704 learned_addr learned my addr 172.21.15.1:0/2623834704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.124+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a29370 0x5595b3a26570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8a6463640 1 -- 172.21.15.1:0/2623834704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a29370 msgr2=0x5595b3a26570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8a6463640 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a29370 0x5595b3a26570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8a6463640 1 -- 172.21.15.1:0/2623834704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 msgr2=0x5595b3a25fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a25fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8a6463640 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a25fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8a6463640 1 -- 172.21.15.1:0/2623834704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b36f53c0 con 0x5595b3a1e5d0 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a25fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8a6463640 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a1e5d0 0x5595b3a28d20 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0ccde0 tx=0x7fe89c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56e556d679f880a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe889ffb640 1 -- 172.21.15.1:0/2623834704 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c020040 con 0x5595b3a1e5d0 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe889ffb640 1 -- 172.21.15.1:0/2623834704 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe89c0860b0 con 0x5595b3a1e5d0 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2623834704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe89001cb00 con 0x5595b3a1e5d0 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe889ffb640 1 -- 172.21.15.1:0/2623834704 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe89c09b9b0 con 0x5595b3a1e5d0 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8aaaa0740 10 client.17574 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2623834704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595b3a37af0 con 0x5595b3a1e5d0 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe889ffb640 1 -- 172.21.15.1:0/2623834704 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe89001cb00 con 0x5595b3a1e5d0 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe889ffb640 1 -- 172.21.15.1:0/2623834704 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe89c0cf3c0 con 0x5595b3a1e5d0 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8aaaa0740 10 client.17574 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.125+0000 7fe8aaaa0740 10 client.17574 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.126+0000 7fe8aaaa0740 20 client.17574 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.126+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5595b3a39000 0x5595b3a3b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.126+0000 7fe8aaaa0740 4 client.17574 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.126+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2623834704 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595b3991860 con 0x5595b3a39000 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.126+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5595b3a39000 0x5595b3a3b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.126+0000 7fe889ffb640 10 client.17574 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.126+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5595b3a39000 0x5595b3a3b470 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4e0235a1ae4a073 server_cookie=b9bc22c5f6302af1 in_seq=0 out_seq=0 2024-09-16T00:37:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe889ffb640 1 -- 172.21.15.1:0/2623834704 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595b3991860 con 0x5595b3a39000 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe889ffb640 10 client.17574 handle_command_reply: tid=0 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 client.17574 shutdown 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2623834704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a1e5d0 msgr2=0x5595b3a28d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a1e5d0 0x5595b3a28d20 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0ccde0 tx=0x7fe89c01e000 comp rx=0 tx=0).stop 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2623834704 shutdown_connections 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a29370 0x5595b3a26570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5595b3a39000 0x5595b3a3b470 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a25fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe889ffb640 0 client.17574 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/2623834704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a1e5d0 0x5595b3a28d20 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2623834704 >> 172.21.15.1:0/2623834704 conn(0x5595b39e7310 msgr2=0x5595b3a20960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2623834704 shutdown_connections 2024-09-16T00:37:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/2623834704 wait complete. 2024-09-16T00:37:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.127+0000 7fe8aaaa0740 20 client.17574 trim_cache size 0 max 16384 2024-09-16T00:37:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8aaaa0740 1 Processor -- start 2024-09-16T00:37:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8aaaa0740 1 -- start start 2024-09-16T00:37:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1e5d0 0x5595b3a35340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1fb40 0x5595b3a2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b38e0 con 0x5595b3a29370 2024-09-16T00:37:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b11b0 con 0x5595b3a1e5d0 2024-09-16T00:37:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01af000 con 0x5595b3a1fb40 2024-09-16T00:37:08.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6c64640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6c64640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51156/0 (socket says 172.21.15.1:51156) 2024-09-16T00:37:08.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6c64640 1 -- 172.21.15.1:0/743837691 learned_addr learned my addr 172.21.15.1:0/743837691 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:08.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1fb40 0x5595b3a2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6463640 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1e5d0 0x5595b3a35340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6c64640 1 -- 172.21.15.1:0/743837691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1fb40 msgr2=0x5595b3a2a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:08.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1fb40 0x5595b3a2a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6c64640 1 -- 172.21.15.1:0/743837691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1e5d0 msgr2=0x5595b3a35340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:08.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1e5d0 0x5595b3a35340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6c64640 1 -- 172.21.15.1:0/743837691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b3a24520 con 0x5595b3a29370 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.143+0000 7fe8a6463640 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1e5d0 0x5595b3a35340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.144+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2aca0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe898009ca0 tx=0x7fe898043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d4514adcfcb314 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.144+0000 7fe88bfff640 1 -- 172.21.15.1:0/743837691 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898052070 con 0x5595b3a29370 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.144+0000 7fe88bfff640 1 -- 172.21.15.1:0/743837691 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe898036040 con 0x5595b3a29370 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.144+0000 7fe88bfff640 1 -- 172.21.15.1:0/743837691 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898049690 con 0x5595b3a29370 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.144+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/743837691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 msgr2=0x5595b3a2aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.144+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2aca0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe898009ca0 tx=0x7fe898043000 comp rx=0 tx=0).stop 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/743837691 shutdown_connections 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a2aca0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1fb40 0x5595b3a2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/743837691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1e5d0 0x5595b3a35340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/743837691 >> 172.21.15.1:0/743837691 conn(0x5595b39e7310 msgr2=0x5595b3a248a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/743837691 shutdown_connections 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/743837691 wait complete. 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 -- start start 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 Processor -- start 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 0x5595b3a40830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a40d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01af670 con 0x5595b3a29370 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b0ae0 con 0x5595b3a1fb40 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01afd40 con 0x5595b3a1e5d0 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8aaaa0740 10 client.0 _ll_register_callbacks cb 0x7fe89c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8a6c64640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8a6c64640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51162/0 (socket says 172.21.15.1:51162) 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8a6c64640 1 -- 172.21.15.1:0/1412301048 learned_addr learned my addr 172.21.15.1:0/1412301048 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.145+0000 7fe8a5c62640 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a40d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8a6463640 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 0x5595b3a40830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8a6c64640 1 -- 172.21.15.1:0/1412301048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 msgr2=0x5595b3a40830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 0x5595b3a40830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8a6c64640 1 -- 172.21.15.1:0/1412301048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 msgr2=0x5595b3a40d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a40d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8a6c64640 1 -- 172.21.15.1:0/1412301048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b3a2f900 con 0x5595b3a29370 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8a6c64640 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a412b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe898006010 tx=0x7fe89802fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b4e6a08c51851c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe889ffb640 1 -- 172.21.15.1:0/1412301048 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898052070 con 0x5595b3a29370 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe889ffb640 1 -- 172.21.15.1:0/1412301048 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe898003970 con 0x5595b3a29370 2024-09-16T00:37:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/1412301048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe898036040 con 0x5595b3a29370 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe889ffb640 1 -- 172.21.15.1:0/1412301048 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe898003b70 con 0x5595b3a29370 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8aaaa0740 10 client.17580 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/1412301048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595b3aa3e90 con 0x5595b3a29370 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.146+0000 7fe889ffb640 1 -- 172.21.15.1:0/1412301048 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe898036040 con 0x5595b3a29370 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe889ffb640 1 -- 172.21.15.1:0/1412301048 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe898004640 con 0x5595b3a29370 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe8aaaa0740 10 client.17580 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe8aaaa0740 10 client.17580 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe8aaaa0740 20 client.17580 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5595b3aa4590 0x5595b3aa69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe8aaaa0740 4 client.17580 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:37:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/1412301048 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595b398bd70 con 0x5595b3aa4590 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe8a6463640 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5595b3aa4590 0x5595b3aa69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe8a6463640 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5595b3aa4590 0x5595b3aa69b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a97a1bc479892eb server_cookie=296d51beea6f0c1 in_seq=0 out_seq=0 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.147+0000 7fe889ffb640 10 client.17580 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe889ffb640 1 -- 172.21.15.1:0/1412301048 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595b398bd70 con 0x5595b3aa4590 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe889ffb640 10 client.17580 handle_command_reply: tid=0 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 client.17580 shutdown 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/1412301048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 msgr2=0x5595b3a412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a412b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe898006010 tx=0x7fe89802fc90 comp rx=0 tx=0).stop 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/1412301048 shutdown_connections 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5595b3a29370 0x5595b3a412b0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5595b3aa4590 0x5595b3aa69b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595b3a1fb40 0x5595b3a40d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 --2- 172.21.15.1:0/1412301048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595b3a1e5d0 0x5595b3a40830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/1412301048 >> 172.21.15.1:0/1412301048 conn(0x5595b39e7310 msgr2=0x5595b3a20110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe889ffb640 0 client.17580 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:37:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/1412301048 shutdown_connections 2024-09-16T00:37:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 1 -- 172.21.15.1:0/1412301048 wait complete. 2024-09-16T00:37:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8aaaa0740 20 client.17580 trim_cache size 0 max 16384 2024-09-16T00:37:08.156 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:37:08.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe88c0a3240 msgr2=0x7fe88c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe88c0a3240 0x7fe88c0a5700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe898004af0 tx=0x7fe898004a80 comp rx=0 tx=0).stop 2024-09-16T00:37:08.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0101de0 msgr2=0x7fe8a00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.148+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0101de0 0x7fe8a00744f0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00d770 tx=0x7fe89c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:37:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.149+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3139958321 shutdown_connections 2024-09-16T00:37:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.149+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a0106f10 0x7fe8a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.149+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe88c0a3240 0x7fe88c0a5700 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.149+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8a0102790 0x7fe8a0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.149+0000 7fe8a86c7640 1 --2- 172.21.15.1:0/3139958321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8a0101de0 0x7fe8a00744f0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.149+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3139958321 >> 172.21.15.1:0/3139958321 conn(0x7fe8a00fd8c0 msgr2=0x7fe8a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.149+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3139958321 shutdown_connections 2024-09-16T00:37:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:08.149+0000 7fe8a86c7640 1 -- 172.21.15.1:0/3139958321 wait complete. 2024-09-16T00:37:08.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:08 smithi001 ceph-mon[30934]: pgmap v612: 129 pgs: 129 active+clean; 4.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:37:08.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:08 smithi153 ceph-mon[35538]: pgmap v612: 129 pgs: 129 active+clean; 4.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:37:08.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:08 smithi019 ceph-mon[35186]: pgmap v612: 129 pgs: 129 active+clean; 4.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:37:08.982 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:37:09.245 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.710+0000 7fca14215640 1 -- 172.21.15.1:0/1278147520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c102ef0 msgr2=0x7fca0c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.710+0000 7fca14215640 1 --2- 172.21.15.1:0/1278147520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c102ef0 0x7fca0c10da40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fca04009ca0 tx=0x7fca0402a950 comp rx=0 tx=0).stop 2024-09-16T00:37:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.710+0000 7fca14215640 1 -- 172.21.15.1:0/1278147520 shutdown_connections 2024-09-16T00:37:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.710+0000 7fca14215640 1 --2- 172.21.15.1:0/1278147520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c102ef0 0x7fca0c10da40 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.710+0000 7fca14215640 1 --2- 172.21.15.1:0/1278147520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0c102570 0x7fca0c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.710+0000 7fca14215640 1 --2- 172.21.15.1:0/1278147520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca0c100e20 0x7fca0c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.710+0000 7fca14215640 1 -- 172.21.15.1:0/1278147520 >> 172.21.15.1:0/1278147520 conn(0x7fca0c0fc7d0 msgr2=0x7fca0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.710+0000 7fca14215640 1 -- 172.21.15.1:0/1278147520 shutdown_connections 2024-09-16T00:37:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.710+0000 7fca14215640 1 -- 172.21.15.1:0/1278147520 wait complete. 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca14215640 1 Processor -- start 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca14215640 1 -- start start 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca14215640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c100e20 0x7fca0c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca14215640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca0c102570 0x7fca0c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca14215640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0c102ef0 0x7fca0c1a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca14215640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c110ff0 con 0x7fca0c100e20 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca14215640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c110e70 con 0x7fca0c102ef0 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca14215640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c111170 con 0x7fca0c102570 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca11fb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c100e20 0x7fca0c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca11fb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c100e20 0x7fca0c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51174/0 (socket says 172.21.15.1:51174) 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.711+0000 7fca11fb1640 1 -- 172.21.15.1:0/2375824421 learned_addr learned my addr 172.21.15.1:0/2375824421 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca127b2640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0c102ef0 0x7fca0c1a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca117b0640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca0c102570 0x7fca0c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca11fb1640 1 -- 172.21.15.1:0/2375824421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca0c102570 msgr2=0x7fca0c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca11fb1640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca0c102570 0x7fca0c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca11fb1640 1 -- 172.21.15.1:0/2375824421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0c102ef0 msgr2=0x7fca0c1a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca11fb1640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0c102ef0 0x7fca0c1a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca11fb1640 1 -- 172.21.15.1:0/2375824421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0c1a4920 con 0x7fca0c100e20 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca11fb1640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c100e20 0x7fca0c199360 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fca0800b4a0 tx=0x7fca0800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca02ffd640 1 -- 172.21.15.1:0/2375824421 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08014070 con 0x7fca0c100e20 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca14215640 1 -- 172.21.15.1:0/2375824421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0c1a4af0 con 0x7fca0c100e20 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca02ffd640 1 -- 172.21.15.1:0/2375824421 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca08010af0 con 0x7fca0c100e20 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca02ffd640 1 -- 172.21.15.1:0/2375824421 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08017d10 con 0x7fca0c100e20 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.712+0000 7fca14215640 1 -- 172.21.15.1:0/2375824421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0c1a5000 con 0x7fca0c100e20 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.714+0000 7fca02ffd640 1 -- 172.21.15.1:0/2375824421 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fca08017070 con 0x7fca0c100e20 2024-09-16T00:37:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca165ee740 1 Processor -- start 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca165ee740 1 -- start start 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca165ee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e4186090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca165ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e4186a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca165ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4193760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca165ee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5562e40ec2f0 con 0x5562e4187b40 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca165ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5562e4093cf0 con 0x5562e4191370 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca165ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5562e40e82b0 con 0x5562e41865d0 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca02ffd640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9f80a3240 0x7fc9f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca127b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e4186090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca127b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e4186090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51176/0 (socket says 172.21.15.1:51176) 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca127b2640 1 -- 172.21.15.1:0/1951836022 learned_addr learned my addr 172.21.15.1:0/1951836022 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.715+0000 7fca11fb1640 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e4186a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca127b2640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9f80a3240 0x7fc9f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca117b0640 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4193760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca02ffd640 1 -- 172.21.15.1:0/2375824421 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fca080cf170 con 0x7fca0c100e20 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca127b2640 1 -- 172.21.15.1:0/1951836022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 msgr2=0x5562e4186a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca127b2640 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e4186a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca127b2640 1 -- 172.21.15.1:0/1951836022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 msgr2=0x5562e4193760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca127b2640 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4193760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca127b2640 1 -- 172.21.15.1:0/1951836022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e40783a0 con 0x5562e4187b40 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca117b0640 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4193760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fca127b2640 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e4186090 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fca04009ca0 tx=0x7fca04003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239af28a14e761b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fc9f7fff640 1 -- 172.21.15.1:0/1951836022 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04043050 con 0x5562e4187b40 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fc9f7fff640 1 -- 172.21.15.1:0/1951836022 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca04044040 con 0x5562e4187b40 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.716+0000 7fc9f7fff640 1 -- 172.21.15.1:0/1951836022 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0404d020 con 0x5562e4187b40 2024-09-16T00:37:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 -- 172.21.15.1:0/1951836022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 msgr2=0x5562e4186090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca127b2640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9f80a3240 0x7fc9f80a5700 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fca0404e000 tx=0x7fca0404c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e4186090 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fca04009ca0 tx=0x7fca04003bb0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 -- 172.21.15.1:0/1951836022 shutdown_connections 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4193760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e4186a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 --2- 172.21.15.1:0/1951836022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e4186090 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 -- 172.21.15.1:0/1951836022 >> 172.21.15.1:0/1951836022 conn(0x5562e414f310 msgr2=0x5562e4182c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 -- 172.21.15.1:0/1951836022 shutdown_connections 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 -- 172.21.15.1:0/1951836022 wait complete. 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.717+0000 7fca165ee740 1 -- start start 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca165ee740 1 Processor -- start 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca165ee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e41865d0 0x5562e4190d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca165ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e4187b40 0x5562e418e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca165ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e418e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca165ee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5562e4194070 con 0x5562e41865d0 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca165ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5562e40a7060 con 0x5562e4191370 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca165ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5562e41941f0 con 0x5562e4187b40 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca165ee740 10 client.0 _ll_register_callbacks cb 0x5562e3dfa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca11fb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e41865d0 0x5562e4190d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca11fb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e41865d0 0x5562e4190d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51186/0 (socket says 172.21.15.1:51186) 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca11fb1640 1 -- 172.21.15.1:0/2117300145 learned_addr learned my addr 172.21.15.1:0/2117300145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca117b0640 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e4187b40 0x5562e418e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca11fb1640 1 -- 172.21.15.1:0/2117300145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e4187b40 msgr2=0x5562e418e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca11fb1640 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e4187b40 0x5562e418e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca11fb1640 1 -- 172.21.15.1:0/2117300145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 msgr2=0x5562e418e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca11fb1640 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e418e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.718+0000 7fca11fb1640 1 -- 172.21.15.1:0/2117300145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e3e5d3c0 con 0x5562e41865d0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca11fb1640 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e41865d0 0x5562e4190d80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fca08097d00 tx=0x7fca0806afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb056b26757bc5c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fc9f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fc9f57fa640 1 -- 172.21.15.1:0/2117300145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0801e040 con 0x5562e41865d0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fc9f57fa640 1 -- 172.21.15.1:0/2117300145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fca08010ca0 con 0x5562e41865d0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca165ee740 1 -- 172.21.15.1:0/2117300145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca04044040 con 0x5562e41865d0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fc9f57fa640 1 -- 172.21.15.1:0/2117300145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca0809b330 con 0x5562e41865d0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca165ee740 10 client.17592 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca165ee740 1 -- 172.21.15.1:0/2117300145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5562e419fb40 con 0x5562e41865d0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fc9f57fa640 1 -- 172.21.15.1:0/2117300145 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca04044040 con 0x5562e41865d0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fc9f57fa640 1 -- 172.21.15.1:0/2117300145 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fca08071030 con 0x5562e41865d0 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca165ee740 10 client.17592 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca165ee740 10 client.17592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca165ee740 20 client.17592 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca165ee740 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5562e41a1000 0x5562e41a3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca165ee740 4 client.17592 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.719+0000 7fca165ee740 1 -- 172.21.15.1:0/2117300145 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562e40f9860 con 0x5562e41a1000 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.720+0000 7fca117b0640 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5562e41a1000 0x5562e41a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.720+0000 7fc9f57fa640 10 client.17592 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.720+0000 7fca117b0640 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5562e41a1000 0x5562e41a3470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=127ef519d01f7c24 server_cookie=8793c87ed05306ad in_seq=0 out_seq=0 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fc9f57fa640 1 -- 172.21.15.1:0/2117300145 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562e40f9860 con 0x5562e41a1000 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fc9f57fa640 10 client.17592 handle_command_reply: tid=0 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 client.17592 shutdown 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 -- 172.21.15.1:0/2117300145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e41865d0 msgr2=0x5562e4190d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e41865d0 0x5562e4190d80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fca08097d00 tx=0x7fca0806afa0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 -- 172.21.15.1:0/2117300145 shutdown_connections 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e418e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e4187b40 0x5562e418e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5562e41a1000 0x5562e41a3470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 --2- 172.21.15.1:0/2117300145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e41865d0 0x5562e4190d80 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 -- 172.21.15.1:0/2117300145 >> 172.21.15.1:0/2117300145 conn(0x5562e414f310 msgr2=0x5562e4188960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fc9f57fa640 0 client.17592 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 -- 172.21.15.1:0/2117300145 shutdown_connections 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 1 -- 172.21.15.1:0/2117300145 wait complete. 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.721+0000 7fca165ee740 20 client.17592 trim_cache size 0 max 16384 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca165ee740 1 Processor -- start 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca165ee740 1 -- start start 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca165ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e419d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca165ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4187b40 0x5562e4181c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca165ee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4191370 0x5562e41a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca165ee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1b17f0 con 0x5562e4191370 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca165ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1af0c0 con 0x5562e4187b40 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca165ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1acf10 con 0x5562e41865d0 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca127b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4191370 0x5562e41a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca127b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4191370 0x5562e41a8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51202/0 (socket says 172.21.15.1:51202) 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca127b2640 1 -- 172.21.15.1:0/1482327111 learned_addr learned my addr 172.21.15.1:0/1482327111 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.734+0000 7fca117b0640 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4187b40 0x5562e4181c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fca127b2640 1 -- 172.21.15.1:0/1482327111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 msgr2=0x5562e419d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fca11fb1640 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e419d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fca127b2640 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e419d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fca127b2640 1 -- 172.21.15.1:0/1482327111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4187b40 msgr2=0x5562e4181c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fca127b2640 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4187b40 0x5562e4181c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fca127b2640 1 -- 172.21.15.1:0/1482327111 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e4189120 con 0x5562e4191370 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fca117b0640 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4187b40 0x5562e4181c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fca127b2640 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4191370 0x5562e41a8d40 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fca0402f810 tx=0x7fca0403c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea5a7c440bc7dcd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fc9f77fe640 1 -- 172.21.15.1:0/1482327111 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0404d050 con 0x5562e4191370 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fc9f77fe640 1 -- 172.21.15.1:0/1482327111 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca04049d40 con 0x5562e4191370 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.735+0000 7fc9f77fe640 1 -- 172.21.15.1:0/1482327111 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04043040 con 0x5562e4191370 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 -- 172.21.15.1:0/1482327111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4191370 msgr2=0x5562e41a8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4191370 0x5562e41a8d40 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fca0402f810 tx=0x7fca0403c030 comp rx=0 tx=0).stop 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 -- 172.21.15.1:0/1482327111 shutdown_connections 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4191370 0x5562e41a8d40 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4187b40 0x5562e4181c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 --2- 172.21.15.1:0/1482327111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e419d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 -- 172.21.15.1:0/1482327111 >> 172.21.15.1:0/1482327111 conn(0x5562e414f310 msgr2=0x5562e4181500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 -- 172.21.15.1:0/1482327111 shutdown_connections 2024-09-16T00:37:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 -- 172.21.15.1:0/1482327111 wait complete. 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 -- start start 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.736+0000 7fca165ee740 1 Processor -- start 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca165ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e4195380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca165ee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e41958c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca165ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4195e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca165ee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1ad580 con 0x5562e4187b40 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca165ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1ae9f0 con 0x5562e4191370 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca165ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1adc50 con 0x5562e41865d0 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca165ee740 10 client.0 _ll_register_callbacks cb 0x7fca08017630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca117b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e41958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca11fb1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e4195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca117b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e41958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51214/0 (socket says 172.21.15.1:51214) 2024-09-16T00:37:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca117b0640 1 -- 172.21.15.1:0/1657762565 learned_addr learned my addr 172.21.15.1:0/1657762565 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca127b2640 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca117b0640 1 -- 172.21.15.1:0/1657762565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 msgr2=0x5562e4195380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca117b0640 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e4195380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca117b0640 1 -- 172.21.15.1:0/1657762565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 msgr2=0x5562e4195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca117b0640 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca117b0640 1 -- 172.21.15.1:0/1657762565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e420aeb0 con 0x5562e4187b40 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca127b2640 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fca117b0640 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e41958c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc005780 tx=0x7fc9fc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d804067083d7c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fc9f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.737+0000 7fc9f57fa640 1 -- 172.21.15.1:0/1657762565 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc004440 con 0x5562e4187b40 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca165ee740 1 -- 172.21.15.1:0/1657762565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca04049d40 con 0x5562e4187b40 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fc9f57fa640 1 -- 172.21.15.1:0/1657762565 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc9fc0035f0 con 0x5562e4187b40 2024-09-16T00:37:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fc9f57fa640 1 -- 172.21.15.1:0/1657762565 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9fc02c050 con 0x5562e4187b40 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca165ee740 10 client.17598 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca165ee740 1 -- 172.21.15.1:0/1657762565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5562e420c1f0 con 0x5562e4187b40 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fc9f57fa640 1 -- 172.21.15.1:0/1657762565 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca04049d40 con 0x5562e4187b40 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fc9f57fa640 1 -- 172.21.15.1:0/1657762565 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc9fc0248d0 con 0x5562e4187b40 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca165ee740 10 client.17598 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca165ee740 10 client.17598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca165ee740 20 client.17598 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca165ee740 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5562e420c940 0x5562e420ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca165ee740 4 client.17598 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca165ee740 1 -- 172.21.15.1:0/1657762565 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562e40f3d70 con 0x5562e420c940 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.738+0000 7fca11fb1640 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5562e420c940 0x5562e420ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca11fb1640 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5562e420c940 0x5562e420ed60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea9f204cbf8a3e69 server_cookie=f25bb0e9fa0cac60 in_seq=0 out_seq=0 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fc9f57fa640 10 client.17598 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fc9f57fa640 1 -- 172.21.15.1:0/1657762565 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562e40f3d70 con 0x5562e420c940 2024-09-16T00:37:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fc9f57fa640 10 client.17598 handle_command_reply: tid=0 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 client.17598 shutdown 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 -- 172.21.15.1:0/1657762565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 msgr2=0x5562e41958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e41958c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc005780 tx=0x7fc9fc012820 comp rx=0 tx=0).stop 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 -- 172.21.15.1:0/1657762565 shutdown_connections 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4191370 0x5562e4195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5562e420c940 0x5562e420ed60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562e4187b40 0x5562e41958c0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 --2- 172.21.15.1:0/1657762565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562e41865d0 0x5562e4195380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fc9f57fa640 0 client.17598 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 -- 172.21.15.1:0/1657762565 >> 172.21.15.1:0/1657762565 conn(0x5562e414f310 msgr2=0x5562e418c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 -- 172.21.15.1:0/1657762565 shutdown_connections 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 1 -- 172.21.15.1:0/1657762565 wait complete. 2024-09-16T00:37:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.739+0000 7fca165ee740 20 client.17598 trim_cache size 0 max 16384 2024-09-16T00:37:09.756 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:37:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 -- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9f80a3240 msgr2=0x7fc9f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9f80a3240 0x7fc9f80a5700 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fca0404e000 tx=0x7fca0404c000 comp rx=0 tx=0).stop 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 -- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c100e20 msgr2=0x7fca0c199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c100e20 0x7fca0c199360 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fca0800b4a0 tx=0x7fca0800b960 comp rx=0 tx=0).stop 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 -- 172.21.15.1:0/2375824421 shutdown_connections 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0c102ef0 0x7fca0c1a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9f80a3240 0x7fc9f80a5700 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca0c102570 0x7fca0c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 --2- 172.21.15.1:0/2375824421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c100e20 0x7fca0c199360 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 -- 172.21.15.1:0/2375824421 >> 172.21.15.1:0/2375824421 conn(0x7fca0c0fc7d0 msgr2=0x7fca0c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 -- 172.21.15.1:0/2375824421 shutdown_connections 2024-09-16T00:37:09.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:09.740+0000 7fca14215640 1 -- 172.21.15.1:0/2375824421 wait complete. 2024-09-16T00:37:10.409 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:10 smithi019 ceph-mon[35186]: pgmap v613: 129 pgs: 129 active+clean; 4.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:10 smithi001 ceph-mon[30934]: pgmap v613: 129 pgs: 129 active+clean; 4.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:10.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:10 smithi153 ceph-mon[35538]: pgmap v613: 129 pgs: 129 active+clean; 4.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:10.754 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:37:11.018 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.514+0000 7fa35e63f640 1 -- 172.21.15.1:0/590008437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358102790 msgr2=0x7fa3581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.514+0000 7fa35e63f640 1 --2- 172.21.15.1:0/590008437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358102790 0x7fa3581067e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009900 tx=0x7fa34c02a410 comp rx=0 tx=0).stop 2024-09-16T00:37:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.514+0000 7fa35e63f640 1 -- 172.21.15.1:0/590008437 shutdown_connections 2024-09-16T00:37:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.514+0000 7fa35e63f640 1 --2- 172.21.15.1:0/590008437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa358106f10 0x7fa35810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.514+0000 7fa35e63f640 1 --2- 172.21.15.1:0/590008437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358102790 0x7fa3581067e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.514+0000 7fa35e63f640 1 --2- 172.21.15.1:0/590008437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa358101de0 0x7fa3581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.514+0000 7fa35e63f640 1 -- 172.21.15.1:0/590008437 >> 172.21.15.1:0/590008437 conn(0x7fa3580fd8c0 msgr2=0x7fa3580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.514+0000 7fa35e63f640 1 -- 172.21.15.1:0/590008437 shutdown_connections 2024-09-16T00:37:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.514+0000 7fa35e63f640 1 -- 172.21.15.1:0/590008437 wait complete. 2024-09-16T00:37:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35e63f640 1 Processor -- start 2024-09-16T00:37:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35e63f640 1 -- start start 2024-09-16T00:37:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35e63f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa358101de0 0x7fa3581983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35e63f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa358102790 0x7fa358198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35e63f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358106f10 0x7fa35819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35e63f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa358111fc0 con 0x7fa358106f10 2024-09-16T00:37:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35e63f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa358111e40 con 0x7fa358102790 2024-09-16T00:37:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35e63f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa358112140 con 0x7fa358101de0 2024-09-16T00:37:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35cbdc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358106f10 0x7fa35819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35cbdc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358106f10 0x7fa35819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38684/0 (socket says 172.21.15.1:38684) 2024-09-16T00:37:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa35cbdc640 1 -- 172.21.15.1:0/723111578 learned_addr learned my addr 172.21.15.1:0/723111578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa3577fe640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa358102790 0x7fa358198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.515+0000 7fa357fff640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa358101de0 0x7fa3581983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa35cbdc640 1 -- 172.21.15.1:0/723111578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa358101de0 msgr2=0x7fa3581983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa35cbdc640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa358101de0 0x7fa3581983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa35cbdc640 1 -- 172.21.15.1:0/723111578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa358102790 msgr2=0x7fa358198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa35cbdc640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa358102790 0x7fa358198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa35cbdc640 1 -- 172.21.15.1:0/723111578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35819d370 con 0x7fa358106f10 2024-09-16T00:37:11.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa35cbdc640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358106f10 0x7fa35819cc90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa35000cce0 tx=0x7fa35000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:11.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa3557fa640 1 -- 172.21.15.1:0/723111578 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35000f450 con 0x7fa358106f10 2024-09-16T00:37:11.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa3557fa640 1 -- 172.21.15.1:0/723111578 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35000fad0 con 0x7fa358106f10 2024-09-16T00:37:11.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa35e63f640 1 -- 172.21.15.1:0/723111578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35819d660 con 0x7fa358106f10 2024-09-16T00:37:11.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa3557fa640 1 -- 172.21.15.1:0/723111578 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350018c10 con 0x7fa358106f10 2024-09-16T00:37:11.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.516+0000 7fa35e63f640 1 -- 172.21.15.1:0/723111578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3581a4f40 con 0x7fa358106f10 2024-09-16T00:37:11.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.518+0000 7fa3557fa640 1 -- 172.21.15.1:0/723111578 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa35000f5f0 con 0x7fa358106f10 2024-09-16T00:37:11.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.518+0000 7fa360a18740 1 Processor -- start 2024-09-16T00:37:11.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa360a18740 1 -- start start 2024-09-16T00:37:11.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa360a18740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b90b40 0x55aa05b8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa360a18740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b8f5d0 0x55aa05b8fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa360a18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b9e780 0x55aa05ba0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa360a18740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aa05af52f0 con 0x55aa05b90b40 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa360a18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aa05a9ccf0 con 0x55aa05b9e780 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa360a18740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aa05af12b0 con 0x55aa05b8f5d0 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b90b40 0x55aa05b8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b90b40 0x55aa05b8f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38686/0 (socket says 172.21.15.1:38686) 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3577fe640 1 -- 172.21.15.1:0/58198570 learned_addr learned my addr 172.21.15.1:0/58198570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3557fa640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3400a32c0 0x7fa3400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa357fff640 1 --2- 172.21.15.1:0/58198570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b8f5d0 0x55aa05b8fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3577fe640 1 -- 172.21.15.1:0/58198570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b8f5d0 msgr2=0x55aa05b8fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3577fe640 1 --2- 172.21.15.1:0/58198570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b8f5d0 0x55aa05b8fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3577fe640 1 -- 172.21.15.1:0/58198570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b9e780 msgr2=0x55aa05ba0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3577fe640 1 --2- 172.21.15.1:0/58198570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b9e780 0x55aa05ba0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3577fe640 1 -- 172.21.15.1:0/58198570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa05a813a0 con 0x55aa05b90b40 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3557fa640 1 -- 172.21.15.1:0/723111578 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa3500d0340 con 0x7fa358106f10 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa35cbdc640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3400a32c0 0x7fa3400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.519+0000 7fa3577fe640 1 --2- 172.21.15.1:0/58198570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b90b40 0x55aa05b8f090 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa34c031550 tx=0x7fa34c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb72a7e8db1391d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa33e7fc640 1 -- 172.21.15.1:0/58198570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c041070 con 0x55aa05b90b40 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa33e7fc640 1 -- 172.21.15.1:0/58198570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa34c005520 con 0x55aa05b90b40 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa33e7fc640 1 -- 172.21.15.1:0/58198570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c035040 con 0x55aa05b90b40 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa360a18740 1 -- 172.21.15.1:0/58198570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b90b40 msgr2=0x55aa05b8f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa360a18740 1 --2- 172.21.15.1:0/58198570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b90b40 0x55aa05b8f090 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa34c031550 tx=0x7fa34c005dd0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa35cbdc640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3400a32c0 0x7fa3400a5780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa35000ab20 tx=0x7fa3500186b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa360a18740 1 -- 172.21.15.1:0/58198570 shutdown_connections 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa360a18740 1 --2- 172.21.15.1:0/58198570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b9e780 0x55aa05ba0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa360a18740 1 --2- 172.21.15.1:0/58198570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b8f5d0 0x55aa05b8fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa360a18740 1 --2- 172.21.15.1:0/58198570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b90b40 0x55aa05b8f090 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa360a18740 1 -- 172.21.15.1:0/58198570 >> 172.21.15.1:0/58198570 conn(0x55aa05b58310 msgr2=0x55aa05b8bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa360a18740 1 -- 172.21.15.1:0/58198570 shutdown_connections 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.520+0000 7fa360a18740 1 -- 172.21.15.1:0/58198570 wait complete. 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa360a18740 1 -- start start 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa360a18740 1 Processor -- start 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa360a18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b8f5d0 0x55aa05b9e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa360a18740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 0x55aa05b9b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa360a18740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b9e780 0x55aa05b9b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa360a18740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aa05ba1420 con 0x55aa05b9e780 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa360a18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aa05ab0060 con 0x55aa05b8f5d0 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa360a18740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aa05ba15a0 con 0x55aa05b90b40 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa360a18740 10 client.0 _ll_register_callbacks cb 0x55aa058038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b9e780 0x55aa05b9b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b9e780 0x55aa05b9b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38702/0 (socket says 172.21.15.1:38702) 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.521+0000 7fa3577fe640 1 -- 172.21.15.1:0/3034976386 learned_addr learned my addr 172.21.15.1:0/3034976386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa357fff640 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 0x55aa05b9b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b8f5d0 0x55aa05b9e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3577fe640 1 -- 172.21.15.1:0/3034976386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 msgr2=0x55aa05b9b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 0x55aa05b9b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3577fe640 1 -- 172.21.15.1:0/3034976386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b8f5d0 msgr2=0x55aa05b9e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b8f5d0 0x55aa05b9e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3577fe640 1 -- 172.21.15.1:0/3034976386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa058663c0 con 0x55aa05b9e780 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b9e780 0x55aa05b9b9e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa34c004fa0 tx=0x7fa34c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97eb64b7de97ac90 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3377fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3377fe640 1 -- 172.21.15.1:0/3034976386 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c035030 con 0x55aa05b9e780 2024-09-16T00:37:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa360a18740 1 -- 172.21.15.1:0/3034976386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa34c005520 con 0x55aa05b9e780 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3377fe640 1 -- 172.21.15.1:0/3034976386 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa34c004910 con 0x55aa05b9e780 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa3377fe640 1 -- 172.21.15.1:0/3034976386 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa34c041070 con 0x55aa05b9e780 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa360a18740 10 client.17610 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.522+0000 7fa360a18740 1 -- 172.21.15.1:0/3034976386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aa05bace80 con 0x55aa05b9e780 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.523+0000 7fa3377fe640 1 -- 172.21.15.1:0/3034976386 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa34c005520 con 0x55aa05b9e780 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.523+0000 7fa3377fe640 1 -- 172.21.15.1:0/3034976386 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa34c053450 con 0x55aa05b9e780 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.523+0000 7fa360a18740 10 client.17610 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.523+0000 7fa360a18740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.523+0000 7fa360a18740 20 client.17610 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.523+0000 7fa360a18740 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aa05bae1d0 0x55aa05bb05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.523+0000 7fa360a18740 4 client.17610 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.523+0000 7fa360a18740 1 -- 172.21.15.1:0/3034976386 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa05b02860 con 0x55aa05bae1d0 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.523+0000 7fa357fff640 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aa05bae1d0 0x55aa05bb05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.524+0000 7fa357fff640 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aa05bae1d0 0x55aa05bb05c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85aa214a3238d20a server_cookie=5685bf1a73752c82 in_seq=0 out_seq=0 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.524+0000 7fa3377fe640 10 client.17610 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.524+0000 7fa3377fe640 1 -- 172.21.15.1:0/3034976386 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa05b02860 con 0x55aa05bae1d0 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.524+0000 7fa3377fe640 10 client.17610 handle_command_reply: tid=0 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.524+0000 7fa360a18740 1 client.17610 shutdown 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 -- 172.21.15.1:0/3034976386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b9e780 msgr2=0x55aa05b9b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b9e780 0x55aa05b9b9e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa34c004fa0 tx=0x7fa34c005850 comp rx=0 tx=0).stop 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 -- 172.21.15.1:0/3034976386 shutdown_connections 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aa05bae1d0 0x55aa05bb05c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b9e780 0x55aa05b9b9e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa3377fe640 0 client.17610 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 0x55aa05b9b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 --2- 172.21.15.1:0/3034976386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b8f5d0 0x55aa05b9e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 -- 172.21.15.1:0/3034976386 >> 172.21.15.1:0/3034976386 conn(0x55aa05b58310 msgr2=0x55aa05b91960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 -- 172.21.15.1:0/3034976386 shutdown_connections 2024-09-16T00:37:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 1 -- 172.21.15.1:0/3034976386 wait complete. 2024-09-16T00:37:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.525+0000 7fa360a18740 20 client.17610 trim_cache size 0 max 16384 2024-09-16T00:37:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa360a18740 1 Processor -- start 2024-09-16T00:37:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa360a18740 1 -- start start 2024-09-16T00:37:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa360a18740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa360a18740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 0x55aa05bb3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa360a18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b9e780 0x55aa05b98650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa360a18740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581af3b0 con 0x55aa05b8f5d0 2024-09-16T00:37:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa360a18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581acc80 con 0x55aa05b9e780 2024-09-16T00:37:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa360a18740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581aaad0 con 0x55aa05b90b40 2024-09-16T00:37:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38710/0 (socket says 172.21.15.1:38710) 2024-09-16T00:37:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.538+0000 7fa3577fe640 1 -- 172.21.15.1:0/3621063279 learned_addr learned my addr 172.21.15.1:0/3621063279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa357fff640 1 --2- 172.21.15.1:0/3621063279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 0x55aa05bb3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa3577fe640 1 -- 172.21.15.1:0/3621063279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 msgr2=0x55aa05bb3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3621063279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 0x55aa05bb3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa3577fe640 1 -- 172.21.15.1:0/3621063279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b9e780 msgr2=0x55aa05b98650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3621063279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b9e780 0x55aa05b98650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa3577fe640 1 -- 172.21.15.1:0/3621063279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa05b92120 con 0x55aa05b8f5d0 2024-09-16T00:37:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa357fff640 1 --2- 172.21.15.1:0/3621063279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 0x55aa05bb3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3621063279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb35e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa34c031460 tx=0x7fa34c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6356fe353b3c4876 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa33dffb640 1 -- 172.21.15.1:0/3621063279 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c035040 con 0x55aa05b8f5d0 2024-09-16T00:37:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa33dffb640 1 -- 172.21.15.1:0/3621063279 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa34c004fd0 con 0x55aa05b8f5d0 2024-09-16T00:37:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.539+0000 7fa33dffb640 1 -- 172.21.15.1:0/3621063279 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0315b0 con 0x55aa05b8f5d0 2024-09-16T00:37:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 -- 172.21.15.1:0/3621063279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 msgr2=0x55aa05bb35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 --2- 172.21.15.1:0/3621063279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb35e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa34c031460 tx=0x7fa34c009250 comp rx=0 tx=0).stop 2024-09-16T00:37:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 -- 172.21.15.1:0/3621063279 shutdown_connections 2024-09-16T00:37:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 --2- 172.21.15.1:0/3621063279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b9e780 0x55aa05b98650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 --2- 172.21.15.1:0/3621063279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b90b40 0x55aa05bb3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 --2- 172.21.15.1:0/3621063279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb35e0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 -- 172.21.15.1:0/3621063279 >> 172.21.15.1:0/3621063279 conn(0x55aa05b58310 msgr2=0x55aa05b9a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 -- 172.21.15.1:0/3621063279 shutdown_connections 2024-09-16T00:37:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 -- 172.21.15.1:0/3621063279 wait complete. 2024-09-16T00:37:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 -- start start 2024-09-16T00:37:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.540+0000 7fa360a18740 1 Processor -- start 2024-09-16T00:37:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa360a18740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa360a18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b90b40 0x55aa05bb1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa360a18740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b9e780 0x55aa05bb1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa360a18740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581ab140 con 0x55aa05b8f5d0 2024-09-16T00:37:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa360a18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581ac5b0 con 0x55aa05b90b40 2024-09-16T00:37:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa360a18740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581ab810 con 0x55aa05b9e780 2024-09-16T00:37:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa360a18740 10 client.0 _ll_register_callbacks cb 0x7fa34c004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38714/0 (socket says 172.21.15.1:38714) 2024-09-16T00:37:11.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3577fe640 1 -- 172.21.15.1:0/3256469982 learned_addr learned my addr 172.21.15.1:0/3256469982 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:11.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa357fff640 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b90b40 0x55aa05bb1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3577fe640 1 -- 172.21.15.1:0/3256469982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b9e780 msgr2=0x55aa05bb1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:11.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b9e780 0x55aa05bb1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3577fe640 1 -- 172.21.15.1:0/3256469982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b90b40 msgr2=0x55aa05bb1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b90b40 0x55aa05bb1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3577fe640 1 -- 172.21.15.1:0/3256469982 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa05c17b10 con 0x55aa05b8f5d0 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3577fe640 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb1310 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0044f0 tx=0x7fa34c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af55e2bda1dd4451 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.541+0000 7fa3377fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa3377fe640 1 -- 172.21.15.1:0/3256469982 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c060020 con 0x55aa05b8f5d0 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa3377fe640 1 -- 172.21.15.1:0/3256469982 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa34c005700 con 0x55aa05b8f5d0 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa360a18740 1 -- 172.21.15.1:0/3256469982 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa34c004fd0 con 0x55aa05b8f5d0 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa3377fe640 1 -- 172.21.15.1:0/3256469982 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa34c035070 con 0x55aa05b8f5d0 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa360a18740 10 client.17616 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa360a18740 1 -- 172.21.15.1:0/3256469982 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aa05c18d00 con 0x55aa05b8f5d0 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa3377fe640 1 -- 172.21.15.1:0/3256469982 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa34c004fd0 con 0x55aa05b8f5d0 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa3377fe640 1 -- 172.21.15.1:0/3256469982 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa34c04f070 con 0x55aa05b8f5d0 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa360a18740 10 client.17616 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa360a18740 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa360a18740 20 client.17616 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa360a18740 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aa05bae1d0 0x55aa05c1b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa360a18740 4 client.17616 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa360a18740 1 -- 172.21.15.1:0/3256469982 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa05afcd70 con 0x55aa05bae1d0 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.542+0000 7fa357fff640 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aa05bae1d0 0x55aa05c1b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:11.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa357fff640 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aa05bae1d0 0x55aa05c1b230 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8809bf904fda66b5 server_cookie=42861c89db5fe714 in_seq=0 out_seq=0 2024-09-16T00:37:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa3377fe640 10 client.17616 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa3377fe640 1 -- 172.21.15.1:0/3256469982 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa05afcd70 con 0x55aa05bae1d0 2024-09-16T00:37:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa3377fe640 10 client.17616 handle_command_reply: tid=0 2024-09-16T00:37:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 client.17616 shutdown 2024-09-16T00:37:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 -- 172.21.15.1:0/3256469982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 msgr2=0x55aa05bb1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb1310 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0044f0 tx=0x7fa34c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 -- 172.21.15.1:0/3256469982 shutdown_connections 2024-09-16T00:37:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aa05bae1d0 0x55aa05c1b230 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa05b9e780 0x55aa05bb1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa3377fe640 0 client.17616 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:11.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa05b90b40 0x55aa05bb1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 --2- 172.21.15.1:0/3256469982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa05b8f5d0 0x55aa05bb1310 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 -- 172.21.15.1:0/3256469982 >> 172.21.15.1:0/3256469982 conn(0x55aa05b58310 msgr2=0x55aa05b9cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:11.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 -- 172.21.15.1:0/3256469982 shutdown_connections 2024-09-16T00:37:11.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.543+0000 7fa360a18740 1 -- 172.21.15.1:0/3256469982 wait complete. 2024-09-16T00:37:11.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa360a18740 20 client.17616 trim_cache size 0 max 16384 2024-09-16T00:37:11.565 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:37:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 -- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3400a32c0 msgr2=0x7fa3400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3400a32c0 0x7fa3400a5780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa35000ab20 tx=0x7fa3500186b0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 -- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358106f10 msgr2=0x7fa35819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358106f10 0x7fa35819cc90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa35000cce0 tx=0x7fa35000a620 comp rx=0 tx=0).stop 2024-09-16T00:37:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 -- 172.21.15.1:0/723111578 shutdown_connections 2024-09-16T00:37:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa358106f10 0x7fa35819cc90 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa358102790 0x7fa358198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa3400a32c0 0x7fa3400a5780 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 --2- 172.21.15.1:0/723111578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa358101de0 0x7fa3581983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:11.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 -- 172.21.15.1:0/723111578 >> 172.21.15.1:0/723111578 conn(0x7fa3580fd8c0 msgr2=0x7fa3580fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:11.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 -- 172.21.15.1:0/723111578 shutdown_connections 2024-09-16T00:37:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:11.544+0000 7fa35e63f640 1 -- 172.21.15.1:0/723111578 wait complete. 2024-09-16T00:37:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:12 smithi001 ceph-mon[30934]: pgmap v614: 129 pgs: 129 active+clean; 4.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:12.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:12 smithi153 ceph-mon[35538]: pgmap v614: 129 pgs: 129 active+clean; 4.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:12 smithi019 ceph-mon[35186]: pgmap v614: 129 pgs: 129 active+clean; 4.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:13.310 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:37:13.311 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:37:13.585 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 -- 172.21.15.1:0/1747773653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 msgr2=0x7f4354100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 --2- 172.21.15.1:0/1747773653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 0x7f4354100ee0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f433c009930 tx=0x7f433c02a420 comp rx=0 tx=0).stop 2024-09-16T00:37:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 -- 172.21.15.1:0/1747773653 shutdown_connections 2024-09-16T00:37:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 --2- 172.21.15.1:0/1747773653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4354101420 0x7f435410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 --2- 172.21.15.1:0/1747773653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 0x7f4354100ee0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 --2- 172.21.15.1:0/1747773653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4354106a80 0x7f4354106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 -- 172.21.15.1:0/1747773653 >> 172.21.15.1:0/1747773653 conn(0x7f43540fc7a0 msgr2=0x7f43540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 -- 172.21.15.1:0/1747773653 shutdown_connections 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 -- 172.21.15.1:0/1747773653 wait complete. 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.052+0000 7f4358c32640 1 Processor -- start 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f4358c32640 1 -- start start 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f4358c32640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 0x7f4354198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f4358c32640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4354101420 0x7f43541989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f4358c32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4354106a80 0x7f435419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f4358c32640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4354110ea0 con 0x7f4354100a80 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f4358c32640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4354110d20 con 0x7f4354106a80 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f4358c32640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4354111020 con 0x7f4354101420 2024-09-16T00:37:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f435259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 0x7f4354198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f435259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 0x7f4354198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38726/0 (socket says 172.21.15.1:38726) 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f435259c640 1 -- 172.21.15.1:0/4149452168 learned_addr learned my addr 172.21.15.1:0/4149452168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f4352d9d640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4354106a80 0x7f435419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f4349d9b640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4354101420 0x7f43541989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f435259c640 1 -- 172.21.15.1:0/4149452168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4354101420 msgr2=0x7f43541989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f435259c640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4354101420 0x7f43541989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f435259c640 1 -- 172.21.15.1:0/4149452168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4354106a80 msgr2=0x7f435419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f435259c640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4354106a80 0x7f435419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.053+0000 7f435259c640 1 -- 172.21.15.1:0/4149452168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435419d440 con 0x7f4354100a80 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.054+0000 7f435259c640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 0x7f4354198490 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f434400b4a0 tx=0x7f434400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.054+0000 7f434bfff640 1 -- 172.21.15.1:0/4149452168 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344014070 con 0x7f4354100a80 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.054+0000 7f434bfff640 1 -- 172.21.15.1:0/4149452168 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4344010af0 con 0x7f4354100a80 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.054+0000 7f4358c32640 1 -- 172.21.15.1:0/4149452168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435419d730 con 0x7f4354100a80 2024-09-16T00:37:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.054+0000 7f434bfff640 1 -- 172.21.15.1:0/4149452168 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344017d10 con 0x7f4354100a80 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.054+0000 7f4358c32640 1 -- 172.21.15.1:0/4149452168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43541a5010 con 0x7f4354100a80 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.056+0000 7f434bfff640 1 -- 172.21.15.1:0/4149452168 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4344017070 con 0x7f4354100a80 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f435b00b740 1 Processor -- start 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f435b00b740 1 -- start start 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f435b00b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 0x55cd409cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f435b00b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 0x55cd409dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f435b00b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409de540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f435b00b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd40932280 con 0x55cd409dc150 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f435b00b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd408d9b10 con 0x55cd409cca40 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f435b00b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd4092e240 con 0x55cd409cd360 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f434bfff640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f43400a3240 0x7f43400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f4349d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409de540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f4349d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409de540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38736/0 (socket says 172.21.15.1:38736) 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f4349d9b640 1 -- 172.21.15.1:0/597128720 learned_addr learned my addr 172.21.15.1:0/597128720 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f4352d9d640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f43400a3240 0x7f43400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f435259c640 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 0x55cd409dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.057+0000 7f4352d9d640 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 0x55cd409cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f434bfff640 1 -- 172.21.15.1:0/4149452168 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f43440cf1f0 con 0x7f4354100a80 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4349d9b640 1 -- 172.21.15.1:0/597128720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 msgr2=0x55cd409dbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4349d9b640 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 0x55cd409dbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4349d9b640 1 -- 172.21.15.1:0/597128720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 msgr2=0x55cd409cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4349d9b640 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 0x55cd409cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4349d9b640 1 -- 172.21.15.1:0/597128720 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd408e61f0 con 0x55cd409dc150 2024-09-16T00:37:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f435259c640 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 0x55cd409dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4352d9d640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f43400a3240 0x7f43400a5700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f434c012770 tx=0x7f434c007410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4349d9b640 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409de540 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f433c005ab0 tx=0x7f433c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cbc3a766060173 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4348d99640 1 -- 172.21.15.1:0/597128720 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c03f070 con 0x55cd409dc150 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4348d99640 1 -- 172.21.15.1:0/597128720 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c031a20 con 0x55cd409dc150 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.058+0000 7f4348d99640 1 -- 172.21.15.1:0/597128720 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c035070 con 0x55cd409dc150 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 -- 172.21.15.1:0/597128720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 msgr2=0x55cd409de540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409de540 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f433c005ab0 tx=0x7f433c031630 comp rx=0 tx=0).stop 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 -- 172.21.15.1:0/597128720 shutdown_connections 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409de540 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 0x55cd409dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 --2- 172.21.15.1:0/597128720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 0x55cd409cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 -- 172.21.15.1:0/597128720 >> 172.21.15.1:0/597128720 conn(0x55cd409c90f0 msgr2=0x55cd409c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 -- 172.21.15.1:0/597128720 shutdown_connections 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 -- 172.21.15.1:0/597128720 wait complete. 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 -- start start 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.059+0000 7f435b00b740 1 Processor -- start 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435b00b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409cca40 0x55cd409d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435b00b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 0x55cd409d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435b00b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409dc150 0x55cd409d9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435b00b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd409ded20 con 0x55cd409cca40 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435b00b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd408ecf10 con 0x55cd409cd360 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435b00b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd409deea0 con 0x55cd409dc150 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435b00b740 10 client.0 _ll_register_callbacks cb 0x55cd408b5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409cca40 0x55cd409d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409cca40 0x55cd409d8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38740/0 (socket says 172.21.15.1:38740) 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435259c640 1 -- 172.21.15.1:0/2243344297 learned_addr learned my addr 172.21.15.1:0/2243344297 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f4352d9d640 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409dc150 0x55cd409d9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f4349d9b640 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 0x55cd409d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435259c640 1 -- 172.21.15.1:0/2243344297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409dc150 msgr2=0x55cd409d9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435259c640 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409dc150 0x55cd409d9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435259c640 1 -- 172.21.15.1:0/2243344297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 msgr2=0x55cd409d9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435259c640 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 0x55cd409d9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435259c640 1 -- 172.21.15.1:0/2243344297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd406a36a0 con 0x55cd409cca40 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f4352d9d640 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409dc150 0x55cd409d9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435259c640 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409cca40 0x55cd409d8ff0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4344097d80 tx=0x7f434401ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2792d2dc1258337c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f4339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f4339ffb640 1 -- 172.21.15.1:0/2243344297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434401e070 con 0x55cd409cca40 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f4339ffb640 1 -- 172.21.15.1:0/2243344297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f434409f4a0 con 0x55cd409cca40 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.060+0000 7f435b00b740 1 -- 172.21.15.1:0/2243344297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433c031a20 con 0x55cd409cca40 2024-09-16T00:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f4339ffb640 1 -- 172.21.15.1:0/2243344297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4344009230 con 0x55cd409cca40 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f435b00b740 10 client.17628 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f435b00b740 1 -- 172.21.15.1:0/2243344297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd409ea990 con 0x55cd409cca40 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f4339ffb640 1 -- 172.21.15.1:0/2243344297 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c031a20 con 0x55cd409cca40 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f4339ffb640 1 -- 172.21.15.1:0/2243344297 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f43440b4230 con 0x55cd409cca40 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f435b00b740 10 client.17628 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f435b00b740 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f435b00b740 20 client.17628 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f435b00b740 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd409ebe50 0x55cd409ee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f435b00b740 4 client.17628 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f435b00b740 1 -- 172.21.15.1:0/2243344297 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd4093f7f0 con 0x55cd409ebe50 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.061+0000 7f4349d9b640 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd409ebe50 0x55cd409ee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.062+0000 7f4349d9b640 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd409ebe50 0x55cd409ee280 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aca7339f77db28d server_cookie=a8d8773513954eb4 in_seq=0 out_seq=0 2024-09-16T00:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.062+0000 7f4339ffb640 10 client.17628 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f4339ffb640 1 -- 172.21.15.1:0/2243344297 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd4093f7f0 con 0x55cd409ebe50 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f4339ffb640 10 client.17628 handle_command_reply: tid=0 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 client.17628 shutdown 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 -- 172.21.15.1:0/2243344297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409cca40 msgr2=0x55cd409d8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409cca40 0x55cd409d8ff0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4344097d80 tx=0x7f434401ec80 comp rx=0 tx=0).stop 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 -- 172.21.15.1:0/2243344297 shutdown_connections 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd409ebe50 0x55cd409ee280 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409dc150 0x55cd409d9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 0x55cd409d9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f4339ffb640 0 client.17628 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 --2- 172.21.15.1:0/2243344297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409cca40 0x55cd409d8ff0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 -- 172.21.15.1:0/2243344297 >> 172.21.15.1:0/2243344297 conn(0x55cd409c90f0 msgr2=0x55cd409c8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 -- 172.21.15.1:0/2243344297 shutdown_connections 2024-09-16T00:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 1 -- 172.21.15.1:0/2243344297 wait complete. 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.063+0000 7f435b00b740 20 client.17628 trim_cache size 0 max 16384 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.080+0000 7f435b00b740 1 Processor -- start 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.080+0000 7f435b00b740 1 -- start start 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f435b00b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cca40 0x55cd409f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f435b00b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 0x55cd409f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f435b00b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409d6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f435b00b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43541af520 con 0x55cd409dc150 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f435b00b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43541acdf0 con 0x55cd409cd360 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f435b00b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43541aac40 con 0x55cd409cca40 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4352d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409d6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4352d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409d6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38746/0 (socket says 172.21.15.1:38746) 2024-09-16T00:37:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4352d9d640 1 -- 172.21.15.1:0/1756351002 learned_addr learned my addr 172.21.15.1:0/1756351002 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4349d9b640 1 --2- 172.21.15.1:0/1756351002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 0x55cd409f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4352d9d640 1 -- 172.21.15.1:0/1756351002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cca40 msgr2=0x55cd409f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f435259c640 1 --2- 172.21.15.1:0/1756351002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cca40 0x55cd409f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4352d9d640 1 --2- 172.21.15.1:0/1756351002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cca40 0x55cd409f3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4352d9d640 1 -- 172.21.15.1:0/1756351002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 msgr2=0x55cd409f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4352d9d640 1 --2- 172.21.15.1:0/1756351002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 0x55cd409f4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4352d9d640 1 -- 172.21.15.1:0/1756351002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd409db8c0 con 0x55cd409dc150 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f4352d9d640 1 --2- 172.21.15.1:0/1756351002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409d6250 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e8a0 tx=0x7f434c014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de4706e299282c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f433bfff640 1 -- 172.21.15.1:0/1756351002 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c018070 con 0x55cd409dc150 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f433bfff640 1 -- 172.21.15.1:0/1756351002 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434c01f400 con 0x55cd409dc150 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.081+0000 7f433bfff640 1 -- 172.21.15.1:0/1756351002 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c027630 con 0x55cd409dc150 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 -- 172.21.15.1:0/1756351002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 msgr2=0x55cd409d6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 --2- 172.21.15.1:0/1756351002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409d6250 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e8a0 tx=0x7f434c014070 comp rx=0 tx=0).stop 2024-09-16T00:37:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 -- 172.21.15.1:0/1756351002 shutdown_connections 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 --2- 172.21.15.1:0/1756351002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd409d6250 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 --2- 172.21.15.1:0/1756351002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cd360 0x55cd409f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 --2- 172.21.15.1:0/1756351002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cca40 0x55cd409f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 -- 172.21.15.1:0/1756351002 >> 172.21.15.1:0/1756351002 conn(0x55cd409c90f0 msgr2=0x55cd409daf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 -- 172.21.15.1:0/1756351002 shutdown_connections 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 -- 172.21.15.1:0/1756351002 wait complete. 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.082+0000 7f435b00b740 1 -- start start 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435b00b740 1 Processor -- start 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435b00b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 0x55cd409e0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435b00b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 0x55cd409e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435b00b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd40a55af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435b00b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43541ab2b0 con 0x55cd409dc150 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435b00b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43541ac720 con 0x55cd409cca40 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435b00b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43541ab980 con 0x55cd409cd360 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f4352d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd40a55af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435b00b740 10 client.0 _ll_register_callbacks cb 0x7f4344017630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f4352d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd40a55af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38758/0 (socket says 172.21.15.1:38758) 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f4352d9d640 1 -- 172.21.15.1:0/3193229767 learned_addr learned my addr 172.21.15.1:0/3193229767 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435259c640 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 0x55cd409e0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f4349d9b640 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 0x55cd409e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f4352d9d640 1 -- 172.21.15.1:0/3193229767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 msgr2=0x55cd409e0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f4352d9d640 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 0x55cd409e0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f4352d9d640 1 -- 172.21.15.1:0/3193229767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 msgr2=0x55cd409e0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f4352d9d640 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 0x55cd409e0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f4352d9d640 1 -- 172.21.15.1:0/3193229767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd40a56030 con 0x55cd409dc150 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.083+0000 7f435259c640 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 0x55cd409e0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f4352d9d640 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd40a55af0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f434c010800 tx=0x7f434c012540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad822ee9c5b25bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f4339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f4339ffb640 1 -- 172.21.15.1:0/3193229767 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c018070 con 0x55cd409dc150 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f4339ffb640 1 -- 172.21.15.1:0/3193229767 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f434c01f7b0 con 0x55cd409dc150 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f435b00b740 1 -- 172.21.15.1:0/3193229767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c01f400 con 0x55cd409dc150 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f4339ffb640 1 -- 172.21.15.1:0/3193229767 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f434c03f050 con 0x55cd409dc150 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f435b00b740 10 client.17634 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f435b00b740 1 -- 172.21.15.1:0/3193229767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd40a57270 con 0x55cd409dc150 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f4339ffb640 1 -- 172.21.15.1:0/3193229767 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434c01f400 con 0x55cd409dc150 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.084+0000 7f4339ffb640 1 -- 172.21.15.1:0/3193229767 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f434c030070 con 0x55cd409dc150 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.085+0000 7f435b00b740 10 client.17634 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.085+0000 7f435b00b740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.085+0000 7f435b00b740 20 client.17634 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.085+0000 7f435b00b740 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd40a57920 0x55cd40a59d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.085+0000 7f435b00b740 4 client.17634 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.085+0000 7f435b00b740 1 -- 172.21.15.1:0/3193229767 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cd40939d00 con 0x55cd40a57920 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.085+0000 7f435259c640 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd40a57920 0x55cd40a59d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.085+0000 7f435259c640 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd40a57920 0x55cd40a59d40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0320cf88b46716 server_cookie=68eafe3514044e2f in_seq=0 out_seq=0 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.085+0000 7f4339ffb640 10 client.17634 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f4339ffb640 1 -- 172.21.15.1:0/3193229767 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cd40939d00 con 0x55cd40a57920 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f4339ffb640 10 client.17634 handle_command_reply: tid=0 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f435b00b740 1 client.17634 shutdown 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f435b00b740 1 -- 172.21.15.1:0/3193229767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 msgr2=0x55cd40a55af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f435b00b740 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd40a55af0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f434c010800 tx=0x7f434c012540 comp rx=0 tx=0).stop 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f435b00b740 1 -- 172.21.15.1:0/3193229767 shutdown_connections 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f435b00b740 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd40a57920 0x55cd40a59d40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f435b00b740 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd409dc150 0x55cd40a55af0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f435b00b740 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd409cd360 0x55cd409e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f435b00b740 1 --2- 172.21.15.1:0/3193229767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd409cca40 0x55cd409e0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.086+0000 7f435b00b740 1 -- 172.21.15.1:0/3193229767 >> 172.21.15.1:0/3193229767 conn(0x55cd409c90f0 msgr2=0x55cd409da5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4339ffb640 0 client.17634 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f435b00b740 1 -- 172.21.15.1:0/3193229767 shutdown_connections 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f435b00b740 1 -- 172.21.15.1:0/3193229767 wait complete. 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f435b00b740 20 client.17634 trim_cache size 0 max 16384 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:37:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "3c76ec75-849a-4122-90ca-69cb500f9e41", 2024-09-16T00:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 -- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f43400a3240 msgr2=0x7f43400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f43400a3240 0x7f43400a5700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f434c012770 tx=0x7f434c007410 comp rx=0 tx=0).stop 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 -- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 msgr2=0x7f4354198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 0x7f4354198490 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f434400b4a0 tx=0x7f434400b960 comp rx=0 tx=0).stop 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 -- 172.21.15.1:0/4149452168 shutdown_connections 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4354106a80 0x7f435419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f43400a3240 0x7f43400a5700 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4354101420 0x7f43541989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 --2- 172.21.15.1:0/4149452168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4354100a80 0x7f4354198490 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.087+0000 7f4358c32640 1 -- 172.21.15.1:0/4149452168 >> 172.21.15.1:0/4149452168 conn(0x7f43540fc7a0 msgr2=0x7f435406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.088+0000 7f4358c32640 1 -- 172.21.15.1:0/4149452168 shutdown_connections 2024-09-16T00:37:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:14.088+0000 7f4358c32640 1 -- 172.21.15.1:0/4149452168 wait complete. 2024-09-16T00:37:14.403 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:14 smithi001 ceph-mon[30934]: pgmap v615: 129 pgs: 129 active+clean; 4.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:14.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:14 smithi153 ceph-mon[35538]: pgmap v615: 129 pgs: 129 active+clean; 4.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:14.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:14 smithi019 ceph-mon[35186]: pgmap v615: 129 pgs: 129 active+clean; 4.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:14.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:37:15.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:15 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:37:15.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:15 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:15.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:15 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:37:15.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:15 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:15.204 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:15.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:15 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:37:15.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:15 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:15.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:15 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:37:15.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:15 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:15 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:37:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:15 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:15 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:37:15.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:15 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:37:15.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 -- 172.21.15.1:0/556582619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014102790 msgr2=0x7f60141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:15.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 --2- 172.21.15.1:0/556582619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014102790 0x7f60141067e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f6008009900 tx=0x7f600802a410 comp rx=0 tx=0).stop 2024-09-16T00:37:15.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 -- 172.21.15.1:0/556582619 shutdown_connections 2024-09-16T00:37:15.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 --2- 172.21.15.1:0/556582619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6014106f10 0x7f601410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 --2- 172.21.15.1:0/556582619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014102790 0x7f60141067e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 --2- 172.21.15.1:0/556582619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6014101de0 0x7f60141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 -- 172.21.15.1:0/556582619 >> 172.21.15.1:0/556582619 conn(0x7f60140fd8c0 msgr2=0x7f60140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 -- 172.21.15.1:0/556582619 shutdown_connections 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 -- 172.21.15.1:0/556582619 wait complete. 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 Processor -- start 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.703+0000 7f601c37a640 1 -- start start 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601c37a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014101de0 0x7f60140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601c37a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6014102790 0x7f6014074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601c37a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6014106f10 0x7f6014074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601c37a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6014111f90 con 0x7f6014101de0 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601c37a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6014111e10 con 0x7f6014106f10 2024-09-16T00:37:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601c37a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6014112110 con 0x7f6014102790 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a116640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014101de0 0x7f60140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a116640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014101de0 0x7f60140744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38776/0 (socket says 172.21.15.1:38776) 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a116640 1 -- 172.21.15.1:0/3010868939 learned_addr learned my addr 172.21.15.1:0/3010868939 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a917640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6014106f10 0x7f6014074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f6019915640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6014102790 0x7f6014074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a116640 1 -- 172.21.15.1:0/3010868939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6014102790 msgr2=0x7f6014074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a116640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6014102790 0x7f6014074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a116640 1 -- 172.21.15.1:0/3010868939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6014106f10 msgr2=0x7f6014074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a116640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6014106f10 0x7f6014074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a116640 1 -- 172.21.15.1:0/3010868939 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60141a68e0 con 0x7f6014101de0 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f6019915640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6014102790 0x7f6014074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601a116640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014101de0 0x7f60140744f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f600400cce0 tx=0x7f600400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f60137fe640 1 -- 172.21.15.1:0/3010868939 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600400f650 con 0x7f6014101de0 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f60137fe640 1 -- 172.21.15.1:0/3010868939 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f600400fcd0 con 0x7f6014101de0 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601c37a640 1 -- 172.21.15.1:0/3010868939 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60141a6bd0 con 0x7f6014101de0 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f60137fe640 1 -- 172.21.15.1:0/3010868939 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004018c70 con 0x7f6014101de0 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.704+0000 7f601c37a640 1 -- 172.21.15.1:0/3010868939 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60141a7110 con 0x7f6014101de0 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.707+0000 7f60137fe640 1 -- 172.21.15.1:0/3010868939 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f600400f7f0 con 0x7f6014101de0 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.707+0000 7f601e753740 1 Processor -- start 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601e753740 1 -- start start 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601e753740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d208414a0 0x563d20841880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601e753740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 0x563d2084c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601e753740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d2084c680 0x563d2084ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601e753740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563d207a72e0 con 0x563d208414a0 2024-09-16T00:37:15.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601e753740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563d2074ece0 con 0x563d20841dc0 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601e753740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563d207a32a0 con 0x563d2084c680 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601a917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d208414a0 0x563d20841880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601a917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d208414a0 0x563d20841880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38790/0 (socket says 172.21.15.1:38790) 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601a917640 1 -- 172.21.15.1:0/551686637 learned_addr learned my addr 172.21.15.1:0/551686637 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601a116640 1 --2- 172.21.15.1:0/551686637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 0x563d2084c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f6019915640 1 --2- 172.21.15.1:0/551686637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d2084c680 0x563d2084ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f60137fe640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ffc0a3240 0x7f5ffc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601a917640 1 -- 172.21.15.1:0/551686637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d2084c680 msgr2=0x563d2084ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601a917640 1 --2- 172.21.15.1:0/551686637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d2084c680 0x563d2084ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601a917640 1 -- 172.21.15.1:0/551686637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 msgr2=0x563d2084c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601a917640 1 --2- 172.21.15.1:0/551686637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 0x563d2084c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.708+0000 7f601a917640 1 -- 172.21.15.1:0/551686637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d20733440 con 0x563d208414a0 2024-09-16T00:37:15.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.709+0000 7f60137fe640 1 -- 172.21.15.1:0/3010868939 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f60040d01b0 con 0x7f6014101de0 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.709+0000 7f601a917640 1 --2- 172.21.15.1:0/551686637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d208414a0 0x563d20841880 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f600c0099e0 tx=0x7f600c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132f70aed260b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.709+0000 7f601a917640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ffc0a3240 0x7f5ffc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.709+0000 7f6010dea640 1 -- 172.21.15.1:0/551686637 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c01f470 con 0x563d208414a0 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.709+0000 7f6010dea640 1 -- 172.21.15.1:0/551686637 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f600c017050 con 0x563d208414a0 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.709+0000 7f6010dea640 1 -- 172.21.15.1:0/551686637 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c025070 con 0x563d208414a0 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 -- 172.21.15.1:0/551686637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d208414a0 msgr2=0x563d20841880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 --2- 172.21.15.1:0/551686637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d208414a0 0x563d20841880 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f600c0099e0 tx=0x7f600c009ea0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601a917640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ffc0a3240 0x7f5ffc0a5700 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f600c01e000 tx=0x7f600c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 -- 172.21.15.1:0/551686637 shutdown_connections 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 --2- 172.21.15.1:0/551686637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d2084c680 0x563d2084ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 --2- 172.21.15.1:0/551686637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 0x563d2084c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 --2- 172.21.15.1:0/551686637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d208414a0 0x563d20841880 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 -- 172.21.15.1:0/551686637 >> 172.21.15.1:0/551686637 conn(0x563d2083da30 msgr2=0x563d2083de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 -- 172.21.15.1:0/551686637 shutdown_connections 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 -- 172.21.15.1:0/551686637 wait complete. 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 -- start start 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.710+0000 7f601e753740 1 Processor -- start 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601e753740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d208414a0 0x563d2084bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601e753740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d20841dc0 0x563d20849210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601e753740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d20849750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601e753740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563d2084f250 con 0x563d2084c680 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601e753740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563d20762050 con 0x563d208414a0 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601e753740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563d2084f3d0 con 0x563d20841dc0 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601e753740 10 client.0 _ll_register_callbacks cb 0x563d207a2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d20849750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d20849750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38802/0 (socket says 172.21.15.1:38802) 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a917640 1 -- 172.21.15.1:0/1170434687 learned_addr learned my addr 172.21.15.1:0/1170434687 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f6019915640 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d20841dc0 0x563d20849210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a116640 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d208414a0 0x563d2084bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a917640 1 -- 172.21.15.1:0/1170434687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d20841dc0 msgr2=0x563d20849210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a917640 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d20841dc0 0x563d20849210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a917640 1 -- 172.21.15.1:0/1170434687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d208414a0 msgr2=0x563d2084bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a917640 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d208414a0 0x563d2084bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a917640 1 -- 172.21.15.1:0/1170434687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d205183c0 con 0x563d2084c680 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601a917640 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d20849750 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f600c00e7b0 tx=0x7f600c01e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b59511e7c5a419 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f5ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/1170434687 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c02d070 con 0x563d2084c680 2024-09-16T00:37:15.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/1170434687 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f600c013530 con 0x563d2084c680 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.711+0000 7f601e753740 1 -- 172.21.15.1:0/1170434687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600c017050 con 0x563d2084c680 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/1170434687 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f600c00eab0 con 0x563d2084c680 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f601e753740 10 client.17646 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f601e753740 1 -- 172.21.15.1:0/1170434687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d2085ac10 con 0x563d2084c680 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/1170434687 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f600c017050 con 0x563d2084c680 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/1170434687 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f600c0063b0 con 0x563d2084c680 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f601e753740 10 client.17646 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f601e753740 10 client.17646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f601e753740 20 client.17646 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f601e753740 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d2085c020 0x563d2085e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f601e753740 4 client.17646 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f601e753740 1 -- 172.21.15.1:0/1170434687 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d207b4850 con 0x563d2085c020 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.712+0000 7f601a116640 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d2085c020 0x563d2085e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.713+0000 7f5ff9ffb640 10 client.17646 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.713+0000 7f601a116640 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d2085c020 0x563d2085e410 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7244b73231d0500 server_cookie=2af21359c9e113b8 in_seq=0 out_seq=0 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.713+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/1170434687 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d207b4850 con 0x563d2085c020 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.713+0000 7f5ff9ffb640 10 client.17646 handle_command_reply: tid=0 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.713+0000 7f601e753740 1 client.17646 shutdown 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 -- 172.21.15.1:0/1170434687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 msgr2=0x563d20849750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d20849750 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f600c00e7b0 tx=0x7f600c01e530 comp rx=0 tx=0).stop 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 -- 172.21.15.1:0/1170434687 shutdown_connections 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d2085c020 0x563d2085e410 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d20849750 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d20841dc0 0x563d20849210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f5ff9ffb640 0 client.17646 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 --2- 172.21.15.1:0/1170434687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d208414a0 0x563d2084bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 -- 172.21.15.1:0/1170434687 >> 172.21.15.1:0/1170434687 conn(0x563d2083da30 msgr2=0x563d2083cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 -- 172.21.15.1:0/1170434687 shutdown_connections 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 1 -- 172.21.15.1:0/1170434687 wait complete. 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.714+0000 7f601e753740 20 client.17646 trim_cache size 0 max 16384 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f601e753740 1 Processor -- start 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f601e753740 1 -- start start 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f601e753740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 0x563d20863a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f601e753740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d20841dc0 0x563d20863fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f601e753740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d2084c680 0x563d20846470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f601e753740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60141b3930 con 0x563d20841dc0 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f601e753740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60141b1200 con 0x563d2084c680 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f601e753740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60141af050 con 0x563d208414a0 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f6019915640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d20841dc0 0x563d20863fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.731+0000 7f6019915640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d20841dc0 0x563d20863fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38814/0 (socket says 172.21.15.1:38814) 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f6019915640 1 -- 172.21.15.1:0/4200496668 learned_addr learned my addr 172.21.15.1:0/4200496668 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f601a917640 1 --2- 172.21.15.1:0/4200496668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d2084c680 0x563d20846470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f601a116640 1 --2- 172.21.15.1:0/4200496668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 0x563d20863a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f6019915640 1 -- 172.21.15.1:0/4200496668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 msgr2=0x563d20863a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f6019915640 1 --2- 172.21.15.1:0/4200496668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 0x563d20863a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f6019915640 1 -- 172.21.15.1:0/4200496668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d2084c680 msgr2=0x563d20846470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f6019915640 1 --2- 172.21.15.1:0/4200496668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d2084c680 0x563d20846470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f6019915640 1 -- 172.21.15.1:0/4200496668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d20840c70 con 0x563d20841dc0 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f6019915640 1 --2- 172.21.15.1:0/4200496668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d20841dc0 0x563d20863fb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6008005ab0 tx=0x7f60080316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4bca7294822278 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f5ffbfff640 1 -- 172.21.15.1:0/4200496668 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008035050 con 0x563d20841dc0 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f5ffbfff640 1 -- 172.21.15.1:0/4200496668 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008005020 con 0x563d20841dc0 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.732+0000 7f5ffbfff640 1 -- 172.21.15.1:0/4200496668 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008004bc0 con 0x563d20841dc0 2024-09-16T00:37:15.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 -- 172.21.15.1:0/4200496668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d20841dc0 msgr2=0x563d20863fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 --2- 172.21.15.1:0/4200496668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d20841dc0 0x563d20863fb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6008005ab0 tx=0x7f60080316f0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 -- 172.21.15.1:0/4200496668 shutdown_connections 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 --2- 172.21.15.1:0/4200496668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d2084c680 0x563d20846470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 --2- 172.21.15.1:0/4200496668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d20841dc0 0x563d20863fb0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 --2- 172.21.15.1:0/4200496668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 0x563d20863a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 -- 172.21.15.1:0/4200496668 >> 172.21.15.1:0/4200496668 conn(0x563d2083da30 msgr2=0x563d2084b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 -- 172.21.15.1:0/4200496668 shutdown_connections 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 -- 172.21.15.1:0/4200496668 wait complete. 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 -- start start 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 Processor -- start 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 0x563d20854d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 0x563d20852080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d208525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60141af6c0 con 0x563d2084c680 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60141b0b30 con 0x563d20841dc0 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60141afd90 con 0x563d208414a0 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.733+0000 7f601e753740 10 client.0 _ll_register_callbacks cb 0x7f60040d3b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d208525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a917640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d208525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38828/0 (socket says 172.21.15.1:38828) 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a917640 1 -- 172.21.15.1:0/4014127302 learned_addr learned my addr 172.21.15.1:0/4014127302 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a116640 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 0x563d20854d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f6019915640 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 0x563d20852080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a917640 1 -- 172.21.15.1:0/4014127302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 msgr2=0x563d20854d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a917640 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 0x563d20854d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a917640 1 -- 172.21.15.1:0/4014127302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 msgr2=0x563d20852080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:15.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a917640 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 0x563d20852080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a917640 1 -- 172.21.15.1:0/4014127302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d20852dd0 con 0x563d2084c680 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601a917640 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d208525c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f600c00e780 tx=0x7f600c00e0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478541dc5d39ed8d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f5ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/4014127302 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c02d040 con 0x563d2084c680 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/4014127302 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f600c013530 con 0x563d2084c680 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601e753740 1 -- 172.21.15.1:0/4014127302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6008005020 con 0x563d2084c680 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/4014127302 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f600c04a050 con 0x563d2084c680 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601e753740 10 client.17652 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.734+0000 7f601e753740 1 -- 172.21.15.1:0/4014127302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d208c69d0 con 0x563d2084c680 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.735+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/4014127302 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008005020 con 0x563d2084c680 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.735+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/4014127302 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f600c03d780 con 0x563d2084c680 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.735+0000 7f601e753740 10 client.17652 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.735+0000 7f601e753740 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:15.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.735+0000 7f601e753740 20 client.17652 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.735+0000 7f601e753740 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d208c70e0 0x563d208c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.735+0000 7f601e753740 4 client.17652 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.735+0000 7f601e753740 1 -- 172.21.15.1:0/4014127302 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d2084eef0 con 0x563d208c70e0 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.735+0000 7f601a116640 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d208c70e0 0x563d208c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601a116640 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d208c70e0 0x563d208c94d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d09c054c23e2b8b server_cookie=b257e44b073d22ce in_seq=0 out_seq=0 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f5ff9ffb640 10 client.17652 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f5ff9ffb640 1 -- 172.21.15.1:0/4014127302 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563d2084eef0 con 0x563d208c70e0 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f5ff9ffb640 10 client.17652 handle_command_reply: tid=0 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 client.17652 shutdown 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 -- 172.21.15.1:0/4014127302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 msgr2=0x563d208525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d208525c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f600c00e780 tx=0x7f600c00e0b0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 -- 172.21.15.1:0/4014127302 shutdown_connections 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563d208c70e0 0x563d208c94d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563d2084c680 0x563d208525c0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563d20841dc0 0x563d20852080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 --2- 172.21.15.1:0/4014127302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d208414a0 0x563d20854d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f5ff9ffb640 0 client.17652 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 -- 172.21.15.1:0/4014127302 >> 172.21.15.1:0/4014127302 conn(0x563d2083da30 msgr2=0x563d2084aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 -- 172.21.15.1:0/4014127302 shutdown_connections 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 1 -- 172.21.15.1:0/4014127302 wait complete. 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.736+0000 7f601e753740 20 client.17652 trim_cache size 0 max 16384 2024-09-16T00:37:15.753 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:37:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:37:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:37:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "3c76ec75-849a-4122-90ca-69cb500f9e41": { 2024-09-16T00:37:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:37:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "3c76ec75-849a-4122-90ca-69cb500f9e41", 2024-09-16T00:37:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:37:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:15.754 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 -- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ffc0a3240 msgr2=0x7f5ffc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ffc0a3240 0x7f5ffc0a5700 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f600c01e000 tx=0x7f600c01f000 comp rx=0 tx=0).stop 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 -- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014101de0 msgr2=0x7f60140744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014101de0 0x7f60140744f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f600400cce0 tx=0x7f600400a620 comp rx=0 tx=0).stop 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 -- 172.21.15.1:0/3010868939 shutdown_connections 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6014106f10 0x7f6014074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ffc0a3240 0x7f5ffc0a5700 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6014102790 0x7f6014074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 --2- 172.21.15.1:0/3010868939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6014101de0 0x7f60140744f0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 -- 172.21.15.1:0/3010868939 >> 172.21.15.1:0/3010868939 conn(0x7f60140fd8c0 msgr2=0x7f60140fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:15.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 -- 172.21.15.1:0/3010868939 shutdown_connections 2024-09-16T00:37:15.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:15.737+0000 7f601c37a640 1 -- 172.21.15.1:0/3010868939 wait complete. 2024-09-16T00:37:16.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:16 smithi001 ceph-mon[30934]: pgmap v616: 129 pgs: 129 active+clean; 4.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:37:16.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:16 smithi153 ceph-mon[35538]: pgmap v616: 129 pgs: 129 active+clean; 4.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:37:16.530 INFO:tasks.cephfs.filesystem:scrub status for tag:3c76ec75-849a-4122-90ca-69cb500f9e41 - {'path': '/', 'tag': '3c76ec75-849a-4122-90ca-69cb500f9e41', 'options': 'recursive,force'} 2024-09-16T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:16 smithi019 ceph-mon[35186]: pgmap v616: 129 pgs: 129 active+clean; 4.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:37:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:18 smithi001 ceph-mon[30934]: pgmap v617: 129 pgs: 129 active+clean; 4.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:18 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:37:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:18 smithi153 ceph-mon[35538]: pgmap v617: 129 pgs: 129 active+clean; 4.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:18 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:37:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:18 smithi019 ceph-mon[35186]: pgmap v617: 129 pgs: 129 active+clean; 4.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:18 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:37:20.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:20 smithi001 ceph-mon[30934]: pgmap v618: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:20.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:20 smithi153 ceph-mon[35538]: pgmap v618: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:20 smithi019 ceph-mon[35186]: pgmap v618: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:22.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:22 smithi001 ceph-mon[30934]: pgmap v619: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:22 smithi153 ceph-mon[35538]: pgmap v619: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:22 smithi019 ceph-mon[35186]: pgmap v619: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:24.354 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.354 INFO:tasks.workunit.client.0.smithi001.stdout:251/600: open fd = 3 2024-09-16T00:37:24.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.354 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.354 INFO:tasks.workunit.client.0.smithi001.stdout:252/600: open fd = 3 2024-09-16T00:37:24.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:253/600: open fd = 3 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:254/600: open fd = 3 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:255/600: open fd = 3 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.355 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:256/600: open fd = 3 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:257/600: open fd = 3 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:258/600: open fd = 3 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.356 INFO:tasks.workunit.client.0.smithi001.stdout:259/600: open fd = 3 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:260/600: open fd = 3 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:261/600: open fd = 3 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:262/600: open fd = 3 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.357 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:263/600: open fd = 3 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:264/600: open fd = 3 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:265/600: open fd = 3 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.358 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:266/600: open fd = 3 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:267/600: open fd = 3 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:268/600: open fd = 3 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:269/600: open fd = 3 2024-09-16T00:37:24.359 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:270/600: open fd = 3 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:271/600: open fd = 3 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:272/600: open fd = 3 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.360 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:273/600: open fd = 3 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:274/600: open fd = 3 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:275/600: open fd = 3 2024-09-16T00:37:24.361 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:276/600: open fd = 3 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:277/600: open fd = 3 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:278/600: open fd = 3 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.362 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:279/600: open fd = 3 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:280/600: open fd = 3 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:281/600: open fd = 3 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.363 INFO:tasks.workunit.client.0.smithi001.stdout:282/600: open fd = 3 2024-09-16T00:37:24.364 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.364 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.364 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.364 INFO:tasks.workunit.client.0.smithi001.stdout:283/600: open fd = 3 2024-09-16T00:37:24.364 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.364 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.364 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.364 INFO:tasks.workunit.client.0.smithi001.stdout:284/600: open fd = 3 2024-09-16T00:37:24.364 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:285/600: open fd = 3 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:286/600: open fd = 3 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:287/600: open fd = 3 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.365 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:288/600: open fd = 3 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:289/600: open fd = 3 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:290/600: open fd = 3 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.366 INFO:tasks.workunit.client.0.smithi001.stdout:291/600: open fd = 3 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:292/600: open fd = 3 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:293/600: open fd = 3 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:294/600: open fd = 3 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.367 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:295/600: open fd = 3 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:296/600: open fd = 3 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:297/600: open fd = 3 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.368 INFO:tasks.workunit.client.0.smithi001.stdout:298/600: open fd = 3 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:299/600: open fd = 3 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:300/600: open fd = 3 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:301/600: open fd = 3 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.369 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:302/600: open fd = 3 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:303/600: open fd = 3 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:304/600: open fd = 3 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.370 INFO:tasks.workunit.client.0.smithi001.stdout:305/600: open fd = 3 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:306/600: open fd = 3 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:307/600: open fd = 3 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:308/600: open fd = 3 2024-09-16T00:37:24.371 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:309/600: open fd = 3 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:310/600: open fd = 3 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.372 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.373 INFO:tasks.workunit.client.0.smithi001.stdout:311/600: open fd = 3 2024-09-16T00:37:24.373 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.373 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.373 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:37:24.373 INFO:tasks.workunit.client.0.smithi001.stdout:312/600: open fd = 3 2024-09-16T00:37:24.373 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.373 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:37:24.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:24 smithi001 ceph-mon[30934]: pgmap v620: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:37:24.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:24 smithi153 ceph-mon[35538]: pgmap v620: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:37:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:24 smithi019 ceph-mon[35186]: pgmap v620: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:37:25.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:25 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:25.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:25 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:25.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:25 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:26.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:26 smithi001 ceph-mon[30934]: pgmap v621: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:37:26.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:26.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:26.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:37:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:26 smithi153 ceph-mon[35538]: pgmap v621: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:37:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:37:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:26 smithi019 ceph-mon[35186]: pgmap v621: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:37:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:37:27.751 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:37:27.751 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-16T00:37:27.751 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:37:28.023 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:28.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:28 smithi001 ceph-mon[30934]: pgmap v622: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:28.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:28 smithi153 ceph-mon[35538]: pgmap v622: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:28.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.567+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/1466731220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102ef0 msgr2=0x7f4fb810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:28.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.567+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/1466731220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102ef0 0x7f4fb810da40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009ca0 tx=0x7f4fb002a950 comp rx=0 tx=0).stop 2024-09-16T00:37:28.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.567+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/1466731220 shutdown_connections 2024-09-16T00:37:28.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.567+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/1466731220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102ef0 0x7f4fb810da40 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:28.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.567+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/1466731220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fb8102570 0x7f4fb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:28.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.567+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/1466731220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb8100e20 0x7f4fb8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.567+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/1466731220 >> 172.21.15.1:0/1466731220 conn(0x7f4fb80fc7d0 msgr2=0x7f4fb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.567+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/1466731220 shutdown_connections 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.567+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/1466731220 wait complete. 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fbd1e9640 1 Processor -- start 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fbd1e9640 1 -- start start 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fbd1e9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb8100e20 0x7f4fb80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fbd1e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102570 0x7f4fb80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fbd1e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fb8102ef0 0x7f4fb80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fbd1e9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8110f80 con 0x7f4fb8102570 2024-09-16T00:37:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fbd1e9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8110e00 con 0x7f4fb8102ef0 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fbd1e9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8111100 con 0x7f4fb8100e20 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4faffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102570 0x7f4fb80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4faffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102570 0x7f4fb80fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56422/0 (socket says 172.21.15.1:56422) 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4faffff640 1 -- 172.21.15.1:0/3185018161 learned_addr learned my addr 172.21.15.1:0/3185018161 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fb6d9d640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb8100e20 0x7f4fb80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fb759e640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fb8102ef0 0x7f4fb80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4faffff640 1 -- 172.21.15.1:0/3185018161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb8100e20 msgr2=0x7f4fb80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4faffff640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb8100e20 0x7f4fb80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4faffff640 1 -- 172.21.15.1:0/3185018161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fb8102ef0 msgr2=0x7f4fb80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4faffff640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fb8102ef0 0x7f4fb80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:28.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4faffff640 1 -- 172.21.15.1:0/3185018161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb81a6910 con 0x7f4fb8102570 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fb6d9d640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb8100e20 0x7f4fb80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4fb759e640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fb8102ef0 0x7f4fb80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.568+0000 7f4faffff640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102570 0x7f4fb80fd1c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000d950 tx=0x7f4fa000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.569+0000 7f4fb4d99640 1 -- 172.21.15.1:0/3185018161 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0017070 con 0x7f4fb8102570 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.569+0000 7f4fb4d99640 1 -- 172.21.15.1:0/3185018161 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa0009c30 con 0x7f4fb8102570 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.569+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb81a6c00 con 0x7f4fb8102570 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.569+0000 7f4fb4d99640 1 -- 172.21.15.1:0/3185018161 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000f6b0 con 0x7f4fb8102570 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.569+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb81a7140 con 0x7f4fb8102570 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.570+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb8101200 con 0x7f4fb8102570 2024-09-16T00:37:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.571+0000 7f4fb4d99640 1 -- 172.21.15.1:0/3185018161 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4fa00109d0 con 0x7f4fb8102570 2024-09-16T00:37:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.571+0000 7f4fb4d99640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fa40a3240 0x7f4fa40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.572+0000 7f4fb4d99640 1 -- 172.21.15.1:0/3185018161 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4fa00cf960 con 0x7f4fb8102570 2024-09-16T00:37:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.572+0000 7f4fb6d9d640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fa40a3240 0x7f4fa40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.572+0000 7f4fb6d9d640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fa40a3240 0x7f4fa40a5700 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80058c0 tx=0x7f4fa8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.576+0000 7f4fb4d99640 1 -- 172.21.15.1:0/3185018161 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fa0009de0 con 0x7f4fb8102570 2024-09-16T00:37:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:28 smithi019 ceph-mon[35186]: pgmap v622: 129 pgs: 129 active+clean; 4.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:37:28.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.786+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fb810a0a0 con 0x7f4fb8102570 2024-09-16T00:37:28.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.787+0000 7f4fb4d99640 1 -- 172.21.15.1:0/3185018161 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f4fa009c280 con 0x7f4fb8102570 2024-09-16T00:37:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:37:28.789 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:37:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:37:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.789+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fa40a3240 msgr2=0x7f4fa40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.789+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fa40a3240 0x7f4fa40a5700 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80058c0 tx=0x7f4fa8005d30 comp rx=0 tx=0).stop 2024-09-16T00:37:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.789+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102570 msgr2=0x7f4fb80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.789+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102570 0x7f4fb80fd1c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000d950 tx=0x7f4fa000de10 comp rx=0 tx=0).stop 2024-09-16T00:37:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.789+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 shutdown_connections 2024-09-16T00:37:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.790+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fb8102ef0 0x7f4fb80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.790+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fb8102570 0x7f4fb80fd1c0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.790+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4fa40a3240 0x7f4fa40a5700 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.790+0000 7f4fbd1e9640 1 --2- 172.21.15.1:0/3185018161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb8100e20 0x7f4fb80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.790+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 >> 172.21.15.1:0/3185018161 conn(0x7f4fb80fc7d0 msgr2=0x7f4fb806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.790+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 shutdown_connections 2024-09-16T00:37:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:28.790+0000 7f4fbd1e9640 1 -- 172.21.15.1:0/3185018161 wait complete. 2024-09-16T00:37:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:29 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3185018161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:29.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:29 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3185018161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:29.517 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-16T00:37:29.518 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-16T00:37:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:29 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3185018161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:29.780 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:30.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.282+0000 7f03ba41e640 1 -- 172.21.15.1:0/2127455250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4102570 msgr2=0x7f03b41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:30.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.282+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2127455250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4102570 0x7f03b41029b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f039c009900 tx=0x7f039c02a480 comp rx=0 tx=0).stop 2024-09-16T00:37:30.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.282+0000 7f03ba41e640 1 -- 172.21.15.1:0/2127455250 shutdown_connections 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.282+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2127455250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b4102ef0 0x7f03b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.282+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2127455250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4102570 0x7f03b41029b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.282+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2127455250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4100e20 0x7f03b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.282+0000 7f03ba41e640 1 -- 172.21.15.1:0/2127455250 >> 172.21.15.1:0/2127455250 conn(0x7f03b40fc7d0 msgr2=0x7f03b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.282+0000 7f03ba41e640 1 -- 172.21.15.1:0/2127455250 shutdown_connections 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.282+0000 7f03ba41e640 1 -- 172.21.15.1:0/2127455250 wait complete. 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03ba41e640 1 Processor -- start 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03ba41e640 1 -- start start 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03ba41e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4100e20 0x7f03b40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03ba41e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4102570 0x7f03b40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03ba41e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b4102ef0 0x7f03b40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03ba41e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4110fd0 con 0x7f03b4100e20 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03ba41e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4110e50 con 0x7f03b4102570 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03ba41e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4111150 con 0x7f03b4102ef0 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4100e20 0x7f03b40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4100e20 0x7f03b40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44212/0 (socket says 172.21.15.1:44212) 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b3fff640 1 -- 172.21.15.1:0/2261924024 learned_addr learned my addr 172.21.15.1:0/2261924024 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03abfff640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4102570 0x7f03b40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b89bb640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b4102ef0 0x7f03b40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b3fff640 1 -- 172.21.15.1:0/2261924024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b4102ef0 msgr2=0x7f03b40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b3fff640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b4102ef0 0x7f03b40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b3fff640 1 -- 172.21.15.1:0/2261924024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4102570 msgr2=0x7f03b40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b3fff640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4102570 0x7f03b40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b3fff640 1 -- 172.21.15.1:0/2261924024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b41a6980 con 0x7f03b4100e20 2024-09-16T00:37:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.283+0000 7f03b89bb640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b4102ef0 0x7f03b40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:30.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.284+0000 7f03b3fff640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4100e20 0x7f03b40fcc80 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f03a400e9f0 tx=0x7f03a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:30.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.284+0000 7f03b1ffb640 1 -- 172.21.15.1:0/2261924024 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a400cd80 con 0x7f03b4100e20 2024-09-16T00:37:30.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.284+0000 7f03ba41e640 1 -- 172.21.15.1:0/2261924024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b41a6c70 con 0x7f03b4100e20 2024-09-16T00:37:30.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.284+0000 7f03b1ffb640 1 -- 172.21.15.1:0/2261924024 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03a4010910 con 0x7f03b4100e20 2024-09-16T00:37:30.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.284+0000 7f03b1ffb640 1 -- 172.21.15.1:0/2261924024 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4018990 con 0x7f03b4100e20 2024-09-16T00:37:30.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.284+0000 7f03ba41e640 1 -- 172.21.15.1:0/2261924024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b41a7180 con 0x7f03b4100e20 2024-09-16T00:37:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.286+0000 7f03b1ffb640 1 -- 172.21.15.1:0/2261924024 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f03a4010ac0 con 0x7f03b4100e20 2024-09-16T00:37:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.286+0000 7f03bc7f7740 1 Processor -- start 2024-09-16T00:37:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.286+0000 7f03bc7f7740 1 -- start start 2024-09-16T00:37:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136fd3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 0x563136fde4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fe0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563136f38160 con 0x563136fdea30 2024-09-16T00:37:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563136edf720 con 0x563136fd4170 2024-09-16T00:37:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563136f34120 con 0x563136fd3850 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fe0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 0x563136fde4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03b1ffb640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f03a00a3280 0x7f03a00a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fe0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44214/0 (socket says 172.21.15.1:44214) 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03abfff640 1 -- 172.21.15.1:0/3818675052 learned_addr learned my addr 172.21.15.1:0/3818675052 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03b89bb640 1 --2- 172.21.15.1:0/3818675052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136fd3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03b1ffb640 1 -- 172.21.15.1:0/2261924024 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f03a40d0110 con 0x7f03b4100e20 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03abfff640 1 -- 172.21.15.1:0/3818675052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 msgr2=0x563136fd3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03abfff640 1 --2- 172.21.15.1:0/3818675052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136fd3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03b89bb640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f03a00a3280 0x7f03a00a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03abfff640 1 -- 172.21.15.1:0/3818675052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 msgr2=0x563136fde4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03abfff640 1 --2- 172.21.15.1:0/3818675052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 0x563136fde4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.287+0000 7f03abfff640 1 -- 172.21.15.1:0/3818675052 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563136ca9ba0 con 0x563136fdea30 2024-09-16T00:37:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.288+0000 7f03abfff640 1 --2- 172.21.15.1:0/3818675052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fe0e20 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f039c000c00 tx=0x7f039c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0053c0fef4302e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.288+0000 7f03b89bb640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f03a00a3280 0x7f03a00a5740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f03ac005fd0 tx=0x7f03ac004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.288+0000 7f03aa7fc640 1 -- 172.21.15.1:0/3818675052 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c035040 con 0x563136fdea30 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.288+0000 7f03aa7fc640 1 -- 172.21.15.1:0/3818675052 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c005520 con 0x563136fdea30 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.288+0000 7f03aa7fc640 1 -- 172.21.15.1:0/3818675052 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c003850 con 0x563136fdea30 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3818675052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 msgr2=0x563136fe0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/3818675052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fe0e20 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f039c000c00 tx=0x7f039c0040f0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3818675052 shutdown_connections 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/3818675052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fe0e20 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/3818675052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 0x563136fde4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/3818675052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136fd3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3818675052 >> 172.21.15.1:0/3818675052 conn(0x563136fcfde0 msgr2=0x563136fd01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:30.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3818675052 shutdown_connections 2024-09-16T00:37:30.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3818675052 wait complete. 2024-09-16T00:37:30.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.289+0000 7f03bc7f7740 1 -- start start 2024-09-16T00:37:30.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03bc7f7740 1 Processor -- start 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136fdb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 0x563136fdbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fdc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563136fe1600 con 0x563136fdea30 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563136ec0900 con 0x563136fd4170 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563136fe1780 con 0x563136fd3850 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03bc7f7740 10 client.0 _ll_register_callbacks cb 0x563136ebe780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03b89bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fdc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 0x563136fdbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136fdb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 0x563136fdbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:38408/0 (socket says 172.21.15.1:38408) 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03abfff640 1 -- 172.21.15.1:0/4251486524 learned_addr learned my addr 172.21.15.1:0/4251486524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:30.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.290+0000 7f03b89bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fdc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44218/0 (socket says 172.21.15.1:44218) 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03b3fff640 1 -- 172.21.15.1:0/4251486524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 msgr2=0x563136fdbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03b3fff640 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 0x563136fdbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03b3fff640 1 -- 172.21.15.1:0/4251486524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 msgr2=0x563136fdc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03b3fff640 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fdc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03b3fff640 1 -- 172.21.15.1:0/4251486524 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563136f00c50 con 0x563136fd3850 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03b89bb640 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fdc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03b3fff640 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136fdb940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03a40cd780 tx=0x7f03a409c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e5384cee1262b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03937fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03937fe640 1 -- 172.21.15.1:0/4251486524 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a40d22e0 con 0x563136fd3850 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03bc7f7740 1 -- 172.21.15.1:0/4251486524 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f039c005520 con 0x563136fd3850 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03937fe640 1 -- 172.21.15.1:0/4251486524 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f03a40bc0b0 con 0x563136fd3850 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03937fe640 1 -- 172.21.15.1:0/4251486524 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03a40a0680 con 0x563136fd3850 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03bc7f7740 10 client.26587 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:30.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.291+0000 7f03bc7f7740 1 -- 172.21.15.1:0/4251486524 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563136fed360 con 0x563136fd3850 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03937fe640 1 -- 172.21.15.1:0/4251486524 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c005520 con 0x563136fd3850 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03937fe640 1 -- 172.21.15.1:0/4251486524 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f03a4018bc0 con 0x563136fd3850 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03bc7f7740 10 client.26587 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03bc7f7740 10 client.26587 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03bc7f7740 20 client.26587 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563136fee770 0x563136ff0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03bc7f7740 4 client.26587 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03bc7f7740 1 -- 172.21.15.1:0/4251486524 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563136f45890 con 0x563136fee770 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03abfff640 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563136fee770 0x563136ff0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03abfff640 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563136fee770 0x563136ff0ba0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=992bb90aa65686d7 server_cookie=d52070a9350ab89a in_seq=0 out_seq=0 2024-09-16T00:37:30.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.292+0000 7f03937fe640 10 client.26587 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:30.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03937fe640 1 -- 172.21.15.1:0/4251486524 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563136f45890 con 0x563136fee770 2024-09-16T00:37:30.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03937fe640 10 client.26587 handle_command_reply: tid=0 2024-09-16T00:37:30.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 client.26587 shutdown 2024-09-16T00:37:30.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 -- 172.21.15.1:0/4251486524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 msgr2=0x563136fdb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:30.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136fdb940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03a40cd780 tx=0x7f03a409c9d0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 -- 172.21.15.1:0/4251486524 shutdown_connections 2024-09-16T00:37:30.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563136fee770 0x563136ff0ba0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563136fdc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd4170 0x563136fdbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/4251486524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136fdb940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03937fe640 0 client.26587 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:30.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 -- 172.21.15.1:0/4251486524 >> 172.21.15.1:0/4251486524 conn(0x563136fcfde0 msgr2=0x563136fd5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:30.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 -- 172.21.15.1:0/4251486524 shutdown_connections 2024-09-16T00:37:30.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.293+0000 7f03bc7f7740 1 -- 172.21.15.1:0/4251486524 wait complete. 2024-09-16T00:37:30.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.294+0000 7f03bc7f7740 20 client.26587 trim_cache size 0 max 16384 2024-09-16T00:37:30.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03bc7f7740 1 Processor -- start 2024-09-16T00:37:30.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03bc7f7740 1 -- start start 2024-09-16T00:37:30.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136ff6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fd4170 0x563136ff6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fdea30 0x563136fd8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41b39b0 con 0x563136fd4170 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41b1280 con 0x563136fdea30 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41af0d0 con 0x563136fd3850 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fd4170 0x563136ff6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fd4170 0x563136ff6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44230/0 (socket says 172.21.15.1:44230) 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03abfff640 1 -- 172.21.15.1:0/3190354567 learned_addr learned my addr 172.21.15.1:0/3190354567 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03abfff640 1 -- 172.21.15.1:0/3190354567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 msgr2=0x563136ff6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03b89bb640 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fdea30 0x563136fd8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03b3fff640 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136ff6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03abfff640 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136ff6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03abfff640 1 -- 172.21.15.1:0/3190354567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fdea30 msgr2=0x563136fd8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03abfff640 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fdea30 0x563136fd8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03abfff640 1 -- 172.21.15.1:0/3190354567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563136fd9e90 con 0x563136fd4170 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03b89bb640 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fdea30 0x563136fd8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.311+0000 7f03b3fff640 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136ff6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03abfff640 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fd4170 0x563136ff6960 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f039c0314b0 tx=0x7f039c02a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39fb0a4a390027d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03a9ffb640 1 -- 172.21.15.1:0/3190354567 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c035040 con 0x563136fd4170 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03a9ffb640 1 -- 172.21.15.1:0/3190354567 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c0048c0 con 0x563136fd4170 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03a9ffb640 1 -- 172.21.15.1:0/3190354567 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c003830 con 0x563136fd4170 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3190354567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fd4170 msgr2=0x563136ff6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fd4170 0x563136ff6960 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f039c0314b0 tx=0x7f039c02a980 comp rx=0 tx=0).stop 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3190354567 shutdown_connections 2024-09-16T00:37:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fdea30 0x563136fd8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fd4170 0x563136ff6960 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/3190354567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd3850 0x563136ff6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.312+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3190354567 >> 172.21.15.1:0/3190354567 conn(0x563136fcfde0 msgr2=0x563136fd5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3190354567 shutdown_connections 2024-09-16T00:37:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 -- 172.21.15.1:0/3190354567 wait complete. 2024-09-16T00:37:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 -- start start 2024-09-16T00:37:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 Processor -- start 2024-09-16T00:37:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd3850 0x563136fe3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd4170 0x563136fe35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563137052430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41af740 con 0x563136fdea30 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41b0bb0 con 0x563136fd3850 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.313+0000 7f03bc7f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41afe10 con 0x563136fd4170 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03bc7f7740 10 client.0 _ll_register_callbacks cb 0x7f03a400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b89bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563137052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b89bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563137052430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44246/0 (socket says 172.21.15.1:44246) 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b89bb640 1 -- 172.21.15.1:0/960283510 learned_addr learned my addr 172.21.15.1:0/960283510 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03abfff640 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd4170 0x563136fe35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b3fff640 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd3850 0x563136fe3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b89bb640 1 -- 172.21.15.1:0/960283510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd4170 msgr2=0x563136fe35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b89bb640 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd4170 0x563136fe35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b89bb640 1 -- 172.21.15.1:0/960283510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd3850 msgr2=0x563136fe3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b89bb640 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd3850 0x563136fe3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b89bb640 1 -- 172.21.15.1:0/960283510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563137052a70 con 0x563136fdea30 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03b89bb640 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563137052430 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f03ac011c80 tx=0x7f03ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7017f6912a1b3423 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03937fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03937fe640 1 -- 172.21.15.1:0/960283510 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac018070 con 0x563136fdea30 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03bc7f7740 1 -- 172.21.15.1:0/960283510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f039c0048c0 con 0x563136fdea30 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.314+0000 7f03937fe640 1 -- 172.21.15.1:0/960283510 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f03ac021db0 con 0x563136fdea30 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03937fe640 1 -- 172.21.15.1:0/960283510 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03ac011e30 con 0x563136fdea30 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03bc7f7740 10 client.17676 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03bc7f7740 1 -- 172.21.15.1:0/960283510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563137059de0 con 0x563136fdea30 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03937fe640 1 -- 172.21.15.1:0/960283510 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c0048c0 con 0x563136fdea30 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03937fe640 1 -- 172.21.15.1:0/960283510 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f03ac028070 con 0x563136fdea30 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03bc7f7740 10 client.17676 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03bc7f7740 10 client.17676 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:37:30.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03bc7f7740 20 client.17676 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:30.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56313705a600 0x56313705ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:30.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03bc7f7740 4 client.17676 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-16T00:37:30.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03bc7f7740 1 -- 172.21.15.1:0/960283510 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x563136f3fbe0 con 0x56313705a600 2024-09-16T00:37:30.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.315+0000 7f03b3fff640 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56313705a600 0x56313705ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:30.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.316+0000 7f03b3fff640 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56313705a600 0x56313705ca20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f2b9206ae99a9 server_cookie=d1fc6a95dd84f93e in_seq=0 out_seq=0 2024-09-16T00:37:30.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.316+0000 7f03937fe640 10 client.17676 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03937fe640 1 -- 172.21.15.1:0/960283510 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563136f3fbe0 con 0x56313705a600 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03937fe640 10 client.17676 handle_command_reply: tid=0 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 client.17676 shutdown 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 -- 172.21.15.1:0/960283510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 msgr2=0x563137052430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563137052430 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f03ac011c80 tx=0x7f03ac004a40 comp rx=0 tx=0).stop 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 -- 172.21.15.1:0/960283510 shutdown_connections 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56313705a600 0x56313705ca20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563136fdea30 0x563137052430 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563136fd4170 0x563136fe35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03937fe640 0 client.17676 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 --2- 172.21.15.1:0/960283510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563136fd3850 0x563136fe3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 -- 172.21.15.1:0/960283510 >> 172.21.15.1:0/960283510 conn(0x563136fcfde0 msgr2=0x563136fce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:30.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 -- 172.21.15.1:0/960283510 shutdown_connections 2024-09-16T00:37:30.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 1 -- 172.21.15.1:0/960283510 wait complete. 2024-09-16T00:37:30.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.328+0000 7f03bc7f7740 20 client.17676 trim_cache size 0 max 16384 2024-09-16T00:37:30.331 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:37:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:37:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:37:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 30, 2024-09-16T00:37:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:37:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: "58fbab5b": { 2024-09-16T00:37:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 30, 2024-09-16T00:37:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:37:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:37:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:37:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:37:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:37:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:37:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 142.3, 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:30.333 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:37:30.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 -- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f03a00a3280 msgr2=0x7f03a00a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:30.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f03a00a3280 0x7f03a00a5740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f03ac005fd0 tx=0x7f03ac004ab0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 -- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4100e20 msgr2=0x7f03b40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:30.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4100e20 0x7f03b40fcc80 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f03a400e9f0 tx=0x7f03a400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 -- 172.21.15.1:0/2261924024 shutdown_connections 2024-09-16T00:37:30.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f03a00a3280 0x7f03a00a5740 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b4102ef0 0x7f03b40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4102570 0x7f03b40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 --2- 172.21.15.1:0/2261924024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b4100e20 0x7f03b40fcc80 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:30.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 -- 172.21.15.1:0/2261924024 >> 172.21.15.1:0/2261924024 conn(0x7f03b40fc7d0 msgr2=0x7f03b406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:30.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 -- 172.21.15.1:0/2261924024 shutdown_connections 2024-09-16T00:37:30.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:30.329+0000 7f03ba41e640 1 -- 172.21.15.1:0/2261924024 wait complete. 2024-09-16T00:37:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:30 smithi001 ceph-mon[30934]: pgmap v623: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:30.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:30 smithi153 ceph-mon[35538]: pgmap v623: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:30.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:30 smithi019 ceph-mon[35186]: pgmap v623: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:37:31.016 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '58fbab5b', quiesce took 3.3 seconds. Will release after: 19.05 2024-09-16T00:37:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:32 smithi001 ceph-mon[30934]: pgmap v624: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:37:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:32 smithi153 ceph-mon[35538]: pgmap v624: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:37:32.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:32 smithi019 ceph-mon[35186]: pgmap v624: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:37:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:34 smithi001 ceph-mon[30934]: pgmap v625: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:37:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:34 smithi153 ceph-mon[35538]: pgmap v625: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:37:34.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:34 smithi019 ceph-mon[35186]: pgmap v625: 129 pgs: 129 active+clean; 4.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:37:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:36 smithi001 ceph-mon[30934]: pgmap v626: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-09-16T00:37:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:36 smithi153 ceph-mon[35538]: pgmap v626: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-09-16T00:37:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:36 smithi019 ceph-mon[35186]: pgmap v626: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-09-16T00:37:38.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:38 smithi001 ceph-mon[30934]: pgmap v627: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-16T00:37:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:38 smithi153 ceph-mon[35538]: pgmap v627: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-16T00:37:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:38 smithi019 ceph-mon[35186]: pgmap v627: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-16T00:37:40.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:40 smithi001 ceph-mon[30934]: pgmap v628: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:37:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:40 smithi153 ceph-mon[35538]: pgmap v628: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:37:40.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:40 smithi019 ceph-mon[35186]: pgmap v628: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:37:42.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:42 smithi001 ceph-mon[30934]: pgmap v629: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-16T00:37:42.465 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:42 smithi153 ceph-mon[35538]: pgmap v629: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-16T00:37:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:42 smithi019 ceph-mon[35186]: pgmap v629: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-16T00:37:44.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:44 smithi001 ceph-mon[30934]: pgmap v630: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-16T00:37:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:44 smithi153 ceph-mon[35538]: pgmap v630: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-16T00:37:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:44 smithi019 ceph-mon[35186]: pgmap v630: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-16T00:37:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:46 smithi153 ceph-mon[35538]: pgmap v631: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-16T00:37:46.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:37:46.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:46 smithi001 ceph-mon[30934]: pgmap v631: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-16T00:37:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:46 smithi019 ceph-mon[35186]: pgmap v631: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-16T00:37:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:47.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.257+0000 7f68795b9640 1 -- 172.21.15.1:0/3115456809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 msgr2=0x7f687410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:47.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.257+0000 7f68795b9640 1 --2- 172.21.15.1:0/3115456809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 0x7f687410da40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f686c009900 tx=0x7f686c02a480 comp rx=0 tx=0).stop 2024-09-16T00:37:47.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.257+0000 7f68795b9640 1 -- 172.21.15.1:0/3115456809 shutdown_connections 2024-09-16T00:37:47.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.257+0000 7f68795b9640 1 --2- 172.21.15.1:0/3115456809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 0x7f687410da40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.257+0000 7f68795b9640 1 --2- 172.21.15.1:0/3115456809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6874102570 0x7f68741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.257+0000 7f68795b9640 1 --2- 172.21.15.1:0/3115456809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874100e20 0x7f6874101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.257+0000 7f68795b9640 1 -- 172.21.15.1:0/3115456809 >> 172.21.15.1:0/3115456809 conn(0x7f68740fc7d0 msgr2=0x7f68740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:47.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.257+0000 7f68795b9640 1 -- 172.21.15.1:0/3115456809 shutdown_connections 2024-09-16T00:37:47.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.257+0000 7f68795b9640 1 -- 172.21.15.1:0/3115456809 wait complete. 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68795b9640 1 Processor -- start 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68795b9640 1 -- start start 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68795b9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874100e20 0x7f6874197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68795b9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6874102570 0x7f6874197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68795b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 0x7f6874197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68795b9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6874110fa0 con 0x7f6874102ef0 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68795b9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6874110e20 con 0x7f6874100e20 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68795b9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6874111120 con 0x7f6874102570 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68737fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 0x7f6874197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68737fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 0x7f6874197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51604/0 (socket says 172.21.15.1:51604) 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.258+0000 7f68737fe640 1 -- 172.21.15.1:0/1893846999 learned_addr learned my addr 172.21.15.1:0/1893846999 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f6872ffd640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874100e20 0x7f6874197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f68727fc640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6874102570 0x7f6874197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f68737fe640 1 -- 172.21.15.1:0/1893846999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6874102570 msgr2=0x7f6874197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:47.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f68737fe640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6874102570 0x7f6874197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f68737fe640 1 -- 172.21.15.1:0/1893846999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874100e20 msgr2=0x7f6874197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:47.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f68737fe640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874100e20 0x7f6874197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f68737fe640 1 -- 172.21.15.1:0/1893846999 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68741a69b0 con 0x7f6874102ef0 2024-09-16T00:37:47.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f68737fe640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 0x7f6874197bc0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f686c004770 tx=0x7f686c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:47.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f685bfff640 1 -- 172.21.15.1:0/1893846999 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c03f070 con 0x7f6874102ef0 2024-09-16T00:37:47.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f685bfff640 1 -- 172.21.15.1:0/1893846999 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686c031c30 con 0x7f6874102ef0 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f68795b9640 1 -- 172.21.15.1:0/1893846999 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68741a6c40 con 0x7f6874102ef0 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f685bfff640 1 -- 172.21.15.1:0/1893846999 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c035030 con 0x7f6874102ef0 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.259+0000 7f68795b9640 1 -- 172.21.15.1:0/1893846999 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68741a71a0 con 0x7f6874102ef0 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f685bfff640 1 -- 172.21.15.1:0/1893846999 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f686c042070 con 0x7f6874102ef0 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f687b992740 1 Processor -- start 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f687b992740 1 -- start start 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f687b992740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e48b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f687b992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 0x55804e496140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f687b992740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e498a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f687b992740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55804e3f12e0 con 0x55804e496680 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f687b992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55804e398ce0 con 0x55804e48bdc0 2024-09-16T00:37:47.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f687b992740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55804e3ed2a0 con 0x55804e48b4a0 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.262+0000 7f685bfff640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f685c0a3240 0x7f685c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e498a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e498a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51608/0 (socket says 172.21.15.1:51608) 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 -- 172.21.15.1:0/2997168291 learned_addr learned my addr 172.21.15.1:0/2997168291 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68737fe640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f685c0a3240 0x7f685c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 --2- 172.21.15.1:0/2997168291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e48b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f6872ffd640 1 --2- 172.21.15.1:0/2997168291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 0x55804e496140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f685bfff640 1 -- 172.21.15.1:0/1893846999 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f686c0f07f0 con 0x7f6874102ef0 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 -- 172.21.15.1:0/2997168291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 msgr2=0x55804e48b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 --2- 172.21.15.1:0/2997168291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e48b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 -- 172.21.15.1:0/2997168291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 msgr2=0x55804e496140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 --2- 172.21.15.1:0/2997168291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 0x55804e496140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 -- 172.21.15.1:0/2997168291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55804e37d440 con 0x55804e496680 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68737fe640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f685c0a3240 0x7f685c0a5700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f686c009a20 tx=0x7f686c036e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68727fc640 1 --2- 172.21.15.1:0/2997168291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e498a70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6868006aa0 tx=0x7f6868009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4bcc141757ea84 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68597fa640 1 -- 172.21.15.1:0/2997168291 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868015760 con 0x55804e496680 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68597fa640 1 -- 172.21.15.1:0/2997168291 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686801d070 con 0x55804e496680 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.263+0000 7f68597fa640 1 -- 172.21.15.1:0/2997168291 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868018b20 con 0x55804e496680 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 -- 172.21.15.1:0/2997168291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 msgr2=0x55804e498a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 --2- 172.21.15.1:0/2997168291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e498a70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6868006aa0 tx=0x7f6868009510 comp rx=0 tx=0).stop 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 -- 172.21.15.1:0/2997168291 shutdown_connections 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 --2- 172.21.15.1:0/2997168291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e498a70 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 --2- 172.21.15.1:0/2997168291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 0x55804e496140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 --2- 172.21.15.1:0/2997168291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e48b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 -- 172.21.15.1:0/2997168291 >> 172.21.15.1:0/2997168291 conn(0x55804e487a30 msgr2=0x55804e487e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 -- 172.21.15.1:0/2997168291 shutdown_connections 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 -- 172.21.15.1:0/2997168291 wait complete. 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.264+0000 7f687b992740 1 -- start start 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f687b992740 1 Processor -- start 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f687b992740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e496060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f687b992740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e493310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f687b992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e496680 0x55804e4938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f687b992740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55804e499250 con 0x55804e48bdc0 2024-09-16T00:37:47.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f687b992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55804e3ac050 con 0x55804e496680 2024-09-16T00:37:47.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f687b992740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55804e4993d0 con 0x55804e48b4a0 2024-09-16T00:37:47.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f687b992740 10 client.0 _ll_register_callbacks cb 0x55804e3ec160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:47.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f6872ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e493310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f6872ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e493310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51624/0 (socket says 172.21.15.1:51624) 2024-09-16T00:37:47.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f6872ffd640 1 -- 172.21.15.1:0/274341285 learned_addr learned my addr 172.21.15.1:0/274341285 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:47.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f68727fc640 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e496060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f68727fc640 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e496680 0x55804e4938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f6872ffd640 1 -- 172.21.15.1:0/274341285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 msgr2=0x55804e496060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:47.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f6872ffd640 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e496060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f6872ffd640 1 -- 172.21.15.1:0/274341285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e496680 msgr2=0x55804e4938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f6872ffd640 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e496680 0x55804e4938e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f6872ffd640 1 -- 172.21.15.1:0/274341285 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55804e1623c0 con 0x55804e48bdc0 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.265+0000 7f68727fc640 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e496060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f6872ffd640 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e493310 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f6864004070 tx=0x7f6864004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8490bddc6300c91d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f6856ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f6856ff5640 1 -- 172.21.15.1:0/274341285 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864013c40 con 0x55804e48bdc0 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f6856ff5640 1 -- 172.21.15.1:0/274341285 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6864013de0 con 0x55804e48bdc0 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f687b992740 1 -- 172.21.15.1:0/274341285 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686801d070 con 0x55804e48bdc0 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f6856ff5640 1 -- 172.21.15.1:0/274341285 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f686401c440 con 0x55804e48bdc0 2024-09-16T00:37:47.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f687b992740 10 client.17688 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:47.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f687b992740 1 -- 172.21.15.1:0/274341285 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55804e4a4d20 con 0x55804e48bdc0 2024-09-16T00:37:47.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f6856ff5640 1 -- 172.21.15.1:0/274341285 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686801d070 con 0x55804e48bdc0 2024-09-16T00:37:47.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f6856ff5640 1 -- 172.21.15.1:0/274341285 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f686401a070 con 0x55804e48bdc0 2024-09-16T00:37:47.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f687b992740 10 client.17688 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:47.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f687b992740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:47.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f687b992740 20 client.17688 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:47.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f687b992740 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55804e4a6280 0x55804e4a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f687b992740 4 client.17688 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.266+0000 7f687b992740 1 -- 172.21.15.1:0/274341285 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55804e3fe850 con 0x55804e4a6280 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.267+0000 7f68727fc640 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55804e4a6280 0x55804e4a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.267+0000 7f6856ff5640 10 client.17688 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.267+0000 7f68727fc640 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55804e4a6280 0x55804e4a86c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd98308eeeee277 server_cookie=a0e23c7a465ba22 in_seq=0 out_seq=0 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f6856ff5640 1 -- 172.21.15.1:0/274341285 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55804e3fe850 con 0x55804e4a6280 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f6856ff5640 10 client.17688 handle_command_reply: tid=0 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 client.17688 shutdown 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 -- 172.21.15.1:0/274341285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 msgr2=0x55804e493310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e493310 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f6864004070 tx=0x7f6864004530 comp rx=0 tx=0).stop 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 -- 172.21.15.1:0/274341285 shutdown_connections 2024-09-16T00:37:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55804e4a6280 0x55804e4a86c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e496680 0x55804e4938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e493310 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f6856ff5640 0 client.17688 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:47.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 --2- 172.21.15.1:0/274341285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e496060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 -- 172.21.15.1:0/274341285 >> 172.21.15.1:0/274341285 conn(0x55804e487a30 msgr2=0x55804e48d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:47.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 -- 172.21.15.1:0/274341285 shutdown_connections 2024-09-16T00:37:47.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 1 -- 172.21.15.1:0/274341285 wait complete. 2024-09-16T00:37:47.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.268+0000 7f687b992740 20 client.17688 trim_cache size 0 max 16384 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.285+0000 7f687b992740 1 Processor -- start 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.285+0000 7f687b992740 1 -- start start 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.285+0000 7f687b992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48b4a0 0x55804e4a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.285+0000 7f687b992740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e495390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.285+0000 7f687b992740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e496680 0x55804e4adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.285+0000 7f687b992740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68741b3910 con 0x55804e48bdc0 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.285+0000 7f687b992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68741b11e0 con 0x55804e48b4a0 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.285+0000 7f687b992740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68741af030 con 0x55804e496680 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.285+0000 7f6872ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e495390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6872ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e495390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51636/0 (socket says 172.21.15.1:51636) 2024-09-16T00:37:47.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6872ffd640 1 -- 172.21.15.1:0/3194964356 learned_addr learned my addr 172.21.15.1:0/3194964356 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f68727fc640 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e496680 0x55804e4adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f68727fc640 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48b4a0 0x55804e4a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6872ffd640 1 -- 172.21.15.1:0/3194964356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e496680 msgr2=0x55804e4adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6872ffd640 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e496680 0x55804e4adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6872ffd640 1 -- 172.21.15.1:0/3194964356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48b4a0 msgr2=0x55804e4a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6872ffd640 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48b4a0 0x55804e4a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6872ffd640 1 -- 172.21.15.1:0/3194964356 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55804e496390 con 0x55804e48bdc0 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f68727fc640 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48b4a0 0x55804e4a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6872ffd640 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e495390 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6864009db0 tx=0x7f68640059a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15086c82e3f6be18 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6858ff9640 1 -- 172.21.15.1:0/3194964356 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686401bb70 con 0x55804e48bdc0 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6858ff9640 1 -- 172.21.15.1:0/3194964356 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686401bd10 con 0x55804e48bdc0 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.286+0000 7f6858ff9640 1 -- 172.21.15.1:0/3194964356 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686401cc10 con 0x55804e48bdc0 2024-09-16T00:37:47.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 -- 172.21.15.1:0/3194964356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 msgr2=0x55804e495390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e495390 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6864009db0 tx=0x7f68640059a0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 -- 172.21.15.1:0/3194964356 shutdown_connections 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e496680 0x55804e4adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e48bdc0 0x55804e495390 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 --2- 172.21.15.1:0/3194964356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48b4a0 0x55804e4a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 -- 172.21.15.1:0/3194964356 >> 172.21.15.1:0/3194964356 conn(0x55804e487a30 msgr2=0x55804e48e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 -- 172.21.15.1:0/3194964356 shutdown_connections 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 -- 172.21.15.1:0/3194964356 wait complete. 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 -- start start 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.287+0000 7f687b992740 1 Processor -- start 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f687b992740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e4904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f687b992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 0x55804e4909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f687b992740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e490f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f687b992740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68741af6a0 con 0x55804e496680 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f687b992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68741b0b10 con 0x55804e48bdc0 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f687b992740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68741afd70 con 0x55804e48b4a0 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f687b992740 10 client.0 _ll_register_callbacks cb 0x7f6864009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e490f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e490f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51648/0 (socket says 172.21.15.1:51648) 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 -- 172.21.15.1:0/752072127 learned_addr learned my addr 172.21.15.1:0/752072127 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e4904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f6872ffd640 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 0x55804e4909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 -- 172.21.15.1:0/752072127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 msgr2=0x55804e4904b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e4904b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 -- 172.21.15.1:0/752072127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 msgr2=0x55804e4909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:47.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 0x55804e4909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 -- 172.21.15.1:0/752072127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55804e510090 con 0x55804e496680 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f68727fc640 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e490f30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6868018630 tx=0x7f6868018700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42dc27bbfc315fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.288+0000 7f6856ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f6856ff5640 1 -- 172.21.15.1:0/752072127 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868018b80 con 0x55804e496680 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f6856ff5640 1 -- 172.21.15.1:0/752072127 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6868018d20 con 0x55804e496680 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f687b992740 1 -- 172.21.15.1:0/752072127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686401bd10 con 0x55804e496680 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f6856ff5640 1 -- 172.21.15.1:0/752072127 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6868015730 con 0x55804e496680 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f687b992740 10 client.17694 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f687b992740 1 -- 172.21.15.1:0/752072127 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55804e5112f0 con 0x55804e496680 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f6856ff5640 1 -- 172.21.15.1:0/752072127 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686401bd10 con 0x55804e496680 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f6856ff5640 1 -- 172.21.15.1:0/752072127 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f68680248d0 con 0x55804e496680 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f687b992740 10 client.17694 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:47.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f687b992740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:47.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f687b992740 20 client.17694 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:47.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f687b992740 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55804e511630 0x55804e513a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:47.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f687b992740 4 client.17694 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:37:47.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f687b992740 1 -- 172.21.15.1:0/752072127 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55804e3f8d60 con 0x55804e511630 2024-09-16T00:37:47.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.289+0000 7f6872ffd640 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55804e511630 0x55804e513a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:47.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f6872ffd640 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55804e511630 0x55804e513a20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aef155471068bc29 server_cookie=6197bd265ba8cb55 in_seq=0 out_seq=0 2024-09-16T00:37:47.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f6856ff5640 10 client.17694 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:47.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f6856ff5640 1 -- 172.21.15.1:0/752072127 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55804e3f8d60 con 0x55804e511630 2024-09-16T00:37:47.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f6856ff5640 10 client.17694 handle_command_reply: tid=0 2024-09-16T00:37:47.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 client.17694 shutdown 2024-09-16T00:37:47.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 -- 172.21.15.1:0/752072127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 msgr2=0x55804e490f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e490f30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6868018630 tx=0x7f6868018700 comp rx=0 tx=0).stop 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 -- 172.21.15.1:0/752072127 shutdown_connections 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55804e511630 0x55804e513a20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55804e496680 0x55804e490f30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55804e48bdc0 0x55804e4909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f6856ff5640 0 client.17694 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 --2- 172.21.15.1:0/752072127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55804e48b4a0 0x55804e4904b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 -- 172.21.15.1:0/752072127 >> 172.21.15.1:0/752072127 conn(0x55804e487a30 msgr2=0x55804e4922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 -- 172.21.15.1:0/752072127 shutdown_connections 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 1 -- 172.21.15.1:0/752072127 wait complete. 2024-09-16T00:37:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.290+0000 7f687b992740 20 client.17694 trim_cache size 0 max 16384 2024-09-16T00:37:47.310 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:37:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:37:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:37:47.310 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:37:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 -- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f685c0a3240 msgr2=0x7f685c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f685c0a3240 0x7f685c0a5700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f686c009a20 tx=0x7f686c036e70 comp rx=0 tx=0).stop 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 -- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 msgr2=0x7f6874197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 0x7f6874197bc0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f686c004770 tx=0x7f686c003c10 comp rx=0 tx=0).stop 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 -- 172.21.15.1:0/1893846999 shutdown_connections 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6874102ef0 0x7f6874197bc0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f685c0a3240 0x7f685c0a5700 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6874102570 0x7f6874197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 --2- 172.21.15.1:0/1893846999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874100e20 0x7f6874197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 -- 172.21.15.1:0/1893846999 >> 172.21.15.1:0/1893846999 conn(0x7f68740fc7d0 msgr2=0x7f6874106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 -- 172.21.15.1:0/1893846999 shutdown_connections 2024-09-16T00:37:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:47.291+0000 7f68795b9640 1 -- 172.21.15.1:0/1893846999 wait complete. 2024-09-16T00:37:48.209 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:37:48.209 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:37:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:48 smithi153 ceph-mon[35538]: pgmap v632: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:37:48.514 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:48 smithi019 ceph-mon[35186]: pgmap v632: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:37:48.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:48 smithi001 ceph-mon[30934]: pgmap v632: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:37:49.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.023+0000 7f301cb7c640 1 -- 172.21.15.1:0/1132911754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 msgr2=0x7f301810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.023+0000 7f301cb7c640 1 --2- 172.21.15.1:0/1132911754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 0x7f301810d950 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3010009930 tx=0x7f301002a420 comp rx=0 tx=0).stop 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.023+0000 7f301cb7c640 1 -- 172.21.15.1:0/1132911754 shutdown_connections 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.023+0000 7f301cb7c640 1 --2- 172.21.15.1:0/1132911754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 0x7f301810d950 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.023+0000 7f301cb7c640 1 --2- 172.21.15.1:0/1132911754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3018100a80 0x7f3018100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.023+0000 7f301cb7c640 1 --2- 172.21.15.1:0/1132911754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018106a80 0x7f3018106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.023+0000 7f301cb7c640 1 -- 172.21.15.1:0/1132911754 >> 172.21.15.1:0/1132911754 conn(0x7f30180fc7a0 msgr2=0x7f30180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.023+0000 7f301cb7c640 1 -- 172.21.15.1:0/1132911754 shutdown_connections 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.023+0000 7f301cb7c640 1 -- 172.21.15.1:0/1132911754 wait complete. 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f301cb7c640 1 Processor -- start 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f301cb7c640 1 -- start start 2024-09-16T00:37:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f301cb7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018100a80 0x7f301810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f301cb7c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 0x7f301810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f301cb7c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3018106a80 0x7f301810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f301cb7c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3018111fa0 con 0x7f3018101420 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f301cb7c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3018111e20 con 0x7f3018100a80 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f301cb7c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3018112120 con 0x7f3018106a80 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f3015d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 0x7f301810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f3015d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 0x7f301810e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51664/0 (socket says 172.21.15.1:51664) 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.024+0000 7f3015d9b640 1 -- 172.21.15.1:0/2981780152 learned_addr learned my addr 172.21.15.1:0/2981780152 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f301659c640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018100a80 0x7f301810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f3016d9d640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3018106a80 0x7f301810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f3015d9b640 1 -- 172.21.15.1:0/2981780152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3018106a80 msgr2=0x7f301810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f3015d9b640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3018106a80 0x7f301810ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f3015d9b640 1 -- 172.21.15.1:0/2981780152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018100a80 msgr2=0x7f301810e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f3015d9b640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018100a80 0x7f301810e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f3015d9b640 1 -- 172.21.15.1:0/2981780152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30181a68b0 con 0x7f3018101420 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f3016d9d640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3018106a80 0x7f301810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f3015d9b640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 0x7f301810e920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f300000c890 tx=0x7f300000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f300f7fe640 1 -- 172.21.15.1:0/2981780152 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000004220 con 0x7f3018101420 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30181a6ba0 con 0x7f3018101420 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f300f7fe640 1 -- 172.21.15.1:0/2981780152 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30000043c0 con 0x7f3018101420 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f300f7fe640 1 -- 172.21.15.1:0/2981780152 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000004a00 con 0x7f3018101420 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.025+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30181a70e0 con 0x7f3018101420 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.026+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3018100ee0 con 0x7f3018101420 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.028+0000 7f300f7fe640 1 -- 172.21.15.1:0/2981780152 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f30000079b0 con 0x7f3018101420 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.029+0000 7f300f7fe640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30040a3240 0x7f30040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.029+0000 7f301659c640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30040a3240 0x7f30040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.029+0000 7f300f7fe640 1 -- 172.21.15.1:0/2981780152 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f30000cfac0 con 0x7f3018101420 2024-09-16T00:37:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.029+0000 7f301659c640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30040a3240 0x7f30040a5700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3008005fd0 tx=0x7f3008009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.033+0000 7f300f7fe640 1 -- 172.21.15.1:0/2981780152 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3000014030 con 0x7f3018101420 2024-09-16T00:37:49.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.254+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f301810f910 con 0x7f3018101420 2024-09-16T00:37:49.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.255+0000 7f300f7fe640 1 -- 172.21.15.1:0/2981780152 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f300009c3e0 con 0x7f3018101420 2024-09-16T00:37:49.255 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:37:49.256 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:37:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:37:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30040a3240 msgr2=0x7f30040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30040a3240 0x7f30040a5700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3008005fd0 tx=0x7f3008009260 comp rx=0 tx=0).stop 2024-09-16T00:37:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 msgr2=0x7f301810e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 0x7f301810e920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f300000c890 tx=0x7f300000cd50 comp rx=0 tx=0).stop 2024-09-16T00:37:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 shutdown_connections 2024-09-16T00:37:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f30040a3240 0x7f30040a5700 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3018106a80 0x7f301810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3018101420 0x7f301810e920 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 --2- 172.21.15.1:0/2981780152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018100a80 0x7f301810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 >> 172.21.15.1:0/2981780152 conn(0x7f30180fc7a0 msgr2=0x7f30180fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 shutdown_connections 2024-09-16T00:37:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:49.257+0000 7f301cb7c640 1 -- 172.21.15.1:0/2981780152 wait complete. 2024-09-16T00:37:50.067 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:37:50.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:37:50.342 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:50.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:50 smithi001 ceph-mon[30934]: pgmap v633: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T00:37:50.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:50 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2981780152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:50 smithi153 ceph-mon[35538]: pgmap v633: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T00:37:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:50 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2981780152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:50.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:50 smithi019 ceph-mon[35186]: pgmap v633: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T00:37:50.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:50 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2981780152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:50.843 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:50.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.844+0000 7fc79847c640 1 -- 172.21.15.1:0/2743658326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7901082e0 msgr2=0x7fc79010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:50.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.844+0000 7fc79847c640 1 --2- 172.21.15.1:0/2743658326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7901082e0 0x7fc79010ec50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc788009cd0 tx=0x7fc78802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:37:50.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.844+0000 7fc79847c640 1 -- 172.21.15.1:0/2743658326 shutdown_connections 2024-09-16T00:37:50.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.844+0000 7fc79847c640 1 --2- 172.21.15.1:0/2743658326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7901082e0 0x7fc79010ec50 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:50.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.844+0000 7fc79847c640 1 --2- 172.21.15.1:0/2743658326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc790103760 0x7fc790107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:50.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.844+0000 7fc79847c640 1 --2- 172.21.15.1:0/2743658326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc790102db0 0x7fc790103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:50.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.844+0000 7fc79847c640 1 -- 172.21.15.1:0/2743658326 >> 172.21.15.1:0/2743658326 conn(0x7fc7900fc7d0 msgr2=0x7fc7900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:50.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.844+0000 7fc79847c640 1 -- 172.21.15.1:0/2743658326 shutdown_connections 2024-09-16T00:37:50.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.844+0000 7fc79847c640 1 -- 172.21.15.1:0/2743658326 wait complete. 2024-09-16T00:37:50.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.845+0000 7fc79847c640 1 Processor -- start 2024-09-16T00:37:50.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.845+0000 7fc79847c640 1 -- start start 2024-09-16T00:37:50.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.845+0000 7fc79847c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc790102db0 0x7fc790100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:50.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc79847c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc790103760 0x7fc790100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:50.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc79847c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7901082e0 0x7fc790101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:50.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc79847c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901120b0 con 0x7fc790102db0 2024-09-16T00:37:50.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc79847c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc790111f30 con 0x7fc7901082e0 2024-09-16T00:37:50.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc79847c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc790112230 con 0x7fc790103760 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796218640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc790102db0 0x7fc790100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796218640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc790102db0 0x7fc790100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40150/0 (socket says 172.21.15.1:40150) 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796218640 1 -- 172.21.15.1:0/1834514308 learned_addr learned my addr 172.21.15.1:0/1834514308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc795a17640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc790103760 0x7fc790100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796a19640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7901082e0 0x7fc790101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796218640 1 -- 172.21.15.1:0/1834514308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc790103760 msgr2=0x7fc790100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796218640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc790103760 0x7fc790100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796218640 1 -- 172.21.15.1:0/1834514308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7901082e0 msgr2=0x7fc790101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796218640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7901082e0 0x7fc790101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796218640 1 -- 172.21.15.1:0/1834514308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7901a6a70 con 0x7fc790102db0 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796a19640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7901082e0 0x7fc790101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.846+0000 7fc796218640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc790102db0 0x7fc790100a10 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00e9f0 tx=0x7fc78c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:50.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.847+0000 7fc7877fe640 1 -- 172.21.15.1:0/1834514308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00cd80 con 0x7fc790102db0 2024-09-16T00:37:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.847+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7901a6d60 con 0x7fc790102db0 2024-09-16T00:37:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.847+0000 7fc7877fe640 1 -- 172.21.15.1:0/1834514308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78c005500 con 0x7fc790102db0 2024-09-16T00:37:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.847+0000 7fc7877fe640 1 -- 172.21.15.1:0/1834514308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c010690 con 0x7fc790102db0 2024-09-16T00:37:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.847+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7901a7240 con 0x7fc790102db0 2024-09-16T00:37:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.848+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc790103190 con 0x7fc790102db0 2024-09-16T00:37:50.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.849+0000 7fc7877fe640 1 -- 172.21.15.1:0/1834514308 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc78c0046c0 con 0x7fc790102db0 2024-09-16T00:37:50.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.850+0000 7fc7877fe640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc77c0a32c0 0x7fc77c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:50.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.850+0000 7fc7877fe640 1 -- 172.21.15.1:0/1834514308 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc78c0d0300 con 0x7fc790102db0 2024-09-16T00:37:50.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.850+0000 7fc795a17640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc77c0a32c0 0x7fc77c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:50.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.850+0000 7fc795a17640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc77c0a32c0 0x7fc77c0a5780 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc7901024a0 tx=0x7fc78000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:50.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:50.853+0000 7fc7877fe640 1 -- 172.21.15.1:0/1834514308 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc78c0541b0 con 0x7fc790102db0 2024-09-16T00:37:51.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.080+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc790101f40 con 0x7fc790102db0 2024-09-16T00:37:51.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.080+0000 7fc7877fe640 1 -- 172.21.15.1:0/1834514308 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fc790101f40 con 0x7fc790102db0 2024-09-16T00:37:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:37:51.081 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:37:51.081 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:37:51.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc77c0a32c0 msgr2=0x7fc77c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc77c0a32c0 0x7fc77c0a5780 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc7901024a0 tx=0x7fc78000b410 comp rx=0 tx=0).stop 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc790102db0 msgr2=0x7fc790100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc790102db0 0x7fc790100a10 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00e9f0 tx=0x7fc78c00edf0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 shutdown_connections 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7901082e0 0x7fc790101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc77c0a32c0 0x7fc77c0a5780 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc790103760 0x7fc790100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 --2- 172.21.15.1:0/1834514308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc790102db0 0x7fc790100a10 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 >> 172.21.15.1:0/1834514308 conn(0x7fc7900fc7d0 msgr2=0x7fc790106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 shutdown_connections 2024-09-16T00:37:51.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.084+0000 7fc79847c640 1 -- 172.21.15.1:0/1834514308 wait complete. 2024-09-16T00:37:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.291+0000 7f3cae5e4640 1 -- 172.21.15.1:0/135886632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8103660 msgr2=0x7f3ca810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.291+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/135886632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8103660 0x7f3ca810bb50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009900 tx=0x7f3c9c02a480 comp rx=0 tx=0).stop 2024-09-16T00:37:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.291+0000 7f3cae5e4640 1 -- 172.21.15.1:0/135886632 shutdown_connections 2024-09-16T00:37:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.291+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/135886632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ca810c170 0x7f3ca810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.291+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/135886632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8103660 0x7f3ca810bb50 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.291+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/135886632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca8102d40 0x7f3ca8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.291+0000 7f3cae5e4640 1 -- 172.21.15.1:0/135886632 >> 172.21.15.1:0/135886632 conn(0x7f3ca80fc7a0 msgr2=0x7f3ca80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.291+0000 7f3cae5e4640 1 -- 172.21.15.1:0/135886632 shutdown_connections 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.291+0000 7f3cae5e4640 1 -- 172.21.15.1:0/135886632 wait complete. 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.292+0000 7f3cae5e4640 1 Processor -- start 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.292+0000 7f3cae5e4640 1 -- start start 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.292+0000 7f3cae5e4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8102d40 0x7f3ca81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.292+0000 7f3cae5e4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ca8103660 0x7f3ca81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.292+0000 7f3cae5e4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca810c170 0x7f3ca819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3cae5e4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8111fb0 con 0x7f3ca8102d40 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3cae5e4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8111e30 con 0x7f3ca810c170 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3cae5e4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8112130 con 0x7f3ca8103660 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8102d40 0x7f3ca81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8102d40 0x7f3ca81984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40174/0 (socket says 172.21.15.1:40174) 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca7fff640 1 -- 172.21.15.1:0/3288846998 learned_addr learned my addr 172.21.15.1:0/3288846998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3cacb81640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca810c170 0x7f3ca819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ca8103660 0x7f3ca81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca7fff640 1 -- 172.21.15.1:0/3288846998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ca8103660 msgr2=0x7f3ca81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ca8103660 0x7f3ca81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca7fff640 1 -- 172.21.15.1:0/3288846998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca810c170 msgr2=0x7f3ca819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca810c170 0x7f3ca819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca7fff640 1 -- 172.21.15.1:0/3288846998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca819d450 con 0x7f3ca8102d40 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8102d40 0x7f3ca81984a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800d950 tx=0x7f3c9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca57fa640 1 -- 172.21.15.1:0/3288846998 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98017070 con 0x7f3ca8102d40 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca57fa640 1 -- 172.21.15.1:0/3288846998 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c98009c30 con 0x7f3ca8102d40 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3cae5e4640 1 -- 172.21.15.1:0/3288846998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca819d740 con 0x7f3ca8102d40 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3ca57fa640 1 -- 172.21.15.1:0/3288846998 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800f6b0 con 0x7f3ca8102d40 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.293+0000 7f3cae5e4640 1 -- 172.21.15.1:0/3288846998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca81a5020 con 0x7f3ca8102d40 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3ca57fa640 1 -- 172.21.15.1:0/3288846998 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3c980109d0 con 0x7f3ca8102d40 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cb09bd740 1 Processor -- start 2024-09-16T00:37:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cb09bd740 1 -- start start 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 0x55a45a7ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a45a7062f0 con 0x55a45a7a1b40 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a45a6adcf0 con 0x55a45a7ab370 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a45a7022b0 con 0x55a45a7a05d0 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3ca57fa640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3c900a3240 0x7f3c900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cacb81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.296+0000 7f3cacb81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40184/0 (socket says 172.21.15.1:40184) 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3cacb81640 1 -- 172.21.15.1:0/65196573 learned_addr learned my addr 172.21.15.1:0/65196573 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3cacb81640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3c900a3240 0x7f3c900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/65196573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 0x55a45a7ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/65196573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3ca57fa640 1 -- 172.21.15.1:0/3288846998 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3c980cf950 con 0x7f3ca8102d40 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3cacb81640 1 -- 172.21.15.1:0/65196573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 msgr2=0x55a45a7a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3cacb81640 1 --2- 172.21.15.1:0/65196573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3cacb81640 1 -- 172.21.15.1:0/65196573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 msgr2=0x55a45a7ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3cacb81640 1 --2- 172.21.15.1:0/65196573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 0x55a45a7ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3cacb81640 1 -- 172.21.15.1:0/65196573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a45a6923a0 con 0x55a45a7a1b40 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3cacb81640 1 --2- 172.21.15.1:0/65196573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7a0090 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000bee0 tx=0x7f3ca0008690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17eb5872589e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3cacb81640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3c900a3240 0x7f3c900a5700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009900 tx=0x7f3ca001b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3c8e7fc640 1 -- 172.21.15.1:0/65196573 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0027070 con 0x55a45a7a1b40 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3c8e7fc640 1 -- 172.21.15.1:0/65196573 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca001ad90 con 0x55a45a7a1b40 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.297+0000 7f3c8e7fc640 1 -- 172.21.15.1:0/65196573 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0022b20 con 0x55a45a7a1b40 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 -- 172.21.15.1:0/65196573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 msgr2=0x55a45a7a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/65196573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7a0090 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000bee0 tx=0x7f3ca0008690 comp rx=0 tx=0).stop 2024-09-16T00:37:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 -- 172.21.15.1:0/65196573 shutdown_connections 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/65196573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 0x55a45a7ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/65196573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/65196573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7a0090 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 -- 172.21.15.1:0/65196573 >> 172.21.15.1:0/65196573 conn(0x55a45a769310 msgr2=0x55a45a79cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 -- 172.21.15.1:0/65196573 shutdown_connections 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 -- 172.21.15.1:0/65196573 wait complete. 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.298+0000 7f3cb09bd740 1 -- start start 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cb09bd740 1 Processor -- start 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a05d0 0x55a45a7aad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a1b40 0x55a45a7a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a45a7ae010 con 0x55a45a7ab370 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a45a6c1060 con 0x55a45a7a05d0 2024-09-16T00:37:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a45a7ae190 con 0x55a45a7a1b40 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cb09bd740 10 client.0 _ll_register_callbacks cb 0x55a45a4148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cacb81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cacb81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40192/0 (socket says 172.21.15.1:40192) 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3cacb81640 1 -- 172.21.15.1:0/1604662288 learned_addr learned my addr 172.21.15.1:0/1604662288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a1b40 0x55a45a7a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.299+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a05d0 0x55a45a7aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3cacb81640 1 -- 172.21.15.1:0/1604662288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a1b40 msgr2=0x55a45a7a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3cacb81640 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a1b40 0x55a45a7a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3cacb81640 1 -- 172.21.15.1:0/1604662288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a05d0 msgr2=0x55a45a7aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3cacb81640 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a05d0 0x55a45a7aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3cacb81640 1 -- 172.21.15.1:0/1604662288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a45a4773c0 con 0x55a45a7ab370 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3cacb81640 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a85e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0022740 tx=0x7f3ca000b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9d7b11f1832a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3c877fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3c877fe640 1 -- 172.21.15.1:0/1604662288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0032020 con 0x55a45a7ab370 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3c877fe640 1 -- 172.21.15.1:0/1604662288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3ca0022ab0 con 0x55a45a7ab370 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1604662288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca001ad90 con 0x55a45a7ab370 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3c877fe640 1 -- 172.21.15.1:0/1604662288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ca003d050 con 0x55a45a7ab370 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3cb09bd740 10 client.17718 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1604662288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a45a7b9ad0 con 0x55a45a7ab370 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3c877fe640 1 -- 172.21.15.1:0/1604662288 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca001ad90 con 0x55a45a7ab370 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.300+0000 7f3c877fe640 1 -- 172.21.15.1:0/1604662288 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3ca002e960 con 0x55a45a7ab370 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.301+0000 7f3cb09bd740 10 client.17718 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.301+0000 7f3cb09bd740 10 client.17718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.301+0000 7f3cb09bd740 20 client.17718 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.301+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55a45a7baf90 0x55a45a7bd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.301+0000 7f3cb09bd740 4 client.17718 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.301+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1604662288 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a45a713860 con 0x55a45a7baf90 2024-09-16T00:37:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.301+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55a45a7baf90 0x55a45a7bd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.301+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55a45a7baf90 0x55a45a7bd380 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d95de847e30e245 server_cookie=22024bb381676292 in_seq=0 out_seq=0 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.301+0000 7f3c877fe640 10 client.17718 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3c877fe640 1 -- 172.21.15.1:0/1604662288 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a45a713860 con 0x55a45a7baf90 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3c877fe640 10 client.17718 handle_command_reply: tid=0 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 client.17718 shutdown 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1604662288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 msgr2=0x55a45a7a85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a85e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0022740 tx=0x7f3ca000b300 comp rx=0 tx=0).stop 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1604662288 shutdown_connections 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a85e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a1b40 0x55a45a7a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55a45a7baf90 0x55a45a7bd380 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3c877fe640 0 client.17718 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:37:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1604662288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a05d0 0x55a45a7aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1604662288 >> 172.21.15.1:0/1604662288 conn(0x55a45a769310 msgr2=0x55a45a7a2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1604662288 shutdown_connections 2024-09-16T00:37:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.302+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1604662288 wait complete. 2024-09-16T00:37:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.303+0000 7f3cb09bd740 20 client.17718 trim_cache size 0 max 16384 2024-09-16T00:37:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3cb09bd740 1 Processor -- start 2024-09-16T00:37:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3cb09bd740 1 -- start start 2024-09-16T00:37:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7bd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 0x55a45a7a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81b1790 con 0x55a45a7a1b40 2024-09-16T00:37:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81af060 con 0x55a45a7ab370 2024-09-16T00:37:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81aceb0 con 0x55a45a7a05d0 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3ca77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3ca77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40194/0 (socket says 172.21.15.1:40194) 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3ca77fe640 1 -- 172.21.15.1:0/4210457880 learned_addr learned my addr 172.21.15.1:0/4210457880 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.316+0000 7f3cacb81640 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 0x55a45a7a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7bd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3ca77fe640 1 -- 172.21.15.1:0/4210457880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 msgr2=0x55a45a7bd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7bd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3ca77fe640 1 -- 172.21.15.1:0/4210457880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 msgr2=0x55a45a7a3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 0x55a45a7a3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3ca77fe640 1 -- 172.21.15.1:0/4210457880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a45a79d730 con 0x55a45a7a1b40 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3cacb81640 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 0x55a45a7a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7bd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:51.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7b72b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c033000 tx=0x7f3c9c031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c09d6f163d67731 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3c8dffb640 1 -- 172.21.15.1:0/4210457880 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c035040 con 0x55a45a7a1b40 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3c8dffb640 1 -- 172.21.15.1:0/4210457880 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c9c031790 con 0x55a45a7a1b40 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.317+0000 7f3c8dffb640 1 -- 172.21.15.1:0/4210457880 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c003d80 con 0x55a45a7a1b40 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 -- 172.21.15.1:0/4210457880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 msgr2=0x55a45a7b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7b72b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c033000 tx=0x7f3c9c031660 comp rx=0 tx=0).stop 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 -- 172.21.15.1:0/4210457880 shutdown_connections 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7ab370 0x55a45a7a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7a1b40 0x55a45a7b72b0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/4210457880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7bd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 -- 172.21.15.1:0/4210457880 >> 172.21.15.1:0/4210457880 conn(0x55a45a769310 msgr2=0x55a45a7a7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 -- 172.21.15.1:0/4210457880 shutdown_connections 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 -- 172.21.15.1:0/4210457880 wait complete. 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.318+0000 7f3cb09bd740 1 -- start start 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cb09bd740 1 Processor -- start 2024-09-16T00:37:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a1b40 0x55a45a7b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cb09bd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81ad520 con 0x55a45a7ab370 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81ae990 con 0x55a45a7a1b40 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cb09bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81adbf0 con 0x55a45a7a05d0 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cb09bd740 10 client.0 _ll_register_callbacks cb 0x7f3ca0025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cacb81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cacb81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40208/0 (socket says 172.21.15.1:40208) 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cacb81640 1 -- 172.21.15.1:0/1587421171 learned_addr learned my addr 172.21.15.1:0/1587421171 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a1b40 0x55a45a7b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cacb81640 1 -- 172.21.15.1:0/1587421171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 msgr2=0x55a45a7b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:51.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cacb81640 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cacb81640 1 -- 172.21.15.1:0/1587421171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a1b40 msgr2=0x55a45a7b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cacb81640 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a1b40 0x55a45a7b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cacb81640 1 -- 172.21.15.1:0/1587421171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a45a7a4f30 con 0x55a45a7ab370 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.319+0000 7f3cacb81640 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a4840 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0002230 tx=0x7f3ca000b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f91a4b407743ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3c877fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3c877fe640 1 -- 172.21.15.1:0/1587421171 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0032040 con 0x55a45a7ab370 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3c877fe640 1 -- 172.21.15.1:0/1587421171 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3ca001ad90 con 0x55a45a7ab370 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1587421171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c9c031790 con 0x55a45a7ab370 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3c877fe640 1 -- 172.21.15.1:0/1587421171 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ca004f050 con 0x55a45a7ab370 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3cb09bd740 10 client.17724 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1587421171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a45a825420 con 0x55a45a7ab370 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3c877fe640 1 -- 172.21.15.1:0/1587421171 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c9c031790 con 0x55a45a7ab370 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3c877fe640 1 -- 172.21.15.1:0/1587421171 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3ca002e510 con 0x55a45a7ab370 2024-09-16T00:37:51.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3cb09bd740 10 client.17724 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3cb09bd740 10 client.17724 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:37:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3cb09bd740 20 client.17724 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55a45a7baf90 0x55a45a827ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3cb09bd740 4 client.17724 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:37:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.320+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1587421171 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a45a70dd70 con 0x55a45a7baf90 2024-09-16T00:37:51.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.321+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55a45a7baf90 0x55a45a827ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:51.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.321+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55a45a7baf90 0x55a45a827ba0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=758297784b44331d server_cookie=868977f8bc232ce0 in_seq=0 out_seq=0 2024-09-16T00:37:51.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.321+0000 7f3c877fe640 10 client.17724 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:37:51.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.321+0000 7f3c877fe640 1 -- 172.21.15.1:0/1587421171 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a45a70dd70 con 0x55a45a7baf90 2024-09-16T00:37:51.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.321+0000 7f3c877fe640 10 client.17724 handle_command_reply: tid=0 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.321+0000 7f3cb09bd740 1 client.17724 shutdown 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.321+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1587421171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 msgr2=0x55a45a7a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.321+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a4840 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0002230 tx=0x7f3ca000b580 comp rx=0 tx=0).stop 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1587421171 shutdown_connections 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45a7ab370 0x55a45a7a4840 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55a45a7baf90 0x55a45a827ba0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a45a7a1b40 0x55a45a7b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cb09bd740 1 --2- 172.21.15.1:0/1587421171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a45a7a05d0 0x55a45a7b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3c877fe640 0 client.17724 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1587421171 >> 172.21.15.1:0/1587421171 conn(0x55a45a769310 msgr2=0x55a45a7a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1587421171 shutdown_connections 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cb09bd740 1 -- 172.21.15.1:0/1587421171 wait complete. 2024-09-16T00:37:51.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cb09bd740 20 client.17724 trim_cache size 0 max 16384 2024-09-16T00:37:51.329 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:37:51.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 -- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3c900a3240 msgr2=0x7f3c900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3c900a3240 0x7f3c900a5700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009900 tx=0x7f3ca001b000 comp rx=0 tx=0).stop 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 -- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8102d40 msgr2=0x7f3ca81984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8102d40 0x7f3ca81984a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800d950 tx=0x7f3c9800de10 comp rx=0 tx=0).stop 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 -- 172.21.15.1:0/3288846998 shutdown_connections 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca810c170 0x7f3ca819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3c900a3240 0x7f3c900a5700 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ca8103660 0x7f3ca81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 --2- 172.21.15.1:0/3288846998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca8102d40 0x7f3ca81984a0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.322+0000 7f3cae5e4640 1 -- 172.21.15.1:0/3288846998 >> 172.21.15.1:0/3288846998 conn(0x7f3ca80fc7a0 msgr2=0x7f3ca80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.323+0000 7f3cae5e4640 1 -- 172.21.15.1:0/3288846998 shutdown_connections 2024-09-16T00:37:51.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:51.323+0000 7f3cae5e4640 1 -- 172.21.15.1:0/3288846998 wait complete. 2024-09-16T00:37:51.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:51 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1834514308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:51.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:51 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1834514308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:51.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:51 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1834514308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:52.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:52 smithi001 ceph-mon[30934]: pgmap v634: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T00:37:52.520 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 58fbab5b --release --await' 2024-09-16T00:37:52.520 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 58fbab5b --release --await 2024-09-16T00:37:52.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:52 smithi019 ceph-mon[35186]: pgmap v634: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T00:37:52.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:52 smithi153 ceph-mon[35538]: pgmap v634: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T00:37:53.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:37:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:53.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 -- 172.21.15.1:0/3205540482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4103660 msgr2=0x7f36d410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:53.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 --2- 172.21.15.1:0/3205540482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4103660 0x7f36d410bb50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009900 tx=0x7f36bc02a480 comp rx=0 tx=0).stop 2024-09-16T00:37:53.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 -- 172.21.15.1:0/3205540482 shutdown_connections 2024-09-16T00:37:53.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 --2- 172.21.15.1:0/3205540482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d410c170 0x7f36d410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 --2- 172.21.15.1:0/3205540482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4103660 0x7f36d410bb50 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 --2- 172.21.15.1:0/3205540482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d4102d40 0x7f36d4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 -- 172.21.15.1:0/3205540482 >> 172.21.15.1:0/3205540482 conn(0x7f36d40fc7a0 msgr2=0x7f36d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 -- 172.21.15.1:0/3205540482 shutdown_connections 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 -- 172.21.15.1:0/3205540482 wait complete. 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 Processor -- start 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.631+0000 7f36d948e640 1 -- start start 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d948e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4102d40 0x7f36d4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d948e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d4103660 0x7f36d4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d948e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d410c170 0x7f36d419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d948e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4111f70 con 0x7f36d4102d40 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d948e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4111df0 con 0x7f36d4103660 2024-09-16T00:37:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d948e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41120f0 con 0x7f36d410c170 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4102d40 0x7f36d4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4102d40 0x7f36d4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40220/0 (socket says 172.21.15.1:40220) 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d2ffd640 1 -- 172.21.15.1:0/953517415 learned_addr learned my addr 172.21.15.1:0/953517415 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36cbfff640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d4103660 0x7f36d4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d37fe640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d410c170 0x7f36d419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d2ffd640 1 -- 172.21.15.1:0/953517415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d410c170 msgr2=0x7f36d419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d2ffd640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d410c170 0x7f36d419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d2ffd640 1 -- 172.21.15.1:0/953517415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d4103660 msgr2=0x7f36d4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d2ffd640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d4103660 0x7f36d4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d2ffd640 1 -- 172.21.15.1:0/953517415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d419d3c0 con 0x7f36d4102d40 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.632+0000 7f36d2ffd640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4102d40 0x7f36d4198410 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f36c400c920 tx=0x7f36c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.633+0000 7f36d0ff9640 1 -- 172.21.15.1:0/953517415 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4007c70 con 0x7f36d4102d40 2024-09-16T00:37:53.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.633+0000 7f36d0ff9640 1 -- 172.21.15.1:0/953517415 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c4007e10 con 0x7f36d4102d40 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.633+0000 7f36d948e640 1 -- 172.21.15.1:0/953517415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d419d6b0 con 0x7f36d4102d40 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.633+0000 7f36d0ff9640 1 -- 172.21.15.1:0/953517415 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4018890 con 0x7f36d4102d40 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.633+0000 7f36d948e640 1 -- 172.21.15.1:0/953517415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d41a4f90 con 0x7f36d4102d40 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.635+0000 7f36d0ff9640 1 -- 172.21.15.1:0/953517415 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f36c4018a70 con 0x7f36d4102d40 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.635+0000 7f36db867740 1 Processor -- start 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.635+0000 7f36db867740 1 -- start start 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36db867740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f9a3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d0ff9640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c00a3240 0x7f36c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36db867740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36db867740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36db867740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dc1f908620 con 0x55dc1f9a3060 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36db867740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc1f8afe50 con 0x55dc1f9a3980 2024-09-16T00:37:53.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36db867740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc1f9045e0 con 0x55dc1f9ae240 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f9a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f9a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40236/0 (socket says 172.21.15.1:40236) 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d37fe640 1 -- 172.21.15.1:0/1528242728 learned_addr learned my addr 172.21.15.1:0/1528242728 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d0ff9640 1 -- 172.21.15.1:0/953517415 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f36c40cff10 con 0x7f36d4102d40 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d2ffd640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c00a3240 0x7f36c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36cbfff640 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d37fe640 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d37fe640 1 -- 172.21.15.1:0/1528242728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 msgr2=0x55dc1f9b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d37fe640 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d37fe640 1 -- 172.21.15.1:0/1528242728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 msgr2=0x55dc1f9add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.636+0000 7f36d37fe640 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.637+0000 7f36d37fe640 1 -- 172.21.15.1:0/1528242728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc1f8a0d40 con 0x55dc1f9a3060 2024-09-16T00:37:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.637+0000 7f36cbfff640 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.637+0000 7f36d2ffd640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c00a3240 0x7f36c00a5700 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f36c409c0c0 tx=0x7f36c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.637+0000 7f36d37fe640 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f9a3440 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0092f0 tx=0x7f36cc009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e6c5c3cdeb3b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.637+0000 7f36c97fa640 1 -- 172.21.15.1:0/1528242728 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc019030 con 0x55dc1f9a3060 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.637+0000 7f36c97fa640 1 -- 172.21.15.1:0/1528242728 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc011540 con 0x55dc1f9a3060 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.637+0000 7f36c97fa640 1 -- 172.21.15.1:0/1528242728 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc005220 con 0x55dc1f9a3060 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 -- 172.21.15.1:0/1528242728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 msgr2=0x55dc1f9a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f9a3440 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0092f0 tx=0x7f36cc009770 comp rx=0 tx=0).stop 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 -- 172.21.15.1:0/1528242728 shutdown_connections 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 --2- 172.21.15.1:0/1528242728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f9a3440 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 -- 172.21.15.1:0/1528242728 >> 172.21.15.1:0/1528242728 conn(0x55dc1f99f5f0 msgr2=0x55dc1f99f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 -- 172.21.15.1:0/1528242728 shutdown_connections 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 -- 172.21.15.1:0/1528242728 wait complete. 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 -- start start 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.638+0000 7f36db867740 1 Processor -- start 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36db867740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 0x55dc1f9ab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36db867740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3980 0x55dc1f9ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36db867740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9abc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36db867740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dc1f9b0e10 con 0x55dc1f9a3980 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36db867740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc1f890f10 con 0x55dc1f9a3060 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36db867740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc1f9b0f90 con 0x55dc1f9ae240 2024-09-16T00:37:53.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36db867740 10 client.0 _ll_register_callbacks cb 0x55dc1f8b4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:53.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3980 0x55dc1f9ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3980 0x55dc1f9ab6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40238/0 (socket says 172.21.15.1:40238) 2024-09-16T00:37:53.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36cbfff640 1 -- 172.21.15.1:0/728150745 learned_addr learned my addr 172.21.15.1:0/728150745 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:53.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36d37fe640 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 0x55dc1f9ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36d37fe640 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36cbfff640 1 -- 172.21.15.1:0/728150745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 msgr2=0x55dc1f9abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:53.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36cbfff640 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36cbfff640 1 -- 172.21.15.1:0/728150745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 msgr2=0x55dc1f9ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36cbfff640 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 0x55dc1f9ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36cbfff640 1 -- 172.21.15.1:0/728150745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1f617580 con 0x55dc1f9a3980 2024-09-16T00:37:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36cbfff640 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3980 0x55dc1f9ab6c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009230 tx=0x7f36bc0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67e4d66c2208b19 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36ba7fc640 1 -- 172.21.15.1:0/728150745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc035030 con 0x55dc1f9a3980 2024-09-16T00:37:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36ba7fc640 1 -- 172.21.15.1:0/728150745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f36bc004250 con 0x55dc1f9a3980 2024-09-16T00:37:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.639+0000 7f36db867740 1 -- 172.21.15.1:0/728150745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36cc011540 con 0x55dc1f9a3980 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36ba7fc640 1 -- 172.21.15.1:0/728150745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36bc04b050 con 0x55dc1f9a3980 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36db867740 10 client.17736 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36db867740 1 -- 172.21.15.1:0/728150745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc1f6e1960 con 0x55dc1f9a3980 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36ba7fc640 1 -- 172.21.15.1:0/728150745 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc011540 con 0x55dc1f9a3980 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36ba7fc640 1 -- 172.21.15.1:0/728150745 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f36bc0049e0 con 0x55dc1f9a3980 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36db867740 10 client.17736 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36db867740 10 client.17736 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36db867740 20 client.17736 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36db867740 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dc1f9bde00 0x55dc1f9c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36db867740 4 client.17736 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36db867740 1 -- 172.21.15.1:0/728150745 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc1f915b90 con 0x55dc1f9bde00 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.640+0000 7f36d37fe640 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dc1f9bde00 0x55dc1f9c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.641+0000 7f36d37fe640 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dc1f9bde00 0x55dc1f9c0230 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ae64ff761b4dab server_cookie=ec55ead94806b248 in_seq=0 out_seq=0 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.641+0000 7f36ba7fc640 10 client.17736 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36ba7fc640 1 -- 172.21.15.1:0/728150745 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc1f915b90 con 0x55dc1f9bde00 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36ba7fc640 10 client.17736 handle_command_reply: tid=0 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 client.17736 shutdown 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 -- 172.21.15.1:0/728150745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3980 msgr2=0x55dc1f9ab6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3980 0x55dc1f9ab6c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009230 tx=0x7f36bc0317e0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 -- 172.21.15.1:0/728150745 shutdown_connections 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dc1f9bde00 0x55dc1f9c0230 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3980 0x55dc1f9ab6c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 --2- 172.21.15.1:0/728150745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 0x55dc1f9ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36ba7fc640 0 client.17736 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:53.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 -- 172.21.15.1:0/728150745 >> 172.21.15.1:0/728150745 conn(0x55dc1f99f5f0 msgr2=0x55dc1f9a5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:53.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 -- 172.21.15.1:0/728150745 shutdown_connections 2024-09-16T00:37:53.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 1 -- 172.21.15.1:0/728150745 wait complete. 2024-09-16T00:37:53.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.642+0000 7f36db867740 20 client.17736 trim_cache size 0 max 16384 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.659+0000 7f36db867740 1 Processor -- start 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.659+0000 7f36db867740 1 -- start start 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.659+0000 7f36db867740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f99fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.659+0000 7f36db867740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.659+0000 7f36db867740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.659+0000 7f36db867740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41b1730 con 0x55dc1f9a3060 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.659+0000 7f36db867740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41af000 con 0x55dc1f9a3980 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.659+0000 7f36db867740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41ace50 con 0x55dc1f9ae240 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f99fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f99fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40252/0 (socket says 172.21.15.1:40252) 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36d37fe640 1 -- 172.21.15.1:0/2792793154 learned_addr learned my addr 172.21.15.1:0/2792793154 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36d37fe640 1 --2- 172.21.15.1:0/2792793154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36cbfff640 1 -- 172.21.15.1:0/2792793154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 msgr2=0x55dc1f9c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36cbfff640 1 --2- 172.21.15.1:0/2792793154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36cbfff640 1 -- 172.21.15.1:0/2792793154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 msgr2=0x55dc1f99fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36cbfff640 1 --2- 172.21.15.1:0/2792793154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f99fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36cbfff640 1 -- 172.21.15.1:0/2792793154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc1f9adaf0 con 0x55dc1f9a3980 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.660+0000 7f36d37fe640 1 --2- 172.21.15.1:0/2792793154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f99fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.661+0000 7f36cbfff640 1 --2- 172.21.15.1:0/2792793154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9a5b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36bc005940 tx=0x7f36bc02a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b8cb7a0b8a537ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.661+0000 7f36c8ff9640 1 -- 172.21.15.1:0/2792793154 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc035070 con 0x55dc1f9a3980 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.661+0000 7f36c8ff9640 1 -- 172.21.15.1:0/2792793154 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36bc048040 con 0x55dc1f9a3980 2024-09-16T00:37:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.661+0000 7f36c8ff9640 1 -- 172.21.15.1:0/2792793154 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc005220 con 0x55dc1f9a3980 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 -- 172.21.15.1:0/2792793154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 msgr2=0x55dc1f9a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 --2- 172.21.15.1:0/2792793154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9a5b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36bc005940 tx=0x7f36bc02a480 comp rx=0 tx=0).stop 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 -- 172.21.15.1:0/2792793154 shutdown_connections 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 --2- 172.21.15.1:0/2792793154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9ae240 0x55dc1f9c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 --2- 172.21.15.1:0/2792793154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3980 0x55dc1f9a5b10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 --2- 172.21.15.1:0/2792793154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9a3060 0x55dc1f99fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 -- 172.21.15.1:0/2792793154 >> 172.21.15.1:0/2792793154 conn(0x55dc1f99f5f0 msgr2=0x55dc1f9ad200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 -- 172.21.15.1:0/2792793154 shutdown_connections 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 -- 172.21.15.1:0/2792793154 wait complete. 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 -- start start 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.662+0000 7f36db867740 1 Processor -- start 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36db867740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 0x55dc1f9b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36db867740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9a3980 0x55dc1f9b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36db867740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36db867740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41ad4c0 con 0x55dc1f9ae240 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36db867740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41ae930 con 0x55dc1f9a3060 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36db867740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41adb90 con 0x55dc1f9a3980 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36db867740 10 client.0 _ll_register_callbacks cb 0x7f36bc0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40268/0 (socket says 172.21.15.1:40268) 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 -- 172.21.15.1:0/1423815581 learned_addr learned my addr 172.21.15.1:0/1423815581 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 0x55dc1f9b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36cbfff640 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9a3980 0x55dc1f9b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 -- 172.21.15.1:0/1423815581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9a3980 msgr2=0x55dc1f9b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9a3980 0x55dc1f9b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 -- 172.21.15.1:0/1423815581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 msgr2=0x55dc1f9b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 0x55dc1f9b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 -- 172.21.15.1:0/1423815581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1fa2c0e0 con 0x55dc1f9ae240 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36d37fe640 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b2b80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0092c0 tx=0x7f36cc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51a4e544fb3af7a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36ba7fc640 1 -- 172.21.15.1:0/1423815581 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc019070 con 0x55dc1f9ae240 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36ba7fc640 1 -- 172.21.15.1:0/1423815581 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f36cc005680 con 0x55dc1f9ae240 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.663+0000 7f36db867740 1 -- 172.21.15.1:0/1423815581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36bc048040 con 0x55dc1f9ae240 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36ba7fc640 1 -- 172.21.15.1:0/1423815581 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36cc028040 con 0x55dc1f9ae240 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36db867740 10 client.17742 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36db867740 1 -- 172.21.15.1:0/1423815581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc1fa2d3c0 con 0x55dc1f9ae240 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36ba7fc640 1 -- 172.21.15.1:0/1423815581 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc033040 con 0x55dc1f9ae240 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36ba7fc640 1 -- 172.21.15.1:0/1423815581 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f36cc037060 con 0x55dc1f9ae240 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36db867740 10 client.17742 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36db867740 10 client.17742 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:37:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36db867740 20 client.17742 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36db867740 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dc1fa2d800 0x55dc1fa2fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36db867740 4 client.17742 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "58fbab5b", "release": true, "await": true}] 2024-09-16T00:37:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36db867740 1 -- 172.21.15.1:0/1423815581 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "58fbab5b", "release": true, "await": true}) -- 0x7f36bc048040 con 0x55dc1fa2d800 2024-09-16T00:37:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.664+0000 7f36cbfff640 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dc1fa2d800 0x55dc1fa2fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.665+0000 7f36cbfff640 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dc1fa2d800 0x55dc1fa2fbf0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7757d9ea7417656 server_cookie=5642a6ba0a96bed8 in_seq=0 out_seq=0 2024-09-16T00:37:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.665+0000 7f36ba7fc640 10 client.17742 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.667+0000 7f36ba7fc640 1 -- 172.21.15.1:0/1423815581 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f36bc048040 con 0x55dc1fa2d800 2024-09-16T00:37:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.667+0000 7f36ba7fc640 10 client.17742 handle_command_reply: tid=0 2024-09-16T00:37:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.667+0000 7f36db867740 1 client.17742 shutdown 2024-09-16T00:37:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 -- 172.21.15.1:0/1423815581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9ae240 msgr2=0x55dc1f9b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b2b80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0092c0 tx=0x7f36cc015040 comp rx=0 tx=0).stop 2024-09-16T00:37:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 -- 172.21.15.1:0/1423815581 shutdown_connections 2024-09-16T00:37:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dc1fa2d800 0x55dc1fa2fbf0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dc1f9ae240 0x55dc1f9b2b80 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc1f9a3980 0x55dc1f9b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 --2- 172.21.15.1:0/1423815581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc1f9a3060 0x55dc1f9b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 -- 172.21.15.1:0/1423815581 >> 172.21.15.1:0/1423815581 conn(0x55dc1f99f5f0 msgr2=0x55dc1f99df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36ba7fc640 0 client.17742 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 -- 172.21.15.1:0/1423815581 shutdown_connections 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 1 -- 172.21.15.1:0/1423815581 wait complete. 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.668+0000 7f36db867740 20 client.17742 trim_cache size 0 max 16384 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 32, 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:37:53.681 INFO:teuthology.orchestra.run.smithi001.stdout: "58fbab5b": { 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 32, 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 142.3, 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:37:53.682 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:37:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:53.683 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:37:53.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 -- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c00a3240 msgr2=0x7f36c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:53.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c00a3240 0x7f36c00a5700 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f36c409c0c0 tx=0x7f36c4000bc0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 -- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4102d40 msgr2=0x7f36d4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4102d40 0x7f36d4198410 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f36c400c920 tx=0x7f36c400cde0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 -- 172.21.15.1:0/953517415 shutdown_connections 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36c00a3240 0x7f36c00a5700 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d410c170 0x7f36d419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d4103660 0x7f36d4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 --2- 172.21.15.1:0/953517415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d4102d40 0x7f36d4198410 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 -- 172.21.15.1:0/953517415 >> 172.21.15.1:0/953517415 conn(0x7f36d40fc7a0 msgr2=0x7f36d40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 -- 172.21.15.1:0/953517415 shutdown_connections 2024-09-16T00:37:53.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:53.669+0000 7f36d948e640 1 -- 172.21.15.1:0/953517415 wait complete. 2024-09-16T00:37:53.928 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:54 smithi001 ceph-mon[30934]: pgmap v635: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:37:54.568 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '58fbab5b', total seconds elapsed: 26.8 2024-09-16T00:37:54.568 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-16T00:37:54.568 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:37:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:54 smithi019 ceph-mon[35186]: pgmap v635: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:37:54.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:54 smithi153 ceph-mon[35538]: pgmap v635: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:37:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.066+0000 7f8f41a37640 1 -- 172.21.15.1:0/3573518838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102570 msgr2=0x7f8f3c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.066+0000 7f8f41a37640 1 --2- 172.21.15.1:0/3573518838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102570 0x7f8f3c1029b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009900 tx=0x7f8f3002a480 comp rx=0 tx=0).stop 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.067+0000 7f8f41a37640 1 -- 172.21.15.1:0/3573518838 shutdown_connections 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.067+0000 7f8f41a37640 1 --2- 172.21.15.1:0/3573518838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f3c102ef0 0x7f8f3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.067+0000 7f8f41a37640 1 --2- 172.21.15.1:0/3573518838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102570 0x7f8f3c1029b0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.067+0000 7f8f41a37640 1 --2- 172.21.15.1:0/3573518838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c100e20 0x7f8f3c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.067+0000 7f8f41a37640 1 -- 172.21.15.1:0/3573518838 >> 172.21.15.1:0/3573518838 conn(0x7f8f3c0fc7d0 msgr2=0x7f8f3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.067+0000 7f8f41a37640 1 -- 172.21.15.1:0/3573518838 shutdown_connections 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.067+0000 7f8f41a37640 1 -- 172.21.15.1:0/3573518838 wait complete. 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.067+0000 7f8f41a37640 1 Processor -- start 2024-09-16T00:37:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.067+0000 7f8f41a37640 1 -- start start 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f41a37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c100e20 0x7f8f3c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f41a37640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f3c102570 0x7f8f3c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f41a37640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102ef0 0x7f8f3c10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f41a37640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c112060 con 0x7f8f3c102ef0 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f41a37640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c111ee0 con 0x7f8f3c100e20 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f41a37640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1121e0 con 0x7f8f3c102570 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102ef0 0x7f8f3c10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102ef0 0x7f8f3c10ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40292/0 (socket says 172.21.15.1:40292) 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3b7fe640 1 -- 172.21.15.1:0/4154499242 learned_addr learned my addr 172.21.15.1:0/4154499242 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3affd640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c100e20 0x7f8f3c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f3c102570 0x7f8f3c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3b7fe640 1 -- 172.21.15.1:0/4154499242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f3c102570 msgr2=0x7f8f3c10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3b7fe640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f3c102570 0x7f8f3c10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3b7fe640 1 -- 172.21.15.1:0/4154499242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c100e20 msgr2=0x7f8f3c10e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3b7fe640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c100e20 0x7f8f3c10e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3b7fe640 1 -- 172.21.15.1:0/4154499242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c1a6a00 con 0x7f8f3c102ef0 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f3c102570 0x7f8f3c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f3b7fe640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102ef0 0x7f8f3c10ef50 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400cad0 tx=0x7f8f3400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f40a35640 1 -- 172.21.15.1:0/4154499242 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400ae20 con 0x7f8f3c102ef0 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f40a35640 1 -- 172.21.15.1:0/4154499242 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f34005020 con 0x7f8f3c102ef0 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.068+0000 7f8f41a37640 1 -- 172.21.15.1:0/4154499242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c1a6cf0 con 0x7f8f3c102ef0 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.069+0000 7f8f40a35640 1 -- 172.21.15.1:0/4154499242 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400f6f0 con 0x7f8f3c102ef0 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.069+0000 7f8f41a37640 1 -- 172.21.15.1:0/4154499242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c1a71e0 con 0x7f8f3c102ef0 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f40a35640 1 -- 172.21.15.1:0/4154499242 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8f340051d0 con 0x7f8f3c102ef0 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f43e10740 1 Processor -- start 2024-09-16T00:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f43e10740 1 -- start start 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e28b40 0x555d75e27090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e275d0 0x555d75e27a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e34760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f43e10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555d75d8d2f0 con 0x555d75e32370 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f43e10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d75d34cf0 con 0x555d75e275d0 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f43e10740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555d75d892b0 con 0x555d75e28b40 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e34760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e34760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40296/0 (socket says 172.21.15.1:40296) 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.071+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/2126772153 learned_addr learned my addr 172.21.15.1:0/2126772153 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f40a35640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8f240a3240 0x7f8f240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3affd640 1 --2- 172.21.15.1:0/2126772153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e275d0 0x555d75e27a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/2126772153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e28b40 0x555d75e27090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3b7fe640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8f240a3240 0x7f8f240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/2126772153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e28b40 msgr2=0x555d75e27090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/2126772153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e28b40 0x555d75e27090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/2126772153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e275d0 msgr2=0x555d75e27a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/2126772153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e275d0 0x555d75e27a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f40a35640 1 -- 172.21.15.1:0/4154499242 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8f340cfcd0 con 0x7f8f3c102ef0 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/2126772153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d75d193a0 con 0x555d75e32370 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/2126772153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e34760 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8f30005770 tx=0x7f8f30005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17671eca08909e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f21ffb640 1 -- 172.21.15.1:0/2126772153 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30043020 con 0x555d75e32370 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f3b7fe640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8f240a3240 0x7f8f240a5700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8f34098350 tx=0x7f8f340122b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f21ffb640 1 -- 172.21.15.1:0/2126772153 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f300046c0 con 0x555d75e32370 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.072+0000 7f8f21ffb640 1 -- 172.21.15.1:0/2126772153 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30035040 con 0x555d75e32370 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 -- 172.21.15.1:0/2126772153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 msgr2=0x555d75e34760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 --2- 172.21.15.1:0/2126772153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e34760 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8f30005770 tx=0x7f8f30005b50 comp rx=0 tx=0).stop 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 -- 172.21.15.1:0/2126772153 shutdown_connections 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 --2- 172.21.15.1:0/2126772153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e34760 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 --2- 172.21.15.1:0/2126772153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e275d0 0x555d75e27a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 --2- 172.21.15.1:0/2126772153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e28b40 0x555d75e27090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 -- 172.21.15.1:0/2126772153 >> 172.21.15.1:0/2126772153 conn(0x555d75df0310 msgr2=0x555d75e23c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 -- 172.21.15.1:0/2126772153 shutdown_connections 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 -- 172.21.15.1:0/2126772153 wait complete. 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.073+0000 7f8f43e10740 1 -- start start 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f43e10740 1 Processor -- start 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 0x555d75e31d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e2efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e2f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f43e10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555d75e35010 con 0x555d75e32370 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f43e10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d75d48060 con 0x555d75e28b40 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f43e10740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555d75e35190 con 0x555d75e275d0 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f43e10740 10 client.0 _ll_register_callbacks cb 0x555d75a9b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e2f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e2f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40298/0 (socket says 172.21.15.1:40298) 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/3513655916 learned_addr learned my addr 172.21.15.1:0/3513655916 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 0x555d75e31d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3affd640 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e2efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/3513655916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 msgr2=0x555d75e31d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 0x555d75e31d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/3513655916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 msgr2=0x555d75e2efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e2efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/3513655916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d75afe3c0 con 0x555d75e32370 2024-09-16T00:37:55.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.074+0000 7f8f3affd640 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e2efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e2f5d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009560 tx=0x7f8f30004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca74ec225ac13d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f1affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f1affd640 1 -- 172.21.15.1:0/3513655916 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30043030 con 0x555d75e32370 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f1affd640 1 -- 172.21.15.1:0/3513655916 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8f300035f0 con 0x555d75e32370 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f43e10740 1 -- 172.21.15.1:0/3513655916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f300046c0 con 0x555d75e32370 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f1affd640 1 -- 172.21.15.1:0/3513655916 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f300037f0 con 0x555d75e32370 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f43e10740 10 client.17754 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f43e10740 1 -- 172.21.15.1:0/3513655916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555d75e40aa0 con 0x555d75e32370 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f1affd640 1 -- 172.21.15.1:0/3513655916 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f300046c0 con 0x555d75e32370 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f1affd640 1 -- 172.21.15.1:0/3513655916 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8f3003a040 con 0x555d75e32370 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f43e10740 10 client.17754 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f43e10740 10 client.17754 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:37:55.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f43e10740 20 client.17754 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:55.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555d75e41df0 0x555d75e441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f43e10740 4 client.17754 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:55.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.075+0000 7f8f43e10740 1 -- 172.21.15.1:0/3513655916 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d75d9a860 con 0x555d75e41df0 2024-09-16T00:37:55.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.076+0000 7f8f3affd640 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555d75e41df0 0x555d75e441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.076+0000 7f8f3affd640 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555d75e41df0 0x555d75e441e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d53cd35dc3c19e64 server_cookie=cceef00f01ec7409 in_seq=0 out_seq=0 2024-09-16T00:37:55.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.076+0000 7f8f1affd640 10 client.17754 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:37:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f1affd640 1 -- 172.21.15.1:0/3513655916 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d75d9a860 con 0x555d75e41df0 2024-09-16T00:37:55.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f1affd640 10 client.17754 handle_command_reply: tid=0 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 client.17754 shutdown 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 -- 172.21.15.1:0/3513655916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 msgr2=0x555d75e2f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e2f5d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009560 tx=0x7f8f30004260 comp rx=0 tx=0).stop 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 -- 172.21.15.1:0/3513655916 shutdown_connections 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555d75e41df0 0x555d75e441e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f1affd640 0 client.17754 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e2f5d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e2efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3513655916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 0x555d75e31d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 -- 172.21.15.1:0/3513655916 >> 172.21.15.1:0/3513655916 conn(0x555d75df0310 msgr2=0x555d75e29960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 -- 172.21.15.1:0/3513655916 shutdown_connections 2024-09-16T00:37:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.077+0000 7f8f43e10740 1 -- 172.21.15.1:0/3513655916 wait complete. 2024-09-16T00:37:55.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.078+0000 7f8f43e10740 20 client.17754 trim_cache size 0 max 16384 2024-09-16T00:37:55.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f43e10740 1 Processor -- start 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f43e10740 1 -- start start 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e275d0 0x555d75e30f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e47200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e32370 0x555d75e47740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f43e10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b3960 con 0x555d75e275d0 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f43e10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b1230 con 0x555d75e28b40 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f43e10740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1af080 con 0x555d75e32370 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e275d0 0x555d75e30f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e275d0 0x555d75e30f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40300/0 (socket says 172.21.15.1:40300) 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/1007692361 learned_addr learned my addr 172.21.15.1:0/1007692361 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.091+0000 7f8f3affd640 1 --2- 172.21.15.1:0/1007692361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e47200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.092+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/1007692361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e32370 msgr2=0x555d75e47740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.092+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/1007692361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e32370 0x555d75e47740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.092+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/1007692361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 msgr2=0x555d75e47200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.092+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/1007692361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e47200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.092+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/1007692361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d75e351b0 con 0x555d75e275d0 2024-09-16T00:37:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.092+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/1007692361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e275d0 0x555d75e30f50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009630 tx=0x7f8f30009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89d8a6a9b61e08e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.092+0000 7f8f217fa640 1 -- 172.21.15.1:0/1007692361 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30035070 con 0x555d75e275d0 2024-09-16T00:37:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.092+0000 7f8f217fa640 1 -- 172.21.15.1:0/1007692361 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f30053070 con 0x555d75e275d0 2024-09-16T00:37:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.092+0000 7f8f217fa640 1 -- 172.21.15.1:0/1007692361 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30043030 con 0x555d75e275d0 2024-09-16T00:37:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 -- 172.21.15.1:0/1007692361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e275d0 msgr2=0x555d75e30f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 --2- 172.21.15.1:0/1007692361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e275d0 0x555d75e30f50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009630 tx=0x7f8f30009810 comp rx=0 tx=0).stop 2024-09-16T00:37:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 -- 172.21.15.1:0/1007692361 shutdown_connections 2024-09-16T00:37:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 --2- 172.21.15.1:0/1007692361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e32370 0x555d75e47740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 --2- 172.21.15.1:0/1007692361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e47200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 --2- 172.21.15.1:0/1007692361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e275d0 0x555d75e30f50 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 -- 172.21.15.1:0/1007692361 >> 172.21.15.1:0/1007692361 conn(0x555d75df0310 msgr2=0x555d75e30840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 -- 172.21.15.1:0/1007692361 shutdown_connections 2024-09-16T00:37:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 -- 172.21.15.1:0/1007692361 wait complete. 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.093+0000 7f8f43e10740 1 -- start start 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f43e10740 1 Processor -- start 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 0x555d75e47790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e44ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f43e10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e45020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f43e10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1af6f0 con 0x555d75e32370 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f43e10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b0b60 con 0x555d75e28b40 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f43e10740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1afdc0 con 0x555d75e275d0 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f43e10740 10 client.0 _ll_register_callbacks cb 0x7f8f2c00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e45020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e45020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40304/0 (socket says 172.21.15.1:40304) 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e44ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3a7fc640 1 -- 172.21.15.1:0/3282223094 learned_addr learned my addr 172.21.15.1:0/3282223094 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 0x555d75e47790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3affd640 1 -- 172.21.15.1:0/3282223094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 msgr2=0x555d75e47790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3affd640 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 0x555d75e47790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3affd640 1 -- 172.21.15.1:0/3282223094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 msgr2=0x555d75e45020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3affd640 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e45020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3affd640 1 -- 172.21.15.1:0/3282223094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d75e45660 con 0x555d75e28b40 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.094+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e45020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.095+0000 7f8f3affd640 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e44ae0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x555d75e2e870 tx=0x7f8f2c011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e9385b2b428f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.095+0000 7f8f1affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.095+0000 7f8f1affd640 1 -- 172.21.15.1:0/3282223094 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c004220 con 0x555d75e28b40 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.095+0000 7f8f1affd640 1 -- 172.21.15.1:0/3282223094 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8f2c0043c0 con 0x555d75e28b40 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.095+0000 7f8f43e10740 1 -- 172.21.15.1:0/3282223094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f30053070 con 0x555d75e28b40 2024-09-16T00:37:55.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.095+0000 7f8f1affd640 1 -- 172.21.15.1:0/3282223094 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f2c02f050 con 0x555d75e28b40 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.095+0000 7f8f43e10740 10 client.26900 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.095+0000 7f8f43e10740 1 -- 172.21.15.1:0/3282223094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555d75eac5c0 con 0x555d75e28b40 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.096+0000 7f8f1affd640 1 -- 172.21.15.1:0/3282223094 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f2c005500 con 0x555d75e28b40 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.096+0000 7f8f1affd640 1 -- 172.21.15.1:0/3282223094 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8f2c01d560 con 0x555d75e28b40 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.096+0000 7f8f43e10740 10 client.26900 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.096+0000 7f8f43e10740 10 client.26900 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.096+0000 7f8f43e10740 20 client.26900 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.096+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555d75e41df0 0x555d75eaed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.096+0000 7f8f43e10740 4 client.26900 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.096+0000 7f8f43e10740 1 -- 172.21.15.1:0/3282223094 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8f30053070 con 0x555d75e41df0 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.096+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555d75e41df0 0x555d75eaed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f3a7fc640 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555d75e41df0 0x555d75eaed20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f814eab06a34484 server_cookie=b8fc3756893c0d71 in_seq=0 out_seq=0 2024-09-16T00:37:55.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f1affd640 10 client.26900 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f1affd640 1 -- 172.21.15.1:0/3282223094 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8f30053070 con 0x555d75e41df0 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f1affd640 10 client.26900 handle_command_reply: tid=0 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 client.26900 shutdown 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 -- 172.21.15.1:0/3282223094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 msgr2=0x555d75e44ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e44ae0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x555d75e2e870 tx=0x7f8f2c011a80 comp rx=0 tx=0).stop 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 -- 172.21.15.1:0/3282223094 shutdown_connections 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555d75e41df0 0x555d75eaed20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d75e32370 0x555d75e45020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f1affd640 0 client.26900 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d75e28b40 0x555d75e44ae0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 --2- 172.21.15.1:0/3282223094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d75e275d0 0x555d75e47790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 -- 172.21.15.1:0/3282223094 >> 172.21.15.1:0/3282223094 conn(0x555d75df0310 msgr2=0x555d75e31060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 -- 172.21.15.1:0/3282223094 shutdown_connections 2024-09-16T00:37:55.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.097+0000 7f8f43e10740 1 -- 172.21.15.1:0/3282223094 wait complete. 2024-09-16T00:37:55.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f43e10740 20 client.26900 trim_cache size 0 max 16384 2024-09-16T00:37:55.103 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 -- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8f240a3240 msgr2=0x7f8f240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8f240a3240 0x7f8f240a5700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8f34098350 tx=0x7f8f340122b0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 -- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102ef0 msgr2=0x7f8f3c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102ef0 0x7f8f3c10ef50 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400cad0 tx=0x7f8f3400cf90 comp rx=0 tx=0).stop 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 -- 172.21.15.1:0/4154499242 shutdown_connections 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f3c102ef0 0x7f8f3c10ef50 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8f240a3240 0x7f8f240a5700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f3c102570 0x7f8f3c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 --2- 172.21.15.1:0/4154499242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c100e20 0x7f8f3c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.098+0000 7f8f41a37640 1 -- 172.21.15.1:0/4154499242 >> 172.21.15.1:0/4154499242 conn(0x7f8f3c0fc7d0 msgr2=0x7f8f3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:55.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.099+0000 7f8f41a37640 1 -- 172.21.15.1:0/4154499242 shutdown_connections 2024-09-16T00:37:55.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:55.099+0000 7f8f41a37640 1 -- 172.21.15.1:0/4154499242 wait complete. 2024-09-16T00:37:55.344 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:55.837 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:37:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 -- 172.21.15.1:0/2734174875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8106f10 msgr2=0x7fbab810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 --2- 172.21.15.1:0/2734174875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8106f10 0x7fbab810ea90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009ca0 tx=0x7fbab002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 -- 172.21.15.1:0/2734174875 shutdown_connections 2024-09-16T00:37:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 --2- 172.21.15.1:0/2734174875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8106f10 0x7fbab810ea90 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 --2- 172.21.15.1:0/2734174875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8102790 0x7fbab81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 --2- 172.21.15.1:0/2734174875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8101de0 0x7fbab81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 -- 172.21.15.1:0/2734174875 >> 172.21.15.1:0/2734174875 conn(0x7fbab80fd8c0 msgr2=0x7fbab80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 -- 172.21.15.1:0/2734174875 shutdown_connections 2024-09-16T00:37:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 -- 172.21.15.1:0/2734174875 wait complete. 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.376+0000 7fbabd040640 1 Processor -- start 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbabd040640 1 -- start start 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbabd040640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8101de0 0x7fbab8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbabd040640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8102790 0x7fbab81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbabd040640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8106f10 0x7fbab819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbabd040640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8111f90 con 0x7fbab8101de0 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbabd040640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8111e10 con 0x7fbab8102790 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbabd040640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8112110 con 0x7fbab8106f10 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8101de0 0x7fbab8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8101de0 0x7fbab8198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40314/0 (socket says 172.21.15.1:40314) 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab6d9d640 1 -- 172.21.15.1:0/204276675 learned_addr learned my addr 172.21.15.1:0/204276675 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbaae59c640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8102790 0x7fbab81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab759e640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8106f10 0x7fbab819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab6d9d640 1 -- 172.21.15.1:0/204276675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8106f10 msgr2=0x7fbab819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab6d9d640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8106f10 0x7fbab819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab6d9d640 1 -- 172.21.15.1:0/204276675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8102790 msgr2=0x7fbab81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab6d9d640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8102790 0x7fbab81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab6d9d640 1 -- 172.21.15.1:0/204276675 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab819d420 con 0x7fbab8101de0 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.377+0000 7fbab759e640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8106f10 0x7fbab819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.378+0000 7fbab6d9d640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8101de0 0x7fbab8198470 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800b4a0 tx=0x7fbaa800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.378+0000 7fbab4d99640 1 -- 172.21.15.1:0/204276675 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8014070 con 0x7fbab8101de0 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.378+0000 7fbab4d99640 1 -- 172.21.15.1:0/204276675 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa8010af0 con 0x7fbab8101de0 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.378+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab819d710 con 0x7fbab8101de0 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.378+0000 7fbab4d99640 1 -- 172.21.15.1:0/204276675 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8017d10 con 0x7fbab8101de0 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.378+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab81a4ff0 con 0x7fbab8101de0 2024-09-16T00:37:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.379+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab81021c0 con 0x7fbab8101de0 2024-09-16T00:37:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.380+0000 7fbab4d99640 1 -- 172.21.15.1:0/204276675 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fbaa8017070 con 0x7fbab8101de0 2024-09-16T00:37:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.381+0000 7fbab4d99640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa40a3240 0x7fbaa40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.381+0000 7fbaae59c640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa40a3240 0x7fbaa40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.381+0000 7fbab4d99640 1 -- 172.21.15.1:0/204276675 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbaa80cf0e0 con 0x7fbab8101de0 2024-09-16T00:37:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.382+0000 7fbaae59c640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa40a3240 0x7fbaa40a5700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbab81088e0 tx=0x7fbaa000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.386+0000 7fbab4d99640 1 -- 172.21.15.1:0/204276675 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbaa809b9a0 con 0x7fbab8101de0 2024-09-16T00:37:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:56 smithi001 ceph-mon[30934]: pgmap v636: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T00:37:56.458 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:56.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:56 smithi019 ceph-mon[35186]: pgmap v636: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T00:37:56.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.615+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbab802d7c0 con 0x7fbab8101de0 2024-09-16T00:37:56.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.615+0000 7fbab4d99640 1 -- 172.21.15.1:0/204276675 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fbaa809b2a0 con 0x7fbab8101de0 2024-09-16T00:37:56.616 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:37:56.616 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:37:56.617 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:37:56.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.617+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa40a3240 msgr2=0x7fbaa40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.617+0000 7fbabd040640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa40a3240 0x7fbaa40a5700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbab81088e0 tx=0x7fbaa000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.617+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8101de0 msgr2=0x7fbab8198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.617+0000 7fbabd040640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8101de0 0x7fbab8198470 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800b4a0 tx=0x7fbaa800b960 comp rx=0 tx=0).stop 2024-09-16T00:37:56.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.617+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 shutdown_connections 2024-09-16T00:37:56.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.617+0000 7fbabd040640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbaa40a3240 0x7fbaa40a5700 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.617+0000 7fbabd040640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab8106f10 0x7fbab819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.617+0000 7fbabd040640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab8102790 0x7fbab81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.617+0000 7fbabd040640 1 --2- 172.21.15.1:0/204276675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbab8101de0 0x7fbab8198470 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.618+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 >> 172.21.15.1:0/204276675 conn(0x7fbab80fd8c0 msgr2=0x7fbab80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:56.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.618+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 shutdown_connections 2024-09-16T00:37:56.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:56.618+0000 7fbabd040640 1 -- 172.21.15.1:0/204276675 wait complete. 2024-09-16T00:37:56.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:56 smithi153 ceph-mon[35538]: pgmap v636: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T00:37:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.126+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/4056663426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036e0 msgr2=0x7fe7ec10bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.126+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/4056663426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036e0 0x7fe7ec10bbd0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc02a480 comp rx=0 tx=0).stop 2024-09-16T00:37:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.126+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/4056663426 shutdown_connections 2024-09-16T00:37:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.126+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/4056663426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7ec10c1a0 0x7fe7ec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.126+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/4056663426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036e0 0x7fe7ec10bbd0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.126+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/4056663426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec102dc0 0x7fe7ec1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.126+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/4056663426 >> 172.21.15.1:0/4056663426 conn(0x7fe7ec0fc7a0 msgr2=0x7fe7ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.126+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/4056663426 shutdown_connections 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.126+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/4056663426 wait complete. 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f3d2b640 1 Processor -- start 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f3d2b640 1 -- start start 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f3d2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102dc0 0x7fe7ec198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f3d2b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec1036e0 0x7fe7ec1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f3d2b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7ec10c1a0 0x7fe7ec19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f3d2b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec111fb0 con 0x7fe7ec102dc0 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f3d2b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec111e30 con 0x7fe7ec1036e0 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f3d2b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec112130 con 0x7fe7ec10c1a0 2024-09-16T00:37:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f1ac7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102dc0 0x7fe7ec198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f1ac7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102dc0 0x7fe7ec198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40322/0 (socket says 172.21.15.1:40322) 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f1ac7640 1 -- 172.21.15.1:0/1611257146 learned_addr learned my addr 172.21.15.1:0/1611257146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f12c6640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec1036e0 0x7fe7ec1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.127+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7ec10c1a0 0x7fe7ec19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7f1ac7640 1 -- 172.21.15.1:0/1611257146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7ec10c1a0 msgr2=0x7fe7ec19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7ec10c1a0 0x7fe7ec19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7f1ac7640 1 -- 172.21.15.1:0/1611257146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec1036e0 msgr2=0x7fe7ec1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec1036e0 0x7fe7ec1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7f1ac7640 1 -- 172.21.15.1:0/1611257146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec19d440 con 0x7fe7ec102dc0 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7ec10c1a0 0x7fe7ec19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102dc0 0x7fe7ec198490 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c890 tx=0x7fe7e800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7e2ffd640 1 -- 172.21.15.1:0/1611257146 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8004220 con 0x7fe7ec102dc0 2024-09-16T00:37:57.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/1611257146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec19d730 con 0x7fe7ec102dc0 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7e2ffd640 1 -- 172.21.15.1:0/1611257146 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e80043c0 con 0x7fe7ec102dc0 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7e2ffd640 1 -- 172.21.15.1:0/1611257146 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80049e0 con 0x7fe7ec102dc0 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.128+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/1611257146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1a5010 con 0x7fe7ec102dc0 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.130+0000 7fe7e2ffd640 1 -- 172.21.15.1:0/1611257146 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe7e80079b0 con 0x7fe7ec102dc0 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f6104740 1 Processor -- start 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f6104740 1 -- start start 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade1964b40 0x55ade1963090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade19635d0 0x55ade1963a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade1974b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f6104740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ade18c92f0 con 0x55ade19635d0 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f6104740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ade1870cf0 con 0x55ade1972780 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f6104740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ade18c52b0 con 0x55ade1964b40 2024-09-16T00:37:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7e2ffd640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7d80a3240 0x7fe7d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f1ac7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade19635d0 0x55ade1963a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f1ac7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade19635d0 0x55ade1963a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40334/0 (socket says 172.21.15.1:40334) 2024-09-16T00:37:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f1ac7640 1 -- 172.21.15.1:0/1086208868 learned_addr learned my addr 172.21.15.1:0/1086208868 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.131+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/1086208868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade1964b40 0x55ade1963090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7f12c6640 1 --2- 172.21.15.1:0/1086208868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade1974b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7d80a3240 0x7fe7d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7e2ffd640 1 -- 172.21.15.1:0/1611257146 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe7e80cfe20 con 0x7fe7ec102dc0 2024-09-16T00:37:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7f1ac7640 1 -- 172.21.15.1:0/1086208868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade1964b40 msgr2=0x55ade1963090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/1086208868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade1964b40 0x55ade1963090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7f1ac7640 1 -- 172.21.15.1:0/1086208868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 msgr2=0x55ade1974b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/1086208868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade1974b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7f1ac7640 1 -- 172.21.15.1:0/1086208868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ade18553a0 con 0x55ade19635d0 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/1086208868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade19635d0 0x55ade1963a10 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8000c00 tx=0x7fe7e80054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf19de077a34bc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7d80a3240 0x7fe7d80a5700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400afd0 tx=0x7fe7e40012a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7d7fff640 1 -- 172.21.15.1:0/1086208868 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e809c9d0 con 0x55ade19635d0 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7d7fff640 1 -- 172.21.15.1:0/1086208868 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e80b02f0 con 0x55ade19635d0 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.132+0000 7fe7d7fff640 1 -- 172.21.15.1:0/1086208868 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e809c2a0 con 0x55ade19635d0 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 -- 172.21.15.1:0/1086208868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade19635d0 msgr2=0x55ade1963a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 --2- 172.21.15.1:0/1086208868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade19635d0 0x55ade1963a10 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8000c00 tx=0x7fe7e80054a0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 -- 172.21.15.1:0/1086208868 shutdown_connections 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 --2- 172.21.15.1:0/1086208868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade1974b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 --2- 172.21.15.1:0/1086208868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade19635d0 0x55ade1963a10 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 --2- 172.21.15.1:0/1086208868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade1964b40 0x55ade1963090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 -- 172.21.15.1:0/1086208868 >> 172.21.15.1:0/1086208868 conn(0x55ade192c310 msgr2=0x55ade195fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 -- 172.21.15.1:0/1086208868 shutdown_connections 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 -- 172.21.15.1:0/1086208868 wait complete. 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.133+0000 7fe7f6104740 1 -- start start 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f6104740 1 Processor -- start 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade19721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade196f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade196fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f6104740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ade1975480 con 0x55ade1964b40 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f6104740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ade1884060 con 0x55ade1972780 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f6104740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ade1975600 con 0x55ade19635d0 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f6104740 10 client.0 _ll_register_callbacks cb 0x55ade15d78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f12c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade196f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f12c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade196f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40338/0 (socket says 172.21.15.1:40338) 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f12c6640 1 -- 172.21.15.1:0/2140010336 learned_addr learned my addr 172.21.15.1:0/2140010336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade196fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.134+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade19721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f12c6640 1 -- 172.21.15.1:0/2140010336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 msgr2=0x55ade19721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f12c6640 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade19721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f12c6640 1 -- 172.21.15.1:0/2140010336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 msgr2=0x55ade196fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f12c6640 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade196fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f12c6640 1 -- 172.21.15.1:0/2140010336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ade163a3c0 con 0x55ade1964b40 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade19721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f12c6640 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade196f460 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc031550 tx=0x7fe7dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c236565571e5cb45 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7d57fa640 1 -- 172.21.15.1:0/2140010336 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc035040 con 0x55ade1964b40 2024-09-16T00:37:57.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f6104740 1 -- 172.21.15.1:0/2140010336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e80b02f0 con 0x55ade1964b40 2024-09-16T00:37:57.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7d57fa640 1 -- 172.21.15.1:0/2140010336 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe7dc005020 con 0x55ade1964b40 2024-09-16T00:37:57.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7d57fa640 1 -- 172.21.15.1:0/2140010336 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc04e050 con 0x55ade1964b40 2024-09-16T00:37:57.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f6104740 10 client.17778 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:57.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7f6104740 1 -- 172.21.15.1:0/2140010336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ade1980fb0 con 0x55ade1964b40 2024-09-16T00:37:57.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.135+0000 7fe7d57fa640 1 -- 172.21.15.1:0/2140010336 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e80b02f0 con 0x55ade1964b40 2024-09-16T00:37:57.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7d57fa640 1 -- 172.21.15.1:0/2140010336 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe7dc0035f0 con 0x55ade1964b40 2024-09-16T00:37:57.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7f6104740 10 client.17778 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:57.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7f6104740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:57.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7f6104740 20 client.17778 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:57.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ade19823c0 0x55ade19847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7f6104740 4 client.17778 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:57.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7f6104740 1 -- 172.21.15.1:0/2140010336 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ade18d6860 con 0x55ade19823c0 2024-09-16T00:37:57.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ade19823c0 0x55ade19847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ade19823c0 0x55ade19847f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0cf6a9e5f8701ef server_cookie=faf9f93376cb36f6 in_seq=0 out_seq=0 2024-09-16T00:37:57.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.136+0000 7fe7d57fa640 10 client.17778 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:57.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7d57fa640 1 -- 172.21.15.1:0/2140010336 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ade18d6860 con 0x55ade19823c0 2024-09-16T00:37:57.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7d57fa640 10 client.17778 handle_command_reply: tid=0 2024-09-16T00:37:57.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 client.17778 shutdown 2024-09-16T00:37:57.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 -- 172.21.15.1:0/2140010336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 msgr2=0x55ade196f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:57.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade196f460 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc031550 tx=0x7fe7dc004270 comp rx=0 tx=0).stop 2024-09-16T00:37:57.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 -- 172.21.15.1:0/2140010336 shutdown_connections 2024-09-16T00:37:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ade19823c0 0x55ade19847f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade196fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade196f460 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7d57fa640 0 client.17778 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2140010336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade19721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 -- 172.21.15.1:0/2140010336 >> 172.21.15.1:0/2140010336 conn(0x55ade192c310 msgr2=0x55ade1965990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 -- 172.21.15.1:0/2140010336 shutdown_connections 2024-09-16T00:37:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.137+0000 7fe7f6104740 1 -- 172.21.15.1:0/2140010336 wait complete. 2024-09-16T00:37:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.138+0000 7fe7f6104740 20 client.17778 trim_cache size 0 max 16384 2024-09-16T00:37:57.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.150+0000 7fe7f6104740 1 Processor -- start 2024-09-16T00:37:57.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.150+0000 7fe7f6104740 1 -- start start 2024-09-16T00:37:57.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade19713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade198a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade198a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f6104740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1af5d0 con 0x55ade1964b40 2024-09-16T00:37:57.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f6104740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1acea0 con 0x55ade1972780 2024-09-16T00:37:57.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f6104740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1aacf0 con 0x55ade19635d0 2024-09-16T00:37:57.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f12c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade198a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f12c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade198a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40348/0 (socket says 172.21.15.1:40348) 2024-09-16T00:37:57.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f12c6640 1 -- 172.21.15.1:0/2506468501 learned_addr learned my addr 172.21.15.1:0/2506468501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:57.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/2506468501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade19713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/2506468501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade198a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f12c6640 1 -- 172.21.15.1:0/2506468501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 msgr2=0x55ade19713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:57.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f12c6640 1 --2- 172.21.15.1:0/2506468501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade19713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f12c6640 1 -- 172.21.15.1:0/2506468501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 msgr2=0x55ade198a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:57.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f12c6640 1 --2- 172.21.15.1:0/2506468501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade198a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f12c6640 1 -- 172.21.15.1:0/2506468501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ade195fed0 con 0x55ade1964b40 2024-09-16T00:37:57.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7f12c6640 1 --2- 172.21.15.1:0/2506468501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade198a0c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0319d0 tx=0x7fe7dc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489e63b18acdf6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:57.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.151+0000 7fe7d77fe640 1 -- 172.21.15.1:0/2506468501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc035050 con 0x55ade1964b40 2024-09-16T00:37:57.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7d77fe640 1 -- 172.21.15.1:0/2506468501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc003bb0 con 0x55ade1964b40 2024-09-16T00:37:57.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7d77fe640 1 -- 172.21.15.1:0/2506468501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc045bc0 con 0x55ade1964b40 2024-09-16T00:37:57.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7f6104740 1 -- 172.21.15.1:0/2506468501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 msgr2=0x55ade198a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2506468501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade198a0c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0319d0 tx=0x7fe7dc001e70 comp rx=0 tx=0).stop 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7f6104740 1 -- 172.21.15.1:0/2506468501 shutdown_connections 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2506468501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1972780 0x55ade198a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2506468501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1964b40 0x55ade198a0c0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7f6104740 1 --2- 172.21.15.1:0/2506468501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade19713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7f6104740 1 -- 172.21.15.1:0/2506468501 >> 172.21.15.1:0/2506468501 conn(0x55ade192c310 msgr2=0x55ade1970cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7f6104740 1 -- 172.21.15.1:0/2506468501 shutdown_connections 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.152+0000 7fe7f6104740 1 -- 172.21.15.1:0/2506468501 wait complete. 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f6104740 1 -- start start 2024-09-16T00:37:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f6104740 1 Processor -- start 2024-09-16T00:37:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade1976960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1964b40 0x55ade1976ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f6104740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1972780 0x55ade19ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f6104740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ab360 con 0x55ade1972780 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f6104740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ac7d0 con 0x55ade1964b40 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f6104740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1aba30 con 0x55ade19635d0 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f6104740 10 client.0 _ll_register_callbacks cb 0x7fe7dc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f22c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1972780 0x55ade19ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f22c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1972780 0x55ade19ec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40356/0 (socket says 172.21.15.1:40356) 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.153+0000 7fe7f22c8640 1 -- 172.21.15.1:0/531256703 learned_addr learned my addr 172.21.15.1:0/531256703 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f12c6640 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1964b40 0x55ade1976ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade1976960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f22c8640 1 -- 172.21.15.1:0/531256703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 msgr2=0x55ade1976960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade1976960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f22c8640 1 -- 172.21.15.1:0/531256703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1964b40 msgr2=0x55ade1976ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1964b40 0x55ade1976ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f22c8640 1 -- 172.21.15.1:0/531256703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ade1977780 con 0x55ade1972780 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade1976960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f22c8640 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1972780 0x55ade19ec090 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4012e00 tx=0x7fe7e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93236dd0a971d23d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7d57fa640 1 -- 172.21.15.1:0/531256703 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4021df0 con 0x55ade1972780 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7d57fa640 1 -- 172.21.15.1:0/531256703 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe7e402b470 con 0x55ade1972780 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f6104740 1 -- 172.21.15.1:0/531256703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7dc003bb0 con 0x55ade1972780 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7d57fa640 1 -- 172.21.15.1:0/531256703 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe7e4034050 con 0x55ade1972780 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f6104740 10 client.17784 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.154+0000 7fe7f6104740 1 -- 172.21.15.1:0/531256703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ade19ed740 con 0x55ade1972780 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7d57fa640 1 -- 172.21.15.1:0/531256703 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc003bb0 con 0x55ade1972780 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7d57fa640 1 -- 172.21.15.1:0/531256703 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe7e4010040 con 0x55ade1972780 2024-09-16T00:37:57.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7f6104740 10 client.17784 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7f6104740 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7f6104740 20 client.17784 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7f6104740 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ade19ede90 0x55ade19f02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7f6104740 4 client.17784 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7f6104740 1 -- 172.21.15.1:0/531256703 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ade18d0d70 con 0x55ade19ede90 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ade19ede90 0x55ade19f02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7f1ac7640 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ade19ede90 0x55ade19f02b0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7ac2506aeb91936 server_cookie=f6b92c819350f125 in_seq=0 out_seq=0 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.155+0000 7fe7d57fa640 10 client.17784 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7d57fa640 1 -- 172.21.15.1:0/531256703 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ade18d0d70 con 0x55ade19ede90 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7d57fa640 10 client.17784 handle_command_reply: tid=0 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 client.17784 shutdown 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 -- 172.21.15.1:0/531256703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1972780 msgr2=0x55ade19ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:57.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1972780 0x55ade19ec090 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4012e00 tx=0x7fe7e4004a40 comp rx=0 tx=0).stop 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 -- 172.21.15.1:0/531256703 shutdown_connections 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ade19ede90 0x55ade19f02b0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ade1972780 0x55ade19ec090 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ade1964b40 0x55ade1976ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 --2- 172.21.15.1:0/531256703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ade19635d0 0x55ade1976960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 -- 172.21.15.1:0/531256703 >> 172.21.15.1:0/531256703 conn(0x55ade192c310 msgr2=0x55ade196d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7d57fa640 0 client.17784 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 -- 172.21.15.1:0/531256703 shutdown_connections 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 1 -- 172.21.15.1:0/531256703 wait complete. 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f6104740 20 client.17784 trim_cache size 0 max 16384 2024-09-16T00:37:57.163 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:37:57.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7d80a3240 msgr2=0x7fe7d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:57.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7d80a3240 0x7fe7d80a5700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400afd0 tx=0x7fe7e40012a0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.156+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102dc0 msgr2=0x7fe7ec198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:57.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.157+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102dc0 0x7fe7ec198490 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c890 tx=0x7fe7e800cd50 comp rx=0 tx=0).stop 2024-09-16T00:37:57.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.157+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/1611257146 shutdown_connections 2024-09-16T00:37:57.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.157+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe7d80a3240 0x7fe7d80a5700 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.157+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7ec10c1a0 0x7fe7ec19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.157+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec1036e0 0x7fe7ec1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.157+0000 7fe7f3d2b640 1 --2- 172.21.15.1:0/1611257146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102dc0 0x7fe7ec198490 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:57.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.157+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/1611257146 >> 172.21.15.1:0/1611257146 conn(0x7fe7ec0fc7a0 msgr2=0x7fe7ec0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:57.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.157+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/1611257146 shutdown_connections 2024-09-16T00:37:57.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:57.157+0000 7fe7f3d2b640 1 -- 172.21.15.1:0/1611257146 wait complete. 2024-09-16T00:37:57.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:57 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/204276675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:57.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:57 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/204276675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:57.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:57 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/204276675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:37:57.993 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-16T00:37:57.993 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-16T00:37:58.589 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:37:58 smithi019 ceph-mon[35186]: pgmap v637: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:37:58.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:37:58 smithi001 ceph-mon[30934]: pgmap v637: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:37:58.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:37:58 smithi153 ceph-mon[35538]: pgmap v637: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:37:59.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.107+0000 7fb9ed41c640 1 -- 172.21.15.1:0/2964937038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 msgr2=0x7fb9e810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.107+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/2964937038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 0x7fb9e810bbb0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009900 tx=0x7fb9d002a480 comp rx=0 tx=0).stop 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.107+0000 7fb9ed41c640 1 -- 172.21.15.1:0/2964937038 shutdown_connections 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.107+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/2964937038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e810c180 0x7fb9e810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.107+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/2964937038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 0x7fb9e810bbb0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.107+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/2964937038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e8102da0 0x7fb9e8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.107+0000 7fb9ed41c640 1 -- 172.21.15.1:0/2964937038 >> 172.21.15.1:0/2964937038 conn(0x7fb9e80fc7a0 msgr2=0x7fb9e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 -- 172.21.15.1:0/2964937038 shutdown_connections 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 -- 172.21.15.1:0/2964937038 wait complete. 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 Processor -- start 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 -- start start 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e8102da0 0x7fb9e8100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 0x7fb9e8100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e810c180 0x7fb9e8101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8111f60 con 0x7fb9e81036c0 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8111de0 con 0x7fb9e8102da0 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9ed41c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81120e0 con 0x7fb9e810c180 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.108+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 0x7fb9e8100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 0x7fb9e8100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40386/0 (socket says 172.21.15.1:40386) 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9dffff640 1 -- 172.21.15.1:0/4001956451 learned_addr learned my addr 172.21.15.1:0/4001956451 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e8102da0 0x7fb9e8100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e810c180 0x7fb9e8101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9dffff640 1 -- 172.21.15.1:0/4001956451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e810c180 msgr2=0x7fb9e8101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9dffff640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e810c180 0x7fb9e8101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9dffff640 1 -- 172.21.15.1:0/4001956451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e8102da0 msgr2=0x7fb9e8100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9dffff640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e8102da0 0x7fb9e8100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9dffff640 1 -- 172.21.15.1:0/4001956451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e81a68a0 con 0x7fb9e81036c0 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9dffff640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 0x7fb9e8100f40 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0006010 tx=0x7fb9d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9e4ff9640 1 -- 172.21.15.1:0/4001956451 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d003f030 con 0x7fb9e81036c0 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9e4ff9640 1 -- 172.21.15.1:0/4001956451 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d0003ce0 con 0x7fb9e81036c0 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9ed41c640 1 -- 172.21.15.1:0/4001956451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e81a6b30 con 0x7fb9e81036c0 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9e4ff9640 1 -- 172.21.15.1:0/4001956451 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0035070 con 0x7fb9e81036c0 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.109+0000 7fb9ed41c640 1 -- 172.21.15.1:0/4001956451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e81a7060 con 0x7fb9e81036c0 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.111+0000 7fb9e4ff9640 1 -- 172.21.15.1:0/4001956451 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fb9d0042030 con 0x7fb9e81036c0 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9ef7f5740 1 Processor -- start 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9ef7f5740 1 -- start start 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 0x555ca10cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10d6a30 0x555ca10d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9e4ff9640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb9d40a32c0 0x7fb9d40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555ca1030160 con 0x555ca10cc170 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ca0fd7720 con 0x555ca10cb850 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555ca102c120 con 0x555ca10d6a30 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40402/0 (socket says 172.21.15.1:40402) 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.112+0000 7fb9e6ffd640 1 -- 172.21.15.1:0/3637759759 learned_addr learned my addr 172.21.15.1:0/3637759759 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb9d40a32c0 0x7fb9d40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e4ff9640 1 -- 172.21.15.1:0/4001956451 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb9d00efe30 con 0x7fb9e81036c0 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 0x555ca10cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9dffff640 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10d6a30 0x555ca10d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e6ffd640 1 -- 172.21.15.1:0/3637759759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10d6a30 msgr2=0x555ca10d8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:59.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10d6a30 0x555ca10d8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e6ffd640 1 -- 172.21.15.1:0/3637759759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 msgr2=0x555ca10cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 0x555ca10cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e6ffd640 1 -- 172.21.15.1:0/3637759759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca0da1ba0 con 0x555ca10cc170 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9dffff640 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10d6a30 0x555ca10d8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb9d40a32c0 0x7fb9d40a5780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007fd0 tx=0x7fb9e00072a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d64f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800b760 tx=0x7fb9d800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e44674e20a41d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9dd7fa640 1 -- 172.21.15.1:0/3637759759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8003dd0 con 0x555ca10cc170 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9dd7fa640 1 -- 172.21.15.1:0/3637759759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d8015070 con 0x555ca10cc170 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.113+0000 7fb9dd7fa640 1 -- 172.21.15.1:0/3637759759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8004e10 con 0x555ca10cc170 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3637759759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 msgr2=0x555ca10d64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d64f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800b760 tx=0x7fb9d800bc20 comp rx=0 tx=0).stop 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3637759759 shutdown_connections 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10d6a30 0x555ca10d8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d64f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3637759759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 0x555ca10cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3637759759 >> 172.21.15.1:0/3637759759 conn(0x555ca10c7de0 msgr2=0x555ca10c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3637759759 shutdown_connections 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3637759759 wait complete. 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.114+0000 7fb9ef7f5740 1 -- start start 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9ef7f5740 1 Processor -- start 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 0x555ca10d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10d6a30 0x555ca10d42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555ca10d9600 con 0x555ca10cc170 2024-09-16T00:37:59.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ca0fb8900 con 0x555ca10d6a30 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555ca10d9780 con 0x555ca10cb850 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9ef7f5740 10 client.0 _ll_register_callbacks cb 0x555ca0fb6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40418/0 (socket says 172.21.15.1:40418) 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9dffff640 1 -- 172.21.15.1:0/3222310942 learned_addr learned my addr 172.21.15.1:0/3222310942 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10d6a30 0x555ca10d42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 0x555ca10d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9dffff640 1 -- 172.21.15.1:0/3222310942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 msgr2=0x555ca10d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9dffff640 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 0x555ca10d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9dffff640 1 -- 172.21.15.1:0/3222310942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10d6a30 msgr2=0x555ca10d42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9dffff640 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10d6a30 0x555ca10d42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9dffff640 1 -- 172.21.15.1:0/3222310942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca0ff8c50 con 0x555ca10cc170 2024-09-16T00:37:59.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9dffff640 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d3da0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00b85c0 tx=0x7fb9d00b84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e9c7cc00d4f460 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.115+0000 7fb9ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/3222310942 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0049030 con 0x555ca10cc170 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/3222310942 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb9d00b87d0 con 0x555ca10cc170 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3222310942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d8015070 con 0x555ca10cc170 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/3222310942 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb9d00ba910 con 0x555ca10cc170 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ef7f5740 10 client.17796 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3222310942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ca10e5200 con 0x555ca10cc170 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/3222310942 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d8015070 con 0x555ca10cc170 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/3222310942 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb9d004c060 con 0x555ca10cc170 2024-09-16T00:37:59.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ef7f5740 10 client.17796 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ef7f5740 10 client.17796 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ef7f5740 20 client.17796 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555ca10e6610 0x555ca10e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ef7f5740 4 client.17796 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.116+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3222310942 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ca103d890 con 0x555ca10e6610 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.117+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555ca10e6610 0x555ca10e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.117+0000 7fb9ce7fc640 10 client.17796 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.117+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555ca10e6610 0x555ca10e8a30 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b8efc5ac290d19 server_cookie=3bcf5dbce6e99048 in_seq=0 out_seq=0 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/3222310942 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ca103d890 con 0x555ca10e6610 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ce7fc640 10 client.17796 handle_command_reply: tid=0 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 client.17796 shutdown 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3222310942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 msgr2=0x555ca10d3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d3da0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00b85c0 tx=0x7fb9d00b84c0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3222310942 shutdown_connections 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555ca10e6610 0x555ca10e8a30 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10d6a30 0x555ca10d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10cc170 0x555ca10d3da0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/3222310942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 0x555ca10d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ce7fc640 0 client.17796 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3222310942 >> 172.21.15.1:0/3222310942 conn(0x555ca10c7de0 msgr2=0x555ca10cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3222310942 shutdown_connections 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/3222310942 wait complete. 2024-09-16T00:37:59.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.118+0000 7fb9ef7f5740 20 client.17796 trim_cache size 0 max 16384 2024-09-16T00:37:59.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.135+0000 7fb9ef7f5740 1 Processor -- start 2024-09-16T00:37:59.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.135+0000 7fb9ef7f5740 1 -- start start 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 0x555ca10d57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cc170 0x555ca10ee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10ee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81b38b0 con 0x555ca10d6a30 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81b1180 con 0x555ca10cc170 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81aefd0 con 0x555ca10cb850 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10ee880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40426/0 (socket says 172.21.15.1:40426) 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e77fe640 1 -- 172.21.15.1:0/2643192630 learned_addr learned my addr 172.21.15.1:0/2643192630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:59.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9dffff640 1 --2- 172.21.15.1:0/2643192630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cc170 0x555ca10ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/2643192630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 0x555ca10d57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e77fe640 1 -- 172.21.15.1:0/2643192630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 msgr2=0x555ca10d57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/2643192630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 0x555ca10d57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e77fe640 1 -- 172.21.15.1:0/2643192630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cc170 msgr2=0x555ca10ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/2643192630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cc170 0x555ca10ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e77fe640 1 -- 172.21.15.1:0/2643192630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca10c7c10 con 0x555ca10d6a30 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/2643192630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10ee880 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0014380 tx=0x7fb9e0007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e332c89b295908fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.136+0000 7fb9dcff9640 1 -- 172.21.15.1:0/2643192630 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0026070 con 0x555ca10d6a30 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9dcff9640 1 -- 172.21.15.1:0/2643192630 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e0004d20 con 0x555ca10d6a30 2024-09-16T00:37:59.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9dcff9640 1 -- 172.21.15.1:0/2643192630 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0005950 con 0x555ca10d6a30 2024-09-16T00:37:59.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/2643192630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 msgr2=0x555ca10ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/2643192630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10ee880 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0014380 tx=0x7fb9e0007520 comp rx=0 tx=0).stop 2024-09-16T00:37:59.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/2643192630 shutdown_connections 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/2643192630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10ee880 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/2643192630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cc170 0x555ca10ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/2643192630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cb850 0x555ca10d57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/2643192630 >> 172.21.15.1:0/2643192630 conn(0x555ca10c7de0 msgr2=0x555ca10d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/2643192630 shutdown_connections 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.137+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/2643192630 wait complete. 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9ef7f5740 1 -- start start 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9ef7f5740 1 Processor -- start 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 0x555ca10df000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cc170 0x555ca10db9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9ef7f5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81af640 con 0x555ca10d6a30 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81b0ab0 con 0x555ca10cb850 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9ef7f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81afd10 con 0x555ca10cc170 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9ef7f5740 10 client.0 _ll_register_callbacks cb 0x7fb9d0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9e77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9e77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10dbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40434/0 (socket says 172.21.15.1:40434) 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9e77fe640 1 -- 172.21.15.1:0/1176544633 learned_addr learned my addr 172.21.15.1:0/1176544633 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9dffff640 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cc170 0x555ca10db9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.138+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 0x555ca10df000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9e77fe640 1 -- 172.21.15.1:0/1176544633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cc170 msgr2=0x555ca10db9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cc170 0x555ca10db9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9e77fe640 1 -- 172.21.15.1:0/1176544633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 msgr2=0x555ca10df000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 0x555ca10df000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9e77fe640 1 -- 172.21.15.1:0/1176544633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca10c33f0 con 0x555ca10d6a30 2024-09-16T00:37:59.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9e77fe640 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10dbee0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007c20 tx=0x7fb9e0004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f207b0f6e5cb56cc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/1176544633 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0026040 con 0x555ca10d6a30 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/1176544633 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb9e00129d0 con 0x555ca10d6a30 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/1176544633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e0004d20 con 0x555ca10d6a30 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/1176544633 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb9e0012bd0 con 0x555ca10d6a30 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9ef7f5740 10 client.17802 fetch_fsmap learned FSMap version 25 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/1176544633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ca1152010 con 0x555ca10d6a30 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.139+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/1176544633 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e0004d20 con 0x555ca10d6a30 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/1176544633 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb9e0031070 con 0x555ca10d6a30 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9ef7f5740 10 client.17802 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9ef7f5740 10 client.17802 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9ef7f5740 20 client.17802 populate_metadata read hostname 'smithi001' 2024-09-16T00:37:59.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555ca11526f0 0x555ca1154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9ef7f5740 4 client.17802 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-16T00:37:59.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/1176544633 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x555ca1037be0 con 0x555ca11526f0 2024-09-16T00:37:59.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555ca11526f0 0x555ca1154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9e6ffd640 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555ca11526f0 0x555ca1154b10 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626dfe123cd05469 server_cookie=75c577959747d2c3 in_seq=0 out_seq=0 2024-09-16T00:37:59.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.140+0000 7fb9ce7fc640 10 client.17802 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ce7fc640 1 -- 172.21.15.1:0/1176544633 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555ca1037be0 con 0x555ca11526f0 2024-09-16T00:37:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ce7fc640 10 client.17802 handle_command_reply: tid=0 2024-09-16T00:37:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 client.17802 shutdown 2024-09-16T00:37:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/1176544633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 msgr2=0x555ca10dbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10dbee0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007c20 tx=0x7fb9e0004c70 comp rx=0 tx=0).stop 2024-09-16T00:37:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/1176544633 shutdown_connections 2024-09-16T00:37:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555ca11526f0 0x555ca1154b10 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca10d6a30 0x555ca10dbee0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ca10cc170 0x555ca10db9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 --2- 172.21.15.1:0/1176544633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca10cb850 0x555ca10df000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ce7fc640 0 client.17802 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/1176544633 >> 172.21.15.1:0/1176544633 conn(0x555ca10c7de0 msgr2=0x555ca10d58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.148+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/1176544633 shutdown_connections 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ef7f5740 1 -- 172.21.15.1:0/1176544633 wait complete. 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ef7f5740 20 client.17802 trim_cache size 0 max 16384 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:37:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 34, 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "328fd998": { 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 34, 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 142.3, 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:37:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:37:59.152 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:37:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 -- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb9d40a32c0 msgr2=0x7fb9d40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb9d40a32c0 0x7fb9d40a5780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007fd0 tx=0x7fb9e00072a0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 -- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 msgr2=0x7fb9e8100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 0x7fb9e8100f40 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0006010 tx=0x7fb9d00042f0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 -- 172.21.15.1:0/4001956451 shutdown_connections 2024-09-16T00:37:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb9d40a32c0 0x7fb9d40a5780 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e810c180 0x7fb9e8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9e81036c0 0x7fb9e8100f40 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 --2- 172.21.15.1:0/4001956451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e8102da0 0x7fb9e8100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 -- 172.21.15.1:0/4001956451 >> 172.21.15.1:0/4001956451 conn(0x7fb9e80fc7a0 msgr2=0x7fb9e80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:59.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 -- 172.21.15.1:0/4001956451 shutdown_connections 2024-09-16T00:37:59.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:37:59.149+0000 7fb9ed41c640 1 -- 172.21.15.1:0/4001956451 wait complete. 2024-09-16T00:37:59.480 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:37:59.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:38:00.068 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '328fd998', quiesce took 5.5 seconds. Will release after: 16.85 2024-09-16T00:38:00.169 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:00.290 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:00 smithi001 ceph-mon[30934]: pgmap v638: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-09-16T00:38:00.609 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:00 smithi019 ceph-mon[35186]: pgmap v638: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-09-16T00:38:00.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.662+0000 7fa58b22c640 1 -- 172.21.15.1:0/654228797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584102790 msgr2=0x7fa5841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.662+0000 7fa58b22c640 1 --2- 172.21.15.1:0/654228797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584102790 0x7fa5841067e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fa570009900 tx=0x7fa57002a410 comp rx=0 tx=0).stop 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.662+0000 7fa58b22c640 1 -- 172.21.15.1:0/654228797 shutdown_connections 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.662+0000 7fa58b22c640 1 --2- 172.21.15.1:0/654228797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa584106f10 0x7fa58410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.662+0000 7fa58b22c640 1 --2- 172.21.15.1:0/654228797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584102790 0x7fa5841067e0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.662+0000 7fa58b22c640 1 --2- 172.21.15.1:0/654228797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa584101de0 0x7fa5841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.662+0000 7fa58b22c640 1 -- 172.21.15.1:0/654228797 >> 172.21.15.1:0/654228797 conn(0x7fa5840fd8c0 msgr2=0x7fa5840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.662+0000 7fa58b22c640 1 -- 172.21.15.1:0/654228797 shutdown_connections 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.662+0000 7fa58b22c640 1 -- 172.21.15.1:0/654228797 wait complete. 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa58b22c640 1 Processor -- start 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa58b22c640 1 -- start start 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa58b22c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584101de0 0x7fa584198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa58b22c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa584102790 0x7fa584198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa58b22c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa584106f10 0x7fa58419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa58b22c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa584111f80 con 0x7fa584101de0 2024-09-16T00:38:00.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa58b22c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa584111e00 con 0x7fa584102790 2024-09-16T00:38:00.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa58b22c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa584112100 con 0x7fa584106f10 2024-09-16T00:38:00.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa588fc8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584101de0 0x7fa584198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa588fc8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584101de0 0x7fa584198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58620/0 (socket says 172.21.15.1:58620) 2024-09-16T00:38:00.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.663+0000 7fa588fc8640 1 -- 172.21.15.1:0/1363212844 learned_addr learned my addr 172.21.15.1:0/1363212844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:00.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa5897c9640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa584106f10 0x7fa58419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa583fff640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa584102790 0x7fa584198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa588fc8640 1 -- 172.21.15.1:0/1363212844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa584106f10 msgr2=0x7fa58419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa588fc8640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa584106f10 0x7fa58419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa588fc8640 1 -- 172.21.15.1:0/1363212844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa584102790 msgr2=0x7fa584198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa588fc8640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa584102790 0x7fa584198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa588fc8640 1 -- 172.21.15.1:0/1363212844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58419d3d0 con 0x7fa584101de0 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa588fc8640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584101de0 0x7fa584198420 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e9f0 tx=0x7fa57800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa581ffb640 1 -- 172.21.15.1:0/1363212844 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57800cd80 con 0x7fa584101de0 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa581ffb640 1 -- 172.21.15.1:0/1363212844 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa578010910 con 0x7fa584101de0 2024-09-16T00:38:00.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa58b22c640 1 -- 172.21.15.1:0/1363212844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58419d6c0 con 0x7fa584101de0 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa581ffb640 1 -- 172.21.15.1:0/1363212844 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578018990 con 0x7fa584101de0 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.664+0000 7fa58b22c640 1 -- 172.21.15.1:0/1363212844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5841a4fa0 con 0x7fa584101de0 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.666+0000 7fa581ffb640 1 -- 172.21.15.1:0/1363212844 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa578010ac0 con 0x7fa584101de0 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa58d605740 1 Processor -- start 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa58d605740 1 -- start start 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa58d605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 0x55e55befee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa58d605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf0dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa58d605740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf10540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa58d605740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e55be64280 con 0x55e55bf0e150 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa58d605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e55be0bb10 con 0x55e55befea60 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa58d605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e55be60240 con 0x55e55beff380 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa581ffb640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5740a3240 0x7fa5740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa583fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf10540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa583fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf10540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58630/0 (socket says 172.21.15.1:58630) 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa5897c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 0x55e55befee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa583fff640 1 -- 172.21.15.1:0/1664220017 learned_addr learned my addr 172.21.15.1:0/1664220017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa5897c9640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5740a3240 0x7fa5740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.667+0000 7fa581ffb640 1 -- 172.21.15.1:0/1363212844 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa5780d01e0 con 0x7fa584101de0 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa588fc8640 1 --2- 172.21.15.1:0/1664220017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf0dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa583fff640 1 -- 172.21.15.1:0/1664220017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 msgr2=0x55e55bf0dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa583fff640 1 --2- 172.21.15.1:0/1664220017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf0dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa583fff640 1 -- 172.21.15.1:0/1664220017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 msgr2=0x55e55befee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa583fff640 1 --2- 172.21.15.1:0/1664220017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 0x55e55befee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa583fff640 1 -- 172.21.15.1:0/1664220017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55be181f0 con 0x55e55bf0e150 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa583fff640 1 --2- 172.21.15.1:0/1664220017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf10540 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa570006010 tx=0x7fa570031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa8ad9a859aa858 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:00.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa5897c9640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5740a3240 0x7fa5740a5700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa57c005c40 tx=0x7fa57c005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa56effd640 1 -- 172.21.15.1:0/1664220017 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57003f070 con 0x55e55bf0e150 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa56effd640 1 -- 172.21.15.1:0/1664220017 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5700043c0 con 0x55e55bf0e150 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.668+0000 7fa56effd640 1 -- 172.21.15.1:0/1664220017 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570035030 con 0x55e55bf0e150 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 -- 172.21.15.1:0/1664220017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 msgr2=0x55e55bf10540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 --2- 172.21.15.1:0/1664220017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf10540 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa570006010 tx=0x7fa570031500 comp rx=0 tx=0).stop 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 -- 172.21.15.1:0/1664220017 shutdown_connections 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 --2- 172.21.15.1:0/1664220017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf10540 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 --2- 172.21.15.1:0/1664220017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf0dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 --2- 172.21.15.1:0/1664220017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 0x55e55befee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 -- 172.21.15.1:0/1664220017 >> 172.21.15.1:0/1664220017 conn(0x55e55befb0f0 msgr2=0x55e55befb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 -- 172.21.15.1:0/1664220017 shutdown_connections 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 -- 172.21.15.1:0/1664220017 wait complete. 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 -- start start 2024-09-16T00:38:00.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.669+0000 7fa58d605740 1 Processor -- start 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa58d605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55befea60 0x55e55bf0afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa58d605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55beff380 0x55e55bf0b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa58d605740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf0ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa58d605740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e55bf10d20 con 0x55e55bf0e150 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa58d605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e55be1ef10 con 0x55e55beff380 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa58d605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e55bf10ea0 con 0x55e55befea60 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa58d605740 10 client.0 _ll_register_callbacks cb 0x55e55bde7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa5897c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf0ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa5897c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf0ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58644/0 (socket says 172.21.15.1:58644) 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa5897c9640 1 -- 172.21.15.1:0/3993184468 learned_addr learned my addr 172.21.15.1:0/3993184468 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:00.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa583fff640 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55beff380 0x55e55bf0b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa588fc8640 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55befea60 0x55e55bf0afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa5897c9640 1 -- 172.21.15.1:0/3993184468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55befea60 msgr2=0x55e55bf0afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa5897c9640 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55befea60 0x55e55bf0afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa5897c9640 1 -- 172.21.15.1:0/3993184468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55beff380 msgr2=0x55e55bf0b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa5897c9640 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55beff380 0x55e55bf0b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa5897c9640 1 -- 172.21.15.1:0/3993184468 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55bbd56a0 con 0x55e55bf0e150 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa588fc8640 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55befea60 0x55e55bf0afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa5897c9640 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf0ba60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00dac0 tx=0x7fa57c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500cc14f2a9ca68b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa563fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.670+0000 7fa563fff640 1 -- 172.21.15.1:0/3993184468 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c010ca0 con 0x55e55bf0e150 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa563fff640 1 -- 172.21.15.1:0/3993184468 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa57c020440 con 0x55e55bf0e150 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa58d605740 1 -- 172.21.15.1:0/3993184468 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5700043c0 con 0x55e55bf0e150 2024-09-16T00:38:00.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa563fff640 1 -- 172.21.15.1:0/3993184468 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa57c032050 con 0x55e55bf0e150 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa58d605740 10 client.17814 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa58d605740 1 -- 172.21.15.1:0/3993184468 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e55bf1c970 con 0x55e55bf0e150 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa563fff640 1 -- 172.21.15.1:0/3993184468 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5700043c0 con 0x55e55bf0e150 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa563fff640 1 -- 172.21.15.1:0/3993184468 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa57c02a5d0 con 0x55e55bf0e150 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa58d605740 10 client.17814 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa58d605740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa58d605740 20 client.17814 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa58d605740 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e55bf1ddd0 0x55e55bf20250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa58d605740 4 client.17814 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa58d605740 1 -- 172.21.15.1:0/3993184468 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e55be717f0 con 0x55e55bf1ddd0 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.671+0000 7fa588fc8640 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e55bf1ddd0 0x55e55bf20250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.672+0000 7fa563fff640 10 client.17814 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.672+0000 7fa588fc8640 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e55bf1ddd0 0x55e55bf20250 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb22ec4d772c34c server_cookie=976198dfb3733793 in_seq=0 out_seq=0 2024-09-16T00:38:00.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa563fff640 1 -- 172.21.15.1:0/3993184468 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e55be717f0 con 0x55e55bf1ddd0 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa563fff640 10 client.17814 handle_command_reply: tid=0 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 client.17814 shutdown 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 -- 172.21.15.1:0/3993184468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 msgr2=0x55e55bf0ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf0ba60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00dac0 tx=0x7fa57c004a40 comp rx=0 tx=0).stop 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 -- 172.21.15.1:0/3993184468 shutdown_connections 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e55bf1ddd0 0x55e55bf20250 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf0ba60 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55beff380 0x55e55bf0b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa563fff640 0 client.17814 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 --2- 172.21.15.1:0/3993184468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55befea60 0x55e55bf0afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 -- 172.21.15.1:0/3993184468 >> 172.21.15.1:0/3993184468 conn(0x55e55befb0f0 msgr2=0x55e55bf00e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 -- 172.21.15.1:0/3993184468 shutdown_connections 2024-09-16T00:38:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 1 -- 172.21.15.1:0/3993184468 wait complete. 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.673+0000 7fa58d605740 20 client.17814 trim_cache size 0 max 16384 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.690+0000 7fa58d605740 1 Processor -- start 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.690+0000 7fa58d605740 1 -- start start 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.690+0000 7fa58d605740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55befea60 0x55e55bf09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.690+0000 7fa58d605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf25b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.690+0000 7fa58d605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55bf0e150 0x55e55bf26060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.690+0000 7fa58d605740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841af5b0 con 0x55e55befea60 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.690+0000 7fa58d605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841ace80 con 0x55e55bf0e150 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.690+0000 7fa58d605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841aacd0 con 0x55e55beff380 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa588fc8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55befea60 0x55e55bf09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa588fc8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55befea60 0x55e55bf09e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58654/0 (socket says 172.21.15.1:58654) 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa588fc8640 1 -- 172.21.15.1:0/559476161 learned_addr learned my addr 172.21.15.1:0/559476161 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa583fff640 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf25b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa5897c9640 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55bf0e150 0x55e55bf26060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa588fc8640 1 -- 172.21.15.1:0/559476161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 msgr2=0x55e55bf25b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa588fc8640 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf25b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa588fc8640 1 -- 172.21.15.1:0/559476161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55bf0e150 msgr2=0x55e55bf26060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa588fc8640 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55bf0e150 0x55e55bf26060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa588fc8640 1 -- 172.21.15.1:0/559476161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55beff1b0 con 0x55e55befea60 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa583fff640 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf25b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa5897c9640 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55bf0e150 0x55e55bf26060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa588fc8640 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55befea60 0x55e55bf09e90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa578000c00 tx=0x7fa5780cd750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b5d12e4e99ae5a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa56e7fc640 1 -- 172.21.15.1:0/559476161 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57809a6a0 con 0x55e55befea60 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa56e7fc640 1 -- 172.21.15.1:0/559476161 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5780cd990 con 0x55e55befea60 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.691+0000 7fa56e7fc640 1 -- 172.21.15.1:0/559476161 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57809b7f0 con 0x55e55befea60 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 -- 172.21.15.1:0/559476161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55befea60 msgr2=0x55e55bf09e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55befea60 0x55e55bf09e90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa578000c00 tx=0x7fa5780cd750 comp rx=0 tx=0).stop 2024-09-16T00:38:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 -- 172.21.15.1:0/559476161 shutdown_connections 2024-09-16T00:38:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55bf0e150 0x55e55bf26060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf25b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 --2- 172.21.15.1:0/559476161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55befea60 0x55e55bf09e90 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 -- 172.21.15.1:0/559476161 >> 172.21.15.1:0/559476161 conn(0x55e55befb0f0 msgr2=0x55e55bf09780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 -- 172.21.15.1:0/559476161 shutdown_connections 2024-09-16T00:38:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 -- 172.21.15.1:0/559476161 wait complete. 2024-09-16T00:38:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 -- start start 2024-09-16T00:38:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.692+0000 7fa58d605740 1 Processor -- start 2024-09-16T00:38:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa58d605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 0x55e55bf08140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa58d605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf08680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa58d605740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf08bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa58d605740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841ab340 con 0x55e55bf0e150 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa58d605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841ac7b0 con 0x55e55befea60 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa58d605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841aba10 con 0x55e55beff380 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa58d605740 10 client.0 _ll_register_callbacks cb 0x7fa57c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa5897c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf08bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa5897c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf08bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58662/0 (socket says 172.21.15.1:58662) 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa5897c9640 1 -- 172.21.15.1:0/2652560401 learned_addr learned my addr 172.21.15.1:0/2652560401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa588fc8640 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 0x55e55bf08140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa583fff640 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf08680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa5897c9640 1 -- 172.21.15.1:0/2652560401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 msgr2=0x55e55bf08680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa5897c9640 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf08680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa5897c9640 1 -- 172.21.15.1:0/2652560401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 msgr2=0x55e55bf08140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa5897c9640 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 0x55e55bf08140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa5897c9640 1 -- 172.21.15.1:0/2652560401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55bf87bf0 con 0x55e55bf0e150 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa5897c9640 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf08bc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00d720 tx=0x7fa57c009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7940764f092e6474 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa563fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa563fff640 1 -- 172.21.15.1:0/2652560401 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c02ac10 con 0x55e55bf0e150 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa563fff640 1 -- 172.21.15.1:0/2652560401 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa57c02adb0 con 0x55e55bf0e150 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.693+0000 7fa58d605740 1 -- 172.21.15.1:0/2652560401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5780cd990 con 0x55e55bf0e150 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa563fff640 1 -- 172.21.15.1:0/2652560401 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa57c044050 con 0x55e55bf0e150 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa58d605740 10 client.17820 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa58d605740 1 -- 172.21.15.1:0/2652560401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e55bf88ea0 con 0x55e55bf0e150 2024-09-16T00:38:00.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa563fff640 1 -- 172.21.15.1:0/2652560401 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5780cd990 con 0x55e55bf0e150 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa563fff640 1 -- 172.21.15.1:0/2652560401 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa57c005320 con 0x55e55bf0e150 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa58d605740 10 client.17820 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa58d605740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa58d605740 20 client.17820 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa58d605740 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e55bf89550 0x55e55bf8b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa58d605740 4 client.17820 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa58d605740 1 -- 172.21.15.1:0/2652560401 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e55be6bd00 con 0x55e55bf89550 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.694+0000 7fa588fc8640 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e55bf89550 0x55e55bf8b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.695+0000 7fa563fff640 10 client.17820 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.695+0000 7fa588fc8640 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e55bf89550 0x55e55bf8b970 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7205aca15cfc4863 server_cookie=660c69a159a59c75 in_seq=0 out_seq=0 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa563fff640 1 -- 172.21.15.1:0/2652560401 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e55be6bd00 con 0x55e55bf89550 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa563fff640 10 client.17820 handle_command_reply: tid=0 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 client.17820 shutdown 2024-09-16T00:38:00.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 -- 172.21.15.1:0/2652560401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 msgr2=0x55e55bf08bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:00.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf08bc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00d720 tx=0x7fa57c009d30 comp rx=0 tx=0).stop 2024-09-16T00:38:00.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 -- 172.21.15.1:0/2652560401 shutdown_connections 2024-09-16T00:38:00.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e55bf89550 0x55e55bf8b970 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e55bf0e150 0x55e55bf08bc0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa563fff640 0 client.17820 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:00.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e55beff380 0x55e55bf08680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 --2- 172.21.15.1:0/2652560401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e55befea60 0x55e55bf08140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 -- 172.21.15.1:0/2652560401 >> 172.21.15.1:0/2652560401 conn(0x55e55befb0f0 msgr2=0x55e55bf0b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:00.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 -- 172.21.15.1:0/2652560401 shutdown_connections 2024-09-16T00:38:00.720 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:38:00.720 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:38:00.720 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "00997f2f-41c6-44cc-ad04-723f9360b824", 2024-09-16T00:38:00.720 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:38:00.720 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:38:00.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 1 -- 172.21.15.1:0/2652560401 wait complete. 2024-09-16T00:38:00.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.696+0000 7fa58d605740 20 client.17820 trim_cache size 0 max 16384 2024-09-16T00:38:00.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 -- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5740a3240 msgr2=0x7fa5740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:00.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5740a3240 0x7fa5740a5700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa57c005c40 tx=0x7fa57c005bd0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 -- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584101de0 msgr2=0x7fa584198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584101de0 0x7fa584198420 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e9f0 tx=0x7fa57800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 -- 172.21.15.1:0/1363212844 shutdown_connections 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa5740a3240 0x7fa5740a5700 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa584106f10 0x7fa58419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa584102790 0x7fa584198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 --2- 172.21.15.1:0/1363212844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa584101de0 0x7fa584198420 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 -- 172.21.15.1:0/1363212844 >> 172.21.15.1:0/1363212844 conn(0x7fa5840fd8c0 msgr2=0x7fa5840fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 -- 172.21.15.1:0/1363212844 shutdown_connections 2024-09-16T00:38:00.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:00.697+0000 7fa58b22c640 1 -- 172.21.15.1:0/1363212844 wait complete. 2024-09-16T00:38:00.722 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:00 smithi153 ceph-mon[35538]: pgmap v638: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-09-16T00:38:01.507 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:01 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:38:01.507 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:01 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:01.507 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:01 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:38:01.507 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:01 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:01.509 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:38:01.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:01 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:38:01.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:01 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:01.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:01 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:38:01.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:01 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:01.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:01 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:38:01.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:01 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:01.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:01 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:38:01.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:01 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:01.772 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:02.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 -- 172.21.15.1:0/4070464839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e8106f10 msgr2=0x7f27e810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 --2- 172.21.15.1:0/4070464839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e8106f10 0x7f27e810ea90 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009ca0 tx=0x7f27d802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 -- 172.21.15.1:0/4070464839 shutdown_connections 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 --2- 172.21.15.1:0/4070464839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e8106f10 0x7f27e810ea90 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 --2- 172.21.15.1:0/4070464839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e8102790 0x7f27e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 --2- 172.21.15.1:0/4070464839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8101de0 0x7f27e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 -- 172.21.15.1:0/4070464839 >> 172.21.15.1:0/4070464839 conn(0x7f27e80fd8c0 msgr2=0x7f27e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 -- 172.21.15.1:0/4070464839 shutdown_connections 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 -- 172.21.15.1:0/4070464839 wait complete. 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.290+0000 7f27ed00e640 1 Processor -- start 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27ed00e640 1 -- start start 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27ed00e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e8101de0 0x7f27e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27ed00e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e8102790 0x7f27e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27ed00e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8106f10 0x7f27e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27ed00e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f27e8111f70 con 0x7f27e8102790 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27ed00e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27e8111df0 con 0x7f27e8101de0 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27ed00e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81120f0 con 0x7f27e8106f10 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27e6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e8101de0 0x7f27e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e8102790 0x7f27e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27e6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e8101de0 0x7f27e80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:38450/0 (socket says 172.21.15.1:38450) 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27e6d9d640 1 -- 172.21.15.1:0/1909828267 learned_addr learned my addr 172.21.15.1:0/1909828267 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.291+0000 7f27e759e640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8106f10 0x7f27e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27e659c640 1 -- 172.21.15.1:0/1909828267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8106f10 msgr2=0x7f27e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27e659c640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8106f10 0x7f27e8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27e659c640 1 -- 172.21.15.1:0/1909828267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e8101de0 msgr2=0x7f27e80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27e659c640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e8101de0 0x7f27e80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27e659c640 1 -- 172.21.15.1:0/1909828267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e81a6890 con 0x7f27e8102790 2024-09-16T00:38:02.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27e6d9d640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e8101de0 0x7f27e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27e659c640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e8102790 0x7f27e8074a30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f27d000e9f0 tx=0x7f27d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27dffff640 1 -- 172.21.15.1:0/1909828267 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d000ccc0 con 0x7f27e8102790 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27ed00e640 1 -- 172.21.15.1:0/1909828267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e81a6b80 con 0x7f27e8102790 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27dffff640 1 -- 172.21.15.1:0/1909828267 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d0010910 con 0x7f27e8102790 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27dffff640 1 -- 172.21.15.1:0/1909828267 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0018960 con 0x7f27e8102790 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.292+0000 7f27ed00e640 1 -- 172.21.15.1:0/1909828267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e81a7110 con 0x7f27e8102790 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.295+0000 7f27dffff640 1 -- 172.21.15.1:0/1909828267 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f27d0010ac0 con 0x7f27e8102790 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.295+0000 7f27ef3e7740 1 Processor -- start 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.295+0000 7f27ef3e7740 1 -- start start 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f48e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 0x56290f499140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f49ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56290f3f42e0 con 0x56290f499680 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56290f39bce0 con 0x56290f48e4a0 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56290f3f02a0 con 0x56290f48edc0 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f49ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f48e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f48e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:38456/0 (socket says 172.21.15.1:38456) 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e759e640 1 -- 172.21.15.1:0/3998845042 learned_addr learned my addr 172.21.15.1:0/3998845042 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e6d9d640 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 0x56290f499140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27dffff640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27d40a32c0 0x7f27d40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e659c640 1 -- 172.21.15.1:0/3998845042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 msgr2=0x56290f499140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e659c640 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 0x56290f499140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e659c640 1 -- 172.21.15.1:0/3998845042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 msgr2=0x56290f48e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e659c640 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f48e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e659c640 1 -- 172.21.15.1:0/3998845042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56290f380440 con 0x56290f499680 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e6d9d640 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 0x56290f499140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e759e640 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f48e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27e759e640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27d40a32c0 0x7f27d40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.296+0000 7f27dffff640 1 -- 172.21.15.1:0/1909828267 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f27d00cffa0 con 0x7f27e8102790 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.297+0000 7f27e659c640 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f49ba70 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f27d00cd670 tx=0x7f27d0024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e69d43ff24a478 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.297+0000 7f27dd7fa640 1 -- 172.21.15.1:0/3998845042 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d009bdb0 con 0x56290f499680 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.297+0000 7f27dd7fa640 1 -- 172.21.15.1:0/3998845042 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d009c270 con 0x56290f499680 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.297+0000 7f27dd7fa640 1 -- 172.21.15.1:0/3998845042 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d001a0d0 con 0x56290f499680 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.297+0000 7f27e759e640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27d40a32c0 0x7f27d40a5780 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f27d802f540 tx=0x7f27d80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 -- 172.21.15.1:0/3998845042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 msgr2=0x56290f49ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f49ba70 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f27d00cd670 tx=0x7f27d0024000 comp rx=0 tx=0).stop 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 -- 172.21.15.1:0/3998845042 shutdown_connections 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f49ba70 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 0x56290f499140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/3998845042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f48e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 -- 172.21.15.1:0/3998845042 >> 172.21.15.1:0/3998845042 conn(0x56290f48aa30 msgr2=0x56290f48ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 -- 172.21.15.1:0/3998845042 shutdown_connections 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 -- 172.21.15.1:0/3998845042 wait complete. 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 -- start start 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.298+0000 7f27ef3e7740 1 Processor -- start 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f498fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 0x56290f496280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56290f49c250 con 0x56290f499680 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56290f3af050 con 0x56290f48e4a0 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56290f49c3d0 con 0x56290f48edc0 2024-09-16T00:38:02.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27ef3e7740 10 client.0 _ll_register_callbacks cb 0x56290f3ef160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f498fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58678/0 (socket says 172.21.15.1:58678) 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e759e640 1 -- 172.21.15.1:0/1792696258 learned_addr learned my addr 172.21.15.1:0/1792696258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e659c640 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 0x56290f496280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e759e640 1 -- 172.21.15.1:0/1792696258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 msgr2=0x56290f496280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e759e640 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 0x56290f496280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e759e640 1 -- 172.21.15.1:0/1792696258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 msgr2=0x56290f498fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e759e640 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f498fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e759e640 1 -- 172.21.15.1:0/1792696258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56290f1653c0 con 0x56290f499680 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27e759e640 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4967e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f27d802ade0 tx=0x7f27d802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb9548341d48a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:02.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.299+0000 7f27ce7fc640 1 -- 172.21.15.1:0/1792696258 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8003e90 con 0x56290f499680 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ce7fc640 1 -- 172.21.15.1:0/1792696258 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f27d80045c0 con 0x56290f499680 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ef3e7740 1 -- 172.21.15.1:0/1792696258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d009c270 con 0x56290f499680 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ce7fc640 1 -- 172.21.15.1:0/1792696258 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27d80047c0 con 0x56290f499680 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ef3e7740 10 client.17832 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ef3e7740 1 -- 172.21.15.1:0/1792696258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56290f4a7c50 con 0x56290f499680 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ce7fc640 1 -- 172.21.15.1:0/1792696258 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d009c270 con 0x56290f499680 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ce7fc640 1 -- 172.21.15.1:0/1792696258 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f27d8051020 con 0x56290f499680 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ef3e7740 10 client.17832 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ef3e7740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ef3e7740 20 client.17832 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56290f4a9060 0x56290f4ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ef3e7740 4 client.17832 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27ef3e7740 1 -- 172.21.15.1:0/1792696258 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56290f401850 con 0x56290f4a9060 2024-09-16T00:38:02.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.300+0000 7f27e6d9d640 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56290f4a9060 0x56290f4ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.301+0000 7f27e6d9d640 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56290f4a9060 0x56290f4ab4b0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e2981ac958df8d server_cookie=b529627e0b5fb970 in_seq=0 out_seq=0 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.301+0000 7f27ce7fc640 10 client.17832 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.301+0000 7f27ce7fc640 1 -- 172.21.15.1:0/1792696258 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56290f401850 con 0x56290f4a9060 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.301+0000 7f27ce7fc640 10 client.17832 handle_command_reply: tid=0 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.301+0000 7f27ef3e7740 1 client.17832 shutdown 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 -- 172.21.15.1:0/1792696258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 msgr2=0x56290f4967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4967e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f27d802ade0 tx=0x7f27d802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 -- 172.21.15.1:0/1792696258 shutdown_connections 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56290f4a9060 0x56290f4ab4b0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4967e0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48edc0 0x56290f496280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/1792696258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48e4a0 0x56290f498fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ce7fc640 0 client.17832 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:02.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 -- 172.21.15.1:0/1792696258 >> 172.21.15.1:0/1792696258 conn(0x56290f48aa30 msgr2=0x56290f490760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:02.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 -- 172.21.15.1:0/1792696258 shutdown_connections 2024-09-16T00:38:02.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 1 -- 172.21.15.1:0/1792696258 wait complete. 2024-09-16T00:38:02.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.302+0000 7f27ef3e7740 20 client.17832 trim_cache size 0 max 16384 2024-09-16T00:38:02.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.319+0000 7f27ef3e7740 1 Processor -- start 2024-09-16T00:38:02.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.319+0000 7f27ef3e7740 1 -- start start 2024-09-16T00:38:02.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 0x56290f4a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48edc0 0x56290f4981f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81b39a0 con 0x56290f499680 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81b1270 con 0x56290f48edc0 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81af0c0 con 0x56290f48e4a0 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58690/0 (socket says 172.21.15.1:58690) 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e759e640 1 -- 172.21.15.1:0/2866210265 learned_addr learned my addr 172.21.15.1:0/2866210265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e6d9d640 1 --2- 172.21.15.1:0/2866210265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 0x56290f4a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e659c640 1 --2- 172.21.15.1:0/2866210265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48edc0 0x56290f4981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e759e640 1 -- 172.21.15.1:0/2866210265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 msgr2=0x56290f4a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e759e640 1 --2- 172.21.15.1:0/2866210265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 0x56290f4a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e759e640 1 -- 172.21.15.1:0/2866210265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48edc0 msgr2=0x56290f4981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e759e640 1 --2- 172.21.15.1:0/2866210265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48edc0 0x56290f4981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.320+0000 7f27e759e640 1 -- 172.21.15.1:0/2866210265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56290f48bbc0 con 0x56290f499680 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.321+0000 7f27e759e640 1 --2- 172.21.15.1:0/2866210265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4b0d80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f27d80308b0 tx=0x7f27d8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e237b73ee6e3adbe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.321+0000 7f27dcff9640 1 -- 172.21.15.1:0/2866210265 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80318e0 con 0x56290f499680 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.321+0000 7f27dcff9640 1 -- 172.21.15.1:0/2866210265 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d8036040 con 0x56290f499680 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.321+0000 7f27dcff9640 1 -- 172.21.15.1:0/2866210265 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d802f870 con 0x56290f499680 2024-09-16T00:38:02.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 -- 172.21.15.1:0/2866210265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 msgr2=0x56290f4b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/2866210265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4b0d80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f27d80308b0 tx=0x7f27d8003f90 comp rx=0 tx=0).stop 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 -- 172.21.15.1:0/2866210265 shutdown_connections 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/2866210265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f499680 0x56290f4b0d80 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/2866210265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f48edc0 0x56290f4981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/2866210265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 0x56290f4a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 -- 172.21.15.1:0/2866210265 >> 172.21.15.1:0/2866210265 conn(0x56290f48aa30 msgr2=0x56290f497ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 -- 172.21.15.1:0/2866210265 shutdown_connections 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 -- 172.21.15.1:0/2866210265 wait complete. 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 -- start start 2024-09-16T00:38:02.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 Processor -- start 2024-09-16T00:38:02.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 0x56290f49dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f48edc0 0x56290f49e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f499680 0x56290f512d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.322+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81af730 con 0x56290f48edc0 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81b0ba0 con 0x56290f499680 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ef3e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81afe00 con 0x56290f48e4a0 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ef3e7740 10 client.0 _ll_register_callbacks cb 0x7f27d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f48edc0 0x56290f49e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f48edc0 0x56290f49e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58694/0 (socket says 172.21.15.1:58694) 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e659c640 1 -- 172.21.15.1:0/971474422 learned_addr learned my addr 172.21.15.1:0/971474422 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e759e640 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f499680 0x56290f512d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e6d9d640 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 0x56290f49dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e659c640 1 -- 172.21.15.1:0/971474422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 msgr2=0x56290f49dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e659c640 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 0x56290f49dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e659c640 1 -- 172.21.15.1:0/971474422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f499680 msgr2=0x56290f512d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e659c640 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f499680 0x56290f512d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e659c640 1 -- 172.21.15.1:0/971474422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56290f513390 con 0x56290f48edc0 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e759e640 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f499680 0x56290f512d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27e659c640 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f48edc0 0x56290f49e150 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f27d00cd6a0 tx=0x7f27d00d02a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896bbd8b5b258554 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ce7fc640 1 -- 172.21.15.1:0/971474422 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0013190 con 0x56290f48edc0 2024-09-16T00:38:02.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ce7fc640 1 -- 172.21.15.1:0/971474422 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f27d009a310 con 0x56290f48edc0 2024-09-16T00:38:02.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ef3e7740 1 -- 172.21.15.1:0/971474422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d8036040 con 0x56290f48edc0 2024-09-16T00:38:02.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ce7fc640 1 -- 172.21.15.1:0/971474422 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27d009bbd0 con 0x56290f48edc0 2024-09-16T00:38:02.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ef3e7740 10 client.17838 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:02.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.323+0000 7f27ef3e7740 1 -- 172.21.15.1:0/971474422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56290f514680 con 0x56290f48edc0 2024-09-16T00:38:02.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.324+0000 7f27ce7fc640 1 -- 172.21.15.1:0/971474422 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d8036040 con 0x56290f48edc0 2024-09-16T00:38:02.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.324+0000 7f27ce7fc640 1 -- 172.21.15.1:0/971474422 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f27d00d3710 con 0x56290f48edc0 2024-09-16T00:38:02.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.324+0000 7f27ef3e7740 10 client.17838 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:02.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.324+0000 7f27ef3e7740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:02.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.324+0000 7f27ef3e7740 20 client.17838 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.324+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56290f5149b0 0x56290f516da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.324+0000 7f27ef3e7740 4 client.17838 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.324+0000 7f27ef3e7740 1 -- 172.21.15.1:0/971474422 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56290f3fbd60 con 0x56290f5149b0 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.324+0000 7f27e6d9d640 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56290f5149b0 0x56290f516da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27e6d9d640 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56290f5149b0 0x56290f516da0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120e35ebfdb3ae64 server_cookie=57d54de80a879dc1 in_seq=0 out_seq=0 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ce7fc640 10 client.17838 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ce7fc640 1 -- 172.21.15.1:0/971474422 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56290f3fbd60 con 0x56290f5149b0 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ce7fc640 10 client.17838 handle_command_reply: tid=0 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 client.17838 shutdown 2024-09-16T00:38:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 -- 172.21.15.1:0/971474422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f48edc0 msgr2=0x56290f49e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f48edc0 0x56290f49e150 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f27d00cd6a0 tx=0x7f27d00d02a0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 -- 172.21.15.1:0/971474422 shutdown_connections 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56290f5149b0 0x56290f516da0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56290f499680 0x56290f512d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56290f48edc0 0x56290f49e150 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 --2- 172.21.15.1:0/971474422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56290f48e4a0 0x56290f49dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 -- 172.21.15.1:0/971474422 >> 172.21.15.1:0/971474422 conn(0x56290f48aa30 msgr2=0x56290f490710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ce7fc640 0 client.17838 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 -- 172.21.15.1:0/971474422 shutdown_connections 2024-09-16T00:38:02.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 1 -- 172.21.15.1:0/971474422 wait complete. 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.325+0000 7f27ef3e7740 20 client.17838 trim_cache size 0 max 16384 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "00997f2f-41c6-44cc-ad04-723f9360b824": { 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "00997f2f-41c6-44cc-ad04-723f9360b824", 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:38:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:38:02.336 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:38:02.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 -- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27d40a32c0 msgr2=0x7f27d40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27d40a32c0 0x7f27d40a5780 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f27d802f540 tx=0x7f27d80095c0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 -- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e8102790 msgr2=0x7f27e8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e8102790 0x7f27e8074a30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f27d000e9f0 tx=0x7f27d000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 -- 172.21.15.1:0/1909828267 shutdown_connections 2024-09-16T00:38:02.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f27d40a32c0 0x7f27d40a5780 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8106f10 0x7f27e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e8102790 0x7f27e8074a30 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 --2- 172.21.15.1:0/1909828267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e8101de0 0x7f27e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 -- 172.21.15.1:0/1909828267 >> 172.21.15.1:0/1909828267 conn(0x7f27e80fd8c0 msgr2=0x7f27e80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:02.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 -- 172.21.15.1:0/1909828267 shutdown_connections 2024-09-16T00:38:02.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:02.326+0000 7f27ed00e640 1 -- 172.21.15.1:0/1909828267 wait complete. 2024-09-16T00:38:02.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:02 smithi019 ceph-mon[35186]: pgmap v639: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-16T00:38:02.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:02 smithi001 ceph-mon[30934]: pgmap v639: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-16T00:38:02.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:02 smithi153 ceph-mon[35538]: pgmap v639: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-16T00:38:03.222 INFO:tasks.cephfs.filesystem:scrub status for tag:00997f2f-41c6-44cc-ad04-723f9360b824 - {'path': '/', 'tag': '00997f2f-41c6-44cc-ad04-723f9360b824', 'options': 'recursive,force'} 2024-09-16T00:38:03.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:03 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:38:03.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:03 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:38:03.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:03 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:38:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:04 smithi019 ceph-mon[35186]: pgmap v640: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-16T00:38:04.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:04 smithi001 ceph-mon[30934]: pgmap v640: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-16T00:38:04.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:04 smithi153 ceph-mon[35538]: pgmap v640: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-16T00:38:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:06 smithi019 ceph-mon[35186]: pgmap v641: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-09-16T00:38:06.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:06 smithi001 ceph-mon[30934]: pgmap v641: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-09-16T00:38:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:06 smithi153 ceph-mon[35538]: pgmap v641: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-09-16T00:38:08.514 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:08 smithi153 ceph-mon[35538]: pgmap v642: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-16T00:38:08.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:08 smithi019 ceph-mon[35186]: pgmap v642: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-16T00:38:08.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:08 smithi001 ceph-mon[30934]: pgmap v642: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-16T00:38:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:10 smithi019 ceph-mon[35186]: pgmap v643: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-16T00:38:10.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:10 smithi001 ceph-mon[30934]: pgmap v643: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-16T00:38:10.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:10 smithi153 ceph-mon[35538]: pgmap v643: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-16T00:38:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:12 smithi019 ceph-mon[35186]: pgmap v644: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-16T00:38:12.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:12 smithi001 ceph-mon[30934]: pgmap v644: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-16T00:38:12.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:12 smithi153 ceph-mon[35538]: pgmap v644: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-16T00:38:14.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:14 smithi019 ceph-mon[35186]: pgmap v645: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-09-16T00:38:14.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:14 smithi001 ceph-mon[30934]: pgmap v645: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-09-16T00:38:14.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:14 smithi153 ceph-mon[35538]: pgmap v645: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-09-16T00:38:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:16 smithi019 ceph-mon[35186]: pgmap v646: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:38:16.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:16 smithi001 ceph-mon[30934]: pgmap v646: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:38:16.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:16 smithi153 ceph-mon[35538]: pgmap v646: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:38:16.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:17.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.665+0000 7fcb577a5640 1 -- 172.21.15.1:0/1114511953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 msgr2=0x7fcb501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:17.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.665+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1114511953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 0x7fcb501067e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009900 tx=0x7fcb4002a410 comp rx=0 tx=0).stop 2024-09-16T00:38:17.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.665+0000 7fcb577a5640 1 -- 172.21.15.1:0/1114511953 shutdown_connections 2024-09-16T00:38:17.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.665+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1114511953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50106f10 0x7fcb5010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.665+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1114511953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 0x7fcb501067e0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.665+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1114511953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb50101de0 0x7fcb501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.665+0000 7fcb577a5640 1 -- 172.21.15.1:0/1114511953 >> 172.21.15.1:0/1114511953 conn(0x7fcb500fd8c0 msgr2=0x7fcb500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.665+0000 7fcb577a5640 1 -- 172.21.15.1:0/1114511953 shutdown_connections 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.665+0000 7fcb577a5640 1 -- 172.21.15.1:0/1114511953 wait complete. 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb577a5640 1 Processor -- start 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb577a5640 1 -- start start 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb577a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50101de0 0x7fcb500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb577a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 0x7fcb50074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb577a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb50106f10 0x7fcb50074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb577a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111f70 con 0x7fcb50102790 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb577a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111df0 con 0x7fcb50106f10 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb577a5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501120f0 con 0x7fcb50101de0 2024-09-16T00:38:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb54d40640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 0x7fcb50074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb54d40640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 0x7fcb50074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42496/0 (socket says 172.21.15.1:42496) 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb54d40640 1 -- 172.21.15.1:0/1676599568 learned_addr learned my addr 172.21.15.1:0/1676599568 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb55541640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50101de0 0x7fcb500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb55d42640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb50106f10 0x7fcb50074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb54d40640 1 -- 172.21.15.1:0/1676599568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50101de0 msgr2=0x7fcb500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb54d40640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50101de0 0x7fcb500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.666+0000 7fcb54d40640 1 -- 172.21.15.1:0/1676599568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb50106f10 msgr2=0x7fcb50074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.667+0000 7fcb54d40640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb50106f10 0x7fcb50074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.667+0000 7fcb54d40640 1 -- 172.21.15.1:0/1676599568 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb501a6900 con 0x7fcb50102790 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.667+0000 7fcb54d40640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 0x7fcb50074a30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fcb40006010 tx=0x7fcb400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.667+0000 7fcb467fc640 1 -- 172.21.15.1:0/1676599568 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4003f030 con 0x7fcb50102790 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.667+0000 7fcb467fc640 1 -- 172.21.15.1:0/1676599568 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb40003ce0 con 0x7fcb50102790 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.667+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb501a6b90 con 0x7fcb50102790 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.667+0000 7fcb467fc640 1 -- 172.21.15.1:0/1676599568 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40035070 con 0x7fcb50102790 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.667+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb501a7070 con 0x7fcb50102790 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.668+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb501021c0 con 0x7fcb50102790 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.669+0000 7fcb467fc640 1 -- 172.21.15.1:0/1676599568 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fcb40042030 con 0x7fcb50102790 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.670+0000 7fcb467fc640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb3c0a3240 0x7fcb3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.670+0000 7fcb55541640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb3c0a3240 0x7fcb3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.670+0000 7fcb467fc640 1 -- 172.21.15.1:0/1676599568 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fcb400f0220 con 0x7fcb50102790 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.671+0000 7fcb55541640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb3c0a3240 0x7fcb3c0a5700 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0058c0 tx=0x7fcb4c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:17.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.675+0000 7fcb467fc640 1 -- 172.21.15.1:0/1676599568 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb400bcc00 con 0x7fcb50102790 2024-09-16T00:38:17.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.887+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb50109b60 con 0x7fcb50102790 2024-09-16T00:38:17.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.887+0000 7fcb467fc640 1 -- 172.21.15.1:0/1676599568 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fcb400bc500 con 0x7fcb50102790 2024-09-16T00:38:17.889 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:38:17.889 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:38:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:38:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb3c0a3240 msgr2=0x7fcb3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb3c0a3240 0x7fcb3c0a5700 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0058c0 tx=0x7fcb4c00a2d0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 msgr2=0x7fcb50074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 0x7fcb50074a30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fcb40006010 tx=0x7fcb400042f0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 shutdown_connections 2024-09-16T00:38:17.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb50106f10 0x7fcb50074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb50102790 0x7fcb50074a30 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fcb3c0a3240 0x7fcb3c0a5700 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 --2- 172.21.15.1:0/1676599568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50101de0 0x7fcb500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 >> 172.21.15.1:0/1676599568 conn(0x7fcb500fd8c0 msgr2=0x7fcb500fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:17.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 shutdown_connections 2024-09-16T00:38:17.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:17.890+0000 7fcb577a5640 1 -- 172.21.15.1:0/1676599568 wait complete. 2024-09-16T00:38:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:18 smithi019 ceph-mon[35186]: pgmap v647: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:38:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:18 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1676599568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:18.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:18 smithi001 ceph-mon[30934]: pgmap v647: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:38:18.656 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:18 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1676599568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:18.689 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 328fd998 --release --await' 2024-09-16T00:38:18.689 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 328fd998 --release --await 2024-09-16T00:38:18.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:18 smithi153 ceph-mon[35538]: pgmap v647: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:38:18.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:18 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1676599568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:18.951 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 -- 172.21.15.1:0/1006779998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8102d30 msgr2=0x7f3fb8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/1006779998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8102d30 0x7f3fb8103110 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009930 tx=0x7f3fa802a420 comp rx=0 tx=0).stop 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 -- 172.21.15.1:0/1006779998 shutdown_connections 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/1006779998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb81082a0 0x7f3fb810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/1006779998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb81036e0 0x7f3fb8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/1006779998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8102d30 0x7f3fb8103110 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 -- 172.21.15.1:0/1006779998 >> 172.21.15.1:0/1006779998 conn(0x7f3fb80fc7d0 msgr2=0x7f3fb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 -- 172.21.15.1:0/1006779998 shutdown_connections 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 -- 172.21.15.1:0/1006779998 wait complete. 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.473+0000 7f3fbe73e640 1 Processor -- start 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fbe73e640 1 -- start start 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fbe73e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb8102d30 0x7f3fb81a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fbe73e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb81036e0 0x7f3fb81a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fbe73e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb81082a0 0x7f3fb819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fbe73e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb8112120 con 0x7f3fb81036e0 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fbe73e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb8111fa0 con 0x7f3fb81082a0 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fbe73e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81122a0 con 0x7f3fb8102d30 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3faffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb81036e0 0x7f3fb81a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3faffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb81036e0 0x7f3fb81a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42516/0 (socket says 172.21.15.1:42516) 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3faffff640 1 -- 172.21.15.1:0/2174755810 learned_addr learned my addr 172.21.15.1:0/2174755810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb8102d30 0x7f3fb81a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb81082a0 0x7f3fb819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3faffff640 1 -- 172.21.15.1:0/2174755810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb8102d30 msgr2=0x7f3fb81a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3faffff640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb8102d30 0x7f3fb81a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3faffff640 1 -- 172.21.15.1:0/2174755810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb81082a0 msgr2=0x7f3fb819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3faffff640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb81082a0 0x7f3fb819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3faffff640 1 -- 172.21.15.1:0/2174755810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb819aea0 con 0x7f3fb81036e0 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3faffff640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb81036e0 0x7f3fb81a0aa0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000d750 tx=0x7f3fa000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.474+0000 7f3fb5ffb640 1 -- 172.21.15.1:0/2174755810 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0017070 con 0x7f3fb81036e0 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.475+0000 7f3fb5ffb640 1 -- 172.21.15.1:0/2174755810 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa0005500 con 0x7f3fb81036e0 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.475+0000 7f3fbe73e640 1 -- 172.21.15.1:0/2174755810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb819b190 con 0x7f3fb81036e0 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.475+0000 7f3fb5ffb640 1 -- 172.21.15.1:0/2174755810 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0004e00 con 0x7f3fb81036e0 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.475+0000 7f3fbe73e640 1 -- 172.21.15.1:0/2174755810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb81a7310 con 0x7f3fb81036e0 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.477+0000 7f3fb5ffb640 1 -- 172.21.15.1:0/2174755810 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3fa0004fa0 con 0x7f3fb81036e0 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.477+0000 7f3fc0b17740 1 Processor -- start 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.477+0000 7f3fc0b17740 1 -- start start 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.477+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 0x560caa54d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.477+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 0x560caa557d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.477+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa55a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.477+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560caa4b2620 con 0x560caa558240 2024-09-16T00:38:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.477+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560caa459e50 con 0x560caa54d980 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.477+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560caa4ae5e0 con 0x560caa54d060 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3faffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa55a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 0x560caa557d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3faffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa55a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42528/0 (socket says 172.21.15.1:42528) 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3faffff640 1 -- 172.21.15.1:0/4261184390 learned_addr learned my addr 172.21.15.1:0/4261184390 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/4261184390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 0x560caa54d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fb5ffb640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fa40a3240 0x7f3fa40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fa40a3240 0x7f3fa40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3faffff640 1 -- 172.21.15.1:0/4261184390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 msgr2=0x560caa54d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3faffff640 1 --2- 172.21.15.1:0/4261184390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 0x560caa54d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3faffff640 1 -- 172.21.15.1:0/4261184390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 msgr2=0x560caa557d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3faffff640 1 --2- 172.21.15.1:0/4261184390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 0x560caa557d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fb5ffb640 1 -- 172.21.15.1:0/2174755810 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3fa00cf780 con 0x7f3fb81036e0 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3faffff640 1 -- 172.21.15.1:0/4261184390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560caa44ad40 con 0x560caa558240 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/4261184390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 0x560caa557d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3faffff640 1 --2- 172.21.15.1:0/4261184390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa55a630 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3fa009b6a0 tx=0x7f3fa0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb8bd920c6bf412 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fa40a3240 0x7f3fa40a5700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009fd0 tx=0x7f3fb0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fae7fc640 1 -- 172.21.15.1:0/4261184390 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0020030 con 0x560caa558240 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fae7fc640 1 -- 172.21.15.1:0/4261184390 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa00046c0 con 0x560caa558240 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.478+0000 7f3fae7fc640 1 -- 172.21.15.1:0/4261184390 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0005290 con 0x560caa558240 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 -- 172.21.15.1:0/4261184390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 msgr2=0x560caa55a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/4261184390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa55a630 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3fa009b6a0 tx=0x7f3fa0009c30 comp rx=0 tx=0).stop 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 -- 172.21.15.1:0/4261184390 shutdown_connections 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/4261184390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa55a630 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/4261184390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 0x560caa557d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/4261184390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 0x560caa54d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 -- 172.21.15.1:0/4261184390 >> 172.21.15.1:0/4261184390 conn(0x560caa5495f0 msgr2=0x560caa5499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 -- 172.21.15.1:0/4261184390 shutdown_connections 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 -- 172.21.15.1:0/4261184390 wait complete. 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.479+0000 7f3fc0b17740 1 -- start start 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fc0b17740 1 Processor -- start 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 0x560caa5550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 0x560caa555630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa555b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560caa55ae10 con 0x560caa558240 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560caa43af10 con 0x560caa54d980 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560caa55af90 con 0x560caa54d060 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fc0b17740 10 client.0 _ll_register_callbacks cb 0x560caa45edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fbccdb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa555b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fbccdb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa555b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42532/0 (socket says 172.21.15.1:42532) 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fbccdb640 1 -- 172.21.15.1:0/1394058056 learned_addr learned my addr 172.21.15.1:0/1394058056 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3faffff640 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 0x560caa555630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.480+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 0x560caa5550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fbccdb640 1 -- 172.21.15.1:0/1394058056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 msgr2=0x560caa5550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 0x560caa5550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fbccdb640 1 -- 172.21.15.1:0/1394058056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 msgr2=0x560caa555630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 0x560caa555630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fbccdb640 1 -- 172.21.15.1:0/1394058056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560caa1c1580 con 0x560caa558240 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 0x560caa5550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa555b70 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0007cd0 tx=0x7f3fb0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41b29d856475c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3f977fe640 1 -- 172.21.15.1:0/1394058056 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0020070 con 0x560caa558240 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3f977fe640 1 -- 172.21.15.1:0/1394058056 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3fb0025660 con 0x560caa558240 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fc0b17740 1 -- 172.21.15.1:0/1394058056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560caa4bfb90 con 0x560caa558240 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3f977fe640 1 -- 172.21.15.1:0/1394058056 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3fb002e050 con 0x560caa558240 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fc0b17740 10 client.17856 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3fc0b17740 1 -- 172.21.15.1:0/1394058056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560caa28b960 con 0x560caa558240 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.481+0000 7f3f977fe640 1 -- 172.21.15.1:0/1394058056 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560caa4bfb90 con 0x560caa558240 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3f977fe640 1 -- 172.21.15.1:0/1394058056 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3fb001b390 con 0x560caa558240 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3fc0b17740 10 client.17856 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3fc0b17740 10 client.17856 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3fc0b17740 20 client.17856 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560caa567dc0 0x560caa56a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3fc0b17740 4 client.17856 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:19.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3fc0b17740 1 -- 172.21.15.1:0/1394058056 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560caa4ba0a0 con 0x560caa567dc0 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560caa567dc0 0x560caa56a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3f977fe640 10 client.17856 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.482+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560caa567dc0 0x560caa56a210 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c18550f129a671 server_cookie=78cc4757739029f2 in_seq=0 out_seq=0 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.483+0000 7f3f977fe640 1 -- 172.21.15.1:0/1394058056 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560caa4ba0a0 con 0x560caa567dc0 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.483+0000 7f3f977fe640 10 client.17856 handle_command_reply: tid=0 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.483+0000 7f3fc0b17740 1 client.17856 shutdown 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.483+0000 7f3fc0b17740 1 -- 172.21.15.1:0/1394058056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 msgr2=0x560caa555b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.483+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa555b70 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0007cd0 tx=0x7f3fb0004b50 comp rx=0 tx=0).stop 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3fc0b17740 1 -- 172.21.15.1:0/1394058056 shutdown_connections 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560caa567dc0 0x560caa56a210 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa558240 0x560caa555b70 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d980 0x560caa555630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3f977fe640 0 client.17856 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/1394058056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d060 0x560caa5550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3fc0b17740 1 -- 172.21.15.1:0/1394058056 >> 172.21.15.1:0/1394058056 conn(0x560caa5495f0 msgr2=0x560caa54f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:19.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3fc0b17740 1 -- 172.21.15.1:0/1394058056 shutdown_connections 2024-09-16T00:38:19.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3fc0b17740 1 -- 172.21.15.1:0/1394058056 wait complete. 2024-09-16T00:38:19.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.484+0000 7f3fc0b17740 20 client.17856 trim_cache size 0 max 16384 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fc0b17740 1 Processor -- start 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fc0b17740 1 -- start start 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d060 0x560caa56fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d980 0x560caa570020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa558240 0x560caa552350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81b39e0 con 0x560caa54d060 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81b12b0 con 0x560caa558240 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81af100 con 0x560caa54d980 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fb7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d060 0x560caa56fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fb7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d060 0x560caa56fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42540/0 (socket says 172.21.15.1:42540) 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.501+0000 7f3fb7fff640 1 -- 172.21.15.1:0/634894334 learned_addr learned my addr 172.21.15.1:0/634894334 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa558240 0x560caa552350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fb7fff640 1 -- 172.21.15.1:0/634894334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d980 msgr2=0x560caa570020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3faffff640 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d980 0x560caa570020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d980 0x560caa570020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fb7fff640 1 -- 172.21.15.1:0/634894334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa558240 msgr2=0x560caa552350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa558240 0x560caa552350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fb7fff640 1 -- 172.21.15.1:0/634894334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560caa549420 con 0x560caa54d060 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa558240 0x560caa552350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d060 0x560caa56fae0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f3fa802a940 tx=0x7f3fa8032cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1caccf62ba7cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fadffb640 1 -- 172.21.15.1:0/634894334 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8035040 con 0x560caa54d060 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fadffb640 1 -- 172.21.15.1:0/634894334 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa8049070 con 0x560caa54d060 2024-09-16T00:38:19.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.502+0000 7f3fadffb640 1 -- 172.21.15.1:0/634894334 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8004920 con 0x560caa54d060 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 -- 172.21.15.1:0/634894334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d060 msgr2=0x560caa56fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d060 0x560caa56fae0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f3fa802a940 tx=0x7f3fa8032cf0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 -- 172.21.15.1:0/634894334 shutdown_connections 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa558240 0x560caa552350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa54d980 0x560caa570020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/634894334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d060 0x560caa56fae0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 -- 172.21.15.1:0/634894334 >> 172.21.15.1:0/634894334 conn(0x560caa5495f0 msgr2=0x560caa557170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 -- 172.21.15.1:0/634894334 shutdown_connections 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 -- 172.21.15.1:0/634894334 wait complete. 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.503+0000 7f3fc0b17740 1 -- start start 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fc0b17740 1 Processor -- start 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d060 0x560caa55c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d980 0x560caa55c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fc0b17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa558240 0x560caa5d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81af770 con 0x560caa54d980 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81b0be0 con 0x560caa54d060 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fc0b17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81afe40 con 0x560caa558240 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fc0b17740 10 client.0 _ll_register_callbacks cb 0x7f3fb0007980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3faffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d980 0x560caa55c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3faffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d980 0x560caa55c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42552/0 (socket says 172.21.15.1:42552) 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3faffff640 1 -- 172.21.15.1:0/2102390467 learned_addr learned my addr 172.21.15.1:0/2102390467 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d060 0x560caa55c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa558240 0x560caa5d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3faffff640 1 -- 172.21.15.1:0/2102390467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa558240 msgr2=0x560caa5d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3faffff640 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa558240 0x560caa5d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3faffff640 1 -- 172.21.15.1:0/2102390467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d060 msgr2=0x560caa55c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3faffff640 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d060 0x560caa55c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3faffff640 1 -- 172.21.15.1:0/2102390467 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560caa55d080 con 0x560caa54d980 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.504+0000 7f3fbccdb640 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa558240 0x560caa5d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3faffff640 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d980 0x560caa55c7f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3fa009b930 tx=0x7f3fa00a4610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7fbbe63a16c735 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3f977fe640 1 -- 172.21.15.1:0/2102390467 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa009fd50 con 0x560caa54d980 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3f977fe640 1 -- 172.21.15.1:0/2102390467 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3fa00860b0 con 0x560caa54d980 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3fc0b17740 1 -- 172.21.15.1:0/2102390467 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fa8049070 con 0x560caa54d980 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3f977fe640 1 -- 172.21.15.1:0/2102390467 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3fa00b1080 con 0x560caa54d980 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3fc0b17740 10 client.17862 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3fc0b17740 1 -- 172.21.15.1:0/2102390467 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560caa5d3130 con 0x560caa54d980 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3f977fe640 1 -- 172.21.15.1:0/2102390467 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa8049070 con 0x560caa54d980 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3f977fe640 1 -- 172.21.15.1:0/2102390467 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3fa0098010 con 0x560caa54d980 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3fc0b17740 10 client.17862 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3fc0b17740 10 client.17862 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:38:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3fc0b17740 20 client.17862 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560caa5d3460 0x560caa5d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3fc0b17740 4 client.17862 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "328fd998", "release": true, "await": true}] 2024-09-16T00:38:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.505+0000 7f3fc0b17740 1 -- 172.21.15.1:0/2102390467 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "328fd998", "release": true, "await": true}) -- 0x560caa544950 con 0x560caa5d3460 2024-09-16T00:38:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.506+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560caa5d3460 0x560caa5d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.506+0000 7f3f977fe640 10 client.17862 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.506+0000 7f3fb7fff640 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560caa5d3460 0x560caa5d5850 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58fa6bc1dd88cdf2 server_cookie=41b57ff95ef4b2f9 in_seq=0 out_seq=0 2024-09-16T00:38:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.508+0000 7f3f977fe640 1 -- 172.21.15.1:0/2102390467 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560caa544950 con 0x560caa5d3460 2024-09-16T00:38:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.508+0000 7f3f977fe640 10 client.17862 handle_command_reply: tid=0 2024-09-16T00:38:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.508+0000 7f3fc0b17740 1 client.17862 shutdown 2024-09-16T00:38:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 -- 172.21.15.1:0/2102390467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d980 msgr2=0x560caa55c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d980 0x560caa55c7f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3fa009b930 tx=0x7f3fa00a4610 comp rx=0 tx=0).stop 2024-09-16T00:38:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 -- 172.21.15.1:0/2102390467 shutdown_connections 2024-09-16T00:38:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560caa5d3460 0x560caa5d5850 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560caa558240 0x560caa5d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560caa54d980 0x560caa55c7f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3f977fe640 0 client.17862 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 --2- 172.21.15.1:0/2102390467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560caa54d060 0x560caa55c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 -- 172.21.15.1:0/2102390467 >> 172.21.15.1:0/2102390467 conn(0x560caa5495f0 msgr2=0x560caa5550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 -- 172.21.15.1:0/2102390467 shutdown_connections 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 1 -- 172.21.15.1:0/2102390467 wait complete. 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fc0b17740 20 client.17862 trim_cache size 0 max 16384 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:38:19.514 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 36, 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "328fd998": { 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 36, 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 142.3, 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:38:19.515 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:38:19.516 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:38:19.516 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:38:19.516 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:38:19.516 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:38:19.516 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:38:19.516 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:38:19.516 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:38:19.516 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:38:19.516 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 -- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fa40a3240 msgr2=0x7f3fa40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fa40a3240 0x7f3fa40a5700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009fd0 tx=0x7f3fb0004a40 comp rx=0 tx=0).stop 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 -- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb81036e0 msgr2=0x7f3fb81a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb81036e0 0x7f3fb81a0aa0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000d750 tx=0x7f3fa000dc10 comp rx=0 tx=0).stop 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 -- 172.21.15.1:0/2174755810 shutdown_connections 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb81082a0 0x7f3fb819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb81036e0 0x7f3fb81a0aa0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3fa40a3240 0x7f3fa40a5700 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 --2- 172.21.15.1:0/2174755810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb8102d30 0x7f3fb81a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.509+0000 7f3fbe73e640 1 -- 172.21.15.1:0/2174755810 >> 172.21.15.1:0/2174755810 conn(0x7f3fb80fc7d0 msgr2=0x7f3fb80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:19.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.510+0000 7f3fbe73e640 1 -- 172.21.15.1:0/2174755810 shutdown_connections 2024-09-16T00:38:19.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:19.510+0000 7f3fbe73e640 1 -- 172.21.15.1:0/2174755810 wait complete. 2024-09-16T00:38:20.240 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '328fd998', total seconds elapsed: 25.7 2024-09-16T00:38:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:20 smithi019 ceph-mon[35186]: pgmap v648: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:38:20.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:20 smithi001 ceph-mon[30934]: pgmap v648: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:38:20.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:20 smithi153 ceph-mon[35538]: pgmap v648: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:38:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:22 smithi019 ceph-mon[35186]: pgmap v649: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:38:22.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:22 smithi001 ceph-mon[30934]: pgmap v649: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:38:22.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:22 smithi153 ceph-mon[35538]: pgmap v649: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:38:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:24 smithi019 ceph-mon[35186]: pgmap v650: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:38:24.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:24 smithi001 ceph-mon[30934]: pgmap v650: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:38:24.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:24 smithi153 ceph-mon[35538]: pgmap v650: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:38:26.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:26 smithi001 ceph-mon[30934]: pgmap v651: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-16T00:38:26.536 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:26 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:26.545 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:26 smithi153 ceph-mon[35538]: pgmap v651: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-16T00:38:26.545 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:26 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:26 smithi019 ceph-mon[35186]: pgmap v651: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-16T00:38:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:26 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:28.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: pgmap v652: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:38:28.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:28.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: pgmap v652: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:38:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:28.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: pgmap v652: 129 pgs: 129 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:38:28.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:28.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:28.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:28.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:38:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:30 smithi019 ceph-mon[35186]: pgmap v653: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-09-16T00:38:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:30 smithi001 ceph-mon[30934]: pgmap v653: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-09-16T00:38:30.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:30 smithi153 ceph-mon[35538]: pgmap v653: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-09-16T00:38:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:32 smithi019 ceph-mon[35186]: pgmap v654: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-16T00:38:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:32 smithi001 ceph-mon[30934]: pgmap v654: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-16T00:38:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:32 smithi153 ceph-mon[35538]: pgmap v654: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-16T00:38:33.223 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:38:33.497 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 -- 172.21.15.1:0/510478335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec102d40 msgr2=0x7f86ec103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 --2- 172.21.15.1:0/510478335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec102d40 0x7f86ec103120 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009900 tx=0x7f86e002a480 comp rx=0 tx=0).stop 2024-09-16T00:38:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 -- 172.21.15.1:0/510478335 shutdown_connections 2024-09-16T00:38:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 --2- 172.21.15.1:0/510478335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86ec10c170 0x7f86ec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 --2- 172.21.15.1:0/510478335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86ec103660 0x7f86ec10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 --2- 172.21.15.1:0/510478335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec102d40 0x7f86ec103120 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 -- 172.21.15.1:0/510478335 >> 172.21.15.1:0/510478335 conn(0x7f86ec0fc7a0 msgr2=0x7f86ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 -- 172.21.15.1:0/510478335 shutdown_connections 2024-09-16T00:38:33.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 -- 172.21.15.1:0/510478335 wait complete. 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 Processor -- start 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.984+0000 7f86f39cf640 1 -- start start 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f39cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86ec102d40 0x7f86ec198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f39cf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86ec103660 0x7f86ec198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f39cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec10c170 0x7f86ec19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f39cf640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec111fc0 con 0x7f86ec10c170 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f39cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec111e40 con 0x7f86ec102d40 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f39cf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec112140 con 0x7f86ec103660 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f1f6c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec10c170 0x7f86ec19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f1f6c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec10c170 0x7f86ec19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43724/0 (socket says 172.21.15.1:43724) 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f1f6c640 1 -- 172.21.15.1:0/2652584453 learned_addr learned my addr 172.21.15.1:0/2652584453 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f176b640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86ec102d40 0x7f86ec198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86ec103660 0x7f86ec198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f1f6c640 1 -- 172.21.15.1:0/2652584453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86ec103660 msgr2=0x7f86ec198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f1f6c640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86ec103660 0x7f86ec198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f1f6c640 1 -- 172.21.15.1:0/2652584453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86ec102d40 msgr2=0x7f86ec198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f1f6c640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86ec102d40 0x7f86ec198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f1f6c640 1 -- 172.21.15.1:0/2652584453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec19d400 con 0x7f86ec10c170 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f1f6c640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec10c170 0x7f86ec19cd20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f86e400eb10 tx=0x7f86e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86de7fc640 1 -- 172.21.15.1:0/2652584453 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4004b20 con 0x7f86ec10c170 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86f39cf640 1 -- 172.21.15.1:0/2652584453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ec19d6f0 con 0x7f86ec10c170 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.985+0000 7f86de7fc640 1 -- 172.21.15.1:0/2652584453 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e4004cc0 con 0x7f86ec10c170 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.986+0000 7f86de7fc640 1 -- 172.21.15.1:0/2652584453 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4010760 con 0x7f86ec10c170 2024-09-16T00:38:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.986+0000 7f86f39cf640 1 -- 172.21.15.1:0/2652584453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ec1a4fd0 con 0x7f86ec10c170 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.988+0000 7f86de7fc640 1 -- 172.21.15.1:0/2652584453 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f86e4005020 con 0x7f86ec10c170 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.988+0000 7f86f5da8740 1 Processor -- start 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.988+0000 7f86f5da8740 1 -- start start 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a140f1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 0x564a14100550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 0x564a14102e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f5da8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564a140572e0 con 0x564a140f1360 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f5da8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a13ffece0 con 0x564a14100a90 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f5da8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564a140532a0 con 0x564a140f1c80 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86de7fc640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86d80a3240 0x7f86d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a140f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a140f1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43726/0 (socket says 172.21.15.1:43726) 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 -- 172.21.15.1:0/2429562712 learned_addr learned my addr 172.21.15.1:0/2429562712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f176b640 1 --2- 172.21.15.1:0/2429562712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 0x564a14100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f1f6c640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86d80a3240 0x7f86d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86de7fc640 1 -- 172.21.15.1:0/2652584453 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f86e40d0390 con 0x7f86ec10c170 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/2429562712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 0x564a14102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 -- 172.21.15.1:0/2429562712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 msgr2=0x564a14100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/2429562712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 0x564a14100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 -- 172.21.15.1:0/2429562712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 msgr2=0x564a14102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/2429562712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 0x564a14102e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 -- 172.21.15.1:0/2429562712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a13fe3440 con 0x564a140f1360 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f1f6c640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86d80a3240 0x7f86d80a5700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f86e4098f80 tx=0x7f86e4091a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.989+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/2429562712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a140f1740 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f86e80135c0 tx=0x7f86e800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845c9a6025c61310 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.990+0000 7f86d77fe640 1 -- 172.21.15.1:0/2429562712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8019030 con 0x564a140f1360 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.990+0000 7f86d77fe640 1 -- 172.21.15.1:0/2429562712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e80119a0 con 0x564a140f1360 2024-09-16T00:38:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.990+0000 7f86d77fe640 1 -- 172.21.15.1:0/2429562712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e801d7e0 con 0x564a140f1360 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.991+0000 7f86f5da8740 1 -- 172.21.15.1:0/2429562712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 msgr2=0x564a140f1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.991+0000 7f86f5da8740 1 --2- 172.21.15.1:0/2429562712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a140f1740 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f86e80135c0 tx=0x7f86e800b6c0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.991+0000 7f86f5da8740 1 -- 172.21.15.1:0/2429562712 shutdown_connections 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.991+0000 7f86f5da8740 1 --2- 172.21.15.1:0/2429562712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 0x564a14102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.991+0000 7f86f5da8740 1 --2- 172.21.15.1:0/2429562712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 0x564a14100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.991+0000 7f86f5da8740 1 --2- 172.21.15.1:0/2429562712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a140f1740 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.991+0000 7f86f5da8740 1 -- 172.21.15.1:0/2429562712 >> 172.21.15.1:0/2429562712 conn(0x564a140eda30 msgr2=0x564a140ede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.991+0000 7f86f5da8740 1 -- 172.21.15.1:0/2429562712 shutdown_connections 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.991+0000 7f86f5da8740 1 -- 172.21.15.1:0/2429562712 wait complete. 2024-09-16T00:38:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f5da8740 1 -- start start 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f5da8740 1 Processor -- start 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1360 0x564a141003a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a140fd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 0x564a140fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f5da8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564a14103660 con 0x564a140f1c80 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f5da8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a14012050 con 0x564a14100a90 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f5da8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564a141037e0 con 0x564a140f1360 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f5da8740 10 client.0 _ll_register_callbacks cb 0x564a14052160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f176b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a140fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f176b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a140fd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43740/0 (socket says 172.21.15.1:43740) 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f176b640 1 -- 172.21.15.1:0/987238798 learned_addr learned my addr 172.21.15.1:0/987238798 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.992+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1360 0x564a141003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 0x564a140fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f176b640 1 -- 172.21.15.1:0/987238798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1360 msgr2=0x564a141003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f176b640 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1360 0x564a141003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f176b640 1 -- 172.21.15.1:0/987238798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 msgr2=0x564a140fdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f176b640 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 0x564a140fdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f176b640 1 -- 172.21.15.1:0/987238798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a13dc83c0 con 0x564a140f1c80 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 0x564a140fdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f176b640 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a140fd650 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009900 tx=0x7f86e0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b45efb8ac0da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86d4ff9640 1 -- 172.21.15.1:0/987238798 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0035040 con 0x564a140f1c80 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86d4ff9640 1 -- 172.21.15.1:0/987238798 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f86e00048c0 con 0x564a140f1c80 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f5da8740 1 -- 172.21.15.1:0/987238798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e80119a0 con 0x564a140f1c80 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86d4ff9640 1 -- 172.21.15.1:0/987238798 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f86e004e050 con 0x564a140f1c80 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f5da8740 10 client.17874 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.993+0000 7f86f5da8740 1 -- 172.21.15.1:0/987238798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a1410f050 con 0x564a140f1c80 2024-09-16T00:38:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.994+0000 7f86d4ff9640 1 -- 172.21.15.1:0/987238798 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e0040070 con 0x564a140f1c80 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.994+0000 7f86d4ff9640 1 -- 172.21.15.1:0/987238798 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f86e0003ab0 con 0x564a140f1c80 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.994+0000 7f86f5da8740 10 client.17874 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.994+0000 7f86f5da8740 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.994+0000 7f86f5da8740 20 client.17874 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.994+0000 7f86f5da8740 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a14110510 0x564a14112930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.994+0000 7f86f5da8740 4 client.17874 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.994+0000 7f86f5da8740 1 -- 172.21.15.1:0/987238798 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f86e80119a0 con 0x564a14110510 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.994+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a14110510 0x564a14112930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.995+0000 7f86d4ff9640 10 client.17874 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.995+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a14110510 0x564a14112930 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5d9b3d0f82ea9b server_cookie=f2e989f0391bbc53 in_seq=0 out_seq=0 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.995+0000 7f86d4ff9640 1 -- 172.21.15.1:0/987238798 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f86e80119a0 con 0x564a14110510 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.995+0000 7f86d4ff9640 10 client.17874 handle_command_reply: tid=0 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.995+0000 7f86f5da8740 1 client.17874 shutdown 2024-09-16T00:38:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 -- 172.21.15.1:0/987238798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 msgr2=0x564a140fd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a140fd650 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009900 tx=0x7f86e0005250 comp rx=0 tx=0).stop 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 -- 172.21.15.1:0/987238798 shutdown_connections 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a14110510 0x564a14112930 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a14100a90 0x564a140fdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a140fd650 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 --2- 172.21.15.1:0/987238798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1360 0x564a141003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 -- 172.21.15.1:0/987238798 >> 172.21.15.1:0/987238798 conn(0x564a140eda30 msgr2=0x564a140ecb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86d4ff9640 0 client.17874 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 -- 172.21.15.1:0/987238798 shutdown_connections 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 1 -- 172.21.15.1:0/987238798 wait complete. 2024-09-16T00:38:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:33.996+0000 7f86f5da8740 20 client.17874 trim_cache size 0 max 16384 2024-09-16T00:38:34.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.013+0000 7f86f5da8740 1 Processor -- start 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.013+0000 7f86f5da8740 1 -- start start 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.013+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140eda30 0x564a1410c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.013+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140f1360 0x564a140f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.013+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a141181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.013+0000 7f86f5da8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1af570 con 0x564a140f1c80 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.013+0000 7f86f5da8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1ace40 con 0x564a140f1360 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.013+0000 7f86f5da8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1aac90 con 0x564a140eda30 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.013+0000 7f86f0f6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a141181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f0f6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a141181c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43742/0 (socket says 172.21.15.1:43742) 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f0f6a640 1 -- 172.21.15.1:0/2423945360 learned_addr learned my addr 172.21.15.1:0/2423945360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f176b640 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140f1360 0x564a140f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140eda30 0x564a1410c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f0f6a640 1 -- 172.21.15.1:0/2423945360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140eda30 msgr2=0x564a1410c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140eda30 0x564a1410c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f0f6a640 1 -- 172.21.15.1:0/2423945360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140f1360 msgr2=0x564a140f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140f1360 0x564a140f8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f0f6a640 1 -- 172.21.15.1:0/2423945360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a140fbba0 con 0x564a140f1c80 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f176b640 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140f1360 0x564a140f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a141181c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f86e8013f10 tx=0x7f86e800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95768fdf5fe759f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86d6ffd640 1 -- 172.21.15.1:0/2423945360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e802f070 con 0x564a140f1c80 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86d6ffd640 1 -- 172.21.15.1:0/2423945360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e8005400 con 0x564a140f1c80 2024-09-16T00:38:34.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.014+0000 7f86d6ffd640 1 -- 172.21.15.1:0/2423945360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8019040 con 0x564a140f1c80 2024-09-16T00:38:34.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 -- 172.21.15.1:0/2423945360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 msgr2=0x564a141181c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:34.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a141181c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f86e8013f10 tx=0x7f86e800b6c0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 -- 172.21.15.1:0/2423945360 shutdown_connections 2024-09-16T00:38:34.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1c80 0x564a141181c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140f1360 0x564a140f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 --2- 172.21.15.1:0/2423945360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140eda30 0x564a1410c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 -- 172.21.15.1:0/2423945360 >> 172.21.15.1:0/2423945360 conn(0x564a140fdcf0 msgr2=0x564a140f84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:34.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 -- 172.21.15.1:0/2423945360 shutdown_connections 2024-09-16T00:38:34.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 -- 172.21.15.1:0/2423945360 wait complete. 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 -- start start 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.015+0000 7f86f5da8740 1 Processor -- start 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140eda30 0x564a14109350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a14104680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f5da8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 0x564a14104e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f5da8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1ab300 con 0x564a140f1360 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f5da8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1ac770 con 0x564a140eda30 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f5da8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1ab9d0 con 0x564a140f1c80 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f5da8740 10 client.0 _ll_register_callbacks cb 0x7f86e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f176b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a14104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f176b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a14104680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43746/0 (socket says 172.21.15.1:43746) 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f176b640 1 -- 172.21.15.1:0/3037180253 learned_addr learned my addr 172.21.15.1:0/3037180253 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 0x564a14104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140eda30 0x564a14109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f176b640 1 -- 172.21.15.1:0/3037180253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 msgr2=0x564a14104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f176b640 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 0x564a14104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f176b640 1 -- 172.21.15.1:0/3037180253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140eda30 msgr2=0x564a14109350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f176b640 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140eda30 0x564a14109350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f176b640 1 -- 172.21.15.1:0/3037180253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1417a190 con 0x564a140f1360 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 0x564a14104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86f176b640 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a14104680 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f86e0000c00 tx=0x7f86e0009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385a1460e2784424 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.016+0000 7f86d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86d4ff9640 1 -- 172.21.15.1:0/3037180253 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0035030 con 0x564a140f1360 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86d4ff9640 1 -- 172.21.15.1:0/3037180253 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f86e0005650 con 0x564a140f1360 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86f5da8740 1 -- 172.21.15.1:0/3037180253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e8005400 con 0x564a140f1360 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86d4ff9640 1 -- 172.21.15.1:0/3037180253 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f86e0009520 con 0x564a140f1360 2024-09-16T00:38:34.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86f5da8740 10 client.17880 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86f5da8740 1 -- 172.21.15.1:0/3037180253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a1417b2a0 con 0x564a140f1360 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86d4ff9640 1 -- 172.21.15.1:0/3037180253 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e8005400 con 0x564a140f1360 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86d4ff9640 1 -- 172.21.15.1:0/3037180253 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f86e00046c0 con 0x564a140f1360 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86f5da8740 10 client.17880 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86f5da8740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86f5da8740 20 client.17880 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86f5da8740 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a14100a90 0x564a1417dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86f5da8740 4 client.17880 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.017+0000 7f86f5da8740 1 -- 172.21.15.1:0/3037180253 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a14064850 con 0x564a14100a90 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a14100a90 0x564a1417dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f0f6a640 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a14100a90 0x564a1417dab0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8660e0a7eaf93df2 server_cookie=e70cf2e16a0b58d2 in_seq=0 out_seq=0 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86d4ff9640 10 client.17880 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86d4ff9640 1 -- 172.21.15.1:0/3037180253 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564a14064850 con 0x564a14100a90 2024-09-16T00:38:34.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86d4ff9640 10 client.17880 handle_command_reply: tid=0 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f5da8740 1 client.17880 shutdown 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f5da8740 1 -- 172.21.15.1:0/3037180253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 msgr2=0x564a14104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f5da8740 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a14104680 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f86e0000c00 tx=0x7f86e0009230 comp rx=0 tx=0).stop 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f5da8740 1 -- 172.21.15.1:0/3037180253 shutdown_connections 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f5da8740 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a14100a90 0x564a1417dab0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f5da8740 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a140f1c80 0x564a14104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f5da8740 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a140f1360 0x564a14104680 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86d4ff9640 0 client.17880 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f5da8740 1 --2- 172.21.15.1:0/3037180253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a140eda30 0x564a14109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.018+0000 7f86f5da8740 1 -- 172.21.15.1:0/3037180253 >> 172.21.15.1:0/3037180253 conn(0x564a140fdcf0 msgr2=0x564a140f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f5da8740 1 -- 172.21.15.1:0/3037180253 shutdown_connections 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f5da8740 1 -- 172.21.15.1:0/3037180253 wait complete. 2024-09-16T00:38:34.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f5da8740 20 client.17880 trim_cache size 0 max 16384 2024-09-16T00:38:34.025 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:38:34.025 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:38:34.025 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:38:34.025 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 -- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86d80a3240 msgr2=0x7f86d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86d80a3240 0x7f86d80a5700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f86e4098f80 tx=0x7f86e4091a90 comp rx=0 tx=0).stop 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 -- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec10c170 msgr2=0x7f86ec19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec10c170 0x7f86ec19cd20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f86e400eb10 tx=0x7f86e400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 -- 172.21.15.1:0/2652584453 shutdown_connections 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86ec10c170 0x7f86ec19cd20 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86d80a3240 0x7f86d80a5700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86ec103660 0x7f86ec198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 --2- 172.21.15.1:0/2652584453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86ec102d40 0x7f86ec198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.019+0000 7f86f39cf640 1 -- 172.21.15.1:0/2652584453 >> 172.21.15.1:0/2652584453 conn(0x7f86ec0fc7a0 msgr2=0x7f86ec0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.020+0000 7f86f39cf640 1 -- 172.21.15.1:0/2652584453 shutdown_connections 2024-09-16T00:38:34.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:34.020+0000 7f86f39cf640 1 -- 172.21.15.1:0/2652584453 wait complete. 2024-09-16T00:38:34.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:34 smithi001 ceph-mon[30934]: pgmap v655: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:38:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:34 smithi019 ceph-mon[35186]: pgmap v655: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:38:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:34 smithi153 ceph-mon[35538]: pgmap v655: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:38:34.786 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:38:34.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:35.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.523+0000 7f86a133f640 1 -- 172.21.15.1:0/141028448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f869c1036c0 msgr2=0x7f869c10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.523+0000 7f86a133f640 1 --2- 172.21.15.1:0/141028448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f869c1036c0 0x7f869c10bbb0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8690009900 tx=0x7f869002a480 comp rx=0 tx=0).stop 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.523+0000 7f86a133f640 1 -- 172.21.15.1:0/141028448 shutdown_connections 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.523+0000 7f86a133f640 1 --2- 172.21.15.1:0/141028448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869c10c180 0x7f869c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.523+0000 7f86a133f640 1 --2- 172.21.15.1:0/141028448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f869c1036c0 0x7f869c10bbb0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.523+0000 7f86a133f640 1 --2- 172.21.15.1:0/141028448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f869c102da0 0x7f869c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.523+0000 7f86a133f640 1 -- 172.21.15.1:0/141028448 >> 172.21.15.1:0/141028448 conn(0x7f869c0fc7a0 msgr2=0x7f869c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.523+0000 7f86a133f640 1 -- 172.21.15.1:0/141028448 shutdown_connections 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.523+0000 7f86a133f640 1 -- 172.21.15.1:0/141028448 wait complete. 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.524+0000 7f86a133f640 1 Processor -- start 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.524+0000 7f86a133f640 1 -- start start 2024-09-16T00:38:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.524+0000 7f86a133f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f869c102da0 0x7f869c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.524+0000 7f86a133f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f869c1036c0 0x7f869c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.525+0000 7f86a133f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869c10c180 0x7f869c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.525+0000 7f86a133f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f869c111f60 con 0x7f869c1036c0 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.525+0000 7f86a133f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f869c111de0 con 0x7f869c102da0 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.525+0000 7f86a133f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f869c1120e0 con 0x7f869c10c180 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.525+0000 7f869affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f869c102da0 0x7f869c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.525+0000 7f869affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f869c102da0 0x7f869c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:60630/0 (socket says 172.21.15.1:60630) 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.525+0000 7f869affd640 1 -- 172.21.15.1:0/1591367958 learned_addr learned my addr 172.21.15.1:0/1591367958 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f869a7fc640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f869c1036c0 0x7f869c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f869b7fe640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869c10c180 0x7f869c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f869affd640 1 -- 172.21.15.1:0/1591367958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869c10c180 msgr2=0x7f869c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f869affd640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869c10c180 0x7f869c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f869affd640 1 -- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f869c1036c0 msgr2=0x7f869c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:35.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f869affd640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f869c1036c0 0x7f869c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f869affd640 1 -- 172.21.15.1:0/1591367958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869c19d3c0 con 0x7f869c102da0 2024-09-16T00:38:35.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f869a7fc640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f869c1036c0 0x7f869c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:38:35.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f869affd640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f869c102da0 0x7f869c198410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f868c00d950 tx=0x7f868c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:35.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f8683fff640 1 -- 172.21.15.1:0/1591367958 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c017030 con 0x7f869c102da0 2024-09-16T00:38:35.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869c19d6b0 con 0x7f869c102da0 2024-09-16T00:38:35.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f8683fff640 1 -- 172.21.15.1:0/1591367958 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f868c004020 con 0x7f869c102da0 2024-09-16T00:38:35.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f8683fff640 1 -- 172.21.15.1:0/1591367958 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c0047a0 con 0x7f869c102da0 2024-09-16T00:38:35.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.526+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869c100930 con 0x7f869c102da0 2024-09-16T00:38:35.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.527+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f869c06b010 con 0x7f869c102da0 2024-09-16T00:38:35.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.529+0000 7f8683fff640 1 -- 172.21.15.1:0/1591367958 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f868c0041d0 con 0x7f869c102da0 2024-09-16T00:38:35.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.529+0000 7f8683fff640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86840a3240 0x7f86840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:35.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.529+0000 7f869a7fc640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86840a3240 0x7f86840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:35.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.530+0000 7f8683fff640 1 -- 172.21.15.1:0/1591367958 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f868c0cfd20 con 0x7f869c102da0 2024-09-16T00:38:35.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.530+0000 7f869a7fc640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86840a3240 0x7f86840a5700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8690009230 tx=0x7f86900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:35.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.533+0000 7f8683fff640 1 -- 172.21.15.1:0/1591367958 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f868c09c5e0 con 0x7f869c102da0 2024-09-16T00:38:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.745+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f869c103180 con 0x7f869c102da0 2024-09-16T00:38:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.746+0000 7f8683fff640 1 -- 172.21.15.1:0/1591367958 <== mon.2 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f868c09bee0 con 0x7f869c102da0 2024-09-16T00:38:35.747 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:38:35.747 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:38:35.747 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:38:35.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86840a3240 msgr2=0x7f86840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:35.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86840a3240 0x7f86840a5700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8690009230 tx=0x7f86900058d0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f869c102da0 msgr2=0x7f869c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:35.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f869c102da0 0x7f869c198410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f868c00d950 tx=0x7f868c00de10 comp rx=0 tx=0).stop 2024-09-16T00:38:35.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 shutdown_connections 2024-09-16T00:38:35.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f86840a3240 0x7f86840a5700 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f869c10c180 0x7f869c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f869c1036c0 0x7f869c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 --2- 172.21.15.1:0/1591367958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f869c102da0 0x7f869c198410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:35.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 >> 172.21.15.1:0/1591367958 conn(0x7f869c0fc7a0 msgr2=0x7f869c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:35.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 shutdown_connections 2024-09-16T00:38:35.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:35.749+0000 7f86a133f640 1 -- 172.21.15.1:0/1591367958 wait complete. 2024-09-16T00:38:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:36 smithi019 ceph-mon[35186]: pgmap v656: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:38:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:36 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1591367958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:36.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:36 smithi001 ceph-mon[30934]: pgmap v656: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:38:36.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:36 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1591367958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:36.456 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:38:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:36 smithi153 ceph-mon[35538]: pgmap v656: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:38:36.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:36 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1591367958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:36.716 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:37.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.208+0000 7f46c6115640 1 -- 172.21.15.1:0/2204744042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c01027a0 msgr2=0x7f46c01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.208+0000 7f46c6115640 1 --2- 172.21.15.1:0/2204744042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c01027a0 0x7f46c01067f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009900 tx=0x7f46b402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.208+0000 7f46c6115640 1 -- 172.21.15.1:0/2204744042 shutdown_connections 2024-09-16T00:38:37.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.208+0000 7f46c6115640 1 --2- 172.21.15.1:0/2204744042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c0106f20 0x7f46c010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.208+0000 7f46c6115640 1 --2- 172.21.15.1:0/2204744042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c01027a0 0x7f46c01067f0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.208+0000 7f46c6115640 1 --2- 172.21.15.1:0/2204744042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0101df0 0x7f46c01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.209+0000 7f46c6115640 1 -- 172.21.15.1:0/2204744042 >> 172.21.15.1:0/2204744042 conn(0x7f46c00fd8f0 msgr2=0x7f46c00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.209+0000 7f46c6115640 1 -- 172.21.15.1:0/2204744042 shutdown_connections 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.209+0000 7f46c6115640 1 -- 172.21.15.1:0/2204744042 wait complete. 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.209+0000 7f46c6115640 1 Processor -- start 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.209+0000 7f46c6115640 1 -- start start 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.209+0000 7f46c6115640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c0101df0 0x7f46c0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.209+0000 7f46c6115640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c01027a0 0x7f46c0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46c6115640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0106f20 0x7f46c019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46c6115640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0111fa0 con 0x7f46c0101df0 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46c6115640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0111e20 con 0x7f46c0106f20 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46c6115640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0112120 con 0x7f46c01027a0 2024-09-16T00:38:37.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c0101df0 0x7f46c0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c0101df0 0x7f46c0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43794/0 (socket says 172.21.15.1:43794) 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bf7fe640 1 -- 172.21.15.1:0/4034928580 learned_addr learned my addr 172.21.15.1:0/4034928580 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bffff640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0106f20 0x7f46c019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46beffd640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c01027a0 0x7f46c0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bf7fe640 1 -- 172.21.15.1:0/4034928580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c01027a0 msgr2=0x7f46c0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c01027a0 0x7f46c0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bf7fe640 1 -- 172.21.15.1:0/4034928580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0106f20 msgr2=0x7f46c019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0106f20 0x7f46c019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bf7fe640 1 -- 172.21.15.1:0/4034928580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c019d3f0 con 0x7f46c0101df0 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c0101df0 0x7f46c0198440 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f46b000cce0 tx=0x7f46b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bcff9640 1 -- 172.21.15.1:0/4034928580 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b000f650 con 0x7f46c0101df0 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bcff9640 1 -- 172.21.15.1:0/4034928580 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b000fcd0 con 0x7f46c0101df0 2024-09-16T00:38:37.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46bcff9640 1 -- 172.21.15.1:0/4034928580 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0018c70 con 0x7f46c0101df0 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.210+0000 7f46c6115640 1 -- 172.21.15.1:0/4034928580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c019d6e0 con 0x7f46c0101df0 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.211+0000 7f46c6115640 1 -- 172.21.15.1:0/4034928580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c01a4fc0 con 0x7f46c0101df0 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.213+0000 7f46bcff9640 1 -- 172.21.15.1:0/4034928580 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f46b000f7f0 con 0x7f46c0101df0 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.213+0000 7f46c84ee740 1 Processor -- start 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.213+0000 7f46c84ee740 1 -- start start 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.213+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ec0b40 0x5648a1ebf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.213+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ebfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.213+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1eca370 0x5648a1ecc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.213+0000 7f46c84ee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5648a1e252f0 con 0x5648a1eca370 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.213+0000 7f46c84ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5648a1dcccf0 con 0x5648a1ec0b40 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.213+0000 7f46c84ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5648a1e212b0 con 0x5648a1ebf5d0 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1eca370 0x5648a1ecc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46bcff9640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46a80a3240 0x7f46a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1eca370 0x5648a1ecc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43798/0 (socket says 172.21.15.1:43798) 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46beffd640 1 -- 172.21.15.1:0/476600863 learned_addr learned my addr 172.21.15.1:0/476600863 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46bffff640 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ec0b40 0x5648a1ebf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ebfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46beffd640 1 -- 172.21.15.1:0/476600863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 msgr2=0x5648a1ebfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46bcff9640 1 -- 172.21.15.1:0/4034928580 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f46b00d0100 con 0x7f46c0101df0 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46beffd640 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ebfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46beffd640 1 -- 172.21.15.1:0/476600863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ec0b40 msgr2=0x5648a1ebf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46beffd640 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ec0b40 0x5648a1ebf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46bffff640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46a80a3240 0x7f46a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46beffd640 1 -- 172.21.15.1:0/476600863 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648a1db13a0 con 0x5648a1eca370 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ebfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46beffd640 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1eca370 0x5648a1ecc760 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009900 tx=0x7f46b40316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd78a902239724a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:37.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46a5ffb640 1 -- 172.21.15.1:0/476600863 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4040070 con 0x5648a1eca370 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46bffff640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46a80a3240 0x7f46a80a5700 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009c10 tx=0x7f46b80090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46a5ffb640 1 -- 172.21.15.1:0/476600863 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b4005020 con 0x5648a1eca370 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.214+0000 7f46a5ffb640 1 -- 172.21.15.1:0/476600863 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4035040 con 0x5648a1eca370 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.215+0000 7f46c84ee740 1 -- 172.21.15.1:0/476600863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1eca370 msgr2=0x5648a1ecc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.215+0000 7f46c84ee740 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1eca370 0x5648a1ecc760 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009900 tx=0x7f46b40316e0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.215+0000 7f46c84ee740 1 -- 172.21.15.1:0/476600863 shutdown_connections 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.215+0000 7f46c84ee740 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1eca370 0x5648a1ecc760 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.215+0000 7f46c84ee740 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ebfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.215+0000 7f46c84ee740 1 --2- 172.21.15.1:0/476600863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ec0b40 0x5648a1ebf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.215+0000 7f46c84ee740 1 -- 172.21.15.1:0/476600863 >> 172.21.15.1:0/476600863 conn(0x5648a1e88310 msgr2=0x5648a1ebbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.215+0000 7f46c84ee740 1 -- 172.21.15.1:0/476600863 shutdown_connections 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.215+0000 7f46c84ee740 1 -- 172.21.15.1:0/476600863 wait complete. 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46c84ee740 1 -- start start 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46c84ee740 1 Processor -- start 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1eca370 0x5648a1ec7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46c84ee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5648a1ecd010 con 0x5648a1ec0b40 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46c84ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5648a1de0060 con 0x5648a1eca370 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46c84ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5648a1ecd190 con 0x5648a1ebf5d0 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46c84ee740 10 client.0 _ll_register_callbacks cb 0x5648a1b338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43812/0 (socket says 172.21.15.1:43812) 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46beffd640 1 -- 172.21.15.1:0/4058616467 learned_addr learned my addr 172.21.15.1:0/4058616467 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.216+0000 7f46bffff640 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1eca370 0x5648a1ec7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f46beffd640 1 -- 172.21.15.1:0/4058616467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 msgr2=0x5648a1ec9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f46beffd640 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f46beffd640 1 -- 172.21.15.1:0/4058616467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1eca370 msgr2=0x5648a1ec7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f46beffd640 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1eca370 0x5648a1ec7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f46beffd640 1 -- 172.21.15.1:0/4058616467 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648a1b963c0 con 0x5648a1ec0b40 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f46beffd640 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec6f50 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f46b402a910 tx=0x7f46b40095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de438da8d107e254 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f469effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f469effd640 1 -- 172.21.15.1:0/4058616467 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b404c020 con 0x5648a1ec0b40 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f469effd640 1 -- 172.21.15.1:0/4058616467 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f46b4004af0 con 0x5648a1ec0b40 2024-09-16T00:38:37.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f46c84ee740 1 -- 172.21.15.1:0/4058616467 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b4005020 con 0x5648a1ec0b40 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f469effd640 1 -- 172.21.15.1:0/4058616467 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f46b4057050 con 0x5648a1ec0b40 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f46c84ee740 10 client.17898 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f46c84ee740 1 -- 172.21.15.1:0/4058616467 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5648a1ed89a0 con 0x5648a1ec0b40 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.217+0000 7f469effd640 1 -- 172.21.15.1:0/4058616467 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b4005020 con 0x5648a1ec0b40 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f469effd640 1 -- 172.21.15.1:0/4058616467 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f46b4044400 con 0x5648a1ec0b40 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f46c84ee740 10 client.17898 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f46c84ee740 10 client.17898 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f46c84ee740 20 client.17898 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5648a1ed9cf0 0x5648a1edc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f46c84ee740 4 client.17898 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f46c84ee740 1 -- 172.21.15.1:0/4058616467 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648a1e32860 con 0x5648a1ed9cf0 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5648a1ed9cf0 0x5648a1edc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f469effd640 10 client.17898 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:38:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.218+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5648a1ed9cf0 0x5648a1edc0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f44d920c5051574 server_cookie=d1f02cc9965b7694 in_seq=0 out_seq=0 2024-09-16T00:38:37.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f469effd640 1 -- 172.21.15.1:0/4058616467 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648a1e32860 con 0x5648a1ed9cf0 2024-09-16T00:38:37.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f469effd640 10 client.17898 handle_command_reply: tid=0 2024-09-16T00:38:37.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 client.17898 shutdown 2024-09-16T00:38:37.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 -- 172.21.15.1:0/4058616467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 msgr2=0x5648a1ec6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec6f50 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f46b402a910 tx=0x7f46b40095a0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 -- 172.21.15.1:0/4058616467 shutdown_connections 2024-09-16T00:38:37.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5648a1ed9cf0 0x5648a1edc0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1eca370 0x5648a1ec7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec6f50 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f469effd640 0 client.17898 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:38:37.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4058616467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 -- 172.21.15.1:0/4058616467 >> 172.21.15.1:0/4058616467 conn(0x5648a1e88310 msgr2=0x5648a1ec1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:37.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 -- 172.21.15.1:0/4058616467 shutdown_connections 2024-09-16T00:38:37.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.219+0000 7f46c84ee740 1 -- 172.21.15.1:0/4058616467 wait complete. 2024-09-16T00:38:37.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.220+0000 7f46c84ee740 20 client.17898 trim_cache size 0 max 16384 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46c84ee740 1 Processor -- start 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46c84ee740 1 -- start start 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1e88310 0x5648a1ed62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ee1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46c84ee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46c01b16f0 con 0x5648a1ec0b40 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46c84ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46c01aefc0 con 0x5648a1e88310 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46c84ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46c01ace10 con 0x5648a1ebf5d0 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ee1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ee1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43822/0 (socket says 172.21.15.1:43822) 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46bffff640 1 -- 172.21.15.1:0/4180692123 learned_addr learned my addr 172.21.15.1:0/4180692123 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.233+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/4180692123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1e88310 0x5648a1ed62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46beffd640 1 --2- 172.21.15.1:0/4180692123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46bffff640 1 -- 172.21.15.1:0/4180692123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 msgr2=0x5648a1ec5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46bffff640 1 --2- 172.21.15.1:0/4180692123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46bffff640 1 -- 172.21.15.1:0/4180692123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1e88310 msgr2=0x5648a1ed62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46bffff640 1 --2- 172.21.15.1:0/4180692123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1e88310 0x5648a1ed62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46bffff640 1 -- 172.21.15.1:0/4180692123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648a1ebf3d0 con 0x5648a1ec0b40 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46bffff640 1 --2- 172.21.15.1:0/4180692123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ee1740 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009f60 tx=0x7f46b80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5139e34a1a0beb4e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46a57fa640 1 -- 172.21.15.1:0/4180692123 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8025070 con 0x5648a1ec0b40 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46a57fa640 1 -- 172.21.15.1:0/4180692123 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b8015d40 con 0x5648a1ec0b40 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.234+0000 7f46a57fa640 1 -- 172.21.15.1:0/4180692123 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8019040 con 0x5648a1ec0b40 2024-09-16T00:38:37.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 -- 172.21.15.1:0/4180692123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 msgr2=0x5648a1ee1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4180692123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ee1740 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009f60 tx=0x7f46b80093f0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 -- 172.21.15.1:0/4180692123 shutdown_connections 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4180692123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ee1740 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4180692123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 --2- 172.21.15.1:0/4180692123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1e88310 0x5648a1ed62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 -- 172.21.15.1:0/4180692123 >> 172.21.15.1:0/4180692123 conn(0x5648a1ec75f0 msgr2=0x5648a1ec5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 -- 172.21.15.1:0/4180692123 shutdown_connections 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 -- 172.21.15.1:0/4180692123 wait complete. 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.235+0000 7f46c84ee740 1 -- start start 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46c84ee740 1 Processor -- start 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1e88310 0x5648a1ec4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46c84ee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46c84ee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46c01ad480 con 0x5648a1ec0b40 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46c84ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46c01ae8f0 con 0x5648a1ebf5d0 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46c84ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46c01adb50 con 0x5648a1e88310 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46c84ee740 10 client.0 _ll_register_callbacks cb 0x7f46b40315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43834/0 (socket says 172.21.15.1:43834) 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bffff640 1 -- 172.21.15.1:0/3382783232 learned_addr learned my addr 172.21.15.1:0/3382783232 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1e88310 0x5648a1ec4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46beffd640 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bffff640 1 -- 172.21.15.1:0/3382783232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1e88310 msgr2=0x5648a1ec4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bffff640 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1e88310 0x5648a1ec4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bffff640 1 -- 172.21.15.1:0/3382783232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ebf5d0 msgr2=0x5648a1ec46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bffff640 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bffff640 1 -- 172.21.15.1:0/3382783232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648a1f43910 con 0x5648a1ec0b40 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.236+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1e88310 0x5648a1ec4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46bffff640 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec4be0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f46b802af00 tx=0x7f46b802a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28d6e01fc2f96eb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f469effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f469effd640 1 -- 172.21.15.1:0/3382783232 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8019030 con 0x5648a1ec0b40 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46c84ee740 1 -- 172.21.15.1:0/3382783232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b8015d40 con 0x5648a1ec0b40 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f469effd640 1 -- 172.21.15.1:0/3382783232 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f46b80059c0 con 0x5648a1ec0b40 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f469effd640 1 -- 172.21.15.1:0/3382783232 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f46b803c050 con 0x5648a1ec0b40 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46c84ee740 10 client.17904 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46c84ee740 1 -- 172.21.15.1:0/3382783232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5648a1f44bb0 con 0x5648a1ec0b40 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f469effd640 1 -- 172.21.15.1:0/3382783232 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b8015d40 con 0x5648a1ec0b40 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f469effd640 1 -- 172.21.15.1:0/3382783232 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f46b8033470 con 0x5648a1ec0b40 2024-09-16T00:38:37.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46c84ee740 10 client.17904 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46c84ee740 10 client.17904 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46c84ee740 20 client.17904 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46c84ee740 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5648a1eca370 0x5648a1f471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46c84ee740 4 client.17904 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.237+0000 7f46c84ee740 1 -- 172.21.15.1:0/3382783232 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648a1e2cd70 con 0x5648a1eca370 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.238+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5648a1eca370 0x5648a1f471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.238+0000 7f46bf7fe640 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5648a1eca370 0x5648a1f471d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c66d50724bc6f2fa server_cookie=28ef9b83f1d82d72 in_seq=0 out_seq=0 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.238+0000 7f469effd640 10 client.17904 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.238+0000 7f469effd640 1 -- 172.21.15.1:0/3382783232 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648a1e2cd70 con 0x5648a1eca370 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.238+0000 7f469effd640 10 client.17904 handle_command_reply: tid=0 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.238+0000 7f46c84ee740 1 client.17904 shutdown 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 -- 172.21.15.1:0/3382783232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 msgr2=0x5648a1ec4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec4be0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f46b802af00 tx=0x7f46b802a670 comp rx=0 tx=0).stop 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 -- 172.21.15.1:0/3382783232 shutdown_connections 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648a1ec0b40 0x5648a1ec4be0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5648a1eca370 0x5648a1f471d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f469effd640 0 client.17904 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648a1ebf5d0 0x5648a1ec46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 --2- 172.21.15.1:0/3382783232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648a1e88310 0x5648a1ec4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 -- 172.21.15.1:0/3382783232 >> 172.21.15.1:0/3382783232 conn(0x5648a1ec75f0 msgr2=0x5648a1ec6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 -- 172.21.15.1:0/3382783232 shutdown_connections 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 1 -- 172.21.15.1:0/3382783232 wait complete. 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c84ee740 20 client.17904 trim_cache size 0 max 16384 2024-09-16T00:38:37.248 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:38:37.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c6115640 1 -- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46a80a3240 msgr2=0x7f46a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c6115640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46a80a3240 0x7f46a80a5700 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009c10 tx=0x7f46b80090e0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c6115640 1 -- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c0101df0 msgr2=0x7f46c0198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.239+0000 7f46c6115640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c0101df0 0x7f46c0198440 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f46b000cce0 tx=0x7f46b000a620 comp rx=0 tx=0).stop 2024-09-16T00:38:37.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.240+0000 7f46c6115640 1 -- 172.21.15.1:0/4034928580 shutdown_connections 2024-09-16T00:38:37.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.240+0000 7f46c6115640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0106f20 0x7f46c019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.240+0000 7f46c6115640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46a80a3240 0x7f46a80a5700 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.240+0000 7f46c6115640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c01027a0 0x7f46c0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.240+0000 7f46c6115640 1 --2- 172.21.15.1:0/4034928580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c0101df0 0x7f46c0198440 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.240+0000 7f46c6115640 1 -- 172.21.15.1:0/4034928580 >> 172.21.15.1:0/4034928580 conn(0x7f46c00fd8f0 msgr2=0x7f46c00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:37.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.240+0000 7f46c6115640 1 -- 172.21.15.1:0/4034928580 shutdown_connections 2024-09-16T00:38:37.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:37.240+0000 7f46c6115640 1 -- 172.21.15.1:0/4034928580 wait complete. 2024-09-16T00:38:37.967 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:38:38.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:38 smithi001 ceph-mon[30934]: pgmap v657: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:38:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:38.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:38 smithi019 ceph-mon[35186]: pgmap v657: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:38:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:38 smithi153 ceph-mon[35538]: pgmap v657: 129 pgs: 129 active+clean; 4.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:38:38.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.754+0000 7fcccde65640 1 -- 172.21.15.1:0/3037049824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 msgr2=0x7fccc81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.754+0000 7fcccde65640 1 --2- 172.21.15.1:0/3037049824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 0x7fccc81021d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009ca0 tx=0x7fccb802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.754+0000 7fcccde65640 1 -- 172.21.15.1:0/3037049824 shutdown_connections 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.754+0000 7fcccde65640 1 --2- 172.21.15.1:0/3037049824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc8106f20 0x7fccc810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.754+0000 7fcccde65640 1 --2- 172.21.15.1:0/3037049824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccc81027a0 0x7fccc81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.754+0000 7fcccde65640 1 --2- 172.21.15.1:0/3037049824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 0x7fccc81021d0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.754+0000 7fcccde65640 1 -- 172.21.15.1:0/3037049824 >> 172.21.15.1:0/3037049824 conn(0x7fccc80fd8f0 msgr2=0x7fccc80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.754+0000 7fcccde65640 1 -- 172.21.15.1:0/3037049824 shutdown_connections 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.754+0000 7fcccde65640 1 -- 172.21.15.1:0/3037049824 wait complete. 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fcccde65640 1 Processor -- start 2024-09-16T00:38:38.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fcccde65640 1 -- start start 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fcccde65640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 0x7fccc80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fcccde65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc81027a0 0x7fccc8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fcccde65640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccc8106f20 0x7fccc8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fcccde65640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8111f80 con 0x7fccc8101df0 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fcccde65640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8111e00 con 0x7fccc81027a0 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fcccde65640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8112100 con 0x7fccc8106f20 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fccc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 0x7fccc80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fccc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 0x7fccc80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43860/0 (socket says 172.21.15.1:43860) 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.755+0000 7fccc77fe640 1 -- 172.21.15.1:0/1018402068 learned_addr learned my addr 172.21.15.1:0/1018402068 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:38.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc7fff640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccc8106f20 0x7fccc8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc81027a0 0x7fccc8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc77fe640 1 -- 172.21.15.1:0/1018402068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccc8106f20 msgr2=0x7fccc8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc77fe640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccc8106f20 0x7fccc8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc77fe640 1 -- 172.21.15.1:0/1018402068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc81027a0 msgr2=0x7fccc8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc77fe640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc81027a0 0x7fccc8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc77fe640 1 -- 172.21.15.1:0/1018402068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc81a67e0 con 0x7fccc8101df0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc77fe640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 0x7fccc80744f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fccb80097b0 tx=0x7fccb8031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc4ff9640 1 -- 172.21.15.1:0/1018402068 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb802fba0 con 0x7fccc8101df0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fcccde65640 1 -- 172.21.15.1:0/1018402068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81a6a70 con 0x7fccc8101df0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc4ff9640 1 -- 172.21.15.1:0/1018402068 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb802fd40 con 0x7fccc8101df0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fccc4ff9640 1 -- 172.21.15.1:0/1018402068 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb803b930 con 0x7fccc8101df0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.756+0000 7fcccde65640 1 -- 172.21.15.1:0/1018402068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81a7000 con 0x7fccc8101df0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.758+0000 7fccc4ff9640 1 -- 172.21.15.1:0/1018402068 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fccb803bad0 con 0x7fccc8101df0 2024-09-16T00:38:38.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccd023e740 1 Processor -- start 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccd023e740 1 -- start start 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccd023e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 0x560e41911090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccd023e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41911a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccd023e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e4191e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccd023e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560e418772f0 con 0x560e4191c370 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccd023e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560e4181ecf0 con 0x560e41912b40 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccd023e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e418732b0 con 0x560e419115d0 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e4191e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e4191e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43876/0 (socket says 172.21.15.1:43876) 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc6ffd640 1 -- 172.21.15.1:0/3877109254 learned_addr learned my addr 172.21.15.1:0/3877109254 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc7fff640 1 --2- 172.21.15.1:0/3877109254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 0x560e41911090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc77fe640 1 --2- 172.21.15.1:0/3877109254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41911a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc4ff9640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fccb00a3240 0x7fccb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc6ffd640 1 -- 172.21.15.1:0/3877109254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 msgr2=0x560e41911a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/3877109254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41911a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc6ffd640 1 -- 172.21.15.1:0/3877109254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 msgr2=0x560e41911090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/3877109254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 0x560e41911090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.759+0000 7fccc6ffd640 1 -- 172.21.15.1:0/3877109254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e418033a0 con 0x560e4191c370 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.760+0000 7fccc7fff640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fccb00a3240 0x7fccb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.760+0000 7fccc4ff9640 1 -- 172.21.15.1:0/1018402068 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fccb80f14e0 con 0x7fccc8101df0 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.760+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/3877109254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e4191e760 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00cc40 tx=0x7fccbc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b4b8c4a8caf670 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.760+0000 7fccadffb640 1 -- 172.21.15.1:0/3877109254 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc014070 con 0x560e4191c370 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.760+0000 7fccadffb640 1 -- 172.21.15.1:0/3877109254 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccbc0047a0 con 0x560e4191c370 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.760+0000 7fccadffb640 1 -- 172.21.15.1:0/3877109254 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc010740 con 0x560e4191c370 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 -- 172.21.15.1:0/3877109254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 msgr2=0x560e4191e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 --2- 172.21.15.1:0/3877109254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e4191e760 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00cc40 tx=0x7fccbc00e720 comp rx=0 tx=0).stop 2024-09-16T00:38:38.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccc7fff640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fccb00a3240 0x7fccb00a5700 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fccc00045e0 tx=0x7fccc00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 -- 172.21.15.1:0/3877109254 shutdown_connections 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 --2- 172.21.15.1:0/3877109254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e4191e760 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 --2- 172.21.15.1:0/3877109254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41911a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 --2- 172.21.15.1:0/3877109254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 0x560e41911090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 -- 172.21.15.1:0/3877109254 >> 172.21.15.1:0/3877109254 conn(0x560e418da310 msgr2=0x560e4190dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 -- 172.21.15.1:0/3877109254 shutdown_connections 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 -- 172.21.15.1:0/3877109254 wait complete. 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 -- start start 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.761+0000 7fccd023e740 1 Processor -- start 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccd023e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e419115d0 0x560e4191bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccd023e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e41912b40 0x560e41918ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccd023e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 0x560e419195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccd023e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560e4191f070 con 0x560e419115d0 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccd023e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560e41832060 con 0x560e4191c370 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccd023e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e4191f1f0 con 0x560e41912b40 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccd023e740 10 client.0 _ll_register_callbacks cb 0x560e415858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e419115d0 0x560e4191bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e419115d0 0x560e4191bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43892/0 (socket says 172.21.15.1:43892) 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc77fe640 1 -- 172.21.15.1:0/3837980530 learned_addr learned my addr 172.21.15.1:0/3837980530 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e41912b40 0x560e41918ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc7fff640 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 0x560e419195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc77fe640 1 -- 172.21.15.1:0/3837980530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e41912b40 msgr2=0x560e41918ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc77fe640 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e41912b40 0x560e41918ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc77fe640 1 -- 172.21.15.1:0/3837980530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 msgr2=0x560e419195e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc77fe640 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 0x560e419195e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc77fe640 1 -- 172.21.15.1:0/3837980530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e415e83c0 con 0x560e419115d0 2024-09-16T00:38:38.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc7fff640 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 0x560e419195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fccc77fe640 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e419115d0 0x560e4191bd40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000f80 tx=0x7fccb8009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2d093206baf4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.762+0000 7fcca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3837980530 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80098d0 con 0x560e419115d0 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3837980530 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fccb80b9d70 con 0x560e419115d0 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fccd023e740 1 -- 172.21.15.1:0/3837980530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccbc0047a0 con 0x560e419115d0 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3837980530 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fccb8043330 con 0x560e419115d0 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fccd023e740 10 client.17916 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fccd023e740 1 -- 172.21.15.1:0/3837980530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e4191f070 con 0x560e419115d0 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3837980530 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccbc0047a0 con 0x560e419115d0 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3837980530 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fccb80f11a0 con 0x560e419115d0 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fccd023e740 10 client.17916 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fccd023e740 10 client.17916 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fccd023e740 20 client.17916 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:38.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fccd023e740 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560e4192bfd0 0x560e4192e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fccd023e740 4 client.17916 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:38.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.763+0000 7fccd023e740 1 -- 172.21.15.1:0/3837980530 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e41884860 con 0x560e4192bfd0 2024-09-16T00:38:38.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.764+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560e4192bfd0 0x560e4192e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.764+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560e4192bfd0 0x560e4192e3c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43cf9633d3df118a server_cookie=5073d59c87cc2ddc in_seq=0 out_seq=0 2024-09-16T00:38:38.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.764+0000 7fcca6ffd640 10 client.17916 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:38:38.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3837980530 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e41884860 con 0x560e4192bfd0 2024-09-16T00:38:38.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fcca6ffd640 10 client.17916 handle_command_reply: tid=0 2024-09-16T00:38:38.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 client.17916 shutdown 2024-09-16T00:38:38.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 -- 172.21.15.1:0/3837980530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e419115d0 msgr2=0x560e4191bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:38.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e419115d0 0x560e4191bd40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000f80 tx=0x7fccb8009660 comp rx=0 tx=0).stop 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 -- 172.21.15.1:0/3837980530 shutdown_connections 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 0x560e419195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e41912b40 0x560e41918ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560e4192bfd0 0x560e4192e3c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 --2- 172.21.15.1:0/3837980530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e419115d0 0x560e4191bd40 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fcca6ffd640 0 client.17916 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 -- 172.21.15.1:0/3837980530 >> 172.21.15.1:0/3837980530 conn(0x560e418da310 msgr2=0x560e41913960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 -- 172.21.15.1:0/3837980530 shutdown_connections 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 1 -- 172.21.15.1:0/3837980530 wait complete. 2024-09-16T00:38:38.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.765+0000 7fccd023e740 20 client.17916 trim_cache size 0 max 16384 2024-09-16T00:38:38.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccd023e740 1 Processor -- start 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccd023e740 1 -- start start 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccd023e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41917f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccd023e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 0x560e4190cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccd023e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e41914120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccd023e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81b3700 con 0x560e4191c370 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccd023e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81b0fd0 con 0x560e41912b40 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccd023e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81aee20 con 0x560e419115d0 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e41914120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccc7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e41914120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43908/0 (socket says 172.21.15.1:43908) 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.778+0000 7fccc7fff640 1 -- 172.21.15.1:0/4182557601 learned_addr learned my addr 172.21.15.1:0/4182557601 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/4182557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 0x560e4190cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccc77fe640 1 --2- 172.21.15.1:0/4182557601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41917f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccc7fff640 1 -- 172.21.15.1:0/4182557601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 msgr2=0x560e41917f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:38.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccc7fff640 1 --2- 172.21.15.1:0/4182557601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41917f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccc7fff640 1 -- 172.21.15.1:0/4182557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 msgr2=0x560e4190cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:38.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccc7fff640 1 --2- 172.21.15.1:0/4182557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 0x560e4190cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccc7fff640 1 -- 172.21.15.1:0/4182557601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e41917540 con 0x560e4191c370 2024-09-16T00:38:38.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccc7fff640 1 --2- 172.21.15.1:0/4182557601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e41914120 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fccc0003e70 tx=0x7fccc0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721c9d0620a13307 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:38.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccad7fa640 1 -- 172.21.15.1:0/4182557601 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc001f630 con 0x560e4191c370 2024-09-16T00:38:38.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccad7fa640 1 -- 172.21.15.1:0/4182557601 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc0023030 con 0x560e4191c370 2024-09-16T00:38:38.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.779+0000 7fccad7fa640 1 -- 172.21.15.1:0/4182557601 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0010b30 con 0x560e4191c370 2024-09-16T00:38:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 -- 172.21.15.1:0/4182557601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 msgr2=0x560e41914120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 --2- 172.21.15.1:0/4182557601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e41914120 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fccc0003e70 tx=0x7fccc0014000 comp rx=0 tx=0).stop 2024-09-16T00:38:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 -- 172.21.15.1:0/4182557601 shutdown_connections 2024-09-16T00:38:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 --2- 172.21.15.1:0/4182557601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e4191c370 0x560e41914120 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 --2- 172.21.15.1:0/4182557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e41912b40 0x560e4190cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 --2- 172.21.15.1:0/4182557601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41917f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 -- 172.21.15.1:0/4182557601 >> 172.21.15.1:0/4182557601 conn(0x560e418da310 msgr2=0x560e4190c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 -- 172.21.15.1:0/4182557601 shutdown_connections 2024-09-16T00:38:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 -- 172.21.15.1:0/4182557601 wait complete. 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 -- start start 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.780+0000 7fccd023e740 1 Processor -- start 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccd023e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41924ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccd023e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e41912b40 0x560e41915c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccd023e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 0x560e41916140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccd023e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81af490 con 0x560e41912b40 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccd023e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81b0900 con 0x560e4191c370 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccd023e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81afb60 con 0x560e419115d0 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccd023e740 10 client.0 _ll_register_callbacks cb 0x7fccb8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e41912b40 0x560e41915c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e41912b40 0x560e41915c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43910/0 (socket says 172.21.15.1:43910) 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc6ffd640 1 -- 172.21.15.1:0/3592029937 learned_addr learned my addr 172.21.15.1:0/3592029937 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:38.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc77fe640 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41924ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc7fff640 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 0x560e41916140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc6ffd640 1 -- 172.21.15.1:0/3592029937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 msgr2=0x560e41924ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41924ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc6ffd640 1 -- 172.21.15.1:0/3592029937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 msgr2=0x560e41916140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 0x560e41916140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc6ffd640 1 -- 172.21.15.1:0/3592029937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e41916830 con 0x560e41912b40 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fccc6ffd640 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e41912b40 0x560e41915c00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00f010 tx=0x7fccbc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c0f08b18de09ca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.781+0000 7fcca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3592029937 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc014070 con 0x560e41912b40 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3592029937 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fccbc0046c0 con 0x560e41912b40 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccd023e740 1 -- 172.21.15.1:0/3592029937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc0023030 con 0x560e41912b40 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3592029937 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fccbc036050 con 0x560e41912b40 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccd023e740 10 client.17922 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccd023e740 1 -- 172.21.15.1:0/3592029937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e41996b90 con 0x560e41912b40 2024-09-16T00:38:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3592029937 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc0023030 con 0x560e41912b40 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3592029937 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fccbc02d070 con 0x560e41912b40 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccd023e740 10 client.17922 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccd023e740 10 client.17922 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccd023e740 20 client.17922 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccd023e740 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560e41997080 0x560e41999470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccd023e740 4 client.17922 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccd023e740 1 -- 172.21.15.1:0/3592029937 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e4187ed70 con 0x560e41997080 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.782+0000 7fccc77fe640 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560e41997080 0x560e41999470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccc77fe640 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560e41997080 0x560e41999470 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21b87d53277658e1 server_cookie=26ce63d054bcf53a in_seq=0 out_seq=0 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fcca6ffd640 10 client.17922 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fcca6ffd640 1 -- 172.21.15.1:0/3592029937 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e4187ed70 con 0x560e41997080 2024-09-16T00:38:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fcca6ffd640 10 client.17922 handle_command_reply: tid=0 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 client.17922 shutdown 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 -- 172.21.15.1:0/3592029937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e41912b40 msgr2=0x560e41915c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e41912b40 0x560e41915c00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00f010 tx=0x7fccbc00c870 comp rx=0 tx=0).stop 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 -- 172.21.15.1:0/3592029937 shutdown_connections 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e4191c370 0x560e41916140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560e41997080 0x560e41999470 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e41912b40 0x560e41915c00 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 --2- 172.21.15.1:0/3592029937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e419115d0 0x560e41924ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 -- 172.21.15.1:0/3592029937 >> 172.21.15.1:0/3592029937 conn(0x560e418da310 msgr2=0x560e4190cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fcca6ffd640 0 client.17922 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 -- 172.21.15.1:0/3592029937 shutdown_connections 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 1 -- 172.21.15.1:0/3592029937 wait complete. 2024-09-16T00:38:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.783+0000 7fccd023e740 20 client.17922 trim_cache size 0 max 16384 2024-09-16T00:38:38.790 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:38:38.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 -- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fccb00a3240 msgr2=0x7fccb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:38.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fccb00a3240 0x7fccb00a5700 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fccc00045e0 tx=0x7fccc00046c0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 -- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 msgr2=0x7fccc80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 0x7fccc80744f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fccb80097b0 tx=0x7fccb8031b10 comp rx=0 tx=0).stop 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 -- 172.21.15.1:0/1018402068 shutdown_connections 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fccb00a3240 0x7fccb00a5700 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccc8106f20 0x7fccc8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc81027a0 0x7fccc8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 --2- 172.21.15.1:0/1018402068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc8101df0 0x7fccc80744f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 -- 172.21.15.1:0/1018402068 >> 172.21.15.1:0/1018402068 conn(0x7fccc80fd8f0 msgr2=0x7fccc80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 -- 172.21.15.1:0/1018402068 shutdown_connections 2024-09-16T00:38:38.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:38.784+0000 7fcccde65640 1 -- 172.21.15.1:0/1018402068 wait complete. 2024-09-16T00:38:39.504 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:38:39.769 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:40.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:40 smithi001 ceph-mon[30934]: pgmap v658: 129 pgs: 129 active+clean; 4.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:38:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.306+0000 7fd11d20f640 1 -- 172.21.15.1:0/1091019758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118106f10 msgr2=0x7fd11810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.306+0000 7fd11d20f640 1 --2- 172.21.15.1:0/1091019758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118106f10 0x7fd11810ea90 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd110009ca0 tx=0x7fd11002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.306+0000 7fd11d20f640 1 -- 172.21.15.1:0/1091019758 shutdown_connections 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.306+0000 7fd11d20f640 1 --2- 172.21.15.1:0/1091019758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118106f10 0x7fd11810ea90 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.306+0000 7fd11d20f640 1 --2- 172.21.15.1:0/1091019758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd118102790 0x7fd1181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.306+0000 7fd11d20f640 1 --2- 172.21.15.1:0/1091019758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118101de0 0x7fd1181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.306+0000 7fd11d20f640 1 -- 172.21.15.1:0/1091019758 >> 172.21.15.1:0/1091019758 conn(0x7fd1180fd8c0 msgr2=0x7fd1180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.306+0000 7fd11d20f640 1 -- 172.21.15.1:0/1091019758 shutdown_connections 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.306+0000 7fd11d20f640 1 -- 172.21.15.1:0/1091019758 wait complete. 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11d20f640 1 Processor -- start 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11d20f640 1 -- start start 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11d20f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd118101de0 0x7fd1180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11d20f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118102790 0x7fd118074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11d20f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118106f10 0x7fd118074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11d20f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd118111f70 con 0x7fd118102790 2024-09-16T00:38:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11d20f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd118111df0 con 0x7fd118106f10 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11d20f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181120f0 con 0x7fd118101de0 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118102790 0x7fd118074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118102790 0x7fd118074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39808/0 (socket says 172.21.15.1:39808) 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd11659c640 1 -- 172.21.15.1:0/4218279120 learned_addr learned my addr 172.21.15.1:0/4218279120 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.307+0000 7fd116d9d640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd118101de0 0x7fd1180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11759e640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118106f10 0x7fd118074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11659c640 1 -- 172.21.15.1:0/4218279120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd118101de0 msgr2=0x7fd1180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11659c640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd118101de0 0x7fd1180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11659c640 1 -- 172.21.15.1:0/4218279120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118106f10 msgr2=0x7fd118074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11659c640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118106f10 0x7fd118074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11659c640 1 -- 172.21.15.1:0/4218279120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1181a67f0 con 0x7fd118102790 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11759e640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118106f10 0x7fd118074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11659c640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118102790 0x7fd118074a30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00c970 tx=0x7fd10c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd0fffff640 1 -- 172.21.15.1:0/4218279120 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c007bb0 con 0x7fd118102790 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11d20f640 1 -- 172.21.15.1:0/4218279120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1181a6ae0 con 0x7fd118102790 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd0fffff640 1 -- 172.21.15.1:0/4218279120 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd10c007d50 con 0x7fd118102790 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd0fffff640 1 -- 172.21.15.1:0/4218279120 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00f450 con 0x7fd118102790 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.308+0000 7fd11d20f640 1 -- 172.21.15.1:0/4218279120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1181a7020 con 0x7fd118102790 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd0fffff640 1 -- 172.21.15.1:0/4218279120 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd10c005020 con 0x7fd118102790 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd11f5e8740 1 Processor -- start 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd11f5e8740 1 -- start start 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 0x559a16f4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f4fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f5a370 0x559a16f5c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559a16eb52f0 con 0x559a16f4f5d0 2024-09-16T00:38:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a16e5ccf0 con 0x559a16f50b40 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559a16eb12b0 con 0x559a16f5a370 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f4fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f4fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39810/0 (socket says 172.21.15.1:39810) 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd116d9d640 1 -- 172.21.15.1:0/227184165 learned_addr learned my addr 172.21.15.1:0/227184165 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd0fffff640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1000a3240 0x7fd1000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd11759e640 1 --2- 172.21.15.1:0/227184165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 0x559a16f4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd116d9d640 1 -- 172.21.15.1:0/227184165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f5a370 msgr2=0x559a16f5c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd116d9d640 1 --2- 172.21.15.1:0/227184165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f5a370 0x559a16f5c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd116d9d640 1 -- 172.21.15.1:0/227184165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 msgr2=0x559a16f4f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd116d9d640 1 --2- 172.21.15.1:0/227184165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 0x559a16f4f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.311+0000 7fd116d9d640 1 -- 172.21.15.1:0/227184165 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a16e413a0 con 0x559a16f4f5d0 2024-09-16T00:38:40.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.312+0000 7fd11759e640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1000a3240 0x7fd1000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.312+0000 7fd0fffff640 1 -- 172.21.15.1:0/4218279120 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd10c0cfbb0 con 0x7fd118102790 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.312+0000 7fd116d9d640 1 --2- 172.21.15.1:0/227184165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f4fa10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd10800cc60 tx=0x7fd10800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedfec27c2969323 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.312+0000 7fd0fd7fa640 1 -- 172.21.15.1:0/227184165 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108014070 con 0x559a16f4f5d0 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.312+0000 7fd0fd7fa640 1 -- 172.21.15.1:0/227184165 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1080047a0 con 0x559a16f4f5d0 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.312+0000 7fd0fd7fa640 1 -- 172.21.15.1:0/227184165 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108010790 con 0x559a16f4f5d0 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.312+0000 7fd11f5e8740 1 -- 172.21.15.1:0/227184165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 msgr2=0x559a16f4fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.312+0000 7fd11759e640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1000a3240 0x7fd1000a5700 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd110005cc0 tx=0x7fd1100095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/227184165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f4fa10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd10800cc60 tx=0x7fd10800e720 comp rx=0 tx=0).stop 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 -- 172.21.15.1:0/227184165 shutdown_connections 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/227184165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f5a370 0x559a16f5c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/227184165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f4fa10 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/227184165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 0x559a16f4f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 -- 172.21.15.1:0/227184165 >> 172.21.15.1:0/227184165 conn(0x559a16f18310 msgr2=0x559a16f4bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:40.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 -- 172.21.15.1:0/227184165 shutdown_connections 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 -- 172.21.15.1:0/227184165 wait complete. 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 -- start start 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 Processor -- start 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f59e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 0x559a16f570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f5a370 0x559a16f57850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559a16f5d070 con 0x559a16f4f5d0 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a16e70060 con 0x559a16f5a370 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559a16f5d1f0 con 0x559a16f50b40 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.313+0000 7fd11f5e8740 10 client.0 _ll_register_callbacks cb 0x559a16bc38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f59e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f59e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39816/0 (socket says 172.21.15.1:39816) 2024-09-16T00:38:40.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd116d9d640 1 -- 172.21.15.1:0/1987750861 learned_addr learned my addr 172.21.15.1:0/1987750861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd11659c640 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 0x559a16f570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd11759e640 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f5a370 0x559a16f57850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd116d9d640 1 -- 172.21.15.1:0/1987750861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 msgr2=0x559a16f570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd116d9d640 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 0x559a16f570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd116d9d640 1 -- 172.21.15.1:0/1987750861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f5a370 msgr2=0x559a16f57850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd116d9d640 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f5a370 0x559a16f57850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd116d9d640 1 -- 172.21.15.1:0/1987750861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a16c263c0 con 0x559a16f4f5d0 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd11759e640 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f5a370 0x559a16f57850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd116d9d640 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f59e10 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd1080057c0 tx=0x7fd108010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48eee456d3e795e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:40.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd0faff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd0faff5640 1 -- 172.21.15.1:0/1987750861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108014070 con 0x559a16f4f5d0 2024-09-16T00:38:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd11f5e8740 1 -- 172.21.15.1:0/1987750861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1080047a0 con 0x559a16f4f5d0 2024-09-16T00:38:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.314+0000 7fd0faff5640 1 -- 172.21.15.1:0/1987750861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd108004020 con 0x559a16f4f5d0 2024-09-16T00:38:40.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd0faff5640 1 -- 172.21.15.1:0/1987750861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd10802c050 con 0x559a16f4f5d0 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd11f5e8740 10 client.17934 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd11f5e8740 1 -- 172.21.15.1:0/1987750861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a16f68bf0 con 0x559a16f4f5d0 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd0faff5640 1 -- 172.21.15.1:0/1987750861 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1080047a0 con 0x559a16f4f5d0 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd0faff5640 1 -- 172.21.15.1:0/1987750861 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd1080035f0 con 0x559a16f4f5d0 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd11f5e8740 10 client.17934 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd11f5e8740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd11f5e8740 20 client.17934 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x559a16f69ff0 0x559a16f6c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd11f5e8740 4 client.17934 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd11f5e8740 1 -- 172.21.15.1:0/1987750861 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a16ec2860 con 0x559a16f69ff0 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.315+0000 7fd11659c640 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x559a16f69ff0 0x559a16f6c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.316+0000 7fd11659c640 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x559a16f69ff0 0x559a16f6c3e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97822c7de867bd60 server_cookie=7ee35c7e060f4278 in_seq=0 out_seq=0 2024-09-16T00:38:40.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.316+0000 7fd0faff5640 10 client.17934 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.316+0000 7fd0faff5640 1 -- 172.21.15.1:0/1987750861 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a16ec2860 con 0x559a16f69ff0 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.316+0000 7fd0faff5640 10 client.17934 handle_command_reply: tid=0 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 client.17934 shutdown 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 -- 172.21.15.1:0/1987750861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 msgr2=0x559a16f59e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f59e10 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd1080057c0 tx=0x7fd108010cd0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 -- 172.21.15.1:0/1987750861 shutdown_connections 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x559a16f69ff0 0x559a16f6c3e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f5a370 0x559a16f57850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 0x559a16f570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd0faff5640 0 client.17934 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/1987750861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f59e10 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 -- 172.21.15.1:0/1987750861 >> 172.21.15.1:0/1987750861 conn(0x559a16f18310 msgr2=0x559a16f51930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 -- 172.21.15.1:0/1987750861 shutdown_connections 2024-09-16T00:38:40.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 1 -- 172.21.15.1:0/1987750861 wait complete. 2024-09-16T00:38:40.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.317+0000 7fd11f5e8740 20 client.17934 trim_cache size 0 max 16384 2024-09-16T00:38:40.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11f5e8740 1 Processor -- start 2024-09-16T00:38:40.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11f5e8740 1 -- start start 2024-09-16T00:38:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f4f5d0 0x559a16f4bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 0x559a16f560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f5a370 0x559a16f6f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181b38c0 con 0x559a16f5a370 2024-09-16T00:38:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181b1190 con 0x559a16f4f5d0 2024-09-16T00:38:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181aefe0 con 0x559a16f50b40 2024-09-16T00:38:40.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f5a370 0x559a16f6f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f5a370 0x559a16f6f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39824/0 (socket says 172.21.15.1:39824) 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11759e640 1 -- 172.21.15.1:0/2701854553 learned_addr learned my addr 172.21.15.1:0/2701854553 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11659c640 1 --2- 172.21.15.1:0/2701854553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 0x559a16f560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11759e640 1 -- 172.21.15.1:0/2701854553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 msgr2=0x559a16f560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11759e640 1 --2- 172.21.15.1:0/2701854553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 0x559a16f560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11759e640 1 -- 172.21.15.1:0/2701854553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f4f5d0 msgr2=0x559a16f4bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11759e640 1 --2- 172.21.15.1:0/2701854553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f4f5d0 0x559a16f4bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.330+0000 7fd11759e640 1 -- 172.21.15.1:0/2701854553 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a16f5d0f0 con 0x559a16f5a370 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd11759e640 1 --2- 172.21.15.1:0/2701854553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f5a370 0x559a16f6f400 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd110031e70 tx=0x7fd11003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849dc461911afc42 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd0fcff9640 1 -- 172.21.15.1:0/2701854553 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110003e00 con 0x559a16f5a370 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd0fcff9640 1 -- 172.21.15.1:0/2701854553 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd110036070 con 0x559a16f5a370 2024-09-16T00:38:40.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd0fcff9640 1 -- 172.21.15.1:0/2701854553 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11002faf0 con 0x559a16f5a370 2024-09-16T00:38:40.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd11f5e8740 1 -- 172.21.15.1:0/2701854553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f5a370 msgr2=0x559a16f6f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:40.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/2701854553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f5a370 0x559a16f6f400 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd110031e70 tx=0x7fd11003e000 comp rx=0 tx=0).stop 2024-09-16T00:38:40.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd11f5e8740 1 -- 172.21.15.1:0/2701854553 shutdown_connections 2024-09-16T00:38:40.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/2701854553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f5a370 0x559a16f6f400 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/2701854553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f50b40 0x559a16f560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/2701854553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f4f5d0 0x559a16f4bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.331+0000 7fd11f5e8740 1 -- 172.21.15.1:0/2701854553 >> 172.21.15.1:0/2701854553 conn(0x559a16f18310 msgr2=0x559a16f55990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 -- 172.21.15.1:0/2701854553 shutdown_connections 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 -- 172.21.15.1:0/2701854553 wait complete. 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 -- start start 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 Processor -- start 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f6faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 0x559a16f6cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f5a370 0x559a16f6d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181af650 con 0x559a16f4f5d0 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181b0ac0 con 0x559a16f50b40 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181afd20 con 0x559a16f5a370 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11f5e8740 10 client.0 _ll_register_callbacks cb 0x7fd10800cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f6faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f6faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39836/0 (socket says 172.21.15.1:39836) 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd116d9d640 1 -- 172.21.15.1:0/3671343737 learned_addr learned my addr 172.21.15.1:0/3671343737 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.332+0000 7fd11759e640 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f5a370 0x559a16f6d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd11659c640 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 0x559a16f6cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd116d9d640 1 -- 172.21.15.1:0/3671343737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f5a370 msgr2=0x559a16f6d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd116d9d640 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f5a370 0x559a16f6d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd116d9d640 1 -- 172.21.15.1:0/3671343737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 msgr2=0x559a16f6cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd116d9d640 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 0x559a16f6cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd116d9d640 1 -- 172.21.15.1:0/3671343737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a16f4b510 con 0x559a16f4f5d0 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd116d9d640 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f6faa0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd108005b90 tx=0x7fd10800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bb2ecd73106dd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd0faff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd0faff5640 1 -- 172.21.15.1:0/3671343737 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10801c030 con 0x559a16f4f5d0 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd0faff5640 1 -- 172.21.15.1:0/3671343737 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd108029070 con 0x559a16f4f5d0 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd11f5e8740 1 -- 172.21.15.1:0/3671343737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd110036070 con 0x559a16f4f5d0 2024-09-16T00:38:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd0faff5640 1 -- 172.21.15.1:0/3671343737 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd108039050 con 0x559a16f4f5d0 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd11f5e8740 10 client.17940 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.333+0000 7fd11f5e8740 1 -- 172.21.15.1:0/3671343737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a16fd46c0 con 0x559a16f4f5d0 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.334+0000 7fd0faff5640 1 -- 172.21.15.1:0/3671343737 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd110036070 con 0x559a16f4f5d0 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.334+0000 7fd0faff5640 1 -- 172.21.15.1:0/3671343737 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd1080042d0 con 0x559a16f4f5d0 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.334+0000 7fd11f5e8740 10 client.17940 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.334+0000 7fd11f5e8740 10 client.17940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.334+0000 7fd11f5e8740 20 client.17940 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.334+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x559a16fd4d80 0x559a16fd71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.334+0000 7fd11f5e8740 4 client.17940 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.334+0000 7fd11f5e8740 1 -- 172.21.15.1:0/3671343737 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a16ebcd70 con 0x559a16fd4d80 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.334+0000 7fd11659c640 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x559a16fd4d80 0x559a16fd71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11659c640 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x559a16fd4d80 0x559a16fd71a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51134d74d0389c95 server_cookie=f0f7af19e636b4d3 in_seq=0 out_seq=0 2024-09-16T00:38:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd0faff5640 10 client.17940 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd0faff5640 1 -- 172.21.15.1:0/3671343737 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a16ebcd70 con 0x559a16fd4d80 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd0faff5640 10 client.17940 handle_command_reply: tid=0 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 client.17940 shutdown 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 -- 172.21.15.1:0/3671343737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 msgr2=0x559a16f6faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f6faa0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd108005b90 tx=0x7fd10800c4d0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 -- 172.21.15.1:0/3671343737 shutdown_connections 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x559a16fd4d80 0x559a16fd71a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a16f5a370 0x559a16f6d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a16f50b40 0x559a16f6cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 --2- 172.21.15.1:0/3671343737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559a16f4f5d0 0x559a16f6faa0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd0faff5640 0 client.17940 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 -- 172.21.15.1:0/3671343737 >> 172.21.15.1:0/3671343737 conn(0x559a16f18310 msgr2=0x559a16f51110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 -- 172.21.15.1:0/3671343737 shutdown_connections 2024-09-16T00:38:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 1 -- 172.21.15.1:0/3671343737 wait complete. 2024-09-16T00:38:40.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.335+0000 7fd11f5e8740 20 client.17940 trim_cache size 0 max 16384 2024-09-16T00:38:40.341 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:38:40.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 -- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1000a3240 msgr2=0x7fd1000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1000a3240 0x7fd1000a5700 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd110005cc0 tx=0x7fd1100095c0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 -- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118102790 msgr2=0x7fd118074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118102790 0x7fd118074a30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00c970 tx=0x7fd10c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 -- 172.21.15.1:0/4218279120 shutdown_connections 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118106f10 0x7fd118074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd118102790 0x7fd118074a30 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd1000a3240 0x7fd1000a5700 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 --2- 172.21.15.1:0/4218279120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd118101de0 0x7fd1180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 -- 172.21.15.1:0/4218279120 >> 172.21.15.1:0/4218279120 conn(0x7fd1180fd8c0 msgr2=0x7fd1180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 -- 172.21.15.1:0/4218279120 shutdown_connections 2024-09-16T00:38:40.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:40.336+0000 7fd11d20f640 1 -- 172.21.15.1:0/4218279120 wait complete. 2024-09-16T00:38:40.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:40 smithi019 ceph-mon[35186]: pgmap v658: 129 pgs: 129 active+clean; 4.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:38:40.465 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:40 smithi153 ceph-mon[35538]: pgmap v658: 129 pgs: 129 active+clean; 4.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:38:42.138 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:38:42.138 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:38:42.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:42 smithi019 ceph-mon[35186]: pgmap v659: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:38:42.412 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:42.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:42 smithi001 ceph-mon[30934]: pgmap v659: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:38:42.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:42 smithi153 ceph-mon[35538]: pgmap v659: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-16T00:38:42.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.909+0000 7fae0a8c4640 1 -- 172.21.15.1:0/661362410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04106f00 msgr2=0x7fae0410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.909+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/661362410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04106f00 0x7fae0410ea80 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009900 tx=0x7fadf802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.909+0000 7fae0a8c4640 1 -- 172.21.15.1:0/661362410 shutdown_connections 2024-09-16T00:38:42.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.909+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/661362410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04106f00 0x7fae0410ea80 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.909+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/661362410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae04102780 0x7fae041067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.909+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/661362410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae04101dd0 0x7fae041021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.909+0000 7fae0a8c4640 1 -- 172.21.15.1:0/661362410 >> 172.21.15.1:0/661362410 conn(0x7fae040fd890 msgr2=0x7fae040ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:42.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.909+0000 7fae0a8c4640 1 -- 172.21.15.1:0/661362410 shutdown_connections 2024-09-16T00:38:42.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.909+0000 7fae0a8c4640 1 -- 172.21.15.1:0/661362410 wait complete. 2024-09-16T00:38:42.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.910+0000 7fae0a8c4640 1 Processor -- start 2024-09-16T00:38:42.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.910+0000 7fae0a8c4640 1 -- start start 2024-09-16T00:38:42.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.910+0000 7fae0a8c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae04101dd0 0x7fae04198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.910+0000 7fae0a8c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04102780 0x7fae041987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.910+0000 7fae0a8c4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae04106f00 0x7fae0419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.910+0000 7fae0a8c4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fae04111f60 con 0x7fae04102780 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.910+0000 7fae0a8c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae04111de0 con 0x7fae04101dd0 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.910+0000 7fae0a8c4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae041120e0 con 0x7fae04106f00 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.910+0000 7fae037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04102780 0x7fae041987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04102780 0x7fae041987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39864/0 (socket says 172.21.15.1:39864) 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae037fe640 1 -- 172.21.15.1:0/4096005042 learned_addr learned my addr 172.21.15.1:0/4096005042 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae08e61640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae04106f00 0x7fae0419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae03fff640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae04101dd0 0x7fae04198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae037fe640 1 -- 172.21.15.1:0/4096005042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae04106f00 msgr2=0x7fae0419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae037fe640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae04106f00 0x7fae0419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae037fe640 1 -- 172.21.15.1:0/4096005042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae04101dd0 msgr2=0x7fae04198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae037fe640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae04101dd0 0x7fae04198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae037fe640 1 -- 172.21.15.1:0/4096005042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0419d210 con 0x7fae04102780 2024-09-16T00:38:42.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae08e61640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae04106f00 0x7fae0419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:42.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae037fe640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04102780 0x7fae041987a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00cce0 tx=0x7fadfc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:42.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae017fa640 1 -- 172.21.15.1:0/4096005042 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00f650 con 0x7fae04102780 2024-09-16T00:38:42.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae0a8c4640 1 -- 172.21.15.1:0/4096005042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0419d500 con 0x7fae04102780 2024-09-16T00:38:42.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae017fa640 1 -- 172.21.15.1:0/4096005042 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadfc00fcd0 con 0x7fae04102780 2024-09-16T00:38:42.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae017fa640 1 -- 172.21.15.1:0/4096005042 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc018c70 con 0x7fae04102780 2024-09-16T00:38:42.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.911+0000 7fae0a8c4640 1 -- 172.21.15.1:0/4096005042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae041a4de0 con 0x7fae04102780 2024-09-16T00:38:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae017fa640 1 -- 172.21.15.1:0/4096005042 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fadfc00f7f0 con 0x7fae04102780 2024-09-16T00:38:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae0cc9d740 1 Processor -- start 2024-09-16T00:38:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae0cc9d740 1 -- start start 2024-09-16T00:38:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 0x55ea251cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 0x55ea251d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ea2512a280 con 0x55ea251c4b60 2024-09-16T00:38:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ea250d1b10 con 0x55ea251cfd40 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea25126240 con 0x55ea251c5480 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae08e61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae017fa640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fadec0a32c0 0x7fadec0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae08e61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39866/0 (socket says 172.21.15.1:39866) 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.914+0000 7fae08e61640 1 -- 172.21.15.1:0/3363918575 learned_addr learned my addr 172.21.15.1:0/3363918575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae03fff640 1 --2- 172.21.15.1:0/3363918575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 0x55ea251cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae017fa640 1 -- 172.21.15.1:0/4096005042 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fadfc0d01e0 con 0x7fae04102780 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae08e61640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fadec0a32c0 0x7fadec0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae037fe640 1 --2- 172.21.15.1:0/3363918575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 0x55ea251d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae08e61640 1 -- 172.21.15.1:0/3363918575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 msgr2=0x55ea251cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae08e61640 1 --2- 172.21.15.1:0/3363918575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 0x55ea251cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae08e61640 1 -- 172.21.15.1:0/3363918575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 msgr2=0x55ea251d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae08e61640 1 --2- 172.21.15.1:0/3363918575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 0x55ea251d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae08e61640 1 -- 172.21.15.1:0/3363918575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea250de1f0 con 0x55ea251c4b60 2024-09-16T00:38:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fae08e61640 1 --2- 172.21.15.1:0/3363918575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c4f40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005940 tx=0x7fadf80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eea6b31449faef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fadea7fc640 1 -- 172.21.15.1:0/3363918575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8035070 con 0x55ea251c4b60 2024-09-16T00:38:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fadea7fc640 1 -- 172.21.15.1:0/3363918575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadf804c070 con 0x55ea251c4b60 2024-09-16T00:38:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.915+0000 7fadea7fc640 1 -- 172.21.15.1:0/3363918575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf803dce0 con 0x55ea251c4b60 2024-09-16T00:38:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae08e61640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fadec0a32c0 0x7fadec0a5780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fadf804b000 tx=0x7fadf8049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3363918575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 msgr2=0x55ea251c4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3363918575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c4f40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005940 tx=0x7fadf80040c0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3363918575 shutdown_connections 2024-09-16T00:38:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3363918575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 0x55ea251d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3363918575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 0x55ea251cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3363918575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c4f40 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3363918575 >> 172.21.15.1:0/3363918575 conn(0x55ea251c10f0 msgr2=0x55ea251c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3363918575 shutdown_connections 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.916+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3363918575 wait complete. 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae0cc9d740 1 -- start start 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae0cc9d740 1 Processor -- start 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c4b60 0x55ea251ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c5480 0x55ea251cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251cfd40 0x55ea251cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ea251d2910 con 0x55ea251cfd40 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ea250e4f10 con 0x55ea251c5480 2024-09-16T00:38:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea251d2a90 con 0x55ea251c4b60 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae0cc9d740 10 client.0 _ll_register_callbacks cb 0x55ea250ada40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae08e61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251cfd40 0x55ea251cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae08e61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251cfd40 0x55ea251cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39870/0 (socket says 172.21.15.1:39870) 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae08e61640 1 -- 172.21.15.1:0/3507846214 learned_addr learned my addr 172.21.15.1:0/3507846214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae03fff640 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c4b60 0x55ea251ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.917+0000 7fae037fe640 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c5480 0x55ea251cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fae08e61640 1 -- 172.21.15.1:0/3507846214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c4b60 msgr2=0x55ea251ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fae08e61640 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c4b60 0x55ea251ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fae08e61640 1 -- 172.21.15.1:0/3507846214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c5480 msgr2=0x55ea251cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fae08e61640 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c5480 0x55ea251cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fae08e61640 1 -- 172.21.15.1:0/3507846214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea24e9b6a0 con 0x55ea251cfd40 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fae08e61640 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251cfd40 0x55ea251cd670 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fadf802a430 tx=0x7fadf8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f7d21e2028d04d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fade37fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fade37fe640 1 -- 172.21.15.1:0/3507846214 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8035070 con 0x55ea251cfd40 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fade37fe640 1 -- 172.21.15.1:0/3507846214 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fadf803d460 con 0x55ea251cfd40 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3507846214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf804c070 con 0x55ea251cfd40 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fade37fe640 1 -- 172.21.15.1:0/3507846214 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadf803dcd0 con 0x55ea251cfd40 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fae0cc9d740 10 client.17952 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3507846214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea251de4e0 con 0x55ea251cfd40 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.918+0000 7fade37fe640 1 -- 172.21.15.1:0/3507846214 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadf804c070 con 0x55ea251cfd40 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fade37fe640 1 -- 172.21.15.1:0/3507846214 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fadf803a960 con 0x55ea251cfd40 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fae0cc9d740 10 client.17952 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fae0cc9d740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fae0cc9d740 20 client.17952 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ea251df830 0x55ea251e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fae0cc9d740 4 client.17952 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3507846214 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea251377f0 con 0x55ea251df830 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fae03fff640 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ea251df830 0x55ea251e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fae03fff640 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ea251df830 0x55ea251e1c20 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f3c144225f107a server_cookie=e134a6870a72f12a in_seq=0 out_seq=0 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.919+0000 7fade37fe640 10 client.17952 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fade37fe640 1 -- 172.21.15.1:0/3507846214 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea251377f0 con 0x55ea251df830 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fade37fe640 10 client.17952 handle_command_reply: tid=0 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fae0cc9d740 1 client.17952 shutdown 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3507846214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251cfd40 msgr2=0x55ea251cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251cfd40 0x55ea251cd670 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fadf802a430 tx=0x7fadf8031400 comp rx=0 tx=0).stop 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3507846214 shutdown_connections 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ea251df830 0x55ea251e1c20 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251cfd40 0x55ea251cd670 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c5480 0x55ea251cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3507846214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c4b60 0x55ea251ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fade37fe640 0 client.17952 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:42.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.920+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3507846214 >> 172.21.15.1:0/3507846214 conn(0x55ea251c10f0 msgr2=0x55ea251c0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:42.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.921+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3507846214 shutdown_connections 2024-09-16T00:38:42.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.921+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3507846214 wait complete. 2024-09-16T00:38:42.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.921+0000 7fae0cc9d740 20 client.17952 trim_cache size 0 max 16384 2024-09-16T00:38:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae0cc9d740 1 Processor -- start 2024-09-16T00:38:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae0cc9d740 1 -- start start 2024-09-16T00:38:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c4b60 0x55ea251c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c5480 0x55ea251cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251cfd40 0x55ea251e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fae041b15e0 con 0x55ea251c5480 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae041aeeb0 con 0x55ea251c4b60 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae041acd00 con 0x55ea251cfd40 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c5480 0x55ea251cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c5480 0x55ea251cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39876/0 (socket says 172.21.15.1:39876) 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae037fe640 1 -- 172.21.15.1:0/3949388539 learned_addr learned my addr 172.21.15.1:0/3949388539 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae08e61640 1 --2- 172.21.15.1:0/3949388539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251cfd40 0x55ea251e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae037fe640 1 -- 172.21.15.1:0/3949388539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251cfd40 msgr2=0x55ea251e71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae03fff640 1 --2- 172.21.15.1:0/3949388539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c4b60 0x55ea251c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae037fe640 1 --2- 172.21.15.1:0/3949388539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251cfd40 0x55ea251e71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.938+0000 7fae037fe640 1 -- 172.21.15.1:0/3949388539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c4b60 msgr2=0x55ea251c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.939+0000 7fae037fe640 1 --2- 172.21.15.1:0/3949388539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c4b60 0x55ea251c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.939+0000 7fae037fe640 1 -- 172.21.15.1:0/3949388539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea251d2470 con 0x55ea251c5480 2024-09-16T00:38:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.939+0000 7fae037fe640 1 --2- 172.21.15.1:0/3949388539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c5480 0x55ea251cbaa0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fadfc098d70 tx=0x7fadfc098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d765e433e73e0077 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.939+0000 7fade9ffb640 1 -- 172.21.15.1:0/3949388539 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0b5070 con 0x55ea251c5480 2024-09-16T00:38:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.939+0000 7fade9ffb640 1 -- 172.21.15.1:0/3949388539 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadfc09c3a0 con 0x55ea251c5480 2024-09-16T00:38:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.939+0000 7fade9ffb640 1 -- 172.21.15.1:0/3949388539 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0b5210 con 0x55ea251c5480 2024-09-16T00:38:42.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.939+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3949388539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c5480 msgr2=0x55ea251cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.939+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3949388539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c5480 0x55ea251cbaa0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fadfc098d70 tx=0x7fadfc098040 comp rx=0 tx=0).stop 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3949388539 shutdown_connections 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3949388539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251cfd40 0x55ea251e71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3949388539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c5480 0x55ea251cbaa0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/3949388539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251c4b60 0x55ea251c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3949388539 >> 172.21.15.1:0/3949388539 conn(0x55ea251c10f0 msgr2=0x55ea251cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3949388539 shutdown_connections 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 -- 172.21.15.1:0/3949388539 wait complete. 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 -- start start 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 Processor -- start 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 0x55ea251ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 0x55ea251ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fae041ad370 con 0x55ea251c4b60 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae041ae7e0 con 0x55ea251cfd40 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.940+0000 7fae0cc9d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae041ada40 con 0x55ea251c5480 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae03fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae03fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39880/0 (socket says 172.21.15.1:39880) 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae03fff640 1 -- 172.21.15.1:0/2141269466 learned_addr learned my addr 172.21.15.1:0/2141269466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae0cc9d740 10 client.0 _ll_register_callbacks cb 0x7fadfc00cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae037fe640 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 0x55ea251ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae08e61640 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 0x55ea251ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae03fff640 1 -- 172.21.15.1:0/2141269466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 msgr2=0x55ea251ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae03fff640 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 0x55ea251ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae03fff640 1 -- 172.21.15.1:0/2141269466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 msgr2=0x55ea251ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae03fff640 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 0x55ea251ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae03fff640 1 -- 172.21.15.1:0/2141269466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea25249390 con 0x55ea251c4b60 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae037fe640 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 0x55ea251ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae08e61640 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 0x55ea251ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae03fff640 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c9e00 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fadf4005940 tx=0x7fadf400ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27daddc884f6f3a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fade37fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fade37fe640 1 -- 172.21.15.1:0/2141269466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf401c470 con 0x55ea251c4b60 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fade37fe640 1 -- 172.21.15.1:0/2141269466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fadf401c610 con 0x55ea251c4b60 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae0cc9d740 1 -- 172.21.15.1:0/2141269466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadfc09c3a0 con 0x55ea251c4b60 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fade37fe640 1 -- 172.21.15.1:0/2141269466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadf402c050 con 0x55ea251c4b60 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae0cc9d740 10 client.17958 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.941+0000 7fae0cc9d740 1 -- 172.21.15.1:0/2141269466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea2524a6f0 con 0x55ea251c4b60 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fade37fe640 1 -- 172.21.15.1:0/2141269466 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadfc09c3a0 con 0x55ea251c4b60 2024-09-16T00:38:42.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fade37fe640 1 -- 172.21.15.1:0/2141269466 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fadf4010b30 con 0x55ea251c4b60 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fae0cc9d740 10 client.17958 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fae0cc9d740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fae0cc9d740 20 client.17958 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ea2524ae50 0x55ea2524d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fae0cc9d740 4 client.17958 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fae0cc9d740 1 -- 172.21.15.1:0/2141269466 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ea25131d00 con 0x55ea2524ae50 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fae037fe640 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ea2524ae50 0x55ea2524d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fae037fe640 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ea2524ae50 0x55ea2524d270 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72354203c42ede8d server_cookie=6e21b962979eb72f in_seq=0 out_seq=0 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.942+0000 7fade37fe640 10 client.17958 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.943+0000 7fade37fe640 1 -- 172.21.15.1:0/2141269466 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ea25131d00 con 0x55ea2524ae50 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.943+0000 7fade37fe640 10 client.17958 handle_command_reply: tid=0 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.943+0000 7fae0cc9d740 1 client.17958 shutdown 2024-09-16T00:38:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 -- 172.21.15.1:0/2141269466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 msgr2=0x55ea251c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c9e00 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fadf4005940 tx=0x7fadf400ce70 comp rx=0 tx=0).stop 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 -- 172.21.15.1:0/2141269466 shutdown_connections 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ea2524ae50 0x55ea2524d270 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea251cfd40 0x55ea251ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea251c5480 0x55ea251ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 --2- 172.21.15.1:0/2141269466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea251c4b60 0x55ea251c9e00 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 -- 172.21.15.1:0/2141269466 >> 172.21.15.1:0/2141269466 conn(0x55ea251c10f0 msgr2=0x55ea251c0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fade37fe640 0 client.17958 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 -- 172.21.15.1:0/2141269466 shutdown_connections 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 1 -- 172.21.15.1:0/2141269466 wait complete. 2024-09-16T00:38:42.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.944+0000 7fae0cc9d740 20 client.17958 trim_cache size 0 max 16384 2024-09-16T00:38:42.950 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:38:42.950 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:38:42.950 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "8969c4cf-8b16-4e8d-9b30-42168f7622f8", 2024-09-16T00:38:42.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:38:42.950 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:38:42.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 -- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fadec0a32c0 msgr2=0x7fadec0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fadec0a32c0 0x7fadec0a5780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fadf804b000 tx=0x7fadf8049000 comp rx=0 tx=0).stop 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 -- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04102780 msgr2=0x7fae041987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04102780 0x7fae041987a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00cce0 tx=0x7fadfc00a620 comp rx=0 tx=0).stop 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 -- 172.21.15.1:0/4096005042 shutdown_connections 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fadec0a32c0 0x7fadec0a5780 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae04106f00 0x7fae0419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae04102780 0x7fae041987a0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 --2- 172.21.15.1:0/4096005042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae04101dd0 0x7fae04198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 -- 172.21.15.1:0/4096005042 >> 172.21.15.1:0/4096005042 conn(0x7fae040fd890 msgr2=0x7fae040ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 -- 172.21.15.1:0/4096005042 shutdown_connections 2024-09-16T00:38:42.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:42.945+0000 7fae0a8c4640 1 -- 172.21.15.1:0/4096005042 wait complete. 2024-09-16T00:38:43.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:43 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:38:43.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:43 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:43.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:43 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:38:43.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:43 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:43.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:43 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:38:43.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:43 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:38:43.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:43 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:43.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:43 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:38:43.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:43 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:43.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:43 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:38:43.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:43 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:38:43.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:43 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:43.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:43 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:38:43.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:43 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:43.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:43 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:38:43.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:38:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:44.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:44 smithi019 ceph-mon[35186]: pgmap v660: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-16T00:38:44.366 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:44 smithi001 ceph-mon[30934]: pgmap v660: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-16T00:38:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:44 smithi153 ceph-mon[35538]: pgmap v660: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-16T00:38:44.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.599+0000 7f4f442fc640 1 -- 172.21.15.1:0/3188499497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 msgr2=0x7f4f3c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.599+0000 7f4f442fc640 1 --2- 172.21.15.1:0/3188499497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 0x7f4f3c100ee0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009930 tx=0x7f4f2c02a420 comp rx=0 tx=0).stop 2024-09-16T00:38:44.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.599+0000 7f4f442fc640 1 -- 172.21.15.1:0/3188499497 shutdown_connections 2024-09-16T00:38:44.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.599+0000 7f4f442fc640 1 --2- 172.21.15.1:0/3188499497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f3c101420 0x7f4f3c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.599+0000 7f4f442fc640 1 --2- 172.21.15.1:0/3188499497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 0x7f4f3c100ee0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.599+0000 7f4f442fc640 1 --2- 172.21.15.1:0/3188499497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f3c106a80 0x7f4f3c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.599+0000 7f4f442fc640 1 -- 172.21.15.1:0/3188499497 >> 172.21.15.1:0/3188499497 conn(0x7f4f3c0fc7a0 msgr2=0x7f4f3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.599+0000 7f4f442fc640 1 -- 172.21.15.1:0/3188499497 shutdown_connections 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.599+0000 7f4f442fc640 1 -- 172.21.15.1:0/3188499497 wait complete. 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f442fc640 1 Processor -- start 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f442fc640 1 -- start start 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f442fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 0x7f4f3c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f442fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f3c101420 0x7f4f3c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f442fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f3c106a80 0x7f4f3c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f442fc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c110e80 con 0x7f4f3c100a80 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f442fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c110d00 con 0x7f4f3c106a80 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f442fc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c111000 con 0x7f4f3c101420 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f42098640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 0x7f4f3c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f42098640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 0x7f4f3c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39908/0 (socket says 172.21.15.1:39908) 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f42098640 1 -- 172.21.15.1:0/367030448 learned_addr learned my addr 172.21.15.1:0/367030448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:44.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f42899640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f3c106a80 0x7f4f3c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.600+0000 7f4f41897640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f3c101420 0x7f4f3c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f42098640 1 -- 172.21.15.1:0/367030448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f3c101420 msgr2=0x7f4f3c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f42098640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f3c101420 0x7f4f3c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f42098640 1 -- 172.21.15.1:0/367030448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f3c106a80 msgr2=0x7f4f3c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f42098640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f3c106a80 0x7f4f3c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f42098640 1 -- 172.21.15.1:0/367030448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c19f630 con 0x7f4f3c100a80 2024-09-16T00:38:44.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f41897640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f3c101420 0x7f4f3c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:44.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f42098640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 0x7f4f3c19a680 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800c890 tx=0x7f4f3800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f337fe640 1 -- 172.21.15.1:0/367030448 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38004220 con 0x7f4f3c100a80 2024-09-16T00:38:44.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f442fc640 1 -- 172.21.15.1:0/367030448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c19f920 con 0x7f4f3c100a80 2024-09-16T00:38:44.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f337fe640 1 -- 172.21.15.1:0/367030448 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f380043c0 con 0x7f4f3c100a80 2024-09-16T00:38:44.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f337fe640 1 -- 172.21.15.1:0/367030448 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38004a00 con 0x7f4f3c100a80 2024-09-16T00:38:44.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.601+0000 7f4f442fc640 1 -- 172.21.15.1:0/367030448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c1a7200 con 0x7f4f3c100a80 2024-09-16T00:38:44.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.603+0000 7f4f337fe640 1 -- 172.21.15.1:0/367030448 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4f380079b0 con 0x7f4f3c100a80 2024-09-16T00:38:44.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f466d5740 1 Processor -- start 2024-09-16T00:38:44.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f466d5740 1 -- start start 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab54a0 0x56287bab5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 0x56287bac2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f466d5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56287ba1b2e0 con 0x56287bab5dc0 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f466d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56287b9c2ce0 con 0x56287bab54a0 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f466d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56287ba172a0 con 0x56287bac0680 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f42098640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f42098640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39916/0 (socket says 172.21.15.1:39916) 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f42098640 1 -- 172.21.15.1:0/2121812428 learned_addr learned my addr 172.21.15.1:0/2121812428 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:44.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.604+0000 7f4f42899640 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab54a0 0x56287bab5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f337fe640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4f280a3240 0x7f4f280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f41897640 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 0x56287bac2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f42098640 1 -- 172.21.15.1:0/2121812428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 msgr2=0x56287bac2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f42098640 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 0x56287bac2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f42098640 1 -- 172.21.15.1:0/2121812428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab54a0 msgr2=0x56287bab5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f42098640 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab54a0 0x56287bab5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f42098640 1 -- 172.21.15.1:0/2121812428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56287b9a7440 con 0x56287bab5dc0 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f41897640 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 0x56287bac2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f42899640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4f280a3240 0x7f4f280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f337fe640 1 -- 172.21.15.1:0/367030448 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4f380cf9a0 con 0x7f4f3c100a80 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f42098640 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac0140 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4f38077700 tx=0x7f4f38066600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc2a2aebf850f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f30dea640 1 -- 172.21.15.1:0/2121812428 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3809b700 con 0x56287bab5dc0 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f30dea640 1 -- 172.21.15.1:0/2121812428 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38066670 con 0x56287bab5dc0 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f42899640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4f280a3240 0x7f4f280a5700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4f34005fd0 tx=0x7f4f34004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.605+0000 7f4f30dea640 1 -- 172.21.15.1:0/2121812428 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3809b520 con 0x56287bab5dc0 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 -- 172.21.15.1:0/2121812428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 msgr2=0x56287bac0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac0140 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4f38077700 tx=0x7f4f38066600 comp rx=0 tx=0).stop 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 -- 172.21.15.1:0/2121812428 shutdown_connections 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 0x56287bac2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac0140 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2121812428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab54a0 0x56287bab5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 -- 172.21.15.1:0/2121812428 >> 172.21.15.1:0/2121812428 conn(0x56287bab1a30 msgr2=0x56287bab1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 -- 172.21.15.1:0/2121812428 shutdown_connections 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 -- 172.21.15.1:0/2121812428 wait complete. 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.606+0000 7f4f466d5740 1 -- start start 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f466d5740 1 Processor -- start 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab54a0 0x56287babffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab5dc0 0x56287babd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 0x56287babd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f466d5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56287bac3250 con 0x56287bab54a0 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f466d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56287b9d6050 con 0x56287bab5dc0 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f466d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56287bac33d0 con 0x56287bac0680 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f466d5740 10 client.0 _ll_register_callbacks cb 0x56287ba16160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f42098640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab54a0 0x56287babffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f42098640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab54a0 0x56287babffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39922/0 (socket says 172.21.15.1:39922) 2024-09-16T00:38:44.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f42098640 1 -- 172.21.15.1:0/2066493266 learned_addr learned my addr 172.21.15.1:0/2066493266 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:44.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f42098640 1 -- 172.21.15.1:0/2066493266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 msgr2=0x56287babd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:44.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f42098640 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 0x56287babd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f41897640 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab5dc0 0x56287babd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f42098640 1 -- 172.21.15.1:0/2066493266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab5dc0 msgr2=0x56287babd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:44.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f42098640 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab5dc0 0x56287babd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f42098640 1 -- 172.21.15.1:0/2066493266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56287b78c3c0 con 0x56287bab54a0 2024-09-16T00:38:44.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f42098640 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab54a0 0x56287babffe0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4f38066830 tx=0x7f4f380076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a65e2bd647b3211 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.607+0000 7f4f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:44.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2066493266 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3801d040 con 0x56287bab54a0 2024-09-16T00:38:44.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2066493266 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4f38099c60 con 0x56287bab54a0 2024-09-16T00:38:44.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f466d5740 1 -- 172.21.15.1:0/2066493266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56287ba28850 con 0x56287bab54a0 2024-09-16T00:38:44.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2066493266 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f3809b700 con 0x56287bab54a0 2024-09-16T00:38:44.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f466d5740 10 client.17970 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:44.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f466d5740 1 -- 172.21.15.1:0/2066493266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56287baceb90 con 0x56287bab54a0 2024-09-16T00:38:44.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2066493266 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56287ba28850 con 0x56287bab54a0 2024-09-16T00:38:44.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2066493266 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4f38007ca0 con 0x56287bab54a0 2024-09-16T00:38:44.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f466d5740 10 client.17970 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:44.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f466d5740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:44.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f466d5740 20 client.17970 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:44.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56287bacfee0 0x56287bad22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f466d5740 4 client.17970 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:44.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.608+0000 7f4f466d5740 1 -- 172.21.15.1:0/2066493266 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56287ba22d60 con 0x56287bacfee0 2024-09-16T00:38:44.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.609+0000 7f4f41897640 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56287bacfee0 0x56287bad22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.609+0000 7f4f25ffb640 10 client.17970 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:44.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.609+0000 7f4f41897640 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56287bacfee0 0x56287bad22d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e21bdca79e0e330 server_cookie=fd905690a8090f25 in_seq=0 out_seq=0 2024-09-16T00:38:44.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2066493266 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56287ba22d60 con 0x56287bacfee0 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f25ffb640 10 client.17970 handle_command_reply: tid=0 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 client.17970 shutdown 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 -- 172.21.15.1:0/2066493266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab54a0 msgr2=0x56287babffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab54a0 0x56287babffe0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4f38066830 tx=0x7f4f380076c0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 -- 172.21.15.1:0/2066493266 shutdown_connections 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56287bacfee0 0x56287bad22d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bac0680 0x56287babd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bab5dc0 0x56287babd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2066493266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab54a0 0x56287babffe0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 -- 172.21.15.1:0/2066493266 >> 172.21.15.1:0/2066493266 conn(0x56287bab1a30 msgr2=0x56287bab7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f25ffb640 0 client.17970 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 -- 172.21.15.1:0/2066493266 shutdown_connections 2024-09-16T00:38:44.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 1 -- 172.21.15.1:0/2066493266 wait complete. 2024-09-16T00:38:44.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.610+0000 7f4f466d5740 20 client.17970 trim_cache size 0 max 16384 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.627+0000 7f4f466d5740 1 Processor -- start 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f466d5740 1 -- start start 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bab5dc0 0x56287bacc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 0x56287bad78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bad7e10 0x56287bab8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f466d5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1b39b0 con 0x56287bad7e10 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f466d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1b1280 con 0x56287bac0680 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f466d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1af0d0 con 0x56287bab5dc0 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42899640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bad7e10 0x56287bab8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42899640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bad7e10 0x56287bab8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39926/0 (socket says 172.21.15.1:39926) 2024-09-16T00:38:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42899640 1 -- 172.21.15.1:0/3202589606 learned_addr learned my addr 172.21.15.1:0/3202589606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42098640 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bab5dc0 0x56287bacc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f41897640 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 0x56287bad78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42899640 1 -- 172.21.15.1:0/3202589606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bab5dc0 msgr2=0x56287bacc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42899640 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bab5dc0 0x56287bacc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42899640 1 -- 172.21.15.1:0/3202589606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 msgr2=0x56287bad78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42899640 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 0x56287bad78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42899640 1 -- 172.21.15.1:0/3202589606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56287bab5220 con 0x56287bad7e10 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f41897640 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 0x56287bad78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.628+0000 7f4f42899640 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bad7e10 0x56287bab8030 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009f50 tx=0x7f4f34015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6149fd9a7c2b0a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f27fff640 1 -- 172.21.15.1:0/3202589606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34012dd0 con 0x56287bad7e10 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f27fff640 1 -- 172.21.15.1:0/3202589606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f34025020 con 0x56287bad7e10 2024-09-16T00:38:44.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f27fff640 1 -- 172.21.15.1:0/3202589606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34010830 con 0x56287bad7e10 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f466d5740 1 -- 172.21.15.1:0/3202589606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bad7e10 msgr2=0x56287bab8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f466d5740 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bad7e10 0x56287bab8030 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009f50 tx=0x7f4f34015040 comp rx=0 tx=0).stop 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f466d5740 1 -- 172.21.15.1:0/3202589606 shutdown_connections 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f466d5740 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bad7e10 0x56287bab8030 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f466d5740 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 0x56287bad78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f466d5740 1 --2- 172.21.15.1:0/3202589606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bab5dc0 0x56287bacc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.629+0000 7f4f466d5740 1 -- 172.21.15.1:0/3202589606 >> 172.21.15.1:0/3202589606 conn(0x56287bab1a30 msgr2=0x56287babc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 -- 172.21.15.1:0/3202589606 shutdown_connections 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 -- 172.21.15.1:0/3202589606 wait complete. 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 -- start start 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 Processor -- start 2024-09-16T00:38:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 0x56287bac3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bad7e10 0x56287bac43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1af740 con 0x56287bab5dc0 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1b0bb0 con 0x56287bac0680 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1afe10 con 0x56287bad7e10 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f466d5740 10 client.0 _ll_register_callbacks cb 0x7f4f34009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.630+0000 7f4f42098640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42098640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39942/0 (socket says 172.21.15.1:39942) 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42098640 1 -- 172.21.15.1:0/2169837595 learned_addr learned my addr 172.21.15.1:0/2169837595 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42899640 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bad7e10 0x56287bac43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f41897640 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 0x56287bac3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42098640 1 -- 172.21.15.1:0/2169837595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bad7e10 msgr2=0x56287bac43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42098640 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bad7e10 0x56287bac43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42098640 1 -- 172.21.15.1:0/2169837595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 msgr2=0x56287bac3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42098640 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 0x56287bac3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42098640 1 -- 172.21.15.1:0/2169837595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56287bac4990 con 0x56287bab5dc0 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42899640 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bad7e10 0x56287bac43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f42098640 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac8b10 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f4f38098020 tx=0x7f4f3800a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4139f43f8455de server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2169837595 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3801d040 con 0x56287bab5dc0 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2169837595 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4f3809be20 con 0x56287bab5dc0 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f466d5740 1 -- 172.21.15.1:0/2169837595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f34025020 con 0x56287bab5dc0 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2169837595 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f380a4880 con 0x56287bab5dc0 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f466d5740 10 client.17976 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.631+0000 7f4f466d5740 1 -- 172.21.15.1:0/2169837595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56287bb3a640 con 0x56287bab5dc0 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2169837595 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f34025020 con 0x56287bab5dc0 2024-09-16T00:38:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2169837595 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4f38086110 con 0x56287bab5dc0 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f466d5740 10 client.17976 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f466d5740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f466d5740 20 client.17976 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56287bb3ab70 0x56287bb3cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f466d5740 4 client.17976 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f466d5740 1 -- 172.21.15.1:0/2169837595 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56287b99f210 con 0x56287bb3ab70 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f41897640 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56287bb3ab70 0x56287bb3cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f41897640 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56287bb3ab70 0x56287bb3cf60 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a6f2a0c06a6a83c server_cookie=7eb41f44cc132328 in_seq=0 out_seq=0 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.632+0000 7f4f25ffb640 10 client.17976 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f25ffb640 1 -- 172.21.15.1:0/2169837595 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56287b99f210 con 0x56287bb3ab70 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f25ffb640 10 client.17976 handle_command_reply: tid=0 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 client.17976 shutdown 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 -- 172.21.15.1:0/2169837595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 msgr2=0x56287bac8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac8b10 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f4f38098020 tx=0x7f4f3800a5f0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 -- 172.21.15.1:0/2169837595 shutdown_connections 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56287bb3ab70 0x56287bb3cf60 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56287bad7e10 0x56287bac43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56287bac0680 0x56287bac3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 --2- 172.21.15.1:0/2169837595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56287bab5dc0 0x56287bac8b10 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 -- 172.21.15.1:0/2169837595 >> 172.21.15.1:0/2169837595 conn(0x56287bab1a30 msgr2=0x56287bab0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f25ffb640 0 client.17976 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 -- 172.21.15.1:0/2169837595 shutdown_connections 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 1 -- 172.21.15.1:0/2169837595 wait complete. 2024-09-16T00:38:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.633+0000 7f4f466d5740 20 client.17976 trim_cache size 0 max 16384 2024-09-16T00:38:44.640 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:38:44.640 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:38:44.640 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:38:44.640 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:38:44.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 -- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4f280a3240 msgr2=0x7f4f280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4f280a3240 0x7f4f280a5700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4f34005fd0 tx=0x7f4f34004a40 comp rx=0 tx=0).stop 2024-09-16T00:38:44.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 -- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 msgr2=0x7f4f3c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 0x7f4f3c19a680 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800c890 tx=0x7f4f3800cd50 comp rx=0 tx=0).stop 2024-09-16T00:38:44.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 -- 172.21.15.1:0/367030448 shutdown_connections 2024-09-16T00:38:44.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f3c106a80 0x7f4f3c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4f280a3240 0x7f4f280a5700 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f3c101420 0x7f4f3c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 --2- 172.21.15.1:0/367030448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f3c100a80 0x7f4f3c19a680 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 -- 172.21.15.1:0/367030448 >> 172.21.15.1:0/367030448 conn(0x7f4f3c0fc7a0 msgr2=0x7f4f3c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 -- 172.21.15.1:0/367030448 shutdown_connections 2024-09-16T00:38:44.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:44.634+0000 7f4f442fc640 1 -- 172.21.15.1:0/367030448 wait complete. 2024-09-16T00:38:45.443 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:38:45.443 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:46.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 -- 172.21.15.1:0/3123556834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c06a370 msgr2=0x7f2b0c1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 --2- 172.21.15.1:0/3123556834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c06a370 0x7f2b0c1009c0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009900 tx=0x7f2afc02a410 comp rx=0 tx=0).stop 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 -- 172.21.15.1:0/3123556834 shutdown_connections 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 --2- 172.21.15.1:0/3123556834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b0c104e80 0x7f2b0c105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 --2- 172.21.15.1:0/3123556834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c06a370 0x7f2b0c1009c0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 --2- 172.21.15.1:0/3123556834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c069f00 0x7f2b0c100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 -- 172.21.15.1:0/3123556834 >> 172.21.15.1:0/3123556834 conn(0x7f2b0c077210 msgr2=0x7f2b0c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 -- 172.21.15.1:0/3123556834 shutdown_connections 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 -- 172.21.15.1:0/3123556834 wait complete. 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 Processor -- start 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.213+0000 7f2b13858640 1 -- start start 2024-09-16T00:38:46.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b13858640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c069f00 0x7f2b0c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b13858640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c06a370 0x7f2b0c1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b13858640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b0c104e80 0x7f2b0c103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b13858640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c110eb0 con 0x7f2b0c069f00 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b13858640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c110d30 con 0x7f2b0c06a370 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b13858640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c111030 con 0x7f2b0c104e80 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b115f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c069f00 0x7f2b0c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b115f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c069f00 0x7f2b0c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39960/0 (socket says 172.21.15.1:39960) 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b115f4640 1 -- 172.21.15.1:0/2943326931 learned_addr learned my addr 172.21.15.1:0/2943326931 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b10df3640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c06a370 0x7f2b0c1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b11df5640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b0c104e80 0x7f2b0c103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:46.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b115f4640 1 -- 172.21.15.1:0/2943326931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b0c104e80 msgr2=0x7f2b0c103600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:46.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b115f4640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b0c104e80 0x7f2b0c103600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:46.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b115f4640 1 -- 172.21.15.1:0/2943326931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c06a370 msgr2=0x7f2b0c1030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:46.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b115f4640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c06a370 0x7f2b0c1030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:46.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b115f4640 1 -- 172.21.15.1:0/2943326931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c1a6930 con 0x7f2b0c069f00 2024-09-16T00:38:46.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b11df5640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b0c104e80 0x7f2b0c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:46.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.214+0000 7f2b115f4640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c069f00 0x7f2b0c102b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800d950 tx=0x7f2b0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:46.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.215+0000 7f2b027fc640 1 -- 172.21.15.1:0/2943326931 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08017070 con 0x7f2b0c069f00 2024-09-16T00:38:46.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.215+0000 7f2b027fc640 1 -- 172.21.15.1:0/2943326931 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b08009c30 con 0x7f2b0c069f00 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.215+0000 7f2b027fc640 1 -- 172.21.15.1:0/2943326931 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800f6b0 con 0x7f2b0c069f00 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.215+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c1a6c20 con 0x7f2b0c069f00 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.215+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c1a71b0 con 0x7f2b0c069f00 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.216+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0c105330 con 0x7f2b0c069f00 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.217+0000 7f2b027fc640 1 -- 172.21.15.1:0/2943326931 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2b080109d0 con 0x7f2b0c069f00 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.218+0000 7f2b027fc640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2af80a32c0 0x7f2af80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.218+0000 7f2b10df3640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2af80a32c0 0x7f2af80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.218+0000 7f2b027fc640 1 -- 172.21.15.1:0/2943326931 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2b080cf950 con 0x7f2b0c069f00 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.218+0000 7f2b10df3640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2af80a32c0 0x7f2af80a5780 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c104610 tx=0x7f2afc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:46.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.222+0000 7f2b027fc640 1 -- 172.21.15.1:0/2943326931 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b08009de0 con 0x7f2b0c069f00 2024-09-16T00:38:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:46 smithi019 ceph-mon[35186]: pgmap v661: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-16T00:38:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.432+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b0c1040b0 con 0x7f2b0c069f00 2024-09-16T00:38:46.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:46 smithi001 ceph-mon[30934]: pgmap v661: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-16T00:38:46.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.433+0000 7f2b027fc640 1 -- 172.21.15.1:0/2943326931 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f2b0809c270 con 0x7f2b0c069f00 2024-09-16T00:38:46.434 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:38:46.435 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:38:46.435 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:38:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2af80a32c0 msgr2=0x7f2af80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2af80a32c0 0x7f2af80a5780 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c104610 tx=0x7f2afc005a20 comp rx=0 tx=0).stop 2024-09-16T00:38:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c069f00 msgr2=0x7f2b0c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c069f00 0x7f2b0c102b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800d950 tx=0x7f2b0800de10 comp rx=0 tx=0).stop 2024-09-16T00:38:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 shutdown_connections 2024-09-16T00:38:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2af80a32c0 0x7f2af80a5780 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b0c104e80 0x7f2b0c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c06a370 0x7f2b0c1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 --2- 172.21.15.1:0/2943326931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b0c069f00 0x7f2b0c102b80 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 >> 172.21.15.1:0/2943326931 conn(0x7f2b0c077210 msgr2=0x7f2b0c0fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 shutdown_connections 2024-09-16T00:38:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:46.436+0000 7f2b13858640 1 -- 172.21.15.1:0/2943326931 wait complete. 2024-09-16T00:38:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:46 smithi153 ceph-mon[35538]: pgmap v661: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-16T00:38:47.111 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:38:47.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:47 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2943326931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:47.374 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:47.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:47 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2943326931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:47.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:47 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2943326931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.894+0000 7fc998c0e640 1 -- 172.21.15.1:0/2557182484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994101420 msgr2=0x7fc99410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.894+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2557182484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994101420 0x7fc99410d950 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009cd0 tx=0x7fc98c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.894+0000 7fc998c0e640 1 -- 172.21.15.1:0/2557182484 shutdown_connections 2024-09-16T00:38:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.894+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2557182484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994101420 0x7fc99410d950 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.894+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2557182484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc994100a80 0x7fc994100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.894+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2557182484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc994106a80 0x7fc994106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.894+0000 7fc998c0e640 1 -- 172.21.15.1:0/2557182484 >> 172.21.15.1:0/2557182484 conn(0x7fc9940fc7a0 msgr2=0x7fc9940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.894+0000 7fc998c0e640 1 -- 172.21.15.1:0/2557182484 shutdown_connections 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.894+0000 7fc998c0e640 1 -- 172.21.15.1:0/2557182484 wait complete. 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc998c0e640 1 Processor -- start 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc998c0e640 1 -- start start 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc998c0e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994100a80 0x7fc994198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc998c0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc994101420 0x7fc994198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc998c0e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc994106a80 0x7fc99419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc998c0e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc994110e80 con 0x7fc994100a80 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc998c0e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc994110d00 con 0x7fc994101420 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc998c0e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc994111000 con 0x7fc994106a80 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc99259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994100a80 0x7fc994198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc99259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994100a80 0x7fc994198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39976/0 (socket says 172.21.15.1:39976) 2024-09-16T00:38:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc99259c640 1 -- 172.21.15.1:0/2482149411 learned_addr learned my addr 172.21.15.1:0/2482149411 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc992d9d640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc994106a80 0x7fc99419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.895+0000 7fc989d9b640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc994101420 0x7fc994198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc99259c640 1 -- 172.21.15.1:0/2482149411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc994106a80 msgr2=0x7fc99419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc99259c640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc994106a80 0x7fc99419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc99259c640 1 -- 172.21.15.1:0/2482149411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc994101420 msgr2=0x7fc994198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc99259c640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc994101420 0x7fc994198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc99259c640 1 -- 172.21.15.1:0/2482149411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99419d3e0 con 0x7fc994100a80 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc989d9b640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc994101420 0x7fc994198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc99259c640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994100a80 0x7fc994198430 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc98400d770 tx=0x7fc98400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc98bfff640 1 -- 172.21.15.1:0/2482149411 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984017070 con 0x7fc994100a80 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc998c0e640 1 -- 172.21.15.1:0/2482149411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99419d6d0 con 0x7fc994100a80 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc98bfff640 1 -- 172.21.15.1:0/2482149411 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc984005500 con 0x7fc994100a80 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc98bfff640 1 -- 172.21.15.1:0/2482149411 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984004e00 con 0x7fc994100a80 2024-09-16T00:38:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.896+0000 7fc998c0e640 1 -- 172.21.15.1:0/2482149411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941a4fb0 con 0x7fc994100a80 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.899+0000 7fc98bfff640 1 -- 172.21.15.1:0/2482149411 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc984004fa0 con 0x7fc994100a80 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.899+0000 7fc99afe7740 1 Processor -- start 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.899+0000 7fc99afe7740 1 -- start start 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f3bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc99afe7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2e902f0 con 0x55b9c2f39780 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc99afe7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2e37cf0 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc99afe7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2e8c2b0 con 0x55b9c2f2bb40 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc989d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f3bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc989d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f3bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39992/0 (socket says 172.21.15.1:39992) 2024-09-16T00:38:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc989d9b640 1 -- 172.21.15.1:0/3646619921 learned_addr learned my addr 172.21.15.1:0/3646619921 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc99259c640 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc992d9d640 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc989d9b640 1 -- 172.21.15.1:0/3646619921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2bb40 msgr2=0x55b9c2f2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc989d9b640 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc989d9b640 1 -- 172.21.15.1:0/3646619921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2a5d0 msgr2=0x55b9c2f2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc989d9b640 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f2aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc989d9b640 1 -- 172.21.15.1:0/3646619921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c2e1c3a0 con 0x55b9c2f39780 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc992d9d640 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc98bfff640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9800a3240 0x7fc9800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc992d9d640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9800a3240 0x7fc9800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.900+0000 7fc989d9b640 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f3bb70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00bd30 tx=0x7fc97c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2396471830c828 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.901+0000 7fc98bfff640 1 -- 172.21.15.1:0/2482149411 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc9840cf6d0 con 0x7fc994100a80 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.901+0000 7fc988d99640 1 -- 172.21.15.1:0/3646619921 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c015070 con 0x55b9c2f39780 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.901+0000 7fc988d99640 1 -- 172.21.15.1:0/3646619921 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c00be20 con 0x55b9c2f39780 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.901+0000 7fc988d99640 1 -- 172.21.15.1:0/3646619921 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0107f0 con 0x55b9c2f39780 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 -- 172.21.15.1:0/3646619921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 msgr2=0x55b9c2f3bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc992d9d640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9800a3240 0x7fc9800a5700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc98c004ab0 tx=0x7fc98c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f3bb70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00bd30 tx=0x7fc97c00f730 comp rx=0 tx=0).stop 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 -- 172.21.15.1:0/3646619921 shutdown_connections 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f3bb70 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 --2- 172.21.15.1:0/3646619921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 -- 172.21.15.1:0/3646619921 >> 172.21.15.1:0/3646619921 conn(0x55b9c2ef3310 msgr2=0x55b9c2f26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 -- 172.21.15.1:0/3646619921 shutdown_connections 2024-09-16T00:38:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 -- 172.21.15.1:0/3646619921 wait complete. 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 -- start start 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 Processor -- start 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f39150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f36400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f39780 0x55b9c2f369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2f3c480 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2e4b060 con 0x55b9c2f2bb40 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2f3c600 con 0x55b9c2f39780 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.902+0000 7fc99afe7740 10 client.0 _ll_register_callbacks cb 0x55b9c2b9e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f39150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f39150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40004/0 (socket says 172.21.15.1:40004) 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99259c640 1 -- 172.21.15.1:0/2808071796 learned_addr learned my addr 172.21.15.1:0/2808071796 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc989d9b640 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f36400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc992d9d640 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f39780 0x55b9c2f369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99259c640 1 -- 172.21.15.1:0/2808071796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f39780 msgr2=0x55b9c2f369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99259c640 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f39780 0x55b9c2f369f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99259c640 1 -- 172.21.15.1:0/2808071796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 msgr2=0x55b9c2f36400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99259c640 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f36400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99259c640 1 -- 172.21.15.1:0/2808071796 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c2c013c0 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99259c640 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f39150 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc9840ccdd0 tx=0x7fc984005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd940639abfc9c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc979ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc979ffb640 1 -- 172.21.15.1:0/2808071796 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984020070 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc979ffb640 1 -- 172.21.15.1:0/2808071796 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc984009cc0 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99afe7740 1 -- 172.21.15.1:0/2808071796 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97c00be20 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc979ffb640 1 -- 172.21.15.1:0/2808071796 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc98409b250 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99afe7740 10 client.17994 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.903+0000 7fc99afe7740 1 -- 172.21.15.1:0/2808071796 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b9c2f47f50 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.904+0000 7fc979ffb640 1 -- 172.21.15.1:0/2808071796 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c00be20 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.904+0000 7fc979ffb640 1 -- 172.21.15.1:0/2808071796 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc984097f60 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.904+0000 7fc99afe7740 10 client.17994 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.904+0000 7fc99afe7740 10 client.17994 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.904+0000 7fc99afe7740 20 client.17994 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.904+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9c2f49410 0x55b9c2f4b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.904+0000 7fc99afe7740 4 client.17994 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.904+0000 7fc99afe7740 1 -- 172.21.15.1:0/2808071796 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9c2e9d860 con 0x55b9c2f49410 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.904+0000 7fc989d9b640 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9c2f49410 0x55b9c2f4b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.905+0000 7fc979ffb640 10 client.17994 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.905+0000 7fc989d9b640 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9c2f49410 0x55b9c2f4b880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae67963c6bd52313 server_cookie=da99a231ce6be92c in_seq=0 out_seq=0 2024-09-16T00:38:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.905+0000 7fc979ffb640 1 -- 172.21.15.1:0/2808071796 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9c2e9d860 con 0x55b9c2f49410 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.905+0000 7fc979ffb640 10 client.17994 handle_command_reply: tid=0 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.905+0000 7fc99afe7740 1 client.17994 shutdown 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.905+0000 7fc99afe7740 1 -- 172.21.15.1:0/2808071796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2a5d0 msgr2=0x55b9c2f39150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.905+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f39150 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc9840ccdd0 tx=0x7fc984005bb0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc99afe7740 1 -- 172.21.15.1:0/2808071796 shutdown_connections 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f39780 0x55b9c2f369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9c2f49410 0x55b9c2f4b880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f36400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2808071796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f39150 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc979ffb640 0 client.17994 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc99afe7740 1 -- 172.21.15.1:0/2808071796 >> 172.21.15.1:0/2808071796 conn(0x55b9c2ef3310 msgr2=0x55b9c2f2c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc99afe7740 1 -- 172.21.15.1:0/2808071796 shutdown_connections 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc99afe7740 1 -- 172.21.15.1:0/2808071796 wait complete. 2024-09-16T00:38:47.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.906+0000 7fc99afe7740 20 client.17994 trim_cache size 0 max 16384 2024-09-16T00:38:47.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.919+0000 7fc99afe7740 1 Processor -- start 2024-09-16T00:38:47.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.919+0000 7fc99afe7740 1 -- start start 2024-09-16T00:38:47.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.919+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f26e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.919+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f45720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.919+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f39780 0x55b9c2f35cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.919+0000 7fc99afe7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941af590 con 0x55b9c2f2bb40 2024-09-16T00:38:47.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.919+0000 7fc99afe7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941ace60 con 0x55b9c2f39780 2024-09-16T00:38:47.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.919+0000 7fc99afe7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941aacb0 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.919+0000 7fc989d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f45720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc989d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f45720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40006/0 (socket says 172.21.15.1:40006) 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc989d9b640 1 -- 172.21.15.1:0/1877176372 learned_addr learned my addr 172.21.15.1:0/1877176372 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc99259c640 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f26e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc992d9d640 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f39780 0x55b9c2f35cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc989d9b640 1 -- 172.21.15.1:0/1877176372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 msgr2=0x55b9c2f26e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc989d9b640 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f26e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc989d9b640 1 -- 172.21.15.1:0/1877176372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f39780 msgr2=0x55b9c2f35cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc989d9b640 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f39780 0x55b9c2f35cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc989d9b640 1 -- 172.21.15.1:0/1877176372 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c2f3c520 con 0x55b9c2f2bb40 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc992d9d640 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f39780 0x55b9c2f35cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc989d9b640 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f45720 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00bd30 tx=0x7fc97c00bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0934f30ec1db97a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:47.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc97bfff640 1 -- 172.21.15.1:0/1877176372 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c015050 con 0x55b9c2f2bb40 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc97bfff640 1 -- 172.21.15.1:0/1877176372 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c005520 con 0x55b9c2f2bb40 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.920+0000 7fc97bfff640 1 -- 172.21.15.1:0/1877176372 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c004be0 con 0x55b9c2f2bb40 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 -- 172.21.15.1:0/1877176372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2bb40 msgr2=0x55b9c2f45720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f45720 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00bd30 tx=0x7fc97c00bc70 comp rx=0 tx=0).stop 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 -- 172.21.15.1:0/1877176372 shutdown_connections 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f39780 0x55b9c2f35cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f45720 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 --2- 172.21.15.1:0/1877176372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f26e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 -- 172.21.15.1:0/1877176372 >> 172.21.15.1:0/1877176372 conn(0x55b9c2ef3310 msgr2=0x55b9c2f26700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 -- 172.21.15.1:0/1877176372 shutdown_connections 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 -- 172.21.15.1:0/1877176372 wait complete. 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.921+0000 7fc99afe7740 1 -- start start 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc99afe7740 1 Processor -- start 2024-09-16T00:38:47.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f33270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc99afe7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f33a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc99afe7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941ab320 con 0x55b9c2f39780 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc99afe7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941ac790 con 0x55b9c2f2bb40 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc99afe7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941ab9f0 con 0x55b9c2f2a5d0 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc99afe7740 10 client.0 _ll_register_callbacks cb 0x7fc98400d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc992d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f33a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc992d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f33a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40018/0 (socket says 172.21.15.1:40018) 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc992d9d640 1 -- 172.21.15.1:0/2466923576 learned_addr learned my addr 172.21.15.1:0/2466923576 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:47.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc99259c640 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc989d9b640 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f33270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc992d9d640 1 -- 172.21.15.1:0/2466923576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 msgr2=0x55b9c2f42130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc992d9d640 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f42130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc992d9d640 1 -- 172.21.15.1:0/2466923576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 msgr2=0x55b9c2f33270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc992d9d640 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f33270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.922+0000 7fc992d9d640 1 -- 172.21.15.1:0/2466923576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c2fb3110 con 0x55b9c2f39780 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc992d9d640 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f33a00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc98c035000 tx=0x7fc98c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3755525b27cf5d02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc979ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc979ffb640 1 -- 172.21.15.1:0/2466923576 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c056060 con 0x55b9c2f39780 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc979ffb640 1 -- 172.21.15.1:0/2466923576 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc98c004250 con 0x55b9c2f39780 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc99afe7740 1 -- 172.21.15.1:0/2466923576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97c005520 con 0x55b9c2f39780 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc979ffb640 1 -- 172.21.15.1:0/2466923576 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc98c05d050 con 0x55b9c2f39780 2024-09-16T00:38:47.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc99afe7740 10 client.18000 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc99afe7740 1 -- 172.21.15.1:0/2466923576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b9c2fb4260 con 0x55b9c2f39780 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc979ffb640 1 -- 172.21.15.1:0/2466923576 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c005520 con 0x55b9c2f39780 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc979ffb640 1 -- 172.21.15.1:0/2466923576 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc98c030070 con 0x55b9c2f39780 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc99afe7740 10 client.18000 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc99afe7740 10 client.18000 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc99afe7740 20 client.18000 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9c2fb4980 0x55b9c2fb6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc99afe7740 4 client.18000 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.923+0000 7fc99afe7740 1 -- 172.21.15.1:0/2466923576 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9c2e97d70 con 0x55b9c2fb4980 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.924+0000 7fc99259c640 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9c2fb4980 0x55b9c2fb6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.924+0000 7fc979ffb640 10 client.18000 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.924+0000 7fc99259c640 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9c2fb4980 0x55b9c2fb6d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72dca2f71b2d3e76 server_cookie=3fb55d1eb9148c9c in_seq=0 out_seq=0 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.924+0000 7fc979ffb640 1 -- 172.21.15.1:0/2466923576 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9c2e97d70 con 0x55b9c2fb4980 2024-09-16T00:38:47.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.924+0000 7fc979ffb640 10 client.18000 handle_command_reply: tid=0 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.924+0000 7fc99afe7740 1 client.18000 shutdown 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.924+0000 7fc99afe7740 1 -- 172.21.15.1:0/2466923576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 msgr2=0x55b9c2f33a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.924+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f33a00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc98c035000 tx=0x7fc98c0095c0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc99afe7740 1 -- 172.21.15.1:0/2466923576 shutdown_connections 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9c2f39780 0x55b9c2f33a00 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9c2fb4980 0x55b9c2fb6d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9c2f2bb40 0x55b9c2f33270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc99afe7740 1 --2- 172.21.15.1:0/2466923576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9c2f2a5d0 0x55b9c2f42130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc99afe7740 1 -- 172.21.15.1:0/2466923576 >> 172.21.15.1:0/2466923576 conn(0x55b9c2ef3310 msgr2=0x55b9c2f266c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc979ffb640 0 client.18000 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc99afe7740 1 -- 172.21.15.1:0/2466923576 shutdown_connections 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc99afe7740 1 -- 172.21.15.1:0/2466923576 wait complete. 2024-09-16T00:38:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc99afe7740 20 client.18000 trim_cache size 0 max 16384 2024-09-16T00:38:47.933 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:38:47.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 -- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9800a3240 msgr2=0x7fc9800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9800a3240 0x7fc9800a5700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc98c004ab0 tx=0x7fc98c004a40 comp rx=0 tx=0).stop 2024-09-16T00:38:47.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 -- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994100a80 msgr2=0x7fc994198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994100a80 0x7fc994198430 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc98400d770 tx=0x7fc98400dc30 comp rx=0 tx=0).stop 2024-09-16T00:38:47.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 -- 172.21.15.1:0/2482149411 shutdown_connections 2024-09-16T00:38:47.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc9800a3240 0x7fc9800a5700 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc994106a80 0x7fc99419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc994101420 0x7fc994198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 --2- 172.21.15.1:0/2482149411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994100a80 0x7fc994198430 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 -- 172.21.15.1:0/2482149411 >> 172.21.15.1:0/2482149411 conn(0x7fc9940fc7a0 msgr2=0x7fc99406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:47.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.925+0000 7fc998c0e640 1 -- 172.21.15.1:0/2482149411 shutdown_connections 2024-09-16T00:38:47.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:47.926+0000 7fc998c0e640 1 -- 172.21.15.1:0/2482149411 wait complete. 2024-09-16T00:38:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:48 smithi019 ceph-mon[35186]: pgmap v662: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-16T00:38:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:48 smithi001 ceph-mon[30934]: pgmap v662: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-16T00:38:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:48 smithi153 ceph-mon[35538]: pgmap v662: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-16T00:38:48.648 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:38:48.906 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:49.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.412+0000 7f71b7b52640 1 -- 172.21.15.1:0/4004687309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0106f10 msgr2=0x7f71b010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.412+0000 7f71b7b52640 1 --2- 172.21.15.1:0/4004687309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0106f10 0x7f71b010ea90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009ca0 tx=0x7f71a802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.412+0000 7f71b7b52640 1 -- 172.21.15.1:0/4004687309 shutdown_connections 2024-09-16T00:38:49.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.412+0000 7f71b7b52640 1 --2- 172.21.15.1:0/4004687309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0106f10 0x7f71b010ea90 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.412+0000 7f71b7b52640 1 --2- 172.21.15.1:0/4004687309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71b0102790 0x7f71b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.412+0000 7f71b7b52640 1 --2- 172.21.15.1:0/4004687309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71b0101de0 0x7f71b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.412+0000 7f71b7b52640 1 -- 172.21.15.1:0/4004687309 >> 172.21.15.1:0/4004687309 conn(0x7f71b00fd8c0 msgr2=0x7f71b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:49.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 -- 172.21.15.1:0/4004687309 shutdown_connections 2024-09-16T00:38:49.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 -- 172.21.15.1:0/4004687309 wait complete. 2024-09-16T00:38:49.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 Processor -- start 2024-09-16T00:38:49.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 -- start start 2024-09-16T00:38:49.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0101de0 0x7f71b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71b0102790 0x7f71b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71b0106f10 0x7f71b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f71b0111f70 con 0x7f71b0101de0 2024-09-16T00:38:49.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f71b0111df0 con 0x7f71b0106f10 2024-09-16T00:38:49.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.413+0000 7f71b7b52640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71b01120f0 con 0x7f71b0102790 2024-09-16T00:38:49.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b58ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0101de0 0x7f71b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b58ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0101de0 0x7f71b00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40044/0 (socket says 172.21.15.1:40044) 2024-09-16T00:38:49.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b58ee640 1 -- 172.21.15.1:0/826532745 learned_addr learned my addr 172.21.15.1:0/826532745 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:49.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b50ed640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71b0102790 0x7f71b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b60ef640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71b0106f10 0x7f71b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b58ee640 1 -- 172.21.15.1:0/826532745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71b0102790 msgr2=0x7f71b0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b58ee640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71b0102790 0x7f71b0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b58ee640 1 -- 172.21.15.1:0/826532745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71b0106f10 msgr2=0x7f71b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b58ee640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71b0106f10 0x7f71b0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b58ee640 1 -- 172.21.15.1:0/826532745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b01a6870 con 0x7f71b0101de0 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b60ef640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71b0106f10 0x7f71b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b58ee640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0101de0 0x7f71b00744f0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00c890 tx=0x7f71ac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71a6ffd640 1 -- 172.21.15.1:0/826532745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac004220 con 0x7f71b0101de0 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b7b52640 1 -- 172.21.15.1:0/826532745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b01a6b60 con 0x7f71b0101de0 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71a6ffd640 1 -- 172.21.15.1:0/826532745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac0043c0 con 0x7f71b0101de0 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71a6ffd640 1 -- 172.21.15.1:0/826532745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac004a00 con 0x7f71b0101de0 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.414+0000 7f71b7b52640 1 -- 172.21.15.1:0/826532745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b01a70f0 con 0x7f71b0101de0 2024-09-16T00:38:49.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.416+0000 7f71a6ffd640 1 -- 172.21.15.1:0/826532745 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f71ac0079b0 con 0x7f71b0101de0 2024-09-16T00:38:49.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b9f2b740 1 Processor -- start 2024-09-16T00:38:49.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b9f2b740 1 -- start start 2024-09-16T00:38:49.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 0x56329691f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x56329691fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x56329692c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5632968852f0 con 0x56329691f5d0 2024-09-16T00:38:49.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56329682ccf0 con 0x56329692a370 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5632968812b0 con 0x563296920b40 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b58ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x56329691fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b58ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x56329691fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40050/0 (socket says 172.21.15.1:40050) 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b58ee640 1 -- 172.21.15.1:0/944330230 learned_addr learned my addr 172.21.15.1:0/944330230 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71b60ef640 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 0x56329691f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.417+0000 7f71a6ffd640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f719c0a3240 0x7f719c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b50ed640 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x56329692c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b60ef640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f719c0a3240 0x7f719c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71a6ffd640 1 -- 172.21.15.1:0/826532745 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f71ac0cfb20 con 0x7f71b0101de0 2024-09-16T00:38:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b58ee640 1 -- 172.21.15.1:0/944330230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 msgr2=0x56329691f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b58ee640 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 0x56329691f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b58ee640 1 -- 172.21.15.1:0/944330230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 msgr2=0x56329692c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b58ee640 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x56329692c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b58ee640 1 -- 172.21.15.1:0/944330230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632968113a0 con 0x56329691f5d0 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b50ed640 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x56329692c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b58ee640 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x56329691fa10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f71ac09b9e0 tx=0x7f71ac005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a44eb55dc6f31a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f719bfff640 1 -- 172.21.15.1:0/944330230 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac01d070 con 0x56329691f5d0 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f71b60ef640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f719c0a3240 0x7f719c0a5700 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009430 tx=0x7f71a802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f719bfff640 1 -- 172.21.15.1:0/944330230 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac09c3e0 con 0x56329691f5d0 2024-09-16T00:38:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.418+0000 7f719bfff640 1 -- 172.21.15.1:0/944330230 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac098350 con 0x56329691f5d0 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.419+0000 7f71b9f2b740 1 -- 172.21.15.1:0/944330230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 msgr2=0x56329691fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.419+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x56329691fa10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f71ac09b9e0 tx=0x7f71ac005300 comp rx=0 tx=0).stop 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.419+0000 7f71b9f2b740 1 -- 172.21.15.1:0/944330230 shutdown_connections 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.419+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x56329692c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.419+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x56329691fa10 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.419+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/944330230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 0x56329691f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.419+0000 7f71b9f2b740 1 -- 172.21.15.1:0/944330230 >> 172.21.15.1:0/944330230 conn(0x5632968e8310 msgr2=0x56329691bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.419+0000 7f71b9f2b740 1 -- 172.21.15.1:0/944330230 shutdown_connections 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.419+0000 7f71b9f2b740 1 -- 172.21.15.1:0/944330230 wait complete. 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b9f2b740 1 -- start start 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b9f2b740 1 Processor -- start 2024-09-16T00:38:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329691f5d0 0x563296929d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563296920b40 0x563296927000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 0x5632969275d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56329692d070 con 0x563296920b40 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563296840060 con 0x56329691f5d0 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56329692d1f0 con 0x56329692a370 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b9f2b740 10 client.0 _ll_register_callbacks cb 0x5632965938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b50ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563296920b40 0x563296927000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b50ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563296920b40 0x563296927000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40066/0 (socket says 172.21.15.1:40066) 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b50ed640 1 -- 172.21.15.1:0/352247232 learned_addr learned my addr 172.21.15.1:0/352247232 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b60ef640 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 0x5632969275d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.420+0000 7f71b58ee640 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329691f5d0 0x563296929d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71b50ed640 1 -- 172.21.15.1:0/352247232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 msgr2=0x5632969275d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71b50ed640 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 0x5632969275d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71b50ed640 1 -- 172.21.15.1:0/352247232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329691f5d0 msgr2=0x563296929d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71b50ed640 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329691f5d0 0x563296929d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71b50ed640 1 -- 172.21.15.1:0/352247232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632965f63c0 con 0x563296920b40 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71b50ed640 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563296920b40 0x563296927000 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f71a0005ca0 tx=0x7f71a00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954cbb0d6be85f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71997fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71997fa640 1 -- 172.21.15.1:0/352247232 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0012650 con 0x563296920b40 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71997fa640 1 -- 172.21.15.1:0/352247232 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f71a00127f0 con 0x563296920b40 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71b9f2b740 1 -- 172.21.15.1:0/352247232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ac09c3e0 con 0x563296920b40 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71997fa640 1 -- 172.21.15.1:0/352247232 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71a00129f0 con 0x563296920b40 2024-09-16T00:38:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71b9f2b740 10 client.18012 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71b9f2b740 1 -- 172.21.15.1:0/352247232 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563296938b80 con 0x563296920b40 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.421+0000 7f71997fa640 1 -- 172.21.15.1:0/352247232 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac09c3e0 con 0x563296920b40 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71997fa640 1 -- 172.21.15.1:0/352247232 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f71a001a3d0 con 0x563296920b40 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71b9f2b740 10 client.18012 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71b9f2b740 10 client.18012 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71b9f2b740 20 client.18012 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56329693a0e0 0x56329693c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71b9f2b740 4 client.18012 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71b9f2b740 1 -- 172.21.15.1:0/352247232 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563296892860 con 0x56329693a0e0 2024-09-16T00:38:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71b58ee640 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56329693a0e0 0x56329693c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71b58ee640 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56329693a0e0 0x56329693c520 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dfb6aead2659686 server_cookie=67bc827c73452024 in_seq=0 out_seq=0 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.422+0000 7f71997fa640 10 client.18012 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71997fa640 1 -- 172.21.15.1:0/352247232 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563296892860 con 0x56329693a0e0 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71997fa640 10 client.18012 handle_command_reply: tid=0 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 client.18012 shutdown 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 -- 172.21.15.1:0/352247232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563296920b40 msgr2=0x563296927000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563296920b40 0x563296927000 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f71a0005ca0 tx=0x7f71a00117b0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 -- 172.21.15.1:0/352247232 shutdown_connections 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 0x5632969275d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56329693a0e0 0x56329693c520 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563296920b40 0x563296927000 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/352247232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329691f5d0 0x563296929d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71997fa640 0 client.18012 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 -- 172.21.15.1:0/352247232 >> 172.21.15.1:0/352247232 conn(0x5632968e8310 msgr2=0x563296921990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 -- 172.21.15.1:0/352247232 shutdown_connections 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 1 -- 172.21.15.1:0/352247232 wait complete. 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.423+0000 7f71b9f2b740 20 client.18012 trim_cache size 0 max 16384 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.436+0000 7f71b9f2b740 1 Processor -- start 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.436+0000 7f71b9f2b740 1 -- start start 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.436+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296936340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563296920b40 0x563296922180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 0x5632969226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f71b01b39b0 con 0x56329691f5d0 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f71b01b1280 con 0x563296920b40 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71b01af0d0 con 0x56329692a370 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b58ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296936340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b58ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296936340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40070/0 (socket says 172.21.15.1:40070) 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b58ee640 1 -- 172.21.15.1:0/1013748214 learned_addr learned my addr 172.21.15.1:0/1013748214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b50ed640 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563296920b40 0x563296922180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b60ef640 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 0x5632969226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b58ee640 1 -- 172.21.15.1:0/1013748214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 msgr2=0x5632969226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b58ee640 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 0x5632969226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b58ee640 1 -- 172.21.15.1:0/1013748214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563296920b40 msgr2=0x563296922180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b58ee640 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563296920b40 0x563296922180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b58ee640 1 -- 172.21.15.1:0/1013748214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563296925590 con 0x56329691f5d0 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b60ef640 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 0x5632969226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f71b58ee640 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296936340 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f71ac007590 tx=0x7f71ac0a4950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa91884ee6ef0a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f719b7fe640 1 -- 172.21.15.1:0/1013748214 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac01d040 con 0x56329691f5d0 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f719b7fe640 1 -- 172.21.15.1:0/1013748214 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac09c3e0 con 0x56329691f5d0 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.437+0000 7f719b7fe640 1 -- 172.21.15.1:0/1013748214 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac098350 con 0x56329691f5d0 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.438+0000 7f71b9f2b740 1 -- 172.21.15.1:0/1013748214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 msgr2=0x563296936340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.438+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296936340 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f71ac007590 tx=0x7f71ac0a4950 comp rx=0 tx=0).stop 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.438+0000 7f71b9f2b740 1 -- 172.21.15.1:0/1013748214 shutdown_connections 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.438+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329692a370 0x5632969226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.438+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563296920b40 0x563296922180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.438+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/1013748214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296936340 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.438+0000 7f71b9f2b740 1 -- 172.21.15.1:0/1013748214 >> 172.21.15.1:0/1013748214 conn(0x5632968e8310 msgr2=0x563296926130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.438+0000 7f71b9f2b740 1 -- 172.21.15.1:0/1013748214 shutdown_connections 2024-09-16T00:38:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.438+0000 7f71b9f2b740 1 -- 172.21.15.1:0/1013748214 wait complete. 2024-09-16T00:38:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b9f2b740 1 -- start start 2024-09-16T00:38:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b9f2b740 1 Processor -- start 2024-09-16T00:38:49.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296932d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 0x563296923e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b9f2b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x5632969243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f71b01af740 con 0x56329691f5d0 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f71b01b0bb0 con 0x56329692a370 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b9f2b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71b01afe10 con 0x563296920b40 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b9f2b740 10 client.0 _ll_register_callbacks cb 0x7f71a0005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b58ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296932d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b58ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296932d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40076/0 (socket says 172.21.15.1:40076) 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b58ee640 1 -- 172.21.15.1:0/184861589 learned_addr learned my addr 172.21.15.1:0/184861589 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.439+0000 7f71b60ef640 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x5632969243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b58ee640 1 -- 172.21.15.1:0/184861589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 msgr2=0x563296923e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:49.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b50ed640 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 0x563296923e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b58ee640 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 0x563296923e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b58ee640 1 -- 172.21.15.1:0/184861589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 msgr2=0x5632969243d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:49.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b58ee640 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x5632969243d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b58ee640 1 -- 172.21.15.1:0/184861589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563296924c00 con 0x56329691f5d0 2024-09-16T00:38:49.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b60ef640 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x5632969243d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:49.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b58ee640 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296932d20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f71ac098140 tx=0x7f71ac0a4a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1f6052ef6025be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:49.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71997fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:49.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71997fa640 1 -- 172.21.15.1:0/184861589 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac01d040 con 0x56329691f5d0 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71997fa640 1 -- 172.21.15.1:0/184861589 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f71ac0cd240 con 0x56329691f5d0 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b9f2b740 1 -- 172.21.15.1:0/184861589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ac09c3e0 con 0x56329691f5d0 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71997fa640 1 -- 172.21.15.1:0/184861589 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71ac098350 con 0x56329691f5d0 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b9f2b740 10 client.18018 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71b9f2b740 1 -- 172.21.15.1:0/184861589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632969a4e90 con 0x56329691f5d0 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.440+0000 7f71997fa640 1 -- 172.21.15.1:0/184861589 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac09c3e0 con 0x56329691f5d0 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71997fa640 1 -- 172.21.15.1:0/184861589 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f71ac09aba0 con 0x56329691f5d0 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71b9f2b740 10 client.18018 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71b9f2b740 10 client.18018 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71b9f2b740 20 client.18018 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5632969a5110 0x5632969a7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71b9f2b740 4 client.18018 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71b9f2b740 1 -- 172.21.15.1:0/184861589 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56329688cd70 con 0x5632969a5110 2024-09-16T00:38:49.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71b50ed640 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5632969a5110 0x5632969a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71997fa640 10 client.18018 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71b50ed640 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5632969a5110 0x5632969a7500 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64712d5872ed145b server_cookie=cda32f17f248485c in_seq=0 out_seq=0 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71997fa640 1 -- 172.21.15.1:0/184861589 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56329688cd70 con 0x5632969a5110 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71997fa640 10 client.18018 handle_command_reply: tid=0 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.441+0000 7f71b9f2b740 1 client.18018 shutdown 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 -- 172.21.15.1:0/184861589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 msgr2=0x563296932d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296932d20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f71ac098140 tx=0x7f71ac0a4a40 comp rx=0 tx=0).stop 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 -- 172.21.15.1:0/184861589 shutdown_connections 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56329692a370 0x5632969243d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563296920b40 0x563296923e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5632969a5110 0x5632969a7500 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 --2- 172.21.15.1:0/184861589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56329691f5d0 0x563296932d20 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 -- 172.21.15.1:0/184861589 >> 172.21.15.1:0/184861589 conn(0x5632968e8310 msgr2=0x56329691a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:49.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71997fa640 0 client.18018 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:38:49.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 -- 172.21.15.1:0/184861589 shutdown_connections 2024-09-16T00:38:49.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 1 -- 172.21.15.1:0/184861589 wait complete. 2024-09-16T00:38:49.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b9f2b740 20 client.18018 trim_cache size 0 max 16384 2024-09-16T00:38:49.463 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b7b52640 1 -- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f719c0a3240 msgr2=0x7f719c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b7b52640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f719c0a3240 0x7f719c0a5700 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009430 tx=0x7f71a802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b7b52640 1 -- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0101de0 msgr2=0x7f71b00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.442+0000 7f71b7b52640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0101de0 0x7f71b00744f0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00c890 tx=0x7f71ac00cd50 comp rx=0 tx=0).stop 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.443+0000 7f71b7b52640 1 -- 172.21.15.1:0/826532745 shutdown_connections 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.443+0000 7f71b7b52640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71b0106f10 0x7f71b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.443+0000 7f71b7b52640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f719c0a3240 0x7f719c0a5700 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.443+0000 7f71b7b52640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71b0102790 0x7f71b0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.443+0000 7f71b7b52640 1 --2- 172.21.15.1:0/826532745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b0101de0 0x7f71b00744f0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.443+0000 7f71b7b52640 1 -- 172.21.15.1:0/826532745 >> 172.21.15.1:0/826532745 conn(0x7f71b00fd8c0 msgr2=0x7f71b00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.443+0000 7f71b7b52640 1 -- 172.21.15.1:0/826532745 shutdown_connections 2024-09-16T00:38:49.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:49.443+0000 7f71b7b52640 1 -- 172.21.15.1:0/826532745 wait complete. 2024-09-16T00:38:50.157 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:38:50.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:50 smithi019 ceph-mon[35186]: pgmap v663: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-16T00:38:50.420 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:50.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:50 smithi001 ceph-mon[30934]: pgmap v663: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-16T00:38:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:50 smithi153 ceph-mon[35538]: pgmap v663: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-16T00:38:50.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.895+0000 7fbc11b95640 1 -- 172.21.15.1:0/3532763695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c106a80 msgr2=0x7fbc0c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:50.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.895+0000 7fbc11b95640 1 --2- 172.21.15.1:0/3532763695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c106a80 0x7fbc0c106e60 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009930 tx=0x7fbbfc02a420 comp rx=0 tx=0).stop 2024-09-16T00:38:50.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.895+0000 7fbc11b95640 1 -- 172.21.15.1:0/3532763695 shutdown_connections 2024-09-16T00:38:50.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.895+0000 7fbc11b95640 1 --2- 172.21.15.1:0/3532763695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc0c101420 0x7fbc0c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.895+0000 7fbc11b95640 1 --2- 172.21.15.1:0/3532763695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc0c100a80 0x7fbc0c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.895+0000 7fbc11b95640 1 --2- 172.21.15.1:0/3532763695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c106a80 0x7fbc0c106e60 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.895+0000 7fbc11b95640 1 -- 172.21.15.1:0/3532763695 >> 172.21.15.1:0/3532763695 conn(0x7fbc0c0fc7a0 msgr2=0x7fbc0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.895+0000 7fbc11b95640 1 -- 172.21.15.1:0/3532763695 shutdown_connections 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.895+0000 7fbc11b95640 1 -- 172.21.15.1:0/3532763695 wait complete. 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc11b95640 1 Processor -- start 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc11b95640 1 -- start start 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc11b95640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc0c100a80 0x7fbc0c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc11b95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c101420 0x7fbc0c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc11b95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc0c106a80 0x7fbc0c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc11b95640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c110e90 con 0x7fbc0c101420 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc11b95640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c110d10 con 0x7fbc0c106a80 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc11b95640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c111010 con 0x7fbc0c100a80 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c101420 0x7fbc0c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c101420 0x7fbc0c1978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34430/0 (socket says 172.21.15.1:34430) 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.896+0000 7fbc0affd640 1 -- 172.21.15.1:0/1573193109 learned_addr learned my addr 172.21.15.1:0/1573193109 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc0c100a80 0x7fbc0c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0bfff640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc0c106a80 0x7fbc0c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0affd640 1 -- 172.21.15.1:0/1573193109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc0c100a80 msgr2=0x7fbc0c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0affd640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc0c100a80 0x7fbc0c197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0affd640 1 -- 172.21.15.1:0/1573193109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc0c106a80 msgr2=0x7fbc0c19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0affd640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc0c106a80 0x7fbc0c19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0affd640 1 -- 172.21.15.1:0/1573193109 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c19c330 con 0x7fbc0c101420 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0bfff640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc0c106a80 0x7fbc0c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc0c100a80 0x7fbc0c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc0affd640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c101420 0x7fbc0c1978c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000d950 tx=0x7fbc0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc08ff9640 1 -- 172.21.15.1:0/1573193109 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00017070 con 0x7fbc0c101420 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc08ff9640 1 -- 172.21.15.1:0/1573193109 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00009c30 con 0x7fbc0c101420 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc11b95640 1 -- 172.21.15.1:0/1573193109 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0c19c620 con 0x7fbc0c101420 2024-09-16T00:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc08ff9640 1 -- 172.21.15.1:0/1573193109 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0000f6b0 con 0x7fbc0c101420 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.897+0000 7fbc11b95640 1 -- 172.21.15.1:0/1573193109 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0c1a3f00 con 0x7fbc0c101420 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.899+0000 7fbc08ff9640 1 -- 172.21.15.1:0/1573193109 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fbc000109d0 con 0x7fbc0c101420 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc13f6e740 1 Processor -- start 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc13f6e740 1 -- start start 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 0x55f8e81bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 0x55f8e81bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81ca580 0x55f8e81cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f8e81232f0 con 0x55f8e81ca580 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f8e80cacf0 con 0x55f8e81bf7e0 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f8e811f2b0 con 0x55f8e81c0d50 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81ca580 0x55f8e81cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc08ff9640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbbf40a3240 0x7fbbf40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81ca580 0x55f8e81cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34446/0 (socket says 172.21.15.1:34446) 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc0affd640 1 -- 172.21.15.1:0/3495624524 learned_addr learned my addr 172.21.15.1:0/3495624524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/3495624524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 0x55f8e81bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc0bfff640 1 --2- 172.21.15.1:0/3495624524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 0x55f8e81bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc0bfff640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbbf40a3240 0x7fbbf40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.900+0000 7fbc08ff9640 1 -- 172.21.15.1:0/1573193109 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbc000cfa50 con 0x7fbc0c101420 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbc0affd640 1 -- 172.21.15.1:0/3495624524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 msgr2=0x55f8e81bf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbc0affd640 1 --2- 172.21.15.1:0/3495624524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 0x55f8e81bf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbc0affd640 1 -- 172.21.15.1:0/3495624524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 msgr2=0x55f8e81bfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbc0affd640 1 --2- 172.21.15.1:0/3495624524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 0x55f8e81bfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbc0affd640 1 -- 172.21.15.1:0/3495624524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8e80af3a0 con 0x55f8e81ca580 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbc0affd640 1 --2- 172.21.15.1:0/3495624524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81ca580 0x55f8e81cc970 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbc00098640 tx=0x7fbc00000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06a1d88abcd1b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbbf1ffb640 1 -- 172.21.15.1:0/3495624524 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00020070 con 0x55f8e81ca580 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbc0bfff640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbbf40a3240 0x7fbbf40a5700 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbc04005c40 tx=0x7fbc04009200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbbf1ffb640 1 -- 172.21.15.1:0/3495624524 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc000a48e0 con 0x55f8e81ca580 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.901+0000 7fbbf1ffb640 1 -- 172.21.15.1:0/3495624524 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0009a050 con 0x55f8e81ca580 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 -- 172.21.15.1:0/3495624524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81ca580 msgr2=0x55f8e81cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/3495624524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81ca580 0x55f8e81cc970 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbc00098640 tx=0x7fbc00000bc0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 -- 172.21.15.1:0/3495624524 shutdown_connections 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/3495624524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81ca580 0x55f8e81cc970 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/3495624524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 0x55f8e81bfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/3495624524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 0x55f8e81bf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 -- 172.21.15.1:0/3495624524 >> 172.21.15.1:0/3495624524 conn(0x55f8e8186310 msgr2=0x55f8e81bbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 -- 172.21.15.1:0/3495624524 shutdown_connections 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 -- 172.21.15.1:0/3495624524 wait complete. 2024-09-16T00:38:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 -- start start 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 Processor -- start 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 0x55f8e81c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81ca580 0x55f8e81c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f8e81cd1c0 con 0x55f8e81bf7e0 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f8e80de060 con 0x55f8e81ca580 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.902+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f8e81cd340 con 0x55f8e81c0d50 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc13f6e740 10 client.0 _ll_register_callbacks cb 0x55f8e7e318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34460/0 (socket says 172.21.15.1:34460) 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0b7fe640 1 -- 172.21.15.1:0/2045131064 learned_addr learned my addr 172.21.15.1:0/2045131064 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0affd640 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 0x55f8e81c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0bfff640 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81ca580 0x55f8e81c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0b7fe640 1 -- 172.21.15.1:0/2045131064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 msgr2=0x55f8e81c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 0x55f8e81c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0b7fe640 1 -- 172.21.15.1:0/2045131064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81ca580 msgr2=0x55f8e81c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81ca580 0x55f8e81c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0b7fe640 1 -- 172.21.15.1:0/2045131064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8e7e943c0 con 0x55f8e81bf7e0 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81c9f70 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc005c40 tx=0x7fbbfc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c774dec075e1222 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbbeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2045131064 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc035040 con 0x55f8e81bf7e0 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2045131064 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbbfc0048c0 con 0x55f8e81bf7e0 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.903+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2045131064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f8e8130860 con 0x55f8e81bf7e0 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2045131064 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbbfc04e050 con 0x55f8e81bf7e0 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbc13f6e740 10 client.18030 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2045131064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8e81d8c60 con 0x55f8e81bf7e0 2024-09-16T00:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2045131064 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f8e8130860 con 0x55f8e81bf7e0 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2045131064 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbbfc003b90 con 0x55f8e81bf7e0 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbc13f6e740 10 client.18030 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbc13f6e740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbc13f6e740 20 client.18030 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8e81da120 0x55f8e81dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbc13f6e740 4 client.18030 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2045131064 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8e812ad70 con 0x55f8e81da120 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.904+0000 7fbc0affd640 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8e81da120 0x55f8e81dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.905+0000 7fbbeaffd640 10 client.18030 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.905+0000 7fbc0affd640 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8e81da120 0x55f8e81dc510 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da893ded665fdfce server_cookie=378bf5adad2dca93 in_seq=0 out_seq=0 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.905+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2045131064 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8e812ad70 con 0x55f8e81da120 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.905+0000 7fbbeaffd640 10 client.18030 handle_command_reply: tid=0 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.905+0000 7fbc13f6e740 1 client.18030 shutdown 2024-09-16T00:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2045131064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 msgr2=0x55f8e81c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81c9f70 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc005c40 tx=0x7fbbfc005270 comp rx=0 tx=0).stop 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2045131064 shutdown_connections 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8e81da120 0x55f8e81dc510 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81ca580 0x55f8e81c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81c0d50 0x55f8e81c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2045131064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81c9f70 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbbeaffd640 0 client.18030 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2045131064 >> 172.21.15.1:0/2045131064 conn(0x55f8e8186310 msgr2=0x55f8e81c1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2045131064 shutdown_connections 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2045131064 wait complete. 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.906+0000 7fbc13f6e740 20 client.18030 trim_cache size 0 max 16384 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc13f6e740 1 Processor -- start 2024-09-16T00:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc13f6e740 1 -- start start 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81c0d50 0x55f8e81b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 0x55f8e81e1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1af5b0 con 0x55f8e81c0d50 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1ace80 con 0x55f8e81bf7e0 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1aacd0 con 0x55f8e81ca580 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81c0d50 0x55f8e81b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81c0d50 0x55f8e81b8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34468/0 (socket says 172.21.15.1:34468) 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0affd640 1 -- 172.21.15.1:0/23964176 learned_addr learned my addr 172.21.15.1:0/23964176 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0bfff640 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 0x55f8e81e1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0affd640 1 -- 172.21.15.1:0/23964176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 msgr2=0x55f8e81e1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0affd640 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 0x55f8e81e1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0affd640 1 -- 172.21.15.1:0/23964176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 msgr2=0x55f8e81d6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0affd640 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0affd640 1 -- 172.21.15.1:0/23964176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8e81bcb70 con 0x55f8e81c0d50 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0bfff640 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 0x55f8e81e1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.919+0000 7fbc0affd640 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81c0d50 0x55f8e81b8bf0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fbc0009bc00 tx=0x7fbc0000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43819ff55853e7a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbbf17fa640 1 -- 172.21.15.1:0/23964176 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00020040 con 0x55f8e81c0d50 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbbf17fa640 1 -- 172.21.15.1:0/23964176 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00009de0 con 0x55f8e81c0d50 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbbf17fa640 1 -- 172.21.15.1:0/23964176 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0009b3b0 con 0x55f8e81c0d50 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbc13f6e740 1 -- 172.21.15.1:0/23964176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81c0d50 msgr2=0x55f8e81b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81c0d50 0x55f8e81b8bf0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fbc0009bc00 tx=0x7fbc0000bf60 comp rx=0 tx=0).stop 2024-09-16T00:38:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbc13f6e740 1 -- 172.21.15.1:0/23964176 shutdown_connections 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 0x55f8e81e1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81c0d50 0x55f8e81b8bf0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/23964176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbc13f6e740 1 -- 172.21.15.1:0/23964176 >> 172.21.15.1:0/23964176 conn(0x55f8e8186310 msgr2=0x55f8e81b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbc13f6e740 1 -- 172.21.15.1:0/23964176 shutdown_connections 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.920+0000 7fbc13f6e740 1 -- 172.21.15.1:0/23964176 wait complete. 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc13f6e740 1 -- start start 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc13f6e740 1 Processor -- start 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81c0d50 0x55f8e81ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc13f6e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 0x55f8e81ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1ab340 con 0x55f8e81bf7e0 2024-09-16T00:38:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1ac7b0 con 0x55f8e81c0d50 2024-09-16T00:38:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc13f6e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1aba10 con 0x55f8e81ca580 2024-09-16T00:38:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc13f6e740 10 client.0 _ll_register_callbacks cb 0x7fbbfc004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34480/0 (socket says 172.21.15.1:34480) 2024-09-16T00:38:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.921+0000 7fbc0b7fe640 1 -- 172.21.15.1:0/2993563321 learned_addr learned my addr 172.21.15.1:0/2993563321 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:50.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc0affd640 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81c0d50 0x55f8e81ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc0b7fe640 1 -- 172.21.15.1:0/2993563321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 msgr2=0x55f8e81ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc0bfff640 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 0x55f8e81ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 0x55f8e81ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc0b7fe640 1 -- 172.21.15.1:0/2993563321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81c0d50 msgr2=0x55f8e81ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81c0d50 0x55f8e81ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc0b7fe640 1 -- 172.21.15.1:0/2993563321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8e81cef50 con 0x55f8e81bf7e0 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc0affd640 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81c0d50 0x55f8e81ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc0b7fe640 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d2e20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc006010 tx=0x7fbbfc003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a69beb4bf281c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbbeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2993563321 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc035030 con 0x55f8e81bf7e0 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2993563321 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbbfc0046c0 con 0x55f8e81bf7e0 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2993563321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc00009de0 con 0x55f8e81bf7e0 2024-09-16T00:38:50.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2993563321 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbbfc05a050 con 0x55f8e81bf7e0 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc13f6e740 10 client.18036 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2993563321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8e8244e50 con 0x55f8e81bf7e0 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.922+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2993563321 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00009de0 con 0x55f8e81bf7e0 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2993563321 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbbfc051070 con 0x55f8e81bf7e0 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbc13f6e740 10 client.18036 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbc13f6e740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbc13f6e740 20 client.18036 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8e82455a0 0x55f8e82479c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbc13f6e740 4 client.18036 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2993563321 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8e80a7170 con 0x55f8e82455a0 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbc0affd640 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8e82455a0 0x55f8e82479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbc0affd640 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8e82455a0 0x55f8e82479c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bceda2ce37038a2d server_cookie=48c682b39e44f3a2 in_seq=0 out_seq=0 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbbeaffd640 10 client.18036 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:50.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbbeaffd640 1 -- 172.21.15.1:0/2993563321 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8e80a7170 con 0x55f8e82455a0 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbbeaffd640 10 client.18036 handle_command_reply: tid=0 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.923+0000 7fbc13f6e740 1 client.18036 shutdown 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2993563321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 msgr2=0x55f8e81d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d2e20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc006010 tx=0x7fbbfc003920 comp rx=0 tx=0).stop 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2993563321 shutdown_connections 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8e82455a0 0x55f8e82479c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8e81ca580 0x55f8e81ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbbeaffd640 0 client.18036 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8e81c0d50 0x55f8e81ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 --2- 172.21.15.1:0/2993563321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8e81bf7e0 0x55f8e81d2e20 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2993563321 >> 172.21.15.1:0/2993563321 conn(0x55f8e8186310 msgr2=0x55f8e81b8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2993563321 shutdown_connections 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 1 -- 172.21.15.1:0/2993563321 wait complete. 2024-09-16T00:38:50.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc13f6e740 20 client.18036 trim_cache size 0 max 16384 2024-09-16T00:38:50.944 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:38:50.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc11b95640 1 -- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbbf40a3240 msgr2=0x7fbbf40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:50.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc11b95640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbbf40a3240 0x7fbbf40a5700 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbc04005c40 tx=0x7fbc04009200 comp rx=0 tx=0).stop 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc11b95640 1 -- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c101420 msgr2=0x7fbc0c1978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc11b95640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c101420 0x7fbc0c1978c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000d950 tx=0x7fbc0000de10 comp rx=0 tx=0).stop 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc11b95640 1 -- 172.21.15.1:0/1573193109 shutdown_connections 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc11b95640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc0c106a80 0x7fbc0c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc11b95640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc0c101420 0x7fbc0c1978c0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.924+0000 7fbc11b95640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbbf40a3240 0x7fbbf40a5700 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.925+0000 7fbc11b95640 1 --2- 172.21.15.1:0/1573193109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc0c100a80 0x7fbc0c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.925+0000 7fbc11b95640 1 -- 172.21.15.1:0/1573193109 >> 172.21.15.1:0/1573193109 conn(0x7fbc0c0fc7a0 msgr2=0x7fbc0c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.925+0000 7fbc11b95640 1 -- 172.21.15.1:0/1573193109 shutdown_connections 2024-09-16T00:38:50.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:50.925+0000 7fbc11b95640 1 -- 172.21.15.1:0/1573193109 wait complete. 2024-09-16T00:38:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:52 smithi019 ceph-mon[35186]: pgmap v664: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-16T00:38:52.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:52 smithi001 ceph-mon[30934]: pgmap v664: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-16T00:38:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:52 smithi153 ceph-mon[35538]: pgmap v664: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-16T00:38:52.676 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:38:52.676 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:38:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 -- 172.21.15.1:0/1349076303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 msgr2=0x7f231010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 --2- 172.21.15.1:0/1349076303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231010ea90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f2308009ca0 tx=0x7f230802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 -- 172.21.15.1:0/1349076303 shutdown_connections 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 --2- 172.21.15.1:0/1349076303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231010ea90 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 --2- 172.21.15.1:0/1349076303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2310102790 0x7f23101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 --2- 172.21.15.1:0/1349076303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2310101de0 0x7f23101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 -- 172.21.15.1:0/1349076303 >> 172.21.15.1:0/1349076303 conn(0x7f23100fd8c0 msgr2=0x7f23100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 -- 172.21.15.1:0/1349076303 shutdown_connections 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 -- 172.21.15.1:0/1349076303 wait complete. 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.462+0000 7f231501d640 1 Processor -- start 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f231501d640 1 -- start start 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f231501d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310101de0 0x7f23100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f231501d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2310102790 0x7f2310074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f231501d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2310106f10 0x7f2310074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f231501d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2310111fe0 con 0x7f2310101de0 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f231501d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2310111e60 con 0x7f2310106f10 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f231501d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2310112160 con 0x7f2310102790 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f230ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310101de0 0x7f23100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f230ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310101de0 0x7f23100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34500/0 (socket says 172.21.15.1:34500) 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f230ed9d640 1 -- 172.21.15.1:0/2850104454 learned_addr learned my addr 172.21.15.1:0/2850104454 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f230f59e640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2310106f10 0x7f2310074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.463+0000 7f230659c640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2310102790 0x7f2310074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f230ed9d640 1 -- 172.21.15.1:0/2850104454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2310102790 msgr2=0x7f2310074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f230ed9d640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2310102790 0x7f2310074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f230ed9d640 1 -- 172.21.15.1:0/2850104454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2310106f10 msgr2=0x7f2310074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f230ed9d640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2310106f10 0x7f2310074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f230ed9d640 1 -- 172.21.15.1:0/2850104454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23101a6840 con 0x7f2310101de0 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f230ed9d640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310101de0 0x7f23100744f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f230000b4a0 tx=0x7f230000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f230cd99640 1 -- 172.21.15.1:0/2850104454 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300015020 con 0x7f2310101de0 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f231501d640 1 -- 172.21.15.1:0/2850104454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23101a6b30 con 0x7f2310101de0 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f230cd99640 1 -- 172.21.15.1:0/2850104454 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2300011d10 con 0x7f2310101de0 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f230cd99640 1 -- 172.21.15.1:0/2850104454 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230000f2e0 con 0x7f2310101de0 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.464+0000 7f231501d640 1 -- 172.21.15.1:0/2850104454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23101a70c0 con 0x7f2310101de0 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.466+0000 7f230cd99640 1 -- 172.21.15.1:0/2850104454 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2300018020 con 0x7f2310101de0 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.466+0000 7f23173f6740 1 Processor -- start 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.466+0000 7f23173f6740 1 -- start start 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f23173f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 0x558920b74f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f23173f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b7f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f23173f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b82130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f23173f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558920ada280 con 0x558920b7fd40 2024-09-16T00:38:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f23173f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558920a81b10 con 0x558920b75480 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f23173f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558920ad6240 con 0x558920b74b60 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b82130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b82130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34506/0 (socket says 172.21.15.1:34506) 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230659c640 1 -- 172.21.15.1:0/2711873248 learned_addr learned my addr 172.21.15.1:0/2711873248 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230f59e640 1 --2- 172.21.15.1:0/2711873248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 0x558920b74f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230ed9d640 1 --2- 172.21.15.1:0/2711873248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b7f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230cd99640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22fc0a3240 0x7f22fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230659c640 1 -- 172.21.15.1:0/2711873248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 msgr2=0x558920b74f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230659c640 1 --2- 172.21.15.1:0/2711873248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 0x558920b74f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230659c640 1 -- 172.21.15.1:0/2711873248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 msgr2=0x558920b7f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230659c640 1 --2- 172.21.15.1:0/2711873248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b7f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.467+0000 7f230659c640 1 -- 172.21.15.1:0/2711873248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558920a8e1f0 con 0x558920b7fd40 2024-09-16T00:38:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.468+0000 7f230f59e640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22fc0a3240 0x7f22fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.468+0000 7f230659c640 1 --2- 172.21.15.1:0/2711873248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b82130 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f22f800aca0 tx=0x7f22f800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc975fb779bf602e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.468+0000 7f230cd99640 1 -- 172.21.15.1:0/2850104454 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f23000cf540 con 0x7f2310101de0 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.468+0000 7f230559a640 1 -- 172.21.15.1:0/2711873248 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f801d070 con 0x558920b7fd40 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.468+0000 7f230559a640 1 -- 172.21.15.1:0/2711873248 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f8018b20 con 0x558920b7fd40 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.468+0000 7f230559a640 1 -- 172.21.15.1:0/2711873248 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8010c20 con 0x558920b7fd40 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 -- 172.21.15.1:0/2711873248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 msgr2=0x558920b82130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f230f59e640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22fc0a3240 0x7f22fc0a5700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2308009430 tx=0x7f230803f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 --2- 172.21.15.1:0/2711873248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b82130 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f22f800aca0 tx=0x7f22f800f790 comp rx=0 tx=0).stop 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 -- 172.21.15.1:0/2711873248 shutdown_connections 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 --2- 172.21.15.1:0/2711873248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b82130 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 --2- 172.21.15.1:0/2711873248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b7f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 --2- 172.21.15.1:0/2711873248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 0x558920b74f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 -- 172.21.15.1:0/2711873248 >> 172.21.15.1:0/2711873248 conn(0x558920b710f0 msgr2=0x558920b714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 -- 172.21.15.1:0/2711873248 shutdown_connections 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 -- 172.21.15.1:0/2711873248 wait complete. 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 -- start start 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 Processor -- start 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 0x558920b7cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b7d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b7d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558920b82910 con 0x558920b7fd40 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558920a94f10 con 0x558920b75480 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.469+0000 7f23173f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558920b82a90 con 0x558920b74b60 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f23173f6740 10 client.0 _ll_register_callbacks cb 0x558920a5da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b7d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b7d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34510/0 (socket says 172.21.15.1:34510) 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230f59e640 1 -- 172.21.15.1:0/86576420 learned_addr learned my addr 172.21.15.1:0/86576420 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230ed9d640 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 0x558920b7cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230659c640 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b7d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230f59e640 1 -- 172.21.15.1:0/86576420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 msgr2=0x558920b7cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230f59e640 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 0x558920b7cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230f59e640 1 -- 172.21.15.1:0/86576420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 msgr2=0x558920b7d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230f59e640 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b7d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230f59e640 1 -- 172.21.15.1:0/86576420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55892084b6a0 con 0x558920b7fd40 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f230f59e640 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b7d660 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2308035000 tx=0x7f2308009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1639eea291dc7aec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f22f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f22f67fc640 1 -- 172.21.15.1:0/86576420 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080305c0 con 0x558920b7fd40 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f22f67fc640 1 -- 172.21.15.1:0/86576420 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2308030760 con 0x558920b7fd40 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f23173f6740 1 -- 172.21.15.1:0/86576420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f8018b20 con 0x558920b7fd40 2024-09-16T00:38:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f22f67fc640 1 -- 172.21.15.1:0/86576420 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f230805a050 con 0x558920b7fd40 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f23173f6740 10 client.18048 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.470+0000 7f23173f6740 1 -- 172.21.15.1:0/86576420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558920b8e510 con 0x558920b7fd40 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.471+0000 7f22f67fc640 1 -- 172.21.15.1:0/86576420 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f8018b20 con 0x558920b7fd40 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.471+0000 7f22f67fc640 1 -- 172.21.15.1:0/86576420 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f230804f020 con 0x558920b7fd40 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.471+0000 7f23173f6740 10 client.18048 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.471+0000 7f23173f6740 10 client.18048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.471+0000 7f23173f6740 20 client.18048 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.471+0000 7f23173f6740 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558920b8f900 0x558920b91d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.471+0000 7f23173f6740 4 client.18048 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.471+0000 7f23173f6740 1 -- 172.21.15.1:0/86576420 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558920ae77f0 con 0x558920b8f900 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.471+0000 7f230ed9d640 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558920b8f900 0x558920b91d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.472+0000 7f22f67fc640 10 client.18048 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.472+0000 7f230ed9d640 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558920b8f900 0x558920b91d70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5eb4f6c9facd65e server_cookie=86d28df2b782756f in_seq=0 out_seq=0 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.472+0000 7f22f67fc640 1 -- 172.21.15.1:0/86576420 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558920ae77f0 con 0x558920b8f900 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.472+0000 7f22f67fc640 10 client.18048 handle_command_reply: tid=0 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.472+0000 7f23173f6740 1 client.18048 shutdown 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 -- 172.21.15.1:0/86576420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 msgr2=0x558920b7d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b7d660 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2308035000 tx=0x7f2308009800 comp rx=0 tx=0).stop 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 -- 172.21.15.1:0/86576420 shutdown_connections 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558920b8f900 0x558920b91d70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b7fd40 0x558920b7d660 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b7d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 --2- 172.21.15.1:0/86576420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b74b60 0x558920b7cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f22f67fc640 0 client.18048 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 -- 172.21.15.1:0/86576420 >> 172.21.15.1:0/86576420 conn(0x558920b710f0 msgr2=0x558920b76dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 -- 172.21.15.1:0/86576420 shutdown_connections 2024-09-16T00:38:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 1 -- 172.21.15.1:0/86576420 wait complete. 2024-09-16T00:38:53.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.473+0000 7f23173f6740 20 client.18048 trim_cache size 0 max 16384 2024-09-16T00:38:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.490+0000 7f23173f6740 1 Processor -- start 2024-09-16T00:38:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.490+0000 7f23173f6740 1 -- start start 2024-09-16T00:38:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f23173f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b97640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f23173f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b75480 0x558920b97b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f23173f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b7fd40 0x558920b6fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f23173f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23101b3940 con 0x558920b74b60 2024-09-16T00:38:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f23173f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23101b1210 con 0x558920b7fd40 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f23173f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23101af060 con 0x558920b75480 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b97640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b97640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34512/0 (socket says 172.21.15.1:34512) 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230ed9d640 1 -- 172.21.15.1:0/2469859621 learned_addr learned my addr 172.21.15.1:0/2469859621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230659c640 1 --2- 172.21.15.1:0/2469859621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b75480 0x558920b97b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230f59e640 1 --2- 172.21.15.1:0/2469859621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b7fd40 0x558920b6fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230ed9d640 1 -- 172.21.15.1:0/2469859621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b75480 msgr2=0x558920b97b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230ed9d640 1 --2- 172.21.15.1:0/2469859621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b75480 0x558920b97b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230ed9d640 1 -- 172.21.15.1:0/2469859621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b7fd40 msgr2=0x558920b6fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230ed9d640 1 --2- 172.21.15.1:0/2469859621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b7fd40 0x558920b6fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.491+0000 7f230ed9d640 1 -- 172.21.15.1:0/2469859621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558920b70f20 con 0x558920b74b60 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.492+0000 7f230ed9d640 1 --2- 172.21.15.1:0/2469859621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b97640 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2300097bc0 tx=0x7f23000ccb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428768e305d1c575 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.492+0000 7f2304d99640 1 -- 172.21.15.1:0/2469859621 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000cf070 con 0x558920b74b60 2024-09-16T00:38:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.492+0000 7f2304d99640 1 -- 172.21.15.1:0/2469859621 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f230009fb10 con 0x558920b74b60 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.492+0000 7f2304d99640 1 -- 172.21.15.1:0/2469859621 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000d2d10 con 0x558920b74b60 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.492+0000 7f23173f6740 1 -- 172.21.15.1:0/2469859621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 msgr2=0x558920b97640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.492+0000 7f23173f6740 1 --2- 172.21.15.1:0/2469859621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b97640 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2300097bc0 tx=0x7f23000ccb10 comp rx=0 tx=0).stop 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 -- 172.21.15.1:0/2469859621 shutdown_connections 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 --2- 172.21.15.1:0/2469859621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b7fd40 0x558920b6fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 --2- 172.21.15.1:0/2469859621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b75480 0x558920b97b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 --2- 172.21.15.1:0/2469859621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b97640 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 -- 172.21.15.1:0/2469859621 >> 172.21.15.1:0/2469859621 conn(0x558920b710f0 msgr2=0x558920b7bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 -- 172.21.15.1:0/2469859621 shutdown_connections 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 -- 172.21.15.1:0/2469859621 wait complete. 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 -- start start 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 Processor -- start 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b87d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b88260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b7fd40 0x558920b82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23101af6d0 con 0x558920b74b60 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23101b0b40 con 0x558920b75480 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23101afda0 con 0x558920b7fd40 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f23173f6740 10 client.0 _ll_register_callbacks cb 0x7f230802f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f230ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b87d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f230ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b87d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34524/0 (socket says 172.21.15.1:34524) 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.493+0000 7f230ed9d640 1 -- 172.21.15.1:0/886847194 learned_addr learned my addr 172.21.15.1:0/886847194 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f230659c640 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b88260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f230ed9d640 1 -- 172.21.15.1:0/886847194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b7fd40 msgr2=0x558920b82cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f230ed9d640 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b7fd40 0x558920b82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f230ed9d640 1 -- 172.21.15.1:0/886847194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 msgr2=0x558920b88260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f230ed9d640 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b88260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f230ed9d640 1 -- 172.21.15.1:0/886847194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558920b83210 con 0x558920b74b60 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f230ed9d640 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b87d20 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f230000b470 tx=0x7f2300099c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990c23e4f3e2f7ba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f22f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f22f67fc640 1 -- 172.21.15.1:0/886847194 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000cf040 con 0x558920b74b60 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f22f67fc640 1 -- 172.21.15.1:0/886847194 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2300097dd0 con 0x558920b74b60 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f23173f6740 1 -- 172.21.15.1:0/886847194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f230009fb10 con 0x558920b74b60 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f22f67fc640 1 -- 172.21.15.1:0/886847194 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2300099800 con 0x558920b74b60 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f23173f6740 10 client.18054 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f23173f6740 1 -- 172.21.15.1:0/886847194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558920bfae90 con 0x558920b74b60 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.494+0000 7f22f67fc640 1 -- 172.21.15.1:0/886847194 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f230009fb10 con 0x558920b74b60 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f22f67fc640 1 -- 172.21.15.1:0/886847194 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f230009ac40 con 0x558920b74b60 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f23173f6740 10 client.18054 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f23173f6740 10 client.18054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f23173f6740 20 client.18054 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f23173f6740 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558920bfb550 0x558920bfd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f23173f6740 4 client.18054 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f23173f6740 1 -- 172.21.15.1:0/886847194 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558920ae1d00 con 0x558920bfb550 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f230659c640 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558920bfb550 0x558920bfd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f230659c640 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558920bfb550 0x558920bfd940 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a868d25e520507 server_cookie=46512a395c12ae33 in_seq=0 out_seq=0 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.495+0000 7f22f67fc640 10 client.18054 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.496+0000 7f22f67fc640 1 -- 172.21.15.1:0/886847194 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558920ae1d00 con 0x558920bfb550 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.496+0000 7f22f67fc640 10 client.18054 handle_command_reply: tid=0 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.496+0000 7f23173f6740 1 client.18054 shutdown 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.496+0000 7f23173f6740 1 -- 172.21.15.1:0/886847194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 msgr2=0x558920b87d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b87d20 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f230000b470 tx=0x7f2300099c50 comp rx=0 tx=0).stop 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 1 -- 172.21.15.1:0/886847194 shutdown_connections 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558920bfb550 0x558920bfd940 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558920b7fd40 0x558920b82cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558920b75480 0x558920b88260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f22f67fc640 0 client.18054 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 1 --2- 172.21.15.1:0/886847194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558920b74b60 0x558920b87d20 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 1 -- 172.21.15.1:0/886847194 >> 172.21.15.1:0/886847194 conn(0x558920b710f0 msgr2=0x558920b765b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 1 -- 172.21.15.1:0/886847194 shutdown_connections 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 1 -- 172.21.15.1:0/886847194 wait complete. 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f23173f6740 20 client.18054 trim_cache size 0 max 16384 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:38:53.511 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "aa49e788-b9f1-416a-bfcc-ef948be8aa24", 2024-09-16T00:38:53.512 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:38:53.512 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:38:53.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f231501d640 1 -- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22fc0a3240 msgr2=0x7f22fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f231501d640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22fc0a3240 0x7f22fc0a5700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2308009430 tx=0x7f230803f040 comp rx=0 tx=0).stop 2024-09-16T00:38:53.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f231501d640 1 -- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310101de0 msgr2=0x7f23100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f231501d640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310101de0 0x7f23100744f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f230000b4a0 tx=0x7f230000b8a0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f231501d640 1 -- 172.21.15.1:0/2850104454 shutdown_connections 2024-09-16T00:38:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.497+0000 7f231501d640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2310106f10 0x7f2310074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.498+0000 7f231501d640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f22fc0a3240 0x7f22fc0a5700 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.498+0000 7f231501d640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2310102790 0x7f2310074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.498+0000 7f231501d640 1 --2- 172.21.15.1:0/2850104454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310101de0 0x7f23100744f0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.498+0000 7f231501d640 1 -- 172.21.15.1:0/2850104454 >> 172.21.15.1:0/2850104454 conn(0x7f23100fd8c0 msgr2=0x7f23100ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.498+0000 7f231501d640 1 -- 172.21.15.1:0/2850104454 shutdown_connections 2024-09-16T00:38:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:53.498+0000 7f231501d640 1 -- 172.21.15.1:0/2850104454 wait complete. 2024-09-16T00:38:54.145 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:38:54.405 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:54 smithi001 ceph-mon[30934]: pgmap v665: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-16T00:38:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:54 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:38:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:54 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:54 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:38:54.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:54 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:54 smithi153 ceph-mon[35538]: pgmap v665: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-16T00:38:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:54 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:38:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:54 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:54 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:38:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:54 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:54 smithi019 ceph-mon[35186]: pgmap v665: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-16T00:38:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:54 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:38:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:54 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:54 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:38:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:54 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:38:54.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.917+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/2171527793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 msgr2=0x7ff1c410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.917+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/2171527793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 0x7ff1c410d950 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009cd0 tx=0x7ff1bc02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.917+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/2171527793 shutdown_connections 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.917+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/2171527793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 0x7ff1c410d950 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.917+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/2171527793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c4100a80 0x7ff1c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.917+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/2171527793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1c4106a80 0x7ff1c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.917+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/2171527793 >> 172.21.15.1:0/2171527793 conn(0x7ff1c40fc7a0 msgr2=0x7ff1c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.918+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/2171527793 shutdown_connections 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.918+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/2171527793 wait complete. 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.918+0000 7ff1cbb7c640 1 Processor -- start 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.918+0000 7ff1cbb7c640 1 -- start start 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1cbb7c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1c4100a80 0x7ff1c4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1cbb7c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 0x7ff1c41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1cbb7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c4106a80 0x7ff1c41a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1cbb7c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4110ed0 con 0x7ff1c4101420 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1cbb7c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4110d50 con 0x7ff1c4106a80 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1cbb7c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4111050 con 0x7ff1c4100a80 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9117640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 0x7ff1c41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9117640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 0x7ff1c41997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34542/0 (socket says 172.21.15.1:34542) 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9117640 1 -- 172.21.15.1:0/1678502300 learned_addr learned my addr 172.21.15.1:0/1678502300 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9918640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1c4100a80 0x7ff1c4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1ca119640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c4106a80 0x7ff1c41a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9117640 1 -- 172.21.15.1:0/1678502300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1c4100a80 msgr2=0x7ff1c4199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9117640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1c4100a80 0x7ff1c4199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9117640 1 -- 172.21.15.1:0/1678502300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c4106a80 msgr2=0x7ff1c41a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9117640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c4106a80 0x7ff1c41a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9117640 1 -- 172.21.15.1:0/1678502300 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c41a4820 con 0x7ff1c4101420 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.919+0000 7ff1c9117640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 0x7ff1c41997a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c920 tx=0x7ff1b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.920+0000 7ff1baffd640 1 -- 172.21.15.1:0/1678502300 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4007c70 con 0x7ff1c4101420 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.920+0000 7ff1baffd640 1 -- 172.21.15.1:0/1678502300 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b4007e10 con 0x7ff1c4101420 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.920+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/1678502300 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c41a49f0 con 0x7ff1c4101420 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.920+0000 7ff1baffd640 1 -- 172.21.15.1:0/1678502300 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4018890 con 0x7ff1c4101420 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.920+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/1678502300 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c41a4f30 con 0x7ff1c4101420 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.922+0000 7ff1baffd640 1 -- 172.21.15.1:0/1678502300 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff1b4018a70 con 0x7ff1c4101420 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.922+0000 7ff1cdf55740 1 Processor -- start 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.922+0000 7ff1cdf55740 1 -- start start 2024-09-16T00:38:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 0x55c88e6a0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0c80 0x55c88e6af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 0x55c88e6b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c88e6062e0 con 0x55c88e6a0c80 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c88e5adce0 con 0x55c88e6a0360 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c88e6022a0 con 0x55c88e6afa90 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0c80 0x55c88e6af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0c80 0x55c88e6af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34548/0 (socket says 172.21.15.1:34548) 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9918640 1 -- 172.21.15.1:0/2133431621 learned_addr learned my addr 172.21.15.1:0/2133431621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9117640 1 --2- 172.21.15.1:0/2133431621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 0x55c88e6b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1ca119640 1 --2- 172.21.15.1:0/2133431621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 0x55c88e6a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1baffd640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1b00a32c0 0x7ff1b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9918640 1 -- 172.21.15.1:0/2133431621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 msgr2=0x55c88e6b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9918640 1 --2- 172.21.15.1:0/2133431621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 0x55c88e6b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9918640 1 -- 172.21.15.1:0/2133431621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 msgr2=0x55c88e6a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9918640 1 --2- 172.21.15.1:0/2133431621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 0x55c88e6a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9918640 1 -- 172.21.15.1:0/2133431621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c88e592440 con 0x55c88e6a0c80 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1ca119640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1b00a32c0 0x7ff1b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1baffd640 1 -- 172.21.15.1:0/1678502300 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff1b40cfe00 con 0x7ff1c4101420 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.923+0000 7ff1c9918640 1 --2- 172.21.15.1:0/2133431621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0c80 0x55c88e6af550 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b710 tx=0x7ff1c0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df43704a0c0dd6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1affff640 1 -- 172.21.15.1:0/2133431621 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0003bd0 con 0x55c88e6a0c80 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1affff640 1 -- 172.21.15.1:0/2133431621 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c0015070 con 0x55c88e6a0c80 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1affff640 1 -- 172.21.15.1:0/2133431621 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0004e40 con 0x55c88e6a0c80 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1cdf55740 1 -- 172.21.15.1:0/2133431621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0c80 msgr2=0x55c88e6af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1ca119640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1b00a32c0 0x7ff1b00a5780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009430 tx=0x7ff1bc03f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/2133431621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0c80 0x55c88e6af550 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b710 tx=0x7ff1c0004610 comp rx=0 tx=0).stop 2024-09-16T00:38:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1cdf55740 1 -- 172.21.15.1:0/2133431621 shutdown_connections 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/2133431621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 0x55c88e6b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/2133431621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0c80 0x55c88e6af550 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/2133431621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 0x55c88e6a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1cdf55740 1 -- 172.21.15.1:0/2133431621 >> 172.21.15.1:0/2133431621 conn(0x55c88e69ca30 msgr2=0x55c88e69ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1cdf55740 1 -- 172.21.15.1:0/2133431621 shutdown_connections 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.924+0000 7ff1cdf55740 1 -- 172.21.15.1:0/2133431621 wait complete. 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1cdf55740 1 -- start start 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1cdf55740 1 Processor -- start 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 0x55c88e6af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6a0c80 0x55c88e6ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6afa90 0x55c88e6acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c88e6b2660 con 0x55c88e6afa90 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c88e5c1050 con 0x55c88e6a0360 2024-09-16T00:38:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c88e6b27e0 con 0x55c88e6a0c80 2024-09-16T00:38:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1cdf55740 10 client.0 _ll_register_callbacks cb 0x55c88e601160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1ca119640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6afa90 0x55c88e6acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1ca119640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6afa90 0x55c88e6acc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34550/0 (socket says 172.21.15.1:34550) 2024-09-16T00:38:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.925+0000 7ff1ca119640 1 -- 172.21.15.1:0/3984798455 learned_addr learned my addr 172.21.15.1:0/3984798455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1c9918640 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 0x55c88e6af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1c9117640 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6a0c80 0x55c88e6ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ca119640 1 -- 172.21.15.1:0/3984798455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6a0c80 msgr2=0x55c88e6ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ca119640 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6a0c80 0x55c88e6ac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ca119640 1 -- 172.21.15.1:0/3984798455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 msgr2=0x55c88e6af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ca119640 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 0x55c88e6af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ca119640 1 -- 172.21.15.1:0/3984798455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c88e3773c0 con 0x55c88e6afa90 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ca119640 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6afa90 0x55c88e6acc10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc035000 tx=0x7ff1bc0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550ee9d5b7f977b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/3984798455 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0305a0 con 0x55c88e6afa90 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/3984798455 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff1bc030740 con 0x55c88e6afa90 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1cdf55740 1 -- 172.21.15.1:0/3984798455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1c0015070 con 0x55c88e6afa90 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/3984798455 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1bc030940 con 0x55c88e6afa90 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1cdf55740 10 client.18066 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.926+0000 7ff1cdf55740 1 -- 172.21.15.1:0/3984798455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c88e6be0d0 con 0x55c88e6afa90 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/3984798455 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c0015070 con 0x55c88e6afa90 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/3984798455 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff1bc04f070 con 0x55c88e6afa90 2024-09-16T00:38:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1cdf55740 10 client.18066 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1cdf55740 10 client.18066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1cdf55740 20 client.18066 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c88e6bf4e0 0x55c88e6c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1cdf55740 4 client.18066 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1cdf55740 1 -- 172.21.15.1:0/3984798455 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c88e613850 con 0x55c88e6bf4e0 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1c9918640 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c88e6bf4e0 0x55c88e6c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1c9918640 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c88e6bf4e0 0x55c88e6c1950 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80636786fab80f9d server_cookie=a87b36a9409901dc in_seq=0 out_seq=0 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.927+0000 7ff1ad7fa640 10 client.18066 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/3984798455 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c88e613850 con 0x55c88e6bf4e0 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1ad7fa640 10 client.18066 handle_command_reply: tid=0 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 client.18066 shutdown 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 -- 172.21.15.1:0/3984798455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6afa90 msgr2=0x55c88e6acc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6afa90 0x55c88e6acc10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc035000 tx=0x7ff1bc0093c0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 -- 172.21.15.1:0/3984798455 shutdown_connections 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c88e6bf4e0 0x55c88e6c1950 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6afa90 0x55c88e6acc10 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6a0c80 0x55c88e6ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/3984798455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0360 0x55c88e6af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 -- 172.21.15.1:0/3984798455 >> 172.21.15.1:0/3984798455 conn(0x55c88e69ca30 msgr2=0x55c88e6a2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1ad7fa640 0 client.18066 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 -- 172.21.15.1:0/3984798455 shutdown_connections 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 1 -- 172.21.15.1:0/3984798455 wait complete. 2024-09-16T00:38:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.928+0000 7ff1cdf55740 20 client.18066 trim_cache size 0 max 16384 2024-09-16T00:38:54.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1cdf55740 1 Processor -- start 2024-09-16T00:38:54.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1cdf55740 1 -- start start 2024-09-16T00:38:54.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6a0c80 0x55c88e6a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6afa90 0x55c88e6c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41af4e0 con 0x55c88e6a0360 2024-09-16T00:38:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41acdb0 con 0x55c88e6afa90 2024-09-16T00:38:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41aac00 con 0x55c88e6a0c80 2024-09-16T00:38:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1c9918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1c9918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6bb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34554/0 (socket says 172.21.15.1:34554) 2024-09-16T00:38:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1c9918640 1 -- 172.21.15.1:0/822186577 learned_addr learned my addr 172.21.15.1:0/822186577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1c9918640 1 -- 172.21.15.1:0/822186577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6a0c80 msgr2=0x55c88e6a7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1c9918640 1 --2- 172.21.15.1:0/822186577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6a0c80 0x55c88e6a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1c9918640 1 -- 172.21.15.1:0/822186577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6afa90 msgr2=0x55c88e6c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1ca119640 1 --2- 172.21.15.1:0/822186577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6afa90 0x55c88e6c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1c9918640 1 --2- 172.21.15.1:0/822186577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6afa90 0x55c88e6c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1c9918640 1 -- 172.21.15.1:0/822186577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c88e6a0ab0 con 0x55c88e6a0360 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.946+0000 7ff1c9918640 1 --2- 172.21.15.1:0/822186577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6bb960 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00039c0 tx=0x7ff1c00049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ded3a6f0b7b59ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1af7fe640 1 -- 172.21.15.1:0/822186577 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0003dd0 con 0x55c88e6a0360 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1af7fe640 1 -- 172.21.15.1:0/822186577 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c0015030 con 0x55c88e6a0360 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1af7fe640 1 -- 172.21.15.1:0/822186577 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0019420 con 0x55c88e6a0360 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1cdf55740 1 -- 172.21.15.1:0/822186577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 msgr2=0x55c88e6bb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/822186577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6bb960 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00039c0 tx=0x7ff1c00049e0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1cdf55740 1 -- 172.21.15.1:0/822186577 shutdown_connections 2024-09-16T00:38:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/822186577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6afa90 0x55c88e6c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/822186577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6a0c80 0x55c88e6a7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/822186577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6bb960 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.947+0000 7ff1cdf55740 1 -- 172.21.15.1:0/822186577 >> 172.21.15.1:0/822186577 conn(0x55c88e69ca30 msgr2=0x55c88e6a74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 -- 172.21.15.1:0/822186577 shutdown_connections 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 -- 172.21.15.1:0/822186577 wait complete. 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 -- start start 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 Processor -- start 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0c80 0x55c88e6b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 0x55c88e6b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41ab270 con 0x55c88e6a0360 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41ac6e0 con 0x55c88e6a0c80 2024-09-16T00:38:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1cdf55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41ab940 con 0x55c88e6afa90 2024-09-16T00:38:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.948+0000 7ff1c9918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1cdf55740 10 client.0 _ll_register_callbacks cb 0x7ff1bc0317e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:38:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1c9918640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34556/0 (socket says 172.21.15.1:34556) 2024-09-16T00:38:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1c9918640 1 -- 172.21.15.1:0/1900275129 learned_addr learned my addr 172.21.15.1:0/1900275129 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:38:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1c9117640 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0c80 0x55c88e6b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1ca119640 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 0x55c88e6b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1c9918640 1 -- 172.21.15.1:0/1900275129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 msgr2=0x55c88e6b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1c9918640 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 0x55c88e6b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1c9918640 1 -- 172.21.15.1:0/1900275129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0c80 msgr2=0x55c88e6b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1c9918640 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0c80 0x55c88e6b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1c9918640 1 -- 172.21.15.1:0/1900275129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c88e6b43f0 con 0x55c88e6a0360 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1ca119640 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 0x55c88e6b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1c9918640 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6b82c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b3d0 tx=0x7ff1c00043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42543c1c471cfeba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/1900275129 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0011e30 con 0x55c88e6a0360 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/1900275129 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff1c0003a70 con 0x55c88e6a0360 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1cdf55740 1 -- 172.21.15.1:0/1900275129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1c0015030 con 0x55c88e6a0360 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/1900275129 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1c0003c70 con 0x55c88e6a0360 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.949+0000 7ff1cdf55740 10 client.18072 fetch_fsmap learned FSMap version 25 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1cdf55740 1 -- 172.21.15.1:0/1900275129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c88e72a2f0 con 0x55c88e6a0360 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/1900275129 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c0015030 con 0x55c88e6a0360 2024-09-16T00:38:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/1900275129 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff1c002e030 con 0x55c88e6a0360 2024-09-16T00:38:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1cdf55740 10 client.18072 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:38:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1cdf55740 10 client.18072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:38:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1cdf55740 20 client.18072 populate_metadata read hostname 'smithi001' 2024-09-16T00:38:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c88e72a7b0 0x55c88e72cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1cdf55740 4 client.18072 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:38:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1cdf55740 1 -- 172.21.15.1:0/1900275129 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c88e60dd60 con 0x55c88e72a7b0 2024-09-16T00:38:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.950+0000 7ff1c9117640 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c88e72a7b0 0x55c88e72cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1c9117640 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c88e72a7b0 0x55c88e72cba0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5fda4a1869396c3 server_cookie=8e26c14e6c21698c in_seq=0 out_seq=0 2024-09-16T00:38:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1ad7fa640 10 client.18072 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1ad7fa640 1 -- 172.21.15.1:0/1900275129 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c88e60dd60 con 0x55c88e72a7b0 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1ad7fa640 10 client.18072 handle_command_reply: tid=0 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 client.18072 shutdown 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 -- 172.21.15.1:0/1900275129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 msgr2=0x55c88e6b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6b82c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b3d0 tx=0x7ff1c00043d0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 -- 172.21.15.1:0/1900275129 shutdown_connections 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55c88e72a7b0 0x55c88e72cba0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c88e6afa90 0x55c88e6b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c88e6a0c80 0x55c88e6b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 --2- 172.21.15.1:0/1900275129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c88e6a0360 0x55c88e6b82c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1ad7fa640 0 client.18072 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 -- 172.21.15.1:0/1900275129 >> 172.21.15.1:0/1900275129 conn(0x55c88e69ca30 msgr2=0x55c88e6adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 -- 172.21.15.1:0/1900275129 shutdown_connections 2024-09-16T00:38:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 1 -- 172.21.15.1:0/1900275129 wait complete. 2024-09-16T00:38:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.951+0000 7ff1cdf55740 20 client.18072 trim_cache size 0 max 16384 2024-09-16T00:38:54.966 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:38:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:38:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:38:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "aa49e788-b9f1-416a-bfcc-ef948be8aa24": { 2024-09-16T00:38:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:38:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "aa49e788-b9f1-416a-bfcc-ef948be8aa24", 2024-09-16T00:38:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:38:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:38:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:38:54.967 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1b00a32c0 msgr2=0x7ff1b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1b00a32c0 0x7ff1b00a5780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009430 tx=0x7ff1bc03f040 comp rx=0 tx=0).stop 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 msgr2=0x7ff1c41997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 0x7ff1c41997a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c920 tx=0x7ff1b400cde0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/1678502300 shutdown_connections 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c4106a80 0x7ff1c41a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1c4101420 0x7ff1c41997a0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff1b00a32c0 0x7ff1b00a5780 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 --2- 172.21.15.1:0/1678502300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1c4100a80 0x7ff1c4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/1678502300 >> 172.21.15.1:0/1678502300 conn(0x7ff1c40fc7a0 msgr2=0x7ff1c40fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/1678502300 shutdown_connections 2024-09-16T00:38:54.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:38:54.952+0000 7ff1cbb7c640 1 -- 172.21.15.1:0/1678502300 wait complete. 2024-09-16T00:38:55.728 INFO:tasks.cephfs.filesystem:scrub status for tag:aa49e788-b9f1-416a-bfcc-ef948be8aa24 - {'path': '/', 'tag': 'aa49e788-b9f1-416a-bfcc-ef948be8aa24', 'options': 'recursive,force'} 2024-09-16T00:38:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:56 smithi001 ceph-mon[30934]: pgmap v666: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:38:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:56 smithi153 ceph-mon[35538]: pgmap v666: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:38:56.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:56 smithi019 ceph-mon[35186]: pgmap v666: 129 pgs: 129 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:38:58.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:58 smithi001 ceph-mon[30934]: pgmap v667: 129 pgs: 129 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-16T00:38:58.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:38:58 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:38:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:58 smithi153 ceph-mon[35538]: pgmap v667: 129 pgs: 129 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-16T00:38:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:38:58 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:38:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:58 smithi019 ceph-mon[35186]: pgmap v667: 129 pgs: 129 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-16T00:38:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:38:58 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:39:00.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:00 smithi001 ceph-mon[30934]: pgmap v668: 129 pgs: 129 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:00 smithi153 ceph-mon[35538]: pgmap v668: 129 pgs: 129 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:00.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:00 smithi019 ceph-mon[35186]: pgmap v668: 129 pgs: 129 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:02 smithi001 ceph-mon[30934]: pgmap v669: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:02.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:02 smithi153 ceph-mon[35538]: pgmap v669: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:02.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:02 smithi019 ceph-mon[35186]: pgmap v669: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:04.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:04 smithi001 ceph-mon[30934]: pgmap v670: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:04.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:04 smithi153 ceph-mon[35538]: pgmap v670: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:04 smithi019 ceph-mon[35186]: pgmap v670: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:06.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:06 smithi001 ceph-mon[30934]: pgmap v671: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:06.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:06 smithi153 ceph-mon[35538]: pgmap v671: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:06 smithi019 ceph-mon[35186]: pgmap v671: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:08.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:08 smithi001 ceph-mon[30934]: pgmap v672: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:39:08.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:08 smithi153 ceph-mon[35538]: pgmap v672: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:39:08.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:08 smithi019 ceph-mon[35186]: pgmap v672: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:39:09.885 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.885 INFO:tasks.workunit.client.0.smithi001.stdout:313/600: open fd = 3 2024-09-16T00:39:09.885 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.885 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:314/600: open fd = 3 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:315/600: open fd = 3 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:316/600: open fd = 3 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.886 INFO:tasks.workunit.client.0.smithi001.stdout:317/600: open fd = 3 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:318/600: open fd = 3 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:319/600: open fd = 3 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:320/600: open fd = 3 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.887 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:321/600: open fd = 3 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:322/600: open fd = 3 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:323/600: open fd = 3 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.888 INFO:tasks.workunit.client.0.smithi001.stdout:324/600: open fd = 3 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:325/600: open fd = 3 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:326/600: open fd = 3 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:327/600: open fd = 3 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.889 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:328/600: open fd = 3 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:329/600: open fd = 3 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:330/600: open fd = 3 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:331/600: open fd = 3 2024-09-16T00:39:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:332/600: open fd = 3 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:333/600: open fd = 3 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:334/600: open fd = 3 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.891 INFO:tasks.workunit.client.0.smithi001.stdout:335/600: open fd = 3 2024-09-16T00:39:09.892 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.892 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.892 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.892 INFO:tasks.workunit.client.0.smithi001.stdout:336/600: open fd = 3 2024-09-16T00:39:09.892 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.892 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.892 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.892 INFO:tasks.workunit.client.0.smithi001.stdout:337/600: open fd = 3 2024-09-16T00:39:09.892 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:338/600: open fd = 3 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:339/600: open fd = 3 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:340/600: open fd = 3 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.893 INFO:tasks.workunit.client.0.smithi001.stdout:341/600: open fd = 3 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:342/600: open fd = 3 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:343/600: open fd = 3 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:344/600: open fd = 3 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:345/600: open fd = 3 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:346/600: open fd = 3 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:347/600: open fd = 3 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.895 INFO:tasks.workunit.client.0.smithi001.stdout:348/600: open fd = 3 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:349/600: open fd = 3 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:350/600: open fd = 3 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:351/600: open fd = 3 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.896 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:352/600: open fd = 3 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:353/600: open fd = 3 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:354/600: open fd = 3 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.897 INFO:tasks.workunit.client.0.smithi001.stdout:355/600: open fd = 3 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:356/600: open fd = 3 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:357/600: open fd = 3 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:358/600: open fd = 3 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:359/600: open fd = 3 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:360/600: open fd = 3 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:361/600: open fd = 3 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.899 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:362/600: open fd = 3 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:363/600: open fd = 3 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:364/600: open fd = 3 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:365/600: open fd = 3 2024-09-16T00:39:09.900 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:366/600: open fd = 3 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:367/600: open fd = 3 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:368/600: open fd = 3 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.901 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:369/600: open fd = 3 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:370/600: open fd = 3 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:371/600: open fd = 3 2024-09-16T00:39:09.902 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:372/600: open fd = 3 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:373/600: open fd = 3 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:374/600: open fd = 3 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:39:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:39:10.409 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:10 smithi019 ceph-mon[35186]: pgmap v673: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:10 smithi001 ceph-mon[30934]: pgmap v673: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:10.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:10 smithi153 ceph-mon[35538]: pgmap v673: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:12 smithi001 ceph-mon[30934]: pgmap v674: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:12.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:12 smithi153 ceph-mon[35538]: pgmap v674: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:12 smithi019 ceph-mon[35186]: pgmap v674: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:14.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:14 smithi001 ceph-mon[30934]: pgmap v675: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:14.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:14 smithi153 ceph-mon[35538]: pgmap v675: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:14.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:14 smithi019 ceph-mon[35186]: pgmap v675: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:16.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:16 smithi001 ceph-mon[30934]: pgmap v676: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:16.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:16 smithi153 ceph-mon[35538]: pgmap v676: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:16 smithi019 ceph-mon[35186]: pgmap v676: 129 pgs: 129 active+clean; 5.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:18 smithi001 ceph-mon[30934]: pgmap v677: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:18 smithi153 ceph-mon[35538]: pgmap v677: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:18 smithi019 ceph-mon[35186]: pgmap v677: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:20.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:20 smithi001 ceph-mon[30934]: pgmap v678: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:20.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:20 smithi153 ceph-mon[35538]: pgmap v678: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:20 smithi019 ceph-mon[35186]: pgmap v678: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:22.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:22 smithi001 ceph-mon[30934]: pgmap v679: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:22.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:22 smithi153 ceph-mon[35538]: pgmap v679: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:22 smithi019 ceph-mon[35186]: pgmap v679: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:24.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:24 smithi001 ceph-mon[30934]: pgmap v680: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:24.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:24 smithi153 ceph-mon[35538]: pgmap v680: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:24 smithi019 ceph-mon[35186]: pgmap v680: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:25.729 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:39:26.001 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:26.409 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:26 smithi001 ceph-mon[30934]: pgmap v681: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:26 smithi153 ceph-mon[35538]: pgmap v681: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:26.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 -- 172.21.15.1:0/1349239415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0102570 msgr2=0x7f83d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:26.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 --2- 172.21.15.1:0/1349239415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0102570 0x7f83d01029b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009900 tx=0x7f83c402a480 comp rx=0 tx=0).stop 2024-09-16T00:39:26.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 -- 172.21.15.1:0/1349239415 shutdown_connections 2024-09-16T00:39:26.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 --2- 172.21.15.1:0/1349239415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d0102ef0 0x7f83d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 --2- 172.21.15.1:0/1349239415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0102570 0x7f83d01029b0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 --2- 172.21.15.1:0/1349239415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d0100e20 0x7f83d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 -- 172.21.15.1:0/1349239415 >> 172.21.15.1:0/1349239415 conn(0x7f83d00fc7d0 msgr2=0x7f83d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 -- 172.21.15.1:0/1349239415 shutdown_connections 2024-09-16T00:39:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 -- 172.21.15.1:0/1349239415 wait complete. 2024-09-16T00:39:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.494+0000 7f83d6205640 1 Processor -- start 2024-09-16T00:39:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83d6205640 1 -- start start 2024-09-16T00:39:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83d6205640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0100e20 0x7f83d01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83d6205640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d0102570 0x7f83d0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83d6205640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d0102ef0 0x7f83d01a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83d6205640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0110fa0 con 0x7f83d0100e20 2024-09-16T00:39:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83d6205640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0110e20 con 0x7f83d0102570 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83d6205640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0111120 con 0x7f83d0102ef0 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83cf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0100e20 0x7f83d01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83cf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0100e20 0x7f83d01993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43056/0 (socket says 172.21.15.1:43056) 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83cf7fe640 1 -- 172.21.15.1:0/811968956 learned_addr learned my addr 172.21.15.1:0/811968956 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83cffff640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d0102ef0 0x7f83d01a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83ceffd640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d0102570 0x7f83d0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83cf7fe640 1 -- 172.21.15.1:0/811968956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d0102ef0 msgr2=0x7f83d01a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d0102ef0 0x7f83d01a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83cf7fe640 1 -- 172.21.15.1:0/811968956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d0102570 msgr2=0x7f83d0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d0102570 0x7f83d0199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83cf7fe640 1 -- 172.21.15.1:0/811968956 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d01a4990 con 0x7f83d0100e20 2024-09-16T00:39:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.495+0000 7f83ceffd640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d0102570 0x7f83d0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.496+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0100e20 0x7f83d01993d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f83c000ba50 tx=0x7f83c000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.496+0000 7f83ccff9640 1 -- 172.21.15.1:0/811968956 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0019070 con 0x7f83d0100e20 2024-09-16T00:39:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.496+0000 7f83d6205640 1 -- 172.21.15.1:0/811968956 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d01a4b60 con 0x7f83d0100e20 2024-09-16T00:39:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.496+0000 7f83ccff9640 1 -- 172.21.15.1:0/811968956 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c000cbe0 con 0x7f83d0100e20 2024-09-16T00:39:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.496+0000 7f83ccff9640 1 -- 172.21.15.1:0/811968956 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c00076e0 con 0x7f83d0100e20 2024-09-16T00:39:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.496+0000 7f83d6205640 1 -- 172.21.15.1:0/811968956 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d01a50a0 con 0x7f83d0100e20 2024-09-16T00:39:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.498+0000 7f83ccff9640 1 -- 172.21.15.1:0/811968956 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f83c001c070 con 0x7f83d0100e20 2024-09-16T00:39:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.498+0000 7f83d85de740 1 Processor -- start 2024-09-16T00:39:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.498+0000 7f83d85de740 1 -- start start 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83d85de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b05820e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83d85de740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b05821d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83d85de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b05821fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83d85de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b0581742e0 con 0x55b05820e380 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83d85de740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b05811bce0 con 0x55b05820eca0 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83d85de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b0581702a0 con 0x55b05821da90 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83ccff9640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83b80a3240 0x7f83b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b05820e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b05821d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b05820e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43066/0 (socket says 172.21.15.1:43066) 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 -- 172.21.15.1:0/2139514233 learned_addr learned my addr 172.21.15.1:0/2139514233 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83b80a3240 0x7f83b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83ccff9640 1 -- 172.21.15.1:0/811968956 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f83c00cf350 con 0x7f83d0100e20 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83ceffd640 1 --2- 172.21.15.1:0/2139514233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b05821fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 -- 172.21.15.1:0/2139514233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 msgr2=0x55b05821fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 --2- 172.21.15.1:0/2139514233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b05821fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 -- 172.21.15.1:0/2139514233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 msgr2=0x55b05821d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 --2- 172.21.15.1:0/2139514233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b05821d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 -- 172.21.15.1:0/2139514233 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b058100440 con 0x55b05820e380 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/2139514233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b05821d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.499+0000 7f83cffff640 1 --2- 172.21.15.1:0/2139514233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b05820e760 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f83c8005fd0 tx=0x7f83c80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86274f2b64e23894 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83cffff640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83b80a3240 0x7f83b80a5700 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f83c80058c0 tx=0x7f83c8015000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83b5ffb640 1 -- 172.21.15.1:0/2139514233 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c800ad20 con 0x55b05820e380 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83b5ffb640 1 -- 172.21.15.1:0/2139514233 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c80035f0 con 0x55b05820e380 2024-09-16T00:39:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83b5ffb640 1 -- 172.21.15.1:0/2139514233 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c801d650 con 0x55b05820e380 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83d85de740 1 -- 172.21.15.1:0/2139514233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 msgr2=0x55b05820e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83d85de740 1 --2- 172.21.15.1:0/2139514233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b05820e760 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f83c8005fd0 tx=0x7f83c80107a0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83d85de740 1 -- 172.21.15.1:0/2139514233 shutdown_connections 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83d85de740 1 --2- 172.21.15.1:0/2139514233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b05821fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83d85de740 1 --2- 172.21.15.1:0/2139514233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b05821d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83d85de740 1 --2- 172.21.15.1:0/2139514233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b05820e760 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83d85de740 1 -- 172.21.15.1:0/2139514233 >> 172.21.15.1:0/2139514233 conn(0x55b05820aa30 msgr2=0x55b05820ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.500+0000 7f83d85de740 1 -- 172.21.15.1:0/2139514233 shutdown_connections 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 1 -- 172.21.15.1:0/2139514233 wait complete. 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 1 -- start start 2024-09-16T00:39:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 1 Processor -- start 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820e380 0x55b05821d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820eca0 0x55b05821a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b05821ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b058220660 con 0x55b05820eca0 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b05812f050 con 0x55b05820e380 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b0582207e0 con 0x55b05821da90 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83d85de740 10 client.0 _ll_register_callbacks cb 0x55b05816f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83ceffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820eca0 0x55b05821a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83ceffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820eca0 0x55b05821a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43074/0 (socket says 172.21.15.1:43074) 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.501+0000 7f83ceffd640 1 -- 172.21.15.1:0/3888601490 learned_addr learned my addr 172.21.15.1:0/3888601490 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83cffff640 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b05821ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820e380 0x55b05821d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83ceffd640 1 -- 172.21.15.1:0/3888601490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 msgr2=0x55b05821ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83ceffd640 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b05821ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83ceffd640 1 -- 172.21.15.1:0/3888601490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820e380 msgr2=0x55b05821d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83ceffd640 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820e380 0x55b05821d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83ceffd640 1 -- 172.21.15.1:0/3888601490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b057ee53c0 con 0x55b05820eca0 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820e380 0x55b05821d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83cffff640 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b05821ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83ceffd640 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820eca0 0x55b05821a6f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009230 tx=0x7f83c4004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58799600cae940a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83aeffd640 1 -- 172.21.15.1:0/3888601490 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4035040 con 0x55b05820eca0 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83aeffd640 1 -- 172.21.15.1:0/3888601490 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f83c40046c0 con 0x55b05820eca0 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83d85de740 1 -- 172.21.15.1:0/3888601490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c80035f0 con 0x55b05820eca0 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83aeffd640 1 -- 172.21.15.1:0/3888601490 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83c404d050 con 0x55b05820eca0 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83d85de740 10 client.18084 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.502+0000 7f83d85de740 1 -- 172.21.15.1:0/3888601490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b05822c0b0 con 0x55b05820eca0 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.503+0000 7f83aeffd640 1 -- 172.21.15.1:0/3888601490 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c80035f0 con 0x55b05820eca0 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.503+0000 7f83aeffd640 1 -- 172.21.15.1:0/3888601490 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f83c4003b10 con 0x55b05820eca0 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.503+0000 7f83d85de740 10 client.18084 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.503+0000 7f83d85de740 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.503+0000 7f83d85de740 20 client.18084 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.503+0000 7f83d85de740 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b05822d570 0x55b05822f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.503+0000 7f83d85de740 4 client.18084 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.503+0000 7f83d85de740 1 -- 172.21.15.1:0/3888601490 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b058181850 con 0x55b05822d570 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.503+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b05822d570 0x55b05822f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.504+0000 7f83aeffd640 10 client.18084 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.504+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b05822d570 0x55b05822f990 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c24e1a6da21d03f3 server_cookie=e5c402d6b8205352 in_seq=0 out_seq=0 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.504+0000 7f83aeffd640 1 -- 172.21.15.1:0/3888601490 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b058181850 con 0x55b05822d570 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.504+0000 7f83aeffd640 10 client.18084 handle_command_reply: tid=0 2024-09-16T00:39:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.504+0000 7f83d85de740 1 client.18084 shutdown 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 -- 172.21.15.1:0/3888601490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820eca0 msgr2=0x55b05821a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820eca0 0x55b05821a6f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009230 tx=0x7f83c4004580 comp rx=0 tx=0).stop 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 -- 172.21.15.1:0/3888601490 shutdown_connections 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b05822d570 0x55b05822f990 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83aeffd640 0 client.18084 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b05821ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820eca0 0x55b05821a6f0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 --2- 172.21.15.1:0/3888601490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820e380 0x55b05821d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 -- 172.21.15.1:0/3888601490 >> 172.21.15.1:0/3888601490 conn(0x55b05820aa30 msgr2=0x55b058210790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 -- 172.21.15.1:0/3888601490 shutdown_connections 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 1 -- 172.21.15.1:0/3888601490 wait complete. 2024-09-16T00:39:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.505+0000 7f83d85de740 20 client.18084 trim_cache size 0 max 16384 2024-09-16T00:39:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83d85de740 1 Processor -- start 2024-09-16T00:39:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83d85de740 1 -- start start 2024-09-16T00:39:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83d85de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05820e380 0x55b058229900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83d85de740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b058235240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83d85de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05821da90 0x55b058235780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83d85de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01af600 con 0x55b05821da90 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83d85de740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01aced0 con 0x55b05820eca0 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83d85de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01aad20 con 0x55b05820e380 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05821da90 0x55b058235780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05821da90 0x55b058235780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43082/0 (socket says 172.21.15.1:43082) 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.522+0000 7f83cffff640 1 -- 172.21.15.1:0/3018194734 learned_addr learned my addr 172.21.15.1:0/3018194734 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83ceffd640 1 --2- 172.21.15.1:0/3018194734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b058235240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/3018194734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05820e380 0x55b058229900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83cffff640 1 -- 172.21.15.1:0/3018194734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05820e380 msgr2=0x55b058229900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83cffff640 1 --2- 172.21.15.1:0/3018194734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05820e380 0x55b058229900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83cffff640 1 -- 172.21.15.1:0/3018194734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 msgr2=0x55b058235240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83cffff640 1 --2- 172.21.15.1:0/3018194734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b058235240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83cffff640 1 -- 172.21.15.1:0/3018194734 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b05820ead0 con 0x55b05821da90 2024-09-16T00:39:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83cffff640 1 --2- 172.21.15.1:0/3018194734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05821da90 0x55b058235780 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f83c80046a0 tx=0x7f83c80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169a3dc9f634933a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83b57fa640 1 -- 172.21.15.1:0/3018194734 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8033070 con 0x55b05821da90 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83b57fa640 1 -- 172.21.15.1:0/3018194734 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c8013790 con 0x55b05821da90 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.523+0000 7f83b57fa640 1 -- 172.21.15.1:0/3018194734 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c801de30 con 0x55b05821da90 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 -- 172.21.15.1:0/3018194734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05821da90 msgr2=0x55b058235780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 --2- 172.21.15.1:0/3018194734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05821da90 0x55b058235780 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f83c80046a0 tx=0x7f83c80046d0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 -- 172.21.15.1:0/3018194734 shutdown_connections 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 --2- 172.21.15.1:0/3018194734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05821da90 0x55b058235780 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 --2- 172.21.15.1:0/3018194734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b058235240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 --2- 172.21.15.1:0/3018194734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05820e380 0x55b058229900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 -- 172.21.15.1:0/3018194734 >> 172.21.15.1:0/3018194734 conn(0x55b05820aa30 msgr2=0x55b058215d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 -- 172.21.15.1:0/3018194734 shutdown_connections 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 -- 172.21.15.1:0/3018194734 wait complete. 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 -- start start 2024-09-16T00:39:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.524+0000 7f83d85de740 1 Processor -- start 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83d85de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b058221f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83d85de740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b058222460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83d85de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b058297210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83d85de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01ab390 con 0x55b05820e380 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83d85de740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01ac800 con 0x55b05820eca0 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83d85de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01aba60 con 0x55b05821da90 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83d85de740 10 client.0 _ll_register_callbacks cb 0x7f83c4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b058221f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b058221f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43092/0 (socket says 172.21.15.1:43092) 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cf7fe640 1 -- 172.21.15.1:0/1976140152 learned_addr learned my addr 172.21.15.1:0/1976140152 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83ceffd640 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b058222460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cffff640 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b058297210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cf7fe640 1 -- 172.21.15.1:0/1976140152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 msgr2=0x55b058297210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b058297210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cf7fe640 1 -- 172.21.15.1:0/1976140152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 msgr2=0x55b058222460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b058222460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cf7fe640 1 -- 172.21.15.1:0/1976140152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b058297750 con 0x55b05820e380 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83cf7fe640 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b058221f20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000f80 tx=0x7f83c006b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc0ecc5deee736a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.525+0000 7f83aeffd640 1 -- 172.21.15.1:0/1976140152 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0023040 con 0x55b05820e380 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83aeffd640 1 -- 172.21.15.1:0/1976140152 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f83c0099870 con 0x55b05820e380 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83d85de740 1 -- 172.21.15.1:0/1976140152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b05817bd60 con 0x55b05820e380 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83aeffd640 1 -- 172.21.15.1:0/1976140152 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83c00cf160 con 0x55b05820e380 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83d85de740 10 client.18090 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83d85de740 1 -- 172.21.15.1:0/1976140152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0582989c0 con 0x55b05820e380 2024-09-16T00:39:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83aeffd640 1 -- 172.21.15.1:0/1976140152 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b05817bd60 con 0x55b05820e380 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83aeffd640 1 -- 172.21.15.1:0/1976140152 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f83c00a3e50 con 0x55b05820e380 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83d85de740 10 client.18090 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83d85de740 10 client.18090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83d85de740 20 client.18090 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83d85de740 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b05822d570 0x55b05829b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83d85de740 4 client.18090 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83d85de740 1 -- 172.21.15.1:0/1976140152 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0580f8210 con 0x55b05822d570 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.526+0000 7f83ceffd640 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b05822d570 0x55b05829b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83ceffd640 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b05822d570 0x55b05829b0f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=527303923014195f server_cookie=c21a17a875a92626 in_seq=0 out_seq=0 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83aeffd640 10 client.18090 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83aeffd640 1 -- 172.21.15.1:0/1976140152 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b0580f8210 con 0x55b05822d570 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83aeffd640 10 client.18090 handle_command_reply: tid=0 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 client.18090 shutdown 2024-09-16T00:39:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 -- 172.21.15.1:0/1976140152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 msgr2=0x55b058221f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b058221f20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000f80 tx=0x7f83c006b1e0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 -- 172.21.15.1:0/1976140152 shutdown_connections 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b05822d570 0x55b05829b0f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b05821da90 0x55b058297210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b05820eca0 0x55b058222460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 --2- 172.21.15.1:0/1976140152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b05820e380 0x55b058221f20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 -- 172.21.15.1:0/1976140152 >> 172.21.15.1:0/1976140152 conn(0x55b05820aa30 msgr2=0x55b05821bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83aeffd640 0 client.18090 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 -- 172.21.15.1:0/1976140152 shutdown_connections 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.527+0000 7f83d85de740 1 -- 172.21.15.1:0/1976140152 wait complete. 2024-09-16T00:39:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d85de740 20 client.18090 trim_cache size 0 max 16384 2024-09-16T00:39:26.535 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:39:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:39:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:39:26.535 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:39:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 -- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83b80a3240 msgr2=0x7f83b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83b80a3240 0x7f83b80a5700 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f83c80058c0 tx=0x7f83c8015000 comp rx=0 tx=0).stop 2024-09-16T00:39:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 -- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0100e20 msgr2=0x7f83d01993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0100e20 0x7f83d01993d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f83c000ba50 tx=0x7f83c000bf10 comp rx=0 tx=0).stop 2024-09-16T00:39:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 -- 172.21.15.1:0/811968956 shutdown_connections 2024-09-16T00:39:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f83b80a3240 0x7f83b80a5700 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d0102ef0 0x7f83d01a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d0102570 0x7f83d0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 --2- 172.21.15.1:0/811968956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83d0100e20 0x7f83d01993d0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 -- 172.21.15.1:0/811968956 >> 172.21.15.1:0/811968956 conn(0x7f83d00fc7d0 msgr2=0x7f83d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 -- 172.21.15.1:0/811968956 shutdown_connections 2024-09-16T00:39:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:26.528+0000 7f83d6205640 1 -- 172.21.15.1:0/811968956 wait complete. 2024-09-16T00:39:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:26 smithi019 ceph-mon[35186]: pgmap v681: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:27.346 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:39:27.347 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:39:27.608 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:28.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.118+0000 7fabdc57d640 1 -- 172.21.15.1:0/215732062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 msgr2=0x7fabd4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:28.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.118+0000 7fabdc57d640 1 --2- 172.21.15.1:0/215732062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 0x7fabd4101200 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009900 tx=0x7fabcc02a480 comp rx=0 tx=0).stop 2024-09-16T00:39:28.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.118+0000 7fabdc57d640 1 -- 172.21.15.1:0/215732062 shutdown_connections 2024-09-16T00:39:28.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.118+0000 7fabdc57d640 1 --2- 172.21.15.1:0/215732062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd4102ef0 0x7fabd410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.118+0000 7fabdc57d640 1 --2- 172.21.15.1:0/215732062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd4102570 0x7fabd41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.118+0000 7fabdc57d640 1 --2- 172.21.15.1:0/215732062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 0x7fabd4101200 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.118+0000 7fabdc57d640 1 -- 172.21.15.1:0/215732062 >> 172.21.15.1:0/215732062 conn(0x7fabd40fc7d0 msgr2=0x7fabd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.119+0000 7fabdc57d640 1 -- 172.21.15.1:0/215732062 shutdown_connections 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.119+0000 7fabdc57d640 1 -- 172.21.15.1:0/215732062 wait complete. 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.119+0000 7fabdc57d640 1 Processor -- start 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.119+0000 7fabdc57d640 1 -- start start 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabdc57d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 0x7fabd4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabdc57d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd4102570 0x7fabd4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabdc57d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd4102ef0 0x7fabd419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabdc57d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4110fd0 con 0x7fabd4100e20 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabdc57d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4110e50 con 0x7fabd4102570 2024-09-16T00:39:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabdc57d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4111150 con 0x7fabd4102ef0 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabda319640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 0x7fabd4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabda319640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 0x7fabd4198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43104/0 (socket says 172.21.15.1:43104) 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabda319640 1 -- 172.21.15.1:0/3417727885 learned_addr learned my addr 172.21.15.1:0/3417727885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabdab1a640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd4102ef0 0x7fabd419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabd9b18640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd4102570 0x7fabd4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabda319640 1 -- 172.21.15.1:0/3417727885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd4102ef0 msgr2=0x7fabd419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabda319640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd4102ef0 0x7fabd419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabda319640 1 -- 172.21.15.1:0/3417727885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd4102570 msgr2=0x7fabd4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabda319640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd4102570 0x7fabd4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabda319640 1 -- 172.21.15.1:0/3417727885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd419d530 con 0x7fabd4100e20 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabd9b18640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd4102570 0x7fabd4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.120+0000 7fabda319640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 0x7fabd4198580 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fabcc02aa70 tx=0x7fabcc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.121+0000 7fabd37fe640 1 -- 172.21.15.1:0/3417727885 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc03f070 con 0x7fabd4100e20 2024-09-16T00:39:28.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.121+0000 7fabd37fe640 1 -- 172.21.15.1:0/3417727885 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabcc031c30 con 0x7fabd4100e20 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.121+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd419d7c0 con 0x7fabd4100e20 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.121+0000 7fabd37fe640 1 -- 172.21.15.1:0/3417727885 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc035030 con 0x7fabd4100e20 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.121+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd41a5100 con 0x7fabd4100e20 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.122+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd4101200 con 0x7fabd4100e20 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.124+0000 7fabd37fe640 1 -- 172.21.15.1:0/3417727885 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fabcc042070 con 0x7fabd4100e20 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.125+0000 7fabd37fe640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabbc0a32c0 0x7fabbc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.125+0000 7fabd9b18640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabbc0a32c0 0x7fabbc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.125+0000 7fabd37fe640 1 -- 172.21.15.1:0/3417727885 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fabcc0f0310 con 0x7fabd4100e20 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.125+0000 7fabd9b18640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabbc0a32c0 0x7fabbc0a5780 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fabd40ffd20 tx=0x7fabc800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:28.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.129+0000 7fabd37fe640 1 -- 172.21.15.1:0/3417727885 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabcc031de0 con 0x7fabd4100e20 2024-09-16T00:39:28.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.344+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabd41a52d0 con 0x7fabd4100e20 2024-09-16T00:39:28.345 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:28 smithi001 ceph-mon[30934]: pgmap v682: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:28.345 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:39:28.345 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:39:28.345 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:28 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:39:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.345+0000 7fabd37fe640 1 -- 172.21.15.1:0/3417727885 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fabcc0bc5e0 con 0x7fabd4100e20 2024-09-16T00:39:28.346 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:39:28.346 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:39:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:39:28.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.347+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabbc0a32c0 msgr2=0x7fabbc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:28.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.347+0000 7fabdc57d640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabbc0a32c0 0x7fabbc0a5780 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fabd40ffd20 tx=0x7fabc800b390 comp rx=0 tx=0).stop 2024-09-16T00:39:28.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.347+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 msgr2=0x7fabd4198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:28.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.347+0000 7fabdc57d640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 0x7fabd4198580 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fabcc02aa70 tx=0x7fabcc003c10 comp rx=0 tx=0).stop 2024-09-16T00:39:28.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.347+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 shutdown_connections 2024-09-16T00:39:28.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.347+0000 7fabdc57d640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fabbc0a32c0 0x7fabbc0a5780 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.348+0000 7fabdc57d640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd4102ef0 0x7fabd419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.348+0000 7fabdc57d640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd4102570 0x7fabd4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.348+0000 7fabdc57d640 1 --2- 172.21.15.1:0/3417727885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabd4100e20 0x7fabd4198580 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.348+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 >> 172.21.15.1:0/3417727885 conn(0x7fabd40fc7d0 msgr2=0x7fabd406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:28.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.348+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 shutdown_connections 2024-09-16T00:39:28.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:28.348+0000 7fabdc57d640 1 -- 172.21.15.1:0/3417727885 wait complete. 2024-09-16T00:39:28.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:28 smithi153 ceph-mon[35538]: pgmap v682: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:28.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:39:28.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:39:28.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:28 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:39:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:28 smithi019 ceph-mon[35186]: pgmap v682: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:39:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:39:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:28 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:39:29.139 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:39:29.402 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:29 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3417727885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:39:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:39:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:39:29.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:29 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3417727885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:29.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:39:29.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:39:29.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:39:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:29 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3417727885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:39:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:39:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:39:29.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 -- 172.21.15.1:0/2400330800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724106f10 msgr2=0x7f372410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 --2- 172.21.15.1:0/2400330800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724106f10 0x7f372410ea90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f371c009ca0 tx=0x7f371c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 -- 172.21.15.1:0/2400330800 shutdown_connections 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 --2- 172.21.15.1:0/2400330800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724106f10 0x7f372410ea90 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 --2- 172.21.15.1:0/2400330800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3724102790 0x7f37241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 --2- 172.21.15.1:0/2400330800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3724101de0 0x7f37241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 -- 172.21.15.1:0/2400330800 >> 172.21.15.1:0/2400330800 conn(0x7f37240fd8c0 msgr2=0x7f37240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 -- 172.21.15.1:0/2400330800 shutdown_connections 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 -- 172.21.15.1:0/2400330800 wait complete. 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 Processor -- start 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.932+0000 7f372a080640 1 -- start start 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f372a080640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724101de0 0x7f37241983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f372a080640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3724102790 0x7f3724198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f372a080640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3724106f10 0x7f372419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f372a080640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3724111fc0 con 0x7f3724101de0 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f372a080640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3724111e40 con 0x7f3724106f10 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f372a080640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3724112140 con 0x7f3724102790 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f37237fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724101de0 0x7f37241983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f37237fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724101de0 0x7f37241983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43122/0 (socket says 172.21.15.1:43122) 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f37237fe640 1 -- 172.21.15.1:0/1854716562 learned_addr learned my addr 172.21.15.1:0/1854716562 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f3723fff640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3724106f10 0x7f372419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f371affd640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3724102790 0x7f3724198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f37237fe640 1 -- 172.21.15.1:0/1854716562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3724102790 msgr2=0x7f3724198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f37237fe640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3724102790 0x7f3724198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f37237fe640 1 -- 172.21.15.1:0/1854716562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3724106f10 msgr2=0x7f372419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f37237fe640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3724106f10 0x7f372419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f37237fe640 1 -- 172.21.15.1:0/1854716562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372419d380 con 0x7f3724101de0 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.933+0000 7f37237fe640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724101de0 0x7f37241983d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f371400e9f0 tx=0x7f371400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.934+0000 7f37217fa640 1 -- 172.21.15.1:0/1854716562 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371400cd80 con 0x7f3724101de0 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.934+0000 7f37217fa640 1 -- 172.21.15.1:0/1854716562 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3714005500 con 0x7f3724101de0 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.934+0000 7f372a080640 1 -- 172.21.15.1:0/1854716562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372419d670 con 0x7f3724101de0 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.934+0000 7f37217fa640 1 -- 172.21.15.1:0/1854716562 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714010690 con 0x7f3724101de0 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.934+0000 7f372a080640 1 -- 172.21.15.1:0/1854716562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37241a4f50 con 0x7f3724101de0 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.936+0000 7f372c459740 1 Processor -- start 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.936+0000 7f372c459740 1 -- start start 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.936+0000 7f372c459740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c949090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.936+0000 7f372c459740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 0x55ee8c949a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f372c459740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 0x55ee8c956760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f372c459740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c8af2f0 con 0x55ee8c9495d0 2024-09-16T00:39:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f372c459740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c856cf0 con 0x55ee8c94ab40 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f372c459740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c8ab2b0 con 0x55ee8c954370 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3723fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c949090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3723fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c949090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:57206/0 (socket says 172.21.15.1:57206) 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3723fff640 1 -- 172.21.15.1:0/1304522592 learned_addr learned my addr 172.21.15.1:0/1304522592 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f371affd640 1 --2- 172.21.15.1:0/1304522592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 0x55ee8c956760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f37217fa640 1 -- 172.21.15.1:0/1854716562 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f37140046c0 con 0x7f3724101de0 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3723fff640 1 -- 172.21.15.1:0/1304522592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 msgr2=0x55ee8c956760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3723fff640 1 --2- 172.21.15.1:0/1304522592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 0x55ee8c956760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3723fff640 1 -- 172.21.15.1:0/1304522592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 msgr2=0x55ee8c949a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3723fff640 1 --2- 172.21.15.1:0/1304522592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 0x55ee8c949a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3723fff640 1 -- 172.21.15.1:0/1304522592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8c83b3a0 con 0x55ee8c94ab40 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3723fff640 1 --2- 172.21.15.1:0/1304522592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c949090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f371c009ca0 tx=0x7f371c0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5795adfae97fe27 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.937+0000 7f3719ffb640 1 -- 172.21.15.1:0/1304522592 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c031db0 con 0x55ee8c94ab40 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f3719ffb640 1 -- 172.21.15.1:0/1304522592 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c036030 con 0x55ee8c94ab40 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f3719ffb640 1 -- 172.21.15.1:0/1304522592 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0042a0 con 0x55ee8c94ab40 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f37217fa640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37100a32c0 0x7f37100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f37237fe640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37100a32c0 0x7f37100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f37217fa640 1 -- 172.21.15.1:0/1854716562 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f37140d0090 con 0x7f3724101de0 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f372c459740 1 -- 172.21.15.1:0/1304522592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 msgr2=0x55ee8c949090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f372c459740 1 --2- 172.21.15.1:0/1304522592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c949090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f371c009ca0 tx=0x7f371c0093e0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f372c459740 1 -- 172.21.15.1:0/1304522592 shutdown_connections 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f372c459740 1 --2- 172.21.15.1:0/1304522592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 0x55ee8c956760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f372c459740 1 --2- 172.21.15.1:0/1304522592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 0x55ee8c949a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f372c459740 1 --2- 172.21.15.1:0/1304522592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c949090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f372c459740 1 -- 172.21.15.1:0/1304522592 >> 172.21.15.1:0/1304522592 conn(0x55ee8c912310 msgr2=0x55ee8c945c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.938+0000 7f372c459740 1 -- 172.21.15.1:0/1304522592 shutdown_connections 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 1 -- 172.21.15.1:0/1304522592 wait complete. 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f37237fe640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37100a32c0 0x7f37100a5780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3714098710 tx=0x7f3714004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 1 -- start start 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 1 Processor -- start 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c9495d0 0x55ee8c958560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c958aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c95cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c956e90 con 0x55ee8c954370 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c86a060 con 0x55ee8c94ab40 2024-09-16T00:39:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c957010 con 0x55ee8c9495d0 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f372c459740 10 client.0 _ll_register_callbacks cb 0x55ee8c5bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.939+0000 7f3723fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c95cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3723fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c95cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43136/0 (socket says 172.21.15.1:43136) 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3723fff640 1 -- 172.21.15.1:0/1943962212 learned_addr learned my addr 172.21.15.1:0/1943962212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f371affd640 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c958aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3723fff640 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c9495d0 0x55ee8c958560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3723fff640 1 -- 172.21.15.1:0/1943962212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c9495d0 msgr2=0x55ee8c958560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3723fff640 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c9495d0 0x55ee8c958560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3723fff640 1 -- 172.21.15.1:0/1943962212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 msgr2=0x55ee8c958aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3723fff640 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c958aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3723fff640 1 -- 172.21.15.1:0/1943962212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8c6203c0 con 0x55ee8c954370 2024-09-16T00:39:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3723fff640 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c95cc10 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f371c009c70 tx=0x7f371c031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cdb69c8fe98be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3702ffd640 1 -- 172.21.15.1:0/1943962212 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c058060 con 0x55ee8c954370 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3702ffd640 1 -- 172.21.15.1:0/1943962212 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f371c0045b0 con 0x55ee8c954370 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f372c459740 1 -- 172.21.15.1:0/1943962212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371c036030 con 0x55ee8c954370 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f3702ffd640 1 -- 172.21.15.1:0/1943962212 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f371c05f050 con 0x55ee8c954370 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f372c459740 10 client.18105 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.940+0000 7f372c459740 1 -- 172.21.15.1:0/1943962212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee8c956e90 con 0x55ee8c954370 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.941+0000 7f3702ffd640 1 -- 172.21.15.1:0/1943962212 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c036030 con 0x55ee8c954370 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.941+0000 7f3702ffd640 1 -- 172.21.15.1:0/1943962212 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f371c050070 con 0x55ee8c954370 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.941+0000 7f372c459740 10 client.18105 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.941+0000 7f372c459740 10 client.18105 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.941+0000 7f372c459740 20 client.18105 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.941+0000 7f372c459740 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ee8c967ee0 0x55ee8c96a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.941+0000 7f372c459740 4 client.18105 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.941+0000 7f372c459740 1 -- 172.21.15.1:0/1943962212 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee8c8bc860 con 0x55ee8c967ee0 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.941+0000 7f371affd640 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ee8c967ee0 0x55ee8c96a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.942+0000 7f3702ffd640 10 client.18105 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.942+0000 7f371affd640 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ee8c967ee0 0x55ee8c96a300 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33d947e4ac996e08 server_cookie=dd50e1445caa9ece in_seq=0 out_seq=0 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.942+0000 7f3702ffd640 1 -- 172.21.15.1:0/1943962212 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee8c8bc860 con 0x55ee8c967ee0 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.942+0000 7f3702ffd640 10 client.18105 handle_command_reply: tid=0 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.942+0000 7f372c459740 1 client.18105 shutdown 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.942+0000 7f372c459740 1 -- 172.21.15.1:0/1943962212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 msgr2=0x55ee8c95cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.942+0000 7f372c459740 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c95cc10 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f371c009c70 tx=0x7f371c031930 comp rx=0 tx=0).stop 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.942+0000 7f372c459740 1 -- 172.21.15.1:0/1943962212 shutdown_connections 2024-09-16T00:39:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.942+0000 7f372c459740 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c95cc10 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.943+0000 7f372c459740 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ee8c967ee0 0x55ee8c96a300 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.943+0000 7f372c459740 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c958aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.943+0000 7f372c459740 1 --2- 172.21.15.1:0/1943962212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c9495d0 0x55ee8c958560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.943+0000 7f372c459740 1 -- 172.21.15.1:0/1943962212 >> 172.21.15.1:0/1943962212 conn(0x55ee8c912310 msgr2=0x55ee8c94b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.943+0000 7f3702ffd640 0 client.18105 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.943+0000 7f372c459740 1 -- 172.21.15.1:0/1943962212 shutdown_connections 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.943+0000 7f372c459740 1 -- 172.21.15.1:0/1943962212 wait complete. 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.943+0000 7f372c459740 20 client.18105 trim_cache size 0 max 16384 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f372c459740 1 Processor -- start 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f372c459740 1 -- start start 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f372c459740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c9495d0 0x55ee8c95cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f372c459740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c945e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f372c459740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c957260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f372c459740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37241af3b0 con 0x55ee8c954370 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f372c459740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37241acc80 con 0x55ee8c94ab40 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f372c459740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37241aaad0 con 0x55ee8c9495d0 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f3723fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c957260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f3723fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c957260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43140/0 (socket says 172.21.15.1:43140) 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f3723fff640 1 -- 172.21.15.1:0/4245953709 learned_addr learned my addr 172.21.15.1:0/4245953709 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f3723fff640 1 -- 172.21.15.1:0/4245953709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c9495d0 msgr2=0x55ee8c95cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f3723fff640 1 --2- 172.21.15.1:0/4245953709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c9495d0 0x55ee8c95cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f3723fff640 1 -- 172.21.15.1:0/4245953709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 msgr2=0x55ee8c945e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f371affd640 1 --2- 172.21.15.1:0/4245953709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c945e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f3723fff640 1 --2- 172.21.15.1:0/4245953709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c945e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f3723fff640 1 -- 172.21.15.1:0/4245953709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8c953960 con 0x55ee8c954370 2024-09-16T00:39:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.956+0000 7f3723fff640 1 --2- 172.21.15.1:0/4245953709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c957260 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f371c02ade0 tx=0x7f371c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d8f95c3654d291 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.957+0000 7f37197fa640 1 -- 172.21.15.1:0/4245953709 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c058030 con 0x55ee8c954370 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.957+0000 7f37197fa640 1 -- 172.21.15.1:0/4245953709 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c036070 con 0x55ee8c954370 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.957+0000 7f37197fa640 1 -- 172.21.15.1:0/4245953709 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c031760 con 0x55ee8c954370 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.957+0000 7f372c459740 1 -- 172.21.15.1:0/4245953709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 msgr2=0x55ee8c957260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.957+0000 7f372c459740 1 --2- 172.21.15.1:0/4245953709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c957260 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f371c02ade0 tx=0x7f371c0095a0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.957+0000 7f372c459740 1 -- 172.21.15.1:0/4245953709 shutdown_connections 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.957+0000 7f372c459740 1 --2- 172.21.15.1:0/4245953709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c954370 0x55ee8c957260 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.957+0000 7f372c459740 1 --2- 172.21.15.1:0/4245953709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c945e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.957+0000 7f372c459740 1 --2- 172.21.15.1:0/4245953709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c9495d0 0x55ee8c95cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.958+0000 7f372c459740 1 -- 172.21.15.1:0/4245953709 >> 172.21.15.1:0/4245953709 conn(0x55ee8c912310 msgr2=0x55ee8c945700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.958+0000 7f372c459740 1 -- 172.21.15.1:0/4245953709 shutdown_connections 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.958+0000 7f372c459740 1 -- 172.21.15.1:0/4245953709 wait complete. 2024-09-16T00:39:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.958+0000 7f372c459740 1 -- start start 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.958+0000 7f372c459740 1 Processor -- start 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f372c459740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 0x55ee8c95fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f372c459740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c9602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f372c459740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 0x55ee8c9607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f372c459740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37241ab140 con 0x55ee8c9495d0 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f372c459740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37241ac5b0 con 0x55ee8c94ab40 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f372c459740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37241ab810 con 0x55ee8c954370 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f372c459740 10 client.0 _ll_register_callbacks cb 0x7f370c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 0x55ee8c95fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 0x55ee8c95fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43144/0 (socket says 172.21.15.1:43144) 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 -- 172.21.15.1:0/2152815439 learned_addr learned my addr 172.21.15.1:0/2152815439 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f371affd640 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c9602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 0x55ee8c9607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 -- 172.21.15.1:0/2152815439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 msgr2=0x55ee8c9607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 0x55ee8c9607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 -- 172.21.15.1:0/2152815439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 msgr2=0x55ee8c9602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c9602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 -- 172.21.15.1:0/2152815439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8c9d1d10 con 0x55ee8c9495d0 2024-09-16T00:39:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f371affd640 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c9602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:39:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3723fff640 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 0x55ee8c95fd70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f371c000f80 tx=0x7f371c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279638f850940b97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:29.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.959+0000 7f3702ffd640 1 -- 172.21.15.1:0/2152815439 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c058040 con 0x55ee8c9495d0 2024-09-16T00:39:29.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f3702ffd640 1 -- 172.21.15.1:0/2152815439 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f371c004390 con 0x55ee8c9495d0 2024-09-16T00:39:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f372c459740 1 -- 172.21.15.1:0/2152815439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371c036070 con 0x55ee8c9495d0 2024-09-16T00:39:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f3702ffd640 1 -- 172.21.15.1:0/2152815439 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f371c02faa0 con 0x55ee8c9495d0 2024-09-16T00:39:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f372c459740 10 client.18111 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f372c459740 1 -- 172.21.15.1:0/2152815439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee8c9d2fd0 con 0x55ee8c9495d0 2024-09-16T00:39:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f3702ffd640 1 -- 172.21.15.1:0/2152815439 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c036070 con 0x55ee8c9495d0 2024-09-16T00:39:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f3702ffd640 1 -- 172.21.15.1:0/2152815439 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f371c050040 con 0x55ee8c9495d0 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f372c459740 10 client.18111 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f372c459740 10 client.18111 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f372c459740 20 client.18111 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f372c459740 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ee8c9d3360 0x55ee8c9d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f372c459740 4 client.18111 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.960+0000 7f372c459740 1 -- 172.21.15.1:0/2152815439 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee8c8b6d70 con 0x55ee8c9d3360 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.961+0000 7f371affd640 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ee8c9d3360 0x55ee8c9d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.961+0000 7f3702ffd640 10 client.18111 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.961+0000 7f371affd640 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ee8c9d3360 0x55ee8c9d5750 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=619aaa475bd939e2 server_cookie=cf185fbcd60cc88d in_seq=0 out_seq=0 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.961+0000 7f3702ffd640 1 -- 172.21.15.1:0/2152815439 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee8c8b6d70 con 0x55ee8c9d3360 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.961+0000 7f3702ffd640 10 client.18111 handle_command_reply: tid=0 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.961+0000 7f372c459740 1 client.18111 shutdown 2024-09-16T00:39:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.961+0000 7f372c459740 1 -- 172.21.15.1:0/2152815439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 msgr2=0x55ee8c95fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.961+0000 7f372c459740 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 0x55ee8c95fd70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f371c000f80 tx=0x7f371c005cf0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.961+0000 7f372c459740 1 -- 172.21.15.1:0/2152815439 shutdown_connections 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372c459740 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ee8c954370 0x55ee8c9607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372c459740 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55ee8c9d3360 0x55ee8c9d5750 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372c459740 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee8c94ab40 0x55ee8c9602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372c459740 1 --2- 172.21.15.1:0/2152815439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ee8c9495d0 0x55ee8c95fd70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372c459740 1 -- 172.21.15.1:0/2152815439 >> 172.21.15.1:0/2152815439 conn(0x55ee8c912310 msgr2=0x55ee8c94b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f3702ffd640 0 client.18111 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372c459740 1 -- 172.21.15.1:0/2152815439 shutdown_connections 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372c459740 1 -- 172.21.15.1:0/2152815439 wait complete. 2024-09-16T00:39:29.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372c459740 20 client.18111 trim_cache size 0 max 16384 2024-09-16T00:39:29.980 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:39:29.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 -- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37100a32c0 msgr2=0x7f37100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37100a32c0 0x7f37100a5780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3714098710 tx=0x7f3714004a40 comp rx=0 tx=0).stop 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 -- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724101de0 msgr2=0x7f37241983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724101de0 0x7f37241983d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f371400e9f0 tx=0x7f371400edf0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 -- 172.21.15.1:0/1854716562 shutdown_connections 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3724106f10 0x7f372419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f37100a32c0 0x7f37100a5780 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3724102790 0x7f3724198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 --2- 172.21.15.1:0/1854716562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3724101de0 0x7f37241983d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.962+0000 7f372a080640 1 -- 172.21.15.1:0/1854716562 >> 172.21.15.1:0/1854716562 conn(0x7f37240fd8c0 msgr2=0x7f37240fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.963+0000 7f372a080640 1 -- 172.21.15.1:0/1854716562 shutdown_connections 2024-09-16T00:39:29.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:29.963+0000 7f372a080640 1 -- 172.21.15.1:0/1854716562 wait complete. 2024-09-16T00:39:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:30 smithi001 ceph-mon[30934]: pgmap v683: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:30.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:30 smithi153 ceph-mon[35538]: pgmap v683: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:30.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:30 smithi019 ceph-mon[35186]: pgmap v683: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:30.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:39:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.549+0000 7ffb3318f640 1 -- 172.21.15.1:0/3388089296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c102790 msgr2=0x7ffb2c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.549+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3388089296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c102790 0x7ffb2c1067e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009900 tx=0x7ffb1802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.549+0000 7ffb3318f640 1 -- 172.21.15.1:0/3388089296 shutdown_connections 2024-09-16T00:39:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.549+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3388089296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb2c106f10 0x7ffb2c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.549+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3388089296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c102790 0x7ffb2c1067e0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.549+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3388089296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb2c101de0 0x7ffb2c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.549+0000 7ffb3318f640 1 -- 172.21.15.1:0/3388089296 >> 172.21.15.1:0/3388089296 conn(0x7ffb2c0fd8c0 msgr2=0x7ffb2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.549+0000 7ffb3318f640 1 -- 172.21.15.1:0/3388089296 shutdown_connections 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.549+0000 7ffb3318f640 1 -- 172.21.15.1:0/3388089296 wait complete. 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3318f640 1 Processor -- start 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3318f640 1 -- start start 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3318f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb2c101de0 0x7ffb2c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3318f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb2c102790 0x7ffb2c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3318f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c106f10 0x7ffb2c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3318f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c111f30 con 0x7ffb2c106f10 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3318f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c111db0 con 0x7ffb2c101de0 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3318f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c1120b0 con 0x7ffb2c102790 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3172c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c106f10 0x7ffb2c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3172c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c106f10 0x7ffb2c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39232/0 (socket says 172.21.15.1:39232) 2024-09-16T00:39:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3172c640 1 -- 172.21.15.1:0/3855900239 learned_addr learned my addr 172.21.15.1:0/3855900239 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb30f2b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb2c101de0 0x7ffb2c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb3172c640 1 -- 172.21.15.1:0/3855900239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb2c102790 msgr2=0x7ffb2c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.550+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb2c102790 0x7ffb2c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb3172c640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb2c102790 0x7ffb2c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb3172c640 1 -- 172.21.15.1:0/3855900239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb2c101de0 msgr2=0x7ffb2c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb3172c640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb2c101de0 0x7ffb2c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb3172c640 1 -- 172.21.15.1:0/3855900239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2c19d2c0 con 0x7ffb2c106f10 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb2c102790 0x7ffb2c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb3172c640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c106f10 0x7ffb2c19cbe0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400cce0 tx=0x7ffb2400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb29ffb640 1 -- 172.21.15.1:0/3855900239 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400f450 con 0x7ffb2c106f10 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb29ffb640 1 -- 172.21.15.1:0/3855900239 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb2400fad0 con 0x7ffb2c106f10 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb3318f640 1 -- 172.21.15.1:0/3855900239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2c19d5b0 con 0x7ffb2c106f10 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb29ffb640 1 -- 172.21.15.1:0/3855900239 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24018b60 con 0x7ffb2c106f10 2024-09-16T00:39:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.551+0000 7ffb3318f640 1 -- 172.21.15.1:0/3855900239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2c1a4e90 con 0x7ffb2c106f10 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.554+0000 7ffb29ffb640 1 -- 172.21.15.1:0/3855900239 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ffb2400fc80 con 0x7ffb2c106f10 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.554+0000 7ffb35568740 1 Processor -- start 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.554+0000 7ffb35568740 1 -- start start 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb35568740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfeb40 0x558046cfd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb35568740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfd5d0 0x558046cfda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb35568740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d0a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb35568740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558046c632f0 con 0x558046cfd5d0 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb35568740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558046c0acf0 con 0x558046d08370 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb35568740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558046c5f2b0 con 0x558046cfeb40 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb30f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfd5d0 0x558046cfda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb30f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfd5d0 0x558046cfda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39234/0 (socket says 172.21.15.1:39234) 2024-09-16T00:39:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb30f2b640 1 -- 172.21.15.1:0/2843352788 learned_addr learned my addr 172.21.15.1:0/2843352788 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfeb40 0x558046cfd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb29ffb640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb1c0a3240 0x7ffb1c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d0a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb30f2b640 1 -- 172.21.15.1:0/2843352788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfeb40 msgr2=0x558046cfd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfeb40 0x558046cfd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb30f2b640 1 -- 172.21.15.1:0/2843352788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 msgr2=0x558046d0a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d0a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb30f2b640 1 -- 172.21.15.1:0/2843352788 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558046bef3a0 con 0x558046cfd5d0 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb3172c640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb1c0a3240 0x7ffb1c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d0a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb29ffb640 1 -- 172.21.15.1:0/3855900239 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ffb240d0220 con 0x7ffb2c106f10 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.555+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfd5d0 0x558046cfda10 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000ad00 tx=0x7ffb2000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33415a0ac1f8600e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb16ffd640 1 -- 172.21.15.1:0/2843352788 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2001d070 con 0x558046cfd5d0 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb3172c640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb1c0a3240 0x7ffb1c0a5700 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffb24098840 tx=0x7ffb24066d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb16ffd640 1 -- 172.21.15.1:0/2843352788 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb20018b20 con 0x558046cfd5d0 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb16ffd640 1 -- 172.21.15.1:0/2843352788 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20010bc0 con 0x558046cfd5d0 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb35568740 1 -- 172.21.15.1:0/2843352788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfd5d0 msgr2=0x558046cfda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb35568740 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfd5d0 0x558046cfda10 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000ad00 tx=0x7ffb2000f790 comp rx=0 tx=0).stop 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb35568740 1 -- 172.21.15.1:0/2843352788 shutdown_connections 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb35568740 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d0a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb35568740 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfd5d0 0x558046cfda10 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb35568740 1 --2- 172.21.15.1:0/2843352788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfeb40 0x558046cfd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb35568740 1 -- 172.21.15.1:0/2843352788 >> 172.21.15.1:0/2843352788 conn(0x558046cc6310 msgr2=0x558046cf9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb35568740 1 -- 172.21.15.1:0/2843352788 shutdown_connections 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.556+0000 7ffb35568740 1 -- 172.21.15.1:0/2843352788 wait complete. 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb35568740 1 -- start start 2024-09-16T00:39:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb35568740 1 Processor -- start 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb35568740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 0x558046d07ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb35568740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d05030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb35568740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d05590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb35568740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558046d0b070 con 0x558046cfeb40 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb35568740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558046c1e060 con 0x558046d08370 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb35568740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558046d0b1f0 con 0x558046cfd5d0 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb35568740 10 client.0 _ll_register_callbacks cb 0x5580469718c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb30f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d05030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb30f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d05030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39244/0 (socket says 172.21.15.1:39244) 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.557+0000 7ffb30f2b640 1 -- 172.21.15.1:0/2532371693 learned_addr learned my addr 172.21.15.1:0/2532371693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d05590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 0x558046d07ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb30f2b640 1 -- 172.21.15.1:0/2532371693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 msgr2=0x558046d07ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 0x558046d07ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb30f2b640 1 -- 172.21.15.1:0/2532371693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 msgr2=0x558046d05590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d05590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb30f2b640 1 -- 172.21.15.1:0/2532371693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580469d43c0 con 0x558046cfeb40 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 0x558046d07ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d05030 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000a550 tx=0x7ffb200186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293d9bbcf281693c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb0bfff640 1 -- 172.21.15.1:0/2532371693 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2001d040 con 0x558046cfeb40 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb0bfff640 1 -- 172.21.15.1:0/2532371693 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ffb200108e0 con 0x558046cfeb40 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb35568740 1 -- 172.21.15.1:0/2532371693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb20018b20 con 0x558046cfeb40 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb0bfff640 1 -- 172.21.15.1:0/2532371693 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb20031050 con 0x558046cfeb40 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb35568740 10 client.18123 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.558+0000 7ffb35568740 1 -- 172.21.15.1:0/2532371693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558046d16a80 con 0x558046cfeb40 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.559+0000 7ffb0bfff640 1 -- 172.21.15.1:0/2532371693 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb20018b20 con 0x558046cfeb40 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.559+0000 7ffb0bfff640 1 -- 172.21.15.1:0/2532371693 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ffb20020030 con 0x558046cfeb40 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.559+0000 7ffb35568740 10 client.18123 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.559+0000 7ffb35568740 10 client.18123 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.559+0000 7ffb35568740 20 client.18123 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.559+0000 7ffb35568740 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x558046d17e80 0x558046d1a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.559+0000 7ffb35568740 4 client.18123 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.559+0000 7ffb35568740 1 -- 172.21.15.1:0/2532371693 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558046c70860 con 0x558046d17e80 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.559+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x558046d17e80 0x558046d1a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.560+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x558046d17e80 0x558046d1a270 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=860bbd96416ae466 server_cookie=9c16e3e4553e3934 in_seq=0 out_seq=0 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.560+0000 7ffb0bfff640 10 client.18123 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.560+0000 7ffb0bfff640 1 -- 172.21.15.1:0/2532371693 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558046c70860 con 0x558046d17e80 2024-09-16T00:39:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.560+0000 7ffb0bfff640 10 client.18123 handle_command_reply: tid=0 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.560+0000 7ffb35568740 1 client.18123 shutdown 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.560+0000 7ffb35568740 1 -- 172.21.15.1:0/2532371693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 msgr2=0x558046d05030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.560+0000 7ffb35568740 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d05030 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000a550 tx=0x7ffb200186f0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb35568740 1 -- 172.21.15.1:0/2532371693 shutdown_connections 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb35568740 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d05590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb35568740 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x558046d17e80 0x558046d1a270 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb35568740 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d05030 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb35568740 1 --2- 172.21.15.1:0/2532371693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 0x558046d07ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb35568740 1 -- 172.21.15.1:0/2532371693 >> 172.21.15.1:0/2532371693 conn(0x558046cc6310 msgr2=0x558046cff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb0bfff640 0 client.18123 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb35568740 1 -- 172.21.15.1:0/2532371693 shutdown_connections 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb35568740 1 -- 172.21.15.1:0/2532371693 wait complete. 2024-09-16T00:39:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.561+0000 7ffb35568740 20 client.18123 trim_cache size 0 max 16384 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.574+0000 7ffb35568740 1 Processor -- start 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.574+0000 7ffb35568740 1 -- start start 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.574+0000 7ffb35568740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 0x558046d14310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.574+0000 7ffb35568740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046cf8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.574+0000 7ffb35568740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d1f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.574+0000 7ffb35568740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c1b1670 con 0x558046cfeb40 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.574+0000 7ffb35568740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c1aef40 con 0x558046d08370 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.574+0000 7ffb35568740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c1acd90 con 0x558046cfd5d0 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb30f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046cf8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb30f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046cf8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39250/0 (socket says 172.21.15.1:39250) 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb30f2b640 1 -- 172.21.15.1:0/3211854136 learned_addr learned my addr 172.21.15.1:0/3211854136 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/3211854136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 0x558046d14310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/3211854136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d1f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb30f2b640 1 -- 172.21.15.1:0/3211854136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 msgr2=0x558046d14310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/3211854136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 0x558046d14310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb30f2b640 1 -- 172.21.15.1:0/3211854136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 msgr2=0x558046d1f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/3211854136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d1f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb30f2b640 1 -- 172.21.15.1:0/3211854136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558046cf8310 con 0x558046cfeb40 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/3211854136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046cf8c10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ffb200186c0 tx=0x7ffb2000a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663d27df59165493 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb167fc640 1 -- 172.21.15.1:0/3211854136 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2001d070 con 0x558046cfeb40 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb167fc640 1 -- 172.21.15.1:0/3211854136 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb20028070 con 0x558046cfeb40 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.575+0000 7ffb167fc640 1 -- 172.21.15.1:0/3211854136 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb200109f0 con 0x558046cfeb40 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 -- 172.21.15.1:0/3211854136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 msgr2=0x558046cf8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 --2- 172.21.15.1:0/3211854136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046cf8c10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ffb200186c0 tx=0x7ffb2000a730 comp rx=0 tx=0).stop 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 -- 172.21.15.1:0/3211854136 shutdown_connections 2024-09-16T00:39:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 --2- 172.21.15.1:0/3211854136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046d08370 0x558046d1f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 --2- 172.21.15.1:0/3211854136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046cf8c10 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 --2- 172.21.15.1:0/3211854136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046cfd5d0 0x558046d14310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 -- 172.21.15.1:0/3211854136 >> 172.21.15.1:0/3211854136 conn(0x558046cc6310 msgr2=0x558046cf8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 -- 172.21.15.1:0/3211854136 shutdown_connections 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 -- 172.21.15.1:0/3211854136 wait complete. 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 -- start start 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.576+0000 7ffb35568740 1 Processor -- start 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb35568740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046cfd5d0 0x558046d10810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb35568740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d10d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb35568740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046d08370 0x558046d01c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb35568740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c1ad400 con 0x558046cfeb40 2024-09-16T00:39:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb35568740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c1ae870 con 0x558046cfd5d0 2024-09-16T00:39:31.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb35568740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c1adad0 con 0x558046d08370 2024-09-16T00:39:31.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb35568740 10 client.0 _ll_register_callbacks cb 0x7ffb2001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:31.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb30f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d10d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb30f2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d10d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39252/0 (socket says 172.21.15.1:39252) 2024-09-16T00:39:31.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb30f2b640 1 -- 172.21.15.1:0/4169662139 learned_addr learned my addr 172.21.15.1:0/4169662139 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:31.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046d08370 0x558046d01c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046cfd5d0 0x558046d10810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb30f2b640 1 -- 172.21.15.1:0/4169662139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046d08370 msgr2=0x558046d01c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:31.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046d08370 0x558046d01c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb30f2b640 1 -- 172.21.15.1:0/4169662139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046cfd5d0 msgr2=0x558046d10810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046cfd5d0 0x558046d10810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb30f2b640 1 -- 172.21.15.1:0/4169662139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558046cfa960 con 0x558046cfeb40 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046cfd5d0 0x558046d10810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb30f2b640 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d10d50 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000f7d0 tx=0x7ffb2000acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a369097dfc3aed3b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.577+0000 7ffb0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb0bfff640 1 -- 172.21.15.1:0/4169662139 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2001d040 con 0x558046cfeb40 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb35568740 1 -- 172.21.15.1:0/4169662139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb20028070 con 0x558046cfeb40 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb0bfff640 1 -- 172.21.15.1:0/4169662139 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ffb20018c40 con 0x558046cfeb40 2024-09-16T00:39:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb0bfff640 1 -- 172.21.15.1:0/4169662139 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb20047050 con 0x558046cfeb40 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb35568740 10 client.18129 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb35568740 1 -- 172.21.15.1:0/4169662139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558046d867b0 con 0x558046cfeb40 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb0bfff640 1 -- 172.21.15.1:0/4169662139 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb20028070 con 0x558046cfeb40 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb0bfff640 1 -- 172.21.15.1:0/4169662139 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ffb20020040 con 0x558046cfeb40 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb35568740 10 client.18129 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb35568740 10 client.18129 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb35568740 20 client.18129 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb35568740 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x558046d86dc0 0x558046d891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb35568740 4 client.18129 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.578+0000 7ffb35568740 1 -- 172.21.15.1:0/4169662139 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558046c6ad70 con 0x558046d86dc0 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x558046d86dc0 0x558046d891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb2bfff640 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x558046d86dc0 0x558046d891e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5193960d9a73a2ad server_cookie=f31b5d66c6b88047 in_seq=0 out_seq=0 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb0bfff640 10 client.18129 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:39:31.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb0bfff640 1 -- 172.21.15.1:0/4169662139 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558046c6ad70 con 0x558046d86dc0 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb0bfff640 10 client.18129 handle_command_reply: tid=0 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb35568740 1 client.18129 shutdown 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb35568740 1 -- 172.21.15.1:0/4169662139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 msgr2=0x558046d10d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb35568740 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d10d50 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000f7d0 tx=0x7ffb2000acc0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb35568740 1 -- 172.21.15.1:0/4169662139 shutdown_connections 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb35568740 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558046d08370 0x558046d01c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb35568740 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x558046d86dc0 0x558046d891e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb35568740 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558046cfeb40 0x558046d10d50 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb35568740 1 --2- 172.21.15.1:0/4169662139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558046cfd5d0 0x558046d10810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb35568740 1 -- 172.21.15.1:0/4169662139 >> 172.21.15.1:0/4169662139 conn(0x558046cc6310 msgr2=0x558046d04010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.579+0000 7ffb0bfff640 0 client.18129 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb35568740 1 -- 172.21.15.1:0/4169662139 shutdown_connections 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb35568740 1 -- 172.21.15.1:0/4169662139 wait complete. 2024-09-16T00:39:31.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb35568740 20 client.18129 trim_cache size 0 max 16384 2024-09-16T00:39:31.590 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:39:31.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 -- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb1c0a3240 msgr2=0x7ffb1c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb1c0a3240 0x7ffb1c0a5700 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffb24098840 tx=0x7ffb24066d70 comp rx=0 tx=0).stop 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 -- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c106f10 msgr2=0x7ffb2c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c106f10 0x7ffb2c19cbe0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400cce0 tx=0x7ffb2400a620 comp rx=0 tx=0).stop 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 -- 172.21.15.1:0/3855900239 shutdown_connections 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb2c106f10 0x7ffb2c19cbe0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ffb1c0a3240 0x7ffb1c0a5700 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb2c102790 0x7ffb2c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 --2- 172.21.15.1:0/3855900239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb2c101de0 0x7ffb2c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 -- 172.21.15.1:0/3855900239 >> 172.21.15.1:0/3855900239 conn(0x7ffb2c0fd8c0 msgr2=0x7ffb2c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.580+0000 7ffb3318f640 1 -- 172.21.15.1:0/3855900239 shutdown_connections 2024-09-16T00:39:31.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:31.581+0000 7ffb3318f640 1 -- 172.21.15.1:0/3855900239 wait complete. 2024-09-16T00:39:32.401 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:39:32.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:32 smithi001 ceph-mon[30934]: pgmap v684: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:32 smithi153 ceph-mon[35538]: pgmap v684: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:32.609 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:32 smithi019 ceph-mon[35186]: pgmap v684: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:32.667 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:33.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.158+0000 7f4488f70640 1 -- 172.21.15.1:0/1569304564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484102790 msgr2=0x7f44841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.158+0000 7f4488f70640 1 --2- 172.21.15.1:0/1569304564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484102790 0x7f44841067e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f4478009900 tx=0x7f447802a410 comp rx=0 tx=0).stop 2024-09-16T00:39:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.159+0000 7f4488f70640 1 -- 172.21.15.1:0/1569304564 shutdown_connections 2024-09-16T00:39:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.159+0000 7f4488f70640 1 --2- 172.21.15.1:0/1569304564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4484106f10 0x7f448410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.159+0000 7f4488f70640 1 --2- 172.21.15.1:0/1569304564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484102790 0x7f44841067e0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.159+0000 7f4488f70640 1 --2- 172.21.15.1:0/1569304564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484101de0 0x7f44841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.159+0000 7f4488f70640 1 -- 172.21.15.1:0/1569304564 >> 172.21.15.1:0/1569304564 conn(0x7f44840fd8c0 msgr2=0x7f44840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.159+0000 7f4488f70640 1 -- 172.21.15.1:0/1569304564 shutdown_connections 2024-09-16T00:39:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.159+0000 7f4488f70640 1 -- 172.21.15.1:0/1569304564 wait complete. 2024-09-16T00:39:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.159+0000 7f4488f70640 1 Processor -- start 2024-09-16T00:39:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4488f70640 1 -- start start 2024-09-16T00:39:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4488f70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484101de0 0x7f4484198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4488f70640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4484102790 0x7f44841989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4488f70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484106f10 0x7f448419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4488f70640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4484111f70 con 0x7f4484106f10 2024-09-16T00:39:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4488f70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4484111df0 con 0x7f4484101de0 2024-09-16T00:39:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4488f70640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44841120f0 con 0x7f4484102790 2024-09-16T00:39:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4482d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484106f10 0x7f448419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4482d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484106f10 0x7f448419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39274/0 (socket says 172.21.15.1:39274) 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4482d9d640 1 -- 172.21.15.1:0/2560112967 learned_addr learned my addr 172.21.15.1:0/2560112967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f4481d9b640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4484102790 0x7f44841989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.160+0000 7f448259c640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484101de0 0x7f4484198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f4482d9d640 1 -- 172.21.15.1:0/2560112967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4484102790 msgr2=0x7f44841989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f4482d9d640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4484102790 0x7f44841989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f4482d9d640 1 -- 172.21.15.1:0/2560112967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484101de0 msgr2=0x7f4484198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f4482d9d640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484101de0 0x7f4484198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f4482d9d640 1 -- 172.21.15.1:0/2560112967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448419d420 con 0x7f4484106f10 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f4482d9d640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484106f10 0x7f448419cd40 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f447c00cad0 tx=0x7f447c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f44737fe640 1 -- 172.21.15.1:0/2560112967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c00ae20 con 0x7f4484106f10 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f4488f70640 1 -- 172.21.15.1:0/2560112967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448419d710 con 0x7f4484106f10 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f44737fe640 1 -- 172.21.15.1:0/2560112967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447c005020 con 0x7f4484106f10 2024-09-16T00:39:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f44737fe640 1 -- 172.21.15.1:0/2560112967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c00f6b0 con 0x7f4484106f10 2024-09-16T00:39:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.161+0000 7f4488f70640 1 -- 172.21.15.1:0/2560112967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44841a4ff0 con 0x7f4484106f10 2024-09-16T00:39:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f44737fe640 1 -- 172.21.15.1:0/2560112967 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f447c0051d0 con 0x7f4484106f10 2024-09-16T00:39:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f448b349740 1 Processor -- start 2024-09-16T00:39:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f448b349740 1 -- start start 2024-09-16T00:39:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f448b349740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3ba5b40 0x563ff3ba4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f448b349740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3ba4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f448b349740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3bb3780 0x563ff3bb5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f448b349740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563ff3b0a2f0 con 0x563ff3bb3780 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f448b349740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563ff3ab1cf0 con 0x563ff3ba45d0 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f448b349740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563ff3b062b0 con 0x563ff3ba5b40 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f4481d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3bb3780 0x563ff3bb5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f4481d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3bb3780 0x563ff3bb5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39280/0 (socket says 172.21.15.1:39280) 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.164+0000 7f4481d9b640 1 -- 172.21.15.1:0/3324433628 learned_addr learned my addr 172.21.15.1:0/3324433628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f448259c640 1 --2- 172.21.15.1:0/3324433628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3ba4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f4481d9b640 1 --2- 172.21.15.1:0/3324433628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3ba5b40 0x563ff3ba4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f448259c640 1 -- 172.21.15.1:0/3324433628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3ba5b40 msgr2=0x563ff3ba4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f448259c640 1 --2- 172.21.15.1:0/3324433628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3ba5b40 0x563ff3ba4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f448259c640 1 -- 172.21.15.1:0/3324433628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3bb3780 msgr2=0x563ff3bb5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f448259c640 1 --2- 172.21.15.1:0/3324433628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3bb3780 0x563ff3bb5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f448259c640 1 -- 172.21.15.1:0/3324433628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff3a963a0 con 0x563ff3ba45d0 2024-09-16T00:39:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f44737fe640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f446c0a3240 0x7f446c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f448259c640 1 --2- 172.21.15.1:0/3324433628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3ba4a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4474005fd0 tx=0x7f44740107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc8423969c755d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f4481d9b640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f446c0a3240 0x7f446c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.165+0000 7f44737fe640 1 -- 172.21.15.1:0/2560112967 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f447c0cffa0 con 0x7f4484106f10 2024-09-16T00:39:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.166+0000 7f4481d9b640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f446c0a3240 0x7f446c0a5700 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4478005ec0 tx=0x7f4478000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:33.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.167+0000 7f4470ff9640 1 -- 172.21.15.1:0/3324433628 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447401c480 con 0x563ff3ba45d0 2024-09-16T00:39:33.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.167+0000 7f4470ff9640 1 -- 172.21.15.1:0/3324433628 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447401cb00 con 0x563ff3ba45d0 2024-09-16T00:39:33.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.167+0000 7f4470ff9640 1 -- 172.21.15.1:0/3324433628 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474011b90 con 0x563ff3ba45d0 2024-09-16T00:39:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 -- 172.21.15.1:0/3324433628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 msgr2=0x563ff3ba4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 --2- 172.21.15.1:0/3324433628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3ba4a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4474005fd0 tx=0x7f44740107a0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 -- 172.21.15.1:0/3324433628 shutdown_connections 2024-09-16T00:39:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 --2- 172.21.15.1:0/3324433628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3bb3780 0x563ff3bb5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 --2- 172.21.15.1:0/3324433628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3ba4a10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 --2- 172.21.15.1:0/3324433628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3ba5b40 0x563ff3ba4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 -- 172.21.15.1:0/3324433628 >> 172.21.15.1:0/3324433628 conn(0x563ff3b6d310 msgr2=0x563ff3ba0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 -- 172.21.15.1:0/3324433628 shutdown_connections 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 -- 172.21.15.1:0/3324433628 wait complete. 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 -- start start 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.168+0000 7f448b349740 1 Processor -- start 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f448b349740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bb3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f448b349740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f448b349740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bb0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f448b349740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563ff3bb6480 con 0x563ff3ba5b40 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f448b349740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563ff3ac5060 con 0x563ff3ba45d0 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f448b349740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563ff3bb6600 con 0x563ff3bb3780 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4481d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f448b349740 10 client.0 _ll_register_callbacks cb 0x563ff38188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4481d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39294/0 (socket says 172.21.15.1:39294) 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4481d9b640 1 -- 172.21.15.1:0/3245402025 learned_addr learned my addr 172.21.15.1:0/3245402025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f448259c640 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bb3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4482d9d640 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bb0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4481d9b640 1 -- 172.21.15.1:0/3245402025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 msgr2=0x563ff3bb0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4481d9b640 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bb0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4481d9b640 1 -- 172.21.15.1:0/3245402025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 msgr2=0x563ff3bb3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4481d9b640 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bb3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4481d9b640 1 -- 172.21.15.1:0/3245402025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff387b3c0 con 0x563ff3ba5b40 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4482d9d640 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bb0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f448259c640 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bb3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.169+0000 7f4481d9b640 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb03e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f4478006010 tx=0x7f447802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01d030560f1c3fe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f4469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f4469ffb640 1 -- 172.21.15.1:0/3245402025 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478035040 con 0x563ff3ba5b40 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f4469ffb640 1 -- 172.21.15.1:0/3245402025 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4478004460 con 0x563ff3ba5b40 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f448b349740 1 -- 172.21.15.1:0/3245402025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f447401cb00 con 0x563ff3ba5b40 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f4469ffb640 1 -- 172.21.15.1:0/3245402025 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4478005630 con 0x563ff3ba5b40 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f448b349740 10 client.18141 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f448b349740 1 -- 172.21.15.1:0/3245402025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563ff3bc1f00 con 0x563ff3ba5b40 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f4469ffb640 1 -- 172.21.15.1:0/3245402025 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447401cb00 con 0x563ff3ba5b40 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f4469ffb640 1 -- 172.21.15.1:0/3245402025 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4478003960 con 0x563ff3ba5b40 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f448b349740 10 client.18141 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f448b349740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f448b349740 20 client.18141 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f448b349740 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563ff3bc3410 0x563ff3bc5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f448b349740 4 client.18141 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.170+0000 7f448b349740 1 -- 172.21.15.1:0/3245402025 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ff3b17860 con 0x563ff3bc3410 2024-09-16T00:39:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.171+0000 7f448259c640 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563ff3bc3410 0x563ff3bc5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.171+0000 7f448259c640 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563ff3bc3410 0x563ff3bc5890 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a6f248bc3ed0eb server_cookie=6f5b66ce76091da9 in_seq=0 out_seq=0 2024-09-16T00:39:33.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.171+0000 7f4469ffb640 10 client.18141 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f4469ffb640 1 -- 172.21.15.1:0/3245402025 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ff3b17860 con 0x563ff3bc3410 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f4469ffb640 10 client.18141 handle_command_reply: tid=0 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 client.18141 shutdown 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 -- 172.21.15.1:0/3245402025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 msgr2=0x563ff3bb03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb03e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f4478006010 tx=0x7f447802a410 comp rx=0 tx=0).stop 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 -- 172.21.15.1:0/3245402025 shutdown_connections 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563ff3bc3410 0x563ff3bc5890 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bb0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb03e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 --2- 172.21.15.1:0/3245402025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bb3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f4469ffb640 0 client.18141 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:33.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 -- 172.21.15.1:0/3245402025 >> 172.21.15.1:0/3245402025 conn(0x563ff3b6d310 msgr2=0x563ff3ba6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:33.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 -- 172.21.15.1:0/3245402025 shutdown_connections 2024-09-16T00:39:33.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 1 -- 172.21.15.1:0/3245402025 wait complete. 2024-09-16T00:39:33.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.172+0000 7f448b349740 20 client.18141 trim_cache size 0 max 16384 2024-09-16T00:39:33.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f448b349740 1 Processor -- start 2024-09-16T00:39:33.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f448b349740 1 -- start start 2024-09-16T00:39:33.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f448b349740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bbf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f448b349740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f448b349740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bb50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f448b349740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f44841af610 con 0x563ff3ba5b40 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f448b349740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44841acee0 con 0x563ff3ba45d0 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f448b349740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44841aad30 con 0x563ff3bb3780 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f4481d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f4481d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39298/0 (socket says 172.21.15.1:39298) 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.185+0000 7f4481d9b640 1 -- 172.21.15.1:0/1471615608 learned_addr learned my addr 172.21.15.1:0/1471615608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f4481d9b640 1 -- 172.21.15.1:0/1471615608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 msgr2=0x563ff3bb50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f4481d9b640 1 --2- 172.21.15.1:0/1471615608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bb50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f448259c640 1 --2- 172.21.15.1:0/1471615608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bbf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f4481d9b640 1 -- 172.21.15.1:0/1471615608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 msgr2=0x563ff3bbf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f4481d9b640 1 --2- 172.21.15.1:0/1471615608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bbf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f4481d9b640 1 -- 172.21.15.1:0/1471615608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff3bb5fb0 con 0x563ff3ba5b40 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f4481d9b640 1 --2- 172.21.15.1:0/1471615608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb4b70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4478003ed0 tx=0x7f447802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8d862d0a958774 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:33.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f446bfff640 1 -- 172.21.15.1:0/1471615608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478064070 con 0x563ff3ba5b40 2024-09-16T00:39:33.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f446bfff640 1 -- 172.21.15.1:0/1471615608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4478003ad0 con 0x563ff3ba5b40 2024-09-16T00:39:33.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.186+0000 7f446bfff640 1 -- 172.21.15.1:0/1471615608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478035040 con 0x563ff3ba5b40 2024-09-16T00:39:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 -- 172.21.15.1:0/1471615608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 msgr2=0x563ff3bb4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 --2- 172.21.15.1:0/1471615608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb4b70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4478003ed0 tx=0x7f447802ab90 comp rx=0 tx=0).stop 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 -- 172.21.15.1:0/1471615608 shutdown_connections 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 --2- 172.21.15.1:0/1471615608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bb50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 --2- 172.21.15.1:0/1471615608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bb4b70 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 --2- 172.21.15.1:0/1471615608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bbf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 -- 172.21.15.1:0/1471615608 >> 172.21.15.1:0/1471615608 conn(0x563ff3b6d310 msgr2=0x563ff3baf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 -- 172.21.15.1:0/1471615608 shutdown_connections 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 -- 172.21.15.1:0/1471615608 wait complete. 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 -- start start 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 Processor -- start 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bcad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bcb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bcb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f44841ab3a0 con 0x563ff3ba5b40 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44841ac810 con 0x563ff3ba45d0 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44841aba70 con 0x563ff3bb3780 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f448b349740 10 client.0 _ll_register_callbacks cb 0x7f4478031ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f4481d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bcb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.187+0000 7f4481d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bcb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39308/0 (socket says 172.21.15.1:39308) 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4481d9b640 1 -- 172.21.15.1:0/158970690 learned_addr learned my addr 172.21.15.1:0/158970690 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f448259c640 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bcad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4482d9d640 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bcb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4481d9b640 1 -- 172.21.15.1:0/158970690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 msgr2=0x563ff3bcb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4481d9b640 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bcb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4481d9b640 1 -- 172.21.15.1:0/158970690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 msgr2=0x563ff3bcad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4481d9b640 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bcad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4481d9b640 1 -- 172.21.15.1:0/158970690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff3bb9f30 con 0x563ff3ba5b40 2024-09-16T00:39:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4481d9b640 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bcb2d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4478033000 tx=0x7f447803d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6782ec43eafe141 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4469ffb640 1 -- 172.21.15.1:0/158970690 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478064070 con 0x563ff3ba5b40 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4469ffb640 1 -- 172.21.15.1:0/158970690 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f447803d970 con 0x563ff3ba5b40 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f448b349740 1 -- 172.21.15.1:0/158970690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4478003ad0 con 0x563ff3ba5b40 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f4469ffb640 1 -- 172.21.15.1:0/158970690 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f447806f050 con 0x563ff3ba5b40 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f448b349740 10 client.18147 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.188+0000 7f448b349740 1 -- 172.21.15.1:0/158970690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563ff3c2e470 con 0x563ff3ba5b40 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f4469ffb640 1 -- 172.21.15.1:0/158970690 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4478003ad0 con 0x563ff3ba5b40 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f4469ffb640 1 -- 172.21.15.1:0/158970690 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f447805f480 con 0x563ff3ba5b40 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f448b349740 10 client.18147 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f448b349740 10 client.18147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f448b349740 20 client.18147 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f448b349740 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563ff3c2ea70 0x563ff3c30e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f448b349740 4 client.18147 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f448b349740 1 -- 172.21.15.1:0/158970690 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ff3b11d70 con 0x563ff3c2ea70 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f448259c640 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563ff3c2ea70 0x563ff3c30e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f448259c640 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563ff3c2ea70 0x563ff3c30e90 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e684a2988547b6 server_cookie=8119eb4e94e98029 in_seq=0 out_seq=0 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.189+0000 7f4469ffb640 10 client.18147 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f4469ffb640 1 -- 172.21.15.1:0/158970690 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ff3b11d70 con 0x563ff3c2ea70 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f4469ffb640 10 client.18147 handle_command_reply: tid=0 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 client.18147 shutdown 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 -- 172.21.15.1:0/158970690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 msgr2=0x563ff3bcb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bcb2d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4478033000 tx=0x7f447803d680 comp rx=0 tx=0).stop 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 -- 172.21.15.1:0/158970690 shutdown_connections 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563ff3c2ea70 0x563ff3c30e90 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ff3bb3780 0x563ff3bcb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ff3ba5b40 0x563ff3bcb2d0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 --2- 172.21.15.1:0/158970690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3ba45d0 0x563ff3bcad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 -- 172.21.15.1:0/158970690 >> 172.21.15.1:0/158970690 conn(0x563ff3b6d310 msgr2=0x563ff3bb1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f4469ffb640 0 client.18147 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 -- 172.21.15.1:0/158970690 shutdown_connections 2024-09-16T00:39:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 1 -- 172.21.15.1:0/158970690 wait complete. 2024-09-16T00:39:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f448b349740 20 client.18147 trim_cache size 0 max 16384 2024-09-16T00:39:33.196 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:39:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f4488f70640 1 -- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f446c0a3240 msgr2=0x7f446c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f4488f70640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f446c0a3240 0x7f446c0a5700 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4478005ec0 tx=0x7f4478000bc0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f4488f70640 1 -- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484106f10 msgr2=0x7f448419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.190+0000 7f4488f70640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484106f10 0x7f448419cd40 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f447c00cad0 tx=0x7f447c00cf90 comp rx=0 tx=0).stop 2024-09-16T00:39:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.191+0000 7f4488f70640 1 -- 172.21.15.1:0/2560112967 shutdown_connections 2024-09-16T00:39:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.191+0000 7f4488f70640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4484106f10 0x7f448419cd40 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.191+0000 7f4488f70640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f446c0a3240 0x7f446c0a5700 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.191+0000 7f4488f70640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4484102790 0x7f44841989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.191+0000 7f4488f70640 1 --2- 172.21.15.1:0/2560112967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484101de0 0x7f4484198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.191+0000 7f4488f70640 1 -- 172.21.15.1:0/2560112967 >> 172.21.15.1:0/2560112967 conn(0x7f44840fd8c0 msgr2=0x7f44840fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.191+0000 7f4488f70640 1 -- 172.21.15.1:0/2560112967 shutdown_connections 2024-09-16T00:39:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:33.191+0000 7f4488f70640 1 -- 172.21.15.1:0/2560112967 wait complete. 2024-09-16T00:39:34.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:34 smithi001 ceph-mon[30934]: pgmap v685: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:34 smithi153 ceph-mon[35538]: pgmap v685: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:34.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:34 smithi019 ceph-mon[35186]: pgmap v685: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:34.974 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:39:34.974 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:39:35.245 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:35.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 -- 172.21.15.1:0/4228704933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694102dd0 msgr2=0x7f16941031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 --2- 172.21.15.1:0/4228704933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694102dd0 0x7f16941031b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f168c009930 tx=0x7f168c02a420 comp rx=0 tx=0).stop 2024-09-16T00:39:35.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 -- 172.21.15.1:0/4228704933 shutdown_connections 2024-09-16T00:39:35.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 --2- 172.21.15.1:0/4228704933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1694108300 0x7f169410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 --2- 172.21.15.1:0/4228704933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1694103780 0x7f1694107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 --2- 172.21.15.1:0/4228704933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694102dd0 0x7f16941031b0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 -- 172.21.15.1:0/4228704933 >> 172.21.15.1:0/4228704933 conn(0x7f16940fc7d0 msgr2=0x7f16940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 -- 172.21.15.1:0/4228704933 shutdown_connections 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 -- 172.21.15.1:0/4228704933 wait complete. 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.756+0000 7f1698b1b640 1 Processor -- start 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1698b1b640 1 -- start start 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1698b1b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1694102dd0 0x7f1694100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1698b1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694103780 0x7f1694100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1698b1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1694108300 0x7f16941014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1698b1b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1694112090 con 0x7f1694103780 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1698b1b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1694111f10 con 0x7f1694108300 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1698b1b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1694112210 con 0x7f1694102dd0 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1691d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694103780 0x7f1694100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1691d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694103780 0x7f1694100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39334/0 (socket says 172.21.15.1:39334) 2024-09-16T00:39:35.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1691d9b640 1 -- 172.21.15.1:0/167405257 learned_addr learned my addr 172.21.15.1:0/167405257 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f169259c640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1694102dd0 0x7f1694100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1692d9d640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1694108300 0x7f16941014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1691d9b640 1 -- 172.21.15.1:0/167405257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1694102dd0 msgr2=0x7f1694100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1691d9b640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1694102dd0 0x7f1694100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1691d9b640 1 -- 172.21.15.1:0/167405257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1694108300 msgr2=0x7f16941014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1691d9b640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1694108300 0x7f16941014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1691d9b640 1 -- 172.21.15.1:0/167405257 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16941a69f0 con 0x7f1694103780 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.757+0000 7f1692d9d640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1694108300 0x7f16941014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.758+0000 7f1691d9b640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694103780 0x7f1694100f70 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f167c00c8c0 tx=0x7f167c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.758+0000 7f168b7fe640 1 -- 172.21.15.1:0/167405257 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c007bb0 con 0x7f1694103780 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.758+0000 7f1698b1b640 1 -- 172.21.15.1:0/167405257 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16941a6ce0 con 0x7f1694103780 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.758+0000 7f168b7fe640 1 -- 172.21.15.1:0/167405257 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f167c007d50 con 0x7f1694103780 2024-09-16T00:39:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.758+0000 7f168b7fe640 1 -- 172.21.15.1:0/167405257 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c018830 con 0x7f1694103780 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.758+0000 7f1698b1b640 1 -- 172.21.15.1:0/167405257 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16941a7210 con 0x7f1694103780 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.760+0000 7f168b7fe640 1 -- 172.21.15.1:0/167405257 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f167c0189f0 con 0x7f1694103780 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.760+0000 7f169aef4740 1 Processor -- start 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169aef4740 1 -- start start 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169aef4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 0x564840f13f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169aef4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f14480 0x564840f1e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169aef4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f1ed40 0x564840f21130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169aef4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564840e79280 con 0x564840f14480 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169aef4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564840e20b10 con 0x564840f1ed40 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169aef4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564840e75240 con 0x564840f13b60 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f14480 0x564840f1e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f14480 0x564840f1e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39336/0 (socket says 172.21.15.1:39336) 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169259c640 1 -- 172.21.15.1:0/2223726576 learned_addr learned my addr 172.21.15.1:0/2223726576 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f168b7fe640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f16800a3240 0x7f16800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f1691d9b640 1 --2- 172.21.15.1:0/2223726576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f1ed40 0x564840f21130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f1692d9d640 1 --2- 172.21.15.1:0/2223726576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 0x564840f13f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169259c640 1 -- 172.21.15.1:0/2223726576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 msgr2=0x564840f13f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169259c640 1 --2- 172.21.15.1:0/2223726576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 0x564840f13f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169259c640 1 -- 172.21.15.1:0/2223726576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f1ed40 msgr2=0x564840f21130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169259c640 1 --2- 172.21.15.1:0/2223726576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f1ed40 0x564840f21130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f1692d9d640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f16800a3240 0x7f16800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f168b7fe640 1 -- 172.21.15.1:0/167405257 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f167c0cfe40 con 0x7f1694103780 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.761+0000 7f169259c640 1 -- 172.21.15.1:0/2223726576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564840e2d1f0 con 0x564840f14480 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.762+0000 7f169259c640 1 --2- 172.21.15.1:0/2223726576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f14480 0x564840f1e800 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f168c009930 tx=0x7f168c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30675780dff2ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.762+0000 7f1688ff9640 1 -- 172.21.15.1:0/2223726576 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c040070 con 0x564840f14480 2024-09-16T00:39:35.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.762+0000 7f1688ff9640 1 -- 172.21.15.1:0/2223726576 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168c005020 con 0x564840f14480 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.762+0000 7f1688ff9640 1 -- 172.21.15.1:0/2223726576 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c035040 con 0x564840f14480 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.762+0000 7f1692d9d640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f16800a3240 0x7f16800a5700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1684005c40 tx=0x7f1684009200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 -- 172.21.15.1:0/2223726576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f14480 msgr2=0x564840f1e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 --2- 172.21.15.1:0/2223726576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f14480 0x564840f1e800 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f168c009930 tx=0x7f168c031630 comp rx=0 tx=0).stop 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 -- 172.21.15.1:0/2223726576 shutdown_connections 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 --2- 172.21.15.1:0/2223726576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f1ed40 0x564840f21130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 --2- 172.21.15.1:0/2223726576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f14480 0x564840f1e800 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 --2- 172.21.15.1:0/2223726576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 0x564840f13f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 -- 172.21.15.1:0/2223726576 >> 172.21.15.1:0/2223726576 conn(0x564840f100f0 msgr2=0x564840f104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 -- 172.21.15.1:0/2223726576 shutdown_connections 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 -- 172.21.15.1:0/2223726576 wait complete. 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 -- start start 2024-09-16T00:39:35.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 Processor -- start 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f1e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f1bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 0x564840f1c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564840f21910 con 0x564840f13b60 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564840e33f10 con 0x564840f14480 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.763+0000 7f169aef4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564840f21a90 con 0x564840f1ed40 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169aef4740 10 client.0 _ll_register_callbacks cb 0x564840dfca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f1e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f1691d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f1bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f1e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39350/0 (socket says 172.21.15.1:39350) 2024-09-16T00:39:35.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169259c640 1 -- 172.21.15.1:0/759486665 learned_addr learned my addr 172.21.15.1:0/759486665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f1692d9d640 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 0x564840f1c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169259c640 1 -- 172.21.15.1:0/759486665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 msgr2=0x564840f1c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169259c640 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 0x564840f1c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169259c640 1 -- 172.21.15.1:0/759486665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 msgr2=0x564840f1bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169259c640 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f1bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169259c640 1 -- 172.21.15.1:0/759486665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564840bea6a0 con 0x564840f13b60 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169259c640 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f1e760 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f168c006010 tx=0x7f168c031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f92c32a5a74626 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f1679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f1679ffb640 1 -- 172.21.15.1:0/759486665 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c040070 con 0x564840f13b60 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f1679ffb640 1 -- 172.21.15.1:0/759486665 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f168c0053e0 con 0x564840f13b60 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f169aef4740 1 -- 172.21.15.1:0/759486665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168c005020 con 0x564840f13b60 2024-09-16T00:39:35.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.764+0000 7f1679ffb640 1 -- 172.21.15.1:0/759486665 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f168c056050 con 0x564840f13b60 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f169aef4740 10 client.18159 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f169aef4740 1 -- 172.21.15.1:0/759486665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564840f2d3d0 con 0x564840f13b60 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f1679ffb640 1 -- 172.21.15.1:0/759486665 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168c005020 con 0x564840f13b60 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f1679ffb640 1 -- 172.21.15.1:0/759486665 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f168c03c9f0 con 0x564840f13b60 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f169aef4740 10 client.18159 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f169aef4740 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f169aef4740 20 client.18159 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f169aef4740 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564840f2e720 0x564840f30b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f169aef4740 4 client.18159 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f169aef4740 1 -- 172.21.15.1:0/759486665 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564840e867f0 con 0x564840f2e720 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.765+0000 7f1691d9b640 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564840f2e720 0x564840f30b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f1679ffb640 10 client.18159 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:35.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f1691d9b640 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564840f2e720 0x564840f30b10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7d7ed8e8331a06 server_cookie=723e4260e1f89078 in_seq=0 out_seq=0 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f1679ffb640 1 -- 172.21.15.1:0/759486665 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564840e867f0 con 0x564840f2e720 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f1679ffb640 10 client.18159 handle_command_reply: tid=0 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f169aef4740 1 client.18159 shutdown 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f169aef4740 1 -- 172.21.15.1:0/759486665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 msgr2=0x564840f1e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f169aef4740 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f1e760 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f168c006010 tx=0x7f168c031b10 comp rx=0 tx=0).stop 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f169aef4740 1 -- 172.21.15.1:0/759486665 shutdown_connections 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f169aef4740 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564840f2e720 0x564840f30b10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f169aef4740 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 0x564840f1c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f1679ffb640 0 client.18159 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f169aef4740 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f1bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f169aef4740 1 --2- 172.21.15.1:0/759486665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f1e760 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.766+0000 7f169aef4740 1 -- 172.21.15.1:0/759486665 >> 172.21.15.1:0/759486665 conn(0x564840f100f0 msgr2=0x564840f15dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.767+0000 7f169aef4740 1 -- 172.21.15.1:0/759486665 shutdown_connections 2024-09-16T00:39:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.767+0000 7f169aef4740 1 -- 172.21.15.1:0/759486665 wait complete. 2024-09-16T00:39:35.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.767+0000 7f169aef4740 20 client.18159 trim_cache size 0 max 16384 2024-09-16T00:39:35.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f169aef4740 1 Processor -- start 2024-09-16T00:39:35.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f169aef4740 1 -- start start 2024-09-16T00:39:35.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f169aef4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 0x564840f2ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f169aef4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f0f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f169aef4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f1ed40 0x564840f33b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f169aef4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16941b38f0 con 0x564840f1ed40 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f169aef4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16941b11c0 con 0x564840f14480 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f169aef4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16941af010 con 0x564840f13b60 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f1692d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f1ed40 0x564840f33b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f1692d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f1ed40 0x564840f33b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39354/0 (socket says 172.21.15.1:39354) 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f1692d9d640 1 -- 172.21.15.1:0/1942807653 learned_addr learned my addr 172.21.15.1:0/1942807653 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f169259c640 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 0x564840f2ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.784+0000 7f1691d9b640 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f0f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f1692d9d640 1 -- 172.21.15.1:0/1942807653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 msgr2=0x564840f2ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f1692d9d640 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 0x564840f2ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f1692d9d640 1 -- 172.21.15.1:0/1942807653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 msgr2=0x564840f0f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f1692d9d640 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f0f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f1692d9d640 1 -- 172.21.15.1:0/1942807653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564840f0ff20 con 0x564840f1ed40 2024-09-16T00:39:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f1691d9b640 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f0f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f169259c640 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 0x564840f2ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f1692d9d640 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f1ed40 0x564840f33b30 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f168400f7b0 tx=0x7f1684009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb233884be1d84a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f167bfff640 1 -- 172.21.15.1:0/1942807653 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684018040 con 0x564840f1ed40 2024-09-16T00:39:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f167bfff640 1 -- 172.21.15.1:0/1942807653 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1684011e50 con 0x564840f1ed40 2024-09-16T00:39:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f167bfff640 1 -- 172.21.15.1:0/1942807653 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840203e0 con 0x564840f1ed40 2024-09-16T00:39:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f169aef4740 1 -- 172.21.15.1:0/1942807653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f1ed40 msgr2=0x564840f33b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.785+0000 7f169aef4740 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f1ed40 0x564840f33b30 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f168400f7b0 tx=0x7f1684009650 comp rx=0 tx=0).stop 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 -- 172.21.15.1:0/1942807653 shutdown_connections 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f1ed40 0x564840f33b30 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f0f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 --2- 172.21.15.1:0/1942807653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f13b60 0x564840f2ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 -- 172.21.15.1:0/1942807653 >> 172.21.15.1:0/1942807653 conn(0x564840f100f0 msgr2=0x564840f0ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 -- 172.21.15.1:0/1942807653 shutdown_connections 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 -- 172.21.15.1:0/1942807653 wait complete. 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 -- start start 2024-09-16T00:39:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 Processor -- start 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f33c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f34140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 0x564840f30060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16941af680 con 0x564840f13b60 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16941b0af0 con 0x564840f14480 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16941afd50 con 0x564840f1ed40 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169aef4740 10 client.0 _ll_register_callbacks cb 0x7f168c02ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.786+0000 7f169259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f33c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f33c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39356/0 (socket says 172.21.15.1:39356) 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169259c640 1 -- 172.21.15.1:0/2783960797 learned_addr learned my addr 172.21.15.1:0/2783960797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f1691d9b640 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f34140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169259c640 1 -- 172.21.15.1:0/2783960797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 msgr2=0x564840f30060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f1692d9d640 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 0x564840f30060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169259c640 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 0x564840f30060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169259c640 1 -- 172.21.15.1:0/2783960797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 msgr2=0x564840f34140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169259c640 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f34140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169259c640 1 -- 172.21.15.1:0/2783960797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564840f1ca50 con 0x564840f13b60 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f1692d9d640 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 0x564840f30060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169259c640 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f33c00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f168c006010 tx=0x7f168c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a68af9fd425ea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f1679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f1679ffb640 1 -- 172.21.15.1:0/2783960797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c040070 con 0x564840f13b60 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f1679ffb640 1 -- 172.21.15.1:0/2783960797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f168c03c440 con 0x564840f13b60 2024-09-16T00:39:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169aef4740 1 -- 172.21.15.1:0/2783960797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1684011e50 con 0x564840f13b60 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f1679ffb640 1 -- 172.21.15.1:0/2783960797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f168c063050 con 0x564840f13b60 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169aef4740 10 client.18165 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.787+0000 7f169aef4740 1 -- 172.21.15.1:0/2783960797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564840f999c0 con 0x564840f13b60 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f1679ffb640 1 -- 172.21.15.1:0/2783960797 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1684011e50 con 0x564840f13b60 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f1679ffb640 1 -- 172.21.15.1:0/2783960797 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f168c004e20 con 0x564840f13b60 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f169aef4740 10 client.18165 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f169aef4740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f169aef4740 20 client.18165 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f169aef4740 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564840f9a050 0x564840f9c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f169aef4740 4 client.18165 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f169aef4740 1 -- 172.21.15.1:0/2783960797 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564840e80d00 con 0x564840f9a050 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f1691d9b640 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564840f9a050 0x564840f9c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f1691d9b640 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564840f9a050 0x564840f9c470 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66bd7c22a94d3b1a server_cookie=c6d4416496d4b0fd in_seq=0 out_seq=0 2024-09-16T00:39:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.788+0000 7f1679ffb640 10 client.18165 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.789+0000 7f1679ffb640 1 -- 172.21.15.1:0/2783960797 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564840e80d00 con 0x564840f9a050 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.789+0000 7f1679ffb640 10 client.18165 handle_command_reply: tid=0 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.789+0000 7f169aef4740 1 client.18165 shutdown 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 -- 172.21.15.1:0/2783960797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 msgr2=0x564840f33c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f33c00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f168c006010 tx=0x7f168c0317e0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 -- 172.21.15.1:0/2783960797 shutdown_connections 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564840f9a050 0x564840f9c470 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564840f1ed40 0x564840f30060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564840f14480 0x564840f34140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 --2- 172.21.15.1:0/2783960797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564840f13b60 0x564840f33c00 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 -- 172.21.15.1:0/2783960797 >> 172.21.15.1:0/2783960797 conn(0x564840f100f0 msgr2=0x564840f0f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1679ffb640 0 client.18165 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 -- 172.21.15.1:0/2783960797 shutdown_connections 2024-09-16T00:39:35.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 1 -- 172.21.15.1:0/2783960797 wait complete. 2024-09-16T00:39:35.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f169aef4740 20 client.18165 trim_cache size 0 max 16384 2024-09-16T00:39:35.796 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:39:35.796 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:39:35.796 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "daea119e-ce17-413d-a89d-bfa75d9dde9b", 2024-09-16T00:39:35.796 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:39:35.796 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:39:35.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 -- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f16800a3240 msgr2=0x7f16800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f16800a3240 0x7f16800a5700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1684005c40 tx=0x7f1684009200 comp rx=0 tx=0).stop 2024-09-16T00:39:35.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 -- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694103780 msgr2=0x7f1694100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694103780 0x7f1694100f70 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f167c00c8c0 tx=0x7f167c00cd80 comp rx=0 tx=0).stop 2024-09-16T00:39:35.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 -- 172.21.15.1:0/167405257 shutdown_connections 2024-09-16T00:39:35.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1694108300 0x7f16941014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1694103780 0x7f1694100f70 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f16800a3240 0x7f16800a5700 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 --2- 172.21.15.1:0/167405257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1694102dd0 0x7f1694100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.790+0000 7f1698b1b640 1 -- 172.21.15.1:0/167405257 >> 172.21.15.1:0/167405257 conn(0x7f16940fc7d0 msgr2=0x7f1694105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:35.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.791+0000 7f1698b1b640 1 -- 172.21.15.1:0/167405257 shutdown_connections 2024-09-16T00:39:35.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:35.791+0000 7f1698b1b640 1 -- 172.21.15.1:0/167405257 wait complete. 2024-09-16T00:39:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:36 smithi001 ceph-mon[30934]: pgmap v686: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:36 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:39:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:36 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:39:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:36 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:39:36.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:36 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:39:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:36 smithi153 ceph-mon[35538]: pgmap v686: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:36 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:39:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:36 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:39:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:36 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:39:36.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:36 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:39:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:36 smithi019 ceph-mon[35186]: pgmap v686: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:36 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:39:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:36 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:39:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:36 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:39:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:36 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:39:36.609 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:39:36.871 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:37.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.348+0000 7f2be3838640 1 -- 172.21.15.1:0/3388603953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc101de0 msgr2=0x7f2bdc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.348+0000 7f2be3838640 1 --2- 172.21.15.1:0/3388603953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc101de0 0x7f2bdc1021c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009900 tx=0x7f2bd802a410 comp rx=0 tx=0).stop 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.348+0000 7f2be3838640 1 -- 172.21.15.1:0/3388603953 shutdown_connections 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.348+0000 7f2be3838640 1 --2- 172.21.15.1:0/3388603953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc106f10 0x7f2bdc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.348+0000 7f2be3838640 1 --2- 172.21.15.1:0/3388603953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bdc102790 0x7f2bdc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.348+0000 7f2be3838640 1 --2- 172.21.15.1:0/3388603953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc101de0 0x7f2bdc1021c0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.348+0000 7f2be3838640 1 -- 172.21.15.1:0/3388603953 >> 172.21.15.1:0/3388603953 conn(0x7f2bdc0fd8c0 msgr2=0x7f2bdc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.348+0000 7f2be3838640 1 -- 172.21.15.1:0/3388603953 shutdown_connections 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.348+0000 7f2be3838640 1 -- 172.21.15.1:0/3388603953 wait complete. 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be3838640 1 Processor -- start 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be3838640 1 -- start start 2024-09-16T00:39:37.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be3838640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc101de0 0x7f2bdc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be3838640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bdc102790 0x7f2bdc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be3838640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc106f10 0x7f2bdc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be3838640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111f30 con 0x7f2bdc106f10 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be3838640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111db0 con 0x7f2bdc101de0 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be3838640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1120b0 con 0x7f2bdc102790 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be1dd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc106f10 0x7f2bdc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be15d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc101de0 0x7f2bdc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be0dd3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bdc102790 0x7f2bdc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be15d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc101de0 0x7f2bdc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:40066/0 (socket says 172.21.15.1:40066) 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be15d4640 1 -- 172.21.15.1:0/3453675707 learned_addr learned my addr 172.21.15.1:0/3453675707 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.349+0000 7f2be1dd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc106f10 0x7f2bdc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39370/0 (socket says 172.21.15.1:39370) 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2be0dd3640 1 -- 172.21.15.1:0/3453675707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc101de0 msgr2=0x7f2bdc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2be0dd3640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc101de0 0x7f2bdc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2be0dd3640 1 -- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc106f10 msgr2=0x7f2bdc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2be0dd3640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc106f10 0x7f2bdc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2be0dd3640 1 -- 172.21.15.1:0/3453675707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc1a6900 con 0x7f2bdc102790 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2be1dd5640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc106f10 0x7f2bdc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2be0dd3640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bdc102790 0x7f2bdc074a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005fd0 tx=0x7f2bcc00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2bd27fc640 1 -- 172.21.15.1:0/3453675707 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc009c30 con 0x7f2bdc102790 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2bd27fc640 1 -- 172.21.15.1:0/3453675707 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc009dd0 con 0x7f2bdc102790 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2be3838640 1 -- 172.21.15.1:0/3453675707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc1a6bf0 con 0x7f2bdc102790 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2bd27fc640 1 -- 172.21.15.1:0/3453675707 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc010460 con 0x7f2bdc102790 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.350+0000 7f2be3838640 1 -- 172.21.15.1:0/3453675707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a7130 con 0x7f2bdc102790 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be5c11740 1 Processor -- start 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2bd27fc640 1 -- 172.21.15.1:0/3453675707 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2bcc003ef0 con 0x7f2bdc102790 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be5c11740 1 -- start start 2024-09-16T00:39:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0f14a0 0x56027e0f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e0fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0fc680 0x56027e0fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be5c11740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56027e0572e0 con 0x56027e0f14a0 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be5c11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56027dffece0 con 0x56027e0fc680 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be5c11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56027e0532a0 con 0x56027e0f1dc0 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be1dd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0f14a0 0x56027e0f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be1dd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0f14a0 0x56027e0f1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39382/0 (socket says 172.21.15.1:39382) 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.353+0000 7f2be1dd5640 1 -- 172.21.15.1:0/1667080450 learned_addr learned my addr 172.21.15.1:0/1667080450 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2be0dd3640 1 --2- 172.21.15.1:0/1667080450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0fc680 0x56027e0fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2be15d4640 1 --2- 172.21.15.1:0/1667080450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e0fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2be1dd5640 1 -- 172.21.15.1:0/1667080450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 msgr2=0x56027e0fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2be1dd5640 1 --2- 172.21.15.1:0/1667080450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e0fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2be1dd5640 1 -- 172.21.15.1:0/1667080450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0fc680 msgr2=0x56027e0fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:37.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2be1dd5640 1 --2- 172.21.15.1:0/1667080450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0fc680 0x56027e0fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2be1dd5640 1 -- 172.21.15.1:0/1667080450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56027dfe3440 con 0x56027e0f14a0 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2bd27fc640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bc80a3240 0x7f2bc80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2be1dd5640 1 --2- 172.21.15.1:0/1667080450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0f14a0 0x56027e0f1880 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400f600 tx=0x7f2bd400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be73bde447a3608 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2be15d4640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bc80a3240 0x7f2bc80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2bd27fc640 1 -- 172.21.15.1:0/3453675707 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2bcc0cf860 con 0x7f2bdc102790 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2bc77fe640 1 -- 172.21.15.1:0/1667080450 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4003b70 con 0x56027e0f14a0 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2bc77fe640 1 -- 172.21.15.1:0/1667080450 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd400dd60 con 0x56027e0f14a0 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.354+0000 7f2bc77fe640 1 -- 172.21.15.1:0/1667080450 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40047e0 con 0x56027e0f14a0 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be5c11740 1 -- 172.21.15.1:0/1667080450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0f14a0 msgr2=0x56027e0f1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be15d4640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bc80a3240 0x7f2bc80a5700 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x56027e0ee890 tx=0x7f2bd8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be5c11740 1 --2- 172.21.15.1:0/1667080450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0f14a0 0x56027e0f1880 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400f600 tx=0x7f2bd400fac0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be5c11740 1 -- 172.21.15.1:0/1667080450 shutdown_connections 2024-09-16T00:39:37.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be5c11740 1 --2- 172.21.15.1:0/1667080450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0fc680 0x56027e0fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be5c11740 1 --2- 172.21.15.1:0/1667080450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e0fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be5c11740 1 --2- 172.21.15.1:0/1667080450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0f14a0 0x56027e0f1880 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be5c11740 1 -- 172.21.15.1:0/1667080450 >> 172.21.15.1:0/1667080450 conn(0x56027e0eda30 msgr2=0x56027e0ede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be5c11740 1 -- 172.21.15.1:0/1667080450 shutdown_connections 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.355+0000 7f2be5c11740 1 -- 172.21.15.1:0/1667080450 wait complete. 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be5c11740 1 -- start start 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be5c11740 1 Processor -- start 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 0x56027e0fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e0f9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0fc680 0x56027e0f9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be5c11740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56027e0ff250 con 0x56027e0fc680 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be5c11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56027e012050 con 0x56027e0f14a0 2024-09-16T00:39:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be5c11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56027e0ff3d0 con 0x56027e0f1dc0 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be5c11740 10 client.0 _ll_register_callbacks cb 0x56027e052160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be1dd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0fc680 0x56027e0f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be1dd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0fc680 0x56027e0f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39394/0 (socket says 172.21.15.1:39394) 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.356+0000 7f2be1dd5640 1 -- 172.21.15.1:0/2749799158 learned_addr learned my addr 172.21.15.1:0/2749799158 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be15d4640 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 0x56027e0fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be0dd3640 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e0f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be1dd5640 1 -- 172.21.15.1:0/2749799158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 msgr2=0x56027e0f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be1dd5640 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e0f9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be1dd5640 1 -- 172.21.15.1:0/2749799158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 msgr2=0x56027e0fbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be1dd5640 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 0x56027e0fbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be1dd5640 1 -- 172.21.15.1:0/2749799158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56027ddc83c0 con 0x56027e0fc680 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be15d4640 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 0x56027e0fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be1dd5640 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0fc680 0x56027e0f9790 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4000f80 tx=0x7f2bd400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57477148c9db756 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:37.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2bc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/2749799158 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4004a40 con 0x56027e0fc680 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/2749799158 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2bd4004be0 con 0x56027e0fc680 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be5c11740 1 -- 172.21.15.1:0/2749799158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd400dd60 con 0x56027e0fc680 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/2749799158 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2bd4030050 con 0x56027e0fc680 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be5c11740 10 client.18177 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2be5c11740 1 -- 172.21.15.1:0/2749799158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56027e10abd0 con 0x56027e0fc680 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.357+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/2749799158 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd400dd60 con 0x56027e0fc680 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/2749799158 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2bd40055a0 con 0x56027e0fc680 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2be5c11740 10 client.18177 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:37.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2be5c11740 10 client.18177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2be5c11740 20 client.18177 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2be5c11740 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56027e10c030 0x56027e10e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2be5c11740 4 client.18177 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2be5c11740 1 -- 172.21.15.1:0/2749799158 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56027e064850 con 0x56027e10c030 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2be15d4640 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56027e10c030 0x56027e10e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2bc4ff9640 10 client.18177 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.358+0000 7f2be15d4640 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56027e10c030 0x56027e10e420 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10722db8ae66f35a server_cookie=8e85556ced43d63e in_seq=0 out_seq=0 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/2749799158 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56027e064850 con 0x56027e10c030 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2bc4ff9640 10 client.18177 handle_command_reply: tid=0 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2be5c11740 1 client.18177 shutdown 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2be5c11740 1 -- 172.21.15.1:0/2749799158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0fc680 msgr2=0x56027e0f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2be5c11740 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0fc680 0x56027e0f9790 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4000f80 tx=0x7f2bd400df10 comp rx=0 tx=0).stop 2024-09-16T00:39:37.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2be5c11740 1 -- 172.21.15.1:0/2749799158 shutdown_connections 2024-09-16T00:39:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2be5c11740 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56027e10c030 0x56027e10e420 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2bc4ff9640 0 client.18177 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2be5c11740 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0fc680 0x56027e0f9790 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2be5c11740 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e0f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2be5c11740 1 --2- 172.21.15.1:0/2749799158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 0x56027e0fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.359+0000 7f2be5c11740 1 -- 172.21.15.1:0/2749799158 >> 172.21.15.1:0/2749799158 conn(0x56027e0eda30 msgr2=0x56027e0f3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.360+0000 7f2be5c11740 1 -- 172.21.15.1:0/2749799158 shutdown_connections 2024-09-16T00:39:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.360+0000 7f2be5c11740 1 -- 172.21.15.1:0/2749799158 wait complete. 2024-09-16T00:39:37.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.360+0000 7f2be5c11740 20 client.18177 trim_cache size 0 max 16384 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be5c11740 1 Processor -- start 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be5c11740 1 -- start start 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e0fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 0x56027e113a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be5c11740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1b3900 con 0x56027e0eda30 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be5c11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1b11d0 con 0x56027e0f14a0 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be5c11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1af020 con 0x56027e0f1dc0 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be15d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e0fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be15d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e0fb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39402/0 (socket says 172.21.15.1:39402) 2024-09-16T00:39:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be15d4640 1 -- 172.21.15.1:0/469416627 learned_addr learned my addr 172.21.15.1:0/469416627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.378+0000 7f2be0dd3640 1 --2- 172.21.15.1:0/469416627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 0x56027e113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.379+0000 7f2be15d4640 1 -- 172.21.15.1:0/469416627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 msgr2=0x56027e113fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.379+0000 7f2be15d4640 1 --2- 172.21.15.1:0/469416627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.379+0000 7f2be15d4640 1 -- 172.21.15.1:0/469416627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 msgr2=0x56027e113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.379+0000 7f2be15d4640 1 --2- 172.21.15.1:0/469416627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 0x56027e113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.379+0000 7f2be15d4640 1 -- 172.21.15.1:0/469416627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56027e0eeae0 con 0x56027e0eda30 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.379+0000 7f2be15d4640 1 --2- 172.21.15.1:0/469416627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e0fb1c0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8006010 tx=0x7f2bd803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6131df9c9e00e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.379+0000 7f2bc6ffd640 1 -- 172.21.15.1:0/469416627 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8056070 con 0x56027e0eda30 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.379+0000 7f2bc6ffd640 1 -- 172.21.15.1:0/469416627 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8005520 con 0x56027e0eda30 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.379+0000 7f2bc6ffd640 1 -- 172.21.15.1:0/469416627 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8035040 con 0x56027e0eda30 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 -- 172.21.15.1:0/469416627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 msgr2=0x56027e0fb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 --2- 172.21.15.1:0/469416627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e0fb1c0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8006010 tx=0x7f2bd803f000 comp rx=0 tx=0).stop 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 -- 172.21.15.1:0/469416627 shutdown_connections 2024-09-16T00:39:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 --2- 172.21.15.1:0/469416627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f1dc0 0x56027e113fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 --2- 172.21.15.1:0/469416627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f14a0 0x56027e113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 --2- 172.21.15.1:0/469416627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e0fb1c0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 -- 172.21.15.1:0/469416627 >> 172.21.15.1:0/469416627 conn(0x56027e0f98f0 msgr2=0x56027e0faab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 -- 172.21.15.1:0/469416627 shutdown_connections 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 -- 172.21.15.1:0/469416627 wait complete. 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 -- start start 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.380+0000 7f2be5c11740 1 Processor -- start 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e104d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f14a0 0x56027e1020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be5c11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f1dc0 0x56027e102610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be5c11740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1af690 con 0x56027e0eda30 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be5c11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1b0b00 con 0x56027e0f1dc0 2024-09-16T00:39:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be5c11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1afd60 con 0x56027e0f14a0 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be5c11740 10 client.0 _ll_register_callbacks cb 0x7f2bcc0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be15d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e104d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be15d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e104d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39410/0 (socket says 172.21.15.1:39410) 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be15d4640 1 -- 172.21.15.1:0/3795083681 learned_addr learned my addr 172.21.15.1:0/3795083681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be0dd3640 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f14a0 0x56027e1020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be1dd5640 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f1dc0 0x56027e102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be15d4640 1 -- 172.21.15.1:0/3795083681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f14a0 msgr2=0x56027e1020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be15d4640 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f14a0 0x56027e1020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be15d4640 1 -- 172.21.15.1:0/3795083681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f1dc0 msgr2=0x56027e102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be15d4640 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f1dc0 0x56027e102610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be15d4640 1 -- 172.21.15.1:0/3795083681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56027e0ee6c0 con 0x56027e0eda30 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2be15d4640 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e104d80 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8031550 tx=0x7f2bd8032e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c760562ff18b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.381+0000 7f2bc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/3795083681 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd805e020 con 0x56027e0eda30 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/3795083681 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2bd8004240 con 0x56027e0eda30 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be5c11740 1 -- 172.21.15.1:0/3795083681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd8005520 con 0x56027e0eda30 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/3795083681 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2bd8056040 con 0x56027e0eda30 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be5c11740 10 client.18183 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be5c11740 1 -- 172.21.15.1:0/3795083681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56027e1768c0 con 0x56027e0eda30 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/3795083681 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8005520 con 0x56027e0eda30 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/3795083681 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2bd8059030 con 0x56027e0eda30 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be5c11740 10 client.18183 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be5c11740 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be5c11740 20 client.18183 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be5c11740 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56027e0fc680 0x56027e179020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be5c11740 4 client.18183 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:39:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be5c11740 1 -- 172.21.15.1:0/3795083681 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56027e05ed60 con 0x56027e0fc680 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.382+0000 7f2be0dd3640 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56027e0fc680 0x56027e179020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be0dd3640 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56027e0fc680 0x56027e179020 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d342004bd80d14 server_cookie=d389057fc36f8079 in_seq=0 out_seq=0 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2bc4ff9640 10 client.18183 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2bc4ff9640 1 -- 172.21.15.1:0/3795083681 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56027e05ed60 con 0x56027e0fc680 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2bc4ff9640 10 client.18183 handle_command_reply: tid=0 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 client.18183 shutdown 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 -- 172.21.15.1:0/3795083681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 msgr2=0x56027e104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e104d80 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8031550 tx=0x7f2bd8032e30 comp rx=0 tx=0).stop 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 -- 172.21.15.1:0/3795083681 shutdown_connections 2024-09-16T00:39:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56027e0fc680 0x56027e179020 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56027e0f1dc0 0x56027e102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56027e0f14a0 0x56027e1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 --2- 172.21.15.1:0/3795083681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56027e0eda30 0x56027e104d80 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 -- 172.21.15.1:0/3795083681 >> 172.21.15.1:0/3795083681 conn(0x56027e0f98f0 msgr2=0x56027e0f76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2bc4ff9640 0 client.18183 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 -- 172.21.15.1:0/3795083681 shutdown_connections 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.383+0000 7f2be5c11740 1 -- 172.21.15.1:0/3795083681 wait complete. 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be5c11740 20 client.18183 trim_cache size 0 max 16384 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:39:37.398 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:39:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "daea119e-ce17-413d-a89d-bfa75d9dde9b": { 2024-09-16T00:39:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:39:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "daea119e-ce17-413d-a89d-bfa75d9dde9b", 2024-09-16T00:39:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:39:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:39:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:39:37.399 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:39:37.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 -- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bc80a3240 msgr2=0x7f2bc80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bc80a3240 0x7f2bc80a5700 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x56027e0ee890 tx=0x7f2bd8005a20 comp rx=0 tx=0).stop 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 -- 172.21.15.1:0/3453675707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bdc102790 msgr2=0x7f2bdc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bdc102790 0x7f2bdc074a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005fd0 tx=0x7f2bcc00dde0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 -- 172.21.15.1:0/3453675707 shutdown_connections 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc106f10 0x7f2bdc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2bc80a3240 0x7f2bc80a5700 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bdc102790 0x7f2bdc074a30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 --2- 172.21.15.1:0/3453675707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc101de0 0x7f2bdc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 -- 172.21.15.1:0/3453675707 >> 172.21.15.1:0/3453675707 conn(0x7f2bdc0fd8c0 msgr2=0x7f2bdc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 -- 172.21.15.1:0/3453675707 shutdown_connections 2024-09-16T00:39:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:37.384+0000 7f2be3838640 1 -- 172.21.15.1:0/3453675707 wait complete. 2024-09-16T00:39:38.143 INFO:tasks.cephfs.filesystem:scrub status for tag:daea119e-ce17-413d-a89d-bfa75d9dde9b - {'path': '/', 'tag': 'daea119e-ce17-413d-a89d-bfa75d9dde9b', 'options': 'recursive,force'} 2024-09-16T00:39:38.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:38 smithi001 ceph-mon[30934]: pgmap v687: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:39:38.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:38 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:39:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:38 smithi153 ceph-mon[35538]: pgmap v687: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:39:38.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:38 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:39:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:38 smithi019 ceph-mon[35186]: pgmap v687: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:39:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:38 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:39:40.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:40 smithi001 ceph-mon[30934]: pgmap v688: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:40.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:40 smithi153 ceph-mon[35538]: pgmap v688: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:40.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:40 smithi019 ceph-mon[35186]: pgmap v688: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:42.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:42 smithi001 ceph-mon[30934]: pgmap v689: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:42.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:42 smithi153 ceph-mon[35538]: pgmap v689: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:42 smithi019 ceph-mon[35186]: pgmap v689: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:43.256 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:39:43.256 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-16T00:39:43.257 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:39:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.014+0000 7f883d065640 1 -- 172.21.15.1:0/3580795059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 msgr2=0x7f88381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.014+0000 7f883d065640 1 --2- 172.21.15.1:0/3580795059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 0x7f88381067e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8820009900 tx=0x7f882002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.014+0000 7f883d065640 1 -- 172.21.15.1:0/3580795059 shutdown_connections 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.014+0000 7f883d065640 1 --2- 172.21.15.1:0/3580795059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8838106f10 0x7f883810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.014+0000 7f883d065640 1 --2- 172.21.15.1:0/3580795059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 0x7f88381067e0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.014+0000 7f883d065640 1 --2- 172.21.15.1:0/3580795059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8838101de0 0x7f88381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.014+0000 7f883d065640 1 -- 172.21.15.1:0/3580795059 >> 172.21.15.1:0/3580795059 conn(0x7f88380fd8c0 msgr2=0x7f88380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.014+0000 7f883d065640 1 -- 172.21.15.1:0/3580795059 shutdown_connections 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.014+0000 7f883d065640 1 -- 172.21.15.1:0/3580795059 wait complete. 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f883d065640 1 Processor -- start 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f883d065640 1 -- start start 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f883d065640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8838101de0 0x7f88380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f883d065640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 0x7f8838074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f883d065640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8838106f10 0x7f8838074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f883d065640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8838111f90 con 0x7f8838102790 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f883d065640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8838111e10 con 0x7f8838106f10 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f883d065640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8838112110 con 0x7f8838101de0 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f882ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 0x7f8838074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f882ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 0x7f8838074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35822/0 (socket says 172.21.15.1:35822) 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f882ffff640 1 -- 172.21.15.1:0/2154488844 learned_addr learned my addr 172.21.15.1:0/2154488844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f883759e640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8838106f10 0x7f8838074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f8836d9d640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8838101de0 0x7f88380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f882ffff640 1 -- 172.21.15.1:0/2154488844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8838101de0 msgr2=0x7f88380744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f882ffff640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8838101de0 0x7f88380744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f882ffff640 1 -- 172.21.15.1:0/2154488844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8838106f10 msgr2=0x7f8838074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f882ffff640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8838106f10 0x7f8838074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f882ffff640 1 -- 172.21.15.1:0/2154488844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88381a68d0 con 0x7f8838102790 2024-09-16T00:39:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.015+0000 7f8836d9d640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8838101de0 0x7f88380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.016+0000 7f883759e640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8838106f10 0x7f8838074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.016+0000 7f882ffff640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 0x7f8838074a30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8820005ab0 tx=0x7f88200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.016+0000 7f8834d99640 1 -- 172.21.15.1:0/2154488844 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882003f030 con 0x7f8838102790 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.016+0000 7f8834d99640 1 -- 172.21.15.1:0/2154488844 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8820003ce0 con 0x7f8838102790 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.016+0000 7f8834d99640 1 -- 172.21.15.1:0/2154488844 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820035070 con 0x7f8838102790 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.016+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381a6b60 con 0x7f8838102790 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.016+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a7040 con 0x7f8838102790 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.017+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88381021c0 con 0x7f8838102790 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.018+0000 7f8834d99640 1 -- 172.21.15.1:0/2154488844 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8820042030 con 0x7f8838102790 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.019+0000 7f8834d99640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88240a3240 0x7f88240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.019+0000 7f8836d9d640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88240a3240 0x7f88240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.019+0000 7f8834d99640 1 -- 172.21.15.1:0/2154488844 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f88200f06d0 con 0x7f8838102790 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.020+0000 7f8836d9d640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88240a3240 0x7f88240a5700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8828006fd0 tx=0x7f8828008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.024+0000 7f8834d99640 1 -- 172.21.15.1:0/2154488844 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88200bcff0 con 0x7f8838102790 2024-09-16T00:39:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:44 smithi001 ceph-mon[30934]: pgmap v690: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:44.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.240+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8838109b60 con 0x7f8838102790 2024-09-16T00:39:44.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.240+0000 7f8834d99640 1 -- 172.21.15.1:0/2154488844 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f88200bc8f0 con 0x7f8838102790 2024-09-16T00:39:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:39:44.241 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:39:44.241 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:39:44.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88240a3240 msgr2=0x7f88240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88240a3240 0x7f88240a5700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8828006fd0 tx=0x7f8828008040 comp rx=0 tx=0).stop 2024-09-16T00:39:44.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 msgr2=0x7f8838074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 0x7f8838074a30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8820005ab0 tx=0x7f88200042f0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 shutdown_connections 2024-09-16T00:39:44.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8838106f10 0x7f8838074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838102790 0x7f8838074a30 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f88240a3240 0x7f88240a5700 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 --2- 172.21.15.1:0/2154488844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8838101de0 0x7f88380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 >> 172.21.15.1:0/2154488844 conn(0x7f88380fd8c0 msgr2=0x7f88380fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:44.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 shutdown_connections 2024-09-16T00:39:44.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:44.243+0000 7f883d065640 1 -- 172.21.15.1:0/2154488844 wait complete. 2024-09-16T00:39:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:44 smithi153 ceph-mon[35538]: pgmap v690: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:44 smithi019 ceph-mon[35186]: pgmap v690: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:39:45.090 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-16T00:39:45.091 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-16T00:39:45.358 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:45.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:45 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2154488844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:45.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:45 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2154488844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:45.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:45 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2154488844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.866+0000 7f141ec93640 1 -- 172.21.15.1:0/3923203490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14181037a0 msgr2=0x7f141810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.866+0000 7f141ec93640 1 --2- 172.21.15.1:0/3923203490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14181037a0 0x7f141810bc00 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f140c009900 tx=0x7f140c02a410 comp rx=0 tx=0).stop 2024-09-16T00:39:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 -- 172.21.15.1:0/3923203490 shutdown_connections 2024-09-16T00:39:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 --2- 172.21.15.1:0/3923203490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f141810c2f0 0x7f141810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 --2- 172.21.15.1:0/3923203490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14181037a0 0x7f141810bc00 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 --2- 172.21.15.1:0/3923203490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1418102df0 0x7f14181031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 -- 172.21.15.1:0/3923203490 >> 172.21.15.1:0/3923203490 conn(0x7f14180fc7d0 msgr2=0x7f14180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 -- 172.21.15.1:0/3923203490 shutdown_connections 2024-09-16T00:39:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 -- 172.21.15.1:0/3923203490 wait complete. 2024-09-16T00:39:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 Processor -- start 2024-09-16T00:39:45.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 -- start start 2024-09-16T00:39:45.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1418102df0 0x7f1418198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14181037a0 0x7f1418198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f141810c2f0 0x7f141819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1418112130 con 0x7f141810c2f0 2024-09-16T00:39:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1418111fb0 con 0x7f14181037a0 2024-09-16T00:39:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.867+0000 7f141ec93640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14181122b0 con 0x7f1418102df0 2024-09-16T00:39:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141d230640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f141810c2f0 0x7f141819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141d230640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f141810c2f0 0x7f141819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35848/0 (socket says 172.21.15.1:35848) 2024-09-16T00:39:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141d230640 1 -- 172.21.15.1:0/18132641 learned_addr learned my addr 172.21.15.1:0/18132641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f1417fff640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14181037a0 0x7f1418198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141ca2f640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1418102df0 0x7f1418198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141d230640 1 -- 172.21.15.1:0/18132641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1418102df0 msgr2=0x7f1418198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141d230640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1418102df0 0x7f1418198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141d230640 1 -- 172.21.15.1:0/18132641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14181037a0 msgr2=0x7f1418198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141d230640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14181037a0 0x7f1418198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141d230640 1 -- 172.21.15.1:0/18132641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141819d620 con 0x7f141810c2f0 2024-09-16T00:39:45.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141d230640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f141810c2f0 0x7f141819cf40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f141000eaa0 tx=0x7f141000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:45.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f1415ffb640 1 -- 172.21.15.1:0/18132641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141000cdb0 con 0x7f141810c2f0 2024-09-16T00:39:45.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f1415ffb640 1 -- 172.21.15.1:0/18132641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1410005020 con 0x7f141810c2f0 2024-09-16T00:39:45.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f1415ffb640 1 -- 172.21.15.1:0/18132641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14100116b0 con 0x7f141810c2f0 2024-09-16T00:39:45.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.868+0000 7f141ec93640 1 -- 172.21.15.1:0/18132641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141819d910 con 0x7f141810c2f0 2024-09-16T00:39:45.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.869+0000 7f141ec93640 1 -- 172.21.15.1:0/18132641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14181a51f0 con 0x7f141810c2f0 2024-09-16T00:39:45.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.871+0000 7f1415ffb640 1 -- 172.21.15.1:0/18132641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f1410005560 con 0x7f141810c2f0 2024-09-16T00:39:45.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.871+0000 7f142106c740 1 Processor -- start 2024-09-16T00:39:45.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.871+0000 7f142106c740 1 -- start start 2024-09-16T00:39:45.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f142106c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d00850 0x560275d00c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f142106c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d0b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f142106c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d0ba30 0x560275d0de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f142106c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560275c65160 con 0x560275d01170 2024-09-16T00:39:45.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f142106c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560275c0c720 con 0x560275d0ba30 2024-09-16T00:39:45.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f142106c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560275c61120 con 0x560275d00850 2024-09-16T00:39:45.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f1415ffb640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f14000a3240 0x7f14000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f141ca2f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d0b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f1417fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d00850 0x560275d00c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f141d230640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f14000a3240 0x7f14000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f1417fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d00850 0x560275d00c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:42220/0 (socket says 172.21.15.1:42220) 2024-09-16T00:39:45.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f1417fff640 1 -- 172.21.15.1:0/399963631 learned_addr learned my addr 172.21.15.1:0/399963631 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:45.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f1417fff640 1 --2- 172.21.15.1:0/399963631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d0ba30 0x560275d0de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f1415ffb640 1 -- 172.21.15.1:0/18132641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f14100cfff0 con 0x7f141810c2f0 2024-09-16T00:39:45.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f141ca2f640 1 -- 172.21.15.1:0/399963631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d00850 msgr2=0x560275d00c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:45.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f141ca2f640 1 --2- 172.21.15.1:0/399963631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d00850 0x560275d00c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f141ca2f640 1 -- 172.21.15.1:0/399963631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d0ba30 msgr2=0x560275d0de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:45.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f141ca2f640 1 --2- 172.21.15.1:0/399963631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d0ba30 0x560275d0de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f141ca2f640 1 -- 172.21.15.1:0/399963631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602759d6ba0 con 0x560275d01170 2024-09-16T00:39:45.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f1417fff640 1 --2- 172.21.15.1:0/399963631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d0ba30 0x560275d0de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:39:45.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.872+0000 7f141d230640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f14000a3240 0x7f14000a5700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f141009c140 tx=0x7f1410066bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:45.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.873+0000 7f141ca2f640 1 --2- 172.21.15.1:0/399963631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d0b4f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f140800b760 tx=0x7f140800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dcabb9a9aaecbd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:45.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.873+0000 7f13feffd640 1 -- 172.21.15.1:0/399963631 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140800d950 con 0x560275d01170 2024-09-16T00:39:45.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.873+0000 7f13feffd640 1 -- 172.21.15.1:0/399963631 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1408019070 con 0x560275d01170 2024-09-16T00:39:45.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.873+0000 7f13feffd640 1 -- 172.21.15.1:0/399963631 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408014d90 con 0x560275d01170 2024-09-16T00:39:45.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.873+0000 7f142106c740 1 -- 172.21.15.1:0/399963631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 msgr2=0x560275d0b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:45.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.873+0000 7f142106c740 1 --2- 172.21.15.1:0/399963631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d0b4f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f140800b760 tx=0x7f140800bc20 comp rx=0 tx=0).stop 2024-09-16T00:39:45.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 -- 172.21.15.1:0/399963631 shutdown_connections 2024-09-16T00:39:45.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 --2- 172.21.15.1:0/399963631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d0ba30 0x560275d0de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 --2- 172.21.15.1:0/399963631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d0b4f0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 --2- 172.21.15.1:0/399963631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d00850 0x560275d00c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 -- 172.21.15.1:0/399963631 >> 172.21.15.1:0/399963631 conn(0x560275cfcde0 msgr2=0x560275cfd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 -- 172.21.15.1:0/399963631 shutdown_connections 2024-09-16T00:39:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 -- 172.21.15.1:0/399963631 wait complete. 2024-09-16T00:39:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 -- start start 2024-09-16T00:39:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 Processor -- start 2024-09-16T00:39:45.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d08860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d08da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 0x560275d092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560275d0e600 con 0x560275d01170 2024-09-16T00:39:45.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560275bed900 con 0x560275d00850 2024-09-16T00:39:45.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560275d0e780 con 0x560275d0ba30 2024-09-16T00:39:45.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f141ca2f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d08da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.874+0000 7f142106c740 10 client.0 _ll_register_callbacks cb 0x560275beb780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:45.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f141ca2f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d08da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35858/0 (socket says 172.21.15.1:35858) 2024-09-16T00:39:45.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f141ca2f640 1 -- 172.21.15.1:0/1577151614 learned_addr learned my addr 172.21.15.1:0/1577151614 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:45.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f1417fff640 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d08860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f1417fff640 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 0x560275d092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f141ca2f640 1 -- 172.21.15.1:0/1577151614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 msgr2=0x560275d092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:45.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f141ca2f640 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 0x560275d092e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f141ca2f640 1 -- 172.21.15.1:0/1577151614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 msgr2=0x560275d08860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:45.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f141ca2f640 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d08860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f141ca2f640 1 -- 172.21.15.1:0/1577151614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560275c2dc50 con 0x560275d01170 2024-09-16T00:39:45.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.875+0000 7f1417fff640 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 0x560275d092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:45.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.876+0000 7f141ca2f640 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d08da0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f1408008010 tx=0x7f1408014dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96598b5b4866409 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:45.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.876+0000 7f13f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:45.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.876+0000 7f13f7fff640 1 -- 172.21.15.1:0/1577151614 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140800d950 con 0x560275d01170 2024-09-16T00:39:45.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.876+0000 7f13f7fff640 1 -- 172.21.15.1:0/1577151614 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f140800daf0 con 0x560275d01170 2024-09-16T00:39:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.876+0000 7f142106c740 1 -- 172.21.15.1:0/1577151614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1408019070 con 0x560275d01170 2024-09-16T00:39:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.876+0000 7f13f7fff640 1 -- 172.21.15.1:0/1577151614 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f140802d050 con 0x560275d01170 2024-09-16T00:39:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.876+0000 7f142106c740 10 client.18201 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.876+0000 7f142106c740 1 -- 172.21.15.1:0/1577151614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560275d1a210 con 0x560275d01170 2024-09-16T00:39:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.877+0000 7f13f7fff640 1 -- 172.21.15.1:0/1577151614 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1408019070 con 0x560275d01170 2024-09-16T00:39:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.877+0000 7f13f7fff640 1 -- 172.21.15.1:0/1577151614 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1408013070 con 0x560275d01170 2024-09-16T00:39:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.877+0000 7f142106c740 10 client.18201 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.877+0000 7f142106c740 10 client.18201 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:39:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.877+0000 7f142106c740 20 client.18201 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.877+0000 7f142106c740 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560275d1b670 0x560275d1da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.877+0000 7f142106c740 4 client.18201 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:39:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.877+0000 7f142106c740 1 -- 172.21.15.1:0/1577151614 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560275c72890 con 0x560275d1b670 2024-09-16T00:39:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.877+0000 7f1417fff640 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560275d1b670 0x560275d1da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f1417fff640 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560275d1b670 0x560275d1da60 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c7392a47db150e server_cookie=31aed530d3a2696d in_seq=0 out_seq=0 2024-09-16T00:39:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f13f7fff640 10 client.18201 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f13f7fff640 1 -- 172.21.15.1:0/1577151614 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560275c72890 con 0x560275d1b670 2024-09-16T00:39:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f13f7fff640 10 client.18201 handle_command_reply: tid=0 2024-09-16T00:39:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f142106c740 1 client.18201 shutdown 2024-09-16T00:39:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f142106c740 1 -- 172.21.15.1:0/1577151614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 msgr2=0x560275d08da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f142106c740 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d08da0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f1408008010 tx=0x7f1408014dc0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f142106c740 1 -- 172.21.15.1:0/1577151614 shutdown_connections 2024-09-16T00:39:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f142106c740 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560275d1b670 0x560275d1da60 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.878+0000 7f13f7fff640 0 client.18201 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.879+0000 7f142106c740 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 0x560275d092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.879+0000 7f142106c740 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d08da0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.879+0000 7f142106c740 1 --2- 172.21.15.1:0/1577151614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d08860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.879+0000 7f142106c740 1 -- 172.21.15.1:0/1577151614 >> 172.21.15.1:0/1577151614 conn(0x560275cfcde0 msgr2=0x560275d02ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.879+0000 7f142106c740 1 -- 172.21.15.1:0/1577151614 shutdown_connections 2024-09-16T00:39:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.879+0000 7f142106c740 1 -- 172.21.15.1:0/1577151614 wait complete. 2024-09-16T00:39:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.879+0000 7f142106c740 20 client.18201 trim_cache size 0 max 16384 2024-09-16T00:39:45.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f142106c740 1 Processor -- start 2024-09-16T00:39:45.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f142106c740 1 -- start start 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f142106c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f142106c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d01170 0x560275cf7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f142106c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d0ba30 0x560275d230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f142106c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f14181b1800 con 0x560275d0ba30 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f142106c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14181af0d0 con 0x560275d00850 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f142106c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14181acf20 con 0x560275d01170 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f1417fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d0ba30 0x560275d230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f1417fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d0ba30 0x560275d230c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35870/0 (socket says 172.21.15.1:35870) 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f1417fff640 1 -- 172.21.15.1:0/2240365536 learned_addr learned my addr 172.21.15.1:0/2240365536 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f1417fff640 1 --2- 172.21.15.1:0/2240365536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.896+0000 7f141ca2f640 1 --2- 172.21.15.1:0/2240365536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d01170 0x560275cf7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.897+0000 7f1417fff640 1 -- 172.21.15.1:0/2240365536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d01170 msgr2=0x560275cf7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.897+0000 7f1417fff640 1 --2- 172.21.15.1:0/2240365536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d01170 0x560275cf7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.897+0000 7f1417fff640 1 -- 172.21.15.1:0/2240365536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 msgr2=0x560275d077a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.897+0000 7f1417fff640 1 --2- 172.21.15.1:0/2240365536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d077a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.897+0000 7f1417fff640 1 -- 172.21.15.1:0/2240365536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560275cfcc10 con 0x560275d0ba30 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.897+0000 7f1417fff640 1 --2- 172.21.15.1:0/2240365536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d0ba30 0x560275d230c0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f14180697d0 tx=0x7f140c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d67b50f91ad70a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.897+0000 7f13fe7fc640 1 -- 172.21.15.1:0/2240365536 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c035040 con 0x560275d0ba30 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.897+0000 7f13fe7fc640 1 -- 172.21.15.1:0/2240365536 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140c0035f0 con 0x560275d0ba30 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.897+0000 7f13fe7fc640 1 -- 172.21.15.1:0/2240365536 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c04c5d0 con 0x560275d0ba30 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 -- 172.21.15.1:0/2240365536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d0ba30 msgr2=0x560275d230c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 --2- 172.21.15.1:0/2240365536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d0ba30 0x560275d230c0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f14180697d0 tx=0x7f140c02a410 comp rx=0 tx=0).stop 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 -- 172.21.15.1:0/2240365536 shutdown_connections 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 --2- 172.21.15.1:0/2240365536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d0ba30 0x560275d230c0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 --2- 172.21.15.1:0/2240365536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d01170 0x560275cf7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 --2- 172.21.15.1:0/2240365536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 -- 172.21.15.1:0/2240365536 >> 172.21.15.1:0/2240365536 conn(0x560275cfcde0 msgr2=0x560275d0a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 -- 172.21.15.1:0/2240365536 shutdown_connections 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 -- 172.21.15.1:0/2240365536 wait complete. 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 -- start start 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.898+0000 7f142106c740 1 Processor -- start 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f142106c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d0fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f142106c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d10000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f142106c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 0x560275d894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f142106c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f14181ad590 con 0x560275d01170 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f142106c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14181aea00 con 0x560275d00850 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f142106c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14181adc60 con 0x560275d0ba30 2024-09-16T00:39:45.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f142106c740 10 client.0 _ll_register_callbacks cb 0x7f140c032de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f141ca2f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d10000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f141ca2f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d10000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35876/0 (socket says 172.21.15.1:35876) 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f141ca2f640 1 -- 172.21.15.1:0/2379143642 learned_addr learned my addr 172.21.15.1:0/2379143642 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f1417fff640 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d0fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f141ca2f640 1 -- 172.21.15.1:0/2379143642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 msgr2=0x560275d894a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f141ca2f640 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 0x560275d894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f141ca2f640 1 -- 172.21.15.1:0/2379143642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 msgr2=0x560275d0fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f141ca2f640 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d0fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f141ca2f640 1 -- 172.21.15.1:0/2379143642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560275d108c0 con 0x560275d01170 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f141ca2f640 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d10000 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f140800d530 tx=0x7f140801d6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e62519d1841010 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.899+0000 7f13f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:39:45.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f13f7fff640 1 -- 172.21.15.1:0/2379143642 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140800b5b0 con 0x560275d01170 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f13f7fff640 1 -- 172.21.15.1:0/2379143642 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f140800d950 con 0x560275d01170 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f142106c740 1 -- 172.21.15.1:0/2379143642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140c0035f0 con 0x560275d01170 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f13f7fff640 1 -- 172.21.15.1:0/2379143642 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f140803a050 con 0x560275d01170 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f142106c740 10 client.18207 fetch_fsmap learned FSMap version 25 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f142106c740 1 -- 172.21.15.1:0/2379143642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560275d8aac0 con 0x560275d01170 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f13f7fff640 1 -- 172.21.15.1:0/2379143642 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140c0035f0 con 0x560275d01170 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f13f7fff640 1 -- 172.21.15.1:0/2379143642 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f140801f4d0 con 0x560275d01170 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f142106c740 10 client.18207 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f142106c740 10 client.18207 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f142106c740 20 client.18207 populate_metadata read hostname 'smithi001' 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f142106c740 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560275d8b250 0x560275d8d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f142106c740 4 client.18207 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-16T00:39:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.900+0000 7f142106c740 1 -- 172.21.15.1:0/2379143642 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x560275c6cbe0 con 0x560275d8b250 2024-09-16T00:39:45.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.901+0000 7f1417fff640 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560275d8b250 0x560275d8d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:45.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.901+0000 7f1417fff640 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560275d8b250 0x560275d8d670 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34fb7bbd01043547 server_cookie=20b8ba52fcbeb835 in_seq=0 out_seq=0 2024-09-16T00:39:45.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.901+0000 7f13f7fff640 10 client.18207 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f13f7fff640 1 -- 172.21.15.1:0/2379143642 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560275c6cbe0 con 0x560275d8b250 2024-09-16T00:39:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f13f7fff640 10 client.18207 handle_command_reply: tid=0 2024-09-16T00:39:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 client.18207 shutdown 2024-09-16T00:39:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 -- 172.21.15.1:0/2379143642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 msgr2=0x560275d10000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d10000 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f140800d530 tx=0x7f140801d6a0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 -- 172.21.15.1:0/2379143642 shutdown_connections 2024-09-16T00:39:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560275d8b250 0x560275d8d670 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560275d0ba30 0x560275d894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560275d01170 0x560275d10000 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 --2- 172.21.15.1:0/2379143642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560275d00850 0x560275d0fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 -- 172.21.15.1:0/2379143642 >> 172.21.15.1:0/2379143642 conn(0x560275cfcde0 msgr2=0x560275d08800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f13f7fff640 0 client.18207 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 -- 172.21.15.1:0/2379143642 shutdown_connections 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 1 -- 172.21.15.1:0/2379143642 wait complete. 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.912+0000 7f142106c740 20 client.18207 trim_cache size 0 max 16384 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:39:45.917 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 38, 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "1b71cc73": { 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 38, 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 141.8, 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:39:45.918 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:39:45.919 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:39:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 -- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f14000a3240 msgr2=0x7f14000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f14000a3240 0x7f14000a5700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f141009c140 tx=0x7f1410066bf0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 -- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f141810c2f0 msgr2=0x7f141819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f141810c2f0 0x7f141819cf40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f141000eaa0 tx=0x7f141000ef60 comp rx=0 tx=0).stop 2024-09-16T00:39:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 -- 172.21.15.1:0/18132641 shutdown_connections 2024-09-16T00:39:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f141810c2f0 0x7f141819cf40 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14181037a0 0x7f1418198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f14000a3240 0x7f14000a5700 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 --2- 172.21.15.1:0/18132641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1418102df0 0x7f1418198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:45.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 -- 172.21.15.1:0/18132641 >> 172.21.15.1:0/18132641 conn(0x7f14180fc7d0 msgr2=0x7f14180fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:45.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 -- 172.21.15.1:0/18132641 shutdown_connections 2024-09-16T00:39:45.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:39:45.913+0000 7f141ec93640 1 -- 172.21.15.1:0/18132641 wait complete. 2024-09-16T00:39:46.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:46 smithi001 ceph-mon[30934]: pgmap v691: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:46 smithi153 ceph-mon[35538]: pgmap v691: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:46 smithi019 ceph-mon[35186]: pgmap v691: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:46.682 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1b71cc73', quiesce took 3.4 seconds. Will release after: 18.450000000000003 2024-09-16T00:39:48.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:48 smithi001 ceph-mon[30934]: pgmap v692: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:48 smithi153 ceph-mon[35538]: pgmap v692: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:48 smithi019 ceph-mon[35186]: pgmap v692: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:39:50.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:50 smithi001 ceph-mon[30934]: pgmap v693: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:50 smithi153 ceph-mon[35538]: pgmap v693: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:50.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:50 smithi019 ceph-mon[35186]: pgmap v693: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:39:52.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:52 smithi001 ceph-mon[30934]: pgmap v694: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:39:52.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:52 smithi153 ceph-mon[35538]: pgmap v694: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:39:52.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:52 smithi019 ceph-mon[35186]: pgmap v694: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:39:54.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:54 smithi001 ceph-mon[30934]: pgmap v695: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-16T00:39:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:54 smithi153 ceph-mon[35538]: pgmap v695: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-16T00:39:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:54 smithi019 ceph-mon[35186]: pgmap v695: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-16T00:39:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:56 smithi001 ceph-mon[30934]: pgmap v696: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-09-16T00:39:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:56 smithi153 ceph-mon[35538]: pgmap v696: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-09-16T00:39:56.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:56 smithi019 ceph-mon[35186]: pgmap v696: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-09-16T00:39:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:39:58 smithi153 ceph-mon[35538]: pgmap v697: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-16T00:39:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:39:58 smithi019 ceph-mon[35186]: pgmap v697: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-16T00:39:58.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:39:58 smithi001 ceph-mon[30934]: pgmap v697: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-16T00:40:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:00 smithi153 ceph-mon[35538]: pgmap v698: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-16T00:40:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:00 smithi153 ceph-mon[35538]: overall HEALTH_OK 2024-09-16T00:40:00.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:00 smithi019 ceph-mon[35186]: pgmap v698: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-16T00:40:00.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:00 smithi019 ceph-mon[35186]: overall HEALTH_OK 2024-09-16T00:40:00.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:00 smithi001 ceph-mon[30934]: pgmap v698: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-16T00:40:00.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:00 smithi001 ceph-mon[30934]: overall HEALTH_OK 2024-09-16T00:40:02.473 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:02 smithi153 ceph-mon[35538]: pgmap v699: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-16T00:40:02.608 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:02 smithi019 ceph-mon[35186]: pgmap v699: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-16T00:40:02.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:02 smithi001 ceph-mon[30934]: pgmap v699: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-16T00:40:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:04 smithi019 ceph-mon[35186]: pgmap v700: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:40:04.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:04 smithi001 ceph-mon[30934]: pgmap v700: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:40:04.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:04 smithi153 ceph-mon[35538]: pgmap v700: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:40:05.134 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:05.409 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:05.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.885+0000 7f8a563f1640 1 -- 172.21.15.1:0/1088151712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 msgr2=0x7f8a50074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.885+0000 7f8a563f1640 1 --2- 172.21.15.1:0/1088151712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 0x7f8a50074a00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009900 tx=0x7f8a4002a480 comp rx=0 tx=0).stop 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.885+0000 7f8a563f1640 1 -- 172.21.15.1:0/1088151712 shutdown_connections 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.885+0000 7f8a563f1640 1 --2- 172.21.15.1:0/1088151712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5010e3c0 0x7f8a50110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.885+0000 7f8a563f1640 1 --2- 172.21.15.1:0/1088151712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50074f40 0x7f8a5010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.885+0000 7f8a563f1640 1 --2- 172.21.15.1:0/1088151712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 0x7f8a50074a00 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.885+0000 7f8a563f1640 1 -- 172.21.15.1:0/1088151712 >> 172.21.15.1:0/1088151712 conn(0x7f8a500fd8c0 msgr2=0x7f8a500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.885+0000 7f8a563f1640 1 -- 172.21.15.1:0/1088151712 shutdown_connections 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.885+0000 7f8a563f1640 1 -- 172.21.15.1:0/1088151712 wait complete. 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.886+0000 7f8a563f1640 1 Processor -- start 2024-09-16T00:40:05.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.886+0000 7f8a563f1640 1 -- start start 2024-09-16T00:40:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.886+0000 7f8a563f1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 0x7f8a5019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.886+0000 7f8a563f1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50074f40 0x7f8a5019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a563f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5010e3c0 0x7f8a5019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a563f1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50114170 con 0x7f8a50074620 2024-09-16T00:40:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a563f1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50113ff0 con 0x7f8a50074f40 2024-09-16T00:40:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a563f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501142f0 con 0x7f8a5010e3c0 2024-09-16T00:40:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 0x7f8a5019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:05.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 0x7f8a5019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54244/0 (socket says 172.21.15.1:54244) 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4ffff640 1 -- 172.21.15.1:0/3551078425 learned_addr learned my addr 172.21.15.1:0/3551078425 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a5498e640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5010e3c0 0x7f8a5019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4f7fe640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50074f40 0x7f8a5019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4ffff640 1 -- 172.21.15.1:0/3551078425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5010e3c0 msgr2=0x7f8a5019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4ffff640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5010e3c0 0x7f8a5019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4ffff640 1 -- 172.21.15.1:0/3551078425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50074f40 msgr2=0x7f8a5019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4ffff640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50074f40 0x7f8a5019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4ffff640 1 -- 172.21.15.1:0/3551078425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5019f600 con 0x7f8a50074620 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4ffff640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 0x7f8a5019a650 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006010 tx=0x7f8a400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/3551078425 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4003f030 con 0x7f8a50074620 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/3551078425 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a40003ce0 con 0x7f8a50074620 2024-09-16T00:40:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.887+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5019f890 con 0x7f8a50074620 2024-09-16T00:40:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.888+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/3551078425 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40035070 con 0x7f8a50074620 2024-09-16T00:40:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.888+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501a71d0 con 0x7f8a50074620 2024-09-16T00:40:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.889+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a50075cd0 con 0x7f8a50074620 2024-09-16T00:40:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.890+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/3551078425 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8a40042030 con 0x7f8a50074620 2024-09-16T00:40:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.890+0000 7f8a4d7fa640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a380a3240 0x7f8a380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:05.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.890+0000 7f8a4f7fe640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a380a3240 0x7f8a380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:05.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.890+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/3551078425 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8a400f0080 con 0x7f8a50074620 2024-09-16T00:40:05.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.891+0000 7f8a4f7fe640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a380a3240 0x7f8a380a5700 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8a50110cf0 tx=0x7f8a4400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:05.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:05.895+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/3551078425 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a400bc9a0 con 0x7f8a50074620 2024-09-16T00:40:06.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.107+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a50074a00 con 0x7f8a50074620 2024-09-16T00:40:06.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.108+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/3551078425 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f8a400bc2a0 con 0x7f8a50074620 2024-09-16T00:40:06.110 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:40:06.110 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:40:06.110 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:40:06.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a380a3240 msgr2=0x7f8a380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a380a3240 0x7f8a380a5700 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8a50110cf0 tx=0x7f8a4400a2d0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 msgr2=0x7f8a5019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 0x7f8a5019a650 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006010 tx=0x7f8a400042f0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 shutdown_connections 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a380a3240 0x7f8a380a5700 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5010e3c0 0x7f8a5019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50074f40 0x7f8a5019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 --2- 172.21.15.1:0/3551078425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a50074620 0x7f8a5019a650 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 >> 172.21.15.1:0/3551078425 conn(0x7f8a500fd8c0 msgr2=0x7f8a500fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 shutdown_connections 2024-09-16T00:40:06.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:06.110+0000 7f8a563f1640 1 -- 172.21.15.1:0/3551078425 wait complete. 2024-09-16T00:40:06.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:06 smithi001 ceph-mon[30934]: pgmap v701: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:40:06.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:06 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3551078425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:06 smithi019 ceph-mon[35186]: pgmap v701: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:40:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:06 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3551078425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:06 smithi153 ceph-mon[35538]: pgmap v701: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:40:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:06 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3551078425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:06.971 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 1b71cc73 --release --await' 2024-09-16T00:40:06.972 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 1b71cc73 --release --await 2024-09-16T00:40:07.236 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:07.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.751+0000 7f6b16e49640 1 -- 172.21.15.1:0/470763848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 msgr2=0x7f6b10106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.751+0000 7f6b16e49640 1 --2- 172.21.15.1:0/470763848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 0x7f6b10106e60 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009930 tx=0x7f6b0c02a420 comp rx=0 tx=0).stop 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.751+0000 7f6b16e49640 1 -- 172.21.15.1:0/470763848 shutdown_connections 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.751+0000 7f6b16e49640 1 --2- 172.21.15.1:0/470763848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b10101420 0x7f6b1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.751+0000 7f6b16e49640 1 --2- 172.21.15.1:0/470763848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b10100a80 0x7f6b10100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.751+0000 7f6b16e49640 1 --2- 172.21.15.1:0/470763848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 0x7f6b10106e60 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.751+0000 7f6b16e49640 1 -- 172.21.15.1:0/470763848 >> 172.21.15.1:0/470763848 conn(0x7f6b100fc7a0 msgr2=0x7f6b100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.751+0000 7f6b16e49640 1 -- 172.21.15.1:0/470763848 shutdown_connections 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.751+0000 7f6b16e49640 1 -- 172.21.15.1:0/470763848 wait complete. 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b16e49640 1 Processor -- start 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b16e49640 1 -- start start 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b16e49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b10100a80 0x7f6b10199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b16e49640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b10101420 0x7f6b10199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b16e49640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 0x7f6b101a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b16e49640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10110e80 con 0x7f6b10106a80 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b16e49640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10110d00 con 0x7f6b10100a80 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b16e49640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10111000 con 0x7f6b10101420 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b153e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 0x7f6b101a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b153e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 0x7f6b101a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54260/0 (socket says 172.21.15.1:54260) 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.752+0000 7f6b153e6640 1 -- 172.21.15.1:0/3151086806 learned_addr learned my addr 172.21.15.1:0/3151086806 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b10100a80 0x7f6b10199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b10101420 0x7f6b10199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b153e6640 1 -- 172.21.15.1:0/3151086806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b10101420 msgr2=0x7f6b10199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b153e6640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b10101420 0x7f6b10199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b153e6640 1 -- 172.21.15.1:0/3151086806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b10100a80 msgr2=0x7f6b10199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b153e6640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b10100a80 0x7f6b10199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b153e6640 1 -- 172.21.15.1:0/3151086806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b101a48d0 con 0x7f6b10106a80 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b153e6640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 0x7f6b101a4390 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800eaa0 tx=0x7f6b0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b05ffb640 1 -- 172.21.15.1:0/3151086806 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800cdb0 con 0x7f6b10106a80 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b16e49640 1 -- 172.21.15.1:0/3151086806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b101a4aa0 con 0x7f6b10106a80 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b05ffb640 1 -- 172.21.15.1:0/3151086806 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b08005020 con 0x7f6b10106a80 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b05ffb640 1 -- 172.21.15.1:0/3151086806 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b080116d0 con 0x7f6b10106a80 2024-09-16T00:40:07.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.753+0000 7f6b16e49640 1 -- 172.21.15.1:0/3151086806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b101a4fb0 con 0x7f6b10106a80 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b05ffb640 1 -- 172.21.15.1:0/3151086806 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6b080054d0 con 0x7f6b10106a80 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b19222740 1 Processor -- start 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b19222740 1 -- start start 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b19222740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 0x5604b7701440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b19222740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701980 0x5604b770bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b19222740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b770e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b19222740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5604b7666620 con 0x5604b770c240 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b19222740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5604b760de50 con 0x5604b7701060 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b19222740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5604b76625e0 con 0x5604b7701980 2024-09-16T00:40:07.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b05ffb640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6af80a3240 0x7f6af80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b770e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b770e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54276/0 (socket says 172.21.15.1:54276) 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.756+0000 7f6b07fff640 1 -- 172.21.15.1:0/2438374342 learned_addr learned my addr 172.21.15.1:0/2438374342 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b14be5640 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701980 0x5604b770bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b07fff640 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 0x5604b7701440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b05ffb640 1 -- 172.21.15.1:0/3151086806 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6b080cfea0 con 0x7f6b10106a80 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b153e6640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6af80a3240 0x7f6af80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b07fff640 1 -- 172.21.15.1:0/2438374342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701980 msgr2=0x5604b770bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b07fff640 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701980 0x5604b770bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b07fff640 1 -- 172.21.15.1:0/2438374342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 msgr2=0x5604b7701440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b07fff640 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 0x5604b7701440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b07fff640 1 -- 172.21.15.1:0/2438374342 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b75fed40 con 0x5604b770c240 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b14be5640 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701980 0x5604b770bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b153e6640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6af80a3240 0x7f6af80a5700 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6b080a47a0 tx=0x7f6b08000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6b07fff640 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b770e630 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6b000125a0 tx=0x7f6b0000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98491c556ecad04 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6af6ffd640 1 -- 172.21.15.1:0/2438374342 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00004250 con 0x5604b770c240 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6af6ffd640 1 -- 172.21.15.1:0/2438374342 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b000043f0 con 0x5604b770c240 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.757+0000 7f6af6ffd640 1 -- 172.21.15.1:0/2438374342 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b000035f0 con 0x5604b770c240 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 -- 172.21.15.1:0/2438374342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 msgr2=0x5604b770e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b770e630 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6b000125a0 tx=0x7f6b0000b870 comp rx=0 tx=0).stop 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 -- 172.21.15.1:0/2438374342 shutdown_connections 2024-09-16T00:40:07.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b770e630 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701980 0x5604b770bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 --2- 172.21.15.1:0/2438374342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 0x5604b7701440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 -- 172.21.15.1:0/2438374342 >> 172.21.15.1:0/2438374342 conn(0x5604b76fd5f0 msgr2=0x5604b76fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 -- 172.21.15.1:0/2438374342 shutdown_connections 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 -- 172.21.15.1:0/2438374342 wait complete. 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 -- start start 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.758+0000 7f6b19222740 1 Processor -- start 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b19222740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701060 0x5604b7709100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b19222740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 0x5604b7709640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b19222740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b770c240 0x5604b7709b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b19222740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5604b770ee10 con 0x5604b7701060 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b19222740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5604b75eef10 con 0x5604b7701980 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b19222740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5604b770ef90 con 0x5604b770c240 2024-09-16T00:40:07.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b19222740 10 client.0 _ll_register_callbacks cb 0x5604b7612db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701060 0x5604b7709100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701060 0x5604b7709100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54286/0 (socket says 172.21.15.1:54286) 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b07fff640 1 -- 172.21.15.1:0/3834074807 learned_addr learned my addr 172.21.15.1:0/3834074807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b07fff640 1 -- 172.21.15.1:0/3834074807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b770c240 msgr2=0x5604b7709b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 0x5604b7709640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b770c240 0x5604b7709b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b07fff640 1 -- 172.21.15.1:0/3834074807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 msgr2=0x5604b7709640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 0x5604b7709640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b07fff640 1 -- 172.21.15.1:0/3834074807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b7375580 con 0x5604b7701060 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.759+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 0x5604b7709640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:07.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701060 0x5604b7709100 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b9b0 tx=0x7f6b000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8a1cc2aca7ee8c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6aeffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6aeffff640 1 -- 172.21.15.1:0/3834074807 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00027070 con 0x5604b7701060 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6aeffff640 1 -- 172.21.15.1:0/3834074807 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6b00005080 con 0x5604b7701060 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b19222740 1 -- 172.21.15.1:0/3834074807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b000043f0 con 0x5604b7701060 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6aeffff640 1 -- 172.21.15.1:0/3834074807 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b00005280 con 0x5604b7701060 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b19222740 10 client.18225 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b19222740 1 -- 172.21.15.1:0/3834074807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604b743f960 con 0x5604b7701060 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6aeffff640 1 -- 172.21.15.1:0/3834074807 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b000043f0 con 0x5604b7701060 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6aeffff640 1 -- 172.21.15.1:0/3834074807 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b0001e440 con 0x5604b7701060 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b19222740 10 client.18225 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b19222740 10 client.18225 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:40:07.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b19222740 20 client.18225 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b19222740 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5604b771bc10 0x5604b771e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b19222740 4 client.18225 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.760+0000 7f6b19222740 1 -- 172.21.15.1:0/3834074807 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604b7673b90 con 0x5604b771bc10 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.761+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5604b771bc10 0x5604b771e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.761+0000 7f6aeffff640 10 client.18225 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.761+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5604b771bc10 0x5604b771e000 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=375a6ef6c6ed347a server_cookie=b4300acb2b6bc5b2 in_seq=0 out_seq=0 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6aeffff640 1 -- 172.21.15.1:0/3834074807 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604b7673b90 con 0x5604b771bc10 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6aeffff640 10 client.18225 handle_command_reply: tid=0 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 client.18225 shutdown 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 -- 172.21.15.1:0/3834074807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701060 msgr2=0x5604b7709100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701060 0x5604b7709100 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b9b0 tx=0x7f6b000048f0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 -- 172.21.15.1:0/3834074807 shutdown_connections 2024-09-16T00:40:07.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5604b771bc10 0x5604b771e000 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b770c240 0x5604b7709b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 0x5604b7709640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6aeffff640 0 client.18225 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:07.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 --2- 172.21.15.1:0/3834074807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701060 0x5604b7709100 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 -- 172.21.15.1:0/3834074807 >> 172.21.15.1:0/3834074807 conn(0x5604b76fd5f0 msgr2=0x5604b77032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:07.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.762+0000 7f6b19222740 1 -- 172.21.15.1:0/3834074807 shutdown_connections 2024-09-16T00:40:07.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.763+0000 7f6b19222740 1 -- 172.21.15.1:0/3834074807 wait complete. 2024-09-16T00:40:07.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.763+0000 7f6b19222740 20 client.18225 trim_cache size 0 max 16384 2024-09-16T00:40:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.779+0000 7f6b19222740 1 Processor -- start 2024-09-16T00:40:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.779+0000 7f6b19222740 1 -- start start 2024-09-16T00:40:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b19222740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 0x5604b7723770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b19222740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701980 0x5604b7723cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b19222740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b770c240 0x5604b7706380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b19222740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b101b1850 con 0x5604b7701980 2024-09-16T00:40:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b19222740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b101af120 con 0x5604b7701060 2024-09-16T00:40:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b19222740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b101acf70 con 0x5604b770c240 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b14be5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701980 0x5604b7723cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b14be5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701980 0x5604b7723cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54288/0 (socket says 172.21.15.1:54288) 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b14be5640 1 -- 172.21.15.1:0/3379833433 learned_addr learned my addr 172.21.15.1:0/3379833433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3379833433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 0x5604b7723770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3379833433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b770c240 0x5604b7706380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b14be5640 1 -- 172.21.15.1:0/3379833433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b770c240 msgr2=0x5604b7706380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3379833433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b770c240 0x5604b7706380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b14be5640 1 -- 172.21.15.1:0/3379833433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 msgr2=0x5604b7723770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3379833433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 0x5604b7723770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b14be5640 1 -- 172.21.15.1:0/3379833433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b77017b0 con 0x5604b7701980 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.780+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3379833433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701980 0x5604b7723cb0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c031550 tx=0x7f6b0c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930ca0b6f7e67c47 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6af67fc640 1 -- 172.21.15.1:0/3379833433 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c04b070 con 0x5604b7701980 2024-09-16T00:40:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6af67fc640 1 -- 172.21.15.1:0/3379833433 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0c031a60 con 0x5604b7701980 2024-09-16T00:40:07.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6af67fc640 1 -- 172.21.15.1:0/3379833433 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c035040 con 0x5604b7701980 2024-09-16T00:40:07.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6b19222740 1 -- 172.21.15.1:0/3379833433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701980 msgr2=0x5604b7723cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6b19222740 1 --2- 172.21.15.1:0/3379833433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701980 0x5604b7723cb0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c031550 tx=0x7f6b0c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6b19222740 1 -- 172.21.15.1:0/3379833433 shutdown_connections 2024-09-16T00:40:07.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6b19222740 1 --2- 172.21.15.1:0/3379833433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b770c240 0x5604b7706380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6b19222740 1 --2- 172.21.15.1:0/3379833433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b7701980 0x5604b7723cb0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6b19222740 1 --2- 172.21.15.1:0/3379833433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701060 0x5604b7723770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6b19222740 1 -- 172.21.15.1:0/3379833433 >> 172.21.15.1:0/3379833433 conn(0x5604b76fd5f0 msgr2=0x5604b770a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6b19222740 1 -- 172.21.15.1:0/3379833433 shutdown_connections 2024-09-16T00:40:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.781+0000 7f6b19222740 1 -- 172.21.15.1:0/3379833433 wait complete. 2024-09-16T00:40:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b19222740 1 -- start start 2024-09-16T00:40:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b19222740 1 Processor -- start 2024-09-16T00:40:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b19222740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701060 0x5604b77062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b19222740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 0x5604b7706820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b19222740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b7706d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b19222740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b101ad5e0 con 0x5604b770c240 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b19222740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b101aea50 con 0x5604b7701980 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b19222740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b101adcb0 con 0x5604b7701060 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b19222740 10 client.0 _ll_register_callbacks cb 0x7f6b0000b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b7706d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b7706d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54304/0 (socket says 172.21.15.1:54304) 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.782+0000 7f6b07fff640 1 -- 172.21.15.1:0/3283579244 learned_addr learned my addr 172.21.15.1:0/3283579244 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 0x5604b7706820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701060 0x5604b77062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b07fff640 1 -- 172.21.15.1:0/3283579244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701060 msgr2=0x5604b77062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701060 0x5604b77062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b07fff640 1 -- 172.21.15.1:0/3283579244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 msgr2=0x5604b7706820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 0x5604b7706820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b07fff640 1 -- 172.21.15.1:0/3283579244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b77858e0 con 0x5604b770c240 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b07fff640 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b7706d60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b6b0 tx=0x7f6b00004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4c6ebcaac5da09 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6aeffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6aeffff640 1 -- 172.21.15.1:0/3283579244 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00027070 con 0x5604b770c240 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6aeffff640 1 -- 172.21.15.1:0/3283579244 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6b00004e80 con 0x5604b770c240 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b19222740 1 -- 172.21.15.1:0/3283579244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5604b766e0a0 con 0x5604b770c240 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6aeffff640 1 -- 172.21.15.1:0/3283579244 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b00005080 con 0x5604b770c240 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b19222740 10 client.18231 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6b19222740 1 -- 172.21.15.1:0/3283579244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604b7786c00 con 0x5604b770c240 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.783+0000 7f6aeffff640 1 -- 172.21.15.1:0/3283579244 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5604b766e0a0 con 0x5604b770c240 2024-09-16T00:40:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6aeffff640 1 -- 172.21.15.1:0/3283579244 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b00043470 con 0x5604b770c240 2024-09-16T00:40:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6b19222740 10 client.18231 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6b19222740 10 client.18231 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:40:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6b19222740 20 client.18231 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6b19222740 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5604b77872c0 0x5604b77896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6b19222740 4 client.18231 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1b71cc73", "release": true, "await": true}] 2024-09-16T00:40:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6b19222740 1 -- 172.21.15.1:0/3283579244 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1b71cc73", "release": true, "await": true}) -- 0x5604b75ea5a0 con 0x5604b77872c0 2024-09-16T00:40:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5604b77872c0 0x5604b77896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6aeffff640 10 client.18231 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.784+0000 7f6b14be5640 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5604b77872c0 0x5604b77896b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61f873e27548983 server_cookie=661bd591c256e18a in_seq=0 out_seq=0 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6aeffff640 1 -- 172.21.15.1:0/3283579244 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5604b75ea5a0 con 0x5604b77872c0 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6aeffff640 10 client.18231 handle_command_reply: tid=0 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 client.18231 shutdown 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 -- 172.21.15.1:0/3283579244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 msgr2=0x5604b7706d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b7706d60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b6b0 tx=0x7f6b00004af0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 -- 172.21.15.1:0/3283579244 shutdown_connections 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5604b77872c0 0x5604b77896b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6aeffff640 0 client.18231 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604b770c240 0x5604b7706d60 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604b7701980 0x5604b7706820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 --2- 172.21.15.1:0/3283579244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604b7701060 0x5604b77062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 -- 172.21.15.1:0/3283579244 >> 172.21.15.1:0/3283579244 conn(0x5604b76fd5f0 msgr2=0x5604b76fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:07.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 -- 172.21.15.1:0/3283579244 shutdown_connections 2024-09-16T00:40:07.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 1 -- 172.21.15.1:0/3283579244 wait complete. 2024-09-16T00:40:07.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.787+0000 7f6b19222740 20 client.18231 trim_cache size 0 max 16384 2024-09-16T00:40:07.795 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:40:07.795 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:40:07.795 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:40:07.795 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 40, 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "1b71cc73": { 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 40, 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 141.8, 2024-09-16T00:40:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:07.797 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:40:07.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 -- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6af80a3240 msgr2=0x7f6af80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6af80a3240 0x7f6af80a5700 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6b080a47a0 tx=0x7f6b08000f40 comp rx=0 tx=0).stop 2024-09-16T00:40:07.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 -- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 msgr2=0x7f6b101a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 0x7f6b101a4390 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800eaa0 tx=0x7f6b0800ef60 comp rx=0 tx=0).stop 2024-09-16T00:40:07.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 -- 172.21.15.1:0/3151086806 shutdown_connections 2024-09-16T00:40:07.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b10106a80 0x7f6b101a4390 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6af80a3240 0x7f6af80a5700 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b10101420 0x7f6b10199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 --2- 172.21.15.1:0/3151086806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b10100a80 0x7f6b10199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 -- 172.21.15.1:0/3151086806 >> 172.21.15.1:0/3151086806 conn(0x7f6b100fc7a0 msgr2=0x7f6b100fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:07.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 -- 172.21.15.1:0/3151086806 shutdown_connections 2024-09-16T00:40:07.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:07.788+0000 7f6b16e49640 1 -- 172.21.15.1:0/3151086806 wait complete. 2024-09-16T00:40:08.144 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:40:08.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:08 smithi001 ceph-mon[30934]: pgmap v702: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:40:08.513 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:08 smithi153 ceph-mon[35538]: pgmap v702: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:40:08.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:08 smithi019 ceph-mon[35186]: pgmap v702: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:40:08.685 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1b71cc73', total seconds elapsed: 25.4 2024-09-16T00:40:08.685 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-16T00:40:08.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:08.742 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:09.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.267+0000 7fe3fffff640 1 -- 172.21.15.1:0/1744256583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4001019e0 msgr2=0x7fe400101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.267+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1744256583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4001019e0 0x7fe400101e40 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009930 tx=0x7fe3f402a420 comp rx=0 tx=0).stop 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.267+0000 7fe3fffff640 1 -- 172.21.15.1:0/1744256583 shutdown_connections 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.267+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1744256583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe400102380 0x7fe40010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.267+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1744256583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4001019e0 0x7fe400101e40 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.267+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1744256583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe400107af0 0x7fe400107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.267+0000 7fe3fffff640 1 -- 172.21.15.1:0/1744256583 >> 172.21.15.1:0/1744256583 conn(0x7fe4000fd700 msgr2=0x7fe4000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.267+0000 7fe3fffff640 1 -- 172.21.15.1:0/1744256583 shutdown_connections 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.267+0000 7fe3fffff640 1 -- 172.21.15.1:0/1744256583 wait complete. 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3fffff640 1 Processor -- start 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3fffff640 1 -- start start 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4001019e0 0x7fe40019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3fffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe400102380 0x7fe40019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe400107af0 0x7fe40019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3fffff640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe400111f10 con 0x7fe400107af0 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3fffff640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe400111d90 con 0x7fe4001019e0 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3fffff640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe400112090 con 0x7fe400102380 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3ff7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe400107af0 0x7fe40019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3ff7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe400107af0 0x7fe40019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54334/0 (socket says 172.21.15.1:54334) 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3ff7fe640 1 -- 172.21.15.1:0/1835863984 learned_addr learned my addr 172.21.15.1:0/1835863984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe400102380 0x7fe40019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3feffd640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4001019e0 0x7fe40019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3ff7fe640 1 -- 172.21.15.1:0/1835863984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe400102380 msgr2=0x7fe40019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3ff7fe640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe400102380 0x7fe40019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3ff7fe640 1 -- 172.21.15.1:0/1835863984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4001019e0 msgr2=0x7fe40019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.268+0000 7fe3ff7fe640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4001019e0 0x7fe40019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.269+0000 7fe3ff7fe640 1 -- 172.21.15.1:0/1835863984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40019f500 con 0x7fe400107af0 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.269+0000 7fe3ff7fe640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe400107af0 0x7fe40019ee20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800b970 tx=0x7fe3f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.269+0000 7fe3e7fff640 1 -- 172.21.15.1:0/1835863984 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8019070 con 0x7fe400107af0 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.269+0000 7fe3fffff640 1 -- 172.21.15.1:0/1835863984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40019f7f0 con 0x7fe400107af0 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.269+0000 7fe3e7fff640 1 -- 172.21.15.1:0/1835863984 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f800cb40 con 0x7fe400107af0 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.269+0000 7fe3e7fff640 1 -- 172.21.15.1:0/1835863984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8007650 con 0x7fe400107af0 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.269+0000 7fe3fffff640 1 -- 172.21.15.1:0/1835863984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4001a70d0 con 0x7fe400107af0 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.271+0000 7fe3e7fff640 1 -- 172.21.15.1:0/1835863984 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe3f801c070 con 0x7fe400107af0 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.271+0000 7fe4079dc740 1 Processor -- start 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.271+0000 7fe4079dc740 1 -- start start 2024-09-16T00:40:09.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 0x560a4d1b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8f90 0x560a4d1c3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe4079dc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560a4d11e2e0 con 0x560a4d1c3850 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe4079dc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a4d0c5ce0 con 0x560a4d1b8f90 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe4079dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560a4d11a2a0 con 0x560a4d1b8670 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3e7fff640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe3e80a3240 0x7fe3e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3fe7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3fe7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54350/0 (socket says 172.21.15.1:54350) 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3fe7fc640 1 -- 172.21.15.1:0/801082917 learned_addr learned my addr 172.21.15.1:0/801082917 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3feffd640 1 --2- 172.21.15.1:0/801082917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8f90 0x560a4d1c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/801082917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 0x560a4d1b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3ff7fe640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe3e80a3240 0x7fe3e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3e7fff640 1 -- 172.21.15.1:0/1835863984 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe3f80cf3d0 con 0x7fe400107af0 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3fe7fc640 1 -- 172.21.15.1:0/801082917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 msgr2=0x560a4d1b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/801082917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 0x560a4d1b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3fe7fc640 1 -- 172.21.15.1:0/801082917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8f90 msgr2=0x560a4d1c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/801082917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8f90 0x560a4d1c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.272+0000 7fe3fe7fc640 1 -- 172.21.15.1:0/801082917 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4d0aa440 con 0x560a4d1c3850 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.273+0000 7fe3ff7fe640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe3e80a3240 0x7fe3e80a5700 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8016000 tx=0x7fe3f806b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.273+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/801082917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1c5c40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4031550 tx=0x7fe3f40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d48424db0d1d6b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.273+0000 7fe3e57fa640 1 -- 172.21.15.1:0/801082917 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4035040 con 0x560a4d1c3850 2024-09-16T00:40:09.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.273+0000 7fe3e57fa640 1 -- 172.21.15.1:0/801082917 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f40319e0 con 0x560a4d1c3850 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.273+0000 7fe3e57fa640 1 -- 172.21.15.1:0/801082917 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4004a00 con 0x560a4d1c3850 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.273+0000 7fe4079dc740 1 -- 172.21.15.1:0/801082917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 msgr2=0x560a4d1c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.273+0000 7fe4079dc740 1 --2- 172.21.15.1:0/801082917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1c5c40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4031550 tx=0x7fe3f40058d0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 -- 172.21.15.1:0/801082917 shutdown_connections 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 --2- 172.21.15.1:0/801082917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1c5c40 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 --2- 172.21.15.1:0/801082917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8f90 0x560a4d1c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 --2- 172.21.15.1:0/801082917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 0x560a4d1b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 -- 172.21.15.1:0/801082917 >> 172.21.15.1:0/801082917 conn(0x560a4d1b4c00 msgr2=0x560a4d1b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 -- 172.21.15.1:0/801082917 shutdown_connections 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 -- 172.21.15.1:0/801082917 wait complete. 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 -- start start 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 Processor -- start 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 0x560a4d1c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1c3850 0x560a4d1c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560a4d1c6420 con 0x560a4d1b8f90 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a4d0d9050 con 0x560a4d1c3850 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560a4d1c65a0 con 0x560a4d1b8670 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe4079dc740 10 client.0 _ll_register_callbacks cb 0x560a4d119160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe3feffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe3feffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54354/0 (socket says 172.21.15.1:54354) 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.274+0000 7fe3feffd640 1 -- 172.21.15.1:0/3710038747 learned_addr learned my addr 172.21.15.1:0/3710038747 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 0x560a4d1c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3feffd640 1 -- 172.21.15.1:0/3710038747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 msgr2=0x560a4d1c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3feffd640 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 0x560a4d1c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1c3850 0x560a4d1c0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3feffd640 1 -- 172.21.15.1:0/3710038747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1c3850 msgr2=0x560a4d1c0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3feffd640 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1c3850 0x560a4d1c0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3feffd640 1 -- 172.21.15.1:0/3710038747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a4ce8f3c0 con 0x560a4d1b8f90 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1c3850 0x560a4d1c0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3feffd640 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c04f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000ce00 tx=0x7fe3f00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9946e5a996f0eb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/3710038747 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0011970 con 0x560a4d1b8f90 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/3710038747 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe3f0011b10 con 0x560a4d1b8f90 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe4079dc740 1 -- 172.21.15.1:0/3710038747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3f40319e0 con 0x560a4d1b8f90 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/3710038747 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe3f0011d10 con 0x560a4d1b8f90 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe4079dc740 10 client.18243 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe4079dc740 1 -- 172.21.15.1:0/3710038747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a4d1d1ee0 con 0x560a4d1b8f90 2024-09-16T00:40:09.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.275+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/3710038747 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f40319e0 con 0x560a4d1b8f90 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.276+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/3710038747 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe3f0018070 con 0x560a4d1b8f90 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.276+0000 7fe4079dc740 10 client.18243 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.276+0000 7fe4079dc740 10 client.18243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.276+0000 7fe4079dc740 20 client.18243 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.276+0000 7fe4079dc740 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560a4d1d3440 0x560a4d1d5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.276+0000 7fe4079dc740 4 client.18243 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.276+0000 7fe4079dc740 1 -- 172.21.15.1:0/3710038747 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a4d12b850 con 0x560a4d1d3440 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.276+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560a4d1d3440 0x560a4d1d5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.276+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560a4d1d3440 0x560a4d1d5880 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53dd55da80eafd48 server_cookie=3b0ae04fa1ee86fd in_seq=0 out_seq=0 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe3e2ff5640 10 client.18243 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/3710038747 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a4d12b850 con 0x560a4d1d3440 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe3e2ff5640 10 client.18243 handle_command_reply: tid=0 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 client.18243 shutdown 2024-09-16T00:40:09.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 -- 172.21.15.1:0/3710038747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 msgr2=0x560a4d1c04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c04f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000ce00 tx=0x7fe3f00039c0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 -- 172.21.15.1:0/3710038747 shutdown_connections 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560a4d1d3440 0x560a4d1d5880 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1c3850 0x560a4d1c0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c04f0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 --2- 172.21.15.1:0/3710038747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8670 0x560a4d1c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 -- 172.21.15.1:0/3710038747 >> 172.21.15.1:0/3710038747 conn(0x560a4d1b4c00 msgr2=0x560a4d1ba8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe3e2ff5640 0 client.18243 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.277+0000 7fe4079dc740 1 -- 172.21.15.1:0/3710038747 shutdown_connections 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.278+0000 7fe4079dc740 1 -- 172.21.15.1:0/3710038747 wait complete. 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.278+0000 7fe4079dc740 20 client.18243 trim_cache size 0 max 16384 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe4079dc740 1 Processor -- start 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe4079dc740 1 -- start start 2024-09-16T00:40:09.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8f90 0x560a4d1c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1db150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe4079dc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001b36d0 con 0x560a4d1c3850 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe4079dc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001b0fa0 con 0x560a4d1b8670 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe4079dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001aedf0 con 0x560a4d1b8f90 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe3fe7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1db150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe3fe7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1db150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54368/0 (socket says 172.21.15.1:54368) 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.295+0000 7fe3fe7fc640 1 -- 172.21.15.1:0/2581114633 learned_addr learned my addr 172.21.15.1:0/2581114633 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/2581114633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3feffd640 1 --2- 172.21.15.1:0/2581114633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8f90 0x560a4d1c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3fe7fc640 1 -- 172.21.15.1:0/2581114633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8f90 msgr2=0x560a4d1c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/2581114633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8f90 0x560a4d1c2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3fe7fc640 1 -- 172.21.15.1:0/2581114633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 msgr2=0x560a4d1cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:09.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/2581114633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3fe7fc640 1 -- 172.21.15.1:0/2581114633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4d1cfbc0 con 0x560a4d1c3850 2024-09-16T00:40:09.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/2581114633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1db150 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40317f0 tx=0x7fe3f40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cd377a64b8202a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3e4ff9640 1 -- 172.21.15.1:0/2581114633 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4035040 con 0x560a4d1c3850 2024-09-16T00:40:09.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3e4ff9640 1 -- 172.21.15.1:0/2581114633 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f404e070 con 0x560a4d1c3850 2024-09-16T00:40:09.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.296+0000 7fe3e4ff9640 1 -- 172.21.15.1:0/2581114633 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f403ee60 con 0x560a4d1c3850 2024-09-16T00:40:09.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.297+0000 7fe4079dc740 1 -- 172.21.15.1:0/2581114633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 msgr2=0x560a4d1db150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.297+0000 7fe4079dc740 1 --2- 172.21.15.1:0/2581114633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1db150 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40317f0 tx=0x7fe3f40058d0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.297+0000 7fe4079dc740 1 -- 172.21.15.1:0/2581114633 shutdown_connections 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.297+0000 7fe4079dc740 1 --2- 172.21.15.1:0/2581114633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1c3850 0x560a4d1db150 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.297+0000 7fe4079dc740 1 --2- 172.21.15.1:0/2581114633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1b8f90 0x560a4d1c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.297+0000 7fe4079dc740 1 --2- 172.21.15.1:0/2581114633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.297+0000 7fe4079dc740 1 -- 172.21.15.1:0/2581114633 >> 172.21.15.1:0/2581114633 conn(0x560a4d1b4c00 msgr2=0x560a4d1c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.298+0000 7fe4079dc740 1 -- 172.21.15.1:0/2581114633 shutdown_connections 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.298+0000 7fe4079dc740 1 -- 172.21.15.1:0/2581114633 wait complete. 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.298+0000 7fe4079dc740 1 -- start start 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.298+0000 7fe4079dc740 1 Processor -- start 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.298+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.298+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.298+0000 7fe4079dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1c3850 0x560a4d1c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe4079dc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001af460 con 0x560a4d1b8f90 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe4079dc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001b08d0 con 0x560a4d1b8670 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe4079dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001afb30 con 0x560a4d1c3850 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe4079dc740 10 client.0 _ll_register_callbacks cb 0x7fe3f0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3feffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3fe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3feffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54376/0 (socket says 172.21.15.1:54376) 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3fe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:43176/0 (socket says 172.21.15.1:43176) 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3feffd640 1 -- 172.21.15.1:0/4173117431 learned_addr learned my addr 172.21.15.1:0/4173117431 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1c3850 0x560a4d1c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3feffd640 1 -- 172.21.15.1:0/4173117431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1c3850 msgr2=0x560a4d1c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3feffd640 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1c3850 0x560a4d1c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3feffd640 1 -- 172.21.15.1:0/4173117431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 msgr2=0x560a4d1cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3feffd640 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3feffd640 1 -- 172.21.15.1:0/4173117431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a4d1c7a60 con 0x560a4d1b8f90 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3feffd640 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c6fe0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00196d0 tx=0x7fe3f0004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9096d3037996295e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.299+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/4173117431 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0019cd0 con 0x560a4d1b8f90 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.300+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/4173117431 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe3f0005030 con 0x560a4d1b8f90 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.300+0000 7fe4079dc740 1 -- 172.21.15.1:0/4173117431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3f404e070 con 0x560a4d1b8f90 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.300+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/4173117431 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe3f0036050 con 0x560a4d1b8f90 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.300+0000 7fe4079dc740 10 client.18249 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.300+0000 7fe4079dc740 1 -- 172.21.15.1:0/4173117431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a4d23ded0 con 0x560a4d1b8f90 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.300+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/4173117431 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f404e070 con 0x560a4d1b8f90 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.300+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/4173117431 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe3f0018070 con 0x560a4d1b8f90 2024-09-16T00:40:09.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.300+0000 7fe4079dc740 10 client.18249 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.300+0000 7fe4079dc740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe4079dc740 20 client.18249 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe4079dc740 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560a4d23e6d0 0x560a4d240af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe4079dc740 4 client.18249 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe4079dc740 1 -- 172.21.15.1:0/4173117431 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a4d125d60 con 0x560a4d23e6d0 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560a4d23e6d0 0x560a4d240af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe3e2ff5640 10 client.18249 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe3fe7fc640 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560a4d23e6d0 0x560a4d240af0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50b87f05be875fb server_cookie=c1727226fb824b31 in_seq=0 out_seq=0 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe3e2ff5640 1 -- 172.21.15.1:0/4173117431 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560a4d125d60 con 0x560a4d23e6d0 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe3e2ff5640 10 client.18249 handle_command_reply: tid=0 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.301+0000 7fe4079dc740 1 client.18249 shutdown 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 -- 172.21.15.1:0/4173117431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 msgr2=0x560a4d1c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c6fe0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00196d0 tx=0x7fe3f0004000 comp rx=0 tx=0).stop 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 -- 172.21.15.1:0/4173117431 shutdown_connections 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560a4d23e6d0 0x560a4d240af0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a4d1c3850 0x560a4d1c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a4d1b8f90 0x560a4d1c6fe0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe3e2ff5640 0 client.18249 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 --2- 172.21.15.1:0/4173117431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4d1b8670 0x560a4d1cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 -- 172.21.15.1:0/4173117431 >> 172.21.15.1:0/4173117431 conn(0x560a4d1b4c00 msgr2=0x560a4d1b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 -- 172.21.15.1:0/4173117431 shutdown_connections 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 1 -- 172.21.15.1:0/4173117431 wait complete. 2024-09-16T00:40:09.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.302+0000 7fe4079dc740 20 client.18249 trim_cache size 0 max 16384 2024-09-16T00:40:09.318 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:40:09.318 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:40:09.318 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:40:09.318 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:40:09.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 -- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe3e80a3240 msgr2=0x7fe3e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe3e80a3240 0x7fe3e80a5700 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8016000 tx=0x7fe3f806b260 comp rx=0 tx=0).stop 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 -- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe400107af0 msgr2=0x7fe40019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe400107af0 0x7fe40019ee20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800b970 tx=0x7fe3f800be30 comp rx=0 tx=0).stop 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 -- 172.21.15.1:0/1835863984 shutdown_connections 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe400107af0 0x7fe40019ee20 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe3e80a3240 0x7fe3e80a5700 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe400102380 0x7fe40019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 --2- 172.21.15.1:0/1835863984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4001019e0 0x7fe40019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 -- 172.21.15.1:0/1835863984 >> 172.21.15.1:0/1835863984 conn(0x7fe4000fd700 msgr2=0x7fe40010d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 -- 172.21.15.1:0/1835863984 shutdown_connections 2024-09-16T00:40:09.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:09.303+0000 7fe3fffff640 1 -- 172.21.15.1:0/1835863984 wait complete. 2024-09-16T00:40:09.531 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.027+0000 7f2c58a39640 1 -- 172.21.15.1:0/1754376994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102ef0 msgr2=0x7f2c5410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.027+0000 7f2c58a39640 1 --2- 172.21.15.1:0/1754376994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102ef0 0x7f2c5410da40 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009ca0 tx=0x7f2c4402a950 comp rx=0 tx=0).stop 2024-09-16T00:40:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.028+0000 7f2c58a39640 1 -- 172.21.15.1:0/1754376994 shutdown_connections 2024-09-16T00:40:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.028+0000 7f2c58a39640 1 --2- 172.21.15.1:0/1754376994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102ef0 0x7f2c5410da40 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.028+0000 7f2c58a39640 1 --2- 172.21.15.1:0/1754376994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c54102570 0x7f2c541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.028+0000 7f2c58a39640 1 --2- 172.21.15.1:0/1754376994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54100e20 0x7f2c54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.028+0000 7f2c58a39640 1 -- 172.21.15.1:0/1754376994 >> 172.21.15.1:0/1754376994 conn(0x7f2c540fc7d0 msgr2=0x7f2c540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.028+0000 7f2c58a39640 1 -- 172.21.15.1:0/1754376994 shutdown_connections 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.028+0000 7f2c58a39640 1 -- 172.21.15.1:0/1754376994 wait complete. 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.028+0000 7f2c58a39640 1 Processor -- start 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.028+0000 7f2c58a39640 1 -- start start 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c58a39640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c54100e20 0x7f2c5410e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c58a39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102570 0x7f2c5410e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c58a39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102ef0 0x7f2c5410edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c58a39640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54112090 con 0x7f2c54102570 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c58a39640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54111f10 con 0x7f2c54102ef0 2024-09-16T00:40:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c58a39640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54112210 con 0x7f2c54100e20 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c51d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102570 0x7f2c5410e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c51d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102570 0x7f2c5410e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54394/0 (socket says 172.21.15.1:54394) 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c51d9b640 1 -- 172.21.15.1:0/482001349 learned_addr learned my addr 172.21.15.1:0/482001349 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c52d9d640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102ef0 0x7f2c5410edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c5259c640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c54100e20 0x7f2c5410e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c51d9b640 1 -- 172.21.15.1:0/482001349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c54100e20 msgr2=0x7f2c5410e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c51d9b640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c54100e20 0x7f2c5410e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c51d9b640 1 -- 172.21.15.1:0/482001349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102ef0 msgr2=0x7f2c5410edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c51d9b640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102ef0 0x7f2c5410edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c51d9b640 1 -- 172.21.15.1:0/482001349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c541a6950 con 0x7f2c54102570 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c52d9d640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102ef0 0x7f2c5410edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c51d9b640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102570 0x7f2c5410e890 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c970 tx=0x7f2c3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c4b7fe640 1 -- 172.21.15.1:0/482001349 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c007bb0 con 0x7f2c54102570 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c4b7fe640 1 -- 172.21.15.1:0/482001349 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3c007d50 con 0x7f2c54102570 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c4b7fe640 1 -- 172.21.15.1:0/482001349 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00f450 con 0x7f2c54102570 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.029+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c541a6c40 con 0x7f2c54102570 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.030+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c541a7180 con 0x7f2c54102570 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.030+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c54101200 con 0x7f2c54102570 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.032+0000 7f2c4b7fe640 1 -- 172.21.15.1:0/482001349 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2c3c004fe0 con 0x7f2c54102570 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.032+0000 7f2c4b7fe640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a32c0 0x7f2c400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.033+0000 7f2c5259c640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a32c0 0x7f2c400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.033+0000 7f2c4b7fe640 1 -- 172.21.15.1:0/482001349 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2c3c0cfeb0 con 0x7f2c54102570 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.033+0000 7f2c5259c640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a32c0 0x7f2c400a5780 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c005fd0 tx=0x7f2c4c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.036+0000 7f2c4b7fe640 1 -- 172.21.15.1:0/482001349 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c3c014030 con 0x7f2c54102570 2024-09-16T00:40:10.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.253+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c5410f880 con 0x7f2c54102570 2024-09-16T00:40:10.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.254+0000 7f2c4b7fe640 1 -- 172.21.15.1:0/482001349 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f2c3c09c7f0 con 0x7f2c54102570 2024-09-16T00:40:10.256 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:40:10.256 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:40:10.256 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:40:10.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.256+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a32c0 msgr2=0x7f2c400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:10.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.256+0000 7f2c58a39640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a32c0 0x7f2c400a5780 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c005fd0 tx=0x7f2c4c005850 comp rx=0 tx=0).stop 2024-09-16T00:40:10.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.256+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102570 msgr2=0x7f2c5410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:10.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.256+0000 7f2c58a39640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102570 0x7f2c5410e890 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c970 tx=0x7f2c3c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:40:10.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.256+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 shutdown_connections 2024-09-16T00:40:10.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.256+0000 7f2c58a39640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c54102ef0 0x7f2c5410edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:10.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.256+0000 7f2c58a39640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c54102570 0x7f2c5410e890 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:10.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.256+0000 7f2c58a39640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2c400a32c0 0x7f2c400a5780 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:10.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.256+0000 7f2c58a39640 1 --2- 172.21.15.1:0/482001349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c54100e20 0x7f2c5410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:10.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.257+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 >> 172.21.15.1:0/482001349 conn(0x7f2c540fc7d0 msgr2=0x7f2c54106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:10.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.257+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 shutdown_connections 2024-09-16T00:40:10.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:10.257+0000 7f2c58a39640 1 -- 172.21.15.1:0/482001349 wait complete. 2024-09-16T00:40:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:10 smithi019 ceph-mon[35186]: pgmap v703: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:40:10.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:10 smithi001 ceph-mon[30934]: pgmap v703: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:40:10.664 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:40:10.664 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:10.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:10 smithi153 ceph-mon[35538]: pgmap v703: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:40:11.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:11 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/482001349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:11.460 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-16T00:40:11.461 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-16T00:40:11.489 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:11.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:11 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/482001349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:11.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:11 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/482001349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:12.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.983+0000 7ff5d10a5640 1 -- 172.21.15.1:0/1736409705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 msgr2=0x7ff5cc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:12.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.983+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/1736409705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 0x7ff5cc103120 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009900 tx=0x7ff5bc02a480 comp rx=0 tx=0).stop 2024-09-16T00:40:12.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.983+0000 7ff5d10a5640 1 -- 172.21.15.1:0/1736409705 shutdown_connections 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.983+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/1736409705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc10c170 0x7ff5cc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.983+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/1736409705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5cc103660 0x7ff5cc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.983+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/1736409705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 0x7ff5cc103120 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.983+0000 7ff5d10a5640 1 -- 172.21.15.1:0/1736409705 >> 172.21.15.1:0/1736409705 conn(0x7ff5cc0fc7a0 msgr2=0x7ff5cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.983+0000 7ff5d10a5640 1 -- 172.21.15.1:0/1736409705 shutdown_connections 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.983+0000 7ff5d10a5640 1 -- 172.21.15.1:0/1736409705 wait complete. 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.984+0000 7ff5d10a5640 1 Processor -- start 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.984+0000 7ff5d10a5640 1 -- start start 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.984+0000 7ff5d10a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.984+0000 7ff5d10a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc103660 0x7ff5cc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.984+0000 7ff5d10a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5cc10c170 0x7ff5cc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:12.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5d10a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111f70 con 0x7ff5cc102d40 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5d10a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111df0 con 0x7ff5cc103660 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5d10a5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1120f0 con 0x7ff5cc10c170 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36108/0 (socket says 172.21.15.1:36108) 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cad9d640 1 -- 172.21.15.1:0/210232123 learned_addr learned my addr 172.21.15.1:0/210232123 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5ca59c640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc103660 0x7ff5cc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cb59e640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5cc10c170 0x7ff5cc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cad9d640 1 -- 172.21.15.1:0/210232123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5cc10c170 msgr2=0x7ff5cc19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cad9d640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5cc10c170 0x7ff5cc19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cad9d640 1 -- 172.21.15.1:0/210232123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc103660 msgr2=0x7ff5cc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:12.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cad9d640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc103660 0x7ff5cc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cad9d640 1 -- 172.21.15.1:0/210232123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc19d4a0 con 0x7ff5cc102d40 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5cad9d640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1984f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc006010 tx=0x7ff5bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5b3fff640 1 -- 172.21.15.1:0/210232123 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc03f030 con 0x7ff5cc102d40 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5b3fff640 1 -- 172.21.15.1:0/210232123 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5bc003ce0 con 0x7ff5cc102d40 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5cc19d730 con 0x7ff5cc102d40 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5b3fff640 1 -- 172.21.15.1:0/210232123 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc035070 con 0x7ff5cc102d40 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.985+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5cc1a5070 con 0x7ff5cc102d40 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.987+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5cc06b070 con 0x7ff5cc102d40 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.988+0000 7ff5b3fff640 1 -- 172.21.15.1:0/210232123 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff5bc042030 con 0x7ff5cc102d40 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.988+0000 7ff5b3fff640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5b40a3240 0x7ff5b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.989+0000 7ff5b3fff640 1 -- 172.21.15.1:0/210232123 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff5bc0f03d0 con 0x7ff5cc102d40 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.989+0000 7ff5ca59c640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5b40a3240 0x7ff5b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:12.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.989+0000 7ff5ca59c640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5b40a3240 0x7ff5b40a5700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc10d640 tx=0x7ff5c0005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:12.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:11.993+0000 7ff5b3fff640 1 -- 172.21.15.1:0/210232123 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5bc0bcdb0 con 0x7ff5cc102d40 2024-09-16T00:40:12.040 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:12.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.210+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5cc1071e0 con 0x7ff5cc102d40 2024-09-16T00:40:12.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.211+0000 7ff5b3fff640 1 -- 172.21.15.1:0/210232123 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7ff5bc0bc6b0 con 0x7ff5cc102d40 2024-09-16T00:40:12.213 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:40:12.213 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:40:12.213 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:40:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5b40a3240 msgr2=0x7ff5b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5b40a3240 0x7ff5b40a5700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc10d640 tx=0x7ff5c0005f50 comp rx=0 tx=0).stop 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 msgr2=0x7ff5cc1984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1984f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc006010 tx=0x7ff5bc0042f0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 shutdown_connections 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5b40a3240 0x7ff5b40a5700 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5cc10c170 0x7ff5cc19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc103660 0x7ff5cc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 --2- 172.21.15.1:0/210232123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1984f0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 >> 172.21.15.1:0/210232123 conn(0x7ff5cc0fc7a0 msgr2=0x7ff5cc0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 shutdown_connections 2024-09-16T00:40:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:12.213+0000 7ff5d10a5640 1 -- 172.21.15.1:0/210232123 wait complete. 2024-09-16T00:40:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:12 smithi001 ceph-mon[30934]: pgmap v704: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-16T00:40:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:12 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/210232123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:12 smithi019 ceph-mon[35186]: pgmap v704: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-16T00:40:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:12 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/210232123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:12.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:12 smithi153 ceph-mon[35538]: pgmap v704: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-16T00:40:12.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:12 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/210232123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:13.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:40:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.305+0000 7f70d73ac640 1 -- 172.21.15.1:0/3299916753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102570 msgr2=0x7f70d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.305+0000 7f70d73ac640 1 --2- 172.21.15.1:0/3299916753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102570 0x7f70d01029b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009900 tx=0x7f70c402a480 comp rx=0 tx=0).stop 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.305+0000 7f70d73ac640 1 -- 172.21.15.1:0/3299916753 shutdown_connections 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.305+0000 7f70d73ac640 1 --2- 172.21.15.1:0/3299916753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d0102ef0 0x7f70d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.305+0000 7f70d73ac640 1 --2- 172.21.15.1:0/3299916753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102570 0x7f70d01029b0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.305+0000 7f70d73ac640 1 --2- 172.21.15.1:0/3299916753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0100e20 0x7f70d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.305+0000 7f70d73ac640 1 -- 172.21.15.1:0/3299916753 >> 172.21.15.1:0/3299916753 conn(0x7f70d00fc7d0 msgr2=0x7f70d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.305+0000 7f70d73ac640 1 -- 172.21.15.1:0/3299916753 shutdown_connections 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.305+0000 7f70d73ac640 1 -- 172.21.15.1:0/3299916753 wait complete. 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d73ac640 1 Processor -- start 2024-09-16T00:40:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d73ac640 1 -- start start 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d73ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0100e20 0x7f70d0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d73ac640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d0102570 0x7f70d0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d73ac640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102ef0 0x7f70d019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d73ac640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0110f50 con 0x7f70d0102ef0 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d73ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0110dd0 con 0x7f70d0100e20 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d73ac640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01110d0 con 0x7f70d0102570 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d5949640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102ef0 0x7f70d019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d5949640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102ef0 0x7f70d019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36128/0 (socket says 172.21.15.1:36128) 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d5949640 1 -- 172.21.15.1:0/119602689 learned_addr learned my addr 172.21.15.1:0/119602689 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d5148640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0100e20 0x7f70d0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.306+0000 7f70d4947640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d0102570 0x7f70d0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70d5949640 1 -- 172.21.15.1:0/119602689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d0102570 msgr2=0x7f70d0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70d5949640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d0102570 0x7f70d0198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70d5949640 1 -- 172.21.15.1:0/119602689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0100e20 msgr2=0x7f70d0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70d5949640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0100e20 0x7f70d0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70d5949640 1 -- 172.21.15.1:0/119602689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d019d500 con 0x7f70d0102ef0 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70d5949640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102ef0 0x7f70d019ce20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f70c000d750 tx=0x7f70c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70ce7fc640 1 -- 172.21.15.1:0/119602689 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0017070 con 0x7f70d0102ef0 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70d73ac640 1 -- 172.21.15.1:0/119602689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d019d7f0 con 0x7f70d0102ef0 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70ce7fc640 1 -- 172.21.15.1:0/119602689 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c0005500 con 0x7f70d0102ef0 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70ce7fc640 1 -- 172.21.15.1:0/119602689 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0004d90 con 0x7f70d0102ef0 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.307+0000 7f70d73ac640 1 -- 172.21.15.1:0/119602689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d01a50d0 con 0x7f70d0102ef0 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.310+0000 7f70ce7fc640 1 -- 172.21.15.1:0/119602689 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f70c0004f30 con 0x7f70d0102ef0 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.310+0000 7f70d9785740 1 Processor -- start 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.310+0000 7f70d9785740 1 -- start start 2024-09-16T00:40:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d9785740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440bdf850 0x565440bdfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d9785740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440be0170 0x565440bea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d9785740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440beaa30 0x565440bece20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d9785740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x565440b44160 con 0x565440be0170 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d9785740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565440aeb720 con 0x565440beaa30 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d9785740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x565440b40120 con 0x565440bdf850 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70ce7fc640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70b80a32c0 0x7f70b80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d5148640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440be0170 0x565440bea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d5148640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440be0170 0x565440bea4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36130/0 (socket says 172.21.15.1:36130) 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d5148640 1 -- 172.21.15.1:0/3092209449 learned_addr learned my addr 172.21.15.1:0/3092209449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d4947640 1 --2- 172.21.15.1:0/3092209449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440beaa30 0x565440bece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d4947640 1 --2- 172.21.15.1:0/3092209449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440bdf850 0x565440bdfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70ce7fc640 1 -- 172.21.15.1:0/119602689 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f70c00cf620 con 0x7f70d0102ef0 2024-09-16T00:40:13.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d5949640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70b80a32c0 0x7f70b80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d5148640 1 -- 172.21.15.1:0/3092209449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440bdf850 msgr2=0x565440bdfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d5148640 1 --2- 172.21.15.1:0/3092209449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440bdf850 0x565440bdfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d5148640 1 -- 172.21.15.1:0/3092209449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440beaa30 msgr2=0x565440bece20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d5148640 1 --2- 172.21.15.1:0/3092209449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440beaa30 0x565440bece20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.311+0000 7f70d5148640 1 -- 172.21.15.1:0/3092209449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654408b5ba0 con 0x565440be0170 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.312+0000 7f70d5949640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70b80a32c0 0x7f70b80a5780 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009610 tx=0x7f70c0004830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.312+0000 7f70d5148640 1 --2- 172.21.15.1:0/3092209449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440be0170 0x565440bea4f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009fc0 tx=0x7f70c800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65af282f86339cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.312+0000 7f70b77fe640 1 -- 172.21.15.1:0/3092209449 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8004d20 con 0x565440be0170 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.312+0000 7f70b77fe640 1 -- 172.21.15.1:0/3092209449 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c8005500 con 0x565440be0170 2024-09-16T00:40:13.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.312+0000 7f70b77fe640 1 -- 172.21.15.1:0/3092209449 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8011a00 con 0x565440be0170 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.313+0000 7f70d9785740 1 -- 172.21.15.1:0/3092209449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440be0170 msgr2=0x565440bea4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.313+0000 7f70d9785740 1 --2- 172.21.15.1:0/3092209449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440be0170 0x565440bea4f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009fc0 tx=0x7f70c800f7f0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.313+0000 7f70d9785740 1 -- 172.21.15.1:0/3092209449 shutdown_connections 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.313+0000 7f70d9785740 1 --2- 172.21.15.1:0/3092209449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440beaa30 0x565440bece20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.313+0000 7f70d9785740 1 --2- 172.21.15.1:0/3092209449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440be0170 0x565440bea4f0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.313+0000 7f70d9785740 1 --2- 172.21.15.1:0/3092209449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440bdf850 0x565440bdfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.314+0000 7f70d9785740 1 -- 172.21.15.1:0/3092209449 >> 172.21.15.1:0/3092209449 conn(0x565440bdbde0 msgr2=0x565440bdc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.314+0000 7f70d9785740 1 -- 172.21.15.1:0/3092209449 shutdown_connections 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.314+0000 7f70d9785740 1 -- 172.21.15.1:0/3092209449 wait complete. 2024-09-16T00:40:13.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.314+0000 7f70d9785740 1 -- start start 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.314+0000 7f70d9785740 1 Processor -- start 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d9785740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440be79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d9785740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 0x565440be7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d9785740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440beaa30 0x565440be8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d4947640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440be79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d4947640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440be79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36134/0 (socket says 172.21.15.1:36134) 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d4947640 1 -- 172.21.15.1:0/3638938160 learned_addr learned my addr 172.21.15.1:0/3638938160 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x565440bed600 con 0x565440bdf850 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565440acc900 con 0x565440beaa30 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x565440bed780 con 0x565440be0170 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d5148640 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 0x565440be7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d9785740 10 client.0 _ll_register_callbacks cb 0x565440aca780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d5148640 1 -- 172.21.15.1:0/3638938160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440beaa30 msgr2=0x565440be8420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d5148640 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440beaa30 0x565440be8420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d5148640 1 -- 172.21.15.1:0/3638938160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 msgr2=0x565440be79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d5148640 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440be79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d5148640 1 -- 172.21.15.1:0/3638938160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565440b0cc50 con 0x565440be0170 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.315+0000 7f70d4947640 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440be79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.316+0000 7f70d5148640 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 0x565440be7ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70c8019000 tx=0x7f70c8009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1881bddd79a99551 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.316+0000 7f70b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.316+0000 7f70b4ff9640 1 -- 172.21.15.1:0/3638938160 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c80058c0 con 0x565440be0170 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.316+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70c8005500 con 0x565440be0170 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.316+0000 7f70b4ff9640 1 -- 172.21.15.1:0/3638938160 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f70c8004a40 con 0x565440be0170 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.316+0000 7f70b4ff9640 1 -- 172.21.15.1:0/3638938160 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70c8004c40 con 0x565440be0170 2024-09-16T00:40:13.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.316+0000 7f70d9785740 10 client.27013 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.316+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565440bfd720 con 0x565440be0170 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.317+0000 7f70b4ff9640 1 -- 172.21.15.1:0/3638938160 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c8005500 con 0x565440be0170 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.317+0000 7f70b4ff9640 1 -- 172.21.15.1:0/3638938160 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f70c8011bb0 con 0x565440be0170 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.317+0000 7f70d9785740 10 client.27013 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.317+0000 7f70d9785740 10 client.27013 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.317+0000 7f70d9785740 20 client.27013 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.317+0000 7f70d9785740 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565440bfeb30 0x565440c00f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.317+0000 7f70d9785740 4 client.27013 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.317+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565440b51890 con 0x565440bfeb30 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.317+0000 7f70d4947640 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565440bfeb30 0x565440c00f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.318+0000 7f70b4ff9640 10 client.27013 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.318+0000 7f70d4947640 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565440bfeb30 0x565440c00f20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa978990db35b317 server_cookie=4e57c3c48e8d7b42 in_seq=0 out_seq=0 2024-09-16T00:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70b4ff9640 1 -- 172.21.15.1:0/3638938160 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565440b51890 con 0x565440bfeb30 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70b4ff9640 10 client.27013 handle_command_reply: tid=0 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 client.27013 shutdown 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 msgr2=0x565440be7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 0x565440be7ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70c8019000 tx=0x7f70c8009d30 comp rx=0 tx=0).stop 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 shutdown_connections 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565440bfeb30 0x565440c00f20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440beaa30 0x565440be8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 0x565440be7ee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 --2- 172.21.15.1:0/3638938160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440be79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 >> 172.21.15.1:0/3638938160 conn(0x565440bdbde0 msgr2=0x565440be1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70b4ff9640 0 client.27013 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 shutdown_connections 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 1 -- 172.21.15.1:0/3638938160 wait complete. 2024-09-16T00:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.319+0000 7f70d9785740 20 client.27013 trim_cache size 0 max 16384 2024-09-16T00:40:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d9785740 1 Processor -- start 2024-09-16T00:40:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d9785740 1 -- start start 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d9785740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440bdf850 0x565440be22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d9785740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 0x565440c06550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d9785740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440beaa30 0x565440c06a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d9785740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01af640 con 0x565440beaa30 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d9785740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01acf10 con 0x565440bdf850 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d9785740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01aad60 con 0x565440be0170 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d4947640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440beaa30 0x565440c06a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d4947640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440beaa30 0x565440c06a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36146/0 (socket says 172.21.15.1:36146) 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d4947640 1 -- 172.21.15.1:0/1074861825 learned_addr learned my addr 172.21.15.1:0/1074861825 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.337+0000 7f70d5148640 1 --2- 172.21.15.1:0/1074861825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 0x565440c06550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70d4947640 1 --2- 172.21.15.1:0/1074861825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440bdf850 0x565440be22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70d4947640 1 -- 172.21.15.1:0/1074861825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 msgr2=0x565440c06550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70d4947640 1 --2- 172.21.15.1:0/1074861825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 0x565440c06550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70d4947640 1 -- 172.21.15.1:0/1074861825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440bdf850 msgr2=0x565440be22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70d4947640 1 --2- 172.21.15.1:0/1074861825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440bdf850 0x565440be22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70d4947640 1 -- 172.21.15.1:0/1074861825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565440be5ef0 con 0x565440beaa30 2024-09-16T00:40:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70d4947640 1 --2- 172.21.15.1:0/1074861825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440beaa30 0x565440c06a90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f70c40314e0 tx=0x7f70c403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a22f9aeb3fd2a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70b6ffd640 1 -- 172.21.15.1:0/1074861825 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4035040 con 0x565440beaa30 2024-09-16T00:40:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70b6ffd640 1 -- 172.21.15.1:0/1074861825 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c4003cb0 con 0x565440beaa30 2024-09-16T00:40:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.338+0000 7f70b6ffd640 1 -- 172.21.15.1:0/1074861825 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c404dac0 con 0x565440beaa30 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 -- 172.21.15.1:0/1074861825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440beaa30 msgr2=0x565440c06a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 --2- 172.21.15.1:0/1074861825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440beaa30 0x565440c06a90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f70c40314e0 tx=0x7f70c403f040 comp rx=0 tx=0).stop 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 -- 172.21.15.1:0/1074861825 shutdown_connections 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 --2- 172.21.15.1:0/1074861825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440beaa30 0x565440c06a90 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 --2- 172.21.15.1:0/1074861825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440be0170 0x565440c06550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 --2- 172.21.15.1:0/1074861825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440bdf850 0x565440be22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 -- 172.21.15.1:0/1074861825 >> 172.21.15.1:0/1074861825 conn(0x565440bdbde0 msgr2=0x565440bdaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 -- 172.21.15.1:0/1074861825 shutdown_connections 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 -- 172.21.15.1:0/1074861825 wait complete. 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 -- start start 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.339+0000 7f70d9785740 1 Processor -- start 2024-09-16T00:40:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d9785740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440bf31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d9785740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440be0170 0x565440bee520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d9785740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440beaa30 0x565440beeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d9785740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01ab3d0 con 0x565440bdf850 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d9785740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01ac840 con 0x565440be0170 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d9785740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01abaa0 con 0x565440beaa30 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d9785740 10 client.0 _ll_register_callbacks cb 0x7f70c8015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d4947640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440bf31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d4947640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440bf31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36156/0 (socket says 172.21.15.1:36156) 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d4947640 1 -- 172.21.15.1:0/703194510 learned_addr learned my addr 172.21.15.1:0/703194510 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d5148640 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440be0170 0x565440bee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.340+0000 7f70d4947640 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440beaa30 0x565440beeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d5148640 1 -- 172.21.15.1:0/703194510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440beaa30 msgr2=0x565440beeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d5148640 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440beaa30 0x565440beeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d5148640 1 -- 172.21.15.1:0/703194510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 msgr2=0x565440bf31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d5148640 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440bf31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d5148640 1 -- 172.21.15.1:0/703194510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565440bef2f0 con 0x565440be0170 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d4947640 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440bf31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d5148640 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440be0170 0x565440bee520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f70c8011f80 tx=0x7f70c8009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619d12ed9818ad86 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70b4ff9640 1 -- 172.21.15.1:0/703194510 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c80058c0 con 0x565440be0170 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d9785740 1 -- 172.21.15.1:0/703194510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70c4003cb0 con 0x565440be0170 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70b4ff9640 1 -- 172.21.15.1:0/703194510 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f70c8022050 con 0x565440be0170 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70b4ff9640 1 -- 172.21.15.1:0/703194510 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70c8040050 con 0x565440be0170 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d9785740 10 client.27275 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.341+0000 7f70d9785740 1 -- 172.21.15.1:0/703194510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565440c695d0 con 0x565440be0170 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.342+0000 7f70b4ff9640 1 -- 172.21.15.1:0/703194510 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c4003cb0 con 0x565440be0170 2024-09-16T00:40:13.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.342+0000 7f70b4ff9640 1 -- 172.21.15.1:0/703194510 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f70c8004de0 con 0x565440be0170 2024-09-16T00:40:13.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.342+0000 7f70d9785740 10 client.27275 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:13.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.342+0000 7f70d9785740 10 client.27275 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:40:13.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.342+0000 7f70d9785740 20 client.27275 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:13.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.342+0000 7f70d9785740 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565440c69db0 0x565440c6c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:13.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.342+0000 7f70d9785740 4 client.27275 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-16T00:40:13.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.342+0000 7f70d9785740 1 -- 172.21.15.1:0/703194510 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x565440b4bbe0 con 0x565440c69db0 2024-09-16T00:40:13.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.343+0000 7f70d4947640 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565440c69db0 0x565440c6c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:13.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.343+0000 7f70d4947640 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565440c69db0 0x565440c6c1d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47590126ab525869 server_cookie=46354e8212968631 in_seq=0 out_seq=0 2024-09-16T00:40:13.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.343+0000 7f70b4ff9640 10 client.27275 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:13.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70b4ff9640 1 -- 172.21.15.1:0/703194510 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565440b4bbe0 con 0x565440c69db0 2024-09-16T00:40:13.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70b4ff9640 10 client.27275 handle_command_reply: tid=0 2024-09-16T00:40:13.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 client.27275 shutdown 2024-09-16T00:40:13.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 -- 172.21.15.1:0/703194510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440be0170 msgr2=0x565440bee520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:13.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440be0170 0x565440bee520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f70c8011f80 tx=0x7f70c8009f70 comp rx=0 tx=0).stop 2024-09-16T00:40:13.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 -- 172.21.15.1:0/703194510 shutdown_connections 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x565440c69db0 0x565440c6c1d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565440beaa30 0x565440beeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565440be0170 0x565440bee520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70b4ff9640 0 client.27275 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 --2- 172.21.15.1:0/703194510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565440bdf850 0x565440bf31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 -- 172.21.15.1:0/703194510 >> 172.21.15.1:0/703194510 conn(0x565440bdbde0 msgr2=0x565440be9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 -- 172.21.15.1:0/703194510 shutdown_connections 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 1 -- 172.21.15.1:0/703194510 wait complete. 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.352+0000 7f70d9785740 20 client.27275 trim_cache size 0 max 16384 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:40:13.355 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 42, 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: "1c7a2282": { 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 42, 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:40:13.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 141.8, 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:13.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:13.358 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 -- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70b80a32c0 msgr2=0x7f70b80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70b80a32c0 0x7f70b80a5780 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009610 tx=0x7f70c0004830 comp rx=0 tx=0).stop 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 -- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102ef0 msgr2=0x7f70d019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102ef0 0x7f70d019ce20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f70c000d750 tx=0x7f70c000dc10 comp rx=0 tx=0).stop 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 -- 172.21.15.1:0/119602689 shutdown_connections 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0102ef0 0x7f70d019ce20 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70b80a32c0 0x7f70b80a5780 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d0102570 0x7f70d0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 --2- 172.21.15.1:0/119602689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0100e20 0x7f70d0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 -- 172.21.15.1:0/119602689 >> 172.21.15.1:0/119602689 conn(0x7f70d00fc7d0 msgr2=0x7f70d006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 -- 172.21.15.1:0/119602689 shutdown_connections 2024-09-16T00:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:13.353+0000 7f70d73ac640 1 -- 172.21.15.1:0/119602689 wait complete. 2024-09-16T00:40:14.090 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1c7a2282', quiesce took 5.4 seconds. Will release after: 16.450000000000003 2024-09-16T00:40:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.345 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:14 smithi001 ceph-mon[30934]: pgmap v705: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:40:14.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:14 smithi019 ceph-mon[35186]: pgmap v705: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:40:14.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:14 smithi153 ceph-mon[35538]: pgmap v705: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:40:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.714+0000 7f1560509640 1 -- 172.21.15.1:0/2652020142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558106a80 msgr2=0x7f1558106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.714+0000 7f1560509640 1 --2- 172.21.15.1:0/2652020142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558106a80 0x7f1558106e60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f1554009930 tx=0x7f155402a420 comp rx=0 tx=0).stop 2024-09-16T00:40:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.714+0000 7f1560509640 1 -- 172.21.15.1:0/2652020142 shutdown_connections 2024-09-16T00:40:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.714+0000 7f1560509640 1 --2- 172.21.15.1:0/2652020142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1558101420 0x7f155810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.714+0000 7f1560509640 1 --2- 172.21.15.1:0/2652020142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558100a80 0x7f1558100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.714+0000 7f1560509640 1 --2- 172.21.15.1:0/2652020142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558106a80 0x7f1558106e60 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.714+0000 7f1560509640 1 -- 172.21.15.1:0/2652020142 >> 172.21.15.1:0/2652020142 conn(0x7f15580fc7a0 msgr2=0x7f15580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.714+0000 7f1560509640 1 -- 172.21.15.1:0/2652020142 shutdown_connections 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.714+0000 7f1560509640 1 -- 172.21.15.1:0/2652020142 wait complete. 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f1560509640 1 Processor -- start 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f1560509640 1 -- start start 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f1560509640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558100a80 0x7f1558198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f1560509640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1558101420 0x7f1558198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f1560509640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558106a80 0x7f155819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f1560509640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1558110e80 con 0x7f1558100a80 2024-09-16T00:40:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f1560509640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1558110d00 con 0x7f1558101420 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f1560509640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1558111000 con 0x7f1558106a80 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f155e2a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558100a80 0x7f1558198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f155e2a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558100a80 0x7f1558198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36180/0 (socket says 172.21.15.1:36180) 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f155e2a5640 1 -- 172.21.15.1:0/3380977898 learned_addr learned my addr 172.21.15.1:0/3380977898 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.715+0000 7f155daa4640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1558101420 0x7f1558198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f155eaa6640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558106a80 0x7f155819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f155e2a5640 1 -- 172.21.15.1:0/3380977898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558106a80 msgr2=0x7f155819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f155e2a5640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558106a80 0x7f155819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f155e2a5640 1 -- 172.21.15.1:0/3380977898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1558101420 msgr2=0x7f1558198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f155e2a5640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1558101420 0x7f1558198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f155e2a5640 1 -- 172.21.15.1:0/3380977898 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155819d400 con 0x7f1558100a80 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f155e2a5640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558100a80 0x7f1558198450 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1554009230 tx=0x7f1554004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f154f7fe640 1 -- 172.21.15.1:0/3380977898 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155403f030 con 0x7f1558100a80 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f154f7fe640 1 -- 172.21.15.1:0/3380977898 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1554003ad0 con 0x7f1558100a80 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f1560509640 1 -- 172.21.15.1:0/3380977898 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155819d690 con 0x7f1558100a80 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f154f7fe640 1 -- 172.21.15.1:0/3380977898 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554035070 con 0x7f1558100a80 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.716+0000 7f1560509640 1 -- 172.21.15.1:0/3380977898 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15581a4fd0 con 0x7f1558100a80 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.718+0000 7f154f7fe640 1 -- 172.21.15.1:0/3380977898 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f1554042030 con 0x7f1558100a80 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f15628e2740 1 Processor -- start 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f15628e2740 1 -- start start 2024-09-16T00:40:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f15628e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f15628e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f15628e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f15628e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d1ef132f0 con 0x562d1efb8370 2024-09-16T00:40:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f15628e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562d1eebacf0 con 0x562d1efaeb40 2024-09-16T00:40:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f15628e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562d1ef0f2b0 con 0x562d1efad5d0 2024-09-16T00:40:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f154f7fe640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15440a3240 0x7f15440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f155daa4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f155daa4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efba760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36192/0 (socket says 172.21.15.1:36192) 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f155daa4640 1 -- 172.21.15.1:0/3225219062 learned_addr learned my addr 172.21.15.1:0/3225219062 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.719+0000 7f155eaa6640 1 --2- 172.21.15.1:0/3225219062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f155e2a5640 1 --2- 172.21.15.1:0/3225219062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f155eaa6640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15440a3240 0x7f15440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f154f7fe640 1 -- 172.21.15.1:0/3380977898 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f15540efe50 con 0x7f1558100a80 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f155daa4640 1 -- 172.21.15.1:0/3225219062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 msgr2=0x562d1efada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f155daa4640 1 --2- 172.21.15.1:0/3225219062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f155daa4640 1 -- 172.21.15.1:0/3225219062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 msgr2=0x562d1efad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f155daa4640 1 --2- 172.21.15.1:0/3225219062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f155daa4640 1 -- 172.21.15.1:0/3225219062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1ee9f3a0 con 0x562d1efb8370 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f155daa4640 1 --2- 172.21.15.1:0/3225219062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efba760 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f154800b740 tx=0x7f154800bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4403eff3ad25e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f155eaa6640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15440a3240 0x7f15440a5700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f155000afd0 tx=0x7f155000a4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f154cff9640 1 -- 172.21.15.1:0/3225219062 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548003dd0 con 0x562d1efb8370 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f154cff9640 1 -- 172.21.15.1:0/3225219062 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1548015070 con 0x562d1efb8370 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.720+0000 7f154cff9640 1 -- 172.21.15.1:0/3225219062 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548004e00 con 0x562d1efb8370 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.721+0000 7f15628e2740 1 -- 172.21.15.1:0/3225219062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 msgr2=0x562d1efba760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.721+0000 7f15628e2740 1 --2- 172.21.15.1:0/3225219062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efba760 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f154800b740 tx=0x7f154800bc00 comp rx=0 tx=0).stop 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.721+0000 7f15628e2740 1 -- 172.21.15.1:0/3225219062 shutdown_connections 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.721+0000 7f15628e2740 1 --2- 172.21.15.1:0/3225219062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efba760 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.721+0000 7f15628e2740 1 --2- 172.21.15.1:0/3225219062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.721+0000 7f15628e2740 1 --2- 172.21.15.1:0/3225219062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.721+0000 7f15628e2740 1 -- 172.21.15.1:0/3225219062 >> 172.21.15.1:0/3225219062 conn(0x562d1ef76310 msgr2=0x562d1efa9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.721+0000 7f15628e2740 1 -- 172.21.15.1:0/3225219062 shutdown_connections 2024-09-16T00:40:14.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.721+0000 7f15628e2740 1 -- 172.21.15.1:0/3225219062 wait complete. 2024-09-16T00:40:14.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f15628e2740 1 -- start start 2024-09-16T00:40:14.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f15628e2740 1 Processor -- start 2024-09-16T00:40:14.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f15628e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efb7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f15628e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efb5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f15628e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f15628e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d1efbb070 con 0x562d1efb8370 2024-09-16T00:40:14.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f15628e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562d1eece060 con 0x562d1efaeb40 2024-09-16T00:40:14.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f15628e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562d1efbb1f0 con 0x562d1efad5d0 2024-09-16T00:40:14.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f15628e2740 10 client.0 _ll_register_callbacks cb 0x562d1ec218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:14.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155eaa6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155eaa6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36200/0 (socket says 172.21.15.1:36200) 2024-09-16T00:40:14.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155eaa6640 1 -- 172.21.15.1:0/339287417 learned_addr learned my addr 172.21.15.1:0/339287417 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155daa4640 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efb5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155e2a5640 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efb7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155eaa6640 1 -- 172.21.15.1:0/339287417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 msgr2=0x562d1efb7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155eaa6640 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efb7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155eaa6640 1 -- 172.21.15.1:0/339287417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 msgr2=0x562d1efb5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155eaa6640 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efb5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155eaa6640 1 -- 172.21.15.1:0/339287417 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1ec843c0 con 0x562d1efb8370 2024-09-16T00:40:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.722+0000 7f155e2a5640 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efb7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f155eaa6640 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb5650 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1550011e00 tx=0x7f1550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7045dcac35ae4d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f1541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f1541ffb640 1 -- 172.21.15.1:0/339287417 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550023070 con 0x562d1efb8370 2024-09-16T00:40:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f1541ffb640 1 -- 172.21.15.1:0/339287417 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1550004d20 con 0x562d1efb8370 2024-09-16T00:40:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f15628e2740 1 -- 172.21.15.1:0/339287417 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1548015070 con 0x562d1efb8370 2024-09-16T00:40:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f1541ffb640 1 -- 172.21.15.1:0/339287417 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f155002f050 con 0x562d1efb8370 2024-09-16T00:40:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f15628e2740 10 client.18291 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f15628e2740 1 -- 172.21.15.1:0/339287417 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d1efc6ba0 con 0x562d1efb8370 2024-09-16T00:40:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f1541ffb640 1 -- 172.21.15.1:0/339287417 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1548015070 con 0x562d1efb8370 2024-09-16T00:40:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f1541ffb640 1 -- 172.21.15.1:0/339287417 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1550004620 con 0x562d1efb8370 2024-09-16T00:40:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f15628e2740 10 client.18291 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f15628e2740 10 client.18291 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:40:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f15628e2740 20 client.18291 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f15628e2740 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562d1efc7fb0 0x562d1efca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f15628e2740 4 client.18291 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.723+0000 7f15628e2740 1 -- 172.21.15.1:0/339287417 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d1ef20860 con 0x562d1efc7fb0 2024-09-16T00:40:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.724+0000 7f155e2a5640 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562d1efc7fb0 0x562d1efca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.724+0000 7f1541ffb640 10 client.18291 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:40:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.724+0000 7f155e2a5640 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562d1efc7fb0 0x562d1efca420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56bf135c5a254dd1 server_cookie=3c28f7c3daa1abaa in_seq=0 out_seq=0 2024-09-16T00:40:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f1541ffb640 1 -- 172.21.15.1:0/339287417 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d1ef20860 con 0x562d1efc7fb0 2024-09-16T00:40:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f1541ffb640 10 client.18291 handle_command_reply: tid=0 2024-09-16T00:40:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 client.18291 shutdown 2024-09-16T00:40:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 -- 172.21.15.1:0/339287417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 msgr2=0x562d1efb5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb5650 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1550011e00 tx=0x7f1550004a40 comp rx=0 tx=0).stop 2024-09-16T00:40:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 -- 172.21.15.1:0/339287417 shutdown_connections 2024-09-16T00:40:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb5650 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562d1efc7fb0 0x562d1efca420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efb5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 --2- 172.21.15.1:0/339287417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efb7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 -- 172.21.15.1:0/339287417 >> 172.21.15.1:0/339287417 conn(0x562d1ef76310 msgr2=0x562d1efaf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f1541ffb640 0 client.18291 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:40:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 -- 172.21.15.1:0/339287417 shutdown_connections 2024-09-16T00:40:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 1 -- 172.21.15.1:0/339287417 wait complete. 2024-09-16T00:40:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.725+0000 7f15628e2740 20 client.18291 trim_cache size 0 max 16384 2024-09-16T00:40:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.738+0000 7f15628e2740 1 Processor -- start 2024-09-16T00:40:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.738+0000 7f15628e2740 1 -- start start 2024-09-16T00:40:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f15628e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efa9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f15628e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efc4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f15628e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f15628e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15581b16b0 con 0x562d1efb8370 2024-09-16T00:40:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f15628e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15581aef80 con 0x562d1efaeb40 2024-09-16T00:40:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f15628e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15581acdd0 con 0x562d1efad5d0 2024-09-16T00:40:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155eaa6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155eaa6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36204/0 (socket says 172.21.15.1:36204) 2024-09-16T00:40:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155eaa6640 1 -- 172.21.15.1:0/2707931751 learned_addr learned my addr 172.21.15.1:0/2707931751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155e2a5640 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efa9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155daa4640 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efc4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155eaa6640 1 -- 172.21.15.1:0/2707931751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 msgr2=0x562d1efa9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155eaa6640 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efa9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155eaa6640 1 -- 172.21.15.1:0/2707931751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 msgr2=0x562d1efc4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155eaa6640 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efc4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155eaa6640 1 -- 172.21.15.1:0/2707931751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1eface80 con 0x562d1efb8370 2024-09-16T00:40:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155daa4640 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efc4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f155eaa6640 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb0120 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f1550011820 tx=0x7f1550012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93160143c7048c13 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.739+0000 7f1543fff640 1 -- 172.21.15.1:0/2707931751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550037070 con 0x562d1efb8370 2024-09-16T00:40:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f1543fff640 1 -- 172.21.15.1:0/2707931751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f155000f810 con 0x562d1efb8370 2024-09-16T00:40:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f1543fff640 1 -- 172.21.15.1:0/2707931751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550023070 con 0x562d1efb8370 2024-09-16T00:40:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 -- 172.21.15.1:0/2707931751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 msgr2=0x562d1efb0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb0120 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f1550011820 tx=0x7f1550012f40 comp rx=0 tx=0).stop 2024-09-16T00:40:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 -- 172.21.15.1:0/2707931751 shutdown_connections 2024-09-16T00:40:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efb8370 0x562d1efb0120 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efc4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 --2- 172.21.15.1:0/2707931751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efad5d0 0x562d1efa9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 -- 172.21.15.1:0/2707931751 >> 172.21.15.1:0/2707931751 conn(0x562d1ef76310 msgr2=0x562d1efa9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 -- 172.21.15.1:0/2707931751 shutdown_connections 2024-09-16T00:40:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 -- 172.21.15.1:0/2707931751 wait complete. 2024-09-16T00:40:14.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.740+0000 7f15628e2740 1 -- start start 2024-09-16T00:40:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f15628e2740 1 Processor -- start 2024-09-16T00:40:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f15628e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efad5d0 0x562d1efc0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f15628e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efc0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f15628e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efb8370 0x562d1efb1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f15628e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15581ad440 con 0x562d1efad5d0 2024-09-16T00:40:14.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f15628e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15581ae8b0 con 0x562d1efaeb40 2024-09-16T00:40:14.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f15628e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15581adb10 con 0x562d1efb8370 2024-09-16T00:40:14.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f15628e2740 10 client.0 _ll_register_callbacks cb 0x7f155000ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:14.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155e2a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efad5d0 0x562d1efc0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155e2a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efad5d0 0x562d1efc0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36220/0 (socket says 172.21.15.1:36220) 2024-09-16T00:40:14.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155e2a5640 1 -- 172.21.15.1:0/1423531609 learned_addr learned my addr 172.21.15.1:0/1423531609 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:14.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155daa4640 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efc0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155eaa6640 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efb8370 0x562d1efb1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155e2a5640 1 -- 172.21.15.1:0/1423531609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efb8370 msgr2=0x562d1efb1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155e2a5640 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efb8370 0x562d1efb1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155e2a5640 1 -- 172.21.15.1:0/1423531609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 msgr2=0x562d1efc0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155e2a5640 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efc0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155e2a5640 1 -- 172.21.15.1:0/1423531609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1efb2450 con 0x562d1efad5d0 2024-09-16T00:40:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.741+0000 7f155eaa6640 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efb8370 0x562d1efb1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f155e2a5640 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efad5d0 0x562d1efc0810 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f15540bc050 tx=0x7f1554036450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65311b02e105c643 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f1541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f1541ffb640 1 -- 172.21.15.1:0/1423531609 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554049040 con 0x562d1efad5d0 2024-09-16T00:40:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f1541ffb640 1 -- 172.21.15.1:0/1423531609 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f155402fdf0 con 0x562d1efad5d0 2024-09-16T00:40:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f15628e2740 1 -- 172.21.15.1:0/1423531609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f155000f810 con 0x562d1efad5d0 2024-09-16T00:40:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f1541ffb640 1 -- 172.21.15.1:0/1423531609 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f15540c4c80 con 0x562d1efad5d0 2024-09-16T00:40:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f15628e2740 10 client.18297 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f15628e2740 1 -- 172.21.15.1:0/1423531609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d1f0328f0 con 0x562d1efad5d0 2024-09-16T00:40:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f1541ffb640 1 -- 172.21.15.1:0/1423531609 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f155000f810 con 0x562d1efad5d0 2024-09-16T00:40:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f1541ffb640 1 -- 172.21.15.1:0/1423531609 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f15540b8730 con 0x562d1efad5d0 2024-09-16T00:40:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f15628e2740 10 client.18297 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f15628e2740 10 client.18297 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:40:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f15628e2740 20 client.18297 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f15628e2740 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562d1f033010 0x562d1f035400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f15628e2740 4 client.18297 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:40:14.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.742+0000 7f15628e2740 1 -- 172.21.15.1:0/1423531609 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d1ef1ad70 con 0x562d1f033010 2024-09-16T00:40:14.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.743+0000 7f155daa4640 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562d1f033010 0x562d1f035400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.743+0000 7f155daa4640 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562d1f033010 0x562d1f035400 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee6f7c11f8bad9ea server_cookie=b07a6f9b2a835604 in_seq=0 out_seq=0 2024-09-16T00:40:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.743+0000 7f1541ffb640 10 client.18297 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:40:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.743+0000 7f1541ffb640 1 -- 172.21.15.1:0/1423531609 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d1ef1ad70 con 0x562d1f033010 2024-09-16T00:40:14.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.743+0000 7f1541ffb640 10 client.18297 handle_command_reply: tid=0 2024-09-16T00:40:14.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 client.18297 shutdown 2024-09-16T00:40:14.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 -- 172.21.15.1:0/1423531609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efad5d0 msgr2=0x562d1efc0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efad5d0 0x562d1efc0810 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f15540bc050 tx=0x7f1554036450 comp rx=0 tx=0).stop 2024-09-16T00:40:14.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 -- 172.21.15.1:0/1423531609 shutdown_connections 2024-09-16T00:40:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d1efb8370 0x562d1efb1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562d1f033010 0x562d1f035400 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d1efaeb40 0x562d1efc0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 --2- 172.21.15.1:0/1423531609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d1efad5d0 0x562d1efc0810 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1541ffb640 0 client.18297 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:40:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 -- 172.21.15.1:0/1423531609 >> 172.21.15.1:0/1423531609 conn(0x562d1ef76310 msgr2=0x562d1efb68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 -- 172.21.15.1:0/1423531609 shutdown_connections 2024-09-16T00:40:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 1 -- 172.21.15.1:0/1423531609 wait complete. 2024-09-16T00:40:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f15628e2740 20 client.18297 trim_cache size 0 max 16384 2024-09-16T00:40:14.778 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:40:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 -- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15440a3240 msgr2=0x7f15440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15440a3240 0x7f15440a5700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f155000afd0 tx=0x7f155000a4d0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 -- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558100a80 msgr2=0x7f1558198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558100a80 0x7f1558198450 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1554009230 tx=0x7f1554004270 comp rx=0 tx=0).stop 2024-09-16T00:40:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 -- 172.21.15.1:0/3380977898 shutdown_connections 2024-09-16T00:40:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f15440a3240 0x7f15440a5700 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1558106a80 0x7f155819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1558101420 0x7f1558198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 --2- 172.21.15.1:0/3380977898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1558100a80 0x7f1558198450 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.744+0000 7f1560509640 1 -- 172.21.15.1:0/3380977898 >> 172.21.15.1:0/3380977898 conn(0x7f15580fc7a0 msgr2=0x7f155806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.745+0000 7f1560509640 1 -- 172.21.15.1:0/3380977898 shutdown_connections 2024-09-16T00:40:14.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:14.745+0000 7f1560509640 1 -- 172.21.15.1:0/3380977898 wait complete. 2024-09-16T00:40:15.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:40:15.829 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 -- 172.21.15.1:0/2502603777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a78102d30 msgr2=0x7f9a78103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/2502603777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a78102d30 0x7f9a78103110 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009930 tx=0x7f9a7002a420 comp rx=0 tx=0).stop 2024-09-16T00:40:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 -- 172.21.15.1:0/2502603777 shutdown_connections 2024-09-16T00:40:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/2502603777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a781082a0 0x7f9a7810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/2502603777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a781036e0 0x7f9a78107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/2502603777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a78102d30 0x7f9a78103110 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 -- 172.21.15.1:0/2502603777 >> 172.21.15.1:0/2502603777 conn(0x7f9a780fc7d0 msgr2=0x7f9a780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 -- 172.21.15.1:0/2502603777 shutdown_connections 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 -- 172.21.15.1:0/2502603777 wait complete. 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 Processor -- start 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.350+0000 7f9a7eb40640 1 -- start start 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7eb40640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a78102d30 0x7f9a781985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7eb40640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a781036e0 0x7f9a78198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7eb40640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a781082a0 0x7f9a7819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7eb40640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781120a0 con 0x7f9a781082a0 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7eb40640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78111f20 con 0x7f9a78102d30 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7eb40640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78112220 con 0x7f9a781036e0 2024-09-16T00:40:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7d0dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a781082a0 0x7f9a7819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7d0dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a781082a0 0x7f9a7819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36230/0 (socket says 172.21.15.1:36230) 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7d0dd640 1 -- 172.21.15.1:0/4107088802 learned_addr learned my addr 172.21.15.1:0/4107088802 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a77fff640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a781036e0 0x7f9a78198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a78102d30 0x7f9a781985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7d0dd640 1 -- 172.21.15.1:0/4107088802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a781036e0 msgr2=0x7f9a78198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7d0dd640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a781036e0 0x7f9a78198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7d0dd640 1 -- 172.21.15.1:0/4107088802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a78102d30 msgr2=0x7f9a781985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7d0dd640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a78102d30 0x7f9a781985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7d0dd640 1 -- 172.21.15.1:0/4107088802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7819d570 con 0x7f9a781082a0 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a78102d30 0x7f9a781985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7d0dd640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a781082a0 0x7f9a7819ce90 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800eb10 tx=0x7f9a6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a75ffb640 1 -- 172.21.15.1:0/4107088802 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68004b20 con 0x7f9a781082a0 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a75ffb640 1 -- 172.21.15.1:0/4107088802 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a68004cc0 con 0x7f9a781082a0 2024-09-16T00:40:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a7eb40640 1 -- 172.21.15.1:0/4107088802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7819d860 con 0x7f9a781082a0 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.351+0000 7f9a75ffb640 1 -- 172.21.15.1:0/4107088802 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68010760 con 0x7f9a781082a0 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.352+0000 7f9a7eb40640 1 -- 172.21.15.1:0/4107088802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a781a5140 con 0x7f9a781082a0 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.354+0000 7f9a75ffb640 1 -- 172.21.15.1:0/4107088802 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9a68005020 con 0x7f9a781082a0 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.354+0000 7f9a80f19740 1 Processor -- start 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.354+0000 7f9a80f19740 1 -- start start 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.354+0000 7f9a75ffb640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a640a32c0 0x7f9a640a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd7795090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd7795a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a80f19740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560cd76fb2f0 con 0x560cd77955d0 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a80f19740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560cd76a2cf0 con 0x560cd7796b40 2024-09-16T00:40:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a80f19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560cd76f72b0 con 0x560cd77a4780 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a77fff640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a640a32c0 0x7f9a640a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7c8dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd7795090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7c8dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd7795090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:42824/0 (socket says 172.21.15.1:42824) 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd7795a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7c8dc640 1 -- 172.21.15.1:0/1262831328 learned_addr learned my addr 172.21.15.1:0/1262831328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a75ffb640 1 -- 172.21.15.1:0/4107088802 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9a680d03d0 con 0x7f9a781082a0 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7d0dd640 1 --2- 172.21.15.1:0/1262831328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7c8dc640 1 -- 172.21.15.1:0/1262831328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 msgr2=0x560cd77a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/1262831328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7c8dc640 1 -- 172.21.15.1:0/1262831328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 msgr2=0x560cd7795a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/1262831328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd7795a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7c8dc640 1 -- 172.21.15.1:0/1262831328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd76873a0 con 0x560cd7796b40 2024-09-16T00:40:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a77fff640 1 --2- 172.21.15.1:0/1262831328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd7795a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a77fff640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a640a32c0 0x7f9a640a5780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9a780fee00 tx=0x7f9a60009b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.355+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/1262831328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd7795090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009930 tx=0x7f9a70005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ced1c21921aebb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a6effd640 1 -- 172.21.15.1:0/1262831328 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7003f070 con 0x560cd7796b40 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a6effd640 1 -- 172.21.15.1:0/1262831328 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a70004030 con 0x560cd7796b40 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a6effd640 1 -- 172.21.15.1:0/1262831328 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70035070 con 0x560cd7796b40 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a80f19740 1 -- 172.21.15.1:0/1262831328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 msgr2=0x560cd7795090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a80f19740 1 --2- 172.21.15.1:0/1262831328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd7795090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009930 tx=0x7f9a70005e60 comp rx=0 tx=0).stop 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a80f19740 1 -- 172.21.15.1:0/1262831328 shutdown_connections 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a80f19740 1 --2- 172.21.15.1:0/1262831328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a80f19740 1 --2- 172.21.15.1:0/1262831328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd7795a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a80f19740 1 --2- 172.21.15.1:0/1262831328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd7795090 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a80f19740 1 -- 172.21.15.1:0/1262831328 >> 172.21.15.1:0/1262831328 conn(0x560cd775e310 msgr2=0x560cd7791c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a80f19740 1 -- 172.21.15.1:0/1262831328 shutdown_connections 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.356+0000 7f9a80f19740 1 -- 172.21.15.1:0/1262831328 wait complete. 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a80f19740 1 -- start start 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a80f19740 1 Processor -- start 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd77a4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd77a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a80f19740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560cd77a7420 con 0x560cd77955d0 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a80f19740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560cd76b6060 con 0x560cd7796b40 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a80f19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560cd77a75a0 con 0x560cd77a4780 2024-09-16T00:40:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a80f19740 10 client.0 _ll_register_callbacks cb 0x560cd74098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.357+0000 7f9a7c8dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd77a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a7c8dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd77a4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36244/0 (socket says 172.21.15.1:36244) 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a7c8dc640 1 -- 172.21.15.1:0/2013888168 learned_addr learned my addr 172.21.15.1:0/2013888168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a77fff640 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd77a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a7d0dd640 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a7c8dc640 1 -- 172.21.15.1:0/2013888168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 msgr2=0x560cd77a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a7c8dc640 1 -- 172.21.15.1:0/2013888168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 msgr2=0x560cd77a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd77a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a7c8dc640 1 -- 172.21.15.1:0/2013888168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd746c3c0 con 0x560cd77955d0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd77a4110 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9a700045b0 tx=0x7f9a700050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a716b4f2eb31042 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a57fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a57fff640 1 -- 172.21.15.1:0/2013888168 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70035070 con 0x560cd77955d0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a57fff640 1 -- 172.21.15.1:0/2013888168 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9a700315b0 con 0x560cd77955d0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a80f19740 1 -- 172.21.15.1:0/2013888168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a70004030 con 0x560cd77955d0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a57fff640 1 -- 172.21.15.1:0/2013888168 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a70055050 con 0x560cd77955d0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a80f19740 10 client.18309 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.358+0000 7f9a80f19740 1 -- 172.21.15.1:0/2013888168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560cd77b2e90 con 0x560cd77955d0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.359+0000 7f9a57fff640 1 -- 172.21.15.1:0/2013888168 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a70004030 con 0x560cd77955d0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.359+0000 7f9a57fff640 1 -- 172.21.15.1:0/2013888168 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9a700035f0 con 0x560cd77955d0 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.359+0000 7f9a80f19740 10 client.18309 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.359+0000 7f9a80f19740 10 client.18309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.359+0000 7f9a80f19740 20 client.18309 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.359+0000 7f9a80f19740 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560cd77b41e0 0x560cd77b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.359+0000 7f9a80f19740 4 client.18309 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.359+0000 7f9a80f19740 1 -- 172.21.15.1:0/2013888168 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cd7708860 con 0x560cd77b41e0 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.359+0000 7f9a77fff640 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560cd77b41e0 0x560cd77b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.360+0000 7f9a57fff640 10 client.18309 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.360+0000 7f9a77fff640 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560cd77b41e0 0x560cd77b65d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21d0ed632fdeed9a server_cookie=6537f670e54a7e16 in_seq=0 out_seq=0 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.360+0000 7f9a57fff640 1 -- 172.21.15.1:0/2013888168 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560cd7708860 con 0x560cd77b41e0 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.360+0000 7f9a57fff640 10 client.18309 handle_command_reply: tid=0 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.360+0000 7f9a80f19740 1 client.18309 shutdown 2024-09-16T00:40:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 -- 172.21.15.1:0/2013888168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 msgr2=0x560cd77a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd77a4110 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9a700045b0 tx=0x7f9a700050e0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 -- 172.21.15.1:0/2013888168 shutdown_connections 2024-09-16T00:40:16.394 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:40:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd7796b40 0x560cd77a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560cd77b41e0 0x560cd77b65d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 --2- 172.21.15.1:0/2013888168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd77955d0 0x560cd77a4110 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a57fff640 0 client.18309 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:40:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 -- 172.21.15.1:0/2013888168 >> 172.21.15.1:0/2013888168 conn(0x560cd775e310 msgr2=0x560cd7797960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 -- 172.21.15.1:0/2013888168 shutdown_connections 2024-09-16T00:40:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 1 -- 172.21.15.1:0/2013888168 wait complete. 2024-09-16T00:40:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.361+0000 7f9a80f19740 20 client.18309 trim_cache size 0 max 16384 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.374+0000 7f9a80f19740 1 Processor -- start 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.374+0000 7f9a80f19740 1 -- start start 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.374+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.374+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.374+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.374+0000 7f9a80f19740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781af4a0 con 0x560cd7796b40 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.374+0000 7f9a80f19740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781acd70 con 0x560cd77955d0 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.374+0000 7f9a80f19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781aabc0 con 0x560cd77a4780 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.374+0000 7f9a77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77bbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36254/0 (socket says 172.21.15.1:36254) 2024-09-16T00:40:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a77fff640 1 -- 172.21.15.1:0/1577856439 learned_addr learned my addr 172.21.15.1:0/1577856439 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a77fff640 1 -- 172.21.15.1:0/1577856439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 msgr2=0x560cd77bc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a7d0dd640 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a77fff640 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77bc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a77fff640 1 -- 172.21.15.1:0/1577856439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 msgr2=0x560cd77b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a77fff640 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a77fff640 1 -- 172.21.15.1:0/1577856439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd7791510 con 0x560cd7796b40 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a77fff640 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77bbb70 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x560cd7792ac0 tx=0x7f9a60021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2021a2dc81883d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a6e7fc640 1 -- 172.21.15.1:0/1577856439 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6000fea0 con 0x560cd7796b40 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a6e7fc640 1 -- 172.21.15.1:0/1577856439 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a6002e070 con 0x560cd7796b40 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.375+0000 7f9a6e7fc640 1 -- 172.21.15.1:0/1577856439 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60004a90 con 0x560cd7796b40 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 -- 172.21.15.1:0/1577856439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 msgr2=0x560cd77bbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77bbb70 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x560cd7792ac0 tx=0x7f9a60021000 comp rx=0 tx=0).stop 2024-09-16T00:40:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 -- 172.21.15.1:0/1577856439 shutdown_connections 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd77bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77bbb70 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 --2- 172.21.15.1:0/1577856439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 -- 172.21.15.1:0/1577856439 >> 172.21.15.1:0/1577856439 conn(0x560cd775e310 msgr2=0x560cd779c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 -- 172.21.15.1:0/1577856439 shutdown_connections 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 -- 172.21.15.1:0/1577856439 wait complete. 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 -- start start 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.376+0000 7f9a80f19740 1 Processor -- start 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a80f19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd781db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a80f19740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781ab230 con 0x560cd7796b40 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a80f19740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781ac6a0 con 0x560cd77955d0 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a80f19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781ab900 con 0x560cd77a4780 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a80f19740 10 client.0 _ll_register_callbacks cb 0x7f9a600114c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77a8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36270/0 (socket says 172.21.15.1:36270) 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a77fff640 1 -- 172.21.15.1:0/456334846 learned_addr learned my addr 172.21.15.1:0/456334846 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a7d0dd640 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd781db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a77fff640 1 -- 172.21.15.1:0/456334846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 msgr2=0x560cd781db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a77fff640 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd781db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a77fff640 1 -- 172.21.15.1:0/456334846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 msgr2=0x560cd77a8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a77fff640 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77a8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a77fff640 1 -- 172.21.15.1:0/456334846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd781e080 con 0x560cd7796b40 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a7d0dd640 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd781db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a77fff640 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77a8ed0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f9a60004700 tx=0x7f9a60004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce1680c11699a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.377+0000 7f9a57fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a57fff640 1 -- 172.21.15.1:0/456334846 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60011280 con 0x560cd7796b40 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a57fff640 1 -- 172.21.15.1:0/456334846 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9a600035f0 con 0x560cd7796b40 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a80f19740 1 -- 172.21.15.1:0/456334846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a6002e070 con 0x560cd7796b40 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a57fff640 1 -- 172.21.15.1:0/456334846 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a60045050 con 0x560cd7796b40 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a80f19740 10 client.18315 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a80f19740 1 -- 172.21.15.1:0/456334846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560cd781f190 con 0x560cd7796b40 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a57fff640 1 -- 172.21.15.1:0/456334846 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a6002e070 con 0x560cd7796b40 2024-09-16T00:40:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a57fff640 1 -- 172.21.15.1:0/456334846 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9a6002b820 con 0x560cd7796b40 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a80f19740 10 client.18315 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a80f19740 10 client.18315 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a80f19740 20 client.18315 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a80f19740 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560cd781f820 0x560cd7821c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a80f19740 4 client.18315 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.378+0000 7f9a80f19740 1 -- 172.21.15.1:0/456334846 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560cd7702d70 con 0x560cd781f820 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560cd781f820 0x560cd7821c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a7c8dc640 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560cd781f820 0x560cd7821c40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6868b925c50568ae server_cookie=f8edbfb21dd59293 in_seq=0 out_seq=0 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a57fff640 10 client.18315 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a57fff640 1 -- 172.21.15.1:0/456334846 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560cd7702d70 con 0x560cd781f820 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a57fff640 10 client.18315 handle_command_reply: tid=0 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a80f19740 1 client.18315 shutdown 2024-09-16T00:40:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a80f19740 1 -- 172.21.15.1:0/456334846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 msgr2=0x560cd77a8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a80f19740 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77a8ed0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f9a60004700 tx=0x7f9a60004500 comp rx=0 tx=0).stop 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a80f19740 1 -- 172.21.15.1:0/456334846 shutdown_connections 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a80f19740 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd77a4780 0x560cd781db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a80f19740 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x560cd781f820 0x560cd7821c40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a80f19740 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560cd7796b40 0x560cd77a8ed0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a80f19740 1 --2- 172.21.15.1:0/456334846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cd77955d0 0x560cd77a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a57fff640 0 client.18315 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.379+0000 7f9a80f19740 1 -- 172.21.15.1:0/456334846 >> 172.21.15.1:0/456334846 conn(0x560cd775e310 msgr2=0x560cd77904e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a80f19740 1 -- 172.21.15.1:0/456334846 shutdown_connections 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a80f19740 1 -- 172.21.15.1:0/456334846 wait complete. 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a80f19740 20 client.18315 trim_cache size 0 max 16384 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 -- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a640a32c0 msgr2=0x7f9a640a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a640a32c0 0x7f9a640a5780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9a780fee00 tx=0x7f9a60009b60 comp rx=0 tx=0).stop 2024-09-16T00:40:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 -- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a781082a0 msgr2=0x7f9a7819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a781082a0 0x7f9a7819ce90 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800eb10 tx=0x7f9a6800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 -- 172.21.15.1:0/4107088802 shutdown_connections 2024-09-16T00:40:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a781082a0 0x7f9a7819ce90 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9a640a32c0 0x7f9a640a5780 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a781036e0 0x7f9a78198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 --2- 172.21.15.1:0/4107088802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a78102d30 0x7f9a781985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 -- 172.21.15.1:0/4107088802 >> 172.21.15.1:0/4107088802 conn(0x7f9a780fc7d0 msgr2=0x7f9a78105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 -- 172.21.15.1:0/4107088802 shutdown_connections 2024-09-16T00:40:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:16.380+0000 7f9a7eb40640 1 -- 172.21.15.1:0/4107088802 wait complete. 2024-09-16T00:40:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:16 smithi019 ceph-mon[35186]: pgmap v706: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-16T00:40:16.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:16 smithi001 ceph-mon[30934]: pgmap v706: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-16T00:40:16.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:16 smithi153 ceph-mon[35538]: pgmap v706: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-16T00:40:17.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:40:17.531 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:18.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 -- 172.21.15.1:0/2731671189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 msgr2=0x7f3400107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 --2- 172.21.15.1:0/2731671189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 0x7f3400107db0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009930 tx=0x7f33f002a420 comp rx=0 tx=0).stop 2024-09-16T00:40:18.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 -- 172.21.15.1:0/2731671189 shutdown_connections 2024-09-16T00:40:18.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 --2- 172.21.15.1:0/2731671189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400102370 0x7f340010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 --2- 172.21.15.1:0/2731671189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34001019d0 0x7f3400101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 --2- 172.21.15.1:0/2731671189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 0x7f3400107db0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 -- 172.21.15.1:0/2731671189 >> 172.21.15.1:0/2731671189 conn(0x7f34000fd710 msgr2=0x7f34000ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:18.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 -- 172.21.15.1:0/2731671189 shutdown_connections 2024-09-16T00:40:18.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 -- 172.21.15.1:0/2731671189 wait complete. 2024-09-16T00:40:18.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 Processor -- start 2024-09-16T00:40:18.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.055+0000 7f33ff59e640 1 -- start start 2024-09-16T00:40:18.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33ff59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34001019d0 0x7f3400198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33ff59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400102370 0x7f34001987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33ff59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 0x7f340019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33ff59e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3400111dd0 con 0x7f34001079d0 2024-09-16T00:40:18.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33ff59e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3400111c50 con 0x7f34001019d0 2024-09-16T00:40:18.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33ff59e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3400111f50 con 0x7f3400102370 2024-09-16T00:40:18.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 0x7f340019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fe59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34001019d0 0x7f3400198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 0x7f340019cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36292/0 (socket says 172.21.15.1:36292) 2024-09-16T00:40:18.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fe59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34001019d0 0x7f3400198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:42854/0 (socket says 172.21.15.1:42854) 2024-09-16T00:40:18.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fed9d640 1 -- 172.21.15.1:0/3400703488 learned_addr learned my addr 172.21.15.1:0/3400703488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:18.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fdd9b640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400102370 0x7f34001987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fed9d640 1 -- 172.21.15.1:0/3400703488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400102370 msgr2=0x7f34001987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fed9d640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400102370 0x7f34001987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fed9d640 1 -- 172.21.15.1:0/3400703488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34001019d0 msgr2=0x7f3400198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fed9d640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34001019d0 0x7f3400198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fed9d640 1 -- 172.21.15.1:0/3400703488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340019d230 con 0x7f34001079d0 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.056+0000 7f33fed9d640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 0x7f340019cb50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f33f800eaa0 tx=0x7f33f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.057+0000 7f33ef7fe640 1 -- 172.21.15.1:0/3400703488 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f800cdb0 con 0x7f34001079d0 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.057+0000 7f33ff59e640 1 -- 172.21.15.1:0/3400703488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340019d520 con 0x7f34001079d0 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.057+0000 7f33ef7fe640 1 -- 172.21.15.1:0/3400703488 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f8005020 con 0x7f34001079d0 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.057+0000 7f33ef7fe640 1 -- 172.21.15.1:0/3400703488 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f80116d0 con 0x7f34001079d0 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.057+0000 7f33ff59e640 1 -- 172.21.15.1:0/3400703488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34001a4e00 con 0x7f34001079d0 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.059+0000 7f3406bdd740 1 Processor -- start 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.059+0000 7f33ef7fe640 1 -- 172.21.15.1:0/3400703488 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f33f80054d0 con 0x7f34001079d0 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.059+0000 7f3406bdd740 1 -- start start 2024-09-16T00:40:18.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.059+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd3c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.059+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3c7b0 0x55aaabd3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.059+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd47550 0x55aaabd49940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.059+0000 7f3406bdd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aaabca22f0 con 0x55aaabd3c7b0 2024-09-16T00:40:18.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.059+0000 7f3406bdd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aaabc49cf0 con 0x55aaabd47550 2024-09-16T00:40:18.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.059+0000 7f3406bdd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aaabc9e2b0 con 0x55aaabd3dd20 2024-09-16T00:40:18.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3c7b0 0x55aaabd3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3c7b0 0x55aaabd3cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36302/0 (socket says 172.21.15.1:36302) 2024-09-16T00:40:18.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fe59c640 1 -- 172.21.15.1:0/4175373532 learned_addr learned my addr 172.21.15.1:0/4175373532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:18.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fdd9b640 1 --2- 172.21.15.1:0/4175373532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd47550 0x55aaabd49940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fdd9b640 1 --2- 172.21.15.1:0/4175373532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd3c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fe59c640 1 -- 172.21.15.1:0/4175373532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 msgr2=0x55aaabd3c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fe59c640 1 --2- 172.21.15.1:0/4175373532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd3c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fe59c640 1 -- 172.21.15.1:0/4175373532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd47550 msgr2=0x55aaabd49940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fe59c640 1 --2- 172.21.15.1:0/4175373532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd47550 0x55aaabd49940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fe59c640 1 -- 172.21.15.1:0/4175373532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaabc2e3a0 con 0x55aaabd3c7b0 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33ef7fe640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f33e80a3240 0x7f33e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fdd9b640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f33e80a3240 0x7f33e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33ef7fe640 1 -- 172.21.15.1:0/3400703488 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f33f80cfec0 con 0x7f34001079d0 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33fe59c640 1 --2- 172.21.15.1:0/4175373532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3c7b0 0x55aaabd3cbf0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009930 tx=0x7f33f0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8bd548da92be3b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33ecff9640 1 -- 172.21.15.1:0/4175373532 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0040070 con 0x55aaabd3c7b0 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33ecff9640 1 -- 172.21.15.1:0/4175373532 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f0005020 con 0x55aaabd3c7b0 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.060+0000 7f33ecff9640 1 -- 172.21.15.1:0/4175373532 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0035040 con 0x55aaabd3c7b0 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f3406bdd740 1 -- 172.21.15.1:0/4175373532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3c7b0 msgr2=0x55aaabd3cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f33fdd9b640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f33e80a3240 0x7f33e80a5700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x55aaabd3d250 tx=0x7f33f4013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4175373532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3c7b0 0x55aaabd3cbf0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009930 tx=0x7f33f0031630 comp rx=0 tx=0).stop 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f3406bdd740 1 -- 172.21.15.1:0/4175373532 shutdown_connections 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4175373532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd47550 0x55aaabd49940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4175373532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3c7b0 0x55aaabd3cbf0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4175373532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd3c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f3406bdd740 1 -- 172.21.15.1:0/4175373532 >> 172.21.15.1:0/4175373532 conn(0x55aaabd05310 msgr2=0x55aaabd38e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f3406bdd740 1 -- 172.21.15.1:0/4175373532 shutdown_connections 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.061+0000 7f3406bdd740 1 -- 172.21.15.1:0/4175373532 wait complete. 2024-09-16T00:40:18.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f3406bdd740 1 -- start start 2024-09-16T00:40:18.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f3406bdd740 1 Processor -- start 2024-09-16T00:40:18.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd46dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3dd20 0x55aaabd440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd47550 0x55aaabd44610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f3406bdd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aaabd4a1f0 con 0x55aaabd3dd20 2024-09-16T00:40:18.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f3406bdd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aaabc5d060 con 0x55aaabd3c7b0 2024-09-16T00:40:18.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f3406bdd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aaabd4a370 con 0x55aaabd47550 2024-09-16T00:40:18.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f3406bdd740 10 client.0 _ll_register_callbacks cb 0x55aaab9b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:18.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fdd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3dd20 0x55aaabd440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fdd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3dd20 0x55aaabd440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36314/0 (socket says 172.21.15.1:36314) 2024-09-16T00:40:18.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fdd9b640 1 -- 172.21.15.1:0/4099659222 learned_addr learned my addr 172.21.15.1:0/4099659222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:18.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fe59c640 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd46dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fed9d640 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd47550 0x55aaabd44610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fdd9b640 1 -- 172.21.15.1:0/4099659222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd47550 msgr2=0x55aaabd44610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:18.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fdd9b640 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd47550 0x55aaabd44610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fdd9b640 1 -- 172.21.15.1:0/4099659222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 msgr2=0x55aaabd46dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fdd9b640 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd46dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.062+0000 7f33fdd9b640 1 -- 172.21.15.1:0/4099659222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaaba133c0 con 0x55aaabd3dd20 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f33fdd9b640 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3dd20 0x55aaabd440d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f33f401fc90 tx=0x7f33f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61197b451b739cca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f33e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f33e5ffb640 1 -- 172.21.15.1:0/4099659222 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4020640 con 0x55aaabd3dd20 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f33e5ffb640 1 -- 172.21.15.1:0/4099659222 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f33f40207e0 con 0x55aaabd3dd20 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f3406bdd740 1 -- 172.21.15.1:0/4099659222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f0005020 con 0x55aaabd3dd20 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f33e5ffb640 1 -- 172.21.15.1:0/4099659222 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33f40209e0 con 0x55aaabd3dd20 2024-09-16T00:40:18.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f3406bdd740 10 client.18327 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f3406bdd740 1 -- 172.21.15.1:0/4099659222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aaabd59ff0 con 0x55aaabd3dd20 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f33e5ffb640 1 -- 172.21.15.1:0/4099659222 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f0005020 con 0x55aaabd3dd20 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.063+0000 7f33e5ffb640 1 -- 172.21.15.1:0/4099659222 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f33f402ac40 con 0x55aaabd3dd20 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.064+0000 7f3406bdd740 10 client.18327 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.064+0000 7f3406bdd740 10 client.18327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.064+0000 7f3406bdd740 20 client.18327 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.064+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aaabd5b4b0 0x55aaabd5d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.064+0000 7f3406bdd740 4 client.18327 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.064+0000 7f3406bdd740 1 -- 172.21.15.1:0/4099659222 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaabcaf860 con 0x55aaabd5b4b0 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.064+0000 7f33fe59c640 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aaabd5b4b0 0x55aaabd5d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.065+0000 7f33e5ffb640 10 client.18327 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.065+0000 7f33fe59c640 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aaabd5b4b0 0x55aaabd5d8e0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e207eed8e5878e11 server_cookie=6958f7c8fa852f68 in_seq=0 out_seq=0 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.065+0000 7f33e5ffb640 1 -- 172.21.15.1:0/4099659222 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaabcaf860 con 0x55aaabd5b4b0 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.065+0000 7f33e5ffb640 10 client.18327 handle_command_reply: tid=0 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.065+0000 7f3406bdd740 1 client.18327 shutdown 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.065+0000 7f3406bdd740 1 -- 172.21.15.1:0/4099659222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3dd20 msgr2=0x55aaabd440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.065+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3dd20 0x55aaabd440d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f33f401fc90 tx=0x7f33f4004730 comp rx=0 tx=0).stop 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.065+0000 7f3406bdd740 1 -- 172.21.15.1:0/4099659222 shutdown_connections 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.066+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aaabd5b4b0 0x55aaabd5d8e0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.066+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd47550 0x55aaabd44610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.066+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd3dd20 0x55aaabd440d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.066+0000 7f3406bdd740 1 --2- 172.21.15.1:0/4099659222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd46dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.066+0000 7f33e5ffb640 0 client.18327 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.066+0000 7f3406bdd740 1 -- 172.21.15.1:0/4099659222 >> 172.21.15.1:0/4099659222 conn(0x55aaabd05310 msgr2=0x55aaabd3eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:18.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.066+0000 7f3406bdd740 1 -- 172.21.15.1:0/4099659222 shutdown_connections 2024-09-16T00:40:18.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.066+0000 7f3406bdd740 1 -- 172.21.15.1:0/4099659222 wait complete. 2024-09-16T00:40:18.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.066+0000 7f3406bdd740 20 client.18327 trim_cache size 0 max 16384 2024-09-16T00:40:18.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 Processor -- start 2024-09-16T00:40:18.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 -- start start 2024-09-16T00:40:18.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd57920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd33b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd48940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f34001af410 con 0x55aaabd47550 2024-09-16T00:40:18.120 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:40:18.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34001acce0 con 0x55aaabd3c7b0 2024-09-16T00:40:18.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f34001aab30 con 0x55aaabd3dd20 2024-09-16T00:40:18.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd48940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd48940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36324/0 (socket says 172.21.15.1:36324) 2024-09-16T00:40:18.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fed9d640 1 -- 172.21.15.1:0/2806176936 learned_addr learned my addr 172.21.15.1:0/2806176936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:18.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fed9d640 1 -- 172.21.15.1:0/2806176936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 msgr2=0x55aaabd33b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:18.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fed9d640 1 --2- 172.21.15.1:0/2806176936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd33b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fed9d640 1 -- 172.21.15.1:0/2806176936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 msgr2=0x55aaabd57920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fe59c640 1 --2- 172.21.15.1:0/2806176936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd57920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fed9d640 1 --2- 172.21.15.1:0/2806176936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd57920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fed9d640 1 -- 172.21.15.1:0/2806176936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaabd5df30 con 0x55aaabd47550 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33fed9d640 1 --2- 172.21.15.1:0/2806176936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd48940 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f33f800cfd0 tx=0x7f33f809c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf55e4cac2ead522 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33e7fff640 1 -- 172.21.15.1:0/2806176936 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f801f040 con 0x55aaabd47550 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33e7fff640 1 -- 172.21.15.1:0/2806176936 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f80b5070 con 0x55aaabd47550 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f33e7fff640 1 -- 172.21.15.1:0/2806176936 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f80a4860 con 0x55aaabd47550 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 -- 172.21.15.1:0/2806176936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 msgr2=0x55aaabd48940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 --2- 172.21.15.1:0/2806176936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd48940 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f33f800cfd0 tx=0x7f33f809c270 comp rx=0 tx=0).stop 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 -- 172.21.15.1:0/2806176936 shutdown_connections 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 --2- 172.21.15.1:0/2806176936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd48940 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 --2- 172.21.15.1:0/2806176936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd33b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 --2- 172.21.15.1:0/2806176936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd57920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 -- 172.21.15.1:0/2806176936 >> 172.21.15.1:0/2806176936 conn(0x55aaabd05310 msgr2=0x55aaabd429a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 -- 172.21.15.1:0/2806176936 shutdown_connections 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.080+0000 7f3406bdd740 1 -- 172.21.15.1:0/2806176936 wait complete. 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f3406bdd740 1 -- start start 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f3406bdd740 1 Processor -- start 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd62af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f3406bdd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd63170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f3406bdd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f34001ab1a0 con 0x55aaabd47550 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f3406bdd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34001ac610 con 0x55aaabd3c7b0 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f3406bdd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f34001ab870 con 0x55aaabd3dd20 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f3406bdd740 10 client.0 _ll_register_callbacks cb 0x7f33f401f910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:18.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f33fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd63170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f33fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd63170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36334/0 (socket says 172.21.15.1:36334) 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.081+0000 7f33fed9d640 1 -- 172.21.15.1:0/1279331547 learned_addr learned my addr 172.21.15.1:0/1279331547 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33fed9d640 1 -- 172.21.15.1:0/1279331547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 msgr2=0x55aaabd62af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33fdd9b640 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd62af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33fe59c640 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33fed9d640 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd62af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33fed9d640 1 -- 172.21.15.1:0/1279331547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 msgr2=0x55aaabd542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33fed9d640 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33fed9d640 1 -- 172.21.15.1:0/1279331547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaabd51d80 con 0x55aaabd47550 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33fe59c640 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33fed9d640 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd63170 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f33f80a05f0 tx=0x7f33f8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55360b25498b668 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:18.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33e5ffb640 1 -- 172.21.15.1:0/1279331547 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f80a4860 con 0x55aaabd47550 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33e5ffb640 1 -- 172.21.15.1:0/1279331547 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f33f80a4a00 con 0x55aaabd47550 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f3406bdd740 1 -- 172.21.15.1:0/1279331547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f80b5070 con 0x55aaabd47550 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33e5ffb640 1 -- 172.21.15.1:0/1279331547 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33f809b580 con 0x55aaabd47550 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f3406bdd740 10 client.18333 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f3406bdd740 1 -- 172.21.15.1:0/1279331547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aaabdc6240 con 0x55aaabd47550 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33e5ffb640 1 -- 172.21.15.1:0/1279331547 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f80b5070 con 0x55aaabd47550 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.082+0000 7f33e5ffb640 1 -- 172.21.15.1:0/1279331547 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f33f800f040 con 0x55aaabd47550 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 10 client.18333 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 20 client.18333 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aaabdc6910 0x55aaabdc8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 4 client.18333 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 -- 172.21.15.1:0/1279331547 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aaabca9d70 con 0x55aaabdc6910 2024-09-16T00:40:18.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f33fe59c640 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aaabdc6910 0x55aaabdc8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f33e5ffb640 10 client.18333 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f33fe59c640 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aaabdc6910 0x55aaabdc8d30 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b2ede63376131e server_cookie=cc63fc8c3f26069c in_seq=0 out_seq=0 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f33e5ffb640 1 -- 172.21.15.1:0/1279331547 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aaabca9d70 con 0x55aaabdc6910 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f33e5ffb640 10 client.18333 handle_command_reply: tid=0 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 client.18333 shutdown 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 -- 172.21.15.1:0/1279331547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 msgr2=0x55aaabd63170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd63170 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f33f80a05f0 tx=0x7f33f8009f60 comp rx=0 tx=0).stop 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 -- 172.21.15.1:0/1279331547 shutdown_connections 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55aaabdc6910 0x55aaabdc8d30 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaabd47550 0x55aaabd63170 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaabd3dd20 0x55aaabd62af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f33e5ffb640 0 client.18333 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 --2- 172.21.15.1:0/1279331547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aaabd3c7b0 0x55aaabd542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.083+0000 7f3406bdd740 1 -- 172.21.15.1:0/1279331547 >> 172.21.15.1:0/1279331547 conn(0x55aaabd05310 msgr2=0x55aaabd45930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f3406bdd740 1 -- 172.21.15.1:0/1279331547 shutdown_connections 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f3406bdd740 1 -- 172.21.15.1:0/1279331547 wait complete. 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f3406bdd740 20 client.18333 trim_cache size 0 max 16384 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 -- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f33e80a3240 msgr2=0x7f33e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f33e80a3240 0x7f33e80a5700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x55aaabd3d250 tx=0x7f33f4013040 comp rx=0 tx=0).stop 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 -- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 msgr2=0x7f340019cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 0x7f340019cb50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f33f800eaa0 tx=0x7f33f800ef60 comp rx=0 tx=0).stop 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 -- 172.21.15.1:0/3400703488 shutdown_connections 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34001079d0 0x7f340019cb50 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f33e80a3240 0x7f33e80a5700 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400102370 0x7f34001987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 --2- 172.21.15.1:0/3400703488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34001019d0 0x7f3400198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 -- 172.21.15.1:0/3400703488 >> 172.21.15.1:0/3400703488 conn(0x7f34000fd710 msgr2=0x7f34000fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:18.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 -- 172.21.15.1:0/3400703488 shutdown_connections 2024-09-16T00:40:18.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:18.084+0000 7f33ff59e640 1 -- 172.21.15.1:0/3400703488 wait complete. 2024-09-16T00:40:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:18 smithi019 ceph-mon[35186]: pgmap v707: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-16T00:40:18.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:18 smithi001 ceph-mon[30934]: pgmap v707: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-16T00:40:18.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:18 smithi153 ceph-mon[35538]: pgmap v707: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-16T00:40:19.815 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:40:19.816 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:40:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:20.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:20 smithi001 ceph-mon[30934]: pgmap v708: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-16T00:40:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:20 smithi019 ceph-mon[35186]: pgmap v708: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-16T00:40:20.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.591+0000 7ff603280640 1 -- 172.21.15.1:0/4191758601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc1036e0 msgr2=0x7ff5fc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.591+0000 7ff603280640 1 --2- 172.21.15.1:0/4191758601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc1036e0 0x7ff5fc107b90 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009930 tx=0x7ff5ec02a420 comp rx=0 tx=0).stop 2024-09-16T00:40:20.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.592+0000 7ff603280640 1 -- 172.21.15.1:0/4191758601 shutdown_connections 2024-09-16T00:40:20.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.592+0000 7ff603280640 1 --2- 172.21.15.1:0/4191758601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc1082a0 0x7ff5fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.592+0000 7ff603280640 1 --2- 172.21.15.1:0/4191758601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc1036e0 0x7ff5fc107b90 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.592+0000 7ff603280640 1 --2- 172.21.15.1:0/4191758601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5fc102d30 0x7ff5fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.592+0000 7ff603280640 1 -- 172.21.15.1:0/4191758601 >> 172.21.15.1:0/4191758601 conn(0x7ff5fc0fc7d0 msgr2=0x7ff5fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.592+0000 7ff603280640 1 -- 172.21.15.1:0/4191758601 shutdown_connections 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.592+0000 7ff603280640 1 -- 172.21.15.1:0/4191758601 wait complete. 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.592+0000 7ff603280640 1 Processor -- start 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.592+0000 7ff603280640 1 -- start start 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff603280640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc102d30 0x7ff5fc198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff603280640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc1036e0 0x7ff5fc198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff603280640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5fc1082a0 0x7ff5fc19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff603280640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc112060 con 0x7ff5fc102d30 2024-09-16T00:40:20.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff603280640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc111ee0 con 0x7ff5fc1082a0 2024-09-16T00:40:20.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff603280640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1121e0 con 0x7ff5fc1036e0 2024-09-16T00:40:20.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60101c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc102d30 0x7ff5fc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60101c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc102d30 0x7ff5fc198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45348/0 (socket says 172.21.15.1:45348) 2024-09-16T00:40:20.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60101c640 1 -- 172.21.15.1:0/536839660 learned_addr learned my addr 172.21.15.1:0/536839660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:20.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60181d640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5fc1082a0 0x7ff5fc19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60081b640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc1036e0 0x7ff5fc198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60101c640 1 -- 172.21.15.1:0/536839660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc1036e0 msgr2=0x7ff5fc198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60101c640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc1036e0 0x7ff5fc198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60101c640 1 -- 172.21.15.1:0/536839660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5fc1082a0 msgr2=0x7ff5fc19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60101c640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5fc1082a0 0x7ff5fc19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60101c640 1 -- 172.21.15.1:0/536839660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19d5b0 con 0x7ff5fc102d30 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff60101c640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc102d30 0x7ff5fc198600 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800d950 tx=0x7ff5f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff5f27fc640 1 -- 172.21.15.1:0/536839660 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8017070 con 0x7ff5fc102d30 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff603280640 1 -- 172.21.15.1:0/536839660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc19d8a0 con 0x7ff5fc102d30 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff5f27fc640 1 -- 172.21.15.1:0/536839660 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f8009c30 con 0x7ff5fc102d30 2024-09-16T00:40:20.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.593+0000 7ff5f27fc640 1 -- 172.21.15.1:0/536839660 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800f6b0 con 0x7ff5fc102d30 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.594+0000 7ff603280640 1 -- 172.21.15.1:0/536839660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a5180 con 0x7ff5fc102d30 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff5f27fc640 1 -- 172.21.15.1:0/536839660 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff5f80109d0 con 0x7ff5fc102d30 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff605659740 1 Processor -- start 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff605659740 1 -- start start 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff605659740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced976b60 0x558ced976f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff605659740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced977480 0x558ced981800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff605659740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced981d40 0x558ced984130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff605659740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ced8dc280 con 0x558ced981d40 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff605659740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ced883b10 con 0x558ced977480 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff605659740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ced8d8240 con 0x558ced976b60 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff60081b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced981d40 0x558ced984130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.596+0000 7ff60081b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced981d40 0x558ced984130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45350/0 (socket says 172.21.15.1:45350) 2024-09-16T00:40:20.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60081b640 1 -- 172.21.15.1:0/1108329571 learned_addr learned my addr 172.21.15.1:0/1108329571 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60101c640 1 --2- 172.21.15.1:0/1108329571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced977480 0x558ced981800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff5f27fc640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5e80a3240 0x7ff5e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60181d640 1 --2- 172.21.15.1:0/1108329571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced976b60 0x558ced976f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60081b640 1 -- 172.21.15.1:0/1108329571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced976b60 msgr2=0x558ced976f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60081b640 1 --2- 172.21.15.1:0/1108329571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced976b60 0x558ced976f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60081b640 1 -- 172.21.15.1:0/1108329571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced977480 msgr2=0x558ced981800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60081b640 1 --2- 172.21.15.1:0/1108329571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced977480 0x558ced981800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60081b640 1 -- 172.21.15.1:0/1108329571 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ced8901f0 con 0x558ced981d40 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60181d640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5e80a3240 0x7ff5e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff5f27fc640 1 -- 172.21.15.1:0/536839660 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff5f80cf9a0 con 0x7ff5fc102d30 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff60081b640 1 --2- 172.21.15.1:0/1108329571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced981d40 0x558ced984130 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec005ab0 tx=0x7ff5ec032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f060734cd70409 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff5e77fe640 1 -- 172.21.15.1:0/1108329571 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec03f070 con 0x558ced981d40 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff5e77fe640 1 -- 172.21.15.1:0/1108329571 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec0317b0 con 0x558ced981d40 2024-09-16T00:40:20.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.597+0000 7ff5e77fe640 1 -- 172.21.15.1:0/1108329571 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec035070 con 0x558ced981d40 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 -- 172.21.15.1:0/1108329571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced981d40 msgr2=0x558ced984130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff60181d640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5e80a3240 0x7ff5e80a5700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400a2c0 tx=0x7ff5f4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 --2- 172.21.15.1:0/1108329571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced981d40 0x558ced984130 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec005ab0 tx=0x7ff5ec032cb0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 -- 172.21.15.1:0/1108329571 shutdown_connections 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 --2- 172.21.15.1:0/1108329571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced981d40 0x558ced984130 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 --2- 172.21.15.1:0/1108329571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced977480 0x558ced981800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 --2- 172.21.15.1:0/1108329571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced976b60 0x558ced976f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 -- 172.21.15.1:0/1108329571 >> 172.21.15.1:0/1108329571 conn(0x558ced9730f0 msgr2=0x558ced9734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 -- 172.21.15.1:0/1108329571 shutdown_connections 2024-09-16T00:40:20.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 -- 172.21.15.1:0/1108329571 wait complete. 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.598+0000 7ff605659740 1 -- start start 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff605659740 1 Processor -- start 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff605659740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 0x558ced97ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff605659740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced97f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff605659740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 0x558ced97f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff605659740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ced984910 con 0x558ced977480 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff605659740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ced896f10 con 0x558ced976b60 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff605659740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ced984a90 con 0x558ced981d40 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff605659740 10 client.0 _ll_register_callbacks cb 0x558ced85fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60081b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced97f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60081b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced97f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45352/0 (socket says 172.21.15.1:45352) 2024-09-16T00:40:20.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60081b640 1 -- 172.21.15.1:0/605760923 learned_addr learned my addr 172.21.15.1:0/605760923 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:20.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60181d640 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 0x558ced97f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60101c640 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 0x558ced97ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60081b640 1 -- 172.21.15.1:0/605760923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 msgr2=0x558ced97f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:20.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60081b640 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 0x558ced97f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60081b640 1 -- 172.21.15.1:0/605760923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 msgr2=0x558ced97ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:20.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60081b640 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 0x558ced97ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.599+0000 7ff60081b640 1 -- 172.21.15.1:0/605760923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ced64d6a0 con 0x558ced977480 2024-09-16T00:40:20.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff60081b640 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced97f150 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec02a940 tx=0x7ff5ec0316a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fb72b0e6641c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff5e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/605760923 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec035070 con 0x558ced977480 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff605659740 1 -- 172.21.15.1:0/605760923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5ec0317b0 con 0x558ced977480 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/605760923 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff5ec0035f0 con 0x558ced977480 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/605760923 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5ec055050 con 0x558ced977480 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff605659740 10 client.18345 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff605659740 1 -- 172.21.15.1:0/605760923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558ced990510 con 0x558ced977480 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/605760923 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec0317b0 con 0x558ced977480 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/605760923 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff5ec03ba40 con 0x558ced977480 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff605659740 10 client.18345 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff605659740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff605659740 20 client.18345 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff605659740 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558ced991860 0x558ced993c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff605659740 4 client.18345 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:20.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.600+0000 7ff605659740 1 -- 172.21.15.1:0/605760923 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ced8e97f0 con 0x558ced991860 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.601+0000 7ff60101c640 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558ced991860 0x558ced993c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.601+0000 7ff60101c640 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558ced991860 0x558ced993c50 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adc722bb225f256 server_cookie=b489cb4532dc18f8 in_seq=0 out_seq=0 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.601+0000 7ff5e4ff9640 10 client.18345 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/605760923 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ced8e97f0 con 0x558ced991860 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff5e4ff9640 10 client.18345 handle_command_reply: tid=0 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 client.18345 shutdown 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 -- 172.21.15.1:0/605760923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 msgr2=0x558ced97f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced97f150 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec02a940 tx=0x7ff5ec0316a0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 -- 172.21.15.1:0/605760923 shutdown_connections 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558ced991860 0x558ced993c50 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 0x558ced97f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced97f150 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 --2- 172.21.15.1:0/605760923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 0x558ced97ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff5e4ff9640 0 client.18345 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:20.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 -- 172.21.15.1:0/605760923 >> 172.21.15.1:0/605760923 conn(0x558ced9730f0 msgr2=0x558ced978dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 -- 172.21.15.1:0/605760923 shutdown_connections 2024-09-16T00:40:20.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 1 -- 172.21.15.1:0/605760923 wait complete. 2024-09-16T00:40:20.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.602+0000 7ff605659740 20 client.18345 trim_cache size 0 max 16384 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.619+0000 7ff605659740 1 Processor -- start 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff605659740 1 -- start start 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff605659740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced976b60 0x558ced98a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff605659740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced977480 0x558ced98a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff605659740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced981d40 0x558ced97d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff605659740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1b16e0 con 0x558ced976b60 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff605659740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1aefb0 con 0x558ced981d40 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff605659740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1ace00 con 0x558ced977480 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60101c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced976b60 0x558ced98a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60101c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced976b60 0x558ced98a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45368/0 (socket says 172.21.15.1:45368) 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60101c640 1 -- 172.21.15.1:0/4179408030 learned_addr learned my addr 172.21.15.1:0/4179408030 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60081b640 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced977480 0x558ced98a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60181d640 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced981d40 0x558ced97d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60101c640 1 -- 172.21.15.1:0/4179408030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced977480 msgr2=0x558ced98a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60101c640 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced977480 0x558ced98a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60101c640 1 -- 172.21.15.1:0/4179408030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced981d40 msgr2=0x558ced97d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60101c640 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced981d40 0x558ced97d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60101c640 1 -- 172.21.15.1:0/4179408030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ced97d160 con 0x558ced976b60 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60081b640 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced977480 0x558ced98a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff60101c640 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced976b60 0x558ced98a0c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80cd0a0 tx=0x7ff5f80d1eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e09c514aecfb62 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.620+0000 7ff5e6ffd640 1 -- 172.21.15.1:0/4179408030 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8020040 con 0x558ced976b60 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff5e6ffd640 1 -- 172.21.15.1:0/4179408030 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f80cfe50 con 0x558ced976b60 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff5e6ffd640 1 -- 172.21.15.1:0/4179408030 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8098230 con 0x558ced976b60 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff605659740 1 -- 172.21.15.1:0/4179408030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced976b60 msgr2=0x558ced98a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff605659740 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced976b60 0x558ced98a0c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80cd0a0 tx=0x7ff5f80d1eb0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff605659740 1 -- 172.21.15.1:0/4179408030 shutdown_connections 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff605659740 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced981d40 0x558ced97d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff605659740 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced977480 0x558ced98a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff605659740 1 --2- 172.21.15.1:0/4179408030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced976b60 0x558ced98a0c0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff605659740 1 -- 172.21.15.1:0/4179408030 >> 172.21.15.1:0/4179408030 conn(0x558ced9730f0 msgr2=0x558ced978dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff605659740 1 -- 172.21.15.1:0/4179408030 shutdown_connections 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.621+0000 7ff605659740 1 -- 172.21.15.1:0/4179408030 wait complete. 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff605659740 1 -- start start 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff605659740 1 Processor -- start 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff605659740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 0x558ced9f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff605659740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced9f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff605659740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 0x558ced9f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff605659740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1ad470 con 0x558ced977480 2024-09-16T00:40:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff605659740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1ae8e0 con 0x558ced976b60 2024-09-16T00:40:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff605659740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1adb40 con 0x558ced981d40 2024-09-16T00:40:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff605659740 10 client.0 _ll_register_callbacks cb 0x7ff5f8097ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff60081b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced9f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff60081b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced9f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45378/0 (socket says 172.21.15.1:45378) 2024-09-16T00:40:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff60081b640 1 -- 172.21.15.1:0/3872818923 learned_addr learned my addr 172.21.15.1:0/3872818923 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff60181d640 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 0x558ced9f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.622+0000 7ff60101c640 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 0x558ced9f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff60081b640 1 -- 172.21.15.1:0/3872818923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 msgr2=0x558ced9f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff60081b640 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 0x558ced9f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff60081b640 1 -- 172.21.15.1:0/3872818923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 msgr2=0x558ced9f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff60081b640 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 0x558ced9f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff60081b640 1 -- 172.21.15.1:0/3872818923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ced9f98d0 con 0x558ced977480 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff60081b640 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced9f4900 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec03be60 tx=0x7ff5ec031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7cbeacd1a36cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff5e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/3872818923 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec035030 con 0x558ced977480 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/3872818923 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff5ec03b430 con 0x558ced977480 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff605659740 1 -- 172.21.15.1:0/3872818923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f80cfe50 con 0x558ced977480 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/3872818923 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5ec065050 con 0x558ced977480 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff605659740 10 client.18351 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff605659740 1 -- 172.21.15.1:0/3872818923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558ceda00bf0 con 0x558ced977480 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.623+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/3872818923 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f80cfe50 con 0x558ced977480 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/3872818923 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff5ec0519e0 con 0x558ced977480 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff605659740 10 client.18351 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff605659740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff605659740 20 client.18351 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:20.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff605659740 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558ceda012e0 0x558ceda03700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff605659740 4 client.18351 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff605659740 1 -- 172.21.15.1:0/3872818923 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ced8e3d00 con 0x558ceda012e0 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff60101c640 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558ceda012e0 0x558ceda03700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff5e4ff9640 10 client.18351 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.624+0000 7ff60101c640 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558ceda012e0 0x558ceda03700 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b712b1997c68a62 server_cookie=47579c8ef471be1d in_seq=0 out_seq=0 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.625+0000 7ff5e4ff9640 1 -- 172.21.15.1:0/3872818923 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ced8e3d00 con 0x558ceda012e0 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.625+0000 7ff5e4ff9640 10 client.18351 handle_command_reply: tid=0 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.625+0000 7ff605659740 1 client.18351 shutdown 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.625+0000 7ff605659740 1 -- 172.21.15.1:0/3872818923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 msgr2=0x558ced9f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.625+0000 7ff605659740 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced9f4900 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec03be60 tx=0x7ff5ec031630 comp rx=0 tx=0).stop 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.625+0000 7ff605659740 1 -- 172.21.15.1:0/3872818923 shutdown_connections 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.625+0000 7ff605659740 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558ceda012e0 0x558ceda03700 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff605659740 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ced981d40 0x558ced9f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff605659740 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ced977480 0x558ced9f4900 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff605659740 1 --2- 172.21.15.1:0/3872818923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ced976b60 0x558ced9f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff605659740 1 -- 172.21.15.1:0/3872818923 >> 172.21.15.1:0/3872818923 conn(0x558ced9730f0 msgr2=0x558ced971a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff5e4ff9640 0 client.18351 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff605659740 1 -- 172.21.15.1:0/3872818923 shutdown_connections 2024-09-16T00:40:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff605659740 1 -- 172.21.15.1:0/3872818923 wait complete. 2024-09-16T00:40:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff605659740 20 client.18351 trim_cache size 0 max 16384 2024-09-16T00:40:20.637 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:40:20.638 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:40:20.638 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "a996bedd-d215-47f7-a42e-91c6568ad89b", 2024-09-16T00:40:20.638 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:40:20.638 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:40:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 -- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5e80a3240 msgr2=0x7ff5e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5e80a3240 0x7ff5e80a5700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400a2c0 tx=0x7ff5f4004a40 comp rx=0 tx=0).stop 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 -- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc102d30 msgr2=0x7ff5fc198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc102d30 0x7ff5fc198600 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800d950 tx=0x7ff5f800de10 comp rx=0 tx=0).stop 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 -- 172.21.15.1:0/536839660 shutdown_connections 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5fc1082a0 0x7ff5fc19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff5e80a3240 0x7ff5e80a5700 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc1036e0 0x7ff5fc198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 --2- 172.21.15.1:0/536839660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5fc102d30 0x7ff5fc198600 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.626+0000 7ff603280640 1 -- 172.21.15.1:0/536839660 >> 172.21.15.1:0/536839660 conn(0x7ff5fc0fc7d0 msgr2=0x7ff5fc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.627+0000 7ff603280640 1 -- 172.21.15.1:0/536839660 shutdown_connections 2024-09-16T00:40:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:20.627+0000 7ff603280640 1 -- 172.21.15.1:0/536839660 wait complete. 2024-09-16T00:40:20.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:20 smithi153 ceph-mon[35538]: pgmap v708: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-16T00:40:21.409 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:40:21.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:21 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:40:21.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:21 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:40:21.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:21 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:40:21.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:21 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:40:21.669 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:21.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:21 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:40:21.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:21 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:40:21.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:21 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:40:21.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:21 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:40:21.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:21 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:40:21.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:21 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:40:21.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:21 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:40:21.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:21 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:40:22.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.181+0000 7fb3bb584640 1 -- 172.21.15.1:0/379985037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b4102d30 msgr2=0x7fb3b4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.181+0000 7fb3bb584640 1 --2- 172.21.15.1:0/379985037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b4102d30 0x7fb3b4103110 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009cd0 tx=0x7fb3a402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.182+0000 7fb3bb584640 1 -- 172.21.15.1:0/379985037 shutdown_connections 2024-09-16T00:40:22.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.182+0000 7fb3bb584640 1 --2- 172.21.15.1:0/379985037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41082a0 0x7fb3b410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.182+0000 7fb3bb584640 1 --2- 172.21.15.1:0/379985037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b41036e0 0x7fb3b4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.182+0000 7fb3bb584640 1 --2- 172.21.15.1:0/379985037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b4102d30 0x7fb3b4103110 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.182+0000 7fb3bb584640 1 -- 172.21.15.1:0/379985037 >> 172.21.15.1:0/379985037 conn(0x7fb3b40fc7d0 msgr2=0x7fb3b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:22.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.182+0000 7fb3bb584640 1 -- 172.21.15.1:0/379985037 shutdown_connections 2024-09-16T00:40:22.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.182+0000 7fb3bb584640 1 -- 172.21.15.1:0/379985037 wait complete. 2024-09-16T00:40:22.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.182+0000 7fb3bb584640 1 Processor -- start 2024-09-16T00:40:22.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.182+0000 7fb3bb584640 1 -- start start 2024-09-16T00:40:22.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3bb584640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b4102d30 0x7fb3b4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3bb584640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b41036e0 0x7fb3b4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3bb584640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41082a0 0x7fb3b4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3bb584640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4112100 con 0x7fb3b41036e0 2024-09-16T00:40:22.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3bb584640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4111f80 con 0x7fb3b4102d30 2024-09-16T00:40:22.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3bb584640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4112280 con 0x7fb3b41082a0 2024-09-16T00:40:22.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b41036e0 0x7fb3b4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b41036e0 0x7fb3b4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45396/0 (socket says 172.21.15.1:45396) 2024-09-16T00:40:22.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/1670838249 learned_addr learned my addr 172.21.15.1:0/1670838249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:22.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b9b21640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41082a0 0x7fb3b4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b9320640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b4102d30 0x7fb3b4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/1670838249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41082a0 msgr2=0x7fb3b4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41082a0 0x7fb3b4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/1670838249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b4102d30 msgr2=0x7fb3b4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b4102d30 0x7fb3b4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/1670838249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b41aaf20 con 0x7fb3b41036e0 2024-09-16T00:40:22.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b41036e0 0x7fb3b4100ed0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c8c0 tx=0x7fb3a800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:22.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b27fc640 1 -- 172.21.15.1:0/1670838249 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8007bb0 con 0x7fb3b41036e0 2024-09-16T00:40:22.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b27fc640 1 -- 172.21.15.1:0/1670838249 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a8007d50 con 0x7fb3b41036e0 2024-09-16T00:40:22.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3bb584640 1 -- 172.21.15.1:0/1670838249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b41ab150 con 0x7fb3b41036e0 2024-09-16T00:40:22.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3b27fc640 1 -- 172.21.15.1:0/1670838249 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8018830 con 0x7fb3b41036e0 2024-09-16T00:40:22.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.183+0000 7fb3bb584640 1 -- 172.21.15.1:0/1670838249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b41ab690 con 0x7fb3b41036e0 2024-09-16T00:40:22.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.186+0000 7fb3b27fc640 1 -- 172.21.15.1:0/1670838249 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fb3a8018a10 con 0x7fb3b41036e0 2024-09-16T00:40:22.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.186+0000 7fb3bd95d740 1 Processor -- start 2024-09-16T00:40:22.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.186+0000 7fb3bd95d740 1 -- start start 2024-09-16T00:40:22.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.186+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3558880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.186+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3558dc0 0x55f1b3563140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.186+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3563680 0x55f1b3565a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.186+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f1b34be2e0 con 0x55f1b3563680 2024-09-16T00:40:22.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.186+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f1b3465ce0 con 0x55f1b35584a0 2024-09-16T00:40:22.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.186+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f1b34ba2a0 con 0x55f1b3558dc0 2024-09-16T00:40:22.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b27fc640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb39c0a3240 0x7fb39c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3563680 0x55f1b3565a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3563680 0x55f1b3565a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45408/0 (socket says 172.21.15.1:45408) 2024-09-16T00:40:22.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/3194961583 learned_addr learned my addr 172.21.15.1:0/3194961583 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:22.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b9b21640 1 --2- 172.21.15.1:0/3194961583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3558880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b9b21640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb39c0a3240 0x7fb39c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b9320640 1 --2- 172.21.15.1:0/3194961583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3558dc0 0x55f1b3563140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b27fc640 1 -- 172.21.15.1:0/1670838249 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb3a80cff80 con 0x7fb3b41036e0 2024-09-16T00:40:22.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/3194961583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3558dc0 msgr2=0x55f1b3563140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:22.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/3194961583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3558dc0 0x55f1b3563140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/3194961583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 msgr2=0x55f1b3558880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:22.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/3194961583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3558880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/3194961583 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b344a440 con 0x55f1b3563680 2024-09-16T00:40:22.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b9b21640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb39c0a3240 0x7fb39c0a5700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00dfd0 tx=0x7fb3ac00d4f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:22.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/3194961583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3563680 0x55f1b3565a70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8007590 tx=0x7fb3a800fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7061671a542b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:22.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.187+0000 7fb39b7fe640 1 -- 172.21.15.1:0/3194961583 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a801f300 con 0x55f1b3563680 2024-09-16T00:40:22.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb39b7fe640 1 -- 172.21.15.1:0/3194961583 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a80a4db0 con 0x55f1b3563680 2024-09-16T00:40:22.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb39b7fe640 1 -- 172.21.15.1:0/3194961583 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8018d00 con 0x55f1b3563680 2024-09-16T00:40:22.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb3bd95d740 1 -- 172.21.15.1:0/3194961583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3563680 msgr2=0x55f1b3565a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/3194961583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3563680 0x55f1b3565a70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8007590 tx=0x7fb3a800fe90 comp rx=0 tx=0).stop 2024-09-16T00:40:22.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb3bd95d740 1 -- 172.21.15.1:0/3194961583 shutdown_connections 2024-09-16T00:40:22.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/3194961583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3563680 0x55f1b3565a70 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/3194961583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3558dc0 0x55f1b3563140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/3194961583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3558880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb3bd95d740 1 -- 172.21.15.1:0/3194961583 >> 172.21.15.1:0/3194961583 conn(0x55f1b3554a30 msgr2=0x55f1b3554e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb3bd95d740 1 -- 172.21.15.1:0/3194961583 shutdown_connections 2024-09-16T00:40:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.188+0000 7fb3bd95d740 1 -- 172.21.15.1:0/3194961583 wait complete. 2024-09-16T00:40:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3bd95d740 1 -- start start 2024-09-16T00:40:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3bd95d740 1 Processor -- start 2024-09-16T00:40:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b35584a0 0x55f1b3562f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3560220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b3563680 0x55f1b3560760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f1b3566250 con 0x55f1b3558dc0 2024-09-16T00:40:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f1b3479050 con 0x55f1b3563680 2024-09-16T00:40:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f1b35663d0 con 0x55f1b35584a0 2024-09-16T00:40:22.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3bd95d740 10 client.0 _ll_register_callbacks cb 0x55f1b34b9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:22.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3560220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3560220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45410/0 (socket says 172.21.15.1:45410) 2024-09-16T00:40:22.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.189+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/73510825 learned_addr learned my addr 172.21.15.1:0/73510825 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:22.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3b9b21640 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b3563680 0x55f1b3560760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3b9320640 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b35584a0 0x55f1b3562f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/73510825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b35584a0 msgr2=0x55f1b3562f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:22.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b35584a0 0x55f1b3562f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/73510825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b3563680 msgr2=0x55f1b3560760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:22.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b3563680 0x55f1b3560760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/73510825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b322f3c0 con 0x55f1b3558dc0 2024-09-16T00:40:22.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3b9320640 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b35584a0 0x55f1b3562f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:22.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3560220 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80d0270 tx=0x7fb3a809c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e47bafb357df92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:22.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:22.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb398ff9640 1 -- 172.21.15.1:0/73510825 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80b5070 con 0x55f1b3558dc0 2024-09-16T00:40:22.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb398ff9640 1 -- 172.21.15.1:0/73510825 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb3a8018d00 con 0x55f1b3558dc0 2024-09-16T00:40:22.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3bd95d740 1 -- 172.21.15.1:0/73510825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a80a4db0 con 0x55f1b3558dc0 2024-09-16T00:40:22.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb398ff9640 1 -- 172.21.15.1:0/73510825 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb3a80a04f0 con 0x55f1b3558dc0 2024-09-16T00:40:22.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3bd95d740 10 client.18363 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:22.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb3bd95d740 1 -- 172.21.15.1:0/73510825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f1b3571bc0 con 0x55f1b3558dc0 2024-09-16T00:40:22.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb398ff9640 1 -- 172.21.15.1:0/73510825 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a80a4db0 con 0x55f1b3558dc0 2024-09-16T00:40:22.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.190+0000 7fb398ff9640 1 -- 172.21.15.1:0/73510825 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb3a80987b0 con 0x55f1b3558dc0 2024-09-16T00:40:22.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.191+0000 7fb3bd95d740 10 client.18363 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:22.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.191+0000 7fb3bd95d740 10 client.18363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:22.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.191+0000 7fb3bd95d740 20 client.18363 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:22.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.191+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f1b3572f10 0x55f1b3575300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.191+0000 7fb3bd95d740 4 client.18363 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.191+0000 7fb3bd95d740 1 -- 172.21.15.1:0/73510825 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1b34cb850 con 0x55f1b3572f10 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.191+0000 7fb3b9320640 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f1b3572f10 0x55f1b3575300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.191+0000 7fb398ff9640 10 client.18363 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.191+0000 7fb3b9320640 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f1b3572f10 0x55f1b3575300 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83376b4749eca9a9 server_cookie=c2f4bcb21750f9eb in_seq=0 out_seq=0 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb398ff9640 1 -- 172.21.15.1:0/73510825 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1b34cb850 con 0x55f1b3572f10 2024-09-16T00:40:22.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb398ff9640 10 client.18363 handle_command_reply: tid=0 2024-09-16T00:40:22.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 client.18363 shutdown 2024-09-16T00:40:22.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 -- 172.21.15.1:0/73510825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 msgr2=0x55f1b3560220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3560220 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80d0270 tx=0x7fb3a809c900 comp rx=0 tx=0).stop 2024-09-16T00:40:22.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 -- 172.21.15.1:0/73510825 shutdown_connections 2024-09-16T00:40:22.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f1b3572f10 0x55f1b3575300 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b3563680 0x55f1b3560760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3560220 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/73510825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b35584a0 0x55f1b3562f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb398ff9640 0 client.18363 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:22.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 -- 172.21.15.1:0/73510825 >> 172.21.15.1:0/73510825 conn(0x55f1b3554a30 msgr2=0x55f1b355a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:22.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 -- 172.21.15.1:0/73510825 shutdown_connections 2024-09-16T00:40:22.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 1 -- 172.21.15.1:0/73510825 wait complete. 2024-09-16T00:40:22.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.192+0000 7fb3bd95d740 20 client.18363 trim_cache size 0 max 16384 2024-09-16T00:40:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3bd95d740 1 Processor -- start 2024-09-16T00:40:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3bd95d740 1 -- start start 2024-09-16T00:40:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3578860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b355d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41b7db0 con 0x55f1b3558dc0 2024-09-16T00:40:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41b5680 con 0x55f1b35584a0 2024-09-16T00:40:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41b34d0 con 0x55f1b3563680 2024-09-16T00:40:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3578860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3578860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45418/0 (socket says 172.21.15.1:45418) 2024-09-16T00:40:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/65440688 learned_addr learned my addr 172.21.15.1:0/65440688 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b9320640 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b9b21640 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b355d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/65440688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 msgr2=0x55f1b355d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b355d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/65440688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 msgr2=0x55f1b3578320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/65440688 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b3560a90 con 0x55f1b3558dc0 2024-09-16T00:40:22.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b9320640 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:22.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b9b21640 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b355d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:22.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.210+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3578860 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb3a809a3f0 tx=0x7fb3a809a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2c42ee1f0d2e1c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:22.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb39affd640 1 -- 172.21.15.1:0/65440688 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80b5050 con 0x55f1b3558dc0 2024-09-16T00:40:22.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb39affd640 1 -- 172.21.15.1:0/65440688 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a801f300 con 0x55f1b3558dc0 2024-09-16T00:40:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb39affd640 1 -- 172.21.15.1:0/65440688 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80987b0 con 0x55f1b3558dc0 2024-09-16T00:40:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb3bd95d740 1 -- 172.21.15.1:0/65440688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 msgr2=0x55f1b3578860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3578860 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb3a809a3f0 tx=0x7fb3a809a650 comp rx=0 tx=0).stop 2024-09-16T00:40:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb3bd95d740 1 -- 172.21.15.1:0/65440688 shutdown_connections 2024-09-16T00:40:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b355d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3578860 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/65440688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb3bd95d740 1 -- 172.21.15.1:0/65440688 >> 172.21.15.1:0/65440688 conn(0x55f1b3554a30 msgr2=0x55f1b355eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:22.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb3bd95d740 1 -- 172.21.15.1:0/65440688 shutdown_connections 2024-09-16T00:40:22.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.211+0000 7fb3bd95d740 1 -- 172.21.15.1:0/65440688 wait complete. 2024-09-16T00:40:22.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3bd95d740 1 -- start start 2024-09-16T00:40:22.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3bd95d740 1 Processor -- start 2024-09-16T00:40:22.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3575fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3bd95d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b3576740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41b3b40 con 0x55f1b3558dc0 2024-09-16T00:40:22.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41b4fb0 con 0x55f1b35584a0 2024-09-16T00:40:22.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3bd95d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41b4210 con 0x55f1b3563680 2024-09-16T00:40:22.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3bd95d740 10 client.0 _ll_register_callbacks cb 0x7fb3ac00d980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:22.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3575fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3b8b1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3575fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45426/0 (socket says 172.21.15.1:45426) 2024-09-16T00:40:22.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/779274646 learned_addr learned my addr 172.21.15.1:0/779274646 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:22.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.212+0000 7fb3b9320640 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3b9b21640 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b3576740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/779274646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 msgr2=0x55f1b3576740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:22.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b3576740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/779274646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 msgr2=0x55f1b3578c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:22.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3b8b1f640 1 -- 172.21.15.1:0/779274646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b35dc850 con 0x55f1b3558dc0 2024-09-16T00:40:22.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3b9b21640 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b3576740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:22.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3b9320640 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:22.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3b8b1f640 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3575fb0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80d0270 tx=0x7fb3a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdbf6a646cd936b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:22.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:22.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb398ff9640 1 -- 172.21.15.1:0/779274646 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80a4db0 con 0x55f1b3558dc0 2024-09-16T00:40:22.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb398ff9640 1 -- 172.21.15.1:0/779274646 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb3a8018d00 con 0x55f1b3558dc0 2024-09-16T00:40:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3bd95d740 1 -- 172.21.15.1:0/779274646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a801f300 con 0x55f1b3558dc0 2024-09-16T00:40:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb398ff9640 1 -- 172.21.15.1:0/779274646 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb3a809c140 con 0x55f1b3558dc0 2024-09-16T00:40:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3bd95d740 10 client.18369 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.213+0000 7fb3bd95d740 1 -- 172.21.15.1:0/779274646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f1b35dc850 con 0x55f1b3558dc0 2024-09-16T00:40:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb398ff9640 1 -- 172.21.15.1:0/779274646 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a801f300 con 0x55f1b3558dc0 2024-09-16T00:40:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb398ff9640 1 -- 172.21.15.1:0/779274646 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb3a809b000 con 0x55f1b3558dc0 2024-09-16T00:40:22.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb3bd95d740 10 client.18369 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:22.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb3bd95d740 10 client.18369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:22.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb3bd95d740 20 client.18369 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:22.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f1b35ddc60 0x55f1b35e0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb3bd95d740 4 client.18369 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:40:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb3bd95d740 1 -- 172.21.15.1:0/779274646 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1b34c5d60 con 0x55f1b35ddc60 2024-09-16T00:40:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb3b9320640 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f1b35ddc60 0x55f1b35e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb3b9320640 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f1b35ddc60 0x55f1b35e0050 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7a99e282312bed server_cookie=a5f39d23a1d78fe in_seq=0 out_seq=0 2024-09-16T00:40:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.214+0000 7fb398ff9640 10 client.18369 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb398ff9640 1 -- 172.21.15.1:0/779274646 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f1b34c5d60 con 0x55f1b35ddc60 2024-09-16T00:40:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb398ff9640 10 client.18369 handle_command_reply: tid=0 2024-09-16T00:40:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 client.18369 shutdown 2024-09-16T00:40:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 -- 172.21.15.1:0/779274646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 msgr2=0x55f1b3575fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3575fb0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80d0270 tx=0x7fb3a8000bc0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 -- 172.21.15.1:0/779274646 shutdown_connections 2024-09-16T00:40:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f1b35ddc60 0x55f1b35e0050 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1b3563680 0x55f1b3576740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1b3558dc0 0x55f1b3575fb0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb398ff9640 0 client.18369 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 --2- 172.21.15.1:0/779274646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1b35584a0 0x55f1b3578c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 -- 172.21.15.1:0/779274646 >> 172.21.15.1:0/779274646 conn(0x55f1b3554a30 msgr2=0x55f1b3561a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 -- 172.21.15.1:0/779274646 shutdown_connections 2024-09-16T00:40:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 1 -- 172.21.15.1:0/779274646 wait complete. 2024-09-16T00:40:22.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.215+0000 7fb3bd95d740 20 client.18369 trim_cache size 0 max 16384 2024-09-16T00:40:22.249 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:40:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:40:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:40:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "a996bedd-d215-47f7-a42e-91c6568ad89b": { 2024-09-16T00:40:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:40:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "a996bedd-d215-47f7-a42e-91c6568ad89b", 2024-09-16T00:40:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:40:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:22.251 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:40:22.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 -- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb39c0a3240 msgr2=0x7fb39c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb39c0a3240 0x7fb39c0a5700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00dfd0 tx=0x7fb3ac00d4f0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 -- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b41036e0 msgr2=0x7fb3b4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b41036e0 0x7fb3b4100ed0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c8c0 tx=0x7fb3a800cd80 comp rx=0 tx=0).stop 2024-09-16T00:40:22.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 -- 172.21.15.1:0/1670838249 shutdown_connections 2024-09-16T00:40:22.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb39c0a3240 0x7fb39c0a5700 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41082a0 0x7fb3b4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b41036e0 0x7fb3b4100ed0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 --2- 172.21.15.1:0/1670838249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b4102d30 0x7fb3b4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 -- 172.21.15.1:0/1670838249 >> 172.21.15.1:0/1670838249 conn(0x7fb3b40fc7d0 msgr2=0x7fb3b4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 -- 172.21.15.1:0/1670838249 shutdown_connections 2024-09-16T00:40:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:22.216+0000 7fb3bb584640 1 -- 172.21.15.1:0/1670838249 wait complete. 2024-09-16T00:40:22.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:22 smithi001 ceph-mon[30934]: pgmap v709: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-16T00:40:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:22 smithi019 ceph-mon[35186]: pgmap v709: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-16T00:40:22.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:22 smithi153 ceph-mon[35538]: pgmap v709: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-16T00:40:22.959 INFO:tasks.cephfs.filesystem:scrub status for tag:a996bedd-d215-47f7-a42e-91c6568ad89b - {'path': '/', 'tag': 'a996bedd-d215-47f7-a42e-91c6568ad89b', 'options': 'recursive,force'} 2024-09-16T00:40:23.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:23 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:40:23.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:23 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:40:23.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:23 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:40:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:24 smithi019 ceph-mon[35186]: pgmap v710: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-09-16T00:40:24.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:24 smithi001 ceph-mon[30934]: pgmap v710: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-09-16T00:40:24.713 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:24 smithi153 ceph-mon[35538]: pgmap v710: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-09-16T00:40:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:26 smithi019 ceph-mon[35186]: pgmap v711: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2024-09-16T00:40:26.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:26 smithi001 ceph-mon[30934]: pgmap v711: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2024-09-16T00:40:26.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:26 smithi153 ceph-mon[35538]: pgmap v711: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2024-09-16T00:40:28.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:28 smithi019 ceph-mon[35186]: pgmap v712: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-16T00:40:28.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:28 smithi001 ceph-mon[30934]: pgmap v712: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-16T00:40:28.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:28 smithi153 ceph-mon[35538]: pgmap v712: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-16T00:40:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:40:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:40:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:40:29.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:40:29.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:40:29.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:40:29.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:40:29.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:29.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:40:29.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:40:29.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:40:30.540 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:30.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:30 smithi001 ceph-mon[30934]: pgmap v713: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T00:40:30.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:30 smithi019 ceph-mon[35186]: pgmap v713: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T00:40:30.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:30 smithi153 ceph-mon[35538]: pgmap v713: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T00:40:30.813 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:31.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 -- 172.21.15.1:0/758337864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240101420 msgr2=0x7f924010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:31.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 --2- 172.21.15.1:0/758337864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240101420 0x7f924010d950 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9238009cd0 tx=0x7f923802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 -- 172.21.15.1:0/758337864 shutdown_connections 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 --2- 172.21.15.1:0/758337864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240101420 0x7f924010d950 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 --2- 172.21.15.1:0/758337864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9240100a80 0x7f9240100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 --2- 172.21.15.1:0/758337864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9240106a80 0x7f9240106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 -- 172.21.15.1:0/758337864 >> 172.21.15.1:0/758337864 conn(0x7f92400fc7a0 msgr2=0x7f92400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 -- 172.21.15.1:0/758337864 shutdown_connections 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 -- 172.21.15.1:0/758337864 wait complete. 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 Processor -- start 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.384+0000 7f9245da6640 1 -- start start 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f9245da6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240100a80 0x7f92401983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f9245da6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9240101420 0x7f9240198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f9245da6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9240106a80 0x7f924019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f9245da6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9240110e80 con 0x7f9240100a80 2024-09-16T00:40:31.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f9245da6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9240110d00 con 0x7f9240106a80 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f9245da6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9240111000 con 0x7f9240101420 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240100a80 0x7f92401983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240100a80 0x7f92401983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34406/0 (socket says 172.21.15.1:34406) 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923f7fe640 1 -- 172.21.15.1:0/4112138987 learned_addr learned my addr 172.21.15.1:0/4112138987 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923ffff640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9240106a80 0x7f924019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923effd640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9240101420 0x7f9240198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923f7fe640 1 -- 172.21.15.1:0/4112138987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9240101420 msgr2=0x7f9240198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923f7fe640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9240101420 0x7f9240198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923f7fe640 1 -- 172.21.15.1:0/4112138987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9240106a80 msgr2=0x7f924019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923f7fe640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9240106a80 0x7f924019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923f7fe640 1 -- 172.21.15.1:0/4112138987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924019d380 con 0x7f9240100a80 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923f7fe640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240100a80 0x7f92401983d0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f923000c920 tx=0x7f923000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:31.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.385+0000 7f923cff9640 1 -- 172.21.15.1:0/4112138987 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230007c70 con 0x7f9240100a80 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.386+0000 7f923cff9640 1 -- 172.21.15.1:0/4112138987 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9230007e10 con 0x7f9240100a80 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.386+0000 7f923cff9640 1 -- 172.21.15.1:0/4112138987 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230018890 con 0x7f9240100a80 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.386+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924019d670 con 0x7f9240100a80 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.386+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92401a4f50 con 0x7f9240100a80 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.387+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9240100ee0 con 0x7f9240100a80 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.388+0000 7f923cff9640 1 -- 172.21.15.1:0/4112138987 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9230018a70 con 0x7f9240100a80 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.389+0000 7f923cff9640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f92280a3240 0x7f92280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.389+0000 7f923effd640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f92280a3240 0x7f92280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.389+0000 7f923cff9640 1 -- 172.21.15.1:0/4112138987 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f92300cfe40 con 0x7f9240100a80 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.389+0000 7f923effd640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f92280a3240 0x7f92280a5700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f92400febf0 tx=0x7f9234005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:31.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.393+0000 7f923cff9640 1 -- 172.21.15.1:0/4112138987 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9230014030 con 0x7f9240100a80 2024-09-16T00:40:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.604+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9240199730 con 0x7f9240100a80 2024-09-16T00:40:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.605+0000 7f923cff9640 1 -- 172.21.15.1:0/4112138987 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f923009c700 con 0x7f9240100a80 2024-09-16T00:40:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:40:31.606 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:40:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:40:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.607+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f92280a3240 msgr2=0x7f92280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.607+0000 7f9245da6640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f92280a3240 0x7f92280a5700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f92400febf0 tx=0x7f9234005f90 comp rx=0 tx=0).stop 2024-09-16T00:40:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.607+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240100a80 msgr2=0x7f92401983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.607+0000 7f9245da6640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240100a80 0x7f92401983d0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f923000c920 tx=0x7f923000cde0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.608+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 shutdown_connections 2024-09-16T00:40:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.608+0000 7f9245da6640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9240106a80 0x7f924019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.608+0000 7f9245da6640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f92280a3240 0x7f92280a5700 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.608+0000 7f9245da6640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9240101420 0x7f9240198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.608+0000 7f9245da6640 1 --2- 172.21.15.1:0/4112138987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9240100a80 0x7f92401983d0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.608+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 >> 172.21.15.1:0/4112138987 conn(0x7f92400fc7a0 msgr2=0x7f924006a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.608+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 shutdown_connections 2024-09-16T00:40:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:31.608+0000 7f9245da6640 1 -- 172.21.15.1:0/4112138987 wait complete. 2024-09-16T00:40:32.329 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 1c7a2282 --release --await' 2024-09-16T00:40:32.329 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 1c7a2282 --release --await 2024-09-16T00:40:32.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:32 smithi019 ceph-mon[35186]: pgmap v714: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:40:32.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:32 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4112138987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:32.629 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:32.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:32 smithi001 ceph-mon[30934]: pgmap v714: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:40:32.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:32 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4112138987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:32.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:32 smithi153 ceph-mon[35538]: pgmap v714: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:40:32.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:32 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4112138987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:33.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.149+0000 7f7a9212b640 1 -- 172.21.15.1:0/811372385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c0ffc40 msgr2=0x7f7a8c1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.149+0000 7f7a9212b640 1 --2- 172.21.15.1:0/811372385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c0ffc40 0x7f7a8c1000c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009930 tx=0x7f7a8002a420 comp rx=0 tx=0).stop 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.149+0000 7f7a9212b640 1 -- 172.21.15.1:0/811372385 shutdown_connections 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.149+0000 7f7a9212b640 1 --2- 172.21.15.1:0/811372385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a8c100600 0x7f7a8c10e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.149+0000 7f7a9212b640 1 --2- 172.21.15.1:0/811372385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c0ffc40 0x7f7a8c1000c0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.149+0000 7f7a9212b640 1 --2- 172.21.15.1:0/811372385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a8c106c80 0x7f7a8c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.149+0000 7f7a9212b640 1 -- 172.21.15.1:0/811372385 >> 172.21.15.1:0/811372385 conn(0x7f7a8c0fba90 msgr2=0x7f7a8c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.149+0000 7f7a9212b640 1 -- 172.21.15.1:0/811372385 shutdown_connections 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.149+0000 7f7a9212b640 1 -- 172.21.15.1:0/811372385 wait complete. 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.150+0000 7f7a9212b640 1 Processor -- start 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.150+0000 7f7a9212b640 1 -- start start 2024-09-16T00:40:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.150+0000 7f7a9212b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a8c0ffc40 0x7f7a8c1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.150+0000 7f7a9212b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c100600 0x7f7a8c101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.150+0000 7f7a9212b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a8c106c80 0x7f7a8c102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a9212b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c111f30 con 0x7f7a8c100600 2024-09-16T00:40:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a9212b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c111db0 con 0x7f7a8c106c80 2024-09-16T00:40:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a9212b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1120b0 con 0x7f7a8c0ffc40 2024-09-16T00:40:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c100600 0x7f7a8c101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c100600 0x7f7a8c101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34418/0 (socket says 172.21.15.1:34418) 2024-09-16T00:40:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8affd640 1 -- 172.21.15.1:0/7384118 learned_addr learned my addr 172.21.15.1:0/7384118 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a8c0ffc40 0x7f7a8c1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8bfff640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a8c106c80 0x7f7a8c102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8affd640 1 -- 172.21.15.1:0/7384118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a8c0ffc40 msgr2=0x7f7a8c1016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8affd640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a8c0ffc40 0x7f7a8c1016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8affd640 1 -- 172.21.15.1:0/7384118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a8c106c80 msgr2=0x7f7a8c102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8affd640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a8c106c80 0x7f7a8c102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8affd640 1 -- 172.21.15.1:0/7384118 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1a68b0 con 0x7f7a8c100600 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a8affd640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c100600 0x7f7a8c101bf0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7a80004770 tx=0x7f7a80003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a88ff9640 1 -- 172.21.15.1:0/7384118 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8003f070 con 0x7f7a8c100600 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a88ff9640 1 -- 172.21.15.1:0/7384118 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a80031c20 con 0x7f7a8c100600 2024-09-16T00:40:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a9212b640 1 -- 172.21.15.1:0/7384118 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1a6b40 con 0x7f7a8c100600 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.151+0000 7f7a88ff9640 1 -- 172.21.15.1:0/7384118 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80035030 con 0x7f7a8c100600 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.152+0000 7f7a9212b640 1 -- 172.21.15.1:0/7384118 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a7020 con 0x7f7a8c100600 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a88ff9640 1 -- 172.21.15.1:0/7384118 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f7a80042070 con 0x7f7a8c100600 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a94504740 1 Processor -- start 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a94504740 1 -- start start 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a94504740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558464fa2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a94504740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2980 0x558464facd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a94504740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a94504740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558464f07620 con 0x558464fad240 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a94504740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558464eaee50 con 0x558464fa2980 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a94504740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558464f035e0 con 0x558464fa2060 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.154+0000 7f7a88ff9640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7a740a3240 0x7f7a740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2980 0x558464facd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2980 0x558464facd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:38420/0 (socket says 172.21.15.1:38420) 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34422/0 (socket says 172.21.15.1:34422) 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8b7fe640 1 -- 172.21.15.1:0/318076012 learned_addr learned my addr 172.21.15.1:0/318076012 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8bfff640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7a740a3240 0x7f7a740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a88ff9640 1 -- 172.21.15.1:0/7384118 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7a800f0420 con 0x7f7a8c100600 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8bfff640 1 --2- 172.21.15.1:0/318076012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558464fa2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8b7fe640 1 -- 172.21.15.1:0/318076012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 msgr2=0x558464fa2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/318076012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558464fa2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8b7fe640 1 -- 172.21.15.1:0/318076012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 msgr2=0x558464faf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/318076012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8b7fe640 1 -- 172.21.15.1:0/318076012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558464e9fd40 con 0x558464fa2980 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8affd640 1 --2- 172.21.15.1:0/318076012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/318076012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2980 0x558464facd00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00bc50 tx=0x7f7a7c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf6dc07b1f86dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a8bfff640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7a740a3240 0x7f7a740a5700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a84005fd0 tx=0x7f7a84009be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.155+0000 7f7a71ffb640 1 -- 172.21.15.1:0/318076012 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01c480 con 0x558464fa2980 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a71ffb640 1 -- 172.21.15.1:0/318076012 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a7c01cb00 con 0x558464fa2980 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a71ffb640 1 -- 172.21.15.1:0/318076012 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c011c50 con 0x558464fa2980 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a94504740 1 -- 172.21.15.1:0/318076012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2980 msgr2=0x558464facd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a94504740 1 --2- 172.21.15.1:0/318076012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2980 0x558464facd00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00bc50 tx=0x7f7a7c0107a0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a94504740 1 -- 172.21.15.1:0/318076012 shutdown_connections 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a94504740 1 --2- 172.21.15.1:0/318076012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a94504740 1 --2- 172.21.15.1:0/318076012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2980 0x558464facd00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a94504740 1 --2- 172.21.15.1:0/318076012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558464fa2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a94504740 1 -- 172.21.15.1:0/318076012 >> 172.21.15.1:0/318076012 conn(0x558464f9e5f0 msgr2=0x558464f9e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a94504740 1 -- 172.21.15.1:0/318076012 shutdown_connections 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.156+0000 7f7a94504740 1 -- 172.21.15.1:0/318076012 wait complete. 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a94504740 1 -- start start 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a94504740 1 Processor -- start 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a94504740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2060 0x558464faa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a94504740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2980 0x558464faa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a94504740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a94504740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558464fafe10 con 0x558464fad240 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a94504740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558464e8ff10 con 0x558464fa2060 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a94504740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558464faff90 con 0x558464fa2980 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a94504740 10 client.0 _ll_register_callbacks cb 0x558464eb3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34432/0 (socket says 172.21.15.1:34432) 2024-09-16T00:40:33.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a8bfff640 1 -- 172.21.15.1:0/2187342269 learned_addr learned my addr 172.21.15.1:0/2187342269 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.157+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2060 0x558464faa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a8affd640 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2980 0x558464faa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a8bfff640 1 -- 172.21.15.1:0/2187342269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2980 msgr2=0x558464faa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a8bfff640 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2980 0x558464faa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a8bfff640 1 -- 172.21.15.1:0/2187342269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2060 msgr2=0x558464faa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a8bfff640 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2060 0x558464faa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a8bfff640 1 -- 172.21.15.1:0/2187342269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558464c16580 con 0x558464fad240 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a8bfff640 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faab60 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400fae0 tx=0x7f7a84004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6af715d95bc661b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a6affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a6affd640 1 -- 172.21.15.1:0/2187342269 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84024030 con 0x558464fad240 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a6affd640 1 -- 172.21.15.1:0/2187342269 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7a8401a040 con 0x558464fad240 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a94504740 1 -- 172.21.15.1:0/2187342269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a7c01cb00 con 0x558464fad240 2024-09-16T00:40:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a6affd640 1 -- 172.21.15.1:0/2187342269 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7a84031050 con 0x558464fad240 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a94504740 10 client.18387 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.158+0000 7f7a94504740 1 -- 172.21.15.1:0/2187342269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558464ce0960 con 0x558464fad240 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a6affd640 1 -- 172.21.15.1:0/2187342269 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a7c01cb00 con 0x558464fad240 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a6affd640 1 -- 172.21.15.1:0/2187342269 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7a84029410 con 0x558464fad240 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a94504740 10 client.18387 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a94504740 10 client.18387 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a94504740 20 client.18387 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a94504740 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558464fbcdc0 0x558464fbf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a94504740 4 client.18387 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a94504740 1 -- 172.21.15.1:0/2187342269 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558464f14b90 con 0x558464fbcdc0 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558464fbcdc0 0x558464fbf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a6affd640 10 client.18387 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.159+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558464fbcdc0 0x558464fbf260 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2fff440981a0fc server_cookie=52e13eefb8525cfd in_seq=0 out_seq=0 2024-09-16T00:40:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a6affd640 1 -- 172.21.15.1:0/2187342269 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558464f14b90 con 0x558464fbcdc0 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a6affd640 10 client.18387 handle_command_reply: tid=0 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a94504740 1 client.18387 shutdown 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a94504740 1 -- 172.21.15.1:0/2187342269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 msgr2=0x558464faab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a94504740 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faab60 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400fae0 tx=0x7f7a84004ab0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a94504740 1 -- 172.21.15.1:0/2187342269 shutdown_connections 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a94504740 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558464fbcdc0 0x558464fbf260 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a94504740 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fad240 0x558464faab60 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a94504740 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2980 0x558464faa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a94504740 1 --2- 172.21.15.1:0/2187342269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fa2060 0x558464faa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a6affd640 0 client.18387 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.160+0000 7f7a94504740 1 -- 172.21.15.1:0/2187342269 >> 172.21.15.1:0/2187342269 conn(0x558464f9e5f0 msgr2=0x558464fa42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.161+0000 7f7a94504740 1 -- 172.21.15.1:0/2187342269 shutdown_connections 2024-09-16T00:40:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.161+0000 7f7a94504740 1 -- 172.21.15.1:0/2187342269 wait complete. 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.161+0000 7f7a94504740 20 client.18387 trim_cache size 0 max 16384 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a94504740 1 Processor -- start 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a94504740 1 -- start start 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a94504740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558464fa4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a94504740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558464f9d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a94504740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 0x558464fc4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a94504740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1b38c0 con 0x558464fa2980 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a94504740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1b1190 con 0x558464fad240 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a94504740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1aefe0 con 0x558464fa2060 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558464f9d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558464f9d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34446/0 (socket says 172.21.15.1:34446) 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8affd640 1 -- 172.21.15.1:0/2877811769 learned_addr learned my addr 172.21.15.1:0/2877811769 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8bfff640 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 0x558464fc4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558464fa4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8affd640 1 -- 172.21.15.1:0/2877811769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 msgr2=0x558464fa4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8affd640 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558464fa4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8affd640 1 -- 172.21.15.1:0/2877811769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 msgr2=0x558464fc4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8affd640 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 0x558464fc4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8affd640 1 -- 172.21.15.1:0/2877811769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558464f9f280 con 0x558464fa2980 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558464fa4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.178+0000 7f7a8affd640 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558464f9d640 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7a80000c00 tx=0x7f7a8008e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fa1a1141a6293 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a717fa640 1 -- 172.21.15.1:0/2877811769 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80049040 con 0x558464fa2980 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a717fa640 1 -- 172.21.15.1:0/2877811769 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a800bc690 con 0x558464fa2980 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a717fa640 1 -- 172.21.15.1:0/2877811769 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800b8d00 con 0x558464fa2980 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a94504740 1 -- 172.21.15.1:0/2877811769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 msgr2=0x558464f9d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a94504740 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558464f9d640 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7a80000c00 tx=0x7f7a8008e460 comp rx=0 tx=0).stop 2024-09-16T00:40:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a94504740 1 -- 172.21.15.1:0/2877811769 shutdown_connections 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a94504740 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 0x558464fc4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a94504740 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558464f9d640 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a94504740 1 --2- 172.21.15.1:0/2877811769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558464fa4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a94504740 1 -- 172.21.15.1:0/2877811769 >> 172.21.15.1:0/2877811769 conn(0x558464f9e5f0 msgr2=0x558464f9cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a94504740 1 -- 172.21.15.1:0/2877811769 shutdown_connections 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.179+0000 7f7a94504740 1 -- 172.21.15.1:0/2877811769 wait complete. 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a94504740 1 -- start start 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a94504740 1 Processor -- start 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a94504740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558465020ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a94504740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558465021420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a94504740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 0x558464fb1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a94504740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1af650 con 0x558464fa2980 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a94504740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1b0ac0 con 0x558464fad240 2024-09-16T00:40:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a94504740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1afd20 con 0x558464fa2060 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a94504740 10 client.0 _ll_register_callbacks cb 0x7f7a84003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558465021420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558465021420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34450/0 (socket says 172.21.15.1:34450) 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a8affd640 1 -- 172.21.15.1:0/807132402 learned_addr learned my addr 172.21.15.1:0/807132402 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.180+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558465020ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a8bfff640 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 0x558464fb1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a8affd640 1 -- 172.21.15.1:0/807132402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 msgr2=0x558465020ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a8affd640 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558465020ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a8affd640 1 -- 172.21.15.1:0/807132402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 msgr2=0x558464fb1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a8affd640 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 0x558464fb1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a8affd640 1 -- 172.21.15.1:0/807132402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558464fb1f80 con 0x558464fa2980 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a8affd640 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558465021420 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f7a800bc390 tx=0x7f7a800bbff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd8b08f4eff97eb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a6affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:33.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a6affd640 1 -- 172.21.15.1:0/807132402 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800d4060 con 0x558464fa2980 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a6affd640 1 -- 172.21.15.1:0/807132402 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7a800b8d00 con 0x558464fa2980 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a94504740 1 -- 172.21.15.1:0/807132402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a800bc690 con 0x558464fa2980 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a6affd640 1 -- 172.21.15.1:0/807132402 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7a80049070 con 0x558464fa2980 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a94504740 10 client.18393 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a94504740 1 -- 172.21.15.1:0/807132402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558465028720 con 0x558464fa2980 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.181+0000 7f7a6affd640 1 -- 172.21.15.1:0/807132402 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a800bc690 con 0x558464fa2980 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a6affd640 1 -- 172.21.15.1:0/807132402 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7a8003bce0 con 0x558464fa2980 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a94504740 10 client.18393 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a94504740 10 client.18393 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a94504740 20 client.18393 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a94504740 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558465028df0 0x55846502b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a94504740 4 client.18393 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1c7a2282", "release": true, "await": true}] 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a94504740 1 -- 172.21.15.1:0/807132402 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1c7a2282", "release": true, "await": true}) -- 0x558464f0f0a0 con 0x558465028df0 2024-09-16T00:40:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558465028df0 0x55846502b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a8b7fe640 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558465028df0 0x55846502b210 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902d2739c8226433 server_cookie=54d6d04872aeac14 in_seq=0 out_seq=0 2024-09-16T00:40:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.182+0000 7f7a6affd640 10 client.18393 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a6affd640 1 -- 172.21.15.1:0/807132402 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558464f0f0a0 con 0x558465028df0 2024-09-16T00:40:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a6affd640 10 client.18393 handle_command_reply: tid=0 2024-09-16T00:40:33.204 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:40:33.204 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:40:33.204 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 44, 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "1c7a2282": { 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 44, 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 141.8, 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:40:33.205 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:40:33.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 client.18393 shutdown 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 -- 172.21.15.1:0/807132402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 msgr2=0x558465021420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558465021420 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f7a800bc390 tx=0x7f7a800bbff0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 -- 172.21.15.1:0/807132402 shutdown_connections 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558465028df0 0x55846502b210 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558464fad240 0x558464fb1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558464fa2980 0x558465021420 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 --2- 172.21.15.1:0/807132402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558464fa2060 0x558465020ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a6affd640 0 client.18393 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 -- 172.21.15.1:0/807132402 >> 172.21.15.1:0/807132402 conn(0x558464f9e5f0 msgr2=0x558464f9d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 -- 172.21.15.1:0/807132402 shutdown_connections 2024-09-16T00:40:33.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 1 -- 172.21.15.1:0/807132402 wait complete. 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a94504740 20 client.18393 trim_cache size 0 max 16384 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a9212b640 1 -- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7a740a3240 msgr2=0x7f7a740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a9212b640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7a740a3240 0x7f7a740a5700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a84005fd0 tx=0x7f7a84009be0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a9212b640 1 -- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c100600 msgr2=0x7f7a8c101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.185+0000 7f7a9212b640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c100600 0x7f7a8c101bf0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7a80004770 tx=0x7f7a80003b90 comp rx=0 tx=0).stop 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.186+0000 7f7a9212b640 1 -- 172.21.15.1:0/7384118 shutdown_connections 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.186+0000 7f7a9212b640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a8c106c80 0x7f7a8c102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.186+0000 7f7a9212b640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a8c100600 0x7f7a8c101bf0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.186+0000 7f7a9212b640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7a740a3240 0x7f7a740a5700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.186+0000 7f7a9212b640 1 --2- 172.21.15.1:0/7384118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a8c0ffc40 0x7f7a8c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.186+0000 7f7a9212b640 1 -- 172.21.15.1:0/7384118 >> 172.21.15.1:0/7384118 conn(0x7f7a8c0fba90 msgr2=0x7f7a8c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.186+0000 7f7a9212b640 1 -- 172.21.15.1:0/7384118 shutdown_connections 2024-09-16T00:40:33.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:33.186+0000 7f7a9212b640 1 -- 172.21.15.1:0/7384118 wait complete. 2024-09-16T00:40:33.984 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1c7a2282', total seconds elapsed: 25.3 2024-09-16T00:40:34.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:34 smithi019 ceph-mon[35186]: pgmap v715: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:40:34.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:34 smithi001 ceph-mon[30934]: pgmap v715: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:40:34.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:34 smithi153 ceph-mon[35538]: pgmap v715: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T00:40:36.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:36 smithi019 ceph-mon[35186]: pgmap v716: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-16T00:40:36.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:36 smithi001 ceph-mon[30934]: pgmap v716: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-16T00:40:36.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:36 smithi153 ceph-mon[35538]: pgmap v716: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-16T00:40:38.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:38 smithi019 ceph-mon[35186]: pgmap v717: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:40:38.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:38 smithi001 ceph-mon[30934]: pgmap v717: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:40:38.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:38 smithi153 ceph-mon[35538]: pgmap v717: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-16T00:40:40.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:40 smithi019 ceph-mon[35186]: pgmap v718: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-16T00:40:40.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:40 smithi001 ceph-mon[30934]: pgmap v718: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-16T00:40:40.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:40 smithi153 ceph-mon[35538]: pgmap v718: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-16T00:40:42.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:42 smithi019 ceph-mon[35186]: pgmap v719: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:40:42.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:42 smithi001 ceph-mon[30934]: pgmap v719: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:40:42.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:42 smithi153 ceph-mon[35538]: pgmap v719: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:40:44.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:44 smithi019 ceph-mon[35186]: pgmap v720: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:40:44.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:44 smithi001 ceph-mon[30934]: pgmap v720: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:40:44.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:44 smithi153 ceph-mon[35538]: pgmap v720: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:40:46.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:46 smithi019 ceph-mon[35186]: pgmap v721: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2024-09-16T00:40:46.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:46 smithi001 ceph-mon[30934]: pgmap v721: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2024-09-16T00:40:46.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:46 smithi153 ceph-mon[35538]: pgmap v721: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2024-09-16T00:40:48.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:48 smithi019 ceph-mon[35186]: pgmap v722: 129 pgs: 129 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:40:48.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:48 smithi001 ceph-mon[30934]: pgmap v722: 129 pgs: 129 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:40:48.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:48 smithi153 ceph-mon[35538]: pgmap v722: 129 pgs: 129 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-16T00:40:50.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:50 smithi019 ceph-mon[35186]: pgmap v723: 129 pgs: 129 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:40:50.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:50 smithi001 ceph-mon[30934]: pgmap v723: 129 pgs: 129 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:40:50.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:50 smithi153 ceph-mon[35538]: pgmap v723: 129 pgs: 129 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:40:52.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:52 smithi019 ceph-mon[35186]: pgmap v724: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:40:52.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:52 smithi001 ceph-mon[30934]: pgmap v724: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:40:52.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:52 smithi153 ceph-mon[35538]: pgmap v724: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:40:52.960 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:40:53.233 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:53.332 INFO:tasks.workunit.client.0.smithi001.stdout:375/600: open fd = 3 2024-09-16T00:40:53.332 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.332 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.333 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.333 INFO:tasks.workunit.client.0.smithi001.stdout:376/600: open fd = 3 2024-09-16T00:40:53.333 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.333 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.333 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.333 INFO:tasks.workunit.client.0.smithi001.stdout:377/600: open fd = 3 2024-09-16T00:40:53.334 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.334 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.334 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.334 INFO:tasks.workunit.client.0.smithi001.stdout:378/600: open fd = 3 2024-09-16T00:40:53.334 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.334 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.335 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.335 INFO:tasks.workunit.client.0.smithi001.stdout:379/600: open fd = 3 2024-09-16T00:40:53.335 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.335 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.335 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.335 INFO:tasks.workunit.client.0.smithi001.stdout:380/600: open fd = 3 2024-09-16T00:40:53.335 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.335 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.336 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.336 INFO:tasks.workunit.client.0.smithi001.stdout:381/600: open fd = 3 2024-09-16T00:40:53.336 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.336 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.336 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.336 INFO:tasks.workunit.client.0.smithi001.stdout:382/600: open fd = 3 2024-09-16T00:40:53.336 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.336 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.337 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.337 INFO:tasks.workunit.client.0.smithi001.stdout:383/600: open fd = 3 2024-09-16T00:40:53.337 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.337 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.337 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.337 INFO:tasks.workunit.client.0.smithi001.stdout:384/600: open fd = 3 2024-09-16T00:40:53.337 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.337 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.337 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.338 INFO:tasks.workunit.client.0.smithi001.stdout:385/600: open fd = 3 2024-09-16T00:40:53.338 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.338 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.338 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.338 INFO:tasks.workunit.client.0.smithi001.stdout:386/600: open fd = 3 2024-09-16T00:40:53.338 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.338 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.338 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.338 INFO:tasks.workunit.client.0.smithi001.stdout:387/600: open fd = 3 2024-09-16T00:40:53.339 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.339 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.339 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.339 INFO:tasks.workunit.client.0.smithi001.stdout:388/600: open fd = 3 2024-09-16T00:40:53.339 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.339 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.339 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.339 INFO:tasks.workunit.client.0.smithi001.stdout:389/600: open fd = 3 2024-09-16T00:40:53.340 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.340 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.340 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.340 INFO:tasks.workunit.client.0.smithi001.stdout:390/600: open fd = 3 2024-09-16T00:40:53.340 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.340 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.340 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.340 INFO:tasks.workunit.client.0.smithi001.stdout:391/600: open fd = 3 2024-09-16T00:40:53.341 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.341 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.341 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.341 INFO:tasks.workunit.client.0.smithi001.stdout:392/600: open fd = 3 2024-09-16T00:40:53.341 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:393/600: open fd = 3 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:394/600: open fd = 3 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:395/600: open fd = 3 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.342 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:396/600: open fd = 3 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:397/600: open fd = 3 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:398/600: open fd = 3 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:399/600: open fd = 3 2024-09-16T00:40:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:400/600: open fd = 3 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:401/600: open fd = 3 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:402/600: open fd = 3 2024-09-16T00:40:53.344 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:403/600: open fd = 3 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:404/600: open fd = 3 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:405/600: open fd = 3 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.345 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:406/600: open fd = 3 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:407/600: open fd = 3 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:408/600: open fd = 3 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.346 INFO:tasks.workunit.client.0.smithi001.stdout:409/600: open fd = 3 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:410/600: open fd = 3 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:411/600: open fd = 3 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:412/600: open fd = 3 2024-09-16T00:40:53.347 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:413/600: open fd = 3 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:414/600: open fd = 3 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:415/600: open fd = 3 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.348 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:416/600: open fd = 3 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:417/600: open fd = 3 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:418/600: open fd = 3 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:419/600: open fd = 3 2024-09-16T00:40:53.349 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:420/600: open fd = 3 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:421/600: open fd = 3 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:422/600: open fd = 3 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:423/600: open fd = 3 2024-09-16T00:40:53.350 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:424/600: open fd = 3 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:425/600: open fd = 3 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:426/600: open fd = 3 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.351 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:427/600: open fd = 3 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:428/600: open fd = 3 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:429/600: open fd = 3 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.352 INFO:tasks.workunit.client.0.smithi001.stdout:430/600: open fd = 3 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:431/600: open fd = 3 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:432/600: open fd = 3 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.353 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:433/600: open fd = 3 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:434/600: open fd = 3 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:435/600: open fd = 3 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:436/600: open fd = 3 2024-09-16T00:40:53.354 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.355 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:40:53.355 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.770+0000 7f2df255f640 1 -- 172.21.15.1:0/2788444094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec10c170 msgr2=0x7f2dec10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.770+0000 7f2df255f640 1 --2- 172.21.15.1:0/2788444094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec10c170 0x7f2dec10e610 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009ca0 tx=0x7f2de402a950 comp rx=0 tx=0).stop 2024-09-16T00:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.770+0000 7f2df255f640 1 -- 172.21.15.1:0/2788444094 shutdown_connections 2024-09-16T00:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.770+0000 7f2df255f640 1 --2- 172.21.15.1:0/2788444094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec10c170 0x7f2dec10e610 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.770+0000 7f2df255f640 1 --2- 172.21.15.1:0/2788444094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec103660 0x7f2dec10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.770+0000 7f2df255f640 1 --2- 172.21.15.1:0/2788444094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec102d40 0x7f2dec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.770+0000 7f2df255f640 1 -- 172.21.15.1:0/2788444094 >> 172.21.15.1:0/2788444094 conn(0x7f2dec0fc7a0 msgr2=0x7f2dec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.771+0000 7f2df255f640 1 -- 172.21.15.1:0/2788444094 shutdown_connections 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.771+0000 7f2df255f640 1 -- 172.21.15.1:0/2788444094 wait complete. 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.771+0000 7f2df255f640 1 Processor -- start 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.771+0000 7f2df255f640 1 -- start start 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2df255f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec102d40 0x7f2dec1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2df255f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec103660 0x7f2dec100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2df255f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec10c170 0x7f2dec101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2df255f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec111f70 con 0x7f2dec103660 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2df255f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec111df0 con 0x7f2dec102d40 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2df255f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1120f0 con 0x7f2dec10c170 2024-09-16T00:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2deb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec103660 0x7f2dec100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2deb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec103660 0x7f2dec100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49678/0 (socket says 172.21.15.1:49678) 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2deb7fe640 1 -- 172.21.15.1:0/3298697206 learned_addr learned my addr 172.21.15.1:0/3298697206 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2debfff640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec102d40 0x7f2dec1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2df0afc640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec10c170 0x7f2dec101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2deb7fe640 1 -- 172.21.15.1:0/3298697206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec10c170 msgr2=0x7f2dec101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec10c170 0x7f2dec101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2deb7fe640 1 -- 172.21.15.1:0/3298697206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec102d40 msgr2=0x7f2dec1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec102d40 0x7f2dec1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2deb7fe640 1 -- 172.21.15.1:0/3298697206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec1a68d0 con 0x7f2dec103660 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2debfff640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec102d40 0x7f2dec1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.772+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec103660 0x7f2dec100ee0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2de000d950 tx=0x7f2de000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.773+0000 7f2de97fa640 1 -- 172.21.15.1:0/3298697206 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0017030 con 0x7f2dec103660 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.773+0000 7f2df255f640 1 -- 172.21.15.1:0/3298697206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dec1a6bc0 con 0x7f2dec103660 2024-09-16T00:40:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.773+0000 7f2de97fa640 1 -- 172.21.15.1:0/3298697206 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de0004020 con 0x7f2dec103660 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.773+0000 7f2de97fa640 1 -- 172.21.15.1:0/3298697206 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de00047a0 con 0x7f2dec103660 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.773+0000 7f2df255f640 1 -- 172.21.15.1:0/3298697206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dec1a7150 con 0x7f2dec103660 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.775+0000 7f2de97fa640 1 -- 172.21.15.1:0/3298697206 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2de00041d0 con 0x7f2dec103660 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.775+0000 7f2df4938740 1 Processor -- start 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.775+0000 7f2df4938740 1 -- start start 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.775+0000 7f2df4938740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fb4a0 0x562d725fb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.775+0000 7f2df4938740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72606140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.775+0000 7f2df4938740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d72606680 0x562d72608a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.775+0000 7f2df4938740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d725612e0 con 0x562d72606680 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.775+0000 7f2df4938740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562d72508ce0 con 0x562d725fbdc0 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.775+0000 7f2df4938740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562d7255d2a0 con 0x562d725fb4a0 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2deb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d72606680 0x562d72608a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2deb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d72606680 0x562d72608a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49692/0 (socket says 172.21.15.1:49692) 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2deb7fe640 1 -- 172.21.15.1:0/1284674681 learned_addr learned my addr 172.21.15.1:0/1284674681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2debfff640 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72606140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2de97fa640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2dd40a3240 0x7f2dd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2df0afc640 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fb4a0 0x562d725fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2de97fa640 1 -- 172.21.15.1:0/3298697206 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2de00cf5d0 con 0x7f2dec103660 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2deb7fe640 1 -- 172.21.15.1:0/1284674681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fb4a0 msgr2=0x562d725fb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fb4a0 0x562d725fb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2deb7fe640 1 -- 172.21.15.1:0/1284674681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 msgr2=0x562d72606140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72606140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2deb7fe640 1 -- 172.21.15.1:0/1284674681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d724ed440 con 0x562d72606680 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2debfff640 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72606140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2df0afc640 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fb4a0 0x562d725fb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2df0afc640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2dd40a3240 0x7f2dd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d72606680 0x562d72608a70 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2de009b490 tx=0x7f2de000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74755c18b80f29ee server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2dd27fc640 1 -- 172.21.15.1:0/1284674681 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de009b560 con 0x562d72606680 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.776+0000 7f2dd27fc640 1 -- 172.21.15.1:0/1284674681 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de009be90 con 0x562d72606680 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2dd27fc640 1 -- 172.21.15.1:0/1284674681 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de009b0f0 con 0x562d72606680 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df4938740 1 -- 172.21.15.1:0/1284674681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d72606680 msgr2=0x562d72608a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df0afc640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2dd40a3240 0x7f2dd40a5700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2de4005e30 tx=0x7f2de40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df4938740 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d72606680 0x562d72608a70 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2de009b490 tx=0x7f2de000bf90 comp rx=0 tx=0).stop 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df4938740 1 -- 172.21.15.1:0/1284674681 shutdown_connections 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df4938740 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d72606680 0x562d72608a70 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df4938740 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72606140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df4938740 1 --2- 172.21.15.1:0/1284674681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fb4a0 0x562d725fb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df4938740 1 -- 172.21.15.1:0/1284674681 >> 172.21.15.1:0/1284674681 conn(0x562d725f7a30 msgr2=0x562d725f7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df4938740 1 -- 172.21.15.1:0/1284674681 shutdown_connections 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.777+0000 7f2df4938740 1 -- 172.21.15.1:0/1284674681 wait complete. 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2df4938740 1 -- start start 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2df4938740 1 Processor -- start 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2df4938740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72605fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2df4938740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fbdc0 0x562d72603290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2df4938740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d72606680 0x562d726037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2df4938740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d726091f0 con 0x562d725fb4a0 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2df4938740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562d7251c050 con 0x562d72606680 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2df4938740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562d72609370 con 0x562d725fbdc0 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2df4938740 10 client.0 _ll_register_callbacks cb 0x562d7255c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2debfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72605fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2debfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72605fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49704/0 (socket says 172.21.15.1:49704) 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.778+0000 7f2debfff640 1 -- 172.21.15.1:0/1972950575 learned_addr learned my addr 172.21.15.1:0/1972950575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fbdc0 0x562d72603290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2df0afc640 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d72606680 0x562d726037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2debfff640 1 -- 172.21.15.1:0/1972950575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fbdc0 msgr2=0x562d72603290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2debfff640 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fbdc0 0x562d72603290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2debfff640 1 -- 172.21.15.1:0/1972950575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d72606680 msgr2=0x562d726037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2debfff640 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d72606680 0x562d726037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2debfff640 1 -- 172.21.15.1:0/1972950575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d722d23c0 con 0x562d725fb4a0 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fbdc0 0x562d72603290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2debfff640 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72605fe0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc010f10 tx=0x7f2ddc00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b6d914bff34ac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2dcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/1972950575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc004020 con 0x562d725fb4a0 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/1972950575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2ddc0041c0 con 0x562d725fb4a0 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2df4938740 1 -- 172.21.15.1:0/1972950575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de009be90 con 0x562d725fb4a0 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/1972950575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ddc0043c0 con 0x562d725fb4a0 2024-09-16T00:40:53.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2df4938740 10 client.18405 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.779+0000 7f2df4938740 1 -- 172.21.15.1:0/1972950575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d72614c50 con 0x562d725fb4a0 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.780+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/1972950575 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de009be90 con 0x562d725fb4a0 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.780+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/1972950575 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2ddc0196b0 con 0x562d725fb4a0 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.780+0000 7f2df4938740 10 client.18405 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.780+0000 7f2df4938740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.780+0000 7f2df4938740 20 client.18405 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.780+0000 7f2df4938740 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562d72616110 0x562d72618590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.780+0000 7f2df4938740 4 client.18405 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.780+0000 7f2df4938740 1 -- 172.21.15.1:0/1972950575 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d7256e850 con 0x562d72616110 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.780+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562d72616110 0x562d72618590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.781+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562d72616110 0x562d72618590 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5bf8342098a854a server_cookie=9364f708459b24b1 in_seq=0 out_seq=0 2024-09-16T00:40:53.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.781+0000 7f2dcb7fe640 10 client.18405 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:53.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.781+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/1972950575 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d7256e850 con 0x562d72616110 2024-09-16T00:40:53.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.781+0000 7f2dcb7fe640 10 client.18405 handle_command_reply: tid=0 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.781+0000 7f2df4938740 1 client.18405 shutdown 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 -- 172.21.15.1:0/1972950575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 msgr2=0x562d72605fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72605fe0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc010f10 tx=0x7f2ddc00f7f0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 -- 172.21.15.1:0/1972950575 shutdown_connections 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562d72616110 0x562d72618590 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d72606680 0x562d726037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d725fbdc0 0x562d72603290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 --2- 172.21.15.1:0/1972950575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72605fe0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 -- 172.21.15.1:0/1972950575 >> 172.21.15.1:0/1972950575 conn(0x562d725f7a30 msgr2=0x562d725fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2dcb7fe640 0 client.18405 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 -- 172.21.15.1:0/1972950575 shutdown_connections 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 1 -- 172.21.15.1:0/1972950575 wait complete. 2024-09-16T00:40:53.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.782+0000 7f2df4938740 20 client.18405 trim_cache size 0 max 16384 2024-09-16T00:40:53.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2df4938740 1 Processor -- start 2024-09-16T00:40:53.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2df4938740 1 -- start start 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2df4938740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72612480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2df4938740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d725f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2df4938740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 0x562d725fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2df4938740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1b3910 con 0x562d725fb4a0 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2df4938740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1b11e0 con 0x562d725fbdc0 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2df4938740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1af030 con 0x562d72606680 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2debfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72612480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2debfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72612480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49718/0 (socket says 172.21.15.1:49718) 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.799+0000 7f2debfff640 1 -- 172.21.15.1:0/3357958413 learned_addr learned my addr 172.21.15.1:0/3357958413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d725f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2debfff640 1 -- 172.21.15.1:0/3357958413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 msgr2=0x562d725fe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2df0afc640 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 0x562d725fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2debfff640 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 0x562d725fe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2debfff640 1 -- 172.21.15.1:0/3357958413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 msgr2=0x562d725f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:53.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2debfff640 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d725f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2debfff640 1 -- 172.21.15.1:0/3357958413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d725f8bc0 con 0x562d725fb4a0 2024-09-16T00:40:53.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d725f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:53.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2df0afc640 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 0x562d725fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:53.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2debfff640 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72612480 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00a500 tx=0x7f2ddc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f8d70359f5ff43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:53.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2dd1ffb640 1 -- 172.21.15.1:0/3357958413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc02f070 con 0x562d725fb4a0 2024-09-16T00:40:53.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2dd1ffb640 1 -- 172.21.15.1:0/3357958413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc005070 con 0x562d725fb4a0 2024-09-16T00:40:53.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.800+0000 7f2dd1ffb640 1 -- 172.21.15.1:0/3357958413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc024030 con 0x562d725fb4a0 2024-09-16T00:40:53.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 -- 172.21.15.1:0/3357958413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 msgr2=0x562d72612480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72612480 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00a500 tx=0x7f2ddc01b000 comp rx=0 tx=0).stop 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 -- 172.21.15.1:0/3357958413 shutdown_connections 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 0x562d725fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d725f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 --2- 172.21.15.1:0/3357958413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d72612480 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 -- 172.21.15.1:0/3357958413 >> 172.21.15.1:0/3357958413 conn(0x562d725f7a30 msgr2=0x562d725fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 -- 172.21.15.1:0/3357958413 shutdown_connections 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 -- 172.21.15.1:0/3357958413 wait complete. 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 -- start start 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.801+0000 7f2df4938740 1 Processor -- start 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2df4938740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d7260eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2df4938740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72600210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2df4938740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 0x562d726009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2df4938740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1af6a0 con 0x562d725fb4a0 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2df4938740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1b0b10 con 0x562d725fbdc0 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2df4938740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1afd70 con 0x562d72606680 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2df4938740 10 client.0 _ll_register_callbacks cb 0x7f2ddc00ab70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2debfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d7260eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2debfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d7260eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49732/0 (socket says 172.21.15.1:49732) 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2debfff640 1 -- 172.21.15.1:0/664842305 learned_addr learned my addr 172.21.15.1:0/664842305 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2df0afc640 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 0x562d726009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72600210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2debfff640 1 -- 172.21.15.1:0/664842305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 msgr2=0x562d726009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2debfff640 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 0x562d726009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2debfff640 1 -- 172.21.15.1:0/664842305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 msgr2=0x562d72600210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2debfff640 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72600210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2debfff640 1 -- 172.21.15.1:0/664842305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d7267fe80 con 0x562d725fb4a0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72600210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2debfff640 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d7260eeb0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc005630 tx=0x7f2ddc005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2a79e67ea9fad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2dcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.802+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/664842305 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc024070 con 0x562d725fb4a0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2df4938740 1 -- 172.21.15.1:0/664842305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ddc005070 con 0x562d725fb4a0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/664842305 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2ddc0193e0 con 0x562d725fb4a0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/664842305 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ddc03e050 con 0x562d725fb4a0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2df4938740 10 client.18411 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2df4938740 1 -- 172.21.15.1:0/664842305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d72680fc0 con 0x562d725fb4a0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/664842305 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc005070 con 0x562d725fb4a0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/664842305 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2ddc029480 con 0x562d725fb4a0 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2df4938740 10 client.18411 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2df4938740 10 client.18411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2df4938740 20 client.18411 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2df4938740 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562d72681670 0x562d72683a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2df4938740 4 client.18411 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.803+0000 7f2df4938740 1 -- 172.21.15.1:0/664842305 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562d72568d60 con 0x562d72681670 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562d72681670 0x562d72683a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2deb7fe640 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562d72681670 0x562d72683a90 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d5717b70c5127e3 server_cookie=4db1eb4b00d6baf9 in_seq=0 out_seq=0 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2dcb7fe640 10 client.18411 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/664842305 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562d72568d60 con 0x562d72681670 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2dcb7fe640 10 client.18411 handle_command_reply: tid=0 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2df4938740 1 client.18411 shutdown 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2df4938740 1 -- 172.21.15.1:0/664842305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 msgr2=0x562d7260eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2df4938740 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d7260eeb0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc005630 tx=0x7f2ddc005660 comp rx=0 tx=0).stop 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2df4938740 1 -- 172.21.15.1:0/664842305 shutdown_connections 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2df4938740 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562d72681670 0x562d72683a90 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2df4938740 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d72606680 0x562d726009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2df4938740 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d725fbdc0 0x562d72600210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2df4938740 1 --2- 172.21.15.1:0/664842305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d725fb4a0 0x562d7260eeb0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2df4938740 1 -- 172.21.15.1:0/664842305 >> 172.21.15.1:0/664842305 conn(0x562d725f7a30 msgr2=0x562d725f6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.804+0000 7f2dcb7fe640 0 client.18411 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df4938740 1 -- 172.21.15.1:0/664842305 shutdown_connections 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df4938740 1 -- 172.21.15.1:0/664842305 wait complete. 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df4938740 20 client.18411 trim_cache size 0 max 16384 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:40:53.810 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 -- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2dd40a3240 msgr2=0x7f2dd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2dd40a3240 0x7f2dd40a5700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2de4005e30 tx=0x7f2de40095c0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 -- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec103660 msgr2=0x7f2dec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec103660 0x7f2dec100ee0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2de000d950 tx=0x7f2de000de10 comp rx=0 tx=0).stop 2024-09-16T00:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 -- 172.21.15.1:0/3298697206 shutdown_connections 2024-09-16T00:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2dd40a3240 0x7f2dd40a5700 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec10c170 0x7f2dec101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dec103660 0x7f2dec100ee0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 --2- 172.21.15.1:0/3298697206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec102d40 0x7f2dec1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 -- 172.21.15.1:0/3298697206 >> 172.21.15.1:0/3298697206 conn(0x7f2dec0fc7a0 msgr2=0x7f2dec0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 -- 172.21.15.1:0/3298697206 shutdown_connections 2024-09-16T00:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:53.805+0000 7f2df255f640 1 -- 172.21.15.1:0/3298697206 wait complete. 2024-09-16T00:40:54.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:54 smithi019 ceph-mon[35186]: pgmap v725: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:40:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:54 smithi001 ceph-mon[30934]: pgmap v725: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:40:54.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:54 smithi153 ceph-mon[35538]: pgmap v725: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:40:54.780 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:40:54.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:55.046 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.530+0000 7f49c53e0640 1 -- 172.21.15.1:0/2506695797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0106f10 msgr2=0x7f49c010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.530+0000 7f49c53e0640 1 --2- 172.21.15.1:0/2506695797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0106f10 0x7f49c010ea90 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009ca0 tx=0x7f49b802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.530+0000 7f49c53e0640 1 -- 172.21.15.1:0/2506695797 shutdown_connections 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.530+0000 7f49c53e0640 1 --2- 172.21.15.1:0/2506695797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0106f10 0x7f49c010ea90 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.530+0000 7f49c53e0640 1 --2- 172.21.15.1:0/2506695797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c0102790 0x7f49c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.530+0000 7f49c53e0640 1 --2- 172.21.15.1:0/2506695797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c0101de0 0x7f49c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.530+0000 7f49c53e0640 1 -- 172.21.15.1:0/2506695797 >> 172.21.15.1:0/2506695797 conn(0x7f49c00fd8c0 msgr2=0x7f49c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.530+0000 7f49c53e0640 1 -- 172.21.15.1:0/2506695797 shutdown_connections 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.530+0000 7f49c53e0640 1 -- 172.21.15.1:0/2506695797 wait complete. 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49c53e0640 1 Processor -- start 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49c53e0640 1 -- start start 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49c53e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0101de0 0x7f49c0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49c53e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c0102790 0x7f49c0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49c53e0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c0106f10 0x7f49c019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49c53e0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0111f70 con 0x7f49c0101de0 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49c53e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0111df0 con 0x7f49c0102790 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49c53e0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49c01120f0 con 0x7f49c0106f10 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0101de0 0x7f49c0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0101de0 0x7f49c0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49746/0 (socket says 172.21.15.1:49746) 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49beffd640 1 -- 172.21.15.1:0/6530013 learned_addr learned my addr 172.21.15.1:0/6530013 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.531+0000 7f49be7fc640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c0102790 0x7f49c0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:55.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49bf7fe640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c0106f10 0x7f49c019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49beffd640 1 -- 172.21.15.1:0/6530013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c0106f10 msgr2=0x7f49c019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49beffd640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c0106f10 0x7f49c019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49beffd640 1 -- 172.21.15.1:0/6530013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c0102790 msgr2=0x7f49c0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49beffd640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c0102790 0x7f49c0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49beffd640 1 -- 172.21.15.1:0/6530013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c019d2e0 con 0x7f49c0101de0 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49be7fc640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c0102790 0x7f49c0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49beffd640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0101de0 0x7f49c0198330 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f49b000c920 tx=0x7f49b000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49a7fff640 1 -- 172.21.15.1:0/6530013 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0007c70 con 0x7f49c0101de0 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c019d5d0 con 0x7f49c0101de0 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49a7fff640 1 -- 172.21.15.1:0/6530013 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b0007e10 con 0x7f49c0101de0 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49a7fff640 1 -- 172.21.15.1:0/6530013 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0018890 con 0x7f49c0101de0 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.532+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c01a4eb0 con 0x7f49c0101de0 2024-09-16T00:40:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.533+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c01021c0 con 0x7f49c0101de0 2024-09-16T00:40:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.534+0000 7f49a7fff640 1 -- 172.21.15.1:0/6530013 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f49b0018a70 con 0x7f49c0101de0 2024-09-16T00:40:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.535+0000 7f49a7fff640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49a80a3240 0x7f49a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.535+0000 7f49be7fc640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49a80a3240 0x7f49a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.535+0000 7f49a7fff640 1 -- 172.21.15.1:0/6530013 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f49b00cff70 con 0x7f49c0101de0 2024-09-16T00:40:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.535+0000 7f49be7fc640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49a80a3240 0x7f49a80a5700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f49b4005e00 tx=0x7f49b4005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.539+0000 7f49a7fff640 1 -- 172.21.15.1:0/6530013 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49b0014030 con 0x7f49c0101de0 2024-09-16T00:40:55.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.750+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49c0109b60 con 0x7f49c0101de0 2024-09-16T00:40:55.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.750+0000 7f49a7fff640 1 -- 172.21.15.1:0/6530013 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f49b000de20 con 0x7f49c0101de0 2024-09-16T00:40:55.751 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:40:55.751 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:40:55.751 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:40:55.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49a80a3240 msgr2=0x7f49a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49a80a3240 0x7f49a80a5700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f49b4005e00 tx=0x7f49b4005c70 comp rx=0 tx=0).stop 2024-09-16T00:40:55.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0101de0 msgr2=0x7f49c0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0101de0 0x7f49c0198330 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f49b000c920 tx=0x7f49b000cde0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 shutdown_connections 2024-09-16T00:40:55.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49a80a3240 0x7f49a80a5700 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49c0106f10 0x7f49c019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49c0102790 0x7f49c0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 --2- 172.21.15.1:0/6530013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c0101de0 0x7f49c0198330 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 >> 172.21.15.1:0/6530013 conn(0x7f49c00fd8c0 msgr2=0x7f49c00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:55.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 shutdown_connections 2024-09-16T00:40:55.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:55.753+0000 7f49c53e0640 1 -- 172.21.15.1:0/6530013 wait complete. 2024-09-16T00:40:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:56 smithi001 ceph-mon[30934]: pgmap v726: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:40:56.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:56 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/6530013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:56.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:40:56.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:56 smithi019 ceph-mon[35186]: pgmap v726: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:40:56.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:56 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/6530013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:56.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:56 smithi153 ceph-mon[35538]: pgmap v726: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:40:56.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:56 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/6530013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.343+0000 7fa50127a640 1 -- 172.21.15.1:0/2933643046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc101dd0 msgr2=0x7fa4fc1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.343+0000 7fa50127a640 1 --2- 172.21.15.1:0/2933643046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc101dd0 0x7fa4fc1021b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009900 tx=0x7fa4ec02a410 comp rx=0 tx=0).stop 2024-09-16T00:40:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.343+0000 7fa50127a640 1 -- 172.21.15.1:0/2933643046 shutdown_connections 2024-09-16T00:40:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.343+0000 7fa50127a640 1 --2- 172.21.15.1:0/2933643046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4fc106f00 0x7fa4fc10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.343+0000 7fa50127a640 1 --2- 172.21.15.1:0/2933643046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc102780 0x7fa4fc1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.343+0000 7fa50127a640 1 --2- 172.21.15.1:0/2933643046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc101dd0 0x7fa4fc1021b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.343+0000 7fa50127a640 1 -- 172.21.15.1:0/2933643046 >> 172.21.15.1:0/2933643046 conn(0x7fa4fc0fd8b0 msgr2=0x7fa4fc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.343+0000 7fa50127a640 1 -- 172.21.15.1:0/2933643046 shutdown_connections 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.343+0000 7fa50127a640 1 -- 172.21.15.1:0/2933643046 wait complete. 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa50127a640 1 Processor -- start 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa50127a640 1 -- start start 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa50127a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4fc101dd0 0x7fa4fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa50127a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc102780 0x7fa4fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa50127a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc106f00 0x7fa4fc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa50127a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc111f20 con 0x7fa4fc102780 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa50127a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc111da0 con 0x7fa4fc101dd0 2024-09-16T00:40:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa50127a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1120a0 con 0x7fa4fc106f00 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc102780 0x7fa4fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc102780 0x7fa4fc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49760/0 (socket says 172.21.15.1:49760) 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4fa7fc640 1 -- 172.21.15.1:0/858079013 learned_addr learned my addr 172.21.15.1:0/858079013 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc106f00 0x7fa4fc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4faffd640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4fc101dd0 0x7fa4fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4fa7fc640 1 -- 172.21.15.1:0/858079013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc106f00 msgr2=0x7fa4fc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4fa7fc640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc106f00 0x7fa4fc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4fa7fc640 1 -- 172.21.15.1:0/858079013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4fc101dd0 msgr2=0x7fa4fc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4fa7fc640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4fc101dd0 0x7fa4fc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.344+0000 7fa4fa7fc640 1 -- 172.21.15.1:0/858079013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc1a6990 con 0x7fa4fc102780 2024-09-16T00:40:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.345+0000 7fa4fa7fc640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc102780 0x7fa4fc074a30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400d950 tx=0x7fa4e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.345+0000 7fa4f3fff640 1 -- 172.21.15.1:0/858079013 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4017070 con 0x7fa4fc102780 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.345+0000 7fa50127a640 1 -- 172.21.15.1:0/858079013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc1a6c80 con 0x7fa4fc102780 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.345+0000 7fa4f3fff640 1 -- 172.21.15.1:0/858079013 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e4009c30 con 0x7fa4fc102780 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.345+0000 7fa4f3fff640 1 -- 172.21.15.1:0/858079013 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400f6b0 con 0x7fa4fc102780 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.345+0000 7fa50127a640 1 -- 172.21.15.1:0/858079013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc1a71c0 con 0x7fa4fc102780 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.347+0000 7fa4f3fff640 1 -- 172.21.15.1:0/858079013 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa4e40109d0 con 0x7fa4fc102780 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.347+0000 7fa503653740 1 Processor -- start 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.347+0000 7fa503653740 1 -- start start 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa503653740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea4501090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa503653740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea45015d0 0x559ea4501a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa503653740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea450c370 0x559ea450e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa503653740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559ea44672f0 con 0x559ea45015d0 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa503653740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559ea440ecf0 con 0x559ea450c370 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa503653740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559ea44632b0 con 0x559ea4502b40 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4faffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea45015d0 0x559ea4501a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4faffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea45015d0 0x559ea4501a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49776/0 (socket says 172.21.15.1:49776) 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4faffd640 1 -- 172.21.15.1:0/1853232967 learned_addr learned my addr 172.21.15.1:0/1853232967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/1853232967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea4501090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4f3fff640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4e80a3240 0x7fa4e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4fa7fc640 1 --2- 172.21.15.1:0/1853232967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea450c370 0x559ea450e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4e80a3240 0x7fa4e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4faffd640 1 -- 172.21.15.1:0/1853232967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 msgr2=0x559ea4501090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4faffd640 1 --2- 172.21.15.1:0/1853232967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea4501090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4faffd640 1 -- 172.21.15.1:0/1853232967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea450c370 msgr2=0x559ea450e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4faffd640 1 --2- 172.21.15.1:0/1853232967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea450c370 0x559ea450e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.348+0000 7fa4faffd640 1 -- 172.21.15.1:0/1853232967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea43f33a0 con 0x559ea45015d0 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.349+0000 7fa4f3fff640 1 -- 172.21.15.1:0/858079013 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa4e40cfa30 con 0x7fa4fc102780 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.349+0000 7fa4faffd640 1 --2- 172.21.15.1:0/1853232967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea45015d0 0x559ea4501a10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006010 tx=0x7fa4ec031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc61c74b1d57e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.349+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4e80a3240 0x7fa4e80a5700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4005fd0 tx=0x7fa4f4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.349+0000 7fa4f17fa640 1 -- 172.21.15.1:0/1853232967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec03f070 con 0x559ea45015d0 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.349+0000 7fa4f17fa640 1 -- 172.21.15.1:0/1853232967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec0317b0 con 0x559ea45015d0 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.349+0000 7fa4f17fa640 1 -- 172.21.15.1:0/1853232967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec035070 con 0x559ea45015d0 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 -- 172.21.15.1:0/1853232967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea45015d0 msgr2=0x559ea4501a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 --2- 172.21.15.1:0/1853232967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea45015d0 0x559ea4501a10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006010 tx=0x7fa4ec031510 comp rx=0 tx=0).stop 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 -- 172.21.15.1:0/1853232967 shutdown_connections 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 --2- 172.21.15.1:0/1853232967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea450c370 0x559ea450e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 --2- 172.21.15.1:0/1853232967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea45015d0 0x559ea4501a10 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 --2- 172.21.15.1:0/1853232967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea4501090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 -- 172.21.15.1:0/1853232967 >> 172.21.15.1:0/1853232967 conn(0x559ea44ca310 msgr2=0x559ea44fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 -- 172.21.15.1:0/1853232967 shutdown_connections 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 -- 172.21.15.1:0/1853232967 wait complete. 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 -- start start 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.350+0000 7fa503653740 1 Processor -- start 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa503653740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea450bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa503653740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea4502b40 0x559ea4508f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa503653740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea450c370 0x559ea45094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa503653740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559ea450f010 con 0x559ea4502b40 2024-09-16T00:40:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa503653740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559ea4422060 con 0x559ea45015d0 2024-09-16T00:40:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa503653740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559ea450f190 con 0x559ea450c370 2024-09-16T00:40:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa503653740 10 client.0 _ll_register_callbacks cb 0x559ea41758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea4502b40 0x559ea4508f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.382 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:40:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea4502b40 0x559ea4508f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49790/0 (socket says 172.21.15.1:49790) 2024-09-16T00:40:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fa7fc640 1 -- 172.21.15.1:0/1810876606 learned_addr learned my addr 172.21.15.1:0/1810876606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea450c370 0x559ea45094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4faffd640 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea450bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fa7fc640 1 -- 172.21.15.1:0/1810876606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea450c370 msgr2=0x559ea45094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fa7fc640 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea450c370 0x559ea45094d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fa7fc640 1 -- 172.21.15.1:0/1810876606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 msgr2=0x559ea450bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fa7fc640 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea450bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fa7fc640 1 -- 172.21.15.1:0/1810876606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea41d83c0 con 0x559ea4502b40 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4fa7fc640 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea4502b40 0x559ea4508f70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4000c00 tx=0x7fa4e4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad75070306aa575 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.351+0000 7fa4e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa4e27fc640 1 -- 172.21.15.1:0/1810876606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4020070 con 0x559ea4502b40 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa4e27fc640 1 -- 172.21.15.1:0/1810876606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa4e4098260 con 0x559ea4502b40 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa503653740 1 -- 172.21.15.1:0/1810876606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ec0317b0 con 0x559ea4502b40 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa4e27fc640 1 -- 172.21.15.1:0/1810876606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa4e409ffa0 con 0x559ea4502b40 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa503653740 10 client.18429 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa503653740 1 -- 172.21.15.1:0/1810876606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ea451aad0 con 0x559ea4502b40 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa4e27fc640 1 -- 172.21.15.1:0/1810876606 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec0317b0 con 0x559ea4502b40 2024-09-16T00:40:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa4e27fc640 1 -- 172.21.15.1:0/1810876606 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa4e409b040 con 0x559ea4502b40 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa503653740 10 client.18429 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa503653740 10 client.18429 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa503653740 20 client.18429 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa503653740 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x559ea451bee0 0x559ea451e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa503653740 4 client.18429 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.352+0000 7fa503653740 1 -- 172.21.15.1:0/1810876606 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ea4474860 con 0x559ea451bee0 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.353+0000 7fa4faffd640 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x559ea451bee0 0x559ea451e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.353+0000 7fa4faffd640 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x559ea451bee0 0x559ea451e370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e38cabb0c294fd3 server_cookie=1a0c71d3e5da8b4 in_seq=0 out_seq=0 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.353+0000 7fa4e27fc640 10 client.18429 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa4e27fc640 1 -- 172.21.15.1:0/1810876606 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ea4474860 con 0x559ea451bee0 2024-09-16T00:40:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa4e27fc640 10 client.18429 handle_command_reply: tid=0 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 client.18429 shutdown 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 -- 172.21.15.1:0/1810876606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea4502b40 msgr2=0x559ea4508f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea4502b40 0x559ea4508f70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4000c00 tx=0x7fa4e4010d70 comp rx=0 tx=0).stop 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 -- 172.21.15.1:0/1810876606 shutdown_connections 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea450c370 0x559ea45094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea4502b40 0x559ea4508f70 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x559ea451bee0 0x559ea451e370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 --2- 172.21.15.1:0/1810876606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea450bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 -- 172.21.15.1:0/1810876606 >> 172.21.15.1:0/1810876606 conn(0x559ea44ca310 msgr2=0x559ea4503960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa4e27fc640 0 client.18429 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 -- 172.21.15.1:0/1810876606 shutdown_connections 2024-09-16T00:40:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 1 -- 172.21.15.1:0/1810876606 wait complete. 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.354+0000 7fa503653740 20 client.18429 trim_cache size 0 max 16384 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.367+0000 7fa503653740 1 Processor -- start 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.367+0000 7fa503653740 1 -- start start 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.367+0000 7fa503653740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea45182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.367+0000 7fa503653740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea4500c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.367+0000 7fa503653740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4507f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.367+0000 7fa503653740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1b38a0 con 0x559ea450c370 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.367+0000 7fa503653740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1b1170 con 0x559ea45015d0 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.367+0000 7fa503653740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1aefc0 con 0x559ea4502b40 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4507f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4507f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49794/0 (socket says 172.21.15.1:49794) 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fb7fe640 1 -- 172.21.15.1:0/2143185275 learned_addr learned my addr 172.21.15.1:0/2143185275 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fa7fc640 1 --2- 172.21.15.1:0/2143185275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea4500c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4faffd640 1 --2- 172.21.15.1:0/2143185275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea45182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fb7fe640 1 -- 172.21.15.1:0/2143185275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 msgr2=0x559ea4500c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/2143185275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea4500c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fb7fe640 1 -- 172.21.15.1:0/2143185275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 msgr2=0x559ea45182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/2143185275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea45182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fb7fe640 1 -- 172.21.15.1:0/2143185275 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea4508490 con 0x559ea450c370 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/2143185275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4507f50 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4014260 tx=0x7fa4f40097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc717c432b7d211 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/2143185275 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f401a040 con 0x559ea450c370 2024-09-16T00:40:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/2143185275 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f40119e0 con 0x559ea450c370 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.368+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/2143185275 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40054c0 con 0x559ea450c370 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 -- 172.21.15.1:0/2143185275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 msgr2=0x559ea4507f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 --2- 172.21.15.1:0/2143185275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4507f50 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4014260 tx=0x7fa4f40097c0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 -- 172.21.15.1:0/2143185275 shutdown_connections 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 --2- 172.21.15.1:0/2143185275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4507f50 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 --2- 172.21.15.1:0/2143185275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea4500c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 --2- 172.21.15.1:0/2143185275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea45182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 -- 172.21.15.1:0/2143185275 >> 172.21.15.1:0/2143185275 conn(0x559ea44ca310 msgr2=0x559ea4507840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 -- 172.21.15.1:0/2143185275 shutdown_connections 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 -- 172.21.15.1:0/2143185275 wait complete. 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.369+0000 7fa503653740 1 -- start start 2024-09-16T00:40:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa503653740 1 Processor -- start 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa503653740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea4514d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa503653740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea45236b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa503653740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4523df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa503653740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1af630 con 0x559ea450c370 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa503653740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1b0aa0 con 0x559ea45015d0 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa503653740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1afd00 con 0x559ea4502b40 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa503653740 10 client.0 _ll_register_callbacks cb 0x7fa4e409b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4523df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4523df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49804/0 (socket says 172.21.15.1:49804) 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fb7fe640 1 -- 172.21.15.1:0/2218873882 learned_addr learned my addr 172.21.15.1:0/2218873882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fa7fc640 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea45236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4faffd640 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea4514d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fb7fe640 1 -- 172.21.15.1:0/2218873882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 msgr2=0x559ea45236b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea45236b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fb7fe640 1 -- 172.21.15.1:0/2218873882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 msgr2=0x559ea4514d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea4514d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fb7fe640 1 -- 172.21.15.1:0/2218873882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea45127a0 con 0x559ea450c370 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4faffd640 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea4514d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4fb7fe640 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4523df0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4012fd0 tx=0x7fa4f4009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec12129b7cf891f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.370+0000 7fa4e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa4e27fc640 1 -- 172.21.15.1:0/2218873882 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f401a040 con 0x559ea450c370 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa4e27fc640 1 -- 172.21.15.1:0/2218873882 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa4f4011b90 con 0x559ea450c370 2024-09-16T00:40:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa503653740 1 -- 172.21.15.1:0/2218873882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f40119e0 con 0x559ea450c370 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa4e27fc640 1 -- 172.21.15.1:0/2218873882 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa4f403f050 con 0x559ea450c370 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa503653740 10 client.18435 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa503653740 1 -- 172.21.15.1:0/2218873882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ea4586d90 con 0x559ea450c370 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa4e27fc640 1 -- 172.21.15.1:0/2218873882 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f40119e0 con 0x559ea450c370 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa4e27fc640 1 -- 172.21.15.1:0/2218873882 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa4f4020040 con 0x559ea450c370 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa503653740 10 client.18435 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa503653740 10 client.18435 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa503653740 20 client.18435 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa503653740 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x559ea45873a0 0x559ea4589790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa503653740 4 client.18435 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:40:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa503653740 1 -- 172.21.15.1:0/2218873882 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ea446ed70 con 0x559ea45873a0 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.371+0000 7fa4faffd640 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x559ea45873a0 0x559ea4589790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa4e27fc640 10 client.18435 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa4faffd640 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x559ea45873a0 0x559ea4589790 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9355c62f7b37fae7 server_cookie=e62056651195389 in_seq=0 out_seq=0 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa4e27fc640 1 -- 172.21.15.1:0/2218873882 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ea446ed70 con 0x559ea45873a0 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa4e27fc640 10 client.18435 handle_command_reply: tid=0 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 client.18435 shutdown 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 -- 172.21.15.1:0/2218873882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 msgr2=0x559ea4523df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4523df0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4012fd0 tx=0x7fa4f4009720 comp rx=0 tx=0).stop 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 -- 172.21.15.1:0/2218873882 shutdown_connections 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea450c370 0x559ea4523df0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ea4502b40 0x559ea45236b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x559ea45873a0 0x559ea4589790 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 --2- 172.21.15.1:0/2218873882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ea45015d0 0x559ea4514d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 -- 172.21.15.1:0/2218873882 >> 172.21.15.1:0/2218873882 conn(0x559ea44ca310 msgr2=0x559ea450a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa4e27fc640 0 client.18435 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 -- 172.21.15.1:0/2218873882 shutdown_connections 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.372+0000 7fa503653740 1 -- 172.21.15.1:0/2218873882 wait complete. 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa503653740 20 client.18435 trim_cache size 0 max 16384 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 -- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4e80a3240 msgr2=0x7fa4e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4e80a3240 0x7fa4e80a5700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4005fd0 tx=0x7fa4f4004ab0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 -- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc102780 msgr2=0x7fa4fc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc102780 0x7fa4fc074a30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400d950 tx=0x7fa4e400de10 comp rx=0 tx=0).stop 2024-09-16T00:40:57.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 -- 172.21.15.1:0/858079013 shutdown_connections 2024-09-16T00:40:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa4e80a3240 0x7fa4e80a5700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc106f00 0x7fa4fc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc102780 0x7fa4fc074a30 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 --2- 172.21.15.1:0/858079013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4fc101dd0 0x7fa4fc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 -- 172.21.15.1:0/858079013 >> 172.21.15.1:0/858079013 conn(0x7fa4fc0fd8b0 msgr2=0x7fa4fc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 -- 172.21.15.1:0/858079013 shutdown_connections 2024-09-16T00:40:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:57.373+0000 7fa50127a640 1 -- 172.21.15.1:0/858079013 wait complete. 2024-09-16T00:40:58.268 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:40:58.537 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:58.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:40:58 smithi019 ceph-mon[35186]: pgmap v727: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:40:58.658 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:40:58 smithi001 ceph-mon[30934]: pgmap v727: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:40:58.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:40:58 smithi153 ceph-mon[35538]: pgmap v727: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:40:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.043+0000 7f00d3197640 1 -- 172.21.15.1:0/97523135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc100a80 msgr2=0x7f00cc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.043+0000 7f00d3197640 1 --2- 172.21.15.1:0/97523135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc100a80 0x7f00cc100ee0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009930 tx=0x7f00b802a420 comp rx=0 tx=0).stop 2024-09-16T00:40:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.043+0000 7f00d3197640 1 -- 172.21.15.1:0/97523135 shutdown_connections 2024-09-16T00:40:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.043+0000 7f00d3197640 1 --2- 172.21.15.1:0/97523135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00cc101420 0x7f00cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.043+0000 7f00d3197640 1 --2- 172.21.15.1:0/97523135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc100a80 0x7f00cc100ee0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.043+0000 7f00d3197640 1 --2- 172.21.15.1:0/97523135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc106a80 0x7f00cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.043+0000 7f00d3197640 1 -- 172.21.15.1:0/97523135 >> 172.21.15.1:0/97523135 conn(0x7f00cc0fc7a0 msgr2=0x7f00cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.043+0000 7f00d3197640 1 -- 172.21.15.1:0/97523135 shutdown_connections 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.043+0000 7f00d3197640 1 -- 172.21.15.1:0/97523135 wait complete. 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d3197640 1 Processor -- start 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d3197640 1 -- start start 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d3197640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc100a80 0x7f00cc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d3197640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc101420 0x7f00cc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d3197640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00cc106a80 0x7f00cc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d3197640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc110e80 con 0x7f00cc101420 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d3197640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc110d00 con 0x7f00cc106a80 2024-09-16T00:40:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d3197640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc111000 con 0x7f00cc100a80 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc101420 0x7f00cc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc101420 0x7f00cc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49834/0 (socket says 172.21.15.1:49834) 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00cbfff640 1 -- 172.21.15.1:0/2335473083 learned_addr learned my addr 172.21.15.1:0/2335473083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d1734640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00cc106a80 0x7f00cc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.044+0000 7f00d0f33640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc100a80 0x7f00cc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00cbfff640 1 -- 172.21.15.1:0/2335473083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc100a80 msgr2=0x7f00cc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00cbfff640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc100a80 0x7f00cc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00cbfff640 1 -- 172.21.15.1:0/2335473083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00cc106a80 msgr2=0x7f00cc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00cbfff640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00cc106a80 0x7f00cc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00cbfff640 1 -- 172.21.15.1:0/2335473083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc19d3b0 con 0x7f00cc101420 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00cbfff640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc101420 0x7f00cc198940 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009230 tx=0x7f00b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00c9ffb640 1 -- 172.21.15.1:0/2335473083 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b803f030 con 0x7f00cc101420 2024-09-16T00:40:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00d3197640 1 -- 172.21.15.1:0/2335473083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc19d640 con 0x7f00cc101420 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00c9ffb640 1 -- 172.21.15.1:0/2335473083 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b8003ad0 con 0x7f00cc101420 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00c9ffb640 1 -- 172.21.15.1:0/2335473083 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8035070 con 0x7f00cc101420 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.045+0000 7f00d3197640 1 -- 172.21.15.1:0/2335473083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc1a4f80 con 0x7f00cc101420 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.047+0000 7f00c9ffb640 1 -- 172.21.15.1:0/2335473083 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f00b8042030 con 0x7f00cc101420 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d5570740 1 Processor -- start 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d5570740 1 -- start start 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d5570740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 0x561cb9130090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d5570740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb9130a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d5570740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 0x561cb9141b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d5570740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561cb90962f0 con 0x561cb91305d0 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d5570740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561cb903dcf0 con 0x561cb913f780 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d5570740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561cb90922b0 con 0x561cb9131b40 2024-09-16T00:40:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d0f33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb9130a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00c9ffb640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00bc0a3240 0x7f00bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d0f33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb9130a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49844/0 (socket says 172.21.15.1:49844) 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00d0f33640 1 -- 172.21.15.1:0/2472480128 learned_addr learned my addr 172.21.15.1:0/2472480128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.048+0000 7f00cbfff640 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 0x561cb9141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00d1734640 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 0x561cb9130090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00d0f33640 1 -- 172.21.15.1:0/2472480128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 msgr2=0x561cb9130090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00d0f33640 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 0x561cb9130090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00d0f33640 1 -- 172.21.15.1:0/2472480128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 msgr2=0x561cb9141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00d0f33640 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 0x561cb9141b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00d0f33640 1 -- 172.21.15.1:0/2472480128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb90223a0 con 0x561cb91305d0 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00d1734640 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 0x561cb9130090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00d1734640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00bc0a3240 0x7f00bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00cbfff640 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 0x561cb9141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:59.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00c9ffb640 1 -- 172.21.15.1:0/2335473083 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f00b80f01d0 con 0x7f00cc101420 2024-09-16T00:40:59.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00d0f33640 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb9130a10 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f00c000bc90 tx=0x7f00c00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e5062f74612cef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:59.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00b6ffd640 1 -- 172.21.15.1:0/2472480128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c001c480 con 0x561cb91305d0 2024-09-16T00:40:59.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00b6ffd640 1 -- 172.21.15.1:0/2472480128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c001cb00 con 0x561cb91305d0 2024-09-16T00:40:59.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.049+0000 7f00b6ffd640 1 -- 172.21.15.1:0/2472480128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0011bb0 con 0x561cb91305d0 2024-09-16T00:40:59.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 -- 172.21.15.1:0/2472480128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 msgr2=0x561cb9130a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:59.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb9130a10 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f00c000bc90 tx=0x7f00c00107a0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d1734640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00bc0a3240 0x7f00bc0a5700 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005fd0 tx=0x7f00c4009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:59.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 -- 172.21.15.1:0/2472480128 shutdown_connections 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 0x561cb9141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb9130a10 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 --2- 172.21.15.1:0/2472480128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 0x561cb9130090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 -- 172.21.15.1:0/2472480128 >> 172.21.15.1:0/2472480128 conn(0x561cb90f9310 msgr2=0x561cb912cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 -- 172.21.15.1:0/2472480128 shutdown_connections 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 -- 172.21.15.1:0/2472480128 wait complete. 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 -- start start 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.050+0000 7f00d5570740 1 Processor -- start 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d5570740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb913f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d5570740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb9131b40 0x561cb913c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d5570740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 0x561cb913c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d5570740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561cb9142480 con 0x561cb91305d0 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d5570740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561cb9051060 con 0x561cb9131b40 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d5570740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561cb9142600 con 0x561cb913f780 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d5570740 10 client.0 _ll_register_callbacks cb 0x561cb8da48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d0f33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb913f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d0f33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb913f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49850/0 (socket says 172.21.15.1:49850) 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d0f33640 1 -- 172.21.15.1:0/1012411908 learned_addr learned my addr 172.21.15.1:0/1012411908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d1734640 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 0x561cb913c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d0f33640 1 -- 172.21.15.1:0/1012411908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 msgr2=0x561cb913c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d0f33640 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 0x561cb913c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d0f33640 1 -- 172.21.15.1:0/1012411908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb9131b40 msgr2=0x561cb913c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00cbfff640 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb9131b40 0x561cb913c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d0f33640 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb9131b40 0x561cb913c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d0f33640 1 -- 172.21.15.1:0/1012411908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb8e073c0 con 0x561cb91305d0 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00d0f33640 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb913f130 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f00c0010ca0 tx=0x7f00c001c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a98e67eee4271a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00abfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.051+0000 7f00abfff640 1 -- 172.21.15.1:0/1012411908 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0004020 con 0x561cb91305d0 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00abfff640 1 -- 172.21.15.1:0/1012411908 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f00c00041c0 con 0x561cb91305d0 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00d5570740 1 -- 172.21.15.1:0/1012411908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c001cb00 con 0x561cb91305d0 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00abfff640 1 -- 172.21.15.1:0/1012411908 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f00c0030050 con 0x561cb91305d0 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00d5570740 10 client.18447 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00d5570740 1 -- 172.21.15.1:0/1012411908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561cb914de40 con 0x561cb91305d0 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00abfff640 1 -- 172.21.15.1:0/1012411908 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c001cb00 con 0x561cb91305d0 2024-09-16T00:40:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00abfff640 1 -- 172.21.15.1:0/1012411908 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f00c00046c0 con 0x561cb91305d0 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00d5570740 10 client.18447 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00d5570740 10 client.18447 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00d5570740 20 client.18447 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00d5570740 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x561cb914f240 0x561cb9151630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00d5570740 4 client.18447 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.052+0000 7f00d5570740 1 -- 172.21.15.1:0/1012411908 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cb90a3860 con 0x561cb914f240 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.053+0000 7f00cbfff640 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x561cb914f240 0x561cb9151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.053+0000 7f00abfff640 10 client.18447 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.053+0000 7f00cbfff640 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x561cb914f240 0x561cb9151630 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa58e25bd98990a server_cookie=86f6a6ccbaa93298 in_seq=0 out_seq=0 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00abfff640 1 -- 172.21.15.1:0/1012411908 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cb90a3860 con 0x561cb914f240 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00abfff640 10 client.18447 handle_command_reply: tid=0 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 client.18447 shutdown 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 -- 172.21.15.1:0/1012411908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 msgr2=0x561cb913f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb913f130 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f00c0010ca0 tx=0x7f00c001c6b0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 -- 172.21.15.1:0/1012411908 shutdown_connections 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 0x561cb913c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb9131b40 0x561cb913c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x561cb914f240 0x561cb9151630 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 --2- 172.21.15.1:0/1012411908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb913f130 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00abfff640 0 client.18447 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 -- 172.21.15.1:0/1012411908 >> 172.21.15.1:0/1012411908 conn(0x561cb90f9310 msgr2=0x561cb9132960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 -- 172.21.15.1:0/1012411908 shutdown_connections 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 1 -- 172.21.15.1:0/1012411908 wait complete. 2024-09-16T00:40:59.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.054+0000 7f00d5570740 20 client.18447 trim_cache size 0 max 16384 2024-09-16T00:40:59.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d5570740 1 Processor -- start 2024-09-16T00:40:59.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d5570740 1 -- start start 2024-09-16T00:40:59.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d5570740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb914b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d5570740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 0x561cb9140b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d5570740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 0x561cb91410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d5570740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1af580 con 0x561cb91305d0 2024-09-16T00:40:59.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d5570740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1ace50 con 0x561cb913f780 2024-09-16T00:40:59.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d5570740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1aaca0 con 0x561cb9131b40 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d0f33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb914b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d0f33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb914b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49856/0 (socket says 172.21.15.1:49856) 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.067+0000 7f00d0f33640 1 -- 172.21.15.1:0/821144318 learned_addr learned my addr 172.21.15.1:0/821144318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00cbfff640 1 --2- 172.21.15.1:0/821144318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 0x561cb9140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00d1734640 1 --2- 172.21.15.1:0/821144318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 0x561cb91410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00d0f33640 1 -- 172.21.15.1:0/821144318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 msgr2=0x561cb9140b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00d0f33640 1 --2- 172.21.15.1:0/821144318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 0x561cb9140b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00d0f33640 1 -- 172.21.15.1:0/821144318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 msgr2=0x561cb91410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00d0f33640 1 --2- 172.21.15.1:0/821144318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 0x561cb91410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00d0f33640 1 -- 172.21.15.1:0/821144318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb913a930 con 0x561cb91305d0 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00d0f33640 1 --2- 172.21.15.1:0/821144318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb914b720 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f00c0020040 tx=0x7f00c000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b192ab1d6201cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00b67fc640 1 -- 172.21.15.1:0/821144318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0004cb0 con 0x561cb91305d0 2024-09-16T00:40:59.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00b67fc640 1 -- 172.21.15.1:0/821144318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c0011440 con 0x561cb91305d0 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.068+0000 7f00b67fc640 1 -- 172.21.15.1:0/821144318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c00052a0 con 0x561cb91305d0 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 -- 172.21.15.1:0/821144318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 msgr2=0x561cb914b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 --2- 172.21.15.1:0/821144318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb914b720 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f00c0020040 tx=0x7f00c000b690 comp rx=0 tx=0).stop 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 -- 172.21.15.1:0/821144318 shutdown_connections 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 --2- 172.21.15.1:0/821144318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb913f780 0x561cb91410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 --2- 172.21.15.1:0/821144318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb9131b40 0x561cb9140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 --2- 172.21.15.1:0/821144318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb91305d0 0x561cb914b720 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 -- 172.21.15.1:0/821144318 >> 172.21.15.1:0/821144318 conn(0x561cb90f9310 msgr2=0x561cb913acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 -- 172.21.15.1:0/821144318 shutdown_connections 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 -- 172.21.15.1:0/821144318 wait complete. 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 -- start start 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.069+0000 7f00d5570740 1 Processor -- start 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d5570740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb91305d0 0x561cb91562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d5570740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb9131b40 0x561cb9156820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d5570740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 0x561cb9146890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d5570740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1ab310 con 0x561cb9131b40 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d5570740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1ac780 con 0x561cb91305d0 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d5570740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1ab9e0 con 0x561cb913f780 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d5570740 10 client.0 _ll_register_callbacks cb 0x7f00c000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb9131b40 0x561cb9156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb9131b40 0x561cb9156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49870/0 (socket says 172.21.15.1:49870) 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00cbfff640 1 -- 172.21.15.1:0/1052025206 learned_addr learned my addr 172.21.15.1:0/1052025206 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d0f33640 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb91305d0 0x561cb91562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d1734640 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 0x561cb9146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00cbfff640 1 -- 172.21.15.1:0/1052025206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 msgr2=0x561cb9146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00cbfff640 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 0x561cb9146890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00cbfff640 1 -- 172.21.15.1:0/1052025206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb91305d0 msgr2=0x561cb91562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00cbfff640 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb91305d0 0x561cb91562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00cbfff640 1 -- 172.21.15.1:0/1052025206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb9146ed0 con 0x561cb9131b40 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00cbfff640 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb9131b40 0x561cb9156820 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f00b80bc140 tx=0x7f00b802ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b4774415350c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00abfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00abfff640 1 -- 172.21.15.1:0/1052025206 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8049070 con 0x561cb9131b40 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00abfff640 1 -- 172.21.15.1:0/1052025206 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f00b8041d80 con 0x561cb9131b40 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.070+0000 7f00d5570740 1 -- 172.21.15.1:0/1052025206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c0011440 con 0x561cb9131b40 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00abfff640 1 -- 172.21.15.1:0/1052025206 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f00b80c0740 con 0x561cb9131b40 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00d5570740 10 client.18453 fetch_fsmap learned FSMap version 25 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00d5570740 1 -- 172.21.15.1:0/1052025206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561cb91481b0 con 0x561cb9131b40 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00abfff640 1 -- 172.21.15.1:0/1052025206 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c0011440 con 0x561cb9131b40 2024-09-16T00:40:59.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00abfff640 1 -- 172.21.15.1:0/1052025206 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f00b80b8ab0 con 0x561cb9131b40 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00d5570740 10 client.18453 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00d5570740 10 client.18453 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00d5570740 20 client.18453 populate_metadata read hostname 'smithi001' 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00d5570740 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x561cb91b9eb0 0x561cb91bc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00d5570740 4 client.18453 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00d5570740 1 -- 172.21.15.1:0/1052025206 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cb909dd70 con 0x561cb91b9eb0 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.071+0000 7f00d0f33640 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x561cb91b9eb0 0x561cb91bc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d0f33640 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x561cb91b9eb0 0x561cb91bc2d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7a799ffda44542b server_cookie=1036da27c17dd9d9 in_seq=0 out_seq=0 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00abfff640 10 client.18453 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00abfff640 1 -- 172.21.15.1:0/1052025206 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cb909dd70 con 0x561cb91b9eb0 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00abfff640 10 client.18453 handle_command_reply: tid=0 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 client.18453 shutdown 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 -- 172.21.15.1:0/1052025206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb9131b40 msgr2=0x561cb9156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:59.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb9131b40 0x561cb9156820 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f00b80bc140 tx=0x7f00b802ffa0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 -- 172.21.15.1:0/1052025206 shutdown_connections 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cb913f780 0x561cb9146890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x561cb91b9eb0 0x561cb91bc2d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cb9131b40 0x561cb9156820 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00abfff640 0 client.18453 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 --2- 172.21.15.1:0/1052025206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561cb91305d0 0x561cb91562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 -- 172.21.15.1:0/1052025206 >> 172.21.15.1:0/1052025206 conn(0x561cb90f9310 msgr2=0x561cb912b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 -- 172.21.15.1:0/1052025206 shutdown_connections 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.072+0000 7f00d5570740 1 -- 172.21.15.1:0/1052025206 wait complete. 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d5570740 20 client.18453 trim_cache size 0 max 16384 2024-09-16T00:40:59.082 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:40:59.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 -- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00bc0a3240 msgr2=0x7f00bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:59.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00bc0a3240 0x7f00bc0a5700 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005fd0 tx=0x7f00c4009600 comp rx=0 tx=0).stop 2024-09-16T00:40:59.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 -- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc101420 msgr2=0x7f00cc198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:59.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc101420 0x7f00cc198940 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009230 tx=0x7f00b8004270 comp rx=0 tx=0).stop 2024-09-16T00:40:59.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 -- 172.21.15.1:0/2335473083 shutdown_connections 2024-09-16T00:40:59.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00cc106a80 0x7f00cc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00cc101420 0x7f00cc198940 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f00bc0a3240 0x7f00bc0a5700 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 --2- 172.21.15.1:0/2335473083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc100a80 0x7f00cc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:59.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 -- 172.21.15.1:0/2335473083 >> 172.21.15.1:0/2335473083 conn(0x7f00cc0fc7a0 msgr2=0x7f00cc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:59.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 -- 172.21.15.1:0/2335473083 shutdown_connections 2024-09-16T00:40:59.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:40:59.073+0000 7f00d3197640 1 -- 172.21.15.1:0/2335473083 wait complete. 2024-09-16T00:40:59.900 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:41:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:00.605 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:00 smithi001 ceph-mon[30934]: pgmap v728: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:00.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:00 smithi019 ceph-mon[35186]: pgmap v728: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:00.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:00 smithi153 ceph-mon[35538]: pgmap v728: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:00.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.689+0000 7f1b605ab640 1 -- 172.21.15.1:0/347957620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58103660 msgr2=0x7f1b5810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.689+0000 7f1b605ab640 1 --2- 172.21.15.1:0/347957620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58103660 0x7f1b5810bb50 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009900 tx=0x7f1b4c02a480 comp rx=0 tx=0).stop 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.689+0000 7f1b605ab640 1 -- 172.21.15.1:0/347957620 shutdown_connections 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.689+0000 7f1b605ab640 1 --2- 172.21.15.1:0/347957620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b5810c170 0x7f1b5810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.689+0000 7f1b605ab640 1 --2- 172.21.15.1:0/347957620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58103660 0x7f1b5810bb50 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.689+0000 7f1b605ab640 1 --2- 172.21.15.1:0/347957620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58102d40 0x7f1b58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.689+0000 7f1b605ab640 1 -- 172.21.15.1:0/347957620 >> 172.21.15.1:0/347957620 conn(0x7f1b580fc7a0 msgr2=0x7f1b580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.689+0000 7f1b605ab640 1 -- 172.21.15.1:0/347957620 shutdown_connections 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.689+0000 7f1b605ab640 1 -- 172.21.15.1:0/347957620 wait complete. 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b605ab640 1 Processor -- start 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b605ab640 1 -- start start 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b605ab640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58102d40 0x7f1b58198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b605ab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58103660 0x7f1b58198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b605ab640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b5810c170 0x7f1b5819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b605ab640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58111fb0 con 0x7f1b58102d40 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b605ab640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58111e30 con 0x7f1b58103660 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b605ab640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58112130 con 0x7f1b5810c170 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b5e347640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58102d40 0x7f1b58198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b5e347640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58102d40 0x7f1b58198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32976/0 (socket says 172.21.15.1:32976) 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b5e347640 1 -- 172.21.15.1:0/1886304297 learned_addr learned my addr 172.21.15.1:0/1886304297 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b5db46640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58103660 0x7f1b58198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b5810c170 0x7f1b5819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.690+0000 7f1b5e347640 1 -- 172.21.15.1:0/1886304297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b5810c170 msgr2=0x7f1b5819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b5e347640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b5810c170 0x7f1b5819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b5e347640 1 -- 172.21.15.1:0/1886304297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58103660 msgr2=0x7f1b58198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b5e347640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58103660 0x7f1b58198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b5e347640 1 -- 172.21.15.1:0/1886304297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5819d3c0 con 0x7f1b58102d40 2024-09-16T00:41:00.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b5e347640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58102d40 0x7f1b58198410 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800cce0 tx=0x7f1b4800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b577fe640 1 -- 172.21.15.1:0/1886304297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4800f650 con 0x7f1b58102d40 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b577fe640 1 -- 172.21.15.1:0/1886304297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b4800fcd0 con 0x7f1b58102d40 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b605ab640 1 -- 172.21.15.1:0/1886304297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5819d6b0 con 0x7f1b58102d40 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b577fe640 1 -- 172.21.15.1:0/1886304297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48018c70 con 0x7f1b58102d40 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.691+0000 7f1b605ab640 1 -- 172.21.15.1:0/1886304297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b581a4f90 con 0x7f1b58102d40 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.693+0000 7f1b577fe640 1 -- 172.21.15.1:0/1886304297 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f1b4800f7f0 con 0x7f1b58102d40 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b62984740 1 Processor -- start 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b62984740 1 -- start start 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b62984740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4973b40 0x55f4a4972090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b62984740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a49725d0 0x55f4a4972a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b62984740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4981780 0x55f4a4983b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b62984740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f4a48d82f0 con 0x55f4a49725d0 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b62984740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4a487fcf0 con 0x55f4a4981780 2024-09-16T00:41:00.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b62984740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f4a48d42b0 con 0x55f4a4973b40 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b577fe640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1b400a32c0 0x7f1b400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b5e347640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a49725d0 0x55f4a4972a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b5e347640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a49725d0 0x55f4a4972a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32980/0 (socket says 172.21.15.1:32980) 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b5e347640 1 -- 172.21.15.1:0/4133673128 learned_addr learned my addr 172.21.15.1:0/4133673128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1b400a32c0 0x7f1b400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/4133673128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4973b40 0x55f4a4972090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.694+0000 7f1b5db46640 1 --2- 172.21.15.1:0/4133673128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4981780 0x55f4a4983b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b577fe640 1 -- 172.21.15.1:0/1886304297 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1b480d0100 con 0x7f1b58102d40 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b5e347640 1 -- 172.21.15.1:0/4133673128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4973b40 msgr2=0x55f4a4972090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b5e347640 1 --2- 172.21.15.1:0/4133673128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4973b40 0x55f4a4972090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b5e347640 1 -- 172.21.15.1:0/4133673128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4981780 msgr2=0x55f4a4983b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b5e347640 1 --2- 172.21.15.1:0/4133673128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4981780 0x55f4a4983b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b5e347640 1 -- 172.21.15.1:0/4133673128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a48643a0 con 0x55f4a49725d0 2024-09-16T00:41:00.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1b400a32c0 0x7f1b400a5780 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1b50006e10 tx=0x7f1b500098b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b5e347640 1 --2- 172.21.15.1:0/4133673128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a49725d0 0x55f4a4972a10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1b48000f80 tx=0x7f1b48098690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9bff25eeabf6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b54ff9640 1 -- 172.21.15.1:0/4133673128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4809be00 con 0x55f4a49725d0 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b54ff9640 1 -- 172.21.15.1:0/4133673128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b4809c9c0 con 0x55f4a49725d0 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.695+0000 7f1b54ff9640 1 -- 172.21.15.1:0/4133673128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4809bc20 con 0x55f4a49725d0 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 -- 172.21.15.1:0/4133673128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a49725d0 msgr2=0x55f4a4972a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 --2- 172.21.15.1:0/4133673128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a49725d0 0x55f4a4972a10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1b48000f80 tx=0x7f1b48098690 comp rx=0 tx=0).stop 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 -- 172.21.15.1:0/4133673128 shutdown_connections 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 --2- 172.21.15.1:0/4133673128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4981780 0x55f4a4983b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 --2- 172.21.15.1:0/4133673128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a49725d0 0x55f4a4972a10 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 --2- 172.21.15.1:0/4133673128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4973b40 0x55f4a4972090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 -- 172.21.15.1:0/4133673128 >> 172.21.15.1:0/4133673128 conn(0x55f4a493b310 msgr2=0x55f4a496ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 -- 172.21.15.1:0/4133673128 shutdown_connections 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 -- 172.21.15.1:0/4133673128 wait complete. 2024-09-16T00:41:00.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.696+0000 7f1b62984740 1 -- start start 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b62984740 1 Processor -- start 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b62984740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a49725d0 0x55f4a4981210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b62984740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4973b40 0x55f4a497e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b62984740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4981780 0x55f4a497ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b62984740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f4a4984480 con 0x55f4a4981780 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b62984740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4a4893060 con 0x55f4a4973b40 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b62984740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f4a4984600 con 0x55f4a49725d0 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b62984740 10 client.0 _ll_register_callbacks cb 0x55f4a45e68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5eb48640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4981780 0x55f4a497ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5eb48640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4981780 0x55f4a497ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32992/0 (socket says 172.21.15.1:32992) 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5eb48640 1 -- 172.21.15.1:0/3000702518 learned_addr learned my addr 172.21.15.1:0/3000702518 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5e347640 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a49725d0 0x55f4a4981210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5db46640 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4973b40 0x55f4a497e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5eb48640 1 -- 172.21.15.1:0/3000702518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a49725d0 msgr2=0x55f4a4981210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a49725d0 0x55f4a4981210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5eb48640 1 -- 172.21.15.1:0/3000702518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4973b40 msgr2=0x55f4a497e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4973b40 0x55f4a497e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5eb48640 1 -- 172.21.15.1:0/3000702518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a46493c0 con 0x55f4a4981780 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.697+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4981780 0x55f4a497ec50 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f1b50017b10 tx=0x7f1b500090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44efb6ad97c6e3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b3dffb640 1 -- 172.21.15.1:0/3000702518 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50018ca0 con 0x55f4a4981780 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b62984740 1 -- 172.21.15.1:0/3000702518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b4809c9c0 con 0x55f4a4981780 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b3dffb640 1 -- 172.21.15.1:0/3000702518 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1b50024460 con 0x55f4a4981780 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b3dffb640 1 -- 172.21.15.1:0/3000702518 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b50032050 con 0x55f4a4981780 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b62984740 10 client.18465 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b62984740 1 -- 172.21.15.1:0/3000702518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f4a4990020 con 0x55f4a4981780 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b3dffb640 1 -- 172.21.15.1:0/3000702518 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b4809c9c0 con 0x55f4a4981780 2024-09-16T00:41:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b3dffb640 1 -- 172.21.15.1:0/3000702518 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1b500224a0 con 0x55f4a4981780 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b62984740 10 client.18465 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b62984740 10 client.18465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b62984740 20 client.18465 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b62984740 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f4a4991430 0x55f4a49938b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b62984740 4 client.18465 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.698+0000 7f1b62984740 1 -- 172.21.15.1:0/3000702518 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4a48e5860 con 0x55f4a4991430 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.699+0000 7f1b5e347640 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f4a4991430 0x55f4a49938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.699+0000 7f1b5e347640 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f4a4991430 0x55f4a49938b0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c622d2d7186615 server_cookie=34a81e06bff6fc56 in_seq=0 out_seq=0 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.699+0000 7f1b3dffb640 10 client.18465 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b3dffb640 1 -- 172.21.15.1:0/3000702518 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4a48e5860 con 0x55f4a4991430 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b3dffb640 10 client.18465 handle_command_reply: tid=0 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 client.18465 shutdown 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 -- 172.21.15.1:0/3000702518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4981780 msgr2=0x55f4a497ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4981780 0x55f4a497ec50 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f1b50017b10 tx=0x7f1b500090e0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 -- 172.21.15.1:0/3000702518 shutdown_connections 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f4a4991430 0x55f4a49938b0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4981780 0x55f4a497ec50 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a4973b40 0x55f4a497e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 --2- 172.21.15.1:0/3000702518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a49725d0 0x55f4a4981210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b3dffb640 0 client.18465 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 -- 172.21.15.1:0/3000702518 >> 172.21.15.1:0/3000702518 conn(0x55f4a493b310 msgr2=0x55f4a4974990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 -- 172.21.15.1:0/3000702518 shutdown_connections 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 1 -- 172.21.15.1:0/3000702518 wait complete. 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.700+0000 7f1b62984740 20 client.18465 trim_cache size 0 max 16384 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b62984740 1 Processor -- start 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b62984740 1 -- start start 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b62984740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 0x55f4a497d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b62984740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a49991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b62984740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 0x55f4a4999710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b62984740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581af4d0 con 0x55f4a4973b40 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b62984740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581acda0 con 0x55f4a49725d0 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b62984740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581aabf0 con 0x55f4a4981780 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b5db46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a49991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b5db46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a49991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32994/0 (socket says 172.21.15.1:32994) 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b5db46640 1 -- 172.21.15.1:0/1037079155 learned_addr learned my addr 172.21.15.1:0/1037079155 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.713+0000 7f1b5e347640 1 --2- 172.21.15.1:0/1037079155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 0x55f4a497d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/1037079155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 0x55f4a4999710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b5db46640 1 -- 172.21.15.1:0/1037079155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 msgr2=0x55f4a4999710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b5db46640 1 --2- 172.21.15.1:0/1037079155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 0x55f4a4999710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b5db46640 1 -- 172.21.15.1:0/1037079155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 msgr2=0x55f4a497d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b5db46640 1 --2- 172.21.15.1:0/1037079155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 0x55f4a497d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b5db46640 1 -- 172.21.15.1:0/1037079155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a496fd60 con 0x55f4a4973b40 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b5db46640 1 --2- 172.21.15.1:0/1037079155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a49991d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009900 tx=0x7f1b4c031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ceea3a3dab98eeb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b3ffff640 1 -- 172.21.15.1:0/1037079155 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c035040 con 0x55f4a4973b40 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b3ffff640 1 -- 172.21.15.1:0/1037079155 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b4c0319f0 con 0x55f4a4973b40 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.714+0000 7f1b3ffff640 1 -- 172.21.15.1:0/1037079155 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c0476e0 con 0x55f4a4973b40 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 -- 172.21.15.1:0/1037079155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 msgr2=0x55f4a49991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 --2- 172.21.15.1:0/1037079155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a49991d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009900 tx=0x7f1b4c031880 comp rx=0 tx=0).stop 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 -- 172.21.15.1:0/1037079155 shutdown_connections 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 --2- 172.21.15.1:0/1037079155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 0x55f4a4999710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 --2- 172.21.15.1:0/1037079155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a49991d0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 --2- 172.21.15.1:0/1037079155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 0x55f4a497d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 -- 172.21.15.1:0/1037079155 >> 172.21.15.1:0/1037079155 conn(0x55f4a493b310 msgr2=0x55f4a497cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 -- 172.21.15.1:0/1037079155 shutdown_connections 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 -- 172.21.15.1:0/1037079155 wait complete. 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 -- start start 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 Processor -- start 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 0x55f4a4989890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a4989dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 0x55f4a4986860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581ab260 con 0x55f4a4973b40 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581ac6d0 con 0x55f4a49725d0 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581ab930 con 0x55f4a4981780 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.715+0000 7f1b62984740 10 client.0 _ll_register_callbacks cb 0x7f1b50009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5db46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a4989dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5db46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a4989dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33010/0 (socket says 172.21.15.1:33010) 2024-09-16T00:41:00.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5db46640 1 -- 172.21.15.1:0/1072613507 learned_addr learned my addr 172.21.15.1:0/1072613507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5e347640 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 0x55f4a4989890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 0x55f4a4986860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5db46640 1 -- 172.21.15.1:0/1072613507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 msgr2=0x55f4a4986860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5db46640 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 0x55f4a4986860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5db46640 1 -- 172.21.15.1:0/1072613507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 msgr2=0x55f4a4989890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5db46640 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 0x55f4a4989890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5db46640 1 -- 172.21.15.1:0/1072613507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a497c3e0 con 0x55f4a4973b40 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5eb48640 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 0x55f4a4986860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b5db46640 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a4989dd0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c005940 tx=0x7f1b4c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943f339531accd53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b3dffb640 1 -- 172.21.15.1:0/1072613507 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c035040 con 0x55f4a4973b40 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b3dffb640 1 -- 172.21.15.1:0/1072613507 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1b4c004120 con 0x55f4a4973b40 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b62984740 1 -- 172.21.15.1:0/1072613507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b4c0319f0 con 0x55f4a4973b40 2024-09-16T00:41:00.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b3dffb640 1 -- 172.21.15.1:0/1072613507 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b4c004320 con 0x55f4a4973b40 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b62984740 10 client.18471 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.716+0000 7f1b62984740 1 -- 172.21.15.1:0/1072613507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f4a49fc690 con 0x55f4a4973b40 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b3dffb640 1 -- 172.21.15.1:0/1072613507 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b4c0319f0 con 0x55f4a4973b40 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b3dffb640 1 -- 172.21.15.1:0/1072613507 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1b4c03d450 con 0x55f4a4973b40 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b62984740 10 client.18471 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b62984740 10 client.18471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b62984740 20 client.18471 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b62984740 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f4a49fcd50 0x55f4a49ff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b62984740 4 client.18471 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b62984740 1 -- 172.21.15.1:0/1072613507 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4a48dfd70 con 0x55f4a49fcd50 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b5e347640 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f4a49fcd50 0x55f4a49ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b3dffb640 10 client.18471 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.717+0000 7f1b5e347640 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f4a49fcd50 0x55f4a49ff170 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e1a7ad8c0a01d0 server_cookie=36307ac2f8d60040 in_seq=0 out_seq=0 2024-09-16T00:41:00.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b3dffb640 1 -- 172.21.15.1:0/1072613507 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4a48dfd70 con 0x55f4a49fcd50 2024-09-16T00:41:00.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b3dffb640 10 client.18471 handle_command_reply: tid=0 2024-09-16T00:41:00.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 client.18471 shutdown 2024-09-16T00:41:00.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 -- 172.21.15.1:0/1072613507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 msgr2=0x55f4a4989dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a4989dd0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c005940 tx=0x7f1b4c003820 comp rx=0 tx=0).stop 2024-09-16T00:41:00.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 -- 172.21.15.1:0/1072613507 shutdown_connections 2024-09-16T00:41:00.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f4a49fcd50 0x55f4a49ff170 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.737 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f4a4981780 0x55f4a4986860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b3dffb640 0 client.18471 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4a4973b40 0x55f4a4989dd0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 --2- 172.21.15.1:0/1072613507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a49725d0 0x55f4a4989890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 -- 172.21.15.1:0/1072613507 >> 172.21.15.1:0/1072613507 conn(0x55f4a493b310 msgr2=0x55f4a497d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 -- 172.21.15.1:0/1072613507 shutdown_connections 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 1 -- 172.21.15.1:0/1072613507 wait complete. 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b62984740 20 client.18471 trim_cache size 0 max 16384 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 -- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1b400a32c0 msgr2=0x7f1b400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1b400a32c0 0x7f1b400a5780 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1b50006e10 tx=0x7f1b500098b0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 -- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58102d40 msgr2=0x7f1b58198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58102d40 0x7f1b58198410 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800cce0 tx=0x7f1b4800a620 comp rx=0 tx=0).stop 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 -- 172.21.15.1:0/1886304297 shutdown_connections 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1b400a32c0 0x7f1b400a5780 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b5810c170 0x7f1b5819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58103660 0x7f1b58198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 --2- 172.21.15.1:0/1886304297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b58102d40 0x7f1b58198410 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.718+0000 7f1b605ab640 1 -- 172.21.15.1:0/1886304297 >> 172.21.15.1:0/1886304297 conn(0x7f1b580fc7a0 msgr2=0x7f1b580fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.719+0000 7f1b605ab640 1 -- 172.21.15.1:0/1886304297 shutdown_connections 2024-09-16T00:41:00.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:00.719+0000 7f1b605ab640 1 -- 172.21.15.1:0/1886304297 wait complete. 2024-09-16T00:41:02.556 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:41:02.556 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:41:02.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:02 smithi001 ceph-mon[30934]: pgmap v729: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:41:02.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:02 smithi019 ceph-mon[35186]: pgmap v729: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:41:02.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:02 smithi153 ceph-mon[35538]: pgmap v729: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:41:02.827 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:03.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.328+0000 7fd085838640 1 -- 172.21.15.1:0/3333290845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0801027a0 msgr2=0x7fd0801067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:03.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.328+0000 7fd085838640 1 --2- 172.21.15.1:0/3333290845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0801027a0 0x7fd0801067f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd068009900 tx=0x7fd06802a410 comp rx=0 tx=0).stop 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.328+0000 7fd085838640 1 -- 172.21.15.1:0/3333290845 shutdown_connections 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.328+0000 7fd085838640 1 --2- 172.21.15.1:0/3333290845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd080106f20 0x7fd08010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.328+0000 7fd085838640 1 --2- 172.21.15.1:0/3333290845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0801027a0 0x7fd0801067f0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.328+0000 7fd085838640 1 --2- 172.21.15.1:0/3333290845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080101df0 0x7fd0801021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.328+0000 7fd085838640 1 -- 172.21.15.1:0/3333290845 >> 172.21.15.1:0/3333290845 conn(0x7fd0800fd8f0 msgr2=0x7fd0800ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.328+0000 7fd085838640 1 -- 172.21.15.1:0/3333290845 shutdown_connections 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.328+0000 7fd085838640 1 -- 172.21.15.1:0/3333290845 wait complete. 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd085838640 1 Processor -- start 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd085838640 1 -- start start 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd085838640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd080101df0 0x7fd080198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd085838640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801027a0 0x7fd0801989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd085838640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd080106f20 0x7fd08019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd085838640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd080111ff0 con 0x7fd080106f20 2024-09-16T00:41:03.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd085838640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd080111e70 con 0x7fd0801027a0 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd085838640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd080112170 con 0x7fd080101df0 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd07f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd080106f20 0x7fd08019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd07f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd080106f20 0x7fd08019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33024/0 (socket says 172.21.15.1:33024) 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd07f7fe640 1 -- 172.21.15.1:0/3948611889 learned_addr learned my addr 172.21.15.1:0/3948611889 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.329+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801027a0 0x7fd0801989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd07effd640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd080101df0 0x7fd080198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd07f7fe640 1 -- 172.21.15.1:0/3948611889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd080101df0 msgr2=0x7fd080198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd07f7fe640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd080101df0 0x7fd080198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd07f7fe640 1 -- 172.21.15.1:0/3948611889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801027a0 msgr2=0x7fd0801989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:03.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd07f7fe640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801027a0 0x7fd0801989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd07f7fe640 1 -- 172.21.15.1:0/3948611889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08019d440 con 0x7fd080106f20 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd07f7fe640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd080106f20 0x7fd08019cd60 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd070009fd0 tx=0x7fd07000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd084836640 1 -- 172.21.15.1:0/3948611889 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070011830 con 0x7fd080106f20 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd084836640 1 -- 172.21.15.1:0/3948611889 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd070016070 con 0x7fd080106f20 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd085838640 1 -- 172.21.15.1:0/3948611889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08019d730 con 0x7fd080106f20 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd084836640 1 -- 172.21.15.1:0/3948611889 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07000f2e0 con 0x7fd080106f20 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.330+0000 7fd085838640 1 -- 172.21.15.1:0/3948611889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0801a5010 con 0x7fd080106f20 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.332+0000 7fd084836640 1 -- 172.21.15.1:0/3948611889 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd0700119d0 con 0x7fd080106f20 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd087c11740 1 Processor -- start 2024-09-16T00:41:03.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd087c11740 1 -- start start 2024-09-16T00:41:03.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd087c11740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a403f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd087c11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a40e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd084836640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd06c0a3240 0x7fd06c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd087c11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 0x55a50a411130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd087c11740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a50a369280 con 0x55a50a403b60 2024-09-16T00:41:03.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd087c11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a50a310b10 con 0x55a50a404480 2024-09-16T00:41:03.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd087c11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a50a365240 con 0x55a50a40ed40 2024-09-16T00:41:03.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd07e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a403f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd084836640 1 -- 172.21.15.1:0/3948611889 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd0700cf390 con 0x7fd080106f20 2024-09-16T00:41:03.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd07e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a403f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33038/0 (socket says 172.21.15.1:33038) 2024-09-16T00:41:03.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd07e7fc640 1 -- 172.21.15.1:0/77226950 learned_addr learned my addr 172.21.15.1:0/77226950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:03.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd07effd640 1 --2- 172.21.15.1:0/77226950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a40e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.333+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd06c0a3240 0x7fd06c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd07e7fc640 1 -- 172.21.15.1:0/77226950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 msgr2=0x55a50a411130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:03.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/77226950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 0x55a50a411130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd07e7fc640 1 -- 172.21.15.1:0/77226950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 msgr2=0x55a50a40e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:03.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/77226950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a40e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd07e7fc640 1 -- 172.21.15.1:0/77226950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50a31d1f0 con 0x55a50a403b60 2024-09-16T00:41:03.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd07effd640 1 --2- 172.21.15.1:0/77226950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a40e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:03.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/77226950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a403f40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd0680096c0 tx=0x7fd068003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c79034287a95b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:03.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd06c0a3240 0x7fd06c0a5700 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd068009230 tx=0x7fd06803a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:03.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd075ffb640 1 -- 172.21.15.1:0/77226950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06804e070 con 0x55a50a403b60 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd075ffb640 1 -- 172.21.15.1:0/77226950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0680047a0 con 0x55a50a403b60 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.334+0000 7fd075ffb640 1 -- 172.21.15.1:0/77226950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068035040 con 0x55a50a403b60 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 -- 172.21.15.1:0/77226950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 msgr2=0x55a50a403f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 --2- 172.21.15.1:0/77226950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a403f40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd0680096c0 tx=0x7fd068003b40 comp rx=0 tx=0).stop 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 -- 172.21.15.1:0/77226950 shutdown_connections 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 --2- 172.21.15.1:0/77226950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 0x55a50a411130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 --2- 172.21.15.1:0/77226950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a40e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 --2- 172.21.15.1:0/77226950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a403f40 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 -- 172.21.15.1:0/77226950 >> 172.21.15.1:0/77226950 conn(0x55a50a4000f0 msgr2=0x55a50a4004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 -- 172.21.15.1:0/77226950 shutdown_connections 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 -- 172.21.15.1:0/77226950 wait complete. 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 -- start start 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.335+0000 7fd087c11740 1 Processor -- start 2024-09-16T00:41:03.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd087c11740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a40bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd087c11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a404480 0x55a50a40c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd087c11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a40ed40 0x55a50a40c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd087c11740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a50a411910 con 0x55a50a403b60 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd087c11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a50a323f10 con 0x55a50a40ed40 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd087c11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a50a411a90 con 0x55a50a404480 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd087c11740 10 client.0 _ll_register_callbacks cb 0x55a50a2eca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a40bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a40bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33048/0 (socket says 172.21.15.1:33048) 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07effd640 1 -- 172.21.15.1:0/3466082703 learned_addr learned my addr 172.21.15.1:0/3466082703 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a404480 0x55a50a40c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07f7fe640 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a40ed40 0x55a50a40c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07effd640 1 -- 172.21.15.1:0/3466082703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a404480 msgr2=0x55a50a40c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:03.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07effd640 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a404480 0x55a50a40c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07effd640 1 -- 172.21.15.1:0/3466082703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a40ed40 msgr2=0x55a50a40c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:03.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07effd640 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a40ed40 0x55a50a40c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07effd640 1 -- 172.21.15.1:0/3466082703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50a0da6a0 con 0x55a50a403b60 2024-09-16T00:41:03.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd07effd640 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a40bce0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd078012720 tx=0x7fd078012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dd85040f1d4a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:03.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd05effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:03.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.336+0000 7fd05effd640 1 -- 172.21.15.1:0/3466082703 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078010dc0 con 0x55a50a403b60 2024-09-16T00:41:03.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd05effd640 1 -- 172.21.15.1:0/3466082703 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd078003970 con 0x55a50a403b60 2024-09-16T00:41:03.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd087c11740 1 -- 172.21.15.1:0/3466082703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0680047a0 con 0x55a50a403b60 2024-09-16T00:41:03.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd05effd640 1 -- 172.21.15.1:0/3466082703 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd078003b70 con 0x55a50a403b60 2024-09-16T00:41:03.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd087c11740 10 client.18483 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:03.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd087c11740 1 -- 172.21.15.1:0/3466082703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a50a421a40 con 0x55a50a403b60 2024-09-16T00:41:03.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd05effd640 1 -- 172.21.15.1:0/3466082703 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0680047a0 con 0x55a50a403b60 2024-09-16T00:41:03.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd05effd640 1 -- 172.21.15.1:0/3466082703 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd078019070 con 0x55a50a403b60 2024-09-16T00:41:03.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd087c11740 10 client.18483 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:03.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd087c11740 10 client.18483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:03.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd087c11740 20 client.18483 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd087c11740 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a50a422f00 0x55a50a425330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd087c11740 4 client.18483 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd087c11740 1 -- 172.21.15.1:0/3466082703 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a50a3767f0 con 0x55a50a422f00 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.337+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a50a422f00 0x55a50a425330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.338+0000 7fd05effd640 10 client.18483 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.338+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a50a422f00 0x55a50a425330 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523647d08a71ceb2 server_cookie=eca1deacbbc06839 in_seq=0 out_seq=0 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd05effd640 1 -- 172.21.15.1:0/3466082703 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a50a3767f0 con 0x55a50a422f00 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd05effd640 10 client.18483 handle_command_reply: tid=0 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 client.18483 shutdown 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 -- 172.21.15.1:0/3466082703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 msgr2=0x55a50a40bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a40bce0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd078012720 tx=0x7fd078012be0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 -- 172.21.15.1:0/3466082703 shutdown_connections 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a50a422f00 0x55a50a425330 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd05effd640 0 client.18483 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:03.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a40ed40 0x55a50a40c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a404480 0x55a50a40c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 --2- 172.21.15.1:0/3466082703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a40bce0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 -- 172.21.15.1:0/3466082703 >> 172.21.15.1:0/3466082703 conn(0x55a50a4000f0 msgr2=0x55a50a405e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:03.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 -- 172.21.15.1:0/3466082703 shutdown_connections 2024-09-16T00:41:03.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 1 -- 172.21.15.1:0/3466082703 wait complete. 2024-09-16T00:41:03.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.339+0000 7fd087c11740 20 client.18483 trim_cache size 0 max 16384 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd087c11740 1 Processor -- start 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd087c11740 1 -- start start 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd087c11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a403b60 0x55a50a405cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd087c11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a42ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd087c11740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a40ed40 0x55a50a42b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd087c11740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801af5b0 con 0x55a50a40ed40 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd087c11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801ace80 con 0x55a50a404480 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd087c11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801aacd0 con 0x55a50a403b60 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd07f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a40ed40 0x55a50a42b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd07f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a40ed40 0x55a50a42b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33064/0 (socket says 172.21.15.1:33064) 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd07f7fe640 1 -- 172.21.15.1:0/618755965 learned_addr learned my addr 172.21.15.1:0/618755965 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd07effd640 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a403b60 0x55a50a405cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a42ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd07f7fe640 1 -- 172.21.15.1:0/618755965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a403b60 msgr2=0x55a50a405cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.357+0000 7fd07f7fe640 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a403b60 0x55a50a405cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.358+0000 7fd07f7fe640 1 -- 172.21.15.1:0/618755965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 msgr2=0x55a50a42ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.358+0000 7fd07f7fe640 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a42ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.358+0000 7fd07f7fe640 1 -- 172.21.15.1:0/618755965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50a40ac90 con 0x55a50a40ed40 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.358+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a42ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.358+0000 7fd07f7fe640 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a40ed40 0x55a50a42b140 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd070097f20 tx=0x7fd070010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55912cb8060dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.358+0000 7fd0757fa640 1 -- 172.21.15.1:0/618755965 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07009bc50 con 0x55a50a40ed40 2024-09-16T00:41:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.358+0000 7fd0757fa640 1 -- 172.21.15.1:0/618755965 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07009b550 con 0x55a50a40ed40 2024-09-16T00:41:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.358+0000 7fd0757fa640 1 -- 172.21.15.1:0/618755965 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070097bc0 con 0x55a50a40ed40 2024-09-16T00:41:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 -- 172.21.15.1:0/618755965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a40ed40 msgr2=0x55a50a42b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a40ed40 0x55a50a42b140 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd070097f20 tx=0x7fd070010910 comp rx=0 tx=0).stop 2024-09-16T00:41:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 -- 172.21.15.1:0/618755965 shutdown_connections 2024-09-16T00:41:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a40ed40 0x55a50a42b140 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a42ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 --2- 172.21.15.1:0/618755965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a403b60 0x55a50a405cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 -- 172.21.15.1:0/618755965 >> 172.21.15.1:0/618755965 conn(0x55a50a4000f0 msgr2=0x55a50a4055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 -- 172.21.15.1:0/618755965 shutdown_connections 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 -- 172.21.15.1:0/618755965 wait complete. 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 -- start start 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.359+0000 7fd087c11740 1 Processor -- start 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd087c11740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a4177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd087c11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a417ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd087c11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 0x55a50a486c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd087c11740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801ab340 con 0x55a50a403b60 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd087c11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801ac7b0 con 0x55a50a404480 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd087c11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801aba10 con 0x55a50a40ed40 2024-09-16T00:41:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd087c11740 10 client.0 _ll_register_callbacks cb 0x7fd07800a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a4177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a4177a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33076/0 (socket says 172.21.15.1:33076) 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07effd640 1 -- 172.21.15.1:0/3732048449 learned_addr learned my addr 172.21.15.1:0/3732048449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a417ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07f7fe640 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 0x55a50a486c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07effd640 1 -- 172.21.15.1:0/3732048449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 msgr2=0x55a50a486c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07effd640 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 0x55a50a486c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07effd640 1 -- 172.21.15.1:0/3732048449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 msgr2=0x55a50a417ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07effd640 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a417ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07effd640 1 -- 172.21.15.1:0/3732048449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50a487250 con 0x55a50a403b60 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07f7fe640 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 0x55a50a486c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd07effd640 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a4177a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fd078005e50 tx=0x7fd0780108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59654f942f0360b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.360+0000 7fd05effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd05effd640 1 -- 172.21.15.1:0/3732048449 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078004bf0 con 0x55a50a403b60 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd087c11740 1 -- 172.21.15.1:0/3732048449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd07009b550 con 0x55a50a403b60 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd05effd640 1 -- 172.21.15.1:0/3732048449 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd078004d90 con 0x55a50a403b60 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd05effd640 1 -- 172.21.15.1:0/3732048449 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd07803a050 con 0x55a50a403b60 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd087c11740 10 client.18489 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd087c11740 1 -- 172.21.15.1:0/3732048449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a50a48e5e0 con 0x55a50a403b60 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd05effd640 1 -- 172.21.15.1:0/3732048449 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07009b550 con 0x55a50a403b60 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd05effd640 1 -- 172.21.15.1:0/3732048449 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd078019070 con 0x55a50a403b60 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd087c11740 10 client.18489 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd087c11740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd087c11740 20 client.18489 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:03.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd087c11740 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a50a422f00 0x55a50a490d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd087c11740 4 client.18489 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.361+0000 7fd087c11740 1 -- 172.21.15.1:0/3732048449 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a50a370d00 con 0x55a50a422f00 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.362+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a50a422f00 0x55a50a490d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.362+0000 7fd05effd640 10 client.18489 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.362+0000 7fd07e7fc640 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a50a422f00 0x55a50a490d40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba80a434bd3962d1 server_cookie=7ae052dc4f408f86 in_seq=0 out_seq=0 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd05effd640 1 -- 172.21.15.1:0/3732048449 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a50a370d00 con 0x55a50a422f00 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd05effd640 10 client.18489 handle_command_reply: tid=0 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 client.18489 shutdown 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 -- 172.21.15.1:0/3732048449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 msgr2=0x55a50a4177a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a4177a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fd078005e50 tx=0x7fd0780108e0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 -- 172.21.15.1:0/3732048449 shutdown_connections 2024-09-16T00:41:03.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a50a422f00 0x55a50a490d40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a50a40ed40 0x55a50a486c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a50a404480 0x55a50a417ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 --2- 172.21.15.1:0/3732048449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a50a403b60 0x55a50a4177a0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd05effd640 0 client.18489 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:03.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 -- 172.21.15.1:0/3732048449 >> 172.21.15.1:0/3732048449 conn(0x55a50a4000f0 msgr2=0x55a50a3ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 -- 172.21.15.1:0/3732048449 shutdown_connections 2024-09-16T00:41:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.363+0000 7fd087c11740 1 -- 172.21.15.1:0/3732048449 wait complete. 2024-09-16T00:41:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd087c11740 20 client.18489 trim_cache size 0 max 16384 2024-09-16T00:41:03.376 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:41:03.377 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:41:03.377 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "a3ff0e5f-e751-4939-9bb7-2b482df69441", 2024-09-16T00:41:03.377 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:41:03.377 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:41:03.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 -- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd06c0a3240 msgr2=0x7fd06c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:03.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd06c0a3240 0x7fd06c0a5700 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd068009230 tx=0x7fd06803a000 comp rx=0 tx=0).stop 2024-09-16T00:41:03.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 -- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd080106f20 msgr2=0x7fd08019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:03.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd080106f20 0x7fd08019cd60 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd070009fd0 tx=0x7fd07000b8e0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 -- 172.21.15.1:0/3948611889 shutdown_connections 2024-09-16T00:41:03.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd080106f20 0x7fd08019cd60 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801027a0 0x7fd0801989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd06c0a3240 0x7fd06c0a5700 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 --2- 172.21.15.1:0/3948611889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd080101df0 0x7fd080198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:03.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 -- 172.21.15.1:0/3948611889 >> 172.21.15.1:0/3948611889 conn(0x7fd0800fd8f0 msgr2=0x7fd0800fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:03.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 -- 172.21.15.1:0/3948611889 shutdown_connections 2024-09-16T00:41:03.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:03.364+0000 7fd085838640 1 -- 172.21.15.1:0/3948611889 wait complete. 2024-09-16T00:41:04.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:41:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:04 smithi019 ceph-mon[35186]: pgmap v730: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:04 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:41:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:04 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:04 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:41:04.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:04 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:04.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:04 smithi001 ceph-mon[30934]: pgmap v730: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:04.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:04 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:41:04.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:04 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:04.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:04 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:41:04.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:04 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:04.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:04 smithi153 ceph-mon[35538]: pgmap v730: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:04.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:04 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:41:04.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:04 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:04.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:04 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:41:04.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:04 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:05.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.060+0000 7f56d8ecf640 1 -- 172.21.15.1:0/772510330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 msgr2=0x7f56d4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.060+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/772510330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 0x7f56d4100ee0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009930 tx=0x7f56bc02a420 comp rx=0 tx=0).stop 2024-09-16T00:41:05.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.060+0000 7f56d8ecf640 1 -- 172.21.15.1:0/772510330 shutdown_connections 2024-09-16T00:41:05.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.060+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/772510330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56d4101420 0x7f56d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.060+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/772510330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 0x7f56d4100ee0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.060+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/772510330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56d4106a80 0x7f56d4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.060+0000 7f56d8ecf640 1 -- 172.21.15.1:0/772510330 >> 172.21.15.1:0/772510330 conn(0x7f56d40fc7a0 msgr2=0x7f56d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.060+0000 7f56d8ecf640 1 -- 172.21.15.1:0/772510330 shutdown_connections 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.060+0000 7f56d8ecf640 1 -- 172.21.15.1:0/772510330 wait complete. 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d8ecf640 1 Processor -- start 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d8ecf640 1 -- start start 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d8ecf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 0x7f56d41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d8ecf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56d4101420 0x7f56d41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d8ecf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56d4106a80 0x7f56d419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d8ecf640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4111fa0 con 0x7f56d4100a80 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d8ecf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4111e20 con 0x7f56d4101420 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d8ecf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4112120 con 0x7f56d4106a80 2024-09-16T00:41:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 0x7f56d41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 0x7f56d41984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33084/0 (socket says 172.21.15.1:33084) 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d259c640 1 -- 172.21.15.1:0/2764736598 learned_addr learned my addr 172.21.15.1:0/2764736598 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56d4101420 0x7f56d41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.061+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56d4106a80 0x7f56d419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56d259c640 1 -- 172.21.15.1:0/2764736598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56d4106a80 msgr2=0x7f56d419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56d259c640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56d4106a80 0x7f56d419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56d259c640 1 -- 172.21.15.1:0/2764736598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56d4101420 msgr2=0x7f56d41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56d259c640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56d4101420 0x7f56d41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56d259c640 1 -- 172.21.15.1:0/2764736598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d419d450 con 0x7f56d4100a80 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56d4101420 0x7f56d41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56d259c640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 0x7f56d41984a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f56c400d950 tx=0x7f56c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56cbfff640 1 -- 172.21.15.1:0/2764736598 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4017030 con 0x7f56d4100a80 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56d8ecf640 1 -- 172.21.15.1:0/2764736598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d419d740 con 0x7f56d4100a80 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56cbfff640 1 -- 172.21.15.1:0/2764736598 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c4004020 con 0x7f56d4100a80 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56cbfff640 1 -- 172.21.15.1:0/2764736598 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c40047a0 con 0x7f56d4100a80 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.062+0000 7f56d8ecf640 1 -- 172.21.15.1:0/2764736598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d41a5020 con 0x7f56d4100a80 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.064+0000 7f56cbfff640 1 -- 172.21.15.1:0/2764736598 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f56c40041d0 con 0x7f56d4100a80 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56db2a8740 1 Processor -- start 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56db2a8740 1 -- start start 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 0x5557c118c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1197140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 0x5557c1199a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56db2a8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5557c10f22e0 con 0x5557c118cdc0 2024-09-16T00:41:05.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56db2a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5557c1099ce0 con 0x5557c118c4a0 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56db2a8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5557c10ee2a0 con 0x5557c1197680 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33090/0 (socket says 172.21.15.1:33090) 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56d259c640 1 -- 172.21.15.1:0/1949363859 learned_addr learned my addr 172.21.15.1:0/1949363859 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 0x5557c1199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.065+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 0x5557c118c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56cbfff640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56c00a3240 0x7f56c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56d259c640 1 -- 172.21.15.1:0/1949363859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 msgr2=0x5557c1199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56d259c640 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 0x5557c1199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56d259c640 1 -- 172.21.15.1:0/1949363859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 msgr2=0x5557c118c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56d259c640 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 0x5557c118c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56d259c640 1 -- 172.21.15.1:0/1949363859 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c107e440 con 0x5557c118cdc0 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 0x5557c118c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56c00a3240 0x7f56c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56cbfff640 1 -- 172.21.15.1:0/2764736598 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f56c40cf820 con 0x7f56d4100a80 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56d259c640 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1197140 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f56c4088d00 tx=0x7f56c4090ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba56bff8415273a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56c8d99640 1 -- 172.21.15.1:0/1949363859 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c409b520 con 0x5557c118cdc0 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56c8d99640 1 -- 172.21.15.1:0/1949363859 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c4066880 con 0x5557c118cdc0 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.066+0000 7f56c8d99640 1 -- 172.21.15.1:0/1949363859 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4088b60 con 0x5557c118cdc0 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 -- 172.21.15.1:0/1949363859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 msgr2=0x5557c1197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56c00a3240 0x7f56c00a5700 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00a4c0 tx=0x7f56cc00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1197140 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f56c4088d00 tx=0x7f56c4090ec0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 -- 172.21.15.1:0/1949363859 shutdown_connections 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 0x5557c1199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1197140 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 --2- 172.21.15.1:0/1949363859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 0x5557c118c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 -- 172.21.15.1:0/1949363859 >> 172.21.15.1:0/1949363859 conn(0x5557c1188a30 msgr2=0x5557c1188e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 -- 172.21.15.1:0/1949363859 shutdown_connections 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 -- 172.21.15.1:0/1949363859 wait complete. 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 -- start start 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.067+0000 7f56db2a8740 1 Processor -- start 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 0x5557c1196f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118cdc0 0x5557c1194230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c1197680 0x5557c1194770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56db2a8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5557c119a250 con 0x5557c1197680 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56db2a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5557c10ad050 con 0x5557c118cdc0 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56db2a8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5557c119a3d0 con 0x5557c118c4a0 2024-09-16T00:41:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56db2a8740 10 client.0 _ll_register_callbacks cb 0x5557c10ed160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c1197680 0x5557c1194770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c1197680 0x5557c1194770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33102/0 (socket says 172.21.15.1:33102) 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d2d9d640 1 -- 172.21.15.1:0/3195954659 learned_addr learned my addr 172.21.15.1:0/3195954659 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118cdc0 0x5557c1194230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d259c640 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 0x5557c1196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d2d9d640 1 -- 172.21.15.1:0/3195954659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 msgr2=0x5557c1196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 0x5557c1196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d2d9d640 1 -- 172.21.15.1:0/3195954659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118cdc0 msgr2=0x5557c1194230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118cdc0 0x5557c1194230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d2d9d640 1 -- 172.21.15.1:0/3195954659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c0e633c0 con 0x5557c1197680 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c1197680 0x5557c1194770 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009740 tx=0x7f56cc009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a9a2bfe55207e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.068+0000 7f56b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56b9ffb640 1 -- 172.21.15.1:0/3195954659 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc026060 con 0x5557c1197680 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56b9ffb640 1 -- 172.21.15.1:0/3195954659 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f56cc0180d0 con 0x5557c1197680 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56db2a8740 1 -- 172.21.15.1:0/3195954659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c4066880 con 0x5557c1197680 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56b9ffb640 1 -- 172.21.15.1:0/3195954659 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f56cc02d050 con 0x5557c1197680 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56db2a8740 10 client.18501 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56db2a8740 1 -- 172.21.15.1:0/3195954659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557c11a5c20 con 0x5557c1197680 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56b9ffb640 1 -- 172.21.15.1:0/3195954659 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c4066880 con 0x5557c1197680 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56b9ffb640 1 -- 172.21.15.1:0/3195954659 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f56cc0219e0 con 0x5557c1197680 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56db2a8740 10 client.18501 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56db2a8740 10 client.18501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56db2a8740 20 client.18501 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5557c11a70e0 0x5557c11a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56db2a8740 4 client.18501 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:05.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.069+0000 7f56db2a8740 1 -- 172.21.15.1:0/3195954659 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c10ff850 con 0x5557c11a70e0 2024-09-16T00:41:05.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.070+0000 7f56d259c640 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5557c11a70e0 0x5557c11a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.070+0000 7f56b9ffb640 10 client.18501 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:05.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.070+0000 7f56d259c640 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5557c11a70e0 0x5557c11a94d0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6d256a67b280ef1 server_cookie=1a15a3d21d166ec2 in_seq=0 out_seq=0 2024-09-16T00:41:05.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56b9ffb640 1 -- 172.21.15.1:0/3195954659 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557c10ff850 con 0x5557c11a70e0 2024-09-16T00:41:05.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56b9ffb640 10 client.18501 handle_command_reply: tid=0 2024-09-16T00:41:05.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 client.18501 shutdown 2024-09-16T00:41:05.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 -- 172.21.15.1:0/3195954659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c1197680 msgr2=0x5557c1194770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c1197680 0x5557c1194770 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009740 tx=0x7f56cc009620 comp rx=0 tx=0).stop 2024-09-16T00:41:05.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 -- 172.21.15.1:0/3195954659 shutdown_connections 2024-09-16T00:41:05.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5557c11a70e0 0x5557c11a94d0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c1197680 0x5557c1194770 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118cdc0 0x5557c1194230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3195954659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 0x5557c1196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56b9ffb640 0 client.18501 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:05.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 -- 172.21.15.1:0/3195954659 >> 172.21.15.1:0/3195954659 conn(0x5557c1188a30 msgr2=0x5557c1187b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 -- 172.21.15.1:0/3195954659 shutdown_connections 2024-09-16T00:41:05.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 1 -- 172.21.15.1:0/3195954659 wait complete. 2024-09-16T00:41:05.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.071+0000 7f56db2a8740 20 client.18501 trim_cache size 0 max 16384 2024-09-16T00:41:05.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56db2a8740 1 Processor -- start 2024-09-16T00:41:05.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56db2a8740 1 -- start start 2024-09-16T00:41:05.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 0x5557c11a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1188e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 0x5557c11aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56db2a8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41b18a0 con 0x5557c118cdc0 2024-09-16T00:41:05.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56db2a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41af170 con 0x5557c118c4a0 2024-09-16T00:41:05.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56db2a8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41acfc0 con 0x5557c1197680 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56c9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1188e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56c9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1188e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33112/0 (socket says 172.21.15.1:33112) 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56c9d9b640 1 -- 172.21.15.1:0/3880542652 learned_addr learned my addr 172.21.15.1:0/3880542652 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56d259c640 1 --2- 172.21.15.1:0/3880542652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 0x5557c11a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/3880542652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 0x5557c11aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56c9d9b640 1 -- 172.21.15.1:0/3880542652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 msgr2=0x5557c11aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/3880542652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 0x5557c11aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56c9d9b640 1 -- 172.21.15.1:0/3880542652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 msgr2=0x5557c11a34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/3880542652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 0x5557c11a34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.089+0000 7f56c9d9b640 1 -- 172.21.15.1:0/3880542652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c11931e0 con 0x5557c118cdc0 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/3880542652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1188e10 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f56bc02a940 tx=0x7f56bc032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cac4a5253d8d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56bbfff640 1 -- 172.21.15.1:0/3880542652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc035040 con 0x5557c118cdc0 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56bbfff640 1 -- 172.21.15.1:0/3880542652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56bc031870 con 0x5557c118cdc0 2024-09-16T00:41:05.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56bbfff640 1 -- 172.21.15.1:0/3880542652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004920 con 0x5557c118cdc0 2024-09-16T00:41:05.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56db2a8740 1 -- 172.21.15.1:0/3880542652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 msgr2=0x5557c1188e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3880542652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1188e10 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f56bc02a940 tx=0x7f56bc032f00 comp rx=0 tx=0).stop 2024-09-16T00:41:05.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56db2a8740 1 -- 172.21.15.1:0/3880542652 shutdown_connections 2024-09-16T00:41:05.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3880542652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c1197680 0x5557c11aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3880542652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1188e10 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56db2a8740 1 --2- 172.21.15.1:0/3880542652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c118c4a0 0x5557c11a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.090+0000 7f56db2a8740 1 -- 172.21.15.1:0/3880542652 >> 172.21.15.1:0/3880542652 conn(0x5557c1188a30 msgr2=0x5557c118f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 -- 172.21.15.1:0/3880542652 shutdown_connections 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 -- 172.21.15.1:0/3880542652 wait complete. 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 -- start start 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 Processor -- start 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 0x5557c11913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1191910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c1197680 0x5557c1191e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41ad630 con 0x5557c118cdc0 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41aeaa0 con 0x5557c1197680 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41add00 con 0x5557c118c4a0 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56db2a8740 10 client.0 _ll_register_callbacks cb 0x7f56cc009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56c9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1191910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56c9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1191910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33124/0 (socket says 172.21.15.1:33124) 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56c9d9b640 1 -- 172.21.15.1:0/2791676088 learned_addr learned my addr 172.21.15.1:0/2791676088 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56d2d9d640 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c1197680 0x5557c1191e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.091+0000 7f56d259c640 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 0x5557c11913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56c9d9b640 1 -- 172.21.15.1:0/2791676088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 msgr2=0x5557c11913d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 0x5557c11913d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56c9d9b640 1 -- 172.21.15.1:0/2791676088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c1197680 msgr2=0x5557c1191e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c1197680 0x5557c1191e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56c9d9b640 1 -- 172.21.15.1:0/2791676088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c11972f0 con 0x5557c118cdc0 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56c9d9b640 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1191910 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f56bc031550 tx=0x7f56bc0469a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910628e012b71ff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56b9ffb640 1 -- 172.21.15.1:0/2791676088 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc04d070 con 0x5557c118cdc0 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56db2a8740 1 -- 172.21.15.1:0/2791676088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56bc031870 con 0x5557c118cdc0 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56b9ffb640 1 -- 172.21.15.1:0/2791676088 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f56bc046d20 con 0x5557c118cdc0 2024-09-16T00:41:05.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56b9ffb640 1 -- 172.21.15.1:0/2791676088 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f56bc059050 con 0x5557c118cdc0 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56db2a8740 10 client.18507 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.092+0000 7f56db2a8740 1 -- 172.21.15.1:0/2791676088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557c1212030 con 0x5557c118cdc0 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56b9ffb640 1 -- 172.21.15.1:0/2791676088 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56bc031870 con 0x5557c118cdc0 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56b9ffb640 1 -- 172.21.15.1:0/2791676088 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f56bc03cac0 con 0x5557c118cdc0 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56db2a8740 10 client.18507 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56db2a8740 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56db2a8740 20 client.18507 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56db2a8740 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5557c12126e0 0x5557c1214b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56db2a8740 4 client.18507 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56db2a8740 1 -- 172.21.15.1:0/2791676088 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557c10f9d60 con 0x5557c12126e0 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56d259c640 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5557c12126e0 0x5557c1214b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56d259c640 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5557c12126e0 0x5557c1214b00 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc9adacf02f810e server_cookie=b5f44add9aefc45b in_seq=0 out_seq=0 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.093+0000 7f56b9ffb640 10 client.18507 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:05.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56b9ffb640 1 -- 172.21.15.1:0/2791676088 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5557c10f9d60 con 0x5557c12126e0 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56b9ffb640 10 client.18507 handle_command_reply: tid=0 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 client.18507 shutdown 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 -- 172.21.15.1:0/2791676088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 msgr2=0x5557c1191910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1191910 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f56bc031550 tx=0x7f56bc0469a0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 -- 172.21.15.1:0/2791676088 shutdown_connections 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5557c12126e0 0x5557c1214b00 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557c1197680 0x5557c1191e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5557c118cdc0 0x5557c1191910 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 --2- 172.21.15.1:0/2791676088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557c118c4a0 0x5557c11913d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56b9ffb640 0 client.18507 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 -- 172.21.15.1:0/2791676088 >> 172.21.15.1:0/2791676088 conn(0x5557c1188a30 msgr2=0x5557c1187350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 -- 172.21.15.1:0/2791676088 shutdown_connections 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 1 -- 172.21.15.1:0/2791676088 wait complete. 2024-09-16T00:41:05.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.094+0000 7f56db2a8740 20 client.18507 trim_cache size 0 max 16384 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout: "a3ff0e5f-e751-4939-9bb7-2b482df69441": { 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "a3ff0e5f-e751-4939-9bb7-2b482df69441", 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:41:05.100 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:41:05.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 -- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56c00a3240 msgr2=0x7f56c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56c00a3240 0x7f56c00a5700 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00a4c0 tx=0x7f56cc00c040 comp rx=0 tx=0).stop 2024-09-16T00:41:05.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 -- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 msgr2=0x7f56d41984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 0x7f56d41984a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f56c400d950 tx=0x7f56c400de10 comp rx=0 tx=0).stop 2024-09-16T00:41:05.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 -- 172.21.15.1:0/2764736598 shutdown_connections 2024-09-16T00:41:05.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f56c00a3240 0x7f56c00a5700 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56d4106a80 0x7f56d419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56d4101420 0x7f56d41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 --2- 172.21.15.1:0/2764736598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56d4100a80 0x7f56d41984a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 -- 172.21.15.1:0/2764736598 >> 172.21.15.1:0/2764736598 conn(0x7f56d40fc7a0 msgr2=0x7f56d40fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 -- 172.21.15.1:0/2764736598 shutdown_connections 2024-09-16T00:41:05.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:05.095+0000 7f56d8ecf640 1 -- 172.21.15.1:0/2764736598 wait complete. 2024-09-16T00:41:05.968 INFO:tasks.cephfs.filesystem:scrub status for tag:a3ff0e5f-e751-4939-9bb7-2b482df69441 - {'path': '/', 'tag': 'a3ff0e5f-e751-4939-9bb7-2b482df69441', 'options': 'recursive,force'} 2024-09-16T00:41:06.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:06 smithi019 ceph-mon[35186]: pgmap v731: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:06.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:06 smithi001 ceph-mon[30934]: pgmap v731: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:06 smithi153 ceph-mon[35538]: pgmap v731: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:08.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:08 smithi019 ceph-mon[35186]: pgmap v732: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:08.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:08 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:41:08.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:08 smithi001 ceph-mon[30934]: pgmap v732: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:08.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:08 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:41:08.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:08 smithi153 ceph-mon[35538]: pgmap v732: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:08.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:08 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:41:10.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:10 smithi019 ceph-mon[35186]: pgmap v733: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:10.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:10 smithi001 ceph-mon[30934]: pgmap v733: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:10.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:10 smithi153 ceph-mon[35538]: pgmap v733: 129 pgs: 129 active+clean; 5.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:12.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:12 smithi019 ceph-mon[35186]: pgmap v734: 129 pgs: 129 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:12.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:12 smithi001 ceph-mon[30934]: pgmap v734: 129 pgs: 129 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:12.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:12 smithi153 ceph-mon[35538]: pgmap v734: 129 pgs: 129 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:14.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:14 smithi019 ceph-mon[35186]: pgmap v735: 129 pgs: 129 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:14.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:14 smithi001 ceph-mon[30934]: pgmap v735: 129 pgs: 129 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:14.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:14 smithi153 ceph-mon[35538]: pgmap v735: 129 pgs: 129 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:16.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:16 smithi019 ceph-mon[35186]: pgmap v736: 129 pgs: 129 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:16.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:16 smithi001 ceph-mon[30934]: pgmap v736: 129 pgs: 129 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:16.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:16 smithi153 ceph-mon[35538]: pgmap v736: 129 pgs: 129 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:18.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:18 smithi019 ceph-mon[35186]: pgmap v737: 129 pgs: 129 active+clean; 5.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:18.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:18 smithi001 ceph-mon[30934]: pgmap v737: 129 pgs: 129 active+clean; 5.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:18.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:18 smithi153 ceph-mon[35538]: pgmap v737: 129 pgs: 129 active+clean; 5.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:20.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:20 smithi019 ceph-mon[35186]: pgmap v738: 129 pgs: 129 active+clean; 5.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:20.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:20 smithi001 ceph-mon[30934]: pgmap v738: 129 pgs: 129 active+clean; 5.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:20.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:20 smithi153 ceph-mon[35538]: pgmap v738: 129 pgs: 129 active+clean; 5.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:22.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:22 smithi019 ceph-mon[35186]: pgmap v739: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:41:22.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:22 smithi001 ceph-mon[30934]: pgmap v739: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:41:22.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:22 smithi153 ceph-mon[35538]: pgmap v739: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:41:24.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:24 smithi019 ceph-mon[35186]: pgmap v740: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:24.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:24 smithi001 ceph-mon[30934]: pgmap v740: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:24.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:24 smithi153 ceph-mon[35538]: pgmap v740: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:26.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:26 smithi019 ceph-mon[35186]: pgmap v741: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:26.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:26 smithi001 ceph-mon[30934]: pgmap v741: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:26.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:26 smithi153 ceph-mon[35538]: pgmap v741: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:28.607 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:28 smithi019 ceph-mon[35186]: pgmap v742: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:28.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:28 smithi001 ceph-mon[30934]: pgmap v742: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:28.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:28 smithi153 ceph-mon[35538]: pgmap v742: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:29.606 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:29 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:41:29.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:29 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:41:29.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:29 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:41:30.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:30 smithi001 ceph-mon[30934]: pgmap v743: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:30.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:41:30.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:41:30.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:30 smithi153 ceph-mon[35538]: pgmap v743: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:30.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:41:30.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:41:30.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:41:30.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:30 smithi019 ceph-mon[35186]: pgmap v743: 129 pgs: 129 active+clean; 6.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:30.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:41:30.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:41:30.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:41:32.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:32 smithi001 ceph-mon[30934]: pgmap v744: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:41:32.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:32 smithi153 ceph-mon[35538]: pgmap v744: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:41:32.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:32 smithi019 ceph-mon[35186]: pgmap v744: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:41:34.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:34 smithi001 ceph-mon[30934]: pgmap v745: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:34.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:34 smithi153 ceph-mon[35538]: pgmap v745: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:34.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:34 smithi019 ceph-mon[35186]: pgmap v745: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:35.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:41:36.241 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:36.670 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:36 smithi001 ceph-mon[30934]: pgmap v746: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:36.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:36 smithi153 ceph-mon[35538]: pgmap v746: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.755+0000 7ff0f6485640 1 -- 172.21.15.1:0/1881332620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 msgr2=0x7ff0f010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.755+0000 7ff0f6485640 1 --2- 172.21.15.1:0/1881332620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 0x7ff0f010e610 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009ca0 tx=0x7ff0e802a950 comp rx=0 tx=0).stop 2024-09-16T00:41:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.755+0000 7ff0f6485640 1 -- 172.21.15.1:0/1881332620 shutdown_connections 2024-09-16T00:41:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.755+0000 7ff0f6485640 1 --2- 172.21.15.1:0/1881332620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 0x7ff0f010e610 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.755+0000 7ff0f6485640 1 --2- 172.21.15.1:0/1881332620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f0103660 0x7ff0f010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.755+0000 7ff0f6485640 1 --2- 172.21.15.1:0/1881332620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f0102d40 0x7ff0f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.755+0000 7ff0f6485640 1 -- 172.21.15.1:0/1881332620 >> 172.21.15.1:0/1881332620 conn(0x7ff0f00fc7a0 msgr2=0x7ff0f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.755+0000 7ff0f6485640 1 -- 172.21.15.1:0/1881332620 shutdown_connections 2024-09-16T00:41:36.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.755+0000 7ff0f6485640 1 -- 172.21.15.1:0/1881332620 wait complete. 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.756+0000 7ff0f6485640 1 Processor -- start 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.756+0000 7ff0f6485640 1 -- start start 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.756+0000 7ff0f6485640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f0102d40 0x7ff0f01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.756+0000 7ff0f6485640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f0103660 0x7ff0f0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.756+0000 7ff0f6485640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 0x7ff0f0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.756+0000 7ff0f6485640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0111f30 con 0x7ff0f010c170 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f6485640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0111db0 con 0x7ff0f0102d40 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f6485640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01120b0 con 0x7ff0f0103660 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f4a22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 0x7ff0f0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f4a22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 0x7ff0f0101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41226/0 (socket says 172.21.15.1:41226) 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f4a22640 1 -- 172.21.15.1:0/167570526 learned_addr learned my addr 172.21.15.1:0/167570526 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0effff640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f0102d40 0x7ff0f01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f0103660 0x7ff0f0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f4a22640 1 -- 172.21.15.1:0/167570526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f0103660 msgr2=0x7ff0f0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f4a22640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f0103660 0x7ff0f0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f4a22640 1 -- 172.21.15.1:0/167570526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f0102d40 msgr2=0x7ff0f01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f4a22640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f0102d40 0x7ff0f01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f4a22640 1 -- 172.21.15.1:0/167570526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f01a6890 con 0x7ff0f010c170 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f0103660 0x7ff0f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f4a22640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 0x7ff0f0101420 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8004af0 tx=0x7ff0e802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0ed7fa640 1 -- 172.21.15.1:0/167570526 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8034040 con 0x7ff0f010c170 2024-09-16T00:41:36.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f6485640 1 -- 172.21.15.1:0/167570526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f01a6b20 con 0x7ff0f010c170 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0ed7fa640 1 -- 172.21.15.1:0/167570526 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e8030920 con 0x7ff0f010c170 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0ed7fa640 1 -- 172.21.15.1:0/167570526 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80047b0 con 0x7ff0f010c170 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.757+0000 7ff0f6485640 1 -- 172.21.15.1:0/167570526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f01a7050 con 0x7ff0f010c170 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0ed7fa640 1 -- 172.21.15.1:0/167570526 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff0e8037020 con 0x7ff0f010c170 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0f885e740 1 Processor -- start 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0f885e740 1 -- start start 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb57880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb62140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb62680 0x55f8ecb64a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0f885e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f8ecabd2e0 con 0x55f8ecb574a0 2024-09-16T00:41:36.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0f885e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f8eca64ce0 con 0x55f8ecb57dc0 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0f885e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f8ecab92a0 con 0x55f8ecb62680 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb57880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb57880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41240/0 (socket says 172.21.15.1:41240) 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/1563272428 learned_addr learned my addr 172.21.15.1:0/1563272428 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0effff640 1 --2- 172.21.15.1:0/1563272428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb62140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.760+0000 7ff0ed7fa640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0d80a3240 0x7ff0d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0f4a22640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0d80a3240 0x7ff0d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/1563272428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb62680 msgr2=0x55f8ecb64a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/1563272428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb62680 0x55f8ecb64a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/1563272428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb57dc0 msgr2=0x55f8ecb62140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/1563272428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb62140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/1563272428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8eca49440 con 0x55f8ecb574a0 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0ed7fa640 1 -- 172.21.15.1:0/167570526 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff0e80f0ae0 con 0x7ff0f010c170 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0effff640 1 --2- 172.21.15.1:0/1563272428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb62140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/1563272428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb57880 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009bf0 tx=0x7ff0e4012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2defcee3fe312c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0d67fc640 1 -- 172.21.15.1:0/1563272428 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4004030 con 0x55f8ecb574a0 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0d67fc640 1 -- 172.21.15.1:0/1563272428 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e40041d0 con 0x55f8ecb574a0 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0f4a22640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0d80a3240 0x7ff0d80a5700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80bcc90 tx=0x7ff0e8087b30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.761+0000 7ff0d67fc640 1 -- 172.21.15.1:0/1563272428 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4013660 con 0x55f8ecb574a0 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 -- 172.21.15.1:0/1563272428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 msgr2=0x55f8ecb57880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 --2- 172.21.15.1:0/1563272428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb57880 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009bf0 tx=0x7ff0e4012760 comp rx=0 tx=0).stop 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 -- 172.21.15.1:0/1563272428 shutdown_connections 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 --2- 172.21.15.1:0/1563272428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb62680 0x55f8ecb64a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 --2- 172.21.15.1:0/1563272428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb62140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 --2- 172.21.15.1:0/1563272428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb57880 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 -- 172.21.15.1:0/1563272428 >> 172.21.15.1:0/1563272428 conn(0x55f8ecb53a30 msgr2=0x55f8ecb53e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 -- 172.21.15.1:0/1563272428 shutdown_connections 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 -- 172.21.15.1:0/1563272428 wait complete. 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 -- start start 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.762+0000 7ff0f885e740 1 Processor -- start 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb574a0 0x55f8ecb61f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb62680 0x55f8ecb5f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0f885e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f8ecb65250 con 0x55f8ecb62680 2024-09-16T00:41:36.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0f885e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f8eca78050 con 0x55f8ecb574a0 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0f885e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f8ecb653d0 con 0x55f8ecb57dc0 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0f885e740 10 client.0 _ll_register_callbacks cb 0x55f8ecab8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb62680 0x55f8ecb5f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb62680 0x55f8ecb5f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41250/0 (socket says 172.21.15.1:41250) 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/126867700 learned_addr learned my addr 172.21.15.1:0/126867700 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0effff640 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb574a0 0x55f8ecb61f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/126867700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 msgr2=0x55f8ecb5f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/126867700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb574a0 msgr2=0x55f8ecb61f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb574a0 0x55f8ecb61f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/126867700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ec82e3c0 con 0x55f8ecb62680 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb62680 0x55f8ecb5f730 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4003f00 tx=0x7ff0e4003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f58febbfa1676a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.763+0000 7ff0cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/126867700 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e401f070 con 0x55f8ecb62680 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/126867700 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff0e4003800 con 0x55f8ecb62680 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0f885e740 1 -- 172.21.15.1:0/126867700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0e40041d0 con 0x55f8ecb62680 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/126867700 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0e4003a00 con 0x55f8ecb62680 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0f885e740 10 client.18519 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0f885e740 1 -- 172.21.15.1:0/126867700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8ecb70bd0 con 0x55f8ecb62680 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/126867700 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e40041d0 con 0x55f8ecb62680 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/126867700 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e400f040 con 0x55f8ecb62680 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0f885e740 10 client.18519 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0f885e740 10 client.18519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:36.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0f885e740 20 client.18519 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0f885e740 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8ecb72090 0x55f8ecb74480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0f885e740 4 client.18519 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0f885e740 1 -- 172.21.15.1:0/126867700 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8ecaca850 con 0x55f8ecb72090 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.764+0000 7ff0effff640 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8ecb72090 0x55f8ecb74480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.765+0000 7ff0cf7fe640 10 client.18519 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.765+0000 7ff0effff640 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8ecb72090 0x55f8ecb74480 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d6e5e9b1c8b0ea server_cookie=ff11cc3fd6ada246 in_seq=0 out_seq=0 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/126867700 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8ecaca850 con 0x55f8ecb72090 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0cf7fe640 10 client.18519 handle_command_reply: tid=0 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 client.18519 shutdown 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 -- 172.21.15.1:0/126867700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb62680 msgr2=0x55f8ecb5f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb62680 0x55f8ecb5f730 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4003f00 tx=0x7ff0e4003e00 comp rx=0 tx=0).stop 2024-09-16T00:41:36.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 -- 172.21.15.1:0/126867700 shutdown_connections 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8ecb72090 0x55f8ecb74480 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb62680 0x55f8ecb5f730 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 --2- 172.21.15.1:0/126867700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb574a0 0x55f8ecb61f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0cf7fe640 0 client.18519 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 -- 172.21.15.1:0/126867700 >> 172.21.15.1:0/126867700 conn(0x55f8ecb53a30 msgr2=0x55f8ecb52b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 -- 172.21.15.1:0/126867700 shutdown_connections 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 1 -- 172.21.15.1:0/126867700 wait complete. 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.766+0000 7ff0f885e740 20 client.18519 trim_cache size 0 max 16384 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0f885e740 1 Processor -- start 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0f885e740 1 -- start start 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb53a30 0x55f8ecb6e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb574a0 0x55f8ecb5f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0f885e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01b38f0 con 0x55f8ecb57dc0 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0f885e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01b11c0 con 0x55f8ecb574a0 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0f885e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01af010 con 0x55f8ecb53a30 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41262/0 (socket says 172.21.15.1:41262) 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/1542244358 learned_addr learned my addr 172.21.15.1:0/1542244358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/1542244358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb53a30 msgr2=0x55f8ecb6e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/1542244358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb53a30 0x55f8ecb6e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.783+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/1542244358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb574a0 msgr2=0x55f8ecb5f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/1542244358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb574a0 0x55f8ecb5f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0ef7fe640 1 -- 172.21.15.1:0/1542244358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ecb57bf0 con 0x55f8ecb57dc0 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/1542244358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5a7f0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005e00 tx=0x7ff0e4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de59aac02861038 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:36.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0d5ffb640 1 -- 172.21.15.1:0/1542244358 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4033040 con 0x55f8ecb57dc0 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0d5ffb640 1 -- 172.21.15.1:0/1542244358 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e4012c60 con 0x55f8ecb57dc0 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0d5ffb640 1 -- 172.21.15.1:0/1542244358 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4003800 con 0x55f8ecb57dc0 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0f885e740 1 -- 172.21.15.1:0/1542244358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb57dc0 msgr2=0x55f8ecb5a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0f885e740 1 --2- 172.21.15.1:0/1542244358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5a7f0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005e00 tx=0x7ff0e4005f90 comp rx=0 tx=0).stop 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0f885e740 1 -- 172.21.15.1:0/1542244358 shutdown_connections 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0f885e740 1 --2- 172.21.15.1:0/1542244358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb5a7f0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0f885e740 1 --2- 172.21.15.1:0/1542244358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb574a0 0x55f8ecb5f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0f885e740 1 --2- 172.21.15.1:0/1542244358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb53a30 0x55f8ecb6e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0f885e740 1 -- 172.21.15.1:0/1542244358 >> 172.21.15.1:0/1542244358 conn(0x55f8ecb5f890 msgr2=0x55f8ecb5a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.784+0000 7ff0f885e740 1 -- 172.21.15.1:0/1542244358 shutdown_connections 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 1 -- 172.21.15.1:0/1542244358 wait complete. 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 1 -- start start 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 1 Processor -- start 2024-09-16T00:41:36.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb53a30 0x55f8ecb66350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb66890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb66dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01af680 con 0x55f8ecb574a0 2024-09-16T00:41:36.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01b0af0 con 0x55f8ecb53a30 2024-09-16T00:41:36.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01afd50 con 0x55f8ecb57dc0 2024-09-16T00:41:36.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0f885e740 10 client.0 _ll_register_callbacks cb 0x7ff0e000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:36.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0effff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb66890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0effff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb66890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41266/0 (socket says 172.21.15.1:41266) 2024-09-16T00:41:36.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0effff640 1 -- 172.21.15.1:0/969777117 learned_addr learned my addr 172.21.15.1:0/969777117 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.785+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb53a30 0x55f8ecb66350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb66dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0effff640 1 -- 172.21.15.1:0/969777117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 msgr2=0x55f8ecb66dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0effff640 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb66dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0effff640 1 -- 172.21.15.1:0/969777117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb53a30 msgr2=0x55f8ecb66350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0effff640 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb53a30 0x55f8ecb66350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0effff640 1 -- 172.21.15.1:0/969777117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ecbdbc10 con 0x55f8ecb574a0 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0effff640 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb66890 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000b7e0 tx=0x7ff0e0011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d687a36f6d3a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/969777117 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0004220 con 0x55f8ecb574a0 2024-09-16T00:41:36.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/969777117 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff0e00043c0 con 0x55f8ecb574a0 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0f885e740 1 -- 172.21.15.1:0/969777117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0e4012c60 con 0x55f8ecb574a0 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/969777117 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0e001d780 con 0x55f8ecb574a0 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0f885e740 10 client.18525 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.786+0000 7ff0f885e740 1 -- 172.21.15.1:0/969777117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8ecbdcf30 con 0x55f8ecb574a0 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.787+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/969777117 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e4012c60 con 0x55f8ecb574a0 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/969777117 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e0003bc0 con 0x55f8ecb574a0 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0f885e740 10 client.18525 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0f885e740 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0f885e740 20 client.18525 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0f885e740 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8ecb62680 0x55f8ecbdf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0f885e740 4 client.18525 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:41:36.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0f885e740 1 -- 172.21.15.1:0/969777117 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8ecac4d60 con 0x55f8ecb62680 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8ecb62680 0x55f8ecbdf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0cf7fe640 10 client.18525 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0ef7fe640 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8ecb62680 0x55f8ecbdf6c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da3b69a77c40b142 server_cookie=81beb5fd184362c3 in_seq=0 out_seq=0 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0cf7fe640 1 -- 172.21.15.1:0/969777117 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f8ecac4d60 con 0x55f8ecb62680 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.788+0000 7ff0cf7fe640 10 client.18525 handle_command_reply: tid=0 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 client.18525 shutdown 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 -- 172.21.15.1:0/969777117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 msgr2=0x55f8ecb66890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb66890 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000b7e0 tx=0x7ff0e0011960 comp rx=0 tx=0).stop 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 -- 172.21.15.1:0/969777117 shutdown_connections 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55f8ecb62680 0x55f8ecbdf6c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ecb57dc0 0x55f8ecb66dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8ecb574a0 0x55f8ecb66890 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 --2- 172.21.15.1:0/969777117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8ecb53a30 0x55f8ecb66350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0cf7fe640 0 client.18525 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:36.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 -- 172.21.15.1:0/969777117 >> 172.21.15.1:0/969777117 conn(0x55f8ecb5f890 msgr2=0x55f8ecb61270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:36.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 -- 172.21.15.1:0/969777117 shutdown_connections 2024-09-16T00:41:36.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 1 -- 172.21.15.1:0/969777117 wait complete. 2024-09-16T00:41:36.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f885e740 20 client.18525 trim_cache size 0 max 16384 2024-09-16T00:41:36.808 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:41:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:41:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:41:36.808 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:41:36.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f6485640 1 -- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0d80a3240 msgr2=0x7ff0d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:36.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f6485640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0d80a3240 0x7ff0d80a5700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80bcc90 tx=0x7ff0e8087b30 comp rx=0 tx=0).stop 2024-09-16T00:41:36.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f6485640 1 -- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 msgr2=0x7ff0f0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:36.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.789+0000 7ff0f6485640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 0x7ff0f0101420 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8004af0 tx=0x7ff0e802f5b0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.790+0000 7ff0f6485640 1 -- 172.21.15.1:0/167570526 shutdown_connections 2024-09-16T00:41:36.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.790+0000 7ff0f6485640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0f010c170 0x7ff0f0101420 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.790+0000 7ff0f6485640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff0d80a3240 0x7ff0d80a5700 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.790+0000 7ff0f6485640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f0103660 0x7ff0f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.790+0000 7ff0f6485640 1 --2- 172.21.15.1:0/167570526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f0102d40 0x7ff0f01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:36.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.790+0000 7ff0f6485640 1 -- 172.21.15.1:0/167570526 >> 172.21.15.1:0/167570526 conn(0x7ff0f00fc7a0 msgr2=0x7ff0f00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:36.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.790+0000 7ff0f6485640 1 -- 172.21.15.1:0/167570526 shutdown_connections 2024-09-16T00:41:36.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:36.790+0000 7ff0f6485640 1 -- 172.21.15.1:0/167570526 wait complete. 2024-09-16T00:41:36.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:36 smithi019 ceph-mon[35186]: pgmap v746: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:37.640 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:41:37.641 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:37.901 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:38.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.414+0000 7f1289326640 1 -- 172.21.15.1:0/3654068033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 msgr2=0x7f128410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:38.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.414+0000 7f1289326640 1 --2- 172.21.15.1:0/3654068033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 0x7f128410ec40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f1274009cd0 tx=0x7f127402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.414+0000 7f1289326640 1 -- 172.21.15.1:0/3654068033 shutdown_connections 2024-09-16T00:41:38.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.414+0000 7f1289326640 1 --2- 172.21.15.1:0/3654068033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 0x7f128410ec40 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.414+0000 7f1289326640 1 --2- 172.21.15.1:0/3654068033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12841036e0 0x7f1284107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.414+0000 7f1289326640 1 --2- 172.21.15.1:0/3654068033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1284102d30 0x7f1284103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.414+0000 7f1289326640 1 -- 172.21.15.1:0/3654068033 >> 172.21.15.1:0/3654068033 conn(0x7f12840fc7d0 msgr2=0x7f12840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:38.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.414+0000 7f1289326640 1 -- 172.21.15.1:0/3654068033 shutdown_connections 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.414+0000 7f1289326640 1 -- 172.21.15.1:0/3654068033 wait complete. 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f1289326640 1 Processor -- start 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f1289326640 1 -- start start 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f1289326640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1284102d30 0x7f1284100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f1289326640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12841036e0 0x7f1284100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f1289326640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 0x7f1284101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f1289326640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12841120a0 con 0x7f12841082a0 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f1289326640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1284111f20 con 0x7f1284102d30 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f1289326640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1284112220 con 0x7f12841036e0 2024-09-16T00:41:38.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f12837fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 0x7f1284101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f12837fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 0x7f1284101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41282/0 (socket says 172.21.15.1:41282) 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f12837fe640 1 -- 172.21.15.1:0/2283755401 learned_addr learned my addr 172.21.15.1:0/2283755401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f1282ffd640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1284102d30 0x7f1284100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.415+0000 7f12827fc640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12841036e0 0x7f1284100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f12837fe640 1 -- 172.21.15.1:0/2283755401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12841036e0 msgr2=0x7f1284100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f12837fe640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12841036e0 0x7f1284100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f12837fe640 1 -- 172.21.15.1:0/2283755401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1284102d30 msgr2=0x7f1284100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f12837fe640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1284102d30 0x7f1284100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f12837fe640 1 -- 172.21.15.1:0/2283755401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12841a6a20 con 0x7f12841082a0 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f1282ffd640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1284102d30 0x7f1284100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f12837fe640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 0x7f1284101410 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f1274005cc0 tx=0x7f1274003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:38.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f127bfff640 1 -- 172.21.15.1:0/2283755401 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274004040 con 0x7f12841082a0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f127bfff640 1 -- 172.21.15.1:0/2283755401 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12740046c0 con 0x7f12841082a0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12841a6cb0 con 0x7f12841082a0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f127bfff640 1 -- 172.21.15.1:0/2283755401 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274046480 con 0x7f12841082a0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.416+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12841a71f0 con 0x7f12841082a0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.417+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1284103110 con 0x7f12841082a0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.418+0000 7f127bfff640 1 -- 172.21.15.1:0/2283755401 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f12740305a0 con 0x7f12841082a0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.419+0000 7f127bfff640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12700a3240 0x7f12700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.419+0000 7f1282ffd640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12700a3240 0x7f12700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.419+0000 7f127bfff640 1 -- 172.21.15.1:0/2283755401 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f127402e030 con 0x7f12841082a0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.420+0000 7f1282ffd640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12700a3240 0x7f12700a5700 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f127c00bfd0 tx=0x7f127c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:38.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.423+0000 7f127bfff640 1 -- 172.21.15.1:0/2283755401 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f127404b060 con 0x7f12841082a0 2024-09-16T00:41:38.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.634+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f128410a2b0 con 0x7f12841082a0 2024-09-16T00:41:38.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:38 smithi001 ceph-mon[30934]: pgmap v747: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.635+0000 7f127bfff640 1 -- 172.21.15.1:0/2283755401 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f12740bdb70 con 0x7f12841082a0 2024-09-16T00:41:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:41:38.636 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:41:38.636 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:41:38.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12700a3240 msgr2=0x7f12700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12700a3240 0x7f12700a5700 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f127c00bfd0 tx=0x7f127c00b390 comp rx=0 tx=0).stop 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 msgr2=0x7f1284101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 0x7f1284101410 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f1274005cc0 tx=0x7f1274003970 comp rx=0 tx=0).stop 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 shutdown_connections 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12841082a0 0x7f1284101410 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f12700a3240 0x7f12700a5700 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12841036e0 0x7f1284100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 --2- 172.21.15.1:0/2283755401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1284102d30 0x7f1284100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 >> 172.21.15.1:0/2283755401 conn(0x7f12840fc7d0 msgr2=0x7f1284105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 shutdown_connections 2024-09-16T00:41:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:38.637+0000 7f1289326640 1 -- 172.21.15.1:0/2283755401 wait complete. 2024-09-16T00:41:38.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:38 smithi153 ceph-mon[35538]: pgmap v747: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:38.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:38 smithi019 ceph-mon[35186]: pgmap v747: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:39.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:41:39.589 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:39.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:39 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2283755401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:39.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:39 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2283755401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:39.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:39 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2283755401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:40.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.098+0000 7fc6150e5640 1 -- 172.21.15.1:0/1261256828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc610102790 msgr2=0x7fc6101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:40.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.098+0000 7fc6150e5640 1 --2- 172.21.15.1:0/1261256828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc610102790 0x7fc6101067e0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009900 tx=0x7fc5f802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.098+0000 7fc6150e5640 1 -- 172.21.15.1:0/1261256828 shutdown_connections 2024-09-16T00:41:40.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.098+0000 7fc6150e5640 1 --2- 172.21.15.1:0/1261256828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610106f10 0x7fc61010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.098+0000 7fc6150e5640 1 --2- 172.21.15.1:0/1261256828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc610102790 0x7fc6101067e0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.098+0000 7fc6150e5640 1 --2- 172.21.15.1:0/1261256828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc610101de0 0x7fc6101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.098+0000 7fc6150e5640 1 -- 172.21.15.1:0/1261256828 >> 172.21.15.1:0/1261256828 conn(0x7fc6100fd8c0 msgr2=0x7fc6100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:40.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.098+0000 7fc6150e5640 1 -- 172.21.15.1:0/1261256828 shutdown_connections 2024-09-16T00:41:40.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.098+0000 7fc6150e5640 1 -- 172.21.15.1:0/1261256828 wait complete. 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc6150e5640 1 Processor -- start 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc6150e5640 1 -- start start 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc6150e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610101de0 0x7fc610198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc6150e5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc610102790 0x7fc6101989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc6150e5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc610106f10 0x7fc61019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc6150e5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc610111f90 con 0x7fc610106f10 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc6150e5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc610111e10 con 0x7fc610101de0 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc6150e5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc610112110 con 0x7fc610102790 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc60ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610101de0 0x7fc610198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc60f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc610106f10 0x7fc61019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc60ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610101de0 0x7fc610198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:44272/0 (socket says 172.21.15.1:44272) 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.099+0000 7fc60ed9d640 1 -- 172.21.15.1:0/2445752897 learned_addr learned my addr 172.21.15.1:0/2445752897 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:40.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc60e59c640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc610102790 0x7fc6101989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc60f59e640 1 -- 172.21.15.1:0/2445752897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc610102790 msgr2=0x7fc6101989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc60f59e640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc610102790 0x7fc6101989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc60f59e640 1 -- 172.21.15.1:0/2445752897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610101de0 msgr2=0x7fc610198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc60f59e640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610101de0 0x7fc610198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc60f59e640 1 -- 172.21.15.1:0/2445752897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61019d440 con 0x7fc610106f10 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610101de0 0x7fc610198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc60f59e640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc610106f10 0x7fc61019cd60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e940 tx=0x7fc60800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc607fff640 1 -- 172.21.15.1:0/2445752897 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608004d20 con 0x7fc610106f10 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc607fff640 1 -- 172.21.15.1:0/2445752897 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc608005020 con 0x7fc610106f10 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc6150e5640 1 -- 172.21.15.1:0/2445752897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61019d730 con 0x7fc610106f10 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc607fff640 1 -- 172.21.15.1:0/2445752897 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080116b0 con 0x7fc610106f10 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.100+0000 7fc6150e5640 1 -- 172.21.15.1:0/2445752897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6101a5010 con 0x7fc610106f10 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.102+0000 7fc607fff640 1 -- 172.21.15.1:0/2445752897 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc6080054d0 con 0x7fc610106f10 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.103+0000 7fc6174be740 1 Processor -- start 2024-09-16T00:41:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.103+0000 7fc6174be740 1 -- start start 2024-09-16T00:41:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.103+0000 7fc6174be740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.103+0000 7fc6174be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.103+0000 7fc6174be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.103+0000 7fc6174be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bff602d2f0 con 0x55bff60d6780 2024-09-16T00:41:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.103+0000 7fc6174be740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bff5fd4cf0 con 0x55bff60c8b40 2024-09-16T00:41:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.103+0000 7fc6174be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bff60292b0 con 0x55bff60c75d0 2024-09-16T00:41:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc607fff640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc5fc0a32c0 0x7fc5fc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60d8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41296/0 (socket says 172.21.15.1:41296) 2024-09-16T00:41:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 -- 172.21.15.1:0/1389854103 learned_addr learned my addr 172.21.15.1:0/1389854103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60f59e640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc5fc0a32c0 0x7fc5fc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc607fff640 1 -- 172.21.15.1:0/2445752897 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc6080cfd70 con 0x7fc610106f10 2024-09-16T00:41:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 -- 172.21.15.1:0/1389854103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 msgr2=0x55bff60c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 -- 172.21.15.1:0/1389854103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 msgr2=0x55bff60c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 -- 172.21.15.1:0/1389854103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff5fb93a0 con 0x55bff60d6780 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60e59c640 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60d8b70 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009900 tx=0x7fc5f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603dd632f67d6d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc60f59e640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc5fc0a32c0 0x7fc5fc0a5780 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc608005c40 tx=0x7fc608076040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.104+0000 7fc6057fa640 1 -- 172.21.15.1:0/1389854103 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8035070 con 0x55bff60d6780 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6057fa640 1 -- 172.21.15.1:0/1389854103 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8005020 con 0x55bff60d6780 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6057fa640 1 -- 172.21.15.1:0/1389854103 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8004a00 con 0x55bff60d6780 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6174be740 1 -- 172.21.15.1:0/1389854103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 msgr2=0x55bff60d8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6174be740 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60d8b70 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009900 tx=0x7fc5f8000bc0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6174be740 1 -- 172.21.15.1:0/1389854103 shutdown_connections 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6174be740 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60d8b70 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6174be740 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6174be740 1 --2- 172.21.15.1:0/1389854103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6174be740 1 -- 172.21.15.1:0/1389854103 >> 172.21.15.1:0/1389854103 conn(0x55bff6090310 msgr2=0x55bff60c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6174be740 1 -- 172.21.15.1:0/1389854103 shutdown_connections 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.105+0000 7fc6174be740 1 -- 172.21.15.1:0/1389854103 wait complete. 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc6174be740 1 -- start start 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc6174be740 1 Processor -- start 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc6174be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c75d0 0x55bff60d61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc6174be740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc6174be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60d6780 0x55bff60d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc6174be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bff60d9420 con 0x55bff60c75d0 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc6174be740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bff5fe8060 con 0x55bff60c8b40 2024-09-16T00:41:40.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc6174be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bff60d95a0 con 0x55bff60d6780 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc6174be740 10 client.0 _ll_register_callbacks cb 0x55bff5d3b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc60e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c75d0 0x55bff60d61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc60e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c75d0 0x55bff60d61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41304/0 (socket says 172.21.15.1:41304) 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc60e59c640 1 -- 172.21.15.1:0/1708847281 learned_addr learned my addr 172.21.15.1:0/1708847281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc60e59c640 1 -- 172.21.15.1:0/1708847281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60d6780 msgr2=0x55bff60d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.106+0000 7fc60e59c640 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60d6780 0x55bff60d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc60e59c640 1 -- 172.21.15.1:0/1708847281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 msgr2=0x55bff60d3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc60e59c640 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60d3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc60e59c640 1 -- 172.21.15.1:0/1708847281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff5d9e3c0 con 0x55bff60c75d0 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60d3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc60e59c640 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c75d0 0x55bff60d61c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8005ec0 tx=0x7fc5f8031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0929a3e8f7c68d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc5f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc5f67fc640 1 -- 172.21.15.1:0/1708847281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f804e020 con 0x55bff60c75d0 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc5f67fc640 1 -- 172.21.15.1:0/1708847281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc5f8004750 con 0x55bff60c75d0 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc6174be740 1 -- 172.21.15.1:0/1708847281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f8005020 con 0x55bff60c75d0 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc5f67fc640 1 -- 172.21.15.1:0/1708847281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5f8059050 con 0x55bff60c75d0 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc6174be740 10 client.18543 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc6174be740 1 -- 172.21.15.1:0/1708847281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bff60d9420 con 0x55bff60c75d0 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc5f67fc640 1 -- 172.21.15.1:0/1708847281 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8005020 con 0x55bff60c75d0 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.107+0000 7fc5f67fc640 1 -- 172.21.15.1:0/1708847281 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc5f803d8a0 con 0x55bff60c75d0 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.108+0000 7fc6174be740 10 client.18543 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.108+0000 7fc6174be740 10 client.18543 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.108+0000 7fc6174be740 20 client.18543 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.108+0000 7fc6174be740 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bff60e6090 0x55bff60e8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.108+0000 7fc6174be740 4 client.18543 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:40.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.108+0000 7fc6174be740 1 -- 172.21.15.1:0/1708847281 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bff603a860 con 0x55bff60e6090 2024-09-16T00:41:40.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.108+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bff60e6090 0x55bff60e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.108+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bff60e6090 0x55bff60e8480 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d1cc9c1e5942c4 server_cookie=8c04084d096c286c in_seq=0 out_seq=0 2024-09-16T00:41:40.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.108+0000 7fc5f67fc640 10 client.18543 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:41:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc5f67fc640 1 -- 172.21.15.1:0/1708847281 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bff603a860 con 0x55bff60e6090 2024-09-16T00:41:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc5f67fc640 10 client.18543 handle_command_reply: tid=0 2024-09-16T00:41:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc6174be740 1 client.18543 shutdown 2024-09-16T00:41:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc6174be740 1 -- 172.21.15.1:0/1708847281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c75d0 msgr2=0x55bff60d61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc6174be740 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c75d0 0x55bff60d61c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8005ec0 tx=0x7fc5f8031b50 comp rx=0 tx=0).stop 2024-09-16T00:41:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc6174be740 1 -- 172.21.15.1:0/1708847281 shutdown_connections 2024-09-16T00:41:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc6174be740 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60d6780 0x55bff60d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc6174be740 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bff60e6090 0x55bff60e8480 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc6174be740 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60d3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc6174be740 1 --2- 172.21.15.1:0/1708847281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c75d0 0x55bff60d61c0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc6174be740 1 -- 172.21.15.1:0/1708847281 >> 172.21.15.1:0/1708847281 conn(0x55bff6090310 msgr2=0x55bff60c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.109+0000 7fc5f67fc640 0 client.18543 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:41:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.110+0000 7fc6174be740 1 -- 172.21.15.1:0/1708847281 shutdown_connections 2024-09-16T00:41:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.110+0000 7fc6174be740 1 -- 172.21.15.1:0/1708847281 wait complete. 2024-09-16T00:41:40.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.110+0000 7fc6174be740 20 client.18543 trim_cache size 0 max 16384 2024-09-16T00:41:40.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.122+0000 7fc6174be740 1 Processor -- start 2024-09-16T00:41:40.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.122+0000 7fc6174be740 1 -- start start 2024-09-16T00:41:40.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc6174be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc6174be740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60edb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc6174be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60ee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc6174be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101af490 con 0x55bff60d6780 2024-09-16T00:41:40.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc6174be740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101acd60 con 0x55bff60c8b40 2024-09-16T00:41:40.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc6174be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101aabb0 con 0x55bff60c75d0 2024-09-16T00:41:40.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60ee050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41314/0 (socket says 172.21.15.1:41314) 2024-09-16T00:41:40.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 -- 172.21.15.1:0/3787381569 learned_addr learned my addr 172.21.15.1:0/3787381569 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60edb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 -- 172.21.15.1:0/3787381569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 msgr2=0x55bff60c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 -- 172.21.15.1:0/3787381569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 msgr2=0x55bff60edb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60edb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 -- 172.21.15.1:0/3787381569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff60d19c0 con 0x55bff60d6780 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60edb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.123+0000 7fc60e59c640 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60ee050 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8031910 tx=0x7fc5f802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a48d8e1000b894e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:40.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc604ff9640 1 -- 172.21.15.1:0/3787381569 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f804e040 con 0x55bff60d6780 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc604ff9640 1 -- 172.21.15.1:0/3787381569 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f803d930 con 0x55bff60d6780 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc604ff9640 1 -- 172.21.15.1:0/3787381569 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8035070 con 0x55bff60d6780 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc6174be740 1 -- 172.21.15.1:0/3787381569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 msgr2=0x55bff60ee050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc6174be740 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60ee050 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8031910 tx=0x7fc5f802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc6174be740 1 -- 172.21.15.1:0/3787381569 shutdown_connections 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc6174be740 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60d6780 0x55bff60ee050 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc6174be740 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60c8b40 0x55bff60edb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc6174be740 1 --2- 172.21.15.1:0/3787381569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc6174be740 1 -- 172.21.15.1:0/3787381569 >> 172.21.15.1:0/3787381569 conn(0x55bff6090310 msgr2=0x55bff60c24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc6174be740 1 -- 172.21.15.1:0/3787381569 shutdown_connections 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.124+0000 7fc6174be740 1 -- 172.21.15.1:0/3787381569 wait complete. 2024-09-16T00:41:40.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc6174be740 1 -- start start 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc6174be740 1 Processor -- start 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc6174be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc6174be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c8b40 0x55bff60da400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc6174be740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60d6780 0x55bff60dab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc6174be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101ab220 con 0x55bff60c8b40 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc6174be740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101ac690 con 0x55bff60d6780 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc6174be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101ab8f0 con 0x55bff60c75d0 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc6174be740 10 client.0 _ll_register_callbacks cb 0x7fc5f8032e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c8b40 0x55bff60da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c8b40 0x55bff60da400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41322/0 (socket says 172.21.15.1:41322) 2024-09-16T00:41:40.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60ed9d640 1 -- 172.21.15.1:0/303441505 learned_addr learned my addr 172.21.15.1:0/303441505 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60e59c640 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60d6780 0x55bff60dab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60e59c640 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60ed9d640 1 -- 172.21.15.1:0/303441505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 msgr2=0x55bff60df0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60df0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60ed9d640 1 -- 172.21.15.1:0/303441505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60d6780 msgr2=0x55bff60dab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60d6780 0x55bff60dab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.125+0000 7fc60ed9d640 1 -- 172.21.15.1:0/303441505 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff614fae0 con 0x55bff60c8b40 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc60ed9d640 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c8b40 0x55bff60da400 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x55bff60d4ab0 tx=0x7fc600014c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a7f730380d5570 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc5f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:40.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc5f67fc640 1 -- 172.21.15.1:0/303441505 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600014e80 con 0x55bff60c8b40 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc5f67fc640 1 -- 172.21.15.1:0/303441505 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc600016a90 con 0x55bff60c8b40 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc6174be740 1 -- 172.21.15.1:0/303441505 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f803d930 con 0x55bff60c8b40 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc5f67fc640 1 -- 172.21.15.1:0/303441505 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc600016c90 con 0x55bff60c8b40 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc6174be740 10 client.18549 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc6174be740 1 -- 172.21.15.1:0/303441505 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bff6150c00 con 0x55bff60c8b40 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc5f67fc640 1 -- 172.21.15.1:0/303441505 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f803d930 con 0x55bff60c8b40 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc5f67fc640 1 -- 172.21.15.1:0/303441505 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc60001d2e0 con 0x55bff60c8b40 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc6174be740 10 client.18549 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.126+0000 7fc6174be740 10 client.18549 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc6174be740 20 client.18549 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:40.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc6174be740 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bff6151300 0x55bff6153720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc6174be740 4 client.18549 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc6174be740 1 -- 172.21.15.1:0/303441505 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bff6034d70 con 0x55bff6151300 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc60e59c640 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bff6151300 0x55bff6153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc5f67fc640 10 client.18549 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc60e59c640 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bff6151300 0x55bff6153720 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc8fd18a99d08862 server_cookie=3058934eccfcb6ba in_seq=0 out_seq=0 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc5f67fc640 1 -- 172.21.15.1:0/303441505 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bff6034d70 con 0x55bff6151300 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc5f67fc640 10 client.18549 handle_command_reply: tid=0 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.127+0000 7fc6174be740 1 client.18549 shutdown 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 -- 172.21.15.1:0/303441505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c8b40 msgr2=0x55bff60da400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c8b40 0x55bff60da400 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x55bff60d4ab0 tx=0x7fc600014c90 comp rx=0 tx=0).stop 2024-09-16T00:41:40.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 -- 172.21.15.1:0/303441505 shutdown_connections 2024-09-16T00:41:40.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55bff6151300 0x55bff6153720 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff60d6780 0x55bff60dab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bff60c8b40 0x55bff60da400 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 --2- 172.21.15.1:0/303441505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bff60c75d0 0x55bff60df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 -- 172.21.15.1:0/303441505 >> 172.21.15.1:0/303441505 conn(0x55bff6090310 msgr2=0x55bff60e16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:40.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc5f67fc640 0 client.18549 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:41:40.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 -- 172.21.15.1:0/303441505 shutdown_connections 2024-09-16T00:41:40.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 1 -- 172.21.15.1:0/303441505 wait complete. 2024-09-16T00:41:40.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6174be740 20 client.18549 trim_cache size 0 max 16384 2024-09-16T00:41:40.137 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:41:40.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 -- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc5fc0a32c0 msgr2=0x7fc5fc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:40.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc5fc0a32c0 0x7fc5fc0a5780 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc608005c40 tx=0x7fc608076040 comp rx=0 tx=0).stop 2024-09-16T00:41:40.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 -- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc610106f10 msgr2=0x7fc61019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:40.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc610106f10 0x7fc61019cd60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e940 tx=0x7fc60800ee00 comp rx=0 tx=0).stop 2024-09-16T00:41:40.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 -- 172.21.15.1:0/2445752897 shutdown_connections 2024-09-16T00:41:40.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc610106f10 0x7fc61019cd60 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc5fc0a32c0 0x7fc5fc0a5780 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc610102790 0x7fc6101989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 --2- 172.21.15.1:0/2445752897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610101de0 0x7fc610198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:40.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.128+0000 7fc6150e5640 1 -- 172.21.15.1:0/2445752897 >> 172.21.15.1:0/2445752897 conn(0x7fc6100fd8c0 msgr2=0x7fc6100fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:40.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.129+0000 7fc6150e5640 1 -- 172.21.15.1:0/2445752897 shutdown_connections 2024-09-16T00:41:40.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:40.129+0000 7fc6150e5640 1 -- 172.21.15.1:0/2445752897 wait complete. 2024-09-16T00:41:40.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:40 smithi001 ceph-mon[30934]: pgmap v748: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:40.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:40 smithi153 ceph-mon[35538]: pgmap v748: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:40.758 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:40 smithi019 ceph-mon[35186]: pgmap v748: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:40.989 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:41:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.742+0000 7fb624ddb640 1 -- 172.21.15.1:0/1592779811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 msgr2=0x7fb6201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.742+0000 7fb624ddb640 1 --2- 172.21.15.1:0/1592779811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 0x7fb6201067e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb608009900 tx=0x7fb60802a410 comp rx=0 tx=0).stop 2024-09-16T00:41:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.742+0000 7fb624ddb640 1 -- 172.21.15.1:0/1592779811 shutdown_connections 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.742+0000 7fb624ddb640 1 --2- 172.21.15.1:0/1592779811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb620106f10 0x7fb62010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.742+0000 7fb624ddb640 1 --2- 172.21.15.1:0/1592779811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 0x7fb6201067e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.742+0000 7fb624ddb640 1 --2- 172.21.15.1:0/1592779811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb620101de0 0x7fb6201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.742+0000 7fb624ddb640 1 -- 172.21.15.1:0/1592779811 >> 172.21.15.1:0/1592779811 conn(0x7fb6200fd8c0 msgr2=0x7fb6200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.742+0000 7fb624ddb640 1 -- 172.21.15.1:0/1592779811 shutdown_connections 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.742+0000 7fb624ddb640 1 -- 172.21.15.1:0/1592779811 wait complete. 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.743+0000 7fb624ddb640 1 Processor -- start 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.743+0000 7fb624ddb640 1 -- start start 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.743+0000 7fb624ddb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb620101de0 0x7fb6201984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.743+0000 7fb624ddb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 0x7fb6201989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.743+0000 7fb624ddb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb620106f10 0x7fb62019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.743+0000 7fb624ddb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb620111fe0 con 0x7fb620102790 2024-09-16T00:41:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.743+0000 7fb624ddb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb620111e60 con 0x7fb620101de0 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.743+0000 7fb624ddb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb620112160 con 0x7fb620106f10 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.743+0000 7fb617fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 0x7fb6201989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb617fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 0x7fb6201989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49488/0 (socket says 172.21.15.1:49488) 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb617fff640 1 -- 172.21.15.1:0/4219248288 learned_addr learned my addr 172.21.15.1:0/4219248288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb61e59c640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb620101de0 0x7fb6201984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb620106f10 0x7fb62019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb617fff640 1 -- 172.21.15.1:0/4219248288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb620106f10 msgr2=0x7fb62019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb617fff640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb620106f10 0x7fb62019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb617fff640 1 -- 172.21.15.1:0/4219248288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb620101de0 msgr2=0x7fb6201984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb617fff640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb620101de0 0x7fb6201984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb617fff640 1 -- 172.21.15.1:0/4219248288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62019d450 con 0x7fb620102790 2024-09-16T00:41:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb61e59c640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb620101de0 0x7fb6201984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb617fff640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 0x7fb6201989e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb608004770 tx=0x7fb608003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb6177fe640 1 -- 172.21.15.1:0/4219248288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60803f070 con 0x7fb620102790 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb624ddb640 1 -- 172.21.15.1:0/4219248288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62019d6e0 con 0x7fb620102790 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb6177fe640 1 -- 172.21.15.1:0/4219248288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb608031ca0 con 0x7fb620102790 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb6177fe640 1 -- 172.21.15.1:0/4219248288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608035030 con 0x7fb620102790 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.744+0000 7fb624ddb640 1 -- 172.21.15.1:0/4219248288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6201a5020 con 0x7fb620102790 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.747+0000 7fb6177fe640 1 -- 172.21.15.1:0/4219248288 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fb608042070 con 0x7fb620102790 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.747+0000 7fb6271b4740 1 Processor -- start 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb6271b4740 1 -- start start 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e4b40 0x55a87a4e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4f2780 0x55a87a4f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb6271b4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a87a4492f0 con 0x55a87a4e35d0 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb6271b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a87a3f0cf0 con 0x55a87a4e4b40 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb6271b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a87a4452b0 con 0x55a87a4f2780 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4e3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49496/0 (socket says 172.21.15.1:49496) 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61e59c640 1 -- 172.21.15.1:0/2034179801 learned_addr learned my addr 172.21.15.1:0/2034179801 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e4b40 0x55a87a4e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb6177fe640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb60c0a3240 0x7fb60c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb617fff640 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4f2780 0x55a87a4f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61e59c640 1 -- 172.21.15.1:0/2034179801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4f2780 msgr2=0x55a87a4f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61e59c640 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4f2780 0x55a87a4f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61e59c640 1 -- 172.21.15.1:0/2034179801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e4b40 msgr2=0x55a87a4e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61e59c640 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e4b40 0x55a87a4e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61e59c640 1 -- 172.21.15.1:0/2034179801 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87a3d53a0 con 0x55a87a4e35d0 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.749+0000 7fb617fff640 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4f2780 0x55a87a4f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.748+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e4b40 0x55a87a4e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.749+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb60c0a3240 0x7fb60c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.749+0000 7fb6177fe640 1 -- 172.21.15.1:0/4219248288 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb6080f0550 con 0x7fb620102790 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.749+0000 7fb61e59c640 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4e3a10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb610007850 tx=0x7fb610007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962e831bb68b9fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.749+0000 7fb614ff9640 1 -- 172.21.15.1:0/2034179801 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61000d950 con 0x55a87a4e35d0 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.749+0000 7fb614ff9640 1 -- 172.21.15.1:0/2034179801 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb610018070 con 0x55a87a4e35d0 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.749+0000 7fb614ff9640 1 -- 172.21.15.1:0/2034179801 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610013e60 con 0x55a87a4e35d0 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 -- 172.21.15.1:0/2034179801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 msgr2=0x55a87a4e3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4e3a10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb610007850 tx=0x7fb610007d10 comp rx=0 tx=0).stop 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb60c0a3240 0x7fb60c0a5700 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb6180098b0 tx=0x7fb618004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 -- 172.21.15.1:0/2034179801 shutdown_connections 2024-09-16T00:41:41.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4f2780 0x55a87a4f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4e3a10 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 --2- 172.21.15.1:0/2034179801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e4b40 0x55a87a4e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 -- 172.21.15.1:0/2034179801 >> 172.21.15.1:0/2034179801 conn(0x55a87a4ac310 msgr2=0x55a87a4dfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 -- 172.21.15.1:0/2034179801 shutdown_connections 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 -- 172.21.15.1:0/2034179801 wait complete. 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 -- start start 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.750+0000 7fb6271b4740 1 Processor -- start 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 0x55a87a4f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4ef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb6271b4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a87a4f5480 con 0x55a87a4f2780 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb6271b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a87a404060 con 0x55a87a4e35d0 2024-09-16T00:41:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb6271b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a87a4f5600 con 0x55a87a4e4b40 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb6271b4740 10 client.0 _ll_register_callbacks cb 0x55a87a1578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4ef900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49512/0 (socket says 172.21.15.1:49512) 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61ed9d640 1 -- 172.21.15.1:0/3539045150 learned_addr learned my addr 172.21.15.1:0/3539045150 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61e59c640 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 0x55a87a4f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb617fff640 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61ed9d640 1 -- 172.21.15.1:0/3539045150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 msgr2=0x55a87a4ef3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4ef3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61ed9d640 1 -- 172.21.15.1:0/3539045150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 msgr2=0x55a87a4f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 0x55a87a4f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61ed9d640 1 -- 172.21.15.1:0/3539045150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87a1ba3c0 con 0x55a87a4f2780 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4ef900 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb618013960 tx=0x7fb618004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51a108e5004cdb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb605ffb640 1 -- 172.21.15.1:0/3539045150 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61801ed70 con 0x55a87a4f2780 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb605ffb640 1 -- 172.21.15.1:0/3539045150 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb618028470 con 0x55a87a4f2780 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.751+0000 7fb6271b4740 1 -- 172.21.15.1:0/3539045150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb610018070 con 0x55a87a4f2780 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb605ffb640 1 -- 172.21.15.1:0/3539045150 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb618028670 con 0x55a87a4f2780 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb6271b4740 10 client.18561 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb6271b4740 1 -- 172.21.15.1:0/3539045150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a87a500e00 con 0x55a87a4f2780 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb605ffb640 1 -- 172.21.15.1:0/3539045150 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb610018070 con 0x55a87a4f2780 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb605ffb640 1 -- 172.21.15.1:0/3539045150 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb61800f040 con 0x55a87a4f2780 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb6271b4740 10 client.18561 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:41.798 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:41:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb6271b4740 10 client.18561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:41:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb6271b4740 20 client.18561 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb6271b4740 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a87a5022c0 0x55a87a504710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb6271b4740 4 client.18561 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb6271b4740 1 -- 172.21.15.1:0/3539045150 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a87a456860 con 0x55a87a5022c0 2024-09-16T00:41:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.752+0000 7fb61e59c640 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a87a5022c0 0x55a87a504710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.753+0000 7fb61e59c640 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a87a5022c0 0x55a87a504710 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=159e7b2a58d6b19d server_cookie=48460a41bb7fcc2a in_seq=0 out_seq=0 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.753+0000 7fb605ffb640 10 client.18561 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.753+0000 7fb605ffb640 1 -- 172.21.15.1:0/3539045150 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a87a456860 con 0x55a87a5022c0 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb605ffb640 10 client.18561 handle_command_reply: tid=0 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 client.18561 shutdown 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 -- 172.21.15.1:0/3539045150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 msgr2=0x55a87a4ef900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4ef900 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb618013960 tx=0x7fb618004ab0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 -- 172.21.15.1:0/3539045150 shutdown_connections 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a87a5022c0 0x55a87a504710 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4ef900 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 --2- 172.21.15.1:0/3539045150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 0x55a87a4f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 -- 172.21.15.1:0/3539045150 >> 172.21.15.1:0/3539045150 conn(0x55a87a4ac310 msgr2=0x55a87a4e5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb605ffb640 0 client.18561 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 -- 172.21.15.1:0/3539045150 shutdown_connections 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 1 -- 172.21.15.1:0/3539045150 wait complete. 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.754+0000 7fb6271b4740 20 client.18561 trim_cache size 0 max 16384 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb6271b4740 1 Processor -- start 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb6271b4740 1 -- start start 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4fe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4eec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4f2780 0x55a87a4df700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb6271b4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201af590 con 0x55a87a4e35d0 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb6271b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201ace60 con 0x55a87a4f2780 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb6271b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201aacb0 con 0x55a87a4e4b40 2024-09-16T00:41:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4fe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4fe720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49516/0 (socket says 172.21.15.1:49516) 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb61e59c640 1 -- 172.21.15.1:0/1158100238 learned_addr learned my addr 172.21.15.1:0/1158100238 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4f2780 0x55a87a4df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb617fff640 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb61e59c640 1 -- 172.21.15.1:0/1158100238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 msgr2=0x55a87a4eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb61e59c640 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb61e59c640 1 -- 172.21.15.1:0/1158100238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4f2780 msgr2=0x55a87a4df700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.767+0000 7fb61e59c640 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4f2780 0x55a87a4df700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb61e59c640 1 -- 172.21.15.1:0/1158100238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87a4ed8f0 con 0x55a87a4e35d0 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb617fff640 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb61e59c640 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4fe720 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb610004810 tx=0x7fb61000f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7470aae750079d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb607fff640 1 -- 172.21.15.1:0/1158100238 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61002a070 con 0x55a87a4e35d0 2024-09-16T00:41:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb607fff640 1 -- 172.21.15.1:0/1158100238 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb610018040 con 0x55a87a4e35d0 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb607fff640 1 -- 172.21.15.1:0/1158100238 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610026610 con 0x55a87a4e35d0 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb6271b4740 1 -- 172.21.15.1:0/1158100238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 msgr2=0x55a87a4fe720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb6271b4740 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4fe720 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb610004810 tx=0x7fb61000f040 comp rx=0 tx=0).stop 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb6271b4740 1 -- 172.21.15.1:0/1158100238 shutdown_connections 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb6271b4740 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4f2780 0x55a87a4df700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb6271b4740 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb6271b4740 1 --2- 172.21.15.1:0/1158100238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4e35d0 0x55a87a4fe720 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb6271b4740 1 -- 172.21.15.1:0/1158100238 >> 172.21.15.1:0/1158100238 conn(0x55a87a4ac310 msgr2=0x55a87a4edc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.768+0000 7fb6271b4740 1 -- 172.21.15.1:0/1158100238 shutdown_connections 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 1 -- 172.21.15.1:0/1158100238 wait complete. 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 1 -- start start 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 1 Processor -- start 2024-09-16T00:41:41.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 0x55a87a4fa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201ab320 con 0x55a87a4f2780 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201ac790 con 0x55a87a4e35d0 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201ab9f0 con 0x55a87a4e4b40 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb6271b4740 10 client.0 _ll_register_callbacks cb 0x7fb618009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4eb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49524/0 (socket says 172.21.15.1:49524) 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb61ed9d640 1 -- 172.21.15.1:0/4058014287 learned_addr learned my addr 172.21.15.1:0/4058014287 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb61e59c640 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 0x55a87a4fa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.769+0000 7fb617fff640 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb61ed9d640 1 -- 172.21.15.1:0/4058014287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 msgr2=0x55a87a4fadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4fadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb61ed9d640 1 -- 172.21.15.1:0/4058014287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 msgr2=0x55a87a4fa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 0x55a87a4fa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb61ed9d640 1 -- 172.21.15.1:0/4058014287 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87a4ebf10 con 0x55a87a4f2780 2024-09-16T00:41:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb61e59c640 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 0x55a87a4fa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb61ed9d640 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4eb9d0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb61801c660 tx=0x7fb61801e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91146290de571146 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb605ffb640 1 -- 172.21.15.1:0/4058014287 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61801c440 con 0x55a87a4f2780 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb605ffb640 1 -- 172.21.15.1:0/4058014287 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb61801cae0 con 0x55a87a4f2780 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb6271b4740 1 -- 172.21.15.1:0/4058014287 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb610018040 con 0x55a87a4f2780 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb605ffb640 1 -- 172.21.15.1:0/4058014287 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb61803d050 con 0x55a87a4f2780 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb6271b4740 10 client.18567 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb6271b4740 1 -- 172.21.15.1:0/4058014287 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a87a56c720 con 0x55a87a4f2780 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb605ffb640 1 -- 172.21.15.1:0/4058014287 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb610018040 con 0x55a87a4f2780 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.770+0000 7fb605ffb640 1 -- 172.21.15.1:0/4058014287 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb61801f6f0 con 0x55a87a4f2780 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb6271b4740 10 client.18567 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:41.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb6271b4740 10 client.18567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb6271b4740 20 client.18567 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb6271b4740 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a87a56ce70 0x55a87a56f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb6271b4740 4 client.18567 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb6271b4740 1 -- 172.21.15.1:0/4058014287 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a87a450d70 con 0x55a87a56ce70 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb61e59c640 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a87a56ce70 0x55a87a56f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb605ffb640 10 client.18567 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb61e59c640 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a87a56ce70 0x55a87a56f290 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95ea57c640aa5833 server_cookie=50543ea5d46a66fb in_seq=0 out_seq=0 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb605ffb640 1 -- 172.21.15.1:0/4058014287 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a87a450d70 con 0x55a87a56ce70 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb605ffb640 10 client.18567 handle_command_reply: tid=0 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.771+0000 7fb6271b4740 1 client.18567 shutdown 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 -- 172.21.15.1:0/4058014287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 msgr2=0x55a87a4eb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4eb9d0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb61801c660 tx=0x7fb61801e7b0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 -- 172.21.15.1:0/4058014287 shutdown_connections 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a87a56ce70 0x55a87a56f290 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a87a4f2780 0x55a87a4eb9d0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a87a4e4b40 0x55a87a4fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb605ffb640 0 client.18567 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 --2- 172.21.15.1:0/4058014287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a87a4e35d0 0x55a87a4fa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 -- 172.21.15.1:0/4058014287 >> 172.21.15.1:0/4058014287 conn(0x55a87a4ac310 msgr2=0x55a87a4edb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 -- 172.21.15.1:0/4058014287 shutdown_connections 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 1 -- 172.21.15.1:0/4058014287 wait complete. 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb6271b4740 20 client.18567 trim_cache size 0 max 16384 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 -- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb60c0a3240 msgr2=0x7fb60c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb60c0a3240 0x7fb60c0a5700 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb6180098b0 tx=0x7fb618004a40 comp rx=0 tx=0).stop 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 -- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 msgr2=0x7fb6201989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 0x7fb6201989e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb608004770 tx=0x7fb608003c10 comp rx=0 tx=0).stop 2024-09-16T00:41:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 -- 172.21.15.1:0/4219248288 shutdown_connections 2024-09-16T00:41:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb60c0a3240 0x7fb60c0a5700 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb620106f10 0x7fb62019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb620102790 0x7fb6201989e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 --2- 172.21.15.1:0/4219248288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb620101de0 0x7fb6201984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 -- 172.21.15.1:0/4219248288 >> 172.21.15.1:0/4219248288 conn(0x7fb6200fd8c0 msgr2=0x7fb6200fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 -- 172.21.15.1:0/4219248288 shutdown_connections 2024-09-16T00:41:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:41.772+0000 7fb624ddb640 1 -- 172.21.15.1:0/4219248288 wait complete. 2024-09-16T00:41:42.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:42 smithi001 ceph-mon[30934]: pgmap v749: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:42.695 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:41:42.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:42 smithi153 ceph-mon[35538]: pgmap v749: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:42.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:42 smithi019 ceph-mon[35186]: pgmap v749: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.488+0000 7f1f87052640 1 -- 172.21.15.1:0/423317612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 msgr2=0x7f1f80101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.488+0000 7f1f87052640 1 --2- 172.21.15.1:0/423317612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 0x7f1f80101200 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009900 tx=0x7f1f7402a480 comp rx=0 tx=0).stop 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.488+0000 7f1f87052640 1 -- 172.21.15.1:0/423317612 shutdown_connections 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.488+0000 7f1f87052640 1 --2- 172.21.15.1:0/423317612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f80102ef0 0x7f1f8010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.488+0000 7f1f87052640 1 --2- 172.21.15.1:0/423317612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f80102570 0x7f1f801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.488+0000 7f1f87052640 1 --2- 172.21.15.1:0/423317612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 0x7f1f80101200 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.488+0000 7f1f87052640 1 -- 172.21.15.1:0/423317612 >> 172.21.15.1:0/423317612 conn(0x7f1f800fc7d0 msgr2=0x7f1f800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.488+0000 7f1f87052640 1 -- 172.21.15.1:0/423317612 shutdown_connections 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.488+0000 7f1f87052640 1 -- 172.21.15.1:0/423317612 wait complete. 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f87052640 1 Processor -- start 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f87052640 1 -- start start 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f87052640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 0x7f1f8019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f87052640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f80102570 0x7f1f8019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f87052640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f80102ef0 0x7f1f8019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f87052640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80110f80 con 0x7f1f80100e20 2024-09-16T00:41:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f87052640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80110e00 con 0x7f1f80102570 2024-09-16T00:41:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f87052640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80111100 con 0x7f1f80102ef0 2024-09-16T00:41:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f84dee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 0x7f1f8019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f84dee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 0x7f1f8019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49536/0 (socket says 172.21.15.1:49536) 2024-09-16T00:41:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f84dee640 1 -- 172.21.15.1:0/2254941252 learned_addr learned my addr 172.21.15.1:0/2254941252 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f80102570 0x7f1f8019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f855ef640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f80102ef0 0x7f1f8019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f84dee640 1 -- 172.21.15.1:0/2254941252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f80102ef0 msgr2=0x7f1f8019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f84dee640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f80102ef0 0x7f1f8019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f84dee640 1 -- 172.21.15.1:0/2254941252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f80102570 msgr2=0x7f1f8019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f84dee640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f80102570 0x7f1f8019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f84dee640 1 -- 172.21.15.1:0/2254941252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8019f6a0 con 0x7f1f80100e20 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.489+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f80102570 0x7f1f8019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.490+0000 7f1f84dee640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 0x7f1f8019a6f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1f74006010 tx=0x7f1f74003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.490+0000 7f1f7dffb640 1 -- 172.21.15.1:0/2254941252 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7403f070 con 0x7f1f80100e20 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.490+0000 7f1f87052640 1 -- 172.21.15.1:0/2254941252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8019f930 con 0x7f1f80100e20 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.490+0000 7f1f7dffb640 1 -- 172.21.15.1:0/2254941252 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f74031c30 con 0x7f1f80100e20 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.490+0000 7f1f7dffb640 1 -- 172.21.15.1:0/2254941252 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74035030 con 0x7f1f80100e20 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.490+0000 7f1f87052640 1 -- 172.21.15.1:0/2254941252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f801a7270 con 0x7f1f80100e20 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.492+0000 7f1f7dffb640 1 -- 172.21.15.1:0/2254941252 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f1f74042070 con 0x7f1f80100e20 2024-09-16T00:41:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f8942b740 1 Processor -- start 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f8942b740 1 -- start start 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e888954b40 0x55e888953090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8889535d0 0x55e888953a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888960760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f8942b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e8888b92f0 con 0x55e88895e370 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f8942b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e888860cf0 con 0x55e8889535d0 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f8942b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8888b52b0 con 0x55e888954b40 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888960760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888960760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49548/0 (socket says 172.21.15.1:49548) 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f7ffff640 1 -- 172.21.15.1:0/2046076369 learned_addr learned my addr 172.21.15.1:0/2046076369 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f84dee640 1 --2- 172.21.15.1:0/2046076369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8889535d0 0x55e888953a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f855ef640 1 --2- 172.21.15.1:0/2046076369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e888954b40 0x55e888953090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f7dffb640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1f700a3240 0x7f1f700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f7ffff640 1 -- 172.21.15.1:0/2046076369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e888954b40 msgr2=0x55e888953090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/2046076369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e888954b40 0x55e888953090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f7ffff640 1 -- 172.21.15.1:0/2046076369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8889535d0 msgr2=0x55e888953a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/2046076369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8889535d0 0x55e888953a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.493+0000 7f1f7ffff640 1 -- 172.21.15.1:0/2046076369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8888453a0 con 0x55e88895e370 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.494+0000 7f1f855ef640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1f700a3240 0x7f1f700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.494+0000 7f1f7dffb640 1 -- 172.21.15.1:0/2254941252 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1f740f0520 con 0x7f1f80100e20 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.494+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/2046076369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888960760 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009c00 tx=0x7f1f6c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f01db4389e03e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.494+0000 7f1f6affd640 1 -- 172.21.15.1:0/2046076369 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c004030 con 0x55e88895e370 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.494+0000 7f1f6affd640 1 -- 172.21.15.1:0/2046076369 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6c0041d0 con 0x55e88895e370 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.494+0000 7f1f6affd640 1 -- 172.21.15.1:0/2046076369 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c012690 con 0x55e88895e370 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 -- 172.21.15.1:0/2046076369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 msgr2=0x55e888960760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:43.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f855ef640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1f700a3240 0x7f1f700a5700 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800bd70 tx=0x7f1f78004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 --2- 172.21.15.1:0/2046076369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888960760 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009c00 tx=0x7f1f6c010740 comp rx=0 tx=0).stop 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 -- 172.21.15.1:0/2046076369 shutdown_connections 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 --2- 172.21.15.1:0/2046076369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888960760 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 --2- 172.21.15.1:0/2046076369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8889535d0 0x55e888953a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 --2- 172.21.15.1:0/2046076369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e888954b40 0x55e888953090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 -- 172.21.15.1:0/2046076369 >> 172.21.15.1:0/2046076369 conn(0x55e88891c310 msgr2=0x55e88894fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 -- 172.21.15.1:0/2046076369 shutdown_connections 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 -- 172.21.15.1:0/2046076369 wait complete. 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 -- start start 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.495+0000 7f1f8942b740 1 Processor -- start 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 0x55e88895dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e88895afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e88895b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f8942b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e888961070 con 0x55e88895e370 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f8942b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e888874060 con 0x55e888954b40 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f8942b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8889611f0 con 0x55e8889535d0 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f8942b740 10 client.0 _ll_register_callbacks cb 0x55e8885c78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f855ef640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e88895b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f855ef640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e88895b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49564/0 (socket says 172.21.15.1:49564) 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f855ef640 1 -- 172.21.15.1:0/1026143868 learned_addr learned my addr 172.21.15.1:0/1026143868 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e88895afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f84dee640 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 0x55e88895dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f855ef640 1 -- 172.21.15.1:0/1026143868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 msgr2=0x55e88895dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f855ef640 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 0x55e88895dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f855ef640 1 -- 172.21.15.1:0/1026143868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 msgr2=0x55e88895afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:43.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f855ef640 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e88895afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f855ef640 1 -- 172.21.15.1:0/1026143868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88862a3c0 con 0x55e88895e370 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f855ef640 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e88895b5d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f1f78011e10 tx=0x7f1f78004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fd773ed94c1ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.496+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1026143868 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7801c430 con 0x55e88895e370 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1026143868 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1f7801c5d0 con 0x55e88895e370 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f8942b740 1 -- 172.21.15.1:0/1026143868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f6c0041d0 con 0x55e88895e370 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1026143868 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f7802e050 con 0x55e88895e370 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f8942b740 10 client.18579 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f8942b740 1 -- 172.21.15.1:0/1026143868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e88896cb80 con 0x55e88895e370 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1026143868 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6c0041d0 con 0x55e88895e370 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1026143868 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1f7801d960 con 0x55e88895e370 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f8942b740 10 client.18579 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f8942b740 10 client.18579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:43.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f8942b740 20 client.18579 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e88896e040 0x55e8889704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f8942b740 4 client.18579 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f8942b740 1 -- 172.21.15.1:0/1026143868 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8888c6860 con 0x55e88896e040 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.497+0000 7f1f84dee640 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e88896e040 0x55e8889704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.498+0000 7f1f5ffff640 10 client.18579 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.498+0000 7f1f84dee640 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e88896e040 0x55e8889704e0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6d7126c48b1e0a server_cookie=495552c75c0755fd in_seq=0 out_seq=0 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1026143868 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8888c6860 con 0x55e88896e040 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f5ffff640 10 client.18579 handle_command_reply: tid=0 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 client.18579 shutdown 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 -- 172.21.15.1:0/1026143868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 msgr2=0x55e88895b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e88895b5d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f1f78011e10 tx=0x7f1f78004ab0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 -- 172.21.15.1:0/1026143868 shutdown_connections 2024-09-16T00:41:43.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e88896e040 0x55e8889704e0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e88895b5d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e88895afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1026143868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 0x55e88895dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 -- 172.21.15.1:0/1026143868 >> 172.21.15.1:0/1026143868 conn(0x55e88891c310 msgr2=0x55e888955960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f5ffff640 0 client.18579 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 -- 172.21.15.1:0/1026143868 shutdown_connections 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 1 -- 172.21.15.1:0/1026143868 wait complete. 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.499+0000 7f1f8942b740 20 client.18579 trim_cache size 0 max 16384 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.512+0000 7f1f8942b740 1 Processor -- start 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.512+0000 7f1f8942b740 1 -- start start 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 0x55e88896a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e8889598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888959df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f8942b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801b3880 con 0x55e88895e370 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f8942b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801b1150 con 0x55e888954b40 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f8942b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801aefa0 con 0x55e8889535d0 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f855ef640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888959df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f855ef640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888959df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49568/0 (socket says 172.21.15.1:49568) 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e8889598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f855ef640 1 -- 172.21.15.1:0/3718972843 learned_addr learned my addr 172.21.15.1:0/3718972843 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f84dee640 1 --2- 172.21.15.1:0/3718972843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 0x55e88896a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f855ef640 1 -- 172.21.15.1:0/3718972843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 msgr2=0x55e88896a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f855ef640 1 --2- 172.21.15.1:0/3718972843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 0x55e88896a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f855ef640 1 -- 172.21.15.1:0/3718972843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 msgr2=0x55e8889598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f855ef640 1 --2- 172.21.15.1:0/3718972843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e8889598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f855ef640 1 -- 172.21.15.1:0/3718972843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e888959530 con 0x55e88895e370 2024-09-16T00:41:43.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f84dee640 1 --2- 172.21.15.1:0/3718972843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 0x55e88896a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f855ef640 1 --2- 172.21.15.1:0/3718972843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888959df0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f1f78010cd0 tx=0x7f1f7800b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4aaa3d66b2d9a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.513+0000 7f1f6a7fc640 1 -- 172.21.15.1:0/3718972843 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800ee90 con 0x55e88895e370 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f6a7fc640 1 -- 172.21.15.1:0/3718972843 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f78036030 con 0x55e88895e370 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f6a7fc640 1 -- 172.21.15.1:0/3718972843 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78026a90 con 0x55e88895e370 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f8942b740 1 -- 172.21.15.1:0/3718972843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 msgr2=0x55e888959df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f8942b740 1 --2- 172.21.15.1:0/3718972843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888959df0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f1f78010cd0 tx=0x7f1f7800b470 comp rx=0 tx=0).stop 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f8942b740 1 -- 172.21.15.1:0/3718972843 shutdown_connections 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f8942b740 1 --2- 172.21.15.1:0/3718972843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e88895e370 0x55e888959df0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f8942b740 1 --2- 172.21.15.1:0/3718972843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e8889598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f8942b740 1 --2- 172.21.15.1:0/3718972843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8889535d0 0x55e88896a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f8942b740 1 -- 172.21.15.1:0/3718972843 >> 172.21.15.1:0/3718972843 conn(0x55e88891c310 msgr2=0x55e88895afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f8942b740 1 -- 172.21.15.1:0/3718972843 shutdown_connections 2024-09-16T00:41:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.514+0000 7f1f8942b740 1 -- 172.21.15.1:0/3718972843 wait complete. 2024-09-16T00:41:43.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f8942b740 1 -- start start 2024-09-16T00:41:43.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f8942b740 1 Processor -- start 2024-09-16T00:41:43.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8889535d0 0x55e888974f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e888975450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f8942b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e88895e370 0x55e888965480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f8942b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801af610 con 0x55e8889535d0 2024-09-16T00:41:43.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f8942b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801b0a80 con 0x55e888954b40 2024-09-16T00:41:43.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f8942b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801afce0 con 0x55e88895e370 2024-09-16T00:41:43.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f8942b740 10 client.0 _ll_register_callbacks cb 0x7f1f7800b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f84dee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8889535d0 0x55e888974f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f84dee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8889535d0 0x55e888974f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49584/0 (socket says 172.21.15.1:49584) 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.515+0000 7f1f84dee640 1 -- 172.21.15.1:0/1779756430 learned_addr learned my addr 172.21.15.1:0/1779756430 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f855ef640 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e88895e370 0x55e888965480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e888975450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f84dee640 1 -- 172.21.15.1:0/1779756430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e88895e370 msgr2=0x55e888965480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f84dee640 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e88895e370 0x55e888965480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f84dee640 1 -- 172.21.15.1:0/1779756430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 msgr2=0x55e888975450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f84dee640 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e888975450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f84dee640 1 -- 172.21.15.1:0/1779756430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8889659c0 con 0x55e8889535d0 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e888975450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f855ef640 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e88895e370 0x55e888965480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:43.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f84dee640 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8889535d0 0x55e888974f10 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f1f740c4e20 tx=0x7f1f7402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5220f5d09b521a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1779756430 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74049070 con 0x55e8889535d0 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1779756430 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1f740edcd0 con 0x55e8889535d0 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f8942b740 1 -- 172.21.15.1:0/1779756430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f78036030 con 0x55e8889535d0 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1779756430 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f740bcde0 con 0x55e8889535d0 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f8942b740 10 client.18585 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f8942b740 1 -- 172.21.15.1:0/1779756430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e888966c90 con 0x55e8889535d0 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.516+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1779756430 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f78036030 con 0x55e8889535d0 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1779756430 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1f740f0180 con 0x55e8889535d0 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f8942b740 10 client.18585 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f8942b740 10 client.18585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f8942b740 20 client.18585 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:43.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e8889d8dd0 0x55e8889db1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f8942b740 4 client.18585 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f8942b740 1 -- 172.21.15.1:0/1779756430 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8888c0d70 con 0x55e8889d8dd0 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e8889d8dd0 0x55e8889db1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f5ffff640 10 client.18585 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f7ffff640 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e8889d8dd0 0x55e8889db1c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6d0fa9308d650e server_cookie=41256498774a255b in_seq=0 out_seq=0 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f5ffff640 1 -- 172.21.15.1:0/1779756430 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8888c0d70 con 0x55e8889d8dd0 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f5ffff640 10 client.18585 handle_command_reply: tid=0 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.517+0000 7f1f8942b740 1 client.18585 shutdown 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 -- 172.21.15.1:0/1779756430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8889535d0 msgr2=0x55e888974f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8889535d0 0x55e888974f10 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f1f740c4e20 tx=0x7f1f7402a480 comp rx=0 tx=0).stop 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 -- 172.21.15.1:0/1779756430 shutdown_connections 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e8889d8dd0 0x55e8889db1c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e88895e370 0x55e888965480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e888954b40 0x55e888975450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 --2- 172.21.15.1:0/1779756430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8889535d0 0x55e888974f10 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 -- 172.21.15.1:0/1779756430 >> 172.21.15.1:0/1779756430 conn(0x55e88891c310 msgr2=0x55e888959870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f5ffff640 0 client.18585 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 -- 172.21.15.1:0/1779756430 shutdown_connections 2024-09-16T00:41:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 1 -- 172.21.15.1:0/1779756430 wait complete. 2024-09-16T00:41:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.518+0000 7f1f8942b740 20 client.18585 trim_cache size 0 max 16384 2024-09-16T00:41:43.534 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 -- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1f700a3240 msgr2=0x7f1f700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1f700a3240 0x7f1f700a5700 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800bd70 tx=0x7f1f78004a40 comp rx=0 tx=0).stop 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 -- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 msgr2=0x7f1f8019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 0x7f1f8019a6f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1f74006010 tx=0x7f1f74003c10 comp rx=0 tx=0).stop 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 -- 172.21.15.1:0/2254941252 shutdown_connections 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1f700a3240 0x7f1f700a5700 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f80102ef0 0x7f1f8019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f80102570 0x7f1f8019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 --2- 172.21.15.1:0/2254941252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f80100e20 0x7f1f8019a6f0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 -- 172.21.15.1:0/2254941252 >> 172.21.15.1:0/2254941252 conn(0x7f1f800fc7d0 msgr2=0x7f1f8006a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 -- 172.21.15.1:0/2254941252 shutdown_connections 2024-09-16T00:41:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:43.519+0000 7f1f87052640 1 -- 172.21.15.1:0/2254941252 wait complete. 2024-09-16T00:41:44.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:44 smithi001 ceph-mon[30934]: pgmap v750: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:44.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:44 smithi153 ceph-mon[35538]: pgmap v750: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:44.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:44 smithi019 ceph-mon[35186]: pgmap v750: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:45.374 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:41:45.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:41:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:46.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.109+0000 7fa017f1d640 1 -- 172.21.15.1:0/96376552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102570 msgr2=0x7fa0101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.109+0000 7fa017f1d640 1 --2- 172.21.15.1:0/96376552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102570 0x7fa0101029b0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa000009900 tx=0x7fa00002a480 comp rx=0 tx=0).stop 2024-09-16T00:41:46.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.109+0000 7fa017f1d640 1 -- 172.21.15.1:0/96376552 shutdown_connections 2024-09-16T00:41:46.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.109+0000 7fa017f1d640 1 --2- 172.21.15.1:0/96376552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa010102ef0 0x7fa01010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.109+0000 7fa017f1d640 1 --2- 172.21.15.1:0/96376552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102570 0x7fa0101029b0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.109+0000 7fa017f1d640 1 --2- 172.21.15.1:0/96376552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010100e20 0x7fa010101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.109+0000 7fa017f1d640 1 -- 172.21.15.1:0/96376552 >> 172.21.15.1:0/96376552 conn(0x7fa0100fc7d0 msgr2=0x7fa0100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.109+0000 7fa017f1d640 1 -- 172.21.15.1:0/96376552 shutdown_connections 2024-09-16T00:41:46.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.109+0000 7fa017f1d640 1 -- 172.21.15.1:0/96376552 wait complete. 2024-09-16T00:41:46.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa017f1d640 1 Processor -- start 2024-09-16T00:41:46.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa017f1d640 1 -- start start 2024-09-16T00:41:46.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa017f1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010100e20 0x7fa0100fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa017f1d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa010102570 0x7fa0100fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa017f1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102ef0 0x7fa0100fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa017f1d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa010110f80 con 0x7fa010102ef0 2024-09-16T00:41:46.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa017f1d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa010110e00 con 0x7fa010100e20 2024-09-16T00:41:46.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa017f1d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa010111100 con 0x7fa010102570 2024-09-16T00:41:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa0164ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102ef0 0x7fa0100fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa0164ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102ef0 0x7fa0100fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49604/0 (socket says 172.21.15.1:49604) 2024-09-16T00:41:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.110+0000 7fa0164ba640 1 -- 172.21.15.1:0/2499127328 learned_addr learned my addr 172.21.15.1:0/2499127328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa015cb9640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010100e20 0x7fa0100fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa0164ba640 1 -- 172.21.15.1:0/2499127328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa010102570 msgr2=0x7fa0100fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa0164ba640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa010102570 0x7fa0100fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa0164ba640 1 -- 172.21.15.1:0/2499127328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010100e20 msgr2=0x7fa0100fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa0164ba640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010100e20 0x7fa0100fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa0164ba640 1 -- 172.21.15.1:0/2499127328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0101a69b0 con 0x7fa010102ef0 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa015cb9640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010100e20 0x7fa0100fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa0164ba640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102ef0 0x7fa0100fd700 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fa00800cce0 tx=0x7fa00800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa006ffd640 1 -- 172.21.15.1:0/2499127328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00800f450 con 0x7fa010102ef0 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa017f1d640 1 -- 172.21.15.1:0/2499127328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0101a6ca0 con 0x7fa010102ef0 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa006ffd640 1 -- 172.21.15.1:0/2499127328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa00800fad0 con 0x7fa010102ef0 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa006ffd640 1 -- 172.21.15.1:0/2499127328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008018c10 con 0x7fa010102ef0 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.111+0000 7fa017f1d640 1 -- 172.21.15.1:0/2499127328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0101a7190 con 0x7fa010102ef0 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.113+0000 7fa006ffd640 1 -- 172.21.15.1:0/2499127328 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa00800f5f0 con 0x7fa010102ef0 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa01a2f6740 1 Processor -- start 2024-09-16T00:41:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa01a2f6740 1 -- start start 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x5637984fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 0x563798506800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x563798509130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563798461280 con 0x5637984fbb60 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563798408b10 con 0x563798506d40 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56379845d240 con 0x5637984fc480 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa0154b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x5637984fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa0154b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x5637984fbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49614/0 (socket says 172.21.15.1:49614) 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa0154b8640 1 -- 172.21.15.1:0/2922405483 learned_addr learned my addr 172.21.15.1:0/2922405483 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.114+0000 7fa006ffd640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa015cb9640 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 0x563798506800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa0154b8640 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x563798509130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa0164ba640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa0154b8640 1 -- 172.21.15.1:0/2922405483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 msgr2=0x563798506800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa0154b8640 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 0x563798506800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa0154b8640 1 -- 172.21.15.1:0/2922405483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 msgr2=0x563798509130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa0154b8640 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x563798509130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa0154b8640 1 -- 172.21.15.1:0/2922405483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637984151f0 con 0x5637984fbb60 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa015cb9640 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 0x563798506800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa006ffd640 1 -- 172.21.15.1:0/2499127328 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa0080d0460 con 0x7fa010102ef0 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa0154b8640 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x5637984fbf40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa00002a9a0 tx=0x7fa0000056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf7f0fa69ff3bac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7f9ffbfff640 1 -- 172.21.15.1:0/2922405483 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000035040 con 0x5637984fbb60 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7fa0164ba640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa0080cdbb0 tx=0x7fa00806f500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7f9ffbfff640 1 -- 172.21.15.1:0/2922405483 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa000044070 con 0x5637984fbb60 2024-09-16T00:41:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.115+0000 7f9ffbfff640 1 -- 172.21.15.1:0/2922405483 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00003da80 con 0x5637984fbb60 2024-09-16T00:41:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2922405483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 msgr2=0x5637984fbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x5637984fbf40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa00002a9a0 tx=0x7fa0000056d0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2922405483 shutdown_connections 2024-09-16T00:41:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x563798509130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 0x563798506800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2922405483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x5637984fbf40 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2922405483 >> 172.21.15.1:0/2922405483 conn(0x5637984f80f0 msgr2=0x5637984f84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2922405483 shutdown_connections 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2922405483 wait complete. 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 -- start start 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.116+0000 7fa01a2f6740 1 Processor -- start 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fbb60 0x563798503be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fc480 0x563798504120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x563798504660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563798509910 con 0x5637984fc480 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56379841bf10 con 0x563798506d40 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563798509a90 con 0x5637984fbb60 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa01a2f6740 10 client.0 _ll_register_callbacks cb 0x5637983e4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa015cb9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fc480 0x563798504120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa015cb9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fc480 0x563798504120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49620/0 (socket says 172.21.15.1:49620) 2024-09-16T00:41:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa015cb9640 1 -- 172.21.15.1:0/2647131946 learned_addr learned my addr 172.21.15.1:0/2647131946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa0154b8640 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fbb60 0x563798503be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa0154b8640 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x563798504660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa015cb9640 1 -- 172.21.15.1:0/2647131946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fbb60 msgr2=0x563798503be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa015cb9640 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fbb60 0x563798503be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa015cb9640 1 -- 172.21.15.1:0/2647131946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 msgr2=0x563798504660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa015cb9640 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x563798504660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa015cb9640 1 -- 172.21.15.1:0/2647131946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637981d26a0 con 0x5637984fc480 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.117+0000 7fa0154b8640 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x563798504660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7fa015cb9640 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fc480 0x563798504120 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00e750 tx=0x7fa00c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f50627dafa56162 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7f9ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7f9ff97fa640 1 -- 172.21.15.1:0/2647131946 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c01c060 con 0x5637984fc480 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7f9ff97fa640 1 -- 172.21.15.1:0/2647131946 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa00c009c90 con 0x5637984fc480 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2647131946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa000044070 con 0x5637984fc480 2024-09-16T00:41:46.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7f9ff97fa640 1 -- 172.21.15.1:0/2647131946 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa00c023050 con 0x5637984fc480 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7fa01a2f6740 10 client.18597 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2647131946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637985154e0 con 0x5637984fc480 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7f9ff97fa640 1 -- 172.21.15.1:0/2647131946 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa000044070 con 0x5637984fc480 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7f9ff97fa640 1 -- 172.21.15.1:0/2647131946 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa00c00f420 con 0x5637984fc480 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7fa01a2f6740 10 client.18597 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7fa01a2f6740 10 client.18597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7fa01a2f6740 20 client.18597 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.118+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5637985169f0 0x563798518e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.119+0000 7fa01a2f6740 4 client.18597 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.119+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2647131946 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56379846e7f0 con 0x5637985169f0 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.119+0000 7fa0154b8640 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5637985169f0 0x563798518e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.119+0000 7f9ff97fa640 10 client.18597 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:46.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.119+0000 7fa0154b8640 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5637985169f0 0x563798518e10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f5cf5ec22b98ae0 server_cookie=73994568d8d48a2c in_seq=0 out_seq=0 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7f9ff97fa640 1 -- 172.21.15.1:0/2647131946 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56379846e7f0 con 0x5637985169f0 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7f9ff97fa640 10 client.18597 handle_command_reply: tid=0 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 client.18597 shutdown 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2647131946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fc480 msgr2=0x563798504120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fc480 0x563798504120 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00e750 tx=0x7fa00c00ec10 comp rx=0 tx=0).stop 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2647131946 shutdown_connections 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5637985169f0 0x563798518e10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x563798504660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fc480 0x563798504120 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7f9ff97fa640 0 client.18597 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/2647131946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fbb60 0x563798503be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2647131946 >> 172.21.15.1:0/2647131946 conn(0x5637984f80f0 msgr2=0x5637984fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2647131946 shutdown_connections 2024-09-16T00:41:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 1 -- 172.21.15.1:0/2647131946 wait complete. 2024-09-16T00:41:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.120+0000 7fa01a2f6740 20 client.18597 trim_cache size 0 max 16384 2024-09-16T00:41:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa01a2f6740 1 Processor -- start 2024-09-16T00:41:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa01a2f6740 1 -- start start 2024-09-16T00:41:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x563798502a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 0x5637985044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x56379851e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101b37e0 con 0x5637984fbb60 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101b10b0 con 0x563798506d40 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101aef00 con 0x5637984fc480 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa0154b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x563798502a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa0154b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x563798502a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49626/0 (socket says 172.21.15.1:49626) 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa0154b8640 1 -- 172.21.15.1:0/846198928 learned_addr learned my addr 172.21.15.1:0/846198928 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa0154b8640 1 -- 172.21.15.1:0/846198928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 msgr2=0x5637985044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa0154b8640 1 --2- 172.21.15.1:0/846198928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 0x5637985044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa0154b8640 1 -- 172.21.15.1:0/846198928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 msgr2=0x56379851e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa0154b8640 1 --2- 172.21.15.1:0/846198928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x56379851e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa0154b8640 1 -- 172.21.15.1:0/846198928 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637985061c0 con 0x5637984fbb60 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.138+0000 7fa0154b8640 1 --2- 172.21.15.1:0/846198928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x563798502a90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fa00003d610 tx=0x7fa00003d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1093620b35c64ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7f9ffb7fe640 1 -- 172.21.15.1:0/846198928 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000051070 con 0x5637984fbb60 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7f9ffb7fe640 1 -- 172.21.15.1:0/846198928 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa000044050 con 0x5637984fbb60 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7f9ffb7fe640 1 -- 172.21.15.1:0/846198928 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000035040 con 0x5637984fbb60 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7fa01a2f6740 1 -- 172.21.15.1:0/846198928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 msgr2=0x563798502a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/846198928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x563798502a90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fa00003d610 tx=0x7fa00003d5a0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7fa01a2f6740 1 -- 172.21.15.1:0/846198928 shutdown_connections 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/846198928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563798506d40 0x56379851e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/846198928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637984fc480 0x5637985044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/846198928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x563798502a90 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7fa01a2f6740 1 -- 172.21.15.1:0/846198928 >> 172.21.15.1:0/846198928 conn(0x5637984f80f0 msgr2=0x563798502380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7fa01a2f6740 1 -- 172.21.15.1:0/846198928 shutdown_connections 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.139+0000 7fa01a2f6740 1 -- 172.21.15.1:0/846198928 wait complete. 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa01a2f6740 1 -- start start 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa01a2f6740 1 Processor -- start 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x56379850ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5637984fc480 0x56379850b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa01a2f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563798506d40 0x56379850b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101af570 con 0x5637984fbb60 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101b09e0 con 0x5637984fc480 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa01a2f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101afc40 con 0x563798506d40 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa01a2f6740 10 client.0 _ll_register_callbacks cb 0x7fa00c00c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:46.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa0154b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x56379850ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa0154b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x56379850ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49640/0 (socket says 172.21.15.1:49640) 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.140+0000 7fa0154b8640 1 -- 172.21.15.1:0/3886220705 learned_addr learned my addr 172.21.15.1:0/3886220705 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa015cb9640 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5637984fc480 0x56379850b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa0154b8640 1 -- 172.21.15.1:0/3886220705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563798506d40 msgr2=0x56379850b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa0154b8640 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563798506d40 0x56379850b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa0154b8640 1 -- 172.21.15.1:0/3886220705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5637984fc480 msgr2=0x56379850b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa0154b8640 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5637984fc480 0x56379850b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa0154b8640 1 -- 172.21.15.1:0/3886220705 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637985809a0 con 0x5637984fbb60 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa0154b8640 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x56379850ad70 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fa000009900 tx=0x7fa00003d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868884a7bf5a6475 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7f9ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7f9ff97fa640 1 -- 172.21.15.1:0/3886220705 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000055020 con 0x5637984fbb60 2024-09-16T00:41:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa01a2f6740 1 -- 172.21.15.1:0/3886220705 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa000044050 con 0x5637984fbb60 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7f9ff97fa640 1 -- 172.21.15.1:0/3886220705 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa000005040 con 0x5637984fbb60 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7f9ff97fa640 1 -- 172.21.15.1:0/3886220705 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa000060050 con 0x5637984fbb60 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa01a2f6740 10 client.18603 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7fa01a2f6740 1 -- 172.21.15.1:0/3886220705 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563798581ce0 con 0x5637984fbb60 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.141+0000 7f9ff97fa640 1 -- 172.21.15.1:0/3886220705 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa000044050 con 0x5637984fbb60 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7f9ff97fa640 1 -- 172.21.15.1:0/3886220705 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa000031540 con 0x5637984fbb60 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7fa01a2f6740 10 client.18603 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7fa01a2f6740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7fa01a2f6740 20 client.18603 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563798581f10 0x563798584300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7fa01a2f6740 4 client.18603 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7fa01a2f6740 1 -- 172.21.15.1:0/3886220705 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563798468d00 con 0x563798581f10 2024-09-16T00:41:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7fa015cb9640 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563798581f10 0x563798584300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7f9ff97fa640 10 client.18603 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.142+0000 7fa015cb9640 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563798581f10 0x563798584300 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e6adbd4199fc92 server_cookie=bec167661bd427fd in_seq=0 out_seq=0 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.143+0000 7f9ff97fa640 1 -- 172.21.15.1:0/3886220705 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563798468d00 con 0x563798581f10 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.143+0000 7f9ff97fa640 10 client.18603 handle_command_reply: tid=0 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.143+0000 7fa01a2f6740 1 client.18603 shutdown 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.143+0000 7fa01a2f6740 1 -- 172.21.15.1:0/3886220705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 msgr2=0x56379850ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.143+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x56379850ad70 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fa000009900 tx=0x7fa00003d420 comp rx=0 tx=0).stop 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa01a2f6740 1 -- 172.21.15.1:0/3886220705 shutdown_connections 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563798581f10 0x563798584300 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563798506d40 0x56379850b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5637984fc480 0x56379850b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7f9ff97fa640 0 client.18603 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa01a2f6740 1 --2- 172.21.15.1:0/3886220705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637984fbb60 0x56379850ad70 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa01a2f6740 1 -- 172.21.15.1:0/3886220705 >> 172.21.15.1:0/3886220705 conn(0x5637984f80f0 msgr2=0x563798502ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa01a2f6740 1 -- 172.21.15.1:0/3886220705 shutdown_connections 2024-09-16T00:41:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa01a2f6740 1 -- 172.21.15.1:0/3886220705 wait complete. 2024-09-16T00:41:46.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa01a2f6740 20 client.18603 trim_cache size 0 max 16384 2024-09-16T00:41:46.155 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:41:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:41:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "78ec0544-3e8d-4d66-905a-d1ca88bd9c56", 2024-09-16T00:41:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:41:46.155 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:41:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa017f1d640 1 -- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ffc0a3240 msgr2=0x7f9ffc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa017f1d640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa0080cdbb0 tx=0x7fa00806f500 comp rx=0 tx=0).stop 2024-09-16T00:41:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa017f1d640 1 -- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102ef0 msgr2=0x7fa0100fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa017f1d640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102ef0 0x7fa0100fd700 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fa00800cce0 tx=0x7fa00800a620 comp rx=0 tx=0).stop 2024-09-16T00:41:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa017f1d640 1 -- 172.21.15.1:0/2499127328 shutdown_connections 2024-09-16T00:41:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa017f1d640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa010102ef0 0x7fa0100fd700 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.144+0000 7fa017f1d640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.145+0000 7fa017f1d640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa010102570 0x7fa0100fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.145+0000 7fa017f1d640 1 --2- 172.21.15.1:0/2499127328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010100e20 0x7fa0100fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.145+0000 7fa017f1d640 1 -- 172.21.15.1:0/2499127328 >> 172.21.15.1:0/2499127328 conn(0x7fa0100fc7d0 msgr2=0x7fa01006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.145+0000 7fa017f1d640 1 -- 172.21.15.1:0/2499127328 shutdown_connections 2024-09-16T00:41:46.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:46.145+0000 7fa017f1d640 1 -- 172.21.15.1:0/2499127328 wait complete. 2024-09-16T00:41:46.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:46 smithi001 ceph-mon[30934]: pgmap v751: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:46.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:46 smithi153 ceph-mon[35538]: pgmap v751: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:46.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:46 smithi019 ceph-mon[35186]: pgmap v751: 129 pgs: 129 active+clean; 6.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:47.005 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:41:47.265 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:47.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:47 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:41:47.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:47 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:47.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:47 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:41:47.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:47 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:47.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:47 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:41:47.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:47 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:47.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:47 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:41:47.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:47 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:47.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 -- 172.21.15.1:0/3456329664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60102790 msgr2=0x7f7f601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:47.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/3456329664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60102790 0x7f7f601067e0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009900 tx=0x7f7f5002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 -- 172.21.15.1:0/3456329664 shutdown_connections 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/3456329664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f60106f10 0x7f7f6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/3456329664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60102790 0x7f7f601067e0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/3456329664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f60101de0 0x7f7f601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 -- 172.21.15.1:0/3456329664 >> 172.21.15.1:0/3456329664 conn(0x7f7f600fd8c0 msgr2=0x7f7f600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 -- 172.21.15.1:0/3456329664 shutdown_connections 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 -- 172.21.15.1:0/3456329664 wait complete. 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 Processor -- start 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.777+0000 7f7f67b2c640 1 -- start start 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f67b2c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f60101de0 0x7f7f600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f67b2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f60102790 0x7f7f60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f67b2c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60106f10 0x7f7f60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f67b2c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7f60111f90 con 0x7f7f60106f10 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f67b2c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f60111e10 con 0x7f7f60102790 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f67b2c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f60112110 con 0x7f7f60101de0 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f660c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60106f10 0x7f7f60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f658c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f60101de0 0x7f7f600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f660c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60106f10 0x7f7f60074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49652/0 (socket says 172.21.15.1:49652) 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f660c9640 1 -- 172.21.15.1:0/144552264 learned_addr learned my addr 172.21.15.1:0/144552264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f650c7640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f60102790 0x7f7f60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f660c9640 1 -- 172.21.15.1:0/144552264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f60101de0 msgr2=0x7f7f600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f660c9640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f60101de0 0x7f7f600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f660c9640 1 -- 172.21.15.1:0/144552264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f60102790 msgr2=0x7f7f60074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f660c9640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f60102790 0x7f7f60074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f660c9640 1 -- 172.21.15.1:0/144552264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f601a68d0 con 0x7f7f60106f10 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f660c9640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60106f10 0x7f7f60074f70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800cce0 tx=0x7f7f5800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.778+0000 7f7f56ffd640 1 -- 172.21.15.1:0/144552264 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800f450 con 0x7f7f60106f10 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.779+0000 7f7f56ffd640 1 -- 172.21.15.1:0/144552264 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5800fad0 con 0x7f7f60106f10 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.779+0000 7f7f67b2c640 1 -- 172.21.15.1:0/144552264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f601a6bc0 con 0x7f7f60106f10 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.779+0000 7f7f56ffd640 1 -- 172.21.15.1:0/144552264 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58018b60 con 0x7f7f60106f10 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.779+0000 7f7f67b2c640 1 -- 172.21.15.1:0/144552264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f601a7080 con 0x7f7f60106f10 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.781+0000 7f7f56ffd640 1 -- 172.21.15.1:0/144552264 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f7f5800fc80 con 0x7f7f60106f10 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.781+0000 7f7f69f05740 1 Processor -- start 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.781+0000 7f7f69f05740 1 -- start start 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.781+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27e4a0 0x55e1bb27e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.781+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27edc0 0x55e1bb289140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb289680 0x55e1bb28ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f69f05740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e1bb1e42e0 con 0x55e1bb27edc0 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f69f05740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e1bb18bce0 con 0x55e1bb27e4a0 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f69f05740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e1bb1e02a0 con 0x55e1bb289680 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f56ffd640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7f4c0a3240 0x7f7f4c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f658c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27edc0 0x55e1bb289140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f658c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27edc0 0x55e1bb289140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49666/0 (socket says 172.21.15.1:49666) 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f658c8640 1 -- 172.21.15.1:0/1565139641 learned_addr learned my addr 172.21.15.1:0/1565139641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f650c7640 1 --2- 172.21.15.1:0/1565139641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27e4a0 0x55e1bb27e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f650c7640 1 --2- 172.21.15.1:0/1565139641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb289680 0x55e1bb28ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f56ffd640 1 -- 172.21.15.1:0/144552264 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7f580d01a0 con 0x7f7f60106f10 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f660c9640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7f4c0a3240 0x7f7f4c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f658c8640 1 -- 172.21.15.1:0/1565139641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb289680 msgr2=0x55e1bb28ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:47.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f658c8640 1 --2- 172.21.15.1:0/1565139641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb289680 0x55e1bb28ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f658c8640 1 -- 172.21.15.1:0/1565139641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27e4a0 msgr2=0x55e1bb27e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f658c8640 1 --2- 172.21.15.1:0/1565139641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27e4a0 0x55e1bb27e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f658c8640 1 -- 172.21.15.1:0/1565139641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1bb170440 con 0x55e1bb27edc0 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f658c8640 1 --2- 172.21.15.1:0/1565139641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27edc0 0x55e1bb289140 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009a70 tx=0x7f7f5c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70399a6c6aade439 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f660c9640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7f4c0a3240 0x7f7f4c0a5700 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7f58098820 tx=0x7f7f5800cf10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.782+0000 7f7f4bfff640 1 -- 172.21.15.1:0/1565139641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c017070 con 0x55e1bb27edc0 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f4bfff640 1 -- 172.21.15.1:0/1565139641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c013470 con 0x55e1bb27edc0 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f4bfff640 1 -- 172.21.15.1:0/1565139641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c012630 con 0x55e1bb27edc0 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f69f05740 1 -- 172.21.15.1:0/1565139641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27edc0 msgr2=0x55e1bb289140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f69f05740 1 --2- 172.21.15.1:0/1565139641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27edc0 0x55e1bb289140 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009a70 tx=0x7f7f5c009f30 comp rx=0 tx=0).stop 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f69f05740 1 -- 172.21.15.1:0/1565139641 shutdown_connections 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f69f05740 1 --2- 172.21.15.1:0/1565139641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb289680 0x55e1bb28ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f69f05740 1 --2- 172.21.15.1:0/1565139641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27edc0 0x55e1bb289140 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f69f05740 1 --2- 172.21.15.1:0/1565139641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27e4a0 0x55e1bb27e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f69f05740 1 -- 172.21.15.1:0/1565139641 >> 172.21.15.1:0/1565139641 conn(0x55e1bb27aa30 msgr2=0x55e1bb27ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f69f05740 1 -- 172.21.15.1:0/1565139641 shutdown_connections 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.783+0000 7f7f69f05740 1 -- 172.21.15.1:0/1565139641 wait complete. 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f69f05740 1 -- start start 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f69f05740 1 Processor -- start 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb288f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27edc0 0x55e1bb2861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb289680 0x55e1bb286710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f69f05740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e1bb28c250 con 0x55e1bb27e4a0 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f69f05740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e1bb19f050 con 0x55e1bb289680 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f69f05740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e1bb28c3d0 con 0x55e1bb27edc0 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f69f05740 10 client.0 _ll_register_callbacks cb 0x55e1bb1df160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f650c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb288f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f650c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb288f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49676/0 (socket says 172.21.15.1:49676) 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f650c7640 1 -- 172.21.15.1:0/395978174 learned_addr learned my addr 172.21.15.1:0/395978174 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.784+0000 7f7f658c8640 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27edc0 0x55e1bb2861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f650c7640 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb289680 0x55e1bb286710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f650c7640 1 -- 172.21.15.1:0/395978174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27edc0 msgr2=0x55e1bb2861d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f650c7640 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27edc0 0x55e1bb2861d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f650c7640 1 -- 172.21.15.1:0/395978174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb289680 msgr2=0x55e1bb286710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f650c7640 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb289680 0x55e1bb286710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f650c7640 1 -- 172.21.15.1:0/395978174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1baf553c0 con 0x55e1bb27e4a0 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f650c7640 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb288f20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009900 tx=0x7f7f5002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d003fd092ba62576 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f497fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f497fa640 1 -- 172.21.15.1:0/395978174 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50035040 con 0x55e1bb27e4a0 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f497fa640 1 -- 172.21.15.1:0/395978174 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7f50005020 con 0x55e1bb27e4a0 2024-09-16T00:41:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f69f05740 1 -- 172.21.15.1:0/395978174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c013470 con 0x55e1bb27e4a0 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f497fa640 1 -- 172.21.15.1:0/395978174 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f500044d0 con 0x55e1bb27e4a0 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f69f05740 10 client.18615 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f69f05740 1 -- 172.21.15.1:0/395978174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1bb297b70 con 0x55e1bb27e4a0 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.785+0000 7f7f497fa640 1 -- 172.21.15.1:0/395978174 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c013470 con 0x55e1bb27e4a0 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f497fa640 1 -- 172.21.15.1:0/395978174 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7f500035f0 con 0x55e1bb27e4a0 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f69f05740 10 client.18615 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f69f05740 10 client.18615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f69f05740 20 client.18615 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f69f05740 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e1bb298f80 0x55e1bb29b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f69f05740 4 client.18615 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f69f05740 1 -- 172.21.15.1:0/395978174 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1bb1f1850 con 0x55e1bb298f80 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f658c8640 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e1bb298f80 0x55e1bb29b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f497fa640 10 client.18615 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:47.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.786+0000 7f7f658c8640 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e1bb298f80 0x55e1bb29b3b0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5afc8fe38adc0bf server_cookie=90b07eec23f9d42d in_seq=0 out_seq=0 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f497fa640 1 -- 172.21.15.1:0/395978174 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1bb1f1850 con 0x55e1bb298f80 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f497fa640 10 client.18615 handle_command_reply: tid=0 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 client.18615 shutdown 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 -- 172.21.15.1:0/395978174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 msgr2=0x55e1bb288f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb288f20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009900 tx=0x7f7f5002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 -- 172.21.15.1:0/395978174 shutdown_connections 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e1bb298f80 0x55e1bb29b3b0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb289680 0x55e1bb286710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27edc0 0x55e1bb2861d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 --2- 172.21.15.1:0/395978174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb288f20 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f497fa640 0 client.18615 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 -- 172.21.15.1:0/395978174 >> 172.21.15.1:0/395978174 conn(0x55e1bb27aa30 msgr2=0x55e1bb279b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 -- 172.21.15.1:0/395978174 shutdown_connections 2024-09-16T00:41:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 1 -- 172.21.15.1:0/395978174 wait complete. 2024-09-16T00:41:47.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.787+0000 7f7f69f05740 20 client.18615 trim_cache size 0 max 16384 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.805+0000 7f7f69f05740 1 Processor -- start 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.805+0000 7f7f69f05740 1 -- start start 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb2954b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb2a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb289680 0x55e1bb2a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f69f05740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601b38d0 con 0x55e1bb27e4a0 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f69f05740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601b11a0 con 0x55e1bb27edc0 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f69f05740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601aeff0 con 0x55e1bb289680 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f650c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb2954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f650c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb2954b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49688/0 (socket says 172.21.15.1:49688) 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f650c7640 1 -- 172.21.15.1:0/372930019 learned_addr learned my addr 172.21.15.1:0/372930019 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f658c8640 1 --2- 172.21.15.1:0/372930019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb2a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f650c7640 1 -- 172.21.15.1:0/372930019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb289680 msgr2=0x55e1bb2a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f650c7640 1 --2- 172.21.15.1:0/372930019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb289680 0x55e1bb2a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f650c7640 1 -- 172.21.15.1:0/372930019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 msgr2=0x55e1bb2a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f650c7640 1 --2- 172.21.15.1:0/372930019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb2a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f650c7640 1 -- 172.21.15.1:0/372930019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1bb27a4d0 con 0x55e1bb27e4a0 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f658c8640 1 --2- 172.21.15.1:0/372930019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb2a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f650c7640 1 --2- 172.21.15.1:0/372930019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb2954b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f7f50033000 tx=0x7f7f500316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ba2045fa677da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f4b7fe640 1 -- 172.21.15.1:0/372930019 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500044d0 con 0x55e1bb27e4a0 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f4b7fe640 1 -- 172.21.15.1:0/372930019 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f500035f0 con 0x55e1bb27e4a0 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.806+0000 7f7f4b7fe640 1 -- 172.21.15.1:0/372930019 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50005260 con 0x55e1bb27e4a0 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.807+0000 7f7f69f05740 1 -- 172.21.15.1:0/372930019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 msgr2=0x55e1bb2954b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.807+0000 7f7f69f05740 1 --2- 172.21.15.1:0/372930019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb2954b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f7f50033000 tx=0x7f7f500316f0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.807+0000 7f7f69f05740 1 -- 172.21.15.1:0/372930019 shutdown_connections 2024-09-16T00:41:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.807+0000 7f7f69f05740 1 --2- 172.21.15.1:0/372930019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb289680 0x55e1bb2a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.807+0000 7f7f69f05740 1 --2- 172.21.15.1:0/372930019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb2a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.807+0000 7f7f69f05740 1 --2- 172.21.15.1:0/372930019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb27e4a0 0x55e1bb2954b0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.807+0000 7f7f69f05740 1 -- 172.21.15.1:0/372930019 >> 172.21.15.1:0/372930019 conn(0x55e1bb27aa30 msgr2=0x55e1bb284aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.807+0000 7f7f69f05740 1 -- 172.21.15.1:0/372930019 shutdown_connections 2024-09-16T00:41:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.807+0000 7f7f69f05740 1 -- 172.21.15.1:0/372930019 wait complete. 2024-09-16T00:41:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f69f05740 1 -- start start 2024-09-16T00:41:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f69f05740 1 Processor -- start 2024-09-16T00:41:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27e4a0 0x55e1bb28d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb28d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f69f05740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb289680 0x55e1bb28dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f69f05740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601af660 con 0x55e1bb289680 2024-09-16T00:41:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f69f05740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601b0ad0 con 0x55e1bb27edc0 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f69f05740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601afd30 con 0x55e1bb27e4a0 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f69f05740 10 client.0 _ll_register_callbacks cb 0x7f7f50009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f650c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb289680 0x55e1bb28dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f650c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb289680 0x55e1bb28dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49694/0 (socket says 172.21.15.1:49694) 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f650c7640 1 -- 172.21.15.1:0/3402087310 learned_addr learned my addr 172.21.15.1:0/3402087310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f658c8640 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb28d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.808+0000 7f7f650c7640 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27e4a0 0x55e1bb28d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f650c7640 1 -- 172.21.15.1:0/3402087310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27e4a0 msgr2=0x55e1bb28d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f650c7640 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27e4a0 0x55e1bb28d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f650c7640 1 -- 172.21.15.1:0/3402087310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 msgr2=0x55e1bb28d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f650c7640 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb28d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f650c7640 1 -- 172.21.15.1:0/3402087310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1bb302c50 con 0x55e1bb289680 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f658c8640 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb28d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f650c7640 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb289680 0x55e1bb28dda0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f7f50031b20 tx=0x7f7f500314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b76420673fad463 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f497fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f497fa640 1 -- 172.21.15.1:0/3402087310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500044e0 con 0x55e1bb289680 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f69f05740 1 -- 172.21.15.1:0/3402087310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f500035f0 con 0x55e1bb289680 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f497fa640 1 -- 172.21.15.1:0/3402087310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7f50003b90 con 0x55e1bb289680 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f497fa640 1 -- 172.21.15.1:0/3402087310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f50003d90 con 0x55e1bb289680 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f69f05740 10 client.18621 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f69f05740 1 -- 172.21.15.1:0/3402087310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1bb303ea0 con 0x55e1bb289680 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.809+0000 7f7f497fa640 1 -- 172.21.15.1:0/3402087310 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f500035f0 con 0x55e1bb289680 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f497fa640 1 -- 172.21.15.1:0/3402087310 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7f50004b10 con 0x55e1bb289680 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f69f05740 10 client.18621 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f69f05740 10 client.18621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f69f05740 20 client.18621 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f69f05740 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e1bb3044c0 0x55e1bb3068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f69f05740 4 client.18621 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f69f05740 1 -- 172.21.15.1:0/3402087310 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1bb286aa0 con 0x55e1bb3044c0 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f658c8640 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e1bb3044c0 0x55e1bb3068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f497fa640 10 client.18621 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.810+0000 7f7f658c8640 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e1bb3044c0 0x55e1bb3068b0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a006b69631623ab2 server_cookie=b56d01ea39805493 in_seq=0 out_seq=0 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f497fa640 1 -- 172.21.15.1:0/3402087310 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e1bb286aa0 con 0x55e1bb3044c0 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f497fa640 10 client.18621 handle_command_reply: tid=0 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 client.18621 shutdown 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 -- 172.21.15.1:0/3402087310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb289680 msgr2=0x55e1bb28dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb289680 0x55e1bb28dda0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f7f50031b20 tx=0x7f7f500314c0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 -- 172.21.15.1:0/3402087310 shutdown_connections 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e1bb3044c0 0x55e1bb3068b0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1bb289680 0x55e1bb28dda0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f497fa640 0 client.18621 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1bb27edc0 0x55e1bb28d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 --2- 172.21.15.1:0/3402087310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1bb27e4a0 0x55e1bb28d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 -- 172.21.15.1:0/3402087310 >> 172.21.15.1:0/3402087310 conn(0x55e1bb27aa30 msgr2=0x55e1bb280710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 -- 172.21.15.1:0/3402087310 shutdown_connections 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 1 -- 172.21.15.1:0/3402087310 wait complete. 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.811+0000 7f7f69f05740 20 client.18621 trim_cache size 0 max 16384 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:41:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "78ec0544-3e8d-4d66-905a-d1ca88bd9c56": { 2024-09-16T00:41:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:41:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "78ec0544-3e8d-4d66-905a-d1ca88bd9c56", 2024-09-16T00:41:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:41:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:41:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:41:47.818 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 -- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7f4c0a3240 msgr2=0x7f7f4c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7f4c0a3240 0x7f7f4c0a5700 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7f58098820 tx=0x7f7f5800cf10 comp rx=0 tx=0).stop 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 -- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60106f10 msgr2=0x7f7f60074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60106f10 0x7f7f60074f70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800cce0 tx=0x7f7f5800a620 comp rx=0 tx=0).stop 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 -- 172.21.15.1:0/144552264 shutdown_connections 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f60106f10 0x7f7f60074f70 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f60102790 0x7f7f60074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7f4c0a3240 0x7f7f4c0a5700 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 --2- 172.21.15.1:0/144552264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f60101de0 0x7f7f600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 -- 172.21.15.1:0/144552264 >> 172.21.15.1:0/144552264 conn(0x7f7f600fd8c0 msgr2=0x7f7f600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 -- 172.21.15.1:0/144552264 shutdown_connections 2024-09-16T00:41:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:47.812+0000 7f7f67b2c640 1 -- 172.21.15.1:0/144552264 wait complete. 2024-09-16T00:41:47.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:47 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:41:47.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:47 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:47.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:47 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:41:47.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:47 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:48.622 INFO:tasks.cephfs.filesystem:scrub status for tag:78ec0544-3e8d-4d66-905a-d1ca88bd9c56 - {'path': '/', 'tag': '78ec0544-3e8d-4d66-905a-d1ca88bd9c56', 'options': 'recursive,force'} 2024-09-16T00:41:48.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:48 smithi001 ceph-mon[30934]: pgmap v752: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:48.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:48 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:41:48.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:48 smithi153 ceph-mon[35538]: pgmap v752: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:48.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:48 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:41:48.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:48 smithi019 ceph-mon[35186]: pgmap v752: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:48.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:48 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:41:50.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:50 smithi001 ceph-mon[30934]: pgmap v753: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:50.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:50 smithi153 ceph-mon[35538]: pgmap v753: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:50.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:50 smithi019 ceph-mon[35186]: pgmap v753: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:52.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:52 smithi001 ceph-mon[30934]: pgmap v754: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:52.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:52 smithi153 ceph-mon[35538]: pgmap v754: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:52.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:52 smithi019 ceph-mon[35186]: pgmap v754: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:54 smithi001 ceph-mon[30934]: pgmap v755: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:54.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:54 smithi153 ceph-mon[35538]: pgmap v755: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:54.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:54 smithi019 ceph-mon[35186]: pgmap v755: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:41:55.142 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.4 2024-09-16T00:41:55.143 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:55.417 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:437/600: open fd = 3 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:438/600: open fd = 3 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:439/600: open fd = 3 2024-09-16T00:41:55.782 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:440/600: open fd = 3 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:441/600: open fd = 3 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:442/600: open fd = 3 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.783 INFO:tasks.workunit.client.0.smithi001.stdout:443/600: open fd = 3 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:444/600: open fd = 3 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:445/600: open fd = 3 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:446/600: open fd = 3 2024-09-16T00:41:55.784 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:447/600: open fd = 3 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:448/600: open fd = 3 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:449/600: open fd = 3 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.785 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:450/600: open fd = 3 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:451/600: open fd = 3 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:452/600: open fd = 3 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:453/600: open fd = 3 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.786 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.787 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.787 INFO:tasks.workunit.client.0.smithi001.stdout:454/600: open fd = 3 2024-09-16T00:41:55.787 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.787 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.787 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.787 INFO:tasks.workunit.client.0.smithi001.stdout:455/600: open fd = 3 2024-09-16T00:41:55.787 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.787 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:456/600: open fd = 3 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:457/600: open fd = 3 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.788 INFO:tasks.workunit.client.0.smithi001.stdout:458/600: open fd = 3 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:459/600: open fd = 3 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:460/600: open fd = 3 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:461/600: open fd = 3 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.789 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:462/600: open fd = 3 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:463/600: open fd = 3 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:464/600: open fd = 3 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.790 INFO:tasks.workunit.client.0.smithi001.stdout:465/600: open fd = 3 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:466/600: open fd = 3 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:467/600: open fd = 3 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:468/600: open fd = 3 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.791 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:469/600: open fd = 3 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:470/600: open fd = 3 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:471/600: open fd = 3 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.792 INFO:tasks.workunit.client.0.smithi001.stdout:472/600: open fd = 3 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:473/600: open fd = 3 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:474/600: open fd = 3 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:475/600: open fd = 3 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.793 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:476/600: open fd = 3 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:477/600: open fd = 3 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:478/600: open fd = 3 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:479/600: open fd = 3 2024-09-16T00:41:55.794 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:480/600: open fd = 3 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:481/600: open fd = 3 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:482/600: open fd = 3 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.795 INFO:tasks.workunit.client.0.smithi001.stdout:483/600: open fd = 3 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:484/600: open fd = 3 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:485/600: open fd = 3 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:486/600: open fd = 3 2024-09-16T00:41:55.796 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:487/600: open fd = 3 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:488/600: open fd = 3 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:489/600: open fd = 3 2024-09-16T00:41:55.797 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:490/600: open fd = 3 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:491/600: open fd = 3 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:492/600: open fd = 3 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.798 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:493/600: open fd = 3 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:494/600: open fd = 3 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:495/600: open fd = 3 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.799 INFO:tasks.workunit.client.0.smithi001.stdout:496/600: open fd = 3 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:497/600: open fd = 3 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:498/600: open fd = 3 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:41:55.800 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:41:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.961+0000 7faeea943640 1 -- 172.21.15.1:0/2603927216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4101420 msgr2=0x7faee410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.961+0000 7faeea943640 1 --2- 172.21.15.1:0/2603927216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4101420 0x7faee410d950 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7faedc009cd0 tx=0x7faedc02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.961+0000 7faeea943640 1 -- 172.21.15.1:0/2603927216 shutdown_connections 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.961+0000 7faeea943640 1 --2- 172.21.15.1:0/2603927216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4101420 0x7faee410d950 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.961+0000 7faeea943640 1 --2- 172.21.15.1:0/2603927216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faee4100a80 0x7faee4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.961+0000 7faeea943640 1 --2- 172.21.15.1:0/2603927216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee4106a80 0x7faee4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.961+0000 7faeea943640 1 -- 172.21.15.1:0/2603927216 >> 172.21.15.1:0/2603927216 conn(0x7faee40fc7a0 msgr2=0x7faee40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.961+0000 7faeea943640 1 -- 172.21.15.1:0/2603927216 shutdown_connections 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.961+0000 7faeea943640 1 -- 172.21.15.1:0/2603927216 wait complete. 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faeea943640 1 Processor -- start 2024-09-16T00:41:55.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faeea943640 1 -- start start 2024-09-16T00:41:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faeea943640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faee4100a80 0x7faee4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faeea943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee4101420 0x7faee41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faeea943640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4106a80 0x7faee419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faeea943640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faee4110ea0 con 0x7faee4106a80 2024-09-16T00:41:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faeea943640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faee4110d20 con 0x7faee4101420 2024-09-16T00:41:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faeea943640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faee4111020 con 0x7faee4100a80 2024-09-16T00:41:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faee8ee0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4106a80 0x7faee419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faee8ee0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4106a80 0x7faee419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48712/0 (socket says 172.21.15.1:48712) 2024-09-16T00:41:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faee8ee0640 1 -- 172.21.15.1:0/3139712703 learned_addr learned my addr 172.21.15.1:0/3139712703 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.962+0000 7faedbfff640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee4101420 0x7faee41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee3fff640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faee4100a80 0x7faee4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee8ee0640 1 -- 172.21.15.1:0/3139712703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faee4100a80 msgr2=0x7faee4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee8ee0640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faee4100a80 0x7faee4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee8ee0640 1 -- 172.21.15.1:0/3139712703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee4101420 msgr2=0x7faee41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee8ee0640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee4101420 0x7faee41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee8ee0640 1 -- 172.21.15.1:0/3139712703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee419d430 con 0x7faee4106a80 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee8ee0640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4106a80 0x7faee419cd50 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7faedc006010 tx=0x7faedc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee1ffb640 1 -- 172.21.15.1:0/3139712703 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc004450 con 0x7faee4106a80 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee1ffb640 1 -- 172.21.15.1:0/3139712703 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faedc003e10 con 0x7faee4106a80 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee419d6c0 con 0x7faee4106a80 2024-09-16T00:41:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faee1ffb640 1 -- 172.21.15.1:0/3139712703 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc046480 con 0x7faee4106a80 2024-09-16T00:41:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.963+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee41a5000 con 0x7faee4106a80 2024-09-16T00:41:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.964+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee4100ee0 con 0x7faee4106a80 2024-09-16T00:41:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.965+0000 7faee1ffb640 1 -- 172.21.15.1:0/3139712703 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7faedc0045f0 con 0x7faee4106a80 2024-09-16T00:41:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.966+0000 7faee1ffb640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faed00a3240 0x7faed00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.966+0000 7faee3fff640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faed00a3240 0x7faed00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.966+0000 7faee1ffb640 1 -- 172.21.15.1:0/3139712703 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7faedc02e030 con 0x7faee4106a80 2024-09-16T00:41:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.966+0000 7faee3fff640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faed00a3240 0x7faed00a5700 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faed4005e10 tx=0x7faed4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:55.970+0000 7faee1ffb640 1 -- 172.21.15.1:0/3139712703 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faedc04b060 con 0x7faee4106a80 2024-09-16T00:41:56.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.182+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faee410a020 con 0x7faee4106a80 2024-09-16T00:41:56.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.183+0000 7faee1ffb640 1 -- 172.21.15.1:0/3139712703 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7faedc0bdfe0 con 0x7faee4106a80 2024-09-16T00:41:56.184 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:41:56.184 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:41:56.184 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:41:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faed00a3240 msgr2=0x7faed00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faed00a3240 0x7faed00a5700 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faed4005e10 tx=0x7faed4005da0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4106a80 msgr2=0x7faee419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4106a80 0x7faee419cd50 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7faedc006010 tx=0x7faedc003c20 comp rx=0 tx=0).stop 2024-09-16T00:41:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 shutdown_connections 2024-09-16T00:41:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faee4106a80 0x7faee419cd50 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee4101420 0x7faee41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faed00a3240 0x7faed00a5700 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 --2- 172.21.15.1:0/3139712703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faee4100a80 0x7faee4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 >> 172.21.15.1:0/3139712703 conn(0x7faee40fc7a0 msgr2=0x7faee406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.186+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 shutdown_connections 2024-09-16T00:41:56.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:56.187+0000 7faeea943640 1 -- 172.21.15.1:0/3139712703 wait complete. 2024-09-16T00:41:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:56 smithi001 ceph-mon[30934]: pgmap v756: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:56.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:56 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3139712703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:56.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:56 smithi153 ceph-mon[35538]: pgmap v756: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:56.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:56 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3139712703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:56.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:56 smithi019 ceph-mon[35186]: pgmap v756: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:41:56.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:56 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3139712703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:56.955 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0' 2024-09-16T00:41:56.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0 2024-09-16T00:41:57.215 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.729+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/2340182765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 msgr2=0x7fa0a01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.729+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/2340182765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 0x7fa0a01031c0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa090009ca0 tx=0x7fa09002a950 comp rx=0 tx=0).stop 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.729+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/2340182765 shutdown_connections 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.729+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/2340182765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0a010c180 0x7fa0a010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.729+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/2340182765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0103700 0x7fa0a010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.729+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/2340182765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 0x7fa0a01031c0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.729+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/2340182765 >> 172.21.15.1:0/2340182765 conn(0x7fa0a00fc7a0 msgr2=0x7fa0a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.729+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/2340182765 shutdown_connections 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.729+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/2340182765 wait complete. 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa0a5d1e640 1 Processor -- start 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa0a5d1e640 1 -- start start 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa0a5d1e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 0x7fa0a01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa0a5d1e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0103700 0x7fa0a01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa0a5d1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0a010c180 0x7fa0a019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa0a5d1e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0111fb0 con 0x7fa0a0102de0 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa0a5d1e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0111e30 con 0x7fa0a0103700 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa0a5d1e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0112130 con 0x7fa0a010c180 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 0x7fa0a01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 0x7fa0a01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48732/0 (socket says 172.21.15.1:48732) 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.730+0000 7fa09f7fe640 1 -- 172.21.15.1:0/1907356334 learned_addr learned my addr 172.21.15.1:0/1907356334 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09effd640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0103700 0x7fa0a01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09ffff640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0a010c180 0x7fa0a019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09f7fe640 1 -- 172.21.15.1:0/1907356334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0a010c180 msgr2=0x7fa0a019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0a010c180 0x7fa0a019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09f7fe640 1 -- 172.21.15.1:0/1907356334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0103700 msgr2=0x7fa0a01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0103700 0x7fa0a01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09f7fe640 1 -- 172.21.15.1:0/1907356334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a019d460 con 0x7fa0a0102de0 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09ffff640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0a010c180 0x7fa0a019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 0x7fa0a01984b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fa090005cc0 tx=0x7fa09002f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09cff9640 1 -- 172.21.15.1:0/1907356334 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090034040 con 0x7fa0a0102de0 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09cff9640 1 -- 172.21.15.1:0/1907356334 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa090030920 con 0x7fa0a0102de0 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/1907356334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a019d6f0 con 0x7fa0a0102de0 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa09cff9640 1 -- 172.21.15.1:0/1907356334 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090004830 con 0x7fa0a0102de0 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.731+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/1907356334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a01a5030 con 0x7fa0a0102de0 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.733+0000 7fa09cff9640 1 -- 172.21.15.1:0/1907356334 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa090037020 con 0x7fa0a0102de0 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa0a80f7740 1 Processor -- start 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa0a80f7740 1 -- start start 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa09cff9640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0880a3240 0x7fa0880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee7760 0x55a33fee7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fef2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fef2940 0x55a33fef4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a33fe4d2a0 con 0x55a33fee8080 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a33fdf4b10 con 0x55a33fee7760 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a33fe49260 con 0x55a33fef2940 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa09cff9640 1 -- 172.21.15.1:0/1907356334 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa0900f0da0 con 0x7fa0a0102de0 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa09ffff640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0880a3240 0x7fa0880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa09effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fef2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa09effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fef2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48734/0 (socket says 172.21.15.1:48734) 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa09effd640 1 -- 172.21.15.1:0/1492530009 learned_addr learned my addr 172.21.15.1:0/1492530009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.734+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/1492530009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee7760 0x55a33fee7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa09effd640 1 -- 172.21.15.1:0/1492530009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fef2940 msgr2=0x55a33fef4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa09effd640 1 --2- 172.21.15.1:0/1492530009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fef2940 0x55a33fef4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa09effd640 1 -- 172.21.15.1:0/1492530009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee7760 msgr2=0x55a33fee7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:57.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa09effd640 1 --2- 172.21.15.1:0/1492530009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee7760 0x55a33fee7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa09effd640 1 -- 172.21.15.1:0/1492530009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33fbbeb90 con 0x55a33fee8080 2024-09-16T00:41:57.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/1492530009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee7760 0x55a33fee7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:57.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa09ffff640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0880a3240 0x7fa0880a5700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa09800a900 tx=0x7fa09800ad00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:57.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa09effd640 1 --2- 172.21.15.1:0/1492530009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fef2400 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa09400bc30 tx=0x7fa094012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a8b73db08c33b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:57.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa085ffb640 1 -- 172.21.15.1:0/1492530009 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09400be40 con 0x55a33fee8080 2024-09-16T00:41:57.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa085ffb640 1 -- 172.21.15.1:0/1492530009 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09400d920 con 0x55a33fee8080 2024-09-16T00:41:57.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.735+0000 7fa085ffb640 1 -- 172.21.15.1:0/1492530009 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094003b40 con 0x55a33fee8080 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 -- 172.21.15.1:0/1492530009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 msgr2=0x55a33fef2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/1492530009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fef2400 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa09400bc30 tx=0x7fa094012b30 comp rx=0 tx=0).stop 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 -- 172.21.15.1:0/1492530009 shutdown_connections 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/1492530009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fef2940 0x55a33fef4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/1492530009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fef2400 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/1492530009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee7760 0x55a33fee7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 -- 172.21.15.1:0/1492530009 >> 172.21.15.1:0/1492530009 conn(0x55a33fee3cf0 msgr2=0x55a33fee40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 -- 172.21.15.1:0/1492530009 shutdown_connections 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 -- 172.21.15.1:0/1492530009 wait complete. 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.736+0000 7fa0a80f7740 1 -- start start 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa0a80f7740 1 Processor -- start 2024-09-16T00:41:57.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee8080 0x55a33feefd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a33fef5510 con 0x55a33fee7760 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a33fdd56e0 con 0x55a33fef2940 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a33fef5690 con 0x55a33fee8080 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa0a80f7740 10 client.0 _ll_register_callbacks cb 0x55a33fdf9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feef830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48736/0 (socket says 172.21.15.1:48736) 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09f7fe640 1 -- 172.21.15.1:0/4554842 learned_addr learned my addr 172.21.15.1:0/4554842 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09ffff640 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09effd640 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee8080 0x55a33feefd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09f7fe640 1 -- 172.21.15.1:0/4554842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee8080 msgr2=0x55a33feefd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee8080 0x55a33feefd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09f7fe640 1 -- 172.21.15.1:0/4554842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 msgr2=0x55a33fef02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09f7fe640 1 -- 172.21.15.1:0/4554842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33fefe9d0 con 0x55a33fee7760 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09ffff640 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feef830 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa0900b93c0 tx=0x7fa0900bd720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa302f38fa218a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.737+0000 7fa07effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa07effd640 1 -- 172.21.15.1:0/4554842 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0900c5bd0 con 0x55a33fee7760 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa07effd640 1 -- 172.21.15.1:0/4554842 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa0900ee550 con 0x55a33fee7760 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa0a80f7740 1 -- 172.21.15.1:0/4554842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09400d920 con 0x55a33fee7760 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa07effd640 1 -- 172.21.15.1:0/4554842 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa090004040 con 0x55a33fee7760 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa0a80f7740 10 client.18639 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa0a80f7740 1 -- 172.21.15.1:0/4554842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a33ff01380 con 0x55a33fee7760 2024-09-16T00:41:57.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa07effd640 1 -- 172.21.15.1:0/4554842 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09400d920 con 0x55a33fee7760 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa07effd640 1 -- 172.21.15.1:0/4554842 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa09004d030 con 0x55a33fee7760 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa0a80f7740 10 client.18639 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa0a80f7740 10 client.18639 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa0a80f7740 20 client.18639 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a33ff02830 0x55a33ff04cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa0a80f7740 4 client.18639 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.738+0000 7fa0a80f7740 1 -- 172.21.15.1:0/4554842 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a33fe5a9d0 con 0x55a33ff02830 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.739+0000 7fa09effd640 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a33ff02830 0x55a33ff04cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.739+0000 7fa09effd640 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a33ff02830 0x55a33ff04cf0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49335112eb537a2d server_cookie=a14679702e861cfe in_seq=0 out_seq=0 2024-09-16T00:41:57.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.739+0000 7fa07effd640 10 client.18639 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa07effd640 1 -- 172.21.15.1:0/4554842 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a33fe5a9d0 con 0x55a33ff02830 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa07effd640 10 client.18639 handle_command_reply: tid=0 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 client.18639 shutdown 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 -- 172.21.15.1:0/4554842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 msgr2=0x55a33feef830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feef830 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa0900b93c0 tx=0x7fa0900bd720 comp rx=0 tx=0).stop 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 -- 172.21.15.1:0/4554842 shutdown_connections 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a33ff02830 0x55a33ff04cf0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa07effd640 0 client.18639 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee8080 0x55a33feefd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/4554842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feef830 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 -- 172.21.15.1:0/4554842 >> 172.21.15.1:0/4554842 conn(0x55a33fee3cf0 msgr2=0x55a33fee2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 -- 172.21.15.1:0/4554842 shutdown_connections 2024-09-16T00:41:57.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 1 -- 172.21.15.1:0/4554842 wait complete. 2024-09-16T00:41:57.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.740+0000 7fa0a80f7740 20 client.18639 trim_cache size 0 max 16384 2024-09-16T00:41:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa0a80f7740 1 Processor -- start 2024-09-16T00:41:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa0a80f7740 1 -- start start 2024-09-16T00:41:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee8080 0x55a33ff0a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fef2940 0x55a33ff0ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01af530 con 0x55a33fee7760 2024-09-16T00:41:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01ace00 con 0x55a33fee8080 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01aac50 con 0x55a33fef2940 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48746/0 (socket says 172.21.15.1:48746) 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.757+0000 7fa09f7fe640 1 -- 172.21.15.1:0/3853195089 learned_addr learned my addr 172.21.15.1:0/3853195089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa09effd640 1 --2- 172.21.15.1:0/3853195089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee8080 0x55a33ff0a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa09f7fe640 1 -- 172.21.15.1:0/3853195089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fef2940 msgr2=0x55a33ff0ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/3853195089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fef2940 0x55a33ff0ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa09f7fe640 1 -- 172.21.15.1:0/3853195089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee8080 msgr2=0x55a33ff0a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/3853195089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee8080 0x55a33ff0a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa09f7fe640 1 -- 172.21.15.1:0/3853195089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33fedf260 con 0x55a33fee7760 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/3853195089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feee100 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fa09002ae70 tx=0x7fa0900c1490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc082927876ef093 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa0857fa640 1 -- 172.21.15.1:0/3853195089 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0900c5740 con 0x55a33fee7760 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa0857fa640 1 -- 172.21.15.1:0/3853195089 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0900dbba0 con 0x55a33fee7760 2024-09-16T00:41:57.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.758+0000 7fa0857fa640 1 -- 172.21.15.1:0/3853195089 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09009bdf0 con 0x55a33fee7760 2024-09-16T00:41:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3853195089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 msgr2=0x55a33feee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3853195089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feee100 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fa09002ae70 tx=0x7fa0900c1490 comp rx=0 tx=0).stop 2024-09-16T00:41:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3853195089 shutdown_connections 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3853195089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fef2940 0x55a33ff0ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3853195089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fee8080 0x55a33ff0a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3853195089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee7760 0x55a33feee100 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3853195089 >> 172.21.15.1:0/3853195089 conn(0x55a33fee3cf0 msgr2=0x55a33fef18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3853195089 shutdown_connections 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3853195089 wait complete. 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 -- start start 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 Processor -- start 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee7760 0x55a33fefa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fefae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01ab2c0 con 0x55a33fee8080 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01ac730 con 0x55a33fef2940 2024-09-16T00:41:57.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01ab990 con 0x55a33fee7760 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.759+0000 7fa0a80f7740 10 client.0 _ll_register_callbacks cb 0x7fa0900a0a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fefae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fefae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48750/0 (socket says 172.21.15.1:48750) 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09effd640 1 -- 172.21.15.1:0/3038533631 learned_addr learned my addr 172.21.15.1:0/3038533631 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09ffff640 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee7760 0x55a33fefa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09effd640 1 -- 172.21.15.1:0/3038533631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee7760 msgr2=0x55a33fefa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09effd640 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee7760 0x55a33fefa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09effd640 1 -- 172.21.15.1:0/3038533631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 msgr2=0x55a33fef78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09effd640 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09effd640 1 -- 172.21.15.1:0/3038533631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33fef7e20 con 0x55a33fee8080 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09ffff640 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:57.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa09effd640 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fefae20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x55a33fee4b50 tx=0x7fa09400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a85fa50b9fcbcf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa07effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa07effd640 1 -- 172.21.15.1:0/3038533631 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0940035f0 con 0x55a33fee8080 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3038533631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0900dbba0 con 0x55a33fee8080 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.760+0000 7fa07effd640 1 -- 172.21.15.1:0/3038533631 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa094021040 con 0x55a33fee8080 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa07effd640 1 -- 172.21.15.1:0/3038533631 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa09400dde0 con 0x55a33fee8080 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa0a80f7740 10 client.18645 fetch_fsmap learned FSMap version 25 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3038533631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a33ff6de80 con 0x55a33fee8080 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa07effd640 1 -- 172.21.15.1:0/3038533631 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0900dbba0 con 0x55a33fee8080 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa07effd640 1 -- 172.21.15.1:0/3038533631 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa094005590 con 0x55a33fee8080 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa0a80f7740 10 client.18645 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa0a80f7740 10 client.18645 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:41:57.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa0a80f7740 20 client.18645 populate_metadata read hostname 'smithi001' 2024-09-16T00:41:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a33ff6e440 0x55a33ff70860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa0a80f7740 4 client.18645 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}] 2024-09-16T00:41:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3038533631 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}) -- 0x55a33fe54d20 con 0x55a33ff6e440 2024-09-16T00:41:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.761+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a33ff6e440 0x55a33ff70860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.762+0000 7fa09f7fe640 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a33ff6e440 0x55a33ff70860 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d0c851badef66e server_cookie=da1f2aeefffe2650 in_seq=0 out_seq=0 2024-09-16T00:41:57.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.762+0000 7fa07effd640 10 client.18645 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa07effd640 1 -- 172.21.15.1:0/3038533631 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55a33fe54d20 con 0x55a33ff6e440 2024-09-16T00:41:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa07effd640 10 client.18645 handle_command_reply: tid=0 2024-09-16T00:41:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 client.18645 shutdown 2024-09-16T00:41:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3038533631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 msgr2=0x55a33fefae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fefae20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x55a33fee4b50 tx=0x7fa09400b950 comp rx=0 tx=0).stop 2024-09-16T00:41:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3038533631 shutdown_connections 2024-09-16T00:41:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a33ff6e440 0x55a33ff70860 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a33fef2940 0x55a33fef78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a33fee8080 0x55a33fefae20 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 --2- 172.21.15.1:0/3038533631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a33fee7760 0x55a33fefa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3038533631 >> 172.21.15.1:0/3038533631 conn(0x55a33fee3cf0 msgr2=0x55a33fee2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa07effd640 0 client.18645 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:41:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3038533631 shutdown_connections 2024-09-16T00:41:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.770+0000 7fa0a80f7740 1 -- 172.21.15.1:0/3038533631 wait complete. 2024-09-16T00:41:57.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a80f7740 20 client.18645 trim_cache size 0 max 16384 2024-09-16T00:41:57.772 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:41:57.772 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 46, 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "89c009e": { 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 46, 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 131.4, 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:41:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:41:57.774 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0880a3240 msgr2=0x7fa0880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0880a3240 0x7fa0880a5700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa09800a900 tx=0x7fa09800ad00 comp rx=0 tx=0).stop 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 msgr2=0x7fa0a01984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 0x7fa0a01984b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fa090005cc0 tx=0x7fa09002f630 comp rx=0 tx=0).stop 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/1907356334 shutdown_connections 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa0880a3240 0x7fa0880a5700 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0a010c180 0x7fa0a019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0103700 0x7fa0a01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 --2- 172.21.15.1:0/1907356334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0a0102de0 0x7fa0a01984b0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:57.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/1907356334 >> 172.21.15.1:0/1907356334 conn(0x7fa0a00fc7a0 msgr2=0x7fa0a00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:57.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/1907356334 shutdown_connections 2024-09-16T00:41:57.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:41:57.771+0000 7fa0a5d1e640 1 -- 172.21.15.1:0/1907356334 wait complete. 2024-09-16T00:41:58.558 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '89c009e', quiesce took 3.4 seconds. Will release after: 8.0 2024-09-16T00:41:58.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:41:58 smithi001 ceph-mon[30934]: pgmap v757: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:58.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:41:58 smithi153 ceph-mon[35538]: pgmap v757: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:41:58.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:41:58 smithi019 ceph-mon[35186]: pgmap v757: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:42:00.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:00 smithi001 ceph-mon[30934]: pgmap v758: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:42:00.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:00 smithi153 ceph-mon[35538]: pgmap v758: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:42:00.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:00 smithi019 ceph-mon[35186]: pgmap v758: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:42:02.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:02 smithi001 ceph-mon[30934]: pgmap v759: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:42:02.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:02 smithi153 ceph-mon[35538]: pgmap v759: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:42:02.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:02 smithi019 ceph-mon[35186]: pgmap v759: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:42:04.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:04 smithi001 ceph-mon[30934]: pgmap v760: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-16T00:42:04.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:04 smithi153 ceph-mon[35538]: pgmap v760: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-16T00:42:04.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:04 smithi019 ceph-mon[35186]: pgmap v760: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-16T00:42:06.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:42:06.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:06 smithi001 ceph-mon[30934]: pgmap v761: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-09-16T00:42:06.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:06 smithi153 ceph-mon[35538]: pgmap v761: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-09-16T00:42:06.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:06 smithi019 ceph-mon[35186]: pgmap v761: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-09-16T00:42:06.857 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.480+0000 7f70d81a4640 1 -- 172.21.15.1:0/770054479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d010e3c0 msgr2=0x7f70d0110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.480+0000 7f70d81a4640 1 --2- 172.21.15.1:0/770054479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d010e3c0 0x7f70d0110810 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009ca0 tx=0x7f70c802a950 comp rx=0 tx=0).stop 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.480+0000 7f70d81a4640 1 -- 172.21.15.1:0/770054479 shutdown_connections 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.480+0000 7f70d81a4640 1 --2- 172.21.15.1:0/770054479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d010e3c0 0x7f70d0110810 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.480+0000 7f70d81a4640 1 --2- 172.21.15.1:0/770054479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0074f40 0x7f70d010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.480+0000 7f70d81a4640 1 --2- 172.21.15.1:0/770054479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d0074620 0x7f70d0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.480+0000 7f70d81a4640 1 -- 172.21.15.1:0/770054479 >> 172.21.15.1:0/770054479 conn(0x7f70d00fd8c0 msgr2=0x7f70d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.480+0000 7f70d81a4640 1 -- 172.21.15.1:0/770054479 shutdown_connections 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.480+0000 7f70d81a4640 1 -- 172.21.15.1:0/770054479 wait complete. 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d81a4640 1 Processor -- start 2024-09-16T00:42:07.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d81a4640 1 -- start start 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d81a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0074620 0x7f70d019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d81a4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0074f40 0x7f70d019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d81a4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d010e3c0 0x7f70d019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d81a4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0114150 con 0x7f70d0074f40 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d81a4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0113fd0 con 0x7f70d0074620 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d81a4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01142d0 con 0x7f70d010e3c0 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d573f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0074f40 0x7f70d019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d573f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0074f40 0x7f70d019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50148/0 (socket says 172.21.15.1:50148) 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d573f640 1 -- 172.21.15.1:0/2054966038 learned_addr learned my addr 172.21.15.1:0/2054966038 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.481+0000 7f70d5f40640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0074620 0x7f70d019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d573f640 1 -- 172.21.15.1:0/2054966038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d010e3c0 msgr2=0x7f70d019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d6741640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d010e3c0 0x7f70d019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:07.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d573f640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d010e3c0 0x7f70d019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d573f640 1 -- 172.21.15.1:0/2054966038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0074620 msgr2=0x7f70d019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d573f640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0074620 0x7f70d019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d573f640 1 -- 172.21.15.1:0/2054966038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d019f5d0 con 0x7f70d0074f40 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d5f40640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0074620 0x7f70d019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d573f640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0074f40 0x7f70d019ab60 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c890 tx=0x7f70c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70c6ffd640 1 -- 172.21.15.1:0/2054966038 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0004220 con 0x7f70d0074f40 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70c6ffd640 1 -- 172.21.15.1:0/2054966038 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c00043c0 con 0x7f70d0074f40 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d019f8c0 con 0x7f70d0074f40 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70c6ffd640 1 -- 172.21.15.1:0/2054966038 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00049e0 con 0x7f70d0074f40 2024-09-16T00:42:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.482+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d01a71a0 con 0x7f70d0074f40 2024-09-16T00:42:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.483+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d0074a00 con 0x7f70d0074f40 2024-09-16T00:42:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.484+0000 7f70c6ffd640 1 -- 172.21.15.1:0/2054966038 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f70c00079b0 con 0x7f70d0074f40 2024-09-16T00:42:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.485+0000 7f70c6ffd640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70bc0a3240 0x7f70bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.485+0000 7f70c6ffd640 1 -- 172.21.15.1:0/2054966038 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f70c00cfdc0 con 0x7f70d0074f40 2024-09-16T00:42:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.485+0000 7f70d5f40640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70bc0a3240 0x7f70bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.486+0000 7f70d5f40640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70bc0a3240 0x7f70bc0a5700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0058c0 tx=0x7f70cc005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:07.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.489+0000 7f70c6ffd640 1 -- 172.21.15.1:0/2054966038 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70c0014030 con 0x7f70d0074f40 2024-09-16T00:42:07.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.699+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70d002d7c0 con 0x7f70d0074f40 2024-09-16T00:42:07.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.700+0000 7f70c6ffd640 1 -- 172.21.15.1:0/2054966038 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f70c009c6e0 con 0x7f70d0074f40 2024-09-16T00:42:07.701 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:42:07.701 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:42:07.701 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:42:07.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70bc0a3240 msgr2=0x7f70bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:07.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70bc0a3240 0x7f70bc0a5700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0058c0 tx=0x7f70cc005ec0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0074f40 msgr2=0x7f70d019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:07.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0074f40 0x7f70d019ab60 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c890 tx=0x7f70c000cd50 comp rx=0 tx=0).stop 2024-09-16T00:42:07.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 shutdown_connections 2024-09-16T00:42:07.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f70bc0a3240 0x7f70bc0a5700 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70d010e3c0 0x7f70d019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70d0074f40 0x7f70d019ab60 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 --2- 172.21.15.1:0/2054966038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d0074620 0x7f70d019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:07.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.702+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 >> 172.21.15.1:0/2054966038 conn(0x7f70d00fd8c0 msgr2=0x7f70d00fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:07.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.703+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 shutdown_connections 2024-09-16T00:42:07.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:07.703+0000 7f70d81a4640 1 -- 172.21.15.1:0/2054966038 wait complete. 2024-09-16T00:42:08.486 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 89c009e --release --await' 2024-09-16T00:42:08.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 89c009e --release --await 2024-09-16T00:42:08.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:08 smithi001 ceph-mon[30934]: pgmap v762: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-16T00:42:08.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:08 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2054966038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:08.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:08 smithi153 ceph-mon[35538]: pgmap v762: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-16T00:42:08.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:08 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2054966038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:08.747 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:08 smithi019 ceph-mon[35186]: pgmap v762: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-16T00:42:08.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:08 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2054966038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.237+0000 7f39a616a640 1 -- 172.21.15.1:0/4122365942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a0103660 msgr2=0x7f39a010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.237+0000 7f39a616a640 1 --2- 172.21.15.1:0/4122365942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a0103660 0x7f39a010bb50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f3994009900 tx=0x7f399402a480 comp rx=0 tx=0).stop 2024-09-16T00:42:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.238+0000 7f39a616a640 1 -- 172.21.15.1:0/4122365942 shutdown_connections 2024-09-16T00:42:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.238+0000 7f39a616a640 1 --2- 172.21.15.1:0/4122365942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a010c170 0x7f39a010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.238+0000 7f39a616a640 1 --2- 172.21.15.1:0/4122365942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a0103660 0x7f39a010bb50 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.238+0000 7f39a616a640 1 --2- 172.21.15.1:0/4122365942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0102d40 0x7f39a0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.238+0000 7f39a616a640 1 -- 172.21.15.1:0/4122365942 >> 172.21.15.1:0/4122365942 conn(0x7f39a00fc7a0 msgr2=0x7f39a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.238+0000 7f39a616a640 1 -- 172.21.15.1:0/4122365942 shutdown_connections 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.238+0000 7f39a616a640 1 -- 172.21.15.1:0/4122365942 wait complete. 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.238+0000 7f39a616a640 1 Processor -- start 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.238+0000 7f39a616a640 1 -- start start 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f39a616a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102d40 0x7f39a01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f39a616a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0103660 0x7f39a0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f39a616a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a010c170 0x7f39a019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f39a616a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f39a0111f70 con 0x7f39a010c170 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f39a616a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39a0111df0 con 0x7f39a0103660 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f39a616a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01120f0 con 0x7f39a0102d40 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a010c170 0x7f39a019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a010c170 0x7f39a019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50160/0 (socket says 172.21.15.1:50160) 2024-09-16T00:42:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399ffff640 1 -- 172.21.15.1:0/3744090056 learned_addr learned my addr 172.21.15.1:0/3744090056 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399effd640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0103660 0x7f39a0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399f7fe640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102d40 0x7f39a01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399ffff640 1 -- 172.21.15.1:0/3744090056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102d40 msgr2=0x7f39a01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399ffff640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102d40 0x7f39a01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399ffff640 1 -- 172.21.15.1:0/3744090056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0103660 msgr2=0x7f39a0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399ffff640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0103660 0x7f39a0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.239+0000 7f399ffff640 1 -- 172.21.15.1:0/3744090056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a019d3a0 con 0x7f39a010c170 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.240+0000 7f399ffff640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a010c170 0x7f39a019ccc0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f399800c970 tx=0x7f399800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.240+0000 7f399cff9640 1 -- 172.21.15.1:0/3744090056 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998007bb0 con 0x7f39a010c170 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.240+0000 7f39a616a640 1 -- 172.21.15.1:0/3744090056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a019d690 con 0x7f39a010c170 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.240+0000 7f399cff9640 1 -- 172.21.15.1:0/3744090056 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3998007d50 con 0x7f39a010c170 2024-09-16T00:42:09.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.240+0000 7f399cff9640 1 -- 172.21.15.1:0/3744090056 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998018800 con 0x7f39a010c170 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.240+0000 7f39a616a640 1 -- 172.21.15.1:0/3744090056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a01a4f70 con 0x7f39a010c170 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.242+0000 7f399cff9640 1 -- 172.21.15.1:0/3744090056 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f399800fa50 con 0x7f39a010c170 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.242+0000 7f39a8543740 1 Processor -- start 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.242+0000 7f39a8543740 1 -- start start 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f39a8543740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c84f60 0x561d51c85340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f39a8543740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c85880 0x561d51c94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f39a8543740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c94650 0x561d51c96a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f39a8543740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561d51bea620 con 0x561d51c85880 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f39a8543740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d51b91e50 con 0x561d51c94650 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f39a8543740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d51be65e0 con 0x561d51c84f60 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c85880 0x561d51c94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c85880 0x561d51c94110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50168/0 (socket says 172.21.15.1:50168) 2024-09-16T00:42:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399f7fe640 1 -- 172.21.15.1:0/2547603049 learned_addr learned my addr 172.21.15.1:0/2547603049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399cff9640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f39880a3240 0x7f39880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399effd640 1 --2- 172.21.15.1:0/2547603049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c84f60 0x561d51c85340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399effd640 1 --2- 172.21.15.1:0/2547603049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c94650 0x561d51c96a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399ffff640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f39880a3240 0x7f39880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399f7fe640 1 -- 172.21.15.1:0/2547603049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c84f60 msgr2=0x561d51c85340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399f7fe640 1 --2- 172.21.15.1:0/2547603049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c84f60 0x561d51c85340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399f7fe640 1 -- 172.21.15.1:0/2547603049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c94650 msgr2=0x561d51c96a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399f7fe640 1 --2- 172.21.15.1:0/2547603049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c94650 0x561d51c96a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399f7fe640 1 -- 172.21.15.1:0/2547603049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d51b82d40 con 0x561d51c85880 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399cff9640 1 -- 172.21.15.1:0/3744090056 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f39980cfc60 con 0x7f39a010c170 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.243+0000 7f399f7fe640 1 --2- 172.21.15.1:0/2547603049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c85880 0x561d51c94110 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f399000af40 tx=0x7f399000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb28276105c5c863 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f3985ffb640 1 -- 172.21.15.1:0/2547603049 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399001d070 con 0x561d51c85880 2024-09-16T00:42:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f399ffff640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f39880a3240 0x7f39880a5700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f399809bb80 tx=0x7f3998091360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f3985ffb640 1 -- 172.21.15.1:0/2547603049 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3990010de0 con 0x561d51c85880 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f3985ffb640 1 -- 172.21.15.1:0/2547603049 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990016340 con 0x561d51c85880 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f39a8543740 1 -- 172.21.15.1:0/2547603049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c85880 msgr2=0x561d51c94110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f39a8543740 1 --2- 172.21.15.1:0/2547603049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c85880 0x561d51c94110 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f399000af40 tx=0x7f399000f850 comp rx=0 tx=0).stop 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f39a8543740 1 -- 172.21.15.1:0/2547603049 shutdown_connections 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f39a8543740 1 --2- 172.21.15.1:0/2547603049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c94650 0x561d51c96a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f39a8543740 1 --2- 172.21.15.1:0/2547603049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c85880 0x561d51c94110 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f39a8543740 1 --2- 172.21.15.1:0/2547603049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c84f60 0x561d51c85340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f39a8543740 1 -- 172.21.15.1:0/2547603049 >> 172.21.15.1:0/2547603049 conn(0x561d51c815f0 msgr2=0x561d51c819d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f39a8543740 1 -- 172.21.15.1:0/2547603049 shutdown_connections 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.244+0000 7f39a8543740 1 -- 172.21.15.1:0/2547603049 wait complete. 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f39a8543740 1 -- start start 2024-09-16T00:42:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f39a8543740 1 Processor -- start 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f39a8543740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 0x561d51c91580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f39a8543740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51c91ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f39a8543740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51c92000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f39a8543740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561d51c97280 con 0x561d51c94650 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f39a8543740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d51b72f10 con 0x561d51c84f60 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f39a8543740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d51c97400 con 0x561d51c85880 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f39a8543740 10 client.0 _ll_register_callbacks cb 0x561d51b96db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f399effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51c92000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f399effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51c92000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50174/0 (socket says 172.21.15.1:50174) 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f399effd640 1 -- 172.21.15.1:0/281453698 learned_addr learned my addr 172.21.15.1:0/281453698 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.245+0000 7f399effd640 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 0x561d51c91580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f399f7fe640 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51c91ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f399effd640 1 -- 172.21.15.1:0/281453698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 msgr2=0x561d51c91ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f399effd640 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51c91ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f399effd640 1 -- 172.21.15.1:0/281453698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 msgr2=0x561d51c91580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f399effd640 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 0x561d51c91580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f399effd640 1 -- 172.21.15.1:0/281453698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d518f9580 con 0x561d51c94650 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f399effd640 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51c92000 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f3994033000 tx=0x7f3994009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100c27615baa98cc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f397effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f397effd640 1 -- 172.21.15.1:0/281453698 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994035070 con 0x561d51c94650 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f397effd640 1 -- 172.21.15.1:0/281453698 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3994004250 con 0x561d51c94650 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f39a8543740 1 -- 172.21.15.1:0/281453698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3990010de0 con 0x561d51c94650 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f397effd640 1 -- 172.21.15.1:0/281453698 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3994054050 con 0x561d51c94650 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f39a8543740 10 client.18663 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f39a8543740 1 -- 172.21.15.1:0/281453698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d519c3960 con 0x561d51c94650 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.246+0000 7f397effd640 1 -- 172.21.15.1:0/281453698 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3990010de0 con 0x561d51c94650 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f397effd640 1 -- 172.21.15.1:0/281453698 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3994049020 con 0x561d51c94650 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f39a8543740 10 client.18663 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f39a8543740 10 client.18663 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f39a8543740 20 client.18663 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f39a8543740 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d51ca42b0 0x561d51ca66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f39a8543740 4 client.18663 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f39a8543740 1 -- 172.21.15.1:0/281453698 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d51bf7b90 con 0x561d51ca42b0 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f399f7fe640 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d51ca42b0 0x561d51ca66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f399f7fe640 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d51ca42b0 0x561d51ca66a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db77525effa7de11 server_cookie=ee83b81fd94f600b in_seq=0 out_seq=0 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.247+0000 7f397effd640 10 client.18663 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f397effd640 1 -- 172.21.15.1:0/281453698 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d51bf7b90 con 0x561d51ca42b0 2024-09-16T00:42:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f397effd640 10 client.18663 handle_command_reply: tid=0 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 client.18663 shutdown 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 -- 172.21.15.1:0/281453698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 msgr2=0x561d51c92000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51c92000 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f3994033000 tx=0x7f3994009260 comp rx=0 tx=0).stop 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 -- 172.21.15.1:0/281453698 shutdown_connections 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d51ca42b0 0x561d51ca66a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51c92000 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51c91ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 --2- 172.21.15.1:0/281453698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 0x561d51c91580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f397effd640 0 client.18663 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 -- 172.21.15.1:0/281453698 >> 172.21.15.1:0/281453698 conn(0x561d51c815f0 msgr2=0x561d51c87320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 -- 172.21.15.1:0/281453698 shutdown_connections 2024-09-16T00:42:09.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.248+0000 7f39a8543740 1 -- 172.21.15.1:0/281453698 wait complete. 2024-09-16T00:42:09.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.249+0000 7f39a8543740 20 client.18663 trim_cache size 0 max 16384 2024-09-16T00:42:09.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f39a8543740 1 Processor -- start 2024-09-16T00:42:09.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f39a8543740 1 -- start start 2024-09-16T00:42:09.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f39a8543740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 0x561d51c871a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f39a8543740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51cabf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f39a8543740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51cac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f39a8543740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01af4c0 con 0x561d51c94650 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f39a8543740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01acd90 con 0x561d51c84f60 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f39a8543740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01aabe0 con 0x561d51c85880 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51cac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51cac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50178/0 (socket says 172.21.15.1:50178) 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 -- 172.21.15.1:0/2968352770 learned_addr learned my addr 172.21.15.1:0/2968352770 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 --2- 172.21.15.1:0/2968352770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 0x561d51c871a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399f7fe640 1 --2- 172.21.15.1:0/2968352770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51cabf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 -- 172.21.15.1:0/2968352770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 msgr2=0x561d51cabf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 --2- 172.21.15.1:0/2968352770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51cabf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 -- 172.21.15.1:0/2968352770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 msgr2=0x561d51c871a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 --2- 172.21.15.1:0/2968352770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 0x561d51c871a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 -- 172.21.15.1:0/2968352770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d51ca6a10 con 0x561d51c94650 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.266+0000 7f399effd640 1 --2- 172.21.15.1:0/2968352770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51cac4b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f3994031810 tx=0x7f399402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755f41b1ac2a9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39857fa640 1 -- 172.21.15.1:0/2968352770 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994035070 con 0x561d51c94650 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39857fa640 1 -- 172.21.15.1:0/2968352770 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f399403d760 con 0x561d51c94650 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39857fa640 1 -- 172.21.15.1:0/2968352770 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994051040 con 0x561d51c94650 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39a8543740 1 -- 172.21.15.1:0/2968352770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 msgr2=0x561d51cac4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39a8543740 1 --2- 172.21.15.1:0/2968352770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51cac4b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f3994031810 tx=0x7f399402a480 comp rx=0 tx=0).stop 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39a8543740 1 -- 172.21.15.1:0/2968352770 shutdown_connections 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39a8543740 1 --2- 172.21.15.1:0/2968352770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c94650 0x561d51cac4b0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39a8543740 1 --2- 172.21.15.1:0/2968352770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51cabf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39a8543740 1 --2- 172.21.15.1:0/2968352770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c84f60 0x561d51c871a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39a8543740 1 -- 172.21.15.1:0/2968352770 >> 172.21.15.1:0/2968352770 conn(0x561d51c815f0 msgr2=0x561d51c86a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39a8543740 1 -- 172.21.15.1:0/2968352770 shutdown_connections 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.267+0000 7f39a8543740 1 -- 172.21.15.1:0/2968352770 wait complete. 2024-09-16T00:42:09.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.268+0000 7f39a8543740 1 -- start start 2024-09-16T00:42:09.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.268+0000 7f39a8543740 1 Processor -- start 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.268+0000 7f39a8543740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c84f60 0x561d51c98bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.268+0000 7f39a8543740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51c99130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.268+0000 7f39a8543740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c94650 0x561d51d0df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.268+0000 7f39a8543740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ab250 con 0x561d51c84f60 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.268+0000 7f39a8543740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ac6c0 con 0x561d51c94650 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.268+0000 7f39a8543740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ab920 con 0x561d51c85880 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.268+0000 7f39a8543740 10 client.0 _ll_register_callbacks cb 0x7f39940318d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c84f60 0x561d51c98bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c84f60 0x561d51c98bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50188/0 (socket says 172.21.15.1:50188) 2024-09-16T00:42:09.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399effd640 1 -- 172.21.15.1:0/1306348974 learned_addr learned my addr 172.21.15.1:0/1306348974 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399f7fe640 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51c99130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399effd640 1 -- 172.21.15.1:0/1306348974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 msgr2=0x561d51c99130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399effd640 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51c99130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399effd640 1 -- 172.21.15.1:0/1306348974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c94650 msgr2=0x561d51d0df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399effd640 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c94650 0x561d51d0df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399effd640 1 -- 172.21.15.1:0/1306348974 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d51d0e580 con 0x561d51c84f60 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f399effd640 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c84f60 0x561d51c98bf0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f3994033000 tx=0x7f3994031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b718b23a906129f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f397effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f397effd640 1 -- 172.21.15.1:0/1306348974 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994035040 con 0x561d51c84f60 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f397effd640 1 -- 172.21.15.1:0/1306348974 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f399403dbb0 con 0x561d51c84f60 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f39a8543740 1 -- 172.21.15.1:0/1306348974 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f399403d760 con 0x561d51c84f60 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f397effd640 1 -- 172.21.15.1:0/1306348974 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3994067050 con 0x561d51c84f60 2024-09-16T00:42:09.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f39a8543740 10 client.18669 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.269+0000 7f39a8543740 1 -- 172.21.15.1:0/1306348974 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d51d0f880 con 0x561d51c84f60 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.270+0000 7f397effd640 1 -- 172.21.15.1:0/1306348974 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f399403d760 con 0x561d51c84f60 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.270+0000 7f397effd640 1 -- 172.21.15.1:0/1306348974 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3994049030 con 0x561d51c84f60 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.270+0000 7f39a8543740 10 client.18669 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.270+0000 7f39a8543740 10 client.18669 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.270+0000 7f39a8543740 20 client.18669 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.270+0000 7f39a8543740 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d51d0ff70 0x561d51d12390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.270+0000 7f39a8543740 4 client.18669 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "89c009e", "release": true, "await": true}] 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.270+0000 7f39a8543740 1 -- 172.21.15.1:0/1306348974 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "89c009e", "release": true, "await": true}) -- 0x561d51bf20a0 con 0x561d51d0ff70 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.270+0000 7f399f7fe640 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d51d0ff70 0x561d51d12390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.271+0000 7f399f7fe640 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d51d0ff70 0x561d51d12390 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc760db3632b433 server_cookie=1467d64d08adafd7 in_seq=0 out_seq=0 2024-09-16T00:42:09.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.271+0000 7f397effd640 10 client.18669 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:09.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.273+0000 7f397effd640 1 -- 172.21.15.1:0/1306348974 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561d51bf20a0 con 0x561d51d0ff70 2024-09-16T00:42:09.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.273+0000 7f397effd640 10 client.18669 handle_command_reply: tid=0 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.273+0000 7f39a8543740 1 client.18669 shutdown 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.273+0000 7f39a8543740 1 -- 172.21.15.1:0/1306348974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c84f60 msgr2=0x561d51c98bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.273+0000 7f39a8543740 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c84f60 0x561d51c98bf0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f3994033000 tx=0x7f3994031980 comp rx=0 tx=0).stop 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.273+0000 7f39a8543740 1 -- 172.21.15.1:0/1306348974 shutdown_connections 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.273+0000 7f39a8543740 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561d51d0ff70 0x561d51d12390 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f397effd640 0 client.18669 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.273+0000 7f39a8543740 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d51c94650 0x561d51d0df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a8543740 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d51c85880 0x561d51c99130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a8543740 1 --2- 172.21.15.1:0/1306348974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561d51c84f60 0x561d51c98bf0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a8543740 1 -- 172.21.15.1:0/1306348974 >> 172.21.15.1:0/1306348974 conn(0x561d51c815f0 msgr2=0x561d51c872b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a8543740 1 -- 172.21.15.1:0/1306348974 shutdown_connections 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a8543740 1 -- 172.21.15.1:0/1306348974 wait complete. 2024-09-16T00:42:09.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a8543740 20 client.18669 trim_cache size 0 max 16384 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 48, 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout: "89c009e": { 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 48, 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:42:09.278 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 131.4, 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:09.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:09.280 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:09.280 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:09.280 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:42:09.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 -- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f39880a3240 msgr2=0x7f39880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:09.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f39880a3240 0x7f39880a5700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f399809bb80 tx=0x7f3998091360 comp rx=0 tx=0).stop 2024-09-16T00:42:09.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 -- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a010c170 msgr2=0x7f39a019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:09.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a010c170 0x7f39a019ccc0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f399800c970 tx=0x7f399800ce30 comp rx=0 tx=0).stop 2024-09-16T00:42:09.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 -- 172.21.15.1:0/3744090056 shutdown_connections 2024-09-16T00:42:09.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39a010c170 0x7f39a019ccc0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0103660 0x7f39a0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f39880a3240 0x7f39880a5700 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 --2- 172.21.15.1:0/3744090056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102d40 0x7f39a01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:09.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.274+0000 7f39a616a640 1 -- 172.21.15.1:0/3744090056 >> 172.21.15.1:0/3744090056 conn(0x7f39a00fc7a0 msgr2=0x7f39a00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:09.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.275+0000 7f39a616a640 1 -- 172.21.15.1:0/3744090056 shutdown_connections 2024-09-16T00:42:09.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:09.275+0000 7f39a616a640 1 -- 172.21.15.1:0/3744090056 wait complete. 2024-09-16T00:42:10.054 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '89c009e', total seconds elapsed: 14.9 2024-09-16T00:42:10.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:10 smithi001 ceph-mon[30934]: pgmap v763: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:42:10.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:10 smithi153 ceph-mon[35538]: pgmap v763: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:42:10.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:10 smithi019 ceph-mon[35186]: pgmap v763: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-16T00:42:12.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:12 smithi001 ceph-mon[30934]: pgmap v764: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-16T00:42:12.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:12 smithi153 ceph-mon[35538]: pgmap v764: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-16T00:42:12.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:12 smithi019 ceph-mon[35186]: pgmap v764: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-16T00:42:14.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:14 smithi001 ceph-mon[30934]: pgmap v765: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T00:42:14.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:14 smithi153 ceph-mon[35538]: pgmap v765: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T00:42:14.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:14 smithi019 ceph-mon[35186]: pgmap v765: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T00:42:16.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:16 smithi001 ceph-mon[30934]: pgmap v766: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-16T00:42:16.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:16 smithi153 ceph-mon[35538]: pgmap v766: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-16T00:42:16.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:16 smithi019 ceph-mon[35186]: pgmap v766: 129 pgs: 129 active+clean; 6.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-16T00:42:18.623 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:42:18.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:18 smithi001 ceph-mon[30934]: pgmap v767: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:42:18.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:18 smithi153 ceph-mon[35538]: pgmap v767: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:42:18.856 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:18 smithi019 ceph-mon[35186]: pgmap v767: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:42:18.911 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.439+0000 7f60107f6640 1 -- 172.21.15.1:0/2844032609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008106f10 msgr2=0x7f600810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.439+0000 7f60107f6640 1 --2- 172.21.15.1:0/2844032609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008106f10 0x7f600810ea90 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f6000009ca0 tx=0x7f600002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 -- 172.21.15.1:0/2844032609 shutdown_connections 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 --2- 172.21.15.1:0/2844032609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008106f10 0x7f600810ea90 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 --2- 172.21.15.1:0/2844032609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6008102790 0x7f60081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 --2- 172.21.15.1:0/2844032609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6008101de0 0x7f60081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 -- 172.21.15.1:0/2844032609 >> 172.21.15.1:0/2844032609 conn(0x7f60080fd8c0 msgr2=0x7f60080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 -- 172.21.15.1:0/2844032609 shutdown_connections 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 -- 172.21.15.1:0/2844032609 wait complete. 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 Processor -- start 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 -- start start 2024-09-16T00:42:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008101de0 0x7f600819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6008102790 0x7f600819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.440+0000 7f60107f6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6008106f10 0x7f600819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f60107f6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6008111f70 con 0x7f6008101de0 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f60107f6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6008111df0 con 0x7f6008102790 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f60107f6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60081120f0 con 0x7f6008106f10 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600e592640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008101de0 0x7f600819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600e592640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008101de0 0x7f600819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51114/0 (socket says 172.21.15.1:51114) 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600e592640 1 -- 172.21.15.1:0/1791440891 learned_addr learned my addr 172.21.15.1:0/1791440891 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600dd91640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6008102790 0x7f600819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600ed93640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6008106f10 0x7f600819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600e592640 1 -- 172.21.15.1:0/1791440891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6008106f10 msgr2=0x7f600819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600e592640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6008106f10 0x7f600819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600e592640 1 -- 172.21.15.1:0/1791440891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6008102790 msgr2=0x7f600819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:19.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600e592640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6008102790 0x7f600819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600e592640 1 -- 172.21.15.1:0/1791440891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600819f580 con 0x7f6008101de0 2024-09-16T00:42:19.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f600e592640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008101de0 0x7f600819a5d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f600400e9f0 tx=0x7f600400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:19.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f5fff7fe640 1 -- 172.21.15.1:0/1791440891 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600400cd80 con 0x7f6008101de0 2024-09-16T00:42:19.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f5fff7fe640 1 -- 172.21.15.1:0/1791440891 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6004010910 con 0x7f6008101de0 2024-09-16T00:42:19.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f60107f6640 1 -- 172.21.15.1:0/1791440891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600819f870 con 0x7f6008101de0 2024-09-16T00:42:19.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.441+0000 7f5fff7fe640 1 -- 172.21.15.1:0/1791440891 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004018990 con 0x7f6008101de0 2024-09-16T00:42:19.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.442+0000 7f60107f6640 1 -- 172.21.15.1:0/1791440891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60081a7150 con 0x7f6008101de0 2024-09-16T00:42:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.444+0000 7f5fff7fe640 1 -- 172.21.15.1:0/1791440891 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6004010ac0 con 0x7f6008101de0 2024-09-16T00:42:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.445+0000 7f6012bcf740 1 Processor -- start 2024-09-16T00:42:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.445+0000 7f6012bcf740 1 -- start start 2024-09-16T00:42:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.445+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512976880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.445+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512976dc0 0x560512981140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.445+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512981680 0x560512983a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.445+0000 7f6012bcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5605128dc2e0 con 0x560512976dc0 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.445+0000 7f6012bcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560512883ce0 con 0x5605129764a0 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.445+0000 7f6012bcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5605128d82a0 con 0x560512981680 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.445+0000 7f5fff7fe640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ff40a3240 0x7f5ff40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600e592640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512976dc0 0x560512981140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512976880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512976880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:33564/0 (socket says 172.21.15.1:33564) 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 -- 172.21.15.1:0/1577735049 learned_addr learned my addr 172.21.15.1:0/1577735049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ff40a3240 0x7f5ff40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600dd91640 1 --2- 172.21.15.1:0/1577735049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512981680 0x560512983a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f5fff7fe640 1 -- 172.21.15.1:0/1791440891 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f60040d0080 con 0x7f6008101de0 2024-09-16T00:42:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 -- 172.21.15.1:0/1577735049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512981680 msgr2=0x560512983a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 --2- 172.21.15.1:0/1577735049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512981680 0x560512983a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 -- 172.21.15.1:0/1577735049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512976dc0 msgr2=0x560512981140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 --2- 172.21.15.1:0/1577735049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512976dc0 0x560512981140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 -- 172.21.15.1:0/1577735049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560512868440 con 0x5605129764a0 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600e592640 1 --2- 172.21.15.1:0/1577735049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512976dc0 0x560512981140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 --2- 172.21.15.1:0/1577735049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512976880 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6000005fe0 tx=0x7f6000031f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f124db5990f5c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f600ed93640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ff40a3240 0x7f5ff40a5700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6000004af0 tx=0x7f6000036000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.446+0000 7f5ffcff9640 1 -- 172.21.15.1:0/1577735049 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000056070 con 0x5605129764a0 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.447+0000 7f5ffcff9640 1 -- 172.21.15.1:0/1577735049 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6000036070 con 0x5605129764a0 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.447+0000 7f5ffcff9640 1 -- 172.21.15.1:0/1577735049 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000051450 con 0x5605129764a0 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.447+0000 7f6012bcf740 1 -- 172.21.15.1:0/1577735049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 msgr2=0x560512976880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.447+0000 7f6012bcf740 1 --2- 172.21.15.1:0/1577735049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512976880 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6000005fe0 tx=0x7f6000031f40 comp rx=0 tx=0).stop 2024-09-16T00:42:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.447+0000 7f6012bcf740 1 -- 172.21.15.1:0/1577735049 shutdown_connections 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.447+0000 7f6012bcf740 1 --2- 172.21.15.1:0/1577735049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512981680 0x560512983a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.447+0000 7f6012bcf740 1 --2- 172.21.15.1:0/1577735049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512976dc0 0x560512981140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.447+0000 7f6012bcf740 1 --2- 172.21.15.1:0/1577735049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512976880 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.447+0000 7f6012bcf740 1 -- 172.21.15.1:0/1577735049 >> 172.21.15.1:0/1577735049 conn(0x560512972a30 msgr2=0x560512972e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 -- 172.21.15.1:0/1577735049 shutdown_connections 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 -- 172.21.15.1:0/1577735049 wait complete. 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 -- start start 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 Processor -- start 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x560512981000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 0x56051297e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x56051297e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5605129840d0 con 0x560512981680 2024-09-16T00:42:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560512897050 con 0x560512976dc0 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560512984250 con 0x5605129764a0 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f6012bcf740 10 client.0 _ll_register_callbacks cb 0x5605128d7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.448+0000 7f600ed93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x56051297e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600ed93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x56051297e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51132/0 (socket says 172.21.15.1:51132) 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600ed93640 1 -- 172.21.15.1:0/3498633769 learned_addr learned my addr 172.21.15.1:0/3498633769 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600dd91640 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 0x56051297e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600e592640 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x560512981000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600ed93640 1 -- 172.21.15.1:0/3498633769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 msgr2=0x560512981000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600ed93640 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x560512981000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600ed93640 1 -- 172.21.15.1:0/3498633769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 msgr2=0x56051297e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600ed93640 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 0x56051297e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600ed93640 1 -- 172.21.15.1:0/3498633769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56051264d3c0 con 0x560512981680 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600e592640 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x560512981000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f600ed93640 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x56051297e830 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f6000009bf0 tx=0x7f6000030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e811dce0399a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f5ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3498633769 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000061060 con 0x560512981680 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3498633769 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f600002fe70 con 0x560512981680 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.449+0000 7f6012bcf740 1 -- 172.21.15.1:0/3498633769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6000036070 con 0x560512981680 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.450+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3498633769 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6000068050 con 0x560512981680 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.450+0000 7f6012bcf740 10 client.18681 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.450+0000 7f6012bcf740 1 -- 172.21.15.1:0/3498633769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5605129840d0 con 0x560512981680 2024-09-16T00:42:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.450+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3498633769 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6000036070 con 0x560512981680 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.451+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3498633769 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6000049400 con 0x560512981680 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.451+0000 7f6012bcf740 10 client.18681 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.451+0000 7f6012bcf740 10 client.18681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.451+0000 7f6012bcf740 20 client.18681 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.451+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560512990d30 0x560512993150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.451+0000 7f6012bcf740 4 client.18681 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.451+0000 7f6012bcf740 1 -- 172.21.15.1:0/3498633769 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605128e9850 con 0x560512990d30 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.451+0000 7f600e592640 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560512990d30 0x560512993150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.452+0000 7f600e592640 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560512990d30 0x560512993150 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f34b126fc0ee92 server_cookie=7ca0d0b1d1493189 in_seq=0 out_seq=0 2024-09-16T00:42:19.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.452+0000 7f5ff1ffb640 10 client.18681 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.452+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3498633769 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605128e9850 con 0x560512990d30 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.452+0000 7f5ff1ffb640 10 client.18681 handle_command_reply: tid=0 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.452+0000 7f6012bcf740 1 client.18681 shutdown 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.452+0000 7f6012bcf740 1 -- 172.21.15.1:0/3498633769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 msgr2=0x56051297e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.452+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x56051297e830 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f6000009bf0 tx=0x7f6000030cc0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f6012bcf740 1 -- 172.21.15.1:0/3498633769 shutdown_connections 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x560512990d30 0x560512993150 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x56051297e830 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 0x56051297e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f5ff1ffb640 0 client.18681 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3498633769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x560512981000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f6012bcf740 1 -- 172.21.15.1:0/3498633769 >> 172.21.15.1:0/3498633769 conn(0x560512972a30 msgr2=0x560512978760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:19.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f6012bcf740 1 -- 172.21.15.1:0/3498633769 shutdown_connections 2024-09-16T00:42:19.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f6012bcf740 1 -- 172.21.15.1:0/3498633769 wait complete. 2024-09-16T00:42:19.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.453+0000 7f6012bcf740 20 client.18681 trim_cache size 0 max 16384 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f6012bcf740 1 Processor -- start 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f6012bcf740 1 -- start start 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x56051298d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 0x560512998a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x560512998f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f6012bcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60081b3710 con 0x560512981680 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f6012bcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60081b0fe0 con 0x560512976dc0 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f6012bcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60081aeee0 con 0x5605129764a0 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f600ed93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x560512998f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f600ed93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x560512998f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51148/0 (socket says 172.21.15.1:51148) 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.470+0000 7f600ed93640 1 -- 172.21.15.1:0/478514937 learned_addr learned my addr 172.21.15.1:0/478514937 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:19.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600dd91640 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 0x560512998a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600e592640 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x56051298d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600ed93640 1 -- 172.21.15.1:0/478514937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 msgr2=0x56051298d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600ed93640 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x56051298d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600ed93640 1 -- 172.21.15.1:0/478514937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 msgr2=0x560512998a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600ed93640 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 0x560512998a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600ed93640 1 -- 172.21.15.1:0/478514937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605129724d0 con 0x560512981680 2024-09-16T00:42:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600dd91640 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 0x560512998a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600e592640 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x56051298d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f600ed93640 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x560512998f60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f6000009620 tx=0x7f60000099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb991f8f75856cc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f5ff3fff640 1 -- 172.21.15.1:0/478514937 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000061040 con 0x560512981680 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f5ff3fff640 1 -- 172.21.15.1:0/478514937 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60000309a0 con 0x560512981680 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.471+0000 7f5ff3fff640 1 -- 172.21.15.1:0/478514937 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000056070 con 0x560512981680 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 -- 172.21.15.1:0/478514937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 msgr2=0x560512998f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x560512998f60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f6000009620 tx=0x7f60000099e0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 -- 172.21.15.1:0/478514937 shutdown_connections 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x560512998f60 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560512976dc0 0x560512998a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 --2- 172.21.15.1:0/478514937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605129764a0 0x56051298d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 -- 172.21.15.1:0/478514937 >> 172.21.15.1:0/478514937 conn(0x560512972a30 msgr2=0x560512979850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 -- 172.21.15.1:0/478514937 shutdown_connections 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 -- 172.21.15.1:0/478514937 wait complete. 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.472+0000 7f6012bcf740 1 -- start start 2024-09-16T00:42:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f6012bcf740 1 Processor -- start 2024-09-16T00:42:19.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512989a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512976dc0 0x560512984da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f6012bcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x5605129852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f6012bcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60081af4a0 con 0x560512981680 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f6012bcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60081b0910 con 0x5605129764a0 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f6012bcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60081afb70 con 0x560512976dc0 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f6012bcf740 10 client.0 _ll_register_callbacks cb 0x7f6000054070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600ed93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x5605129852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600ed93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x5605129852e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51150/0 (socket says 172.21.15.1:51150) 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600ed93640 1 -- 172.21.15.1:0/3368427998 learned_addr learned my addr 172.21.15.1:0/3368427998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600e592640 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512989a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600dd91640 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512976dc0 0x560512984da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600ed93640 1 -- 172.21.15.1:0/3368427998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512976dc0 msgr2=0x560512984da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:19.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600ed93640 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512976dc0 0x560512984da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600ed93640 1 -- 172.21.15.1:0/3368427998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 msgr2=0x560512989a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600ed93640 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512989a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600ed93640 1 -- 172.21.15.1:0/3368427998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560512985980 con 0x560512981680 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.473+0000 7f600e592640 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512989a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.474+0000 7f600ed93640 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x5605129852e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f60000497a0 tx=0x7f60000496d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6dbd5a89a06846 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.474+0000 7f5ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.474+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3368427998 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000056040 con 0x560512981680 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.474+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3368427998 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6000049ba0 con 0x560512981680 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.474+0000 7f6012bcf740 1 -- 172.21.15.1:0/3368427998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60000309a0 con 0x560512981680 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.474+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3368427998 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6000077050 con 0x560512981680 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.474+0000 7f6012bcf740 10 client.18687 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.474+0000 7f6012bcf740 1 -- 172.21.15.1:0/3368427998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5605129fb7e0 con 0x560512981680 2024-09-16T00:42:19.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.474+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3368427998 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60000309a0 con 0x560512981680 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.475+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3368427998 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f600005c640 con 0x560512981680 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.475+0000 7f6012bcf740 10 client.18687 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.475+0000 7f6012bcf740 10 client.18687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.475+0000 7f6012bcf740 20 client.18687 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.475+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5605129fbdf0 0x5605129fe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.475+0000 7f6012bcf740 4 client.18687 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.475+0000 7f6012bcf740 1 -- 172.21.15.1:0/3368427998 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5605128e3d60 con 0x5605129fbdf0 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.475+0000 7f600e592640 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5605129fbdf0 0x5605129fe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f600e592640 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5605129fbdf0 0x5605129fe1e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f139167ef1074748 server_cookie=6fbd21bb14f02bea in_seq=0 out_seq=0 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f5ff1ffb640 10 client.18687 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f5ff1ffb640 1 -- 172.21.15.1:0/3368427998 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5605128e3d60 con 0x5605129fbdf0 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f5ff1ffb640 10 client.18687 handle_command_reply: tid=0 2024-09-16T00:42:19.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 client.18687 shutdown 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 -- 172.21.15.1:0/3368427998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 msgr2=0x5605129852e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x5605129852e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f60000497a0 tx=0x7f60000496d0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 -- 172.21.15.1:0/3368427998 shutdown_connections 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5605129fbdf0 0x5605129fe1e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560512981680 0x5605129852e0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560512976dc0 0x560512984da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f5ff1ffb640 0 client.18687 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 --2- 172.21.15.1:0/3368427998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605129764a0 0x560512989a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 -- 172.21.15.1:0/3368427998 >> 172.21.15.1:0/3368427998 conn(0x560512972a30 msgr2=0x56051297fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 -- 172.21.15.1:0/3368427998 shutdown_connections 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.476+0000 7f6012bcf740 1 -- 172.21.15.1:0/3368427998 wait complete. 2024-09-16T00:42:19.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f6012bcf740 20 client.18687 trim_cache size 0 max 16384 2024-09-16T00:42:19.482 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:42:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:42:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:42:19.482 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:42:19.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 -- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ff40a3240 msgr2=0x7f5ff40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ff40a3240 0x7f5ff40a5700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6000004af0 tx=0x7f6000036000 comp rx=0 tx=0).stop 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 -- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008101de0 msgr2=0x7f600819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008101de0 0x7f600819a5d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f600400e9f0 tx=0x7f600400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 -- 172.21.15.1:0/1791440891 shutdown_connections 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f5ff40a3240 0x7f5ff40a5700 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6008106f10 0x7f600819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6008102790 0x7f600819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 --2- 172.21.15.1:0/1791440891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6008101de0 0x7f600819a5d0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 -- 172.21.15.1:0/1791440891 >> 172.21.15.1:0/1791440891 conn(0x7f60080fd8c0 msgr2=0x7f60080ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 -- 172.21.15.1:0/1791440891 shutdown_connections 2024-09-16T00:42:19.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:19.477+0000 7f60107f6640 1 -- 172.21.15.1:0/1791440891 wait complete. 2024-09-16T00:42:20.328 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:42:20.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:42:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:20.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:20 smithi001 ceph-mon[30934]: pgmap v768: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-16T00:42:20.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:20 smithi153 ceph-mon[35538]: pgmap v768: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-16T00:42:20.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:20 smithi019 ceph-mon[35186]: pgmap v768: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-16T00:42:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.077+0000 7f455376f640 1 -- 172.21.15.1:0/238231348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 msgr2=0x7f454c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.077+0000 7f455376f640 1 --2- 172.21.15.1:0/238231348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 0x7f454c1067e0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f453c009900 tx=0x7f453c02a410 comp rx=0 tx=0).stop 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.077+0000 7f455376f640 1 -- 172.21.15.1:0/238231348 shutdown_connections 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.077+0000 7f455376f640 1 --2- 172.21.15.1:0/238231348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f454c106f10 0x7f454c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.077+0000 7f455376f640 1 --2- 172.21.15.1:0/238231348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 0x7f454c1067e0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.077+0000 7f455376f640 1 --2- 172.21.15.1:0/238231348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454c101de0 0x7f454c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.077+0000 7f455376f640 1 -- 172.21.15.1:0/238231348 >> 172.21.15.1:0/238231348 conn(0x7f454c0fd8c0 msgr2=0x7f454c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.077+0000 7f455376f640 1 -- 172.21.15.1:0/238231348 shutdown_connections 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.077+0000 7f455376f640 1 -- 172.21.15.1:0/238231348 wait complete. 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455376f640 1 Processor -- start 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455376f640 1 -- start start 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455376f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f454c101de0 0x7f454c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455376f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 0x7f454c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455376f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454c106f10 0x7f454c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455376f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f454c111f70 con 0x7f454c102790 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455376f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f454c111df0 con 0x7f454c106f10 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455376f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1120f0 con 0x7f454c101de0 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f4550d0a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 0x7f454c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f4550d0a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 0x7f454c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47934/0 (socket says 172.21.15.1:47934) 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f4550d0a640 1 -- 172.21.15.1:0/3449223532 learned_addr learned my addr 172.21.15.1:0/3449223532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455150b640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f454c101de0 0x7f454c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f4551d0c640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454c106f10 0x7f454c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f4550d0a640 1 -- 172.21.15.1:0/3449223532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f454c101de0 msgr2=0x7f454c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f4550d0a640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f454c101de0 0x7f454c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f4550d0a640 1 -- 172.21.15.1:0/3449223532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454c106f10 msgr2=0x7f454c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f4550d0a640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454c106f10 0x7f454c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f4550d0a640 1 -- 172.21.15.1:0/3449223532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454c1a6990 con 0x7f454c102790 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.079+0000 7f4551d0c640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454c106f10 0x7f454c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.078+0000 7f455150b640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f454c101de0 0x7f454c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.079+0000 7f4550d0a640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 0x7f454c074a30 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f453c006010 tx=0x7f453c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.079+0000 7f45427fc640 1 -- 172.21.15.1:0/3449223532 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c03f030 con 0x7f454c102790 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.079+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454c1a6c20 con 0x7f454c102790 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.079+0000 7f45427fc640 1 -- 172.21.15.1:0/3449223532 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f453c003ce0 con 0x7f454c102790 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.079+0000 7f45427fc640 1 -- 172.21.15.1:0/3449223532 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c035070 con 0x7f454c102790 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.079+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f454c1a7100 con 0x7f454c102790 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.080+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f454c1021c0 con 0x7f454c102790 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.082+0000 7f45427fc640 1 -- 172.21.15.1:0/3449223532 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f453c042030 con 0x7f454c102790 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.082+0000 7f45427fc640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45380a3240 0x7f45380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.083+0000 7f455150b640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45380a3240 0x7f45380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.083+0000 7f45427fc640 1 -- 172.21.15.1:0/3449223532 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f453c0f0680 con 0x7f454c102790 2024-09-16T00:42:21.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.083+0000 7f455150b640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45380a3240 0x7f45380a5700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f45480058c0 tx=0x7f4548005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:21.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.086+0000 7f45427fc640 1 -- 172.21.15.1:0/3449223532 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f453c0bd000 con 0x7f454c102790 2024-09-16T00:42:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.297+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f454c10a150 con 0x7f454c102790 2024-09-16T00:42:21.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.298+0000 7f45427fc640 1 -- 172.21.15.1:0/3449223532 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f453c0bc900 con 0x7f454c102790 2024-09-16T00:42:21.298 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:42:21.299 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:42:21.299 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:42:21.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45380a3240 msgr2=0x7f45380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:21.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45380a3240 0x7f45380a5700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f45480058c0 tx=0x7f4548005ec0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 msgr2=0x7f454c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 0x7f454c074a30 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f453c006010 tx=0x7f453c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 shutdown_connections 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454c106f10 0x7f454c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f454c102790 0x7f454c074a30 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45380a3240 0x7f45380a5700 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 --2- 172.21.15.1:0/3449223532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f454c101de0 0x7f454c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 >> 172.21.15.1:0/3449223532 conn(0x7f454c0fd8c0 msgr2=0x7f454c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 shutdown_connections 2024-09-16T00:42:21.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:21.300+0000 7f455376f640 1 -- 172.21.15.1:0/3449223532 wait complete. 2024-09-16T00:42:21.609 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:21 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3449223532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:21.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:21 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3449223532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:21.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:21 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3449223532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:21.939 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:42:22.197 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:22.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:22 smithi001 ceph-mon[30934]: pgmap v769: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:42:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.647+0000 7f6a5b9db640 1 -- 172.21.15.1:0/3465245926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a54074eb0 msgr2=0x7f6a54112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.647+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/3465245926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a54074eb0 0x7f6a54112650 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009900 tx=0x7f6a4402a480 comp rx=0 tx=0).stop 2024-09-16T00:42:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.647+0000 7f6a5b9db640 1 -- 172.21.15.1:0/3465245926 shutdown_connections 2024-09-16T00:42:22.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.647+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/3465245926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54112b90 0x7f6a54114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.647+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/3465245926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a54074eb0 0x7f6a54112650 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.647+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/3465245926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a54074590 0x7f6a54074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.647+0000 7f6a5b9db640 1 -- 172.21.15.1:0/3465245926 >> 172.21.15.1:0/3465245926 conn(0x7f6a540fd8c0 msgr2=0x7f6a540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:22.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.647+0000 7f6a5b9db640 1 -- 172.21.15.1:0/3465245926 shutdown_connections 2024-09-16T00:42:22.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.647+0000 7f6a5b9db640 1 -- 172.21.15.1:0/3465245926 wait complete. 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a5b9db640 1 Processor -- start 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a5b9db640 1 -- start start 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a5b9db640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a54074590 0x7f6a541a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a5b9db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54074eb0 0x7f6a541a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a5b9db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a54112b90 0x7f6a5419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a5b9db640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54118590 con 0x7f6a54112b90 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a5b9db640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54118410 con 0x7f6a54074eb0 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a5b9db640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54118710 con 0x7f6a54074590 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a59777640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a54074590 0x7f6a541a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a59777640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a54074590 0x7f6a541a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:38586/0 (socket says 172.21.15.1:38586) 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a59777640 1 -- 172.21.15.1:0/2862579398 learned_addr learned my addr 172.21.15.1:0/2862579398 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a58f76640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54074eb0 0x7f6a541a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.648+0000 7f6a59f78640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a54112b90 0x7f6a5419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a59777640 1 -- 172.21.15.1:0/2862579398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54074eb0 msgr2=0x7f6a541a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a59777640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54074eb0 0x7f6a541a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a59777640 1 -- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a54112b90 msgr2=0x7f6a5419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a59777640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a54112b90 0x7f6a5419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a59777640 1 -- 172.21.15.1:0/2862579398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5419efe0 con 0x7f6a54074590 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a59f78640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a54112b90 0x7f6a5419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a59777640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a54074590 0x7f6a541a46f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000d950 tx=0x7f6a5000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a4a7fc640 1 -- 172.21.15.1:0/2862579398 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50017070 con 0x7f6a54074590 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a5b9db640 1 -- 172.21.15.1:0/2862579398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5419f2d0 con 0x7f6a54074590 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a4a7fc640 1 -- 172.21.15.1:0/2862579398 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a50009c30 con 0x7f6a54074590 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a4a7fc640 1 -- 172.21.15.1:0/2862579398 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000f6b0 con 0x7f6a54074590 2024-09-16T00:42:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.649+0000 7f6a5b9db640 1 -- 172.21.15.1:0/2862579398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a541ab5c0 con 0x7f6a54074590 2024-09-16T00:42:22.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.652+0000 7f6a5ddb4740 1 Processor -- start 2024-09-16T00:42:22.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.652+0000 7f6a5ddb4740 1 -- start start 2024-09-16T00:42:22.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 0x56242da8e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242da8ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a4a7fc640 1 -- 172.21.15.1:0/2862579398 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6a500109d0 con 0x7f6a54074590 2024-09-16T00:42:22.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da9b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56242d9f42f0 con 0x56242da99370 2024-09-16T00:42:22.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56242d99bcf0 con 0x56242da8e5d0 2024-09-16T00:42:22.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56242d9f02b0 con 0x56242da8fb40 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a58f76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da9b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a59777640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242da8ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a59f78640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 0x56242da8e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a58f76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da9b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47962/0 (socket says 172.21.15.1:47962) 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a58f76640 1 -- 172.21.15.1:0/3966536295 learned_addr learned my addr 172.21.15.1:0/3966536295 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a58f76640 1 -- 172.21.15.1:0/3966536295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 msgr2=0x56242da8e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a58f76640 1 --2- 172.21.15.1:0/3966536295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 0x56242da8e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a58f76640 1 -- 172.21.15.1:0/3966536295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 msgr2=0x56242da8ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a58f76640 1 --2- 172.21.15.1:0/3966536295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242da8ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a58f76640 1 -- 172.21.15.1:0/3966536295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242d9803a0 con 0x56242da99370 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.653+0000 7f6a58f76640 1 --2- 172.21.15.1:0/3966536295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da9b760 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009900 tx=0x7f6a440040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8320d19fb2c6193 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.654+0000 7f6a4a7fc640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a400a3240 0x7f6a400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.654+0000 7f6a3f7fe640 1 -- 172.21.15.1:0/3966536295 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44035040 con 0x56242da99370 2024-09-16T00:42:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.654+0000 7f6a3f7fe640 1 -- 172.21.15.1:0/3966536295 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a44005520 con 0x56242da99370 2024-09-16T00:42:22.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.654+0000 7f6a3f7fe640 1 -- 172.21.15.1:0/3966536295 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44004e60 con 0x56242da99370 2024-09-16T00:42:22.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.654+0000 7f6a59f78640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a400a3240 0x7f6a400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.654+0000 7f6a4a7fc640 1 -- 172.21.15.1:0/2862579398 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6a500cf8d0 con 0x7f6a54074590 2024-09-16T00:42:22.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.654+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/3966536295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 msgr2=0x56242da9b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.654+0000 7f6a59f78640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a400a3240 0x7f6a400a5700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6a54076600 tx=0x7f6a4c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.654+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/3966536295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da9b760 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009900 tx=0x7f6a440040f0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.655+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/3966536295 shutdown_connections 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.655+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/3966536295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da9b760 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.655+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/3966536295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242da8ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.655+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/3966536295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 0x56242da8e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.655+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/3966536295 >> 172.21.15.1:0/3966536295 conn(0x56242da57310 msgr2=0x56242da8ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.655+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/3966536295 shutdown_connections 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.655+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/3966536295 wait complete. 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.655+0000 7f6a5ddb4740 1 -- start start 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.655+0000 7f6a5ddb4740 1 Processor -- start 2024-09-16T00:42:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242da98d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 0x56242da95ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56242da9c010 con 0x56242da99370 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56242d9af060 con 0x56242da8e5d0 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56242da9c190 con 0x56242da8fb40 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a5ddb4740 10 client.0 _ll_register_callbacks cb 0x56242d7028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59f78640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59f78640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47976/0 (socket says 172.21.15.1:47976) 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59f78640 1 -- 172.21.15.1:0/4064179620 learned_addr learned my addr 172.21.15.1:0/4064179620 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a58f76640 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 0x56242da95ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59777640 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242da98d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59f78640 1 -- 172.21.15.1:0/4064179620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 msgr2=0x56242da95ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59f78640 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 0x56242da95ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59f78640 1 -- 172.21.15.1:0/4064179620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 msgr2=0x56242da98d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59f78640 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242da98d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59f78640 1 -- 172.21.15.1:0/4064179620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242d7653c0 con 0x56242da99370 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a59f78640 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da965e0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005f40 tx=0x7f6a4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377105195c48420a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a3cff9640 1 -- 172.21.15.1:0/4064179620 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c02b060 con 0x56242da99370 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a3cff9640 1 -- 172.21.15.1:0/4064179620 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6a4c01c730 con 0x56242da99370 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.656+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/4064179620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a44005520 con 0x56242da99370 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a3cff9640 1 -- 172.21.15.1:0/4064179620 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6a4c032050 con 0x56242da99370 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a5ddb4740 10 client.18705 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/4064179620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56242daa7ae0 con 0x56242da99370 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a3cff9640 1 -- 172.21.15.1:0/4064179620 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a44005520 con 0x56242da99370 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a3cff9640 1 -- 172.21.15.1:0/4064179620 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6a4c01a040 con 0x56242da99370 2024-09-16T00:42:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a5ddb4740 10 client.18705 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a5ddb4740 10 client.18705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a5ddb4740 20 client.18705 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56242daa8fa0 0x56242daab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a5ddb4740 4 client.18705 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/4064179620 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56242da01860 con 0x56242daa8fa0 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.657+0000 7f6a59777640 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56242daa8fa0 0x56242daab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.658+0000 7f6a59777640 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56242daa8fa0 0x56242daab3b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ccdbaf39a2345bb server_cookie=c4593cceebec9915 in_seq=0 out_seq=0 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.658+0000 7f6a3cff9640 10 client.18705 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.658+0000 7f6a3cff9640 1 -- 172.21.15.1:0/4064179620 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56242da01860 con 0x56242daa8fa0 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.658+0000 7f6a3cff9640 10 client.18705 handle_command_reply: tid=0 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.658+0000 7f6a5ddb4740 1 client.18705 shutdown 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/4064179620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 msgr2=0x56242da965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da965e0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005f40 tx=0x7f6a4c004a40 comp rx=0 tx=0).stop 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/4064179620 shutdown_connections 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da99370 0x56242da965e0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da8fb40 0x56242da95ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56242daa8fa0 0x56242daab3b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/4064179620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242da98d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a3cff9640 0 client.18705 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/4064179620 >> 172.21.15.1:0/4064179620 conn(0x56242da57310 msgr2=0x56242da90960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/4064179620 shutdown_connections 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/4064179620 wait complete. 2024-09-16T00:42:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.659+0000 7f6a5ddb4740 20 client.18705 trim_cache size 0 max 16384 2024-09-16T00:42:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a5ddb4740 1 Processor -- start 2024-09-16T00:42:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a5ddb4740 1 -- start start 2024-09-16T00:42:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8e5d0 0x56242daa52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8fb40 0x56242da8a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 0x56242da8ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b7d90 con 0x56242da8e5d0 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b5660 con 0x56242da8fb40 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b34b0 con 0x56242da99370 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a59777640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8e5d0 0x56242daa52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a59777640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8e5d0 0x56242daa52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47988/0 (socket says 172.21.15.1:47988) 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.672+0000 7f6a59777640 1 -- 172.21.15.1:0/2493923972 learned_addr learned my addr 172.21.15.1:0/2493923972 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a59f78640 1 --2- 172.21.15.1:0/2493923972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 0x56242da8ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a58f76640 1 --2- 172.21.15.1:0/2493923972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8fb40 0x56242da8a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a59777640 1 -- 172.21.15.1:0/2493923972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 msgr2=0x56242da8ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a59777640 1 --2- 172.21.15.1:0/2493923972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 0x56242da8ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a59777640 1 -- 172.21.15.1:0/2493923972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8fb40 msgr2=0x56242da8a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:22.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a59777640 1 --2- 172.21.15.1:0/2493923972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8fb40 0x56242da8a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a59777640 1 -- 172.21.15.1:0/2493923972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242da990b0 con 0x56242da8e5d0 2024-09-16T00:42:22.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a59777640 1 --2- 172.21.15.1:0/2493923972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8e5d0 0x56242daa52b0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6a50098570 tx=0x7f6a500d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f65b016bc91c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:22.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a3effd640 1 -- 172.21.15.1:0/2493923972 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50020040 con 0x56242da8e5d0 2024-09-16T00:42:22.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a3effd640 1 -- 172.21.15.1:0/2493923972 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a50009de0 con 0x56242da8e5d0 2024-09-16T00:42:22.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.673+0000 7f6a3effd640 1 -- 172.21.15.1:0/2493923972 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a500860b0 con 0x56242da8e5d0 2024-09-16T00:42:22.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/2493923972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8e5d0 msgr2=0x56242daa52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/2493923972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8e5d0 0x56242daa52b0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6a50098570 tx=0x7f6a500d5550 comp rx=0 tx=0).stop 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/2493923972 shutdown_connections 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/2493923972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 0x56242da8ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/2493923972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8fb40 0x56242da8a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/2493923972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8e5d0 0x56242daa52b0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/2493923972 >> 172.21.15.1:0/2493923972 conn(0x56242da57310 msgr2=0x56242da95fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/2493923972 shutdown_connections 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/2493923972 wait complete. 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 -- start start 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.674+0000 7f6a5ddb4740 1 Processor -- start 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242daa1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8fb40 0x56242da92b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a5ddb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 0x56242da93080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b3b20 con 0x56242da8fb40 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b4f90 con 0x56242da8e5d0 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a5ddb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b41f0 con 0x56242da99370 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a5ddb4740 10 client.0 _ll_register_callbacks cb 0x7f6a4c011500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a58f76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8fb40 0x56242da92b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a58f76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8fb40 0x56242da92b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47994/0 (socket says 172.21.15.1:47994) 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a58f76640 1 -- 172.21.15.1:0/126480762 learned_addr learned my addr 172.21.15.1:0/126480762 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a59f78640 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 0x56242da93080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a59777640 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242daa1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a58f76640 1 -- 172.21.15.1:0/126480762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 msgr2=0x56242da93080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a58f76640 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 0x56242da93080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a58f76640 1 -- 172.21.15.1:0/126480762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 msgr2=0x56242daa1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a58f76640 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242daa1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a58f76640 1 -- 172.21.15.1:0/126480762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242da93640 con 0x56242da8fb40 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a58f76640 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8fb40 0x56242da92b40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x56242da8f070 tx=0x7f6a44009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3239fe47b1e2cdfc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.675+0000 7f6a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a3cff9640 1 -- 172.21.15.1:0/126480762 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44035040 con 0x56242da8fb40 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a3cff9640 1 -- 172.21.15.1:0/126480762 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6a440035f0 con 0x56242da8fb40 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/126480762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a50009de0 con 0x56242da8fb40 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a3cff9640 1 -- 172.21.15.1:0/126480762 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6a4405c050 con 0x56242da8fb40 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a5ddb4740 10 client.18711 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/126480762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56242db13a10 con 0x56242da8fb40 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a3cff9640 1 -- 172.21.15.1:0/126480762 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a50009de0 con 0x56242da8fb40 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a3cff9640 1 -- 172.21.15.1:0/126480762 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6a44031880 con 0x56242da8fb40 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a5ddb4740 10 client.18711 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a5ddb4740 10 client.18711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:42:22.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a5ddb4740 20 client.18711 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56242db13d40 0x56242db16160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a5ddb4740 4 client.18711 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/126480762 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56242d9fbd70 con 0x56242db13d40 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.676+0000 7f6a59777640 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56242db13d40 0x56242db16160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a3cff9640 10 client.18711 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a59777640 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56242db13d40 0x56242db16160 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32c5bf17ddf33c51 server_cookie=fae33fd8cf605336 in_seq=0 out_seq=0 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a3cff9640 1 -- 172.21.15.1:0/126480762 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56242d9fbd70 con 0x56242db13d40 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a3cff9640 10 client.18711 handle_command_reply: tid=0 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 client.18711 shutdown 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/126480762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8fb40 msgr2=0x56242da92b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8fb40 0x56242da92b40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x56242da8f070 tx=0x7f6a44009650 comp rx=0 tx=0).stop 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/126480762 shutdown_connections 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56242da99370 0x56242da93080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56242da8fb40 0x56242da92b40 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x56242db13d40 0x56242db16160 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 --2- 172.21.15.1:0/126480762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242da8e5d0 0x56242daa1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a3cff9640 0 client.18711 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:42:22.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/126480762 >> 172.21.15.1:0/126480762 conn(0x56242da57310 msgr2=0x56242da8a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:22.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/126480762 shutdown_connections 2024-09-16T00:42:22.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.677+0000 7f6a5ddb4740 1 -- 172.21.15.1:0/126480762 wait complete. 2024-09-16T00:42:22.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5ddb4740 20 client.18711 trim_cache size 0 max 16384 2024-09-16T00:42:22.683 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:42:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 -- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a400a3240 msgr2=0x7f6a400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a400a3240 0x7f6a400a5700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6a54076600 tx=0x7f6a4c009600 comp rx=0 tx=0).stop 2024-09-16T00:42:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 -- 172.21.15.1:0/2862579398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a54074590 msgr2=0x7f6a541a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a54074590 0x7f6a541a46f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000d950 tx=0x7f6a5000de10 comp rx=0 tx=0).stop 2024-09-16T00:42:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 -- 172.21.15.1:0/2862579398 shutdown_connections 2024-09-16T00:42:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a54112b90 0x7f6a5419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54074eb0 0x7f6a541a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f6a400a3240 0x7f6a400a5700 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 --2- 172.21.15.1:0/2862579398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a54074590 0x7f6a541a46f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:22.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 -- 172.21.15.1:0/2862579398 >> 172.21.15.1:0/2862579398 conn(0x7f6a540fd8c0 msgr2=0x7f6a540ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:22.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 -- 172.21.15.1:0/2862579398 shutdown_connections 2024-09-16T00:42:22.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:22.678+0000 7f6a5b9db640 1 -- 172.21.15.1:0/2862579398 wait complete. 2024-09-16T00:42:22.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:22 smithi153 ceph-mon[35538]: pgmap v769: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:42:22.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:22 smithi019 ceph-mon[35186]: pgmap v769: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:42:23.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:42:23.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:23 smithi001 ceph-mon[30934]: pgmap v770: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:42:23.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:23 smithi153 ceph-mon[35538]: pgmap v770: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:42:23.733 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:23.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:23 smithi019 ceph-mon[35186]: pgmap v770: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:42:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.214+0000 7f9dc50cd640 1 -- 172.21.15.1:0/3222111507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0102780 msgr2=0x7f9dc01067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.214+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/3222111507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0102780 0x7f9dc01067d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009900 tx=0x7f9da802a410 comp rx=0 tx=0).stop 2024-09-16T00:42:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.215+0000 7f9dc50cd640 1 -- 172.21.15.1:0/3222111507 shutdown_connections 2024-09-16T00:42:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.215+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/3222111507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc0106f00 0x7f9dc010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.215+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/3222111507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0102780 0x7f9dc01067d0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.215+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/3222111507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc0101dd0 0x7f9dc01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.215+0000 7f9dc50cd640 1 -- 172.21.15.1:0/3222111507 >> 172.21.15.1:0/3222111507 conn(0x7f9dc00fd8b0 msgr2=0x7f9dc00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.215+0000 7f9dc50cd640 1 -- 172.21.15.1:0/3222111507 shutdown_connections 2024-09-16T00:42:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.215+0000 7f9dc50cd640 1 -- 172.21.15.1:0/3222111507 wait complete. 2024-09-16T00:42:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.216+0000 7f9dc50cd640 1 Processor -- start 2024-09-16T00:42:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.216+0000 7f9dc50cd640 1 -- start start 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dc50cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc0101dd0 0x7f9dc019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dc50cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc0102780 0x7f9dc019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dc50cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0106f00 0x7f9dc019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dc50cd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0111f20 con 0x7f9dc0106f00 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dc50cd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0111da0 con 0x7f9dc0101dd0 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dc50cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01120a0 con 0x7f9dc0102780 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbf59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0106f00 0x7f9dc019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbf59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0106f00 0x7f9dc019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48008/0 (socket says 172.21.15.1:48008) 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbf59e640 1 -- 172.21.15.1:0/2374734244 learned_addr learned my addr 172.21.15.1:0/2374734244 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc0101dd0 0x7f9dc019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc0102780 0x7f9dc019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbf59e640 1 -- 172.21.15.1:0/2374734244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc0102780 msgr2=0x7f9dc019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbf59e640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc0102780 0x7f9dc019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbf59e640 1 -- 172.21.15.1:0/2374734244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc0101dd0 msgr2=0x7f9dc019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbf59e640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc0101dd0 0x7f9dc019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbf59e640 1 -- 172.21.15.1:0/2374734244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc019f630 con 0x7f9dc0106f00 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.217+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc0101dd0 0x7f9dc019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.218+0000 7f9dbf59e640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0106f00 0x7f9dc019ef50 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f9db800e970 tx=0x7f9db800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.218+0000 7f9db7fff640 1 -- 172.21.15.1:0/2374734244 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db800ccc0 con 0x7f9dc0106f00 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.218+0000 7f9dc50cd640 1 -- 172.21.15.1:0/2374734244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc019f920 con 0x7f9dc0106f00 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.218+0000 7f9db7fff640 1 -- 172.21.15.1:0/2374734244 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db800ce60 con 0x7f9dc0106f00 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.218+0000 7f9db7fff640 1 -- 172.21.15.1:0/2374734244 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8010490 con 0x7f9dc0106f00 2024-09-16T00:42:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.218+0000 7f9dc50cd640 1 -- 172.21.15.1:0/2374734244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc01a7200 con 0x7f9dc0106f00 2024-09-16T00:42:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.220+0000 7f9db7fff640 1 -- 172.21.15.1:0/2374734244 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9db8005020 con 0x7f9dc0106f00 2024-09-16T00:42:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dc74a6740 1 Processor -- start 2024-09-16T00:42:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dc74a6740 1 -- start start 2024-09-16T00:42:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e48e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e48ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 0x56226e49b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56226e3f42f0 con 0x56226e48fb40 2024-09-16T00:42:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56226e39bcf0 con 0x56226e499370 2024-09-16T00:42:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56226e3f02b0 con 0x56226e48e5d0 2024-09-16T00:42:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e48e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e48e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48012/0 (socket says 172.21.15.1:48012) 2024-09-16T00:42:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.221+0000 7f9dbe59c640 1 -- 172.21.15.1:0/1507501700 learned_addr learned my addr 172.21.15.1:0/1507501700 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9db7fff640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dac0a3240 0x7f9dac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/1507501700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 0x56226e49b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/1507501700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e48ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbe59c640 1 -- 172.21.15.1:0/1507501700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 msgr2=0x56226e48ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/1507501700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e48ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbe59c640 1 -- 172.21.15.1:0/1507501700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 msgr2=0x56226e49b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/1507501700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 0x56226e49b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbf59e640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dac0a3240 0x7f9dac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbe59c640 1 -- 172.21.15.1:0/1507501700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226e3803a0 con 0x56226e48fb40 2024-09-16T00:42:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9db7fff640 1 -- 172.21.15.1:0/2374734244 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9db80d0150 con 0x7f9dc0106f00 2024-09-16T00:42:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/1507501700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e48e090 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x56226e48f070 tx=0x7f9da8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bba469dd5fca85c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9db57fa640 1 -- 172.21.15.1:0/1507501700 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8044030 con 0x56226e48fb40 2024-09-16T00:42:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9dbf59e640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dac0a3240 0x7f9dac0a5700 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9db809c300 tx=0x7f9db8063040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9db57fa640 1 -- 172.21.15.1:0/1507501700 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da80319d0 con 0x56226e48fb40 2024-09-16T00:42:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.222+0000 7f9db57fa640 1 -- 172.21.15.1:0/1507501700 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8035040 con 0x56226e48fb40 2024-09-16T00:42:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.223+0000 7f9dc74a6740 1 -- 172.21.15.1:0/1507501700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 msgr2=0x56226e48e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.223+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/1507501700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e48e090 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x56226e48f070 tx=0x7f9da8000bc0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.223+0000 7f9dc74a6740 1 -- 172.21.15.1:0/1507501700 shutdown_connections 2024-09-16T00:42:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.223+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/1507501700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 0x56226e49b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.223+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/1507501700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e48ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.223+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/1507501700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e48e090 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.223+0000 7f9dc74a6740 1 -- 172.21.15.1:0/1507501700 >> 172.21.15.1:0/1507501700 conn(0x56226e457310 msgr2=0x56226e48ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.223+0000 7f9dc74a6740 1 -- 172.21.15.1:0/1507501700 shutdown_connections 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.223+0000 7f9dc74a6740 1 -- 172.21.15.1:0/1507501700 wait complete. 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dc74a6740 1 -- start start 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dc74a6740 1 Processor -- start 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e498db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 0x56226e496060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56226e49c010 con 0x56226e499370 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56226e3af060 con 0x56226e48fb40 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56226e49c190 con 0x56226e48e5d0 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dc74a6740 10 client.0 _ll_register_callbacks cb 0x56226e1028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48024/0 (socket says 172.21.15.1:48024) 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dbe59c640 1 -- 172.21.15.1:0/3063174498 learned_addr learned my addr 172.21.15.1:0/3063174498 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 0x56226e496060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.224+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e498db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9dbe59c640 1 -- 172.21.15.1:0/3063174498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 msgr2=0x56226e498db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e498db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9dbe59c640 1 -- 172.21.15.1:0/3063174498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 msgr2=0x56226e496060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 0x56226e496060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9dbe59c640 1 -- 172.21.15.1:0/3063174498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56226e1653c0 con 0x56226e499370 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4967f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f9da8031ba0 tx=0x7f9da8005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58115f0a58dc419 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9da67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9da67fc640 1 -- 172.21.15.1:0/3063174498 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8044030 con 0x56226e499370 2024-09-16T00:42:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9da67fc640 1 -- 172.21.15.1:0/3063174498 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9da80042f0 con 0x56226e499370 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9dc74a6740 1 -- 172.21.15.1:0/3063174498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da80319d0 con 0x56226e499370 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9da67fc640 1 -- 172.21.15.1:0/3063174498 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9da8035040 con 0x56226e499370 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9dc74a6740 10 client.18723 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9dc74a6740 1 -- 172.21.15.1:0/3063174498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56226e4a7b00 con 0x56226e499370 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.225+0000 7f9da67fc640 1 -- 172.21.15.1:0/3063174498 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da80319d0 con 0x56226e499370 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9da67fc640 1 -- 172.21.15.1:0/3063174498 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9da8003b80 con 0x56226e499370 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9dc74a6740 10 client.18723 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9dc74a6740 10 client.18723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9dc74a6740 20 client.18723 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56226e4a8e50 0x56226e4ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9dc74a6740 4 client.18723 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9dc74a6740 1 -- 172.21.15.1:0/3063174498 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56226e401860 con 0x56226e4a8e50 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56226e4a8e50 0x56226e4ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56226e4a8e50 0x56226e4ab240 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab9cf0576a5cfd04 server_cookie=b84c704be3922434 in_seq=0 out_seq=0 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.226+0000 7f9da67fc640 10 client.18723 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9da67fc640 1 -- 172.21.15.1:0/3063174498 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56226e401860 con 0x56226e4a8e50 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9da67fc640 10 client.18723 handle_command_reply: tid=0 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 client.18723 shutdown 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 -- 172.21.15.1:0/3063174498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 msgr2=0x56226e4967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4967f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f9da8031ba0 tx=0x7f9da8005500 comp rx=0 tx=0).stop 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 -- 172.21.15.1:0/3063174498 shutdown_connections 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56226e4a8e50 0x56226e4ab240 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9da67fc640 0 client.18723 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4967f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 0x56226e496060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/3063174498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e498db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 -- 172.21.15.1:0/3063174498 >> 172.21.15.1:0/3063174498 conn(0x56226e457310 msgr2=0x56226e490960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 -- 172.21.15.1:0/3063174498 shutdown_connections 2024-09-16T00:42:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.227+0000 7f9dc74a6740 1 -- 172.21.15.1:0/3063174498 wait complete. 2024-09-16T00:42:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.228+0000 7f9dc74a6740 20 client.18723 trim_cache size 0 max 16384 2024-09-16T00:42:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.241+0000 7f9dc74a6740 1 Processor -- start 2024-09-16T00:42:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.241+0000 7f9dc74a6740 1 -- start start 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e495920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e4a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 0x56226e4ae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01b3800 con 0x56226e48fb40 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01b10d0 con 0x56226e499370 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01aef20 con 0x56226e48e5d0 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e4a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e4a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48026/0 (socket says 172.21.15.1:48026) 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbed9d640 1 -- 172.21.15.1:0/454540496 learned_addr learned my addr 172.21.15.1:0/454540496 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 0x56226e4ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e495920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbed9d640 1 -- 172.21.15.1:0/454540496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 msgr2=0x56226e495920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e495920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbed9d640 1 -- 172.21.15.1:0/454540496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 msgr2=0x56226e4ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 0x56226e4ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbed9d640 1 -- 172.21.15.1:0/454540496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226e491120 con 0x56226e48fb40 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e495920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e4a4b30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009fd0 tx=0x7f9db0014e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10237beed2adb40 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9db4ff9640 1 -- 172.21.15.1:0/454540496 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0004d20 con 0x56226e48fb40 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9db4ff9640 1 -- 172.21.15.1:0/454540496 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db0003970 con 0x56226e48fb40 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.242+0000 7f9db4ff9640 1 -- 172.21.15.1:0/454540496 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0004470 con 0x56226e48fb40 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.243+0000 7f9dc74a6740 1 -- 172.21.15.1:0/454540496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 msgr2=0x56226e4a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.243+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e4a4b30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009fd0 tx=0x7f9db0014e20 comp rx=0 tx=0).stop 2024-09-16T00:42:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.243+0000 7f9dc74a6740 1 -- 172.21.15.1:0/454540496 shutdown_connections 2024-09-16T00:42:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.243+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e499370 0x56226e4ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.243+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e48fb40 0x56226e4a4b30 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.243+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/454540496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e495920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.243+0000 7f9dc74a6740 1 -- 172.21.15.1:0/454540496 >> 172.21.15.1:0/454540496 conn(0x56226e457310 msgr2=0x56226e4a4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.243+0000 7f9dc74a6740 1 -- 172.21.15.1:0/454540496 shutdown_connections 2024-09-16T00:42:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.243+0000 7f9dc74a6740 1 -- 172.21.15.1:0/454540496 wait complete. 2024-09-16T00:42:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dc74a6740 1 -- start start 2024-09-16T00:42:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dc74a6740 1 Processor -- start 2024-09-16T00:42:24.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e4ac150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 0x56226e4ac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dc74a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01af590 con 0x56226e499370 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01b0a00 con 0x56226e48fb40 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dc74a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01afc60 con 0x56226e48e5d0 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dc74a6740 10 client.0 _ll_register_callbacks cb 0x7f9da8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4acbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48042/0 (socket says 172.21.15.1:48042) 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dbe59c640 1 -- 172.21.15.1:0/279865048 learned_addr learned my addr 172.21.15.1:0/279865048 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e4ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 0x56226e4ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dbe59c640 1 -- 172.21.15.1:0/279865048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 msgr2=0x56226e4ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.244+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e4ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dbe59c640 1 -- 172.21.15.1:0/279865048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 msgr2=0x56226e4ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 0x56226e4ac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dbe59c640 1 -- 172.21.15.1:0/279865048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56226e512900 con 0x56226e499370 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dbe59c640 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4acbd0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f9da8006010 tx=0x7f9da8005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c355054ff84ed1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9da67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9da67fc640 1 -- 172.21.15.1:0/279865048 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8044040 con 0x56226e499370 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9da67fc640 1 -- 172.21.15.1:0/279865048 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9da804ab70 con 0x56226e499370 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dc74a6740 1 -- 172.21.15.1:0/279865048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9db0003970 con 0x56226e499370 2024-09-16T00:42:24.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9da67fc640 1 -- 172.21.15.1:0/279865048 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9da8005230 con 0x56226e499370 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dc74a6740 10 client.18729 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dc74a6740 1 -- 172.21.15.1:0/279865048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56226e513bc0 con 0x56226e499370 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9da67fc640 1 -- 172.21.15.1:0/279865048 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db0003970 con 0x56226e499370 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9da67fc640 1 -- 172.21.15.1:0/279865048 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9da80035f0 con 0x56226e499370 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dc74a6740 10 client.18729 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dc74a6740 10 client.18729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dc74a6740 20 client.18729 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.245+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56226e4a8e50 0x56226e516240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 4 client.18729 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 1 -- 172.21.15.1:0/279865048 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56226e3fbd70 con 0x56226e4a8e50 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56226e4a8e50 0x56226e516240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:24.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9da67fc640 10 client.18729 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dbed9d640 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56226e4a8e50 0x56226e516240 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a59220f6231e2022 server_cookie=28b2be33bb00026e in_seq=0 out_seq=0 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9da67fc640 1 -- 172.21.15.1:0/279865048 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56226e3fbd70 con 0x56226e4a8e50 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9da67fc640 10 client.18729 handle_command_reply: tid=0 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 1 client.18729 shutdown 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 1 -- 172.21.15.1:0/279865048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 msgr2=0x56226e4acbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4acbd0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f9da8006010 tx=0x7f9da8005500 comp rx=0 tx=0).stop 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 1 -- 172.21.15.1:0/279865048 shutdown_connections 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x56226e4a8e50 0x56226e516240 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9da67fc640 0 client.18729 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:42:24.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56226e499370 0x56226e4acbd0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56226e48fb40 0x56226e4ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.246+0000 7f9dc74a6740 1 --2- 172.21.15.1:0/279865048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56226e48e5d0 0x56226e4ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc74a6740 1 -- 172.21.15.1:0/279865048 >> 172.21.15.1:0/279865048 conn(0x56226e457310 msgr2=0x56226e490930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:24.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc74a6740 1 -- 172.21.15.1:0/279865048 shutdown_connections 2024-09-16T00:42:24.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc74a6740 1 -- 172.21.15.1:0/279865048 wait complete. 2024-09-16T00:42:24.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc74a6740 20 client.18729 trim_cache size 0 max 16384 2024-09-16T00:42:24.254 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:42:24.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 -- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dac0a3240 msgr2=0x7f9dac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:24.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dac0a3240 0x7f9dac0a5700 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9db809c300 tx=0x7f9db8063040 comp rx=0 tx=0).stop 2024-09-16T00:42:24.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 -- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0106f00 msgr2=0x7f9dc019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:24.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0106f00 0x7f9dc019ef50 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f9db800e970 tx=0x7f9db800ed70 comp rx=0 tx=0).stop 2024-09-16T00:42:24.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 -- 172.21.15.1:0/2374734244 shutdown_connections 2024-09-16T00:42:24.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dc0106f00 0x7f9dc019ef50 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9dac0a3240 0x7f9dac0a5700 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc0102780 0x7f9dc019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 --2- 172.21.15.1:0/2374734244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc0101dd0 0x7f9dc019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:24.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 -- 172.21.15.1:0/2374734244 >> 172.21.15.1:0/2374734244 conn(0x7f9dc00fd8b0 msgr2=0x7f9dc00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:24.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 -- 172.21.15.1:0/2374734244 shutdown_connections 2024-09-16T00:42:24.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:24.247+0000 7f9dc50cd640 1 -- 172.21.15.1:0/2374734244 wait complete. 2024-09-16T00:42:24.892 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:42:25.152 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.668+0000 7f4d1cc28640 1 -- 172.21.15.1:0/158694098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 msgr2=0x7f4d1810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.668+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/158694098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 0x7f4d1810ea90 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009900 tx=0x7f4d1002a410 comp rx=0 tx=0).stop 2024-09-16T00:42:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.669+0000 7f4d1cc28640 1 -- 172.21.15.1:0/158694098 shutdown_connections 2024-09-16T00:42:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.669+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/158694098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 0x7f4d1810ea90 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.669+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/158694098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18102790 0x7f4d181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.669+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/158694098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d18101de0 0x7f4d181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.669+0000 7f4d1cc28640 1 -- 172.21.15.1:0/158694098 >> 172.21.15.1:0/158694098 conn(0x7f4d180fd8c0 msgr2=0x7f4d180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.669+0000 7f4d1cc28640 1 -- 172.21.15.1:0/158694098 shutdown_connections 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.669+0000 7f4d1cc28640 1 -- 172.21.15.1:0/158694098 wait complete. 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.669+0000 7f4d1cc28640 1 Processor -- start 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.669+0000 7f4d1cc28640 1 -- start start 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d1cc28640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d18101de0 0x7f4d1819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d1cc28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18102790 0x7f4d1819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d1cc28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 0x7f4d1819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d1cc28640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4d18111f90 con 0x7f4d18106f10 2024-09-16T00:42:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d1cc28640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d18111e10 con 0x7f4d18102790 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d1cc28640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d18112110 con 0x7f4d18101de0 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d16d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 0x7f4d1819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d16d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 0x7f4d1819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48056/0 (socket says 172.21.15.1:48056) 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d16d9d640 1 -- 172.21.15.1:0/2404197237 learned_addr learned my addr 172.21.15.1:0/2404197237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d1659c640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d18101de0 0x7f4d1819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18102790 0x7f4d1819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d16d9d640 1 -- 172.21.15.1:0/2404197237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d18101de0 msgr2=0x7f4d1819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d16d9d640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d18101de0 0x7f4d1819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d16d9d640 1 -- 172.21.15.1:0/2404197237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18102790 msgr2=0x7f4d1819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d16d9d640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18102790 0x7f4d1819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d16d9d640 1 -- 172.21.15.1:0/2404197237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1819f670 con 0x7f4d18106f10 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.670+0000 7f4d16d9d640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 0x7f4d1819ef90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009230 tx=0x7f4d100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.671+0000 7f4d0f7fe640 1 -- 172.21.15.1:0/2404197237 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1003f030 con 0x7f4d18106f10 2024-09-16T00:42:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.671+0000 7f4d0f7fe640 1 -- 172.21.15.1:0/2404197237 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d10003ce0 con 0x7f4d18106f10 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.671+0000 7f4d1cc28640 1 -- 172.21.15.1:0/2404197237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1819f900 con 0x7f4d18106f10 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.671+0000 7f4d0f7fe640 1 -- 172.21.15.1:0/2404197237 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10035070 con 0x7f4d18106f10 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.671+0000 7f4d1cc28640 1 -- 172.21.15.1:0/2404197237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d181a7240 con 0x7f4d18106f10 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.673+0000 7f4d0f7fe640 1 -- 172.21.15.1:0/2404197237 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4d10042030 con 0x7f4d18106f10 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.673+0000 7f4d1f001740 1 Processor -- start 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.673+0000 7f4d1f001740 1 -- start start 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 0x55b5aaff7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 0x55b5aaff7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab004760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d1f001740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b5aaf5d2f0 con 0x55b5ab002370 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d1f001740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5aaf04cf0 con 0x55b5aaff75d0 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d1f001740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b5aaf592b0 con 0x55b5aaff8b40 2024-09-16T00:42:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab004760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab004760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48068/0 (socket says 172.21.15.1:48068) 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 -- 172.21.15.1:0/1115099068 learned_addr learned my addr 172.21.15.1:0/1115099068 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/1115099068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 0x55b5aaff7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d1659c640 1 --2- 172.21.15.1:0/1115099068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 0x55b5aaff7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d0f7fe640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4d040a3240 0x7f4d040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d16d9d640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4d040a3240 0x7f4d040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 -- 172.21.15.1:0/1115099068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 msgr2=0x55b5aaff7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/1115099068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 0x55b5aaff7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 -- 172.21.15.1:0/1115099068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 msgr2=0x55b5aaff7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/1115099068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 0x55b5aaff7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 -- 172.21.15.1:0/1115099068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5aaee93a0 con 0x55b5ab002370 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d0f7fe640 1 -- 172.21.15.1:0/2404197237 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4d100f05a0 con 0x7f4d18106f10 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.674+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/1115099068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab004760 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4d00011730 tx=0x7f4d0000a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed75ddeef4a739cc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d0cff9640 1 -- 172.21.15.1:0/1115099068 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00004020 con 0x55b5ab002370 2024-09-16T00:42:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d0cff9640 1 -- 172.21.15.1:0/1115099068 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d00005500 con 0x55b5ab002370 2024-09-16T00:42:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d0cff9640 1 -- 172.21.15.1:0/1115099068 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00004b80 con 0x55b5ab002370 2024-09-16T00:42:25.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d1f001740 1 -- 172.21.15.1:0/1115099068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 msgr2=0x55b5ab004760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:25.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1115099068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab004760 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4d00011730 tx=0x7f4d0000a6a0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d16d9d640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4d040a3240 0x7f4d040a5700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4d100b8c70 tx=0x7f4d1008b550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:25.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d1f001740 1 -- 172.21.15.1:0/1115099068 shutdown_connections 2024-09-16T00:42:25.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1115099068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab004760 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1115099068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 0x55b5aaff7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.675+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1115099068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 0x55b5aaff7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 -- 172.21.15.1:0/1115099068 >> 172.21.15.1:0/1115099068 conn(0x55b5aafc0310 msgr2=0x55b5aaff3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 -- 172.21.15.1:0/1115099068 shutdown_connections 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 -- 172.21.15.1:0/1115099068 wait complete. 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 -- start start 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 Processor -- start 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 0x55b5ab001d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 0x55b5aafff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5aafff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b5ab005070 con 0x55b5ab002370 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5aaf18060 con 0x55b5aaff75d0 2024-09-16T00:42:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b5ab0051f0 con 0x55b5aaff8b40 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d1f001740 10 client.0 _ll_register_callbacks cb 0x55b5aac6b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d15d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5aafff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d15d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5aafff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48076/0 (socket says 172.21.15.1:48076) 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.676+0000 7f4d15d9b640 1 -- 172.21.15.1:0/3768991357 learned_addr learned my addr 172.21.15.1:0/3768991357 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d1659c640 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 0x55b5aafff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 0x55b5ab001d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d15d9b640 1 -- 172.21.15.1:0/3768991357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 msgr2=0x55b5aafff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 0x55b5aafff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d15d9b640 1 -- 172.21.15.1:0/3768991357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 msgr2=0x55b5ab001d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 0x55b5ab001d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d15d9b640 1 -- 172.21.15.1:0/3768991357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5aacce3c0 con 0x55b5ab002370 2024-09-16T00:42:25.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5aafff610 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4d000058c0 tx=0x7f4d000045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d11bf78dc206bd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4cfdffb640 1 -- 172.21.15.1:0/3768991357 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00004de0 con 0x55b5ab002370 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d1f001740 1 -- 172.21.15.1:0/3768991357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d00005500 con 0x55b5ab002370 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4cfdffb640 1 -- 172.21.15.1:0/3768991357 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4d0000f790 con 0x55b5ab002370 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4cfdffb640 1 -- 172.21.15.1:0/3768991357 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d00032050 con 0x55b5ab002370 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d1f001740 10 client.18741 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.677+0000 7f4d1f001740 1 -- 172.21.15.1:0/3768991357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b5ab010b10 con 0x55b5ab002370 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4cfdffb640 1 -- 172.21.15.1:0/3768991357 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d00005500 con 0x55b5ab002370 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4cfdffb640 1 -- 172.21.15.1:0/3768991357 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4d0001e650 con 0x55b5ab002370 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4d1f001740 10 client.18741 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:25.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4d1f001740 10 client.18741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4d1f001740 20 client.18741 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5ab011ec0 0x55b5ab0142b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4d1f001740 4 client.18741 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4d1f001740 1 -- 172.21.15.1:0/3768991357 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5aaf6a860 con 0x55b5ab011ec0 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4d1659c640 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5ab011ec0 0x55b5ab0142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4d1659c640 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5ab011ec0 0x55b5ab0142b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa134ab488d0b152 server_cookie=8ce19c35ac28e404 in_seq=0 out_seq=0 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.678+0000 7f4cfdffb640 10 client.18741 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4cfdffb640 1 -- 172.21.15.1:0/3768991357 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5aaf6a860 con 0x55b5ab011ec0 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4cfdffb640 10 client.18741 handle_command_reply: tid=0 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 client.18741 shutdown 2024-09-16T00:42:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 -- 172.21.15.1:0/3768991357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 msgr2=0x55b5aafff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5aafff610 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4d000058c0 tx=0x7f4d000045a0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 -- 172.21.15.1:0/3768991357 shutdown_connections 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5ab011ec0 0x55b5ab0142b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5aafff610 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff8b40 0x55b5aafff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3768991357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff75d0 0x55b5ab001d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4cfdffb640 0 client.18741 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 -- 172.21.15.1:0/3768991357 >> 172.21.15.1:0/3768991357 conn(0x55b5aafc0310 msgr2=0x55b5aaff9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.679+0000 7f4d1f001740 1 -- 172.21.15.1:0/3768991357 shutdown_connections 2024-09-16T00:42:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.680+0000 7f4d1f001740 1 -- 172.21.15.1:0/3768991357 wait complete. 2024-09-16T00:42:25.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.680+0000 7f4d1f001740 20 client.18741 trim_cache size 0 max 16384 2024-09-16T00:42:25.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d1f001740 1 Processor -- start 2024-09-16T00:42:25.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d1f001740 1 -- start start 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 0x55b5ab00e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff8b40 0x55b5ab000f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab019a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d1f001740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181b38f0 con 0x55b5ab002370 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d1f001740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181b11c0 con 0x55b5aaff8b40 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d1f001740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181af010 con 0x55b5aaff75d0 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab019a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab019a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48080/0 (socket says 172.21.15.1:48080) 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 -- 172.21.15.1:0/3508685951 learned_addr learned my addr 172.21.15.1:0/3508685951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d1659c640 1 --2- 172.21.15.1:0/3508685951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff8b40 0x55b5ab000f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/3508685951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 0x55b5ab00e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 -- 172.21.15.1:0/3508685951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 msgr2=0x55b5ab00e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/3508685951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 0x55b5ab00e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 -- 172.21.15.1:0/3508685951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff8b40 msgr2=0x55b5ab000f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/3508685951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff8b40 0x55b5ab000f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 -- 172.21.15.1:0/3508685951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5aaff72d0 con 0x55b5ab002370 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.693+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/3508685951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab019a40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4d0001eaa0 tx=0x7f4d0000ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3449bda5595d3b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4cfffff640 1 -- 172.21.15.1:0/3508685951 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0003b070 con 0x55b5ab002370 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4cfffff640 1 -- 172.21.15.1:0/3508685951 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d00004660 con 0x55b5ab002370 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4cfffff640 1 -- 172.21.15.1:0/3508685951 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000f790 con 0x55b5ab002370 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4d1f001740 1 -- 172.21.15.1:0/3508685951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 msgr2=0x55b5ab019a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3508685951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab019a40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4d0001eaa0 tx=0x7f4d0000ab20 comp rx=0 tx=0).stop 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4d1f001740 1 -- 172.21.15.1:0/3508685951 shutdown_connections 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3508685951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5ab002370 0x55b5ab019a40 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3508685951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5aaff8b40 0x55b5ab000f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4d1f001740 1 --2- 172.21.15.1:0/3508685951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 0x55b5ab00e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4d1f001740 1 -- 172.21.15.1:0/3508685951 >> 172.21.15.1:0/3508685951 conn(0x55b5aafc0310 msgr2=0x55b5ab000880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:25.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4d1f001740 1 -- 172.21.15.1:0/3508685951 shutdown_connections 2024-09-16T00:42:25.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.694+0000 7f4d1f001740 1 -- 172.21.15.1:0/3508685951 wait complete. 2024-09-16T00:42:25.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1f001740 1 -- start start 2024-09-16T00:42:25.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1f001740 1 Processor -- start 2024-09-16T00:42:25.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 0x55b5ab00a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5aaff8b40 0x55b5ab00aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1f001740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ab002370 0x55b5aaffb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1f001740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181af680 con 0x55b5aaff8b40 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1f001740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181b0af0 con 0x55b5ab002370 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1f001740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181afd50 con 0x55b5aaff75d0 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1f001740 10 client.0 _ll_register_callbacks cb 0x7f4d08009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5aaff8b40 0x55b5ab00aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5aaff8b40 0x55b5ab00aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48094/0 (socket says 172.21.15.1:48094) 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d1659c640 1 -- 172.21.15.1:0/1998208517 learned_addr learned my addr 172.21.15.1:0/1998208517 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ab002370 0x55b5aaffb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.695+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 0x55b5ab00a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d1659c640 1 -- 172.21.15.1:0/1998208517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 msgr2=0x55b5ab00a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d1659c640 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 0x55b5ab00a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d1659c640 1 -- 172.21.15.1:0/1998208517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ab002370 msgr2=0x55b5aaffb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d1659c640 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ab002370 0x55b5aaffb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d1659c640 1 -- 172.21.15.1:0/1998208517 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5aaffbf60 con 0x55b5aaff8b40 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 0x55b5ab00a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d1659c640 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5aaff8b40 0x55b5ab00aba0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x55b5ab000660 tx=0x7f4d0800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96e0363458c6531 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4cfdffb640 1 -- 172.21.15.1:0/1998208517 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08017db0 con 0x55b5aaff8b40 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4cfdffb640 1 -- 172.21.15.1:0/1998208517 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4d08027420 con 0x55b5aaff8b40 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d1f001740 1 -- 172.21.15.1:0/1998208517 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d00004660 con 0x55b5aaff8b40 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4cfdffb640 1 -- 172.21.15.1:0/1998208517 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d08030050 con 0x55b5aaff8b40 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d1f001740 10 client.18747 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4d1f001740 1 -- 172.21.15.1:0/1998208517 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b5ab07c420 con 0x55b5aaff8b40 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.696+0000 7f4cfdffb640 1 -- 172.21.15.1:0/1998208517 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d00004660 con 0x55b5aaff8b40 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4cfdffb640 1 -- 172.21.15.1:0/1998208517 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4d080167f0 con 0x55b5aaff8b40 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4d1f001740 10 client.18747 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4d1f001740 10 client.18747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4d1f001740 20 client.18747 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5ab07cbd0 0x55b5ab07eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4d1f001740 4 client.18747 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4d1f001740 1 -- 172.21.15.1:0/1998208517 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5aaf64d70 con 0x55b5ab07cbd0 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5ab07cbd0 0x55b5ab07eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4d15d9b640 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5ab07cbd0 0x55b5ab07eff0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d875e15de469adb2 server_cookie=6735a7454a1c5b0e in_seq=0 out_seq=0 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4cfdffb640 10 client.18747 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4cfdffb640 1 -- 172.21.15.1:0/1998208517 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5aaf64d70 con 0x55b5ab07cbd0 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4cfdffb640 10 client.18747 handle_command_reply: tid=0 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.697+0000 7f4d1f001740 1 client.18747 shutdown 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 -- 172.21.15.1:0/1998208517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5aaff8b40 msgr2=0x55b5ab00aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5aaff8b40 0x55b5ab00aba0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x55b5ab000660 tx=0x7f4d0800be50 comp rx=0 tx=0).stop 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 -- 172.21.15.1:0/1998208517 shutdown_connections 2024-09-16T00:42:25.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b5ab07cbd0 0x55b5ab07eff0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ab002370 0x55b5aaffb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5aaff8b40 0x55b5ab00aba0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4cfdffb640 0 client.18747 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:25.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 --2- 172.21.15.1:0/1998208517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5aaff75d0 0x55b5ab00a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 -- 172.21.15.1:0/1998208517 >> 172.21.15.1:0/1998208517 conn(0x55b5aafc0310 msgr2=0x55b5ab0010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:25.704 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:42:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 -- 172.21.15.1:0/1998208517 shutdown_connections 2024-09-16T00:42:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 1 -- 172.21.15.1:0/1998208517 wait complete. 2024-09-16T00:42:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1f001740 20 client.18747 trim_cache size 0 max 16384 2024-09-16T00:42:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1cc28640 1 -- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4d040a3240 msgr2=0x7f4d040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4d040a3240 0x7f4d040a5700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4d100b8c70 tx=0x7f4d1008b550 comp rx=0 tx=0).stop 2024-09-16T00:42:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1cc28640 1 -- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 msgr2=0x7f4d1819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 0x7f4d1819ef90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009230 tx=0x7f4d100042f0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.698+0000 7f4d1cc28640 1 -- 172.21.15.1:0/2404197237 shutdown_connections 2024-09-16T00:42:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.699+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d18106f10 0x7f4d1819ef90 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.699+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18102790 0x7f4d1819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.699+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4d040a3240 0x7f4d040a5700 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.699+0000 7f4d1cc28640 1 --2- 172.21.15.1:0/2404197237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d18101de0 0x7f4d1819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:25.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.699+0000 7f4d1cc28640 1 -- 172.21.15.1:0/2404197237 >> 172.21.15.1:0/2404197237 conn(0x7f4d180fd8c0 msgr2=0x7f4d180fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:25.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.699+0000 7f4d1cc28640 1 -- 172.21.15.1:0/2404197237 shutdown_connections 2024-09-16T00:42:25.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:25.699+0000 7f4d1cc28640 1 -- 172.21.15.1:0/2404197237 wait complete. 2024-09-16T00:42:25.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:25 smithi001 ceph-mon[30934]: pgmap v771: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:42:25.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:25 smithi153 ceph-mon[35538]: pgmap v771: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:42:26.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:25 smithi019 ceph-mon[35186]: pgmap v771: 129 pgs: 129 active+clean; 6.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:42:27.454 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:42:27.455 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:42:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:27.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:27 smithi001 ceph-mon[30934]: pgmap v772: 129 pgs: 129 active+clean; 6.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:42:27.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:27 smithi153 ceph-mon[35538]: pgmap v772: 129 pgs: 129 active+clean; 6.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:42:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:27 smithi019 ceph-mon[35186]: pgmap v772: 129 pgs: 129 active+clean; 6.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:42:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.246+0000 7f7e01d1f640 1 -- 172.21.15.1:0/3978642836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc103660 msgr2=0x7f7dfc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.246+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/3978642836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc103660 0x7f7dfc10bb50 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009900 tx=0x7f7df002a480 comp rx=0 tx=0).stop 2024-09-16T00:42:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.246+0000 7f7e01d1f640 1 -- 172.21.15.1:0/3978642836 shutdown_connections 2024-09-16T00:42:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.246+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/3978642836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc10c170 0x7f7dfc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.246+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/3978642836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc103660 0x7f7dfc10bb50 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.246+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/3978642836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dfc102d40 0x7f7dfc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.246+0000 7f7e01d1f640 1 -- 172.21.15.1:0/3978642836 >> 172.21.15.1:0/3978642836 conn(0x7f7dfc0fc7a0 msgr2=0x7f7dfc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.246+0000 7f7e01d1f640 1 -- 172.21.15.1:0/3978642836 shutdown_connections 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.246+0000 7f7e01d1f640 1 -- 172.21.15.1:0/3978642836 wait complete. 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7e01d1f640 1 Processor -- start 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7e01d1f640 1 -- start start 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7e01d1f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc102d40 0x7f7dfc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7e01d1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dfc103660 0x7f7dfc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7e01d1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc10c170 0x7f7dfc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7e01d1f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc111fb0 con 0x7f7dfc10c170 2024-09-16T00:42:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7e01d1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc111e30 con 0x7f7dfc103660 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7e01d1f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc112130 con 0x7f7dfc102d40 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7dfbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc10c170 0x7f7dfc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7dfbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc10c170 0x7f7dfc101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48116/0 (socket says 172.21.15.1:48116) 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7dfbfff640 1 -- 172.21.15.1:0/1563206570 learned_addr learned my addr 172.21.15.1:0/1563206570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.247+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dfc103660 0x7f7dfc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc102d40 0x7f7dfc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7dfbfff640 1 -- 172.21.15.1:0/1563206570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc102d40 msgr2=0x7f7dfc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7dfbfff640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc102d40 0x7f7dfc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7dfbfff640 1 -- 172.21.15.1:0/1563206570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dfc103660 msgr2=0x7f7dfc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7dfbfff640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dfc103660 0x7f7dfc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7dfbfff640 1 -- 172.21.15.1:0/1563206570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc1a6960 con 0x7f7dfc10c170 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7dfbfff640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc10c170 0x7f7dfc101420 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7df400b970 tx=0x7f7df400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7df8ff9640 1 -- 172.21.15.1:0/1563206570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4019070 con 0x7f7dfc10c170 2024-09-16T00:42:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7e01d1f640 1 -- 172.21.15.1:0/1563206570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc1a6c50 con 0x7f7dfc10c170 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7df8ff9640 1 -- 172.21.15.1:0/1563206570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df400cb40 con 0x7f7dfc10c170 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7df8ff9640 1 -- 172.21.15.1:0/1563206570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4007610 con 0x7f7dfc10c170 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.248+0000 7f7e01d1f640 1 -- 172.21.15.1:0/1563206570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1a7160 con 0x7f7dfc10c170 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.250+0000 7f7df8ff9640 1 -- 172.21.15.1:0/1563206570 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f7df401c070 con 0x7f7dfc10c170 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7e040f8740 1 Processor -- start 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7e040f8740 1 -- start start 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 0x55a9f82f2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7e040f8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a9f824a280 con 0x55a9f82e5480 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7e040f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a9f81f1b10 con 0x55a9f82e4b60 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7e040f8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a9f8246240 con 0x55a9f82efd40 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7df8ff9640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7de40a3240 0x7f7de40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7dfb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7dfb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48132/0 (socket says 172.21.15.1:48132) 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7dfb7fe640 1 -- 172.21.15.1:0/3152974938 learned_addr learned my addr 172.21.15.1:0/3152974938 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7dfbfff640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7de40a3240 0x7f7de40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 0x55a9f82f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.251+0000 7f7df8ff9640 1 -- 172.21.15.1:0/1563206570 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7df40cf2e0 con 0x7f7dfc10c170 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7dfb7fe640 1 -- 172.21.15.1:0/3152974938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 msgr2=0x55a9f82f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 0x55a9f82f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7dfb7fe640 1 -- 172.21.15.1:0/3152974938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 msgr2=0x55a9f82e4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82e4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7dfb7fe640 1 -- 172.21.15.1:0/3152974938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9f81fe1f0 con 0x55a9f82e5480 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 0x55a9f82f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82ef800 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00bcb0 tx=0x7f7dec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca69e332b21989f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7dfbfff640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7de40a3240 0x7f7de40a5700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7df4000f80 tx=0x7f7df400f300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7de1ffb640 1 -- 172.21.15.1:0/3152974938 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec015070 con 0x55a9f82e5480 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7de1ffb640 1 -- 172.21.15.1:0/3152974938 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec01a940 con 0x55a9f82e5480 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.252+0000 7f7de1ffb640 1 -- 172.21.15.1:0/3152974938 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec011a40 con 0x55a9f82e5480 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 -- 172.21.15.1:0/3152974938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 msgr2=0x55a9f82ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82ef800 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00bcb0 tx=0x7f7dec00f730 comp rx=0 tx=0).stop 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 -- 172.21.15.1:0/3152974938 shutdown_connections 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 0x55a9f82f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82ef800 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 --2- 172.21.15.1:0/3152974938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82e4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 -- 172.21.15.1:0/3152974938 >> 172.21.15.1:0/3152974938 conn(0x55a9f82e10f0 msgr2=0x55a9f82e14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 -- 172.21.15.1:0/3152974938 shutdown_connections 2024-09-16T00:42:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 -- 172.21.15.1:0/3152974938 wait complete. 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 -- start start 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.253+0000 7f7e040f8740 1 Processor -- start 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 0x55a9f82ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82efd40 0x55a9f82ed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7e040f8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a9f82f2910 con 0x55a9f82efd40 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7e040f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a9f8204f10 con 0x55a9f82e4b60 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7e040f8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a9f82f2a90 con 0x55a9f82e5480 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7e040f8740 10 client.0 _ll_register_callbacks cb 0x55a9f81cda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82efd40 0x55a9f82ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82efd40 0x55a9f82ed620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48140/0 (socket says 172.21.15.1:48140) 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 -- 172.21.15.1:0/2983164608 learned_addr learned my addr 172.21.15.1:0/2983164608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 0x55a9f82ed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 -- 172.21.15.1:0/2983164608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 msgr2=0x55a9f82ed0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 0x55a9f82ed0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 -- 172.21.15.1:0/2983164608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 msgr2=0x55a9f82ecba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82ecba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 -- 172.21.15.1:0/2983164608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9f7fbb6a0 con 0x55a9f82efd40 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82efd40 0x55a9f82ed620 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009900 tx=0x7f7df002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5822b8c9417587 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.254+0000 7f7ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2983164608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0035040 con 0x55a9f82efd40 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2983164608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7df003ea70 con 0x55a9f82efd40 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7e040f8740 1 -- 172.21.15.1:0/2983164608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dec01a940 con 0x55a9f82efd40 2024-09-16T00:42:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2983164608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7df003ec70 con 0x55a9f82efd40 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7e040f8740 10 client.18759 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7e040f8740 1 -- 172.21.15.1:0/2983164608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9f82fe410 con 0x55a9f82efd40 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2983164608 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec01a940 con 0x55a9f82efd40 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2983164608 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7df0004ba0 con 0x55a9f82efd40 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7e040f8740 10 client.18759 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7e040f8740 10 client.18759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7e040f8740 20 client.18759 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a9f82ff8d0 0x55a9f8301cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7e040f8740 4 client.18759 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.255+0000 7f7e040f8740 1 -- 172.21.15.1:0/2983164608 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9f82577f0 con 0x55a9f82ff8d0 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.256+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a9f82ff8d0 0x55a9f8301cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.256+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a9f82ff8d0 0x55a9f8301cc0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0bd80aaafb7932 server_cookie=faf7ad1e493279e2 in_seq=0 out_seq=0 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.256+0000 7f7ddaffd640 10 client.18759 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2983164608 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9f82577f0 con 0x55a9f82ff8d0 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7ddaffd640 10 client.18759 handle_command_reply: tid=0 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 client.18759 shutdown 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 -- 172.21.15.1:0/2983164608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82efd40 msgr2=0x55a9f82ed620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82efd40 0x55a9f82ed620 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009900 tx=0x7f7df002a480 comp rx=0 tx=0).stop 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 -- 172.21.15.1:0/2983164608 shutdown_connections 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a9f82ff8d0 0x55a9f8301cc0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82efd40 0x55a9f82ed620 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 0x55a9f82ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7ddaffd640 0 client.18759 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2983164608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82ecba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 -- 172.21.15.1:0/2983164608 >> 172.21.15.1:0/2983164608 conn(0x55a9f82e10f0 msgr2=0x55a9f82e0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 -- 172.21.15.1:0/2983164608 shutdown_connections 2024-09-16T00:42:28.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 1 -- 172.21.15.1:0/2983164608 wait complete. 2024-09-16T00:42:28.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.257+0000 7f7e040f8740 20 client.18759 trim_cache size 0 max 16384 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7e040f8740 1 Processor -- start 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7e040f8740 1 -- start start 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82ec480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82eea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 0x55a9f8307520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7e040f8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1b3a10 con 0x55a9f82e5480 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7e040f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1b12e0 con 0x55a9f82e4b60 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7e040f8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1af130 con 0x55a9f82efd40 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7dfb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7dfb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82eea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48156/0 (socket says 172.21.15.1:48156) 2024-09-16T00:42:28.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7dfb7fe640 1 -- 172.21.15.1:0/473619079 learned_addr learned my addr 172.21.15.1:0/473619079 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/473619079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 0x55a9f8307520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/473619079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7dfb7fe640 1 -- 172.21.15.1:0/473619079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 msgr2=0x55a9f8307520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/473619079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 0x55a9f8307520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7dfb7fe640 1 -- 172.21.15.1:0/473619079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 msgr2=0x55a9f82ec480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.275+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/473619079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82ec480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.276+0000 7f7dfb7fe640 1 -- 172.21.15.1:0/473619079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9f83022f0 con 0x55a9f82e5480 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.276+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/473619079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82eea00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00bcb0 tx=0x7f7dec0115e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e767b6e6f699a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.276+0000 7f7de17fa640 1 -- 172.21.15.1:0/473619079 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec015050 con 0x55a9f82e5480 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.276+0000 7f7de17fa640 1 -- 172.21.15.1:0/473619079 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec02b070 con 0x55a9f82e5480 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.276+0000 7f7de17fa640 1 -- 172.21.15.1:0/473619079 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0117b0 con 0x55a9f82e5480 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.276+0000 7f7e040f8740 1 -- 172.21.15.1:0/473619079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 msgr2=0x55a9f82eea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.276+0000 7f7e040f8740 1 --2- 172.21.15.1:0/473619079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82eea00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00bcb0 tx=0x7f7dec0115e0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 -- 172.21.15.1:0/473619079 shutdown_connections 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 --2- 172.21.15.1:0/473619079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82efd40 0x55a9f8307520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 --2- 172.21.15.1:0/473619079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e5480 0x55a9f82eea00 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 --2- 172.21.15.1:0/473619079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82e4b60 0x55a9f82ec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 -- 172.21.15.1:0/473619079 >> 172.21.15.1:0/473619079 conn(0x55a9f82e10f0 msgr2=0x55a9f82ee2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 -- 172.21.15.1:0/473619079 shutdown_connections 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 -- 172.21.15.1:0/473619079 wait complete. 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 -- start start 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 Processor -- start 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e4b60 0x55a9f82f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 0x55a9f82f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82efd40 0x55a9f83694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1af7a0 con 0x55a9f82e4b60 2024-09-16T00:42:28.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1b0c10 con 0x55a9f82efd40 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1afe70 con 0x55a9f82e5480 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.277+0000 7f7e040f8740 10 client.0 _ll_register_callbacks cb 0x7f7df00315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e4b60 0x55a9f82f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e4b60 0x55a9f82f3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48170/0 (socket says 172.21.15.1:48170) 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfaffd640 1 -- 172.21.15.1:0/2640238113 learned_addr learned my addr 172.21.15.1:0/2640238113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 0x55a9f82f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfaffd640 1 -- 172.21.15.1:0/2640238113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 msgr2=0x55a9f82f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 0x55a9f82f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfaffd640 1 -- 172.21.15.1:0/2640238113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82efd40 msgr2=0x55a9f83694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82efd40 0x55a9f83694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfaffd640 1 -- 172.21.15.1:0/2640238113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9f82e7490 con 0x55a9f82e4b60 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7dfaffd640 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e4b60 0x55a9f82f3db0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7df002a9a0 tx=0x7f7df00045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901c592829d5ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:28.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2640238113 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0035040 con 0x55a9f82e4b60 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7e040f8740 1 -- 172.21.15.1:0/2640238113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dec02b070 con 0x55a9f82e4b60 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2640238113 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7df003ea70 con 0x55a9f82e4b60 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2640238113 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7df0064050 con 0x55a9f82e4b60 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7e040f8740 10 client.18765 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.278+0000 7f7e040f8740 1 -- 172.21.15.1:0/2640238113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9f82e0f20 con 0x55a9f82e4b60 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.279+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2640238113 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec02b070 con 0x55a9f82e4b60 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.279+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2640238113 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7df0004fd0 con 0x55a9f82e4b60 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.279+0000 7f7e040f8740 10 client.18765 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.279+0000 7f7e040f8740 10 client.18765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.279+0000 7f7e040f8740 20 client.18765 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.279+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a9f836af00 0x55a9f836d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.279+0000 7f7e040f8740 4 client.18765 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:42:28.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.279+0000 7f7e040f8740 1 -- 172.21.15.1:0/2640238113 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a9f8251d00 con 0x55a9f836af00 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.279+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a9f836af00 0x55a9f836d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.280+0000 7f7dfb7fe640 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a9f836af00 0x55a9f836d2f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f281555e492ac9 server_cookie=7823b6de00e839a6 in_seq=0 out_seq=0 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.280+0000 7f7ddaffd640 10 client.18765 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7ddaffd640 1 -- 172.21.15.1:0/2640238113 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a9f8251d00 con 0x55a9f836af00 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7ddaffd640 10 client.18765 handle_command_reply: tid=0 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 client.18765 shutdown 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 -- 172.21.15.1:0/2640238113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e4b60 msgr2=0x55a9f82f3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e4b60 0x55a9f82f3db0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7df002a9a0 tx=0x7f7df00045f0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 -- 172.21.15.1:0/2640238113 shutdown_connections 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a9f836af00 0x55a9f836d2f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9f82efd40 0x55a9f83694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9f82e5480 0x55a9f82f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 --2- 172.21.15.1:0/2640238113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f82e4b60 0x55a9f82f3db0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 -- 172.21.15.1:0/2640238113 >> 172.21.15.1:0/2640238113 conn(0x55a9f82e10f0 msgr2=0x55a9f82e0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7ddaffd640 0 client.18765 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:28.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 -- 172.21.15.1:0/2640238113 shutdown_connections 2024-09-16T00:42:28.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 1 -- 172.21.15.1:0/2640238113 wait complete. 2024-09-16T00:42:28.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.281+0000 7f7e040f8740 20 client.18765 trim_cache size 0 max 16384 2024-09-16T00:42:28.290 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:42:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:42:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "595cbb5d-9a91-4e1a-9fad-236572259159", 2024-09-16T00:42:28.291 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:42:28.291 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:42:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 -- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7de40a3240 msgr2=0x7f7de40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7de40a3240 0x7f7de40a5700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7df4000f80 tx=0x7f7df400f300 comp rx=0 tx=0).stop 2024-09-16T00:42:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 -- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc10c170 msgr2=0x7f7dfc101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc10c170 0x7f7dfc101420 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7df400b970 tx=0x7f7df400be30 comp rx=0 tx=0).stop 2024-09-16T00:42:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 -- 172.21.15.1:0/1563206570 shutdown_connections 2024-09-16T00:42:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dfc10c170 0x7f7dfc101420 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dfc103660 0x7f7dfc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f7de40a3240 0x7f7de40a5700 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 --2- 172.21.15.1:0/1563206570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc102d40 0x7f7dfc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 -- 172.21.15.1:0/1563206570 >> 172.21.15.1:0/1563206570 conn(0x7f7dfc0fc7a0 msgr2=0x7f7dfc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 -- 172.21.15.1:0/1563206570 shutdown_connections 2024-09-16T00:42:28.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:28.282+0000 7f7e01d1f640 1 -- 172.21.15.1:0/1563206570 wait complete. 2024-09-16T00:42:29.097 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:42:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:29.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:29 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:42:29.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:29 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:29.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:29 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:42:29.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:29 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:29.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:29 smithi001 ceph-mon[30934]: pgmap v773: 129 pgs: 129 active+clean; 6.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:42:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.879+0000 7fafbc44c640 1 -- 172.21.15.1:0/1466849864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 msgr2=0x7fafb410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.879+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1466849864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 0x7fafb410da40 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fafac009ca0 tx=0x7fafac02a950 comp rx=0 tx=0).stop 2024-09-16T00:42:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.879+0000 7fafbc44c640 1 -- 172.21.15.1:0/1466849864 shutdown_connections 2024-09-16T00:42:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.879+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1466849864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 0x7fafb410da40 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.879+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1466849864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb4102570 0x7fafb41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.879+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1466849864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb4100e20 0x7fafb4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.879+0000 7fafbc44c640 1 -- 172.21.15.1:0/1466849864 >> 172.21.15.1:0/1466849864 conn(0x7fafb40fc7d0 msgr2=0x7fafb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.879+0000 7fafbc44c640 1 -- 172.21.15.1:0/1466849864 shutdown_connections 2024-09-16T00:42:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.879+0000 7fafbc44c640 1 -- 172.21.15.1:0/1466849864 wait complete. 2024-09-16T00:42:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafbc44c640 1 Processor -- start 2024-09-16T00:42:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafbc44c640 1 -- start start 2024-09-16T00:42:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafbc44c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb4100e20 0x7fafb4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafbc44c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb4102570 0x7fafb4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafbc44c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 0x7fafb419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafbc44c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4110fa0 con 0x7fafb4102ef0 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafbc44c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4110e20 con 0x7fafb4102570 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafbc44c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4111120 con 0x7fafb4100e20 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafba9e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 0x7fafb419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafba9e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 0x7fafb419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48190/0 (socket says 172.21.15.1:48190) 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafba9e9640 1 -- 172.21.15.1:0/1788090889 learned_addr learned my addr 172.21.15.1:0/1788090889 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.880+0000 7fafb99e7640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb4102570 0x7fafb4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafba1e8640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb4100e20 0x7fafb4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafba9e9640 1 -- 172.21.15.1:0/1788090889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb4100e20 msgr2=0x7fafb4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafba9e9640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb4100e20 0x7fafb4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafba9e9640 1 -- 172.21.15.1:0/1788090889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb4102570 msgr2=0x7fafb4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafba9e9640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb4102570 0x7fafb4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafba9e9640 1 -- 172.21.15.1:0/1788090889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb419d540 con 0x7fafb4102ef0 2024-09-16T00:42:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafba9e9640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 0x7fafb419ce60 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fafac02ae70 tx=0x7fafac02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafab7fe640 1 -- 172.21.15.1:0/1788090889 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac034040 con 0x7fafb4102ef0 2024-09-16T00:42:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafbc44c640 1 -- 172.21.15.1:0/1788090889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb419d7d0 con 0x7fafb4102ef0 2024-09-16T00:42:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafab7fe640 1 -- 172.21.15.1:0/1788090889 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafac030920 con 0x7fafb4102ef0 2024-09-16T00:42:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafab7fe640 1 -- 172.21.15.1:0/1788090889 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0047f0 con 0x7fafb4102ef0 2024-09-16T00:42:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.881+0000 7fafbc44c640 1 -- 172.21.15.1:0/1788090889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb41a5110 con 0x7fafb4102ef0 2024-09-16T00:42:29.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.884+0000 7fafbe825740 1 Processor -- start 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.884+0000 7fafab7fe640 1 -- 172.21.15.1:0/1788090889 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fafac037020 con 0x7fafb4102ef0 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.884+0000 7fafbe825740 1 -- start start 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.884+0000 7fafbe825740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8d3a0 0x55a86fa8d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.884+0000 7fafbe825740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 0x55a86fa9c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.884+0000 7fafbe825740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 0x55a86fa9ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.884+0000 7fafbe825740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a86f9f32e0 con 0x55a86fa8d3a0 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.884+0000 7fafbe825740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a86f99ace0 con 0x55a86fa8dcc0 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.884+0000 7fafbe825740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a86f9ef2a0 con 0x55a86fa9ca90 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8d3a0 0x55a86fa8d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8d3a0 0x55a86fa8d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48202/0 (socket says 172.21.15.1:48202) 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 -- 172.21.15.1:0/4046696185 learned_addr learned my addr 172.21.15.1:0/4046696185 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafba1e8640 1 --2- 172.21.15.1:0/4046696185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 0x55a86fa9c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 --2- 172.21.15.1:0/4046696185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 0x55a86fa9ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafab7fe640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafa00a3240 0x7fafa00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 -- 172.21.15.1:0/4046696185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 msgr2=0x55a86fa9ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 --2- 172.21.15.1:0/4046696185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 0x55a86fa9ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 -- 172.21.15.1:0/4046696185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 msgr2=0x55a86fa9c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 --2- 172.21.15.1:0/4046696185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 0x55a86fa9c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 -- 172.21.15.1:0/4046696185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86f97f440 con 0x55a86fa8d3a0 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafab7fe640 1 -- 172.21.15.1:0/1788090889 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fafac0f0b80 con 0x7fafb4102ef0 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafba9e9640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafa00a3240 0x7fafa00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafb99e7640 1 --2- 172.21.15.1:0/4046696185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8d3a0 0x55a86fa8d780 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fafa4005cf0 tx=0x7fafa400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3893543cbeb24df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafa8dea640 1 -- 172.21.15.1:0/4046696185 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004030 con 0x55a86fa8d3a0 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.885+0000 7fafa8dea640 1 -- 172.21.15.1:0/4046696185 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4005500 con 0x55a86fa8d3a0 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafa8dea640 1 -- 172.21.15.1:0/4046696185 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004b20 con 0x55a86fa8d3a0 2024-09-16T00:42:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafbe825740 1 -- 172.21.15.1:0/4046696185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8d3a0 msgr2=0x55a86fa8d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafba9e9640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafa00a3240 0x7fafa00a5700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fafac0b9770 tx=0x7fafac02a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafbe825740 1 --2- 172.21.15.1:0/4046696185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8d3a0 0x55a86fa8d780 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fafa4005cf0 tx=0x7fafa400b840 comp rx=0 tx=0).stop 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafbe825740 1 -- 172.21.15.1:0/4046696185 shutdown_connections 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafbe825740 1 --2- 172.21.15.1:0/4046696185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 0x55a86fa9ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafbe825740 1 --2- 172.21.15.1:0/4046696185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 0x55a86fa9c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafbe825740 1 --2- 172.21.15.1:0/4046696185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8d3a0 0x55a86fa8d780 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafbe825740 1 -- 172.21.15.1:0/4046696185 >> 172.21.15.1:0/4046696185 conn(0x55a86fa89a30 msgr2=0x55a86fa89e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafbe825740 1 -- 172.21.15.1:0/4046696185 shutdown_connections 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.886+0000 7fafbe825740 1 -- 172.21.15.1:0/4046696185 wait complete. 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafbe825740 1 -- start start 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafbe825740 1 Processor -- start 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafbe825740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8d3a0 0x55a86fa9c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafbe825740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86fa99690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafbe825740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 0x55a86fa99bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafbe825740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a86fa9f710 con 0x55a86fa8dcc0 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafbe825740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a86f9ae050 con 0x55a86fa8d3a0 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafbe825740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a86fa9f890 con 0x55a86fa9ca90 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafbe825740 10 client.0 _ll_register_callbacks cb 0x55a86f9ee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafba1e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86fa99690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafba1e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86fa99690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48214/0 (socket says 172.21.15.1:48214) 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafba1e8640 1 -- 172.21.15.1:0/3441668181 learned_addr learned my addr 172.21.15.1:0/3441668181 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.887+0000 7fafb99e7640 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8d3a0 0x55a86fa9c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafb99e7640 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 0x55a86fa99bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafba1e8640 1 -- 172.21.15.1:0/3441668181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 msgr2=0x55a86fa99bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafba1e8640 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 0x55a86fa99bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafba1e8640 1 -- 172.21.15.1:0/3441668181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8d3a0 msgr2=0x55a86fa9c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafba1e8640 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8d3a0 0x55a86fa9c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafba1e8640 1 -- 172.21.15.1:0/3441668181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86f7643c0 con 0x55a86fa8dcc0 2024-09-16T00:42:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafba1e8640 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86fa99690 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009ee0 tx=0x7fafb0011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c29e1915292a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7faf9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7faf9dffb640 1 -- 172.21.15.1:0/3441668181 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0019070 con 0x55a86fa8dcc0 2024-09-16T00:42:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7faf9dffb640 1 -- 172.21.15.1:0/3441668181 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fafb0015660 con 0x55a86fa8dcc0 2024-09-16T00:42:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafbe825740 1 -- 172.21.15.1:0/3441668181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa4005500 con 0x55a86fa8dcc0 2024-09-16T00:42:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7faf9dffb640 1 -- 172.21.15.1:0/3441668181 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fafb0025050 con 0x55a86fa8dcc0 2024-09-16T00:42:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafbe825740 10 client.18777 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.888+0000 7fafbe825740 1 -- 172.21.15.1:0/3441668181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a86faab180 con 0x55a86fa8dcc0 2024-09-16T00:42:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7faf9dffb640 1 -- 172.21.15.1:0/3441668181 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4005500 con 0x55a86fa8dcc0 2024-09-16T00:42:29.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7faf9dffb640 1 -- 172.21.15.1:0/3441668181 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fafb0014a60 con 0x55a86fa8dcc0 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7fafbe825740 10 client.18777 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7fafbe825740 10 client.18777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7fafbe825740 20 client.18777 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7fafbe825740 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a86faac630 0x55a86faaea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7fafbe825740 4 client.18777 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7fafbe825740 1 -- 172.21.15.1:0/3441668181 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86fa00850 con 0x55a86faac630 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7fafb99e7640 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a86faac630 0x55a86faaea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7fafb99e7640 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a86faac630 0x55a86faaea50 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428537d26613e388 server_cookie=da6788b99ce82449 in_seq=0 out_seq=0 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.889+0000 7faf9dffb640 10 client.18777 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7faf9dffb640 1 -- 172.21.15.1:0/3441668181 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a86fa00850 con 0x55a86faac630 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7faf9dffb640 10 client.18777 handle_command_reply: tid=0 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7fafbe825740 1 client.18777 shutdown 2024-09-16T00:42:29.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7fafbe825740 1 -- 172.21.15.1:0/3441668181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 msgr2=0x55a86fa99690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7fafbe825740 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86fa99690 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009ee0 tx=0x7fafb0011630 comp rx=0 tx=0).stop 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7fafbe825740 1 -- 172.21.15.1:0/3441668181 shutdown_connections 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7fafbe825740 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a86faac630 0x55a86faaea50 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7fafbe825740 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa9ca90 0x55a86fa99bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7fafbe825740 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86fa99690 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7fafbe825740 1 --2- 172.21.15.1:0/3441668181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8d3a0 0x55a86fa9c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7faf9dffb640 0 client.18777 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.890+0000 7fafbe825740 1 -- 172.21.15.1:0/3441668181 >> 172.21.15.1:0/3441668181 conn(0x55a86fa89a30 msgr2=0x55a86fa8f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.891+0000 7fafbe825740 1 -- 172.21.15.1:0/3441668181 shutdown_connections 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.891+0000 7fafbe825740 1 -- 172.21.15.1:0/3441668181 wait complete. 2024-09-16T00:42:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.891+0000 7fafbe825740 20 client.18777 trim_cache size 0 max 16384 2024-09-16T00:42:29.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafbe825740 1 Processor -- start 2024-09-16T00:42:29.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafbe825740 1 -- start start 2024-09-16T00:42:29.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafbe825740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa8d3a0 0x55a86faa89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafbe825740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 0x55a86fa98670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafbe825740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa9ca90 0x55a86fab4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafbe825740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41af4a0 con 0x55a86fa9ca90 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafbe825740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41acd70 con 0x55a86fa8dcc0 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafbe825740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41aabc0 con 0x55a86fa8d3a0 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafb99e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa9ca90 0x55a86fab4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafb99e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa9ca90 0x55a86fab4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48220/0 (socket says 172.21.15.1:48220) 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafb99e7640 1 -- 172.21.15.1:0/2191068928 learned_addr learned my addr 172.21.15.1:0/2191068928 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafba1e8640 1 --2- 172.21.15.1:0/2191068928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 0x55a86fa98670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafb99e7640 1 -- 172.21.15.1:0/2191068928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa8d3a0 msgr2=0x55a86faa89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafb99e7640 1 --2- 172.21.15.1:0/2191068928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa8d3a0 0x55a86faa89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.908+0000 7fafb99e7640 1 -- 172.21.15.1:0/2191068928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 msgr2=0x55a86fa98670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafb99e7640 1 --2- 172.21.15.1:0/2191068928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 0x55a86fa98670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafb99e7640 1 -- 172.21.15.1:0/2191068928 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86fa8e530 con 0x55a86fa9ca90 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafb99e7640 1 --2- 172.21.15.1:0/2191068928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa9ca90 0x55a86fab4320 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fafa4004510 tx=0x7fafa400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b726d9c252ac42c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7faf9ffff640 1 -- 172.21.15.1:0/2191068928 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4025040 con 0x55a86fa9ca90 2024-09-16T00:42:29.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7faf9ffff640 1 -- 172.21.15.1:0/2191068928 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4010b40 con 0x55a86fa9ca90 2024-09-16T00:42:29.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7faf9ffff640 1 -- 172.21.15.1:0/2191068928 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa402ca40 con 0x55a86fa9ca90 2024-09-16T00:42:29.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafbe825740 1 -- 172.21.15.1:0/2191068928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa9ca90 msgr2=0x55a86fab4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafbe825740 1 --2- 172.21.15.1:0/2191068928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa9ca90 0x55a86fab4320 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fafa4004510 tx=0x7fafa400bd10 comp rx=0 tx=0).stop 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafbe825740 1 -- 172.21.15.1:0/2191068928 shutdown_connections 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafbe825740 1 --2- 172.21.15.1:0/2191068928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa9ca90 0x55a86fab4320 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafbe825740 1 --2- 172.21.15.1:0/2191068928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa8dcc0 0x55a86fa98670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafbe825740 1 --2- 172.21.15.1:0/2191068928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa8d3a0 0x55a86faa89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.909+0000 7fafbe825740 1 -- 172.21.15.1:0/2191068928 >> 172.21.15.1:0/2191068928 conn(0x55a86fa89a30 msgr2=0x55a86fa97f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 -- 172.21.15.1:0/2191068928 shutdown_connections 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 -- 172.21.15.1:0/2191068928 wait complete. 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 -- start start 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 Processor -- start 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa8d3a0 0x55a86faa5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86faa03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa9ca90 0x55a86faa08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41ab230 con 0x55a86fa8dcc0 2024-09-16T00:42:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41ac6a0 con 0x55a86fa9ca90 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41ab900 con 0x55a86fa8d3a0 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafbe825740 10 client.0 _ll_register_callbacks cb 0x7fafb00094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafba1e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86faa03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafba1e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86faa03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48224/0 (socket says 172.21.15.1:48224) 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafba1e8640 1 -- 172.21.15.1:0/1678806986 learned_addr learned my addr 172.21.15.1:0/1678806986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.910+0000 7fafb99e7640 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa8d3a0 0x55a86faa5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafb99e7640 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa9ca90 0x55a86faa08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafba1e8640 1 -- 172.21.15.1:0/1678806986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa8d3a0 msgr2=0x55a86faa5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafba1e8640 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa8d3a0 0x55a86faa5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafba1e8640 1 -- 172.21.15.1:0/1678806986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa9ca90 msgr2=0x55a86faa08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafba1e8640 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa9ca90 0x55a86faa08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafba1e8640 1 -- 172.21.15.1:0/1678806986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86faa0ed0 con 0x55a86fa8dcc0 2024-09-16T00:42:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafb99e7640 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa9ca90 0x55a86faa08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafba1e8640 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86faa03a0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb0009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffe5bf7e897e6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7faf9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7faf9dffb640 1 -- 172.21.15.1:0/1678806986 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0019070 con 0x55a86fa8dcc0 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7faf9dffb640 1 -- 172.21.15.1:0/1678806986 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fafb0005bc0 con 0x55a86fa8dcc0 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafbe825740 1 -- 172.21.15.1:0/1678806986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa4010b40 con 0x55a86fa8dcc0 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7faf9dffb640 1 -- 172.21.15.1:0/1678806986 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fafb0037050 con 0x55a86fa8dcc0 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafbe825740 10 client.18783 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.911+0000 7fafbe825740 1 -- 172.21.15.1:0/1678806986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a86fb17100 con 0x55a86fa8dcc0 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7faf9dffb640 1 -- 172.21.15.1:0/1678806986 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4010b40 con 0x55a86fa8dcc0 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7faf9dffb640 1 -- 172.21.15.1:0/1678806986 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fafb00145a0 con 0x55a86fa8dcc0 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7fafbe825740 10 client.18783 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7fafbe825740 10 client.18783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7fafbe825740 20 client.18783 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7fafbe825740 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a86fb17870 0x55a86fb19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7fafbe825740 4 client.18783 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7fafbe825740 1 -- 172.21.15.1:0/1678806986 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a86f9fad60 con 0x55a86fb17870 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7fafb99e7640 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a86fb17870 0x55a86fb19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7fafb99e7640 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a86fb17870 0x55a86fb19c90 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648e7a92f9ea56e2 server_cookie=f7334b62b46b4352 in_seq=0 out_seq=0 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.912+0000 7faf9dffb640 10 client.18783 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7faf9dffb640 1 -- 172.21.15.1:0/1678806986 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a86f9fad60 con 0x55a86fb17870 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7faf9dffb640 10 client.18783 handle_command_reply: tid=0 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 client.18783 shutdown 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 -- 172.21.15.1:0/1678806986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 msgr2=0x55a86faa03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86faa03a0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb0009110 comp rx=0 tx=0).stop 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 -- 172.21.15.1:0/1678806986 shutdown_connections 2024-09-16T00:42:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55a86fb17870 0x55a86fb19c90 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86fa9ca90 0x55a86faa08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a86fa8dcc0 0x55a86faa03a0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7faf9dffb640 0 client.18783 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 --2- 172.21.15.1:0/1678806986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a86fa8d3a0 0x55a86faa5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 -- 172.21.15.1:0/1678806986 >> 172.21.15.1:0/1678806986 conn(0x55a86fa89a30 msgr2=0x55a86fa98780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 -- 172.21.15.1:0/1678806986 shutdown_connections 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 1 -- 172.21.15.1:0/1678806986 wait complete. 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.913+0000 7fafbe825740 20 client.18783 trim_cache size 0 max 16384 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:42:29.919 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:42:29.920 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:42:29.920 INFO:teuthology.orchestra.run.smithi001.stdout: "595cbb5d-9a91-4e1a-9fad-236572259159": { 2024-09-16T00:42:29.920 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:42:29.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "595cbb5d-9a91-4e1a-9fad-236572259159", 2024-09-16T00:42:29.920 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:42:29.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:29.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:29.920 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 -- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafa00a3240 msgr2=0x7fafa00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafa00a3240 0x7fafa00a5700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fafac0b9770 tx=0x7fafac02a950 comp rx=0 tx=0).stop 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 -- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 msgr2=0x7fafb419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 0x7fafb419ce60 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fafac02ae70 tx=0x7fafac02f5d0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 -- 172.21.15.1:0/1788090889 shutdown_connections 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafb4102ef0 0x7fafb419ce60 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb4102570 0x7fafb4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fafa00a3240 0x7fafa00a5700 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 --2- 172.21.15.1:0/1788090889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb4100e20 0x7fafb4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 -- 172.21.15.1:0/1788090889 >> 172.21.15.1:0/1788090889 conn(0x7fafb40fc7d0 msgr2=0x7fafb406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 -- 172.21.15.1:0/1788090889 shutdown_connections 2024-09-16T00:42:29.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:29.914+0000 7fafbc44c640 1 -- 172.21.15.1:0/1788090889 wait complete. 2024-09-16T00:42:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:29 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:42:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:29 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:29 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:42:29.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:29 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:29.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:29 smithi153 ceph-mon[35538]: pgmap v773: 129 pgs: 129 active+clean; 6.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:42:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:29 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:42:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:29 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:29 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:42:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:29 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:29 smithi019 ceph-mon[35186]: pgmap v773: 129 pgs: 129 active+clean; 6.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:42:30.781 INFO:tasks.cephfs.filesystem:scrub status for tag:595cbb5d-9a91-4e1a-9fad-236572259159 - {'path': '/', 'tag': '595cbb5d-9a91-4e1a-9fad-236572259159', 'options': 'recursive,force'} 2024-09-16T00:42:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:42:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:42:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:42:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:42:30.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:42:30.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:42:30.961 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:42:30.962 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:42:31.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:42:31.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:42:31.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:42:31.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:42:31.227 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.5 2024-09-16T00:42:31.228 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:42:31.496 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:32.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 -- 172.21.15.1:0/2034485903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 msgr2=0x7fed0c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 --2- 172.21.15.1:0/2034485903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 0x7fed0c1021c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009900 tx=0x7fecf402a410 comp rx=0 tx=0).stop 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 -- 172.21.15.1:0/2034485903 shutdown_connections 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 --2- 172.21.15.1:0/2034485903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c106f10 0x7fed0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 --2- 172.21.15.1:0/2034485903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c102790 0x7fed0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 --2- 172.21.15.1:0/2034485903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 0x7fed0c1021c0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 -- 172.21.15.1:0/2034485903 >> 172.21.15.1:0/2034485903 conn(0x7fed0c0fd8c0 msgr2=0x7fed0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 -- 172.21.15.1:0/2034485903 shutdown_connections 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 -- 172.21.15.1:0/2034485903 wait complete. 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 Processor -- start 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.989+0000 7fed12f22640 1 -- start start 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed12f22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 0x7fed0c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed12f22640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c102790 0x7fed0c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed12f22640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c106f10 0x7fed0c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed12f22640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c111f90 con 0x7fed0c101de0 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed12f22640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c111e10 con 0x7fed0c102790 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed12f22640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c112110 con 0x7fed0c106f10 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed10cbe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 0x7fed0c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed10cbe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 0x7fed0c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51976/0 (socket says 172.21.15.1:51976) 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed10cbe640 1 -- 172.21.15.1:0/3611025756 learned_addr learned my addr 172.21.15.1:0/3611025756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed0bfff640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c102790 0x7fed0c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed114bf640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c106f10 0x7fed0c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed10cbe640 1 -- 172.21.15.1:0/3611025756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c106f10 msgr2=0x7fed0c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed10cbe640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c106f10 0x7fed0c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed10cbe640 1 -- 172.21.15.1:0/3611025756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c102790 msgr2=0x7fed0c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed10cbe640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c102790 0x7fed0c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed10cbe640 1 -- 172.21.15.1:0/3611025756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c19d4c0 con 0x7fed0c101de0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.990+0000 7fed10cbe640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 0x7fed0c198510 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005ab0 tx=0x7fecf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.991+0000 7fed09ffb640 1 -- 172.21.15.1:0/3611025756 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf403f030 con 0x7fed0c101de0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.991+0000 7fed09ffb640 1 -- 172.21.15.1:0/3611025756 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4003ce0 con 0x7fed0c101de0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.991+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c19d750 con 0x7fed0c101de0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.991+0000 7fed09ffb640 1 -- 172.21.15.1:0/3611025756 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4035070 con 0x7fed0c101de0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.991+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c1a5090 con 0x7fed0c101de0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.992+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0c1032c0 con 0x7fed0c101de0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.993+0000 7fed09ffb640 1 -- 172.21.15.1:0/3611025756 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fecf4042030 con 0x7fed0c101de0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.994+0000 7fed09ffb640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fecf80a3240 0x7fecf80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.994+0000 7fed0bfff640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fecf80a3240 0x7fecf80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.994+0000 7fed09ffb640 1 -- 172.21.15.1:0/3611025756 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fecf40f0800 con 0x7fed0c101de0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.994+0000 7fed0bfff640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fecf80a3240 0x7fecf80a5700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fed0c1087d0 tx=0x7fecfc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:32.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:31.997+0000 7fed09ffb640 1 -- 172.21.15.1:0/3611025756 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fecf40bca70 con 0x7fed0c101de0 2024-09-16T00:42:32.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.210+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed0c1021c0 con 0x7fed0c101de0 2024-09-16T00:42:32.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.211+0000 7fed09ffb640 1 -- 172.21.15.1:0/3611025756 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fecf40c0d70 con 0x7fed0c101de0 2024-09-16T00:42:32.211 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:42:32.211 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:42:32.212 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:42:32.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fecf80a3240 msgr2=0x7fecf80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:32.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fecf80a3240 0x7fecf80a5700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fed0c1087d0 tx=0x7fecfc00a2d0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 msgr2=0x7fed0c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 0x7fed0c198510 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005ab0 tx=0x7fecf40042f0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 shutdown_connections 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fecf80a3240 0x7fecf80a5700 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c106f10 0x7fed0c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c102790 0x7fed0c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 --2- 172.21.15.1:0/3611025756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c101de0 0x7fed0c198510 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 >> 172.21.15.1:0/3611025756 conn(0x7fed0c0fd8c0 msgr2=0x7fed0c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 shutdown_connections 2024-09-16T00:42:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:32.213+0000 7fed12f22640 1 -- 172.21.15.1:0/3611025756 wait complete. 2024-09-16T00:42:32.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:32 smithi001 ceph-mon[30934]: pgmap v774: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:42:32.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:32 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3611025756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:32.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:32 smithi153 ceph-mon[35538]: pgmap v774: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:42:32.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:32 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3611025756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:32.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:32 smithi019 ceph-mon[35186]: pgmap v774: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:42:32.857 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:32 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3611025756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:32.938 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0' 2024-09-16T00:42:32.938 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0 2024-09-16T00:42:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.683+0000 7fc244a0f640 1 -- 172.21.15.1:0/3875603618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074590 msgr2=0x7fc240074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.683+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3875603618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074590 0x7fc240074970 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc230009900 tx=0x7fc23002a480 comp rx=0 tx=0).stop 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.684+0000 7fc244a0f640 1 -- 172.21.15.1:0/3875603618 shutdown_connections 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.684+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3875603618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc240112b90 0x7fc240114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.684+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3875603618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc240074eb0 0x7fc240112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.684+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3875603618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074590 0x7fc240074970 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.684+0000 7fc244a0f640 1 -- 172.21.15.1:0/3875603618 >> 172.21.15.1:0/3875603618 conn(0x7fc2400fd8c0 msgr2=0x7fc2400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.684+0000 7fc244a0f640 1 -- 172.21.15.1:0/3875603618 shutdown_connections 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.684+0000 7fc244a0f640 1 -- 172.21.15.1:0/3875603618 wait complete. 2024-09-16T00:42:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc244a0f640 1 Processor -- start 2024-09-16T00:42:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc244a0f640 1 -- start start 2024-09-16T00:42:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc244a0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc240074590 0x7fc2401a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc244a0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074eb0 0x7fc2401a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc244a0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc240112b90 0x7fc24019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc244a0f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401185a0 con 0x7fc240074eb0 2024-09-16T00:42:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc244a0f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc240118420 con 0x7fc240074590 2024-09-16T00:42:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc244a0f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc240118720 con 0x7fc240112b90 2024-09-16T00:42:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074eb0 0x7fc2401a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074eb0 0x7fc2401a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52002/0 (socket says 172.21.15.1:52002) 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23dd9b640 1 -- 172.21.15.1:0/3445272641 learned_addr learned my addr 172.21.15.1:0/3445272641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23e59c640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc240074590 0x7fc2401a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc240112b90 0x7fc24019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23dd9b640 1 -- 172.21.15.1:0/3445272641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc240112b90 msgr2=0x7fc24019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc240112b90 0x7fc24019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23dd9b640 1 -- 172.21.15.1:0/3445272641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc240074590 msgr2=0x7fc2401a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc240074590 0x7fc2401a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.685+0000 7fc23dd9b640 1 -- 172.21.15.1:0/3445272641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24019f000 con 0x7fc240074eb0 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.686+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074eb0 0x7fc2401a4c50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc23400c920 tx=0x7fc23400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.686+0000 7fc22f7fe640 1 -- 172.21.15.1:0/3445272641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234007c70 con 0x7fc240074eb0 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.686+0000 7fc244a0f640 1 -- 172.21.15.1:0/3445272641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24019f2f0 con 0x7fc240074eb0 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.686+0000 7fc22f7fe640 1 -- 172.21.15.1:0/3445272641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc234007e10 con 0x7fc240074eb0 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.686+0000 7fc22f7fe640 1 -- 172.21.15.1:0/3445272641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234018890 con 0x7fc240074eb0 2024-09-16T00:42:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.686+0000 7fc244a0f640 1 -- 172.21.15.1:0/3445272641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2401ab5e0 con 0x7fc240074eb0 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.688+0000 7fc22f7fe640 1 -- 172.21.15.1:0/3445272641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fc234018a70 con 0x7fc240074eb0 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc246de8740 1 Processor -- start 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc246de8740 1 -- start start 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc246de8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 0x55ad874e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc246de8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad874f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc246de8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874f0940 0x55ad874f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc246de8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad8744b2a0 con 0x55ad874e6080 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc246de8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad873f2b10 con 0x55ad874f0940 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc246de8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad87447260 con 0x55ad874e5760 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc23e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad874f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc23e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad874f0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52018/0 (socket says 172.21.15.1:52018) 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc23e59c640 1 -- 172.21.15.1:0/515951849 learned_addr learned my addr 172.21.15.1:0/515951849 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.689+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/515951849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 0x55ad874e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc22f7fe640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2280a3240 0x7fc2280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/515951849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874f0940 0x55ad874f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc23e59c640 1 -- 172.21.15.1:0/515951849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 msgr2=0x55ad874e5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc23e59c640 1 --2- 172.21.15.1:0/515951849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 0x55ad874e5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc23e59c640 1 -- 172.21.15.1:0/515951849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874f0940 msgr2=0x55ad874f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc23e59c640 1 --2- 172.21.15.1:0/515951849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874f0940 0x55ad874f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc23e59c640 1 -- 172.21.15.1:0/515951849 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad871bcb90 con 0x55ad874e6080 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2280a3240 0x7fc2280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc22f7fe640 1 -- 172.21.15.1:0/3445272641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc2340cff70 con 0x7fc240074eb0 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc23e59c640 1 --2- 172.21.15.1:0/515951849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad874f0400 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc230009900 tx=0x7fc230004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26528ea352db4eb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.690+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2280a3240 0x7fc2280a5700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc240076600 tx=0x7fc238004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:33.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc22cff9640 1 -- 172.21.15.1:0/515951849 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230035030 con 0x55ad874e6080 2024-09-16T00:42:33.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc22cff9640 1 -- 172.21.15.1:0/515951849 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc230005520 con 0x55ad874e6080 2024-09-16T00:42:33.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc22cff9640 1 -- 172.21.15.1:0/515951849 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230004e60 con 0x55ad874e6080 2024-09-16T00:42:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc246de8740 1 -- 172.21.15.1:0/515951849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 msgr2=0x55ad874f0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc246de8740 1 --2- 172.21.15.1:0/515951849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad874f0400 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc230009900 tx=0x7fc230004170 comp rx=0 tx=0).stop 2024-09-16T00:42:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc246de8740 1 -- 172.21.15.1:0/515951849 shutdown_connections 2024-09-16T00:42:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc246de8740 1 --2- 172.21.15.1:0/515951849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874f0940 0x55ad874f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc246de8740 1 --2- 172.21.15.1:0/515951849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad874f0400 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc246de8740 1 --2- 172.21.15.1:0/515951849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 0x55ad874e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.691+0000 7fc246de8740 1 -- 172.21.15.1:0/515951849 >> 172.21.15.1:0/515951849 conn(0x55ad874e1cf0 msgr2=0x55ad874e20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 -- 172.21.15.1:0/515951849 shutdown_connections 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 -- 172.21.15.1:0/515951849 wait complete. 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 -- start start 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 Processor -- start 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 0x55ad874ed840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e6080 0x55ad874edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad874f3510 con 0x55ad874f0940 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad873d36e0 con 0x55ad874e6080 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad874f3690 con 0x55ad874e5760 2024-09-16T00:42:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc246de8740 10 client.0 _ll_register_callbacks cb 0x55ad873f7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.692+0000 7fc23ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874ee2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52030/0 (socket says 172.21.15.1:52030) 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23ed9d640 1 -- 172.21.15.1:0/651668095 learned_addr learned my addr 172.21.15.1:0/651668095 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e6080 0x55ad874edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23e59c640 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 0x55ad874ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23ed9d640 1 -- 172.21.15.1:0/651668095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 msgr2=0x55ad874ed840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 0x55ad874ed840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23ed9d640 1 -- 172.21.15.1:0/651668095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e6080 msgr2=0x55ad874edd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e6080 0x55ad874edd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23ed9d640 1 -- 172.21.15.1:0/651668095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad874fca30 con 0x55ad874f0940 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23e59c640 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 0x55ad874ed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874ee2c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc2380128d0 tx=0x7fc238004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e97fe545f170f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc225ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc225ffb640 1 -- 172.21.15.1:0/651668095 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23801dd70 con 0x55ad874f0940 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc225ffb640 1 -- 172.21.15.1:0/651668095 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc238027470 con 0x55ad874f0940 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc246de8740 1 -- 172.21.15.1:0/651668095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc230005520 con 0x55ad874f0940 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc225ffb640 1 -- 172.21.15.1:0/651668095 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc238011e90 con 0x55ad874f0940 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc246de8740 10 client.18801 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.693+0000 7fc246de8740 1 -- 172.21.15.1:0/651668095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad874ff400 con 0x55ad874f0940 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc225ffb640 1 -- 172.21.15.1:0/651668095 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc230005520 con 0x55ad874f0940 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc225ffb640 1 -- 172.21.15.1:0/651668095 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc23800f040 con 0x55ad874f0940 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc246de8740 10 client.18801 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc246de8740 10 client.18801 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc246de8740 20 client.18801 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc246de8740 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad875008c0 0x55ad87502d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc246de8740 4 client.18801 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc246de8740 1 -- 172.21.15.1:0/651668095 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad874589d0 con 0x55ad875008c0 2024-09-16T00:42:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc23e59c640 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad875008c0 0x55ad87502d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc225ffb640 10 client.18801 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.694+0000 7fc23e59c640 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad875008c0 0x55ad87502d10 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80201c6d77d60be server_cookie=13a1d3497be60589 in_seq=0 out_seq=0 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc225ffb640 1 -- 172.21.15.1:0/651668095 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad874589d0 con 0x55ad875008c0 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc225ffb640 10 client.18801 handle_command_reply: tid=0 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 client.18801 shutdown 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 -- 172.21.15.1:0/651668095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 msgr2=0x55ad874ee2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874ee2c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc2380128d0 tx=0x7fc238004ab0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 -- 172.21.15.1:0/651668095 shutdown_connections 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad875008c0 0x55ad87502d10 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874ee2c0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e6080 0x55ad874edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 --2- 172.21.15.1:0/651668095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e5760 0x55ad874ed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc225ffb640 0 client.18801 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 -- 172.21.15.1:0/651668095 >> 172.21.15.1:0/651668095 conn(0x55ad874e1cf0 msgr2=0x55ad874e79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:33.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.695+0000 7fc246de8740 1 -- 172.21.15.1:0/651668095 shutdown_connections 2024-09-16T00:42:33.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.696+0000 7fc246de8740 1 -- 172.21.15.1:0/651668095 wait complete. 2024-09-16T00:42:33.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.696+0000 7fc246de8740 20 client.18801 trim_cache size 0 max 16384 2024-09-16T00:42:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.712+0000 7fc246de8740 1 Processor -- start 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc246de8740 1 -- start start 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc246de8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 0x55ad874e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc246de8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad875085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc246de8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874f0940 0x55ad87508b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc246de8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401b7cc0 con 0x55ad874e6080 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc246de8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401b5590 con 0x55ad874e5760 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc246de8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401b33e0 con 0x55ad874f0940 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad875085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad875085e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52036/0 (socket says 172.21.15.1:52036) 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23dd9b640 1 -- 172.21.15.1:0/137072458 learned_addr learned my addr 172.21.15.1:0/137072458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23dd9b640 1 -- 172.21.15.1:0/137072458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874f0940 msgr2=0x55ad87508b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23e59c640 1 --2- 172.21.15.1:0/137072458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 0x55ad874e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/137072458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874f0940 0x55ad87508b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/137072458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874f0940 0x55ad87508b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23dd9b640 1 -- 172.21.15.1:0/137072458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 msgr2=0x55ad874e78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/137072458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 0x55ad874e78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23dd9b640 1 -- 172.21.15.1:0/137072458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad874ec7f0 con 0x55ad874e6080 2024-09-16T00:42:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/137072458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad875085e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc2340cd610 tx=0x7fc23400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe4ec261d07cf40 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.713+0000 7fc227fff640 1 -- 172.21.15.1:0/137072458 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340a4da0 con 0x55ad874e6080 2024-09-16T00:42:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc227fff640 1 -- 172.21.15.1:0/137072458 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2340cd720 con 0x55ad874e6080 2024-09-16T00:42:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc227fff640 1 -- 172.21.15.1:0/137072458 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340bc070 con 0x55ad874e6080 2024-09-16T00:42:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc246de8740 1 -- 172.21.15.1:0/137072458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 msgr2=0x55ad875085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc246de8740 1 --2- 172.21.15.1:0/137072458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad875085e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc2340cd610 tx=0x7fc23400ff30 comp rx=0 tx=0).stop 2024-09-16T00:42:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc246de8740 1 -- 172.21.15.1:0/137072458 shutdown_connections 2024-09-16T00:42:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc246de8740 1 --2- 172.21.15.1:0/137072458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874f0940 0x55ad87508b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc246de8740 1 --2- 172.21.15.1:0/137072458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874e6080 0x55ad875085e0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc246de8740 1 --2- 172.21.15.1:0/137072458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 0x55ad874e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc246de8740 1 -- 172.21.15.1:0/137072458 >> 172.21.15.1:0/137072458 conn(0x55ad874e1cf0 msgr2=0x55ad874e71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc246de8740 1 -- 172.21.15.1:0/137072458 shutdown_connections 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.714+0000 7fc246de8740 1 -- 172.21.15.1:0/137072458 wait complete. 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc246de8740 1 -- start start 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc246de8740 1 Processor -- start 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc246de8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 0x55ad874f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc246de8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e6080 0x55ad874f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc246de8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc246de8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401b3a50 con 0x55ad874f0940 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc246de8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401b4ec0 con 0x55ad874e5760 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc246de8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401b4120 con 0x55ad874e6080 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc246de8740 10 client.0 _ll_register_callbacks cb 0x7fc238012480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52044/0 (socket says 172.21.15.1:52044) 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23ed9d640 1 -- 172.21.15.1:0/4275011295 learned_addr learned my addr 172.21.15.1:0/4275011295 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23e59c640 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 0x55ad874f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e6080 0x55ad874f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23ed9d640 1 -- 172.21.15.1:0/4275011295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e6080 msgr2=0x55ad874f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e6080 0x55ad874f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23ed9d640 1 -- 172.21.15.1:0/4275011295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 msgr2=0x55ad874f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 0x55ad874f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23ed9d640 1 -- 172.21.15.1:0/4275011295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad8756b5c0 con 0x55ad874f0940 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23e59c640 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 0x55ad874f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.715+0000 7fc23dd9b640 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e6080 0x55ad874f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc23ed9d640 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874f5d20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc2380128a0 tx=0x7fc238011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a071dab923d8963e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc225ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:42:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc225ffb640 1 -- 172.21.15.1:0/4275011295 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238039070 con 0x55ad874f0940 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc225ffb640 1 -- 172.21.15.1:0/4275011295 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fc23801e660 con 0x55ad874f0940 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc246de8740 1 -- 172.21.15.1:0/4275011295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2340cd720 con 0x55ad874f0940 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc225ffb640 1 -- 172.21.15.1:0/4275011295 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2380200d0 con 0x55ad874f0940 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc246de8740 10 client.18807 fetch_fsmap learned FSMap version 25 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc246de8740 1 -- 172.21.15.1:0/4275011295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad8756c780 con 0x55ad874f0940 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc225ffb640 1 -- 172.21.15.1:0/4275011295 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2340cd720 con 0x55ad874f0940 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.716+0000 7fc225ffb640 1 -- 172.21.15.1:0/4275011295 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc238027850 con 0x55ad874f0940 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.717+0000 7fc246de8740 10 client.18807 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.717+0000 7fc246de8740 10 client.18807 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.717+0000 7fc246de8740 20 client.18807 populate_metadata read hostname 'smithi001' 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.717+0000 7fc246de8740 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad8756cf10 0x55ad8756f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.717+0000 7fc246de8740 4 client.18807 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}] 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.717+0000 7fc246de8740 1 -- 172.21.15.1:0/4275011295 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}) -- 0x55ad87452d20 con 0x55ad8756cf10 2024-09-16T00:42:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.717+0000 7fc23e59c640 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad8756cf10 0x55ad8756f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:33.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.717+0000 7fc23e59c640 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad8756cf10 0x55ad8756f330 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3334ac4ec4e35d server_cookie=f6dd5c09221904f5 in_seq=0 out_seq=0 2024-09-16T00:42:33.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.717+0000 7fc225ffb640 10 client.18807 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc225ffb640 1 -- 172.21.15.1:0/4275011295 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ad87452d20 con 0x55ad8756cf10 2024-09-16T00:42:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc225ffb640 10 client.18807 handle_command_reply: tid=0 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 client.18807 shutdown 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 -- 172.21.15.1:0/4275011295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 msgr2=0x55ad874f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874f5d20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc2380128a0 tx=0x7fc238011fd0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 -- 172.21.15.1:0/4275011295 shutdown_connections 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55ad8756cf10 0x55ad8756f330 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad874f0940 0x55ad874f5d20 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad874e6080 0x55ad874f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 --2- 172.21.15.1:0/4275011295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad874e5760 0x55ad874f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 -- 172.21.15.1:0/4275011295 >> 172.21.15.1:0/4275011295 conn(0x55ad874e1cf0 msgr2=0x55ad874ee020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc225ffb640 0 client.18807 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 -- 172.21.15.1:0/4275011295 shutdown_connections 2024-09-16T00:42:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.728+0000 7fc246de8740 1 -- 172.21.15.1:0/4275011295 wait complete. 2024-09-16T00:42:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc246de8740 20 client.18807 trim_cache size 0 max 16384 2024-09-16T00:42:33.731 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:42:33.731 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:42:33.731 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:42:33.731 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 50, 2024-09-16T00:42:33.731 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:42:33.731 INFO:teuthology.orchestra.run.smithi001.stdout: "93c3a287": { 2024-09-16T00:42:33.731 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 50, 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 146.5, 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:42:33.732 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:42:33.733 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:42:33.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:42:33.733 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:42:33.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:33.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:33.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:33.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:33.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:42:33.734 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 -- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2280a3240 msgr2=0x7fc2280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2280a3240 0x7fc2280a5700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc240076600 tx=0x7fc238004a40 comp rx=0 tx=0).stop 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 -- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074eb0 msgr2=0x7fc2401a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074eb0 0x7fc2401a4c50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc23400c920 tx=0x7fc23400cde0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 -- 172.21.15.1:0/3445272641 shutdown_connections 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fc2280a3240 0x7fc2280a5700 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc240112b90 0x7fc24019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc240074eb0 0x7fc2401a4c50 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 --2- 172.21.15.1:0/3445272641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc240074590 0x7fc2401a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 -- 172.21.15.1:0/3445272641 >> 172.21.15.1:0/3445272641 conn(0x7fc2400fd8c0 msgr2=0x7fc2400fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 -- 172.21.15.1:0/3445272641 shutdown_connections 2024-09-16T00:42:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:33.729+0000 7fc244a0f640 1 -- 172.21.15.1:0/3445272641 wait complete. 2024-09-16T00:42:33.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:33 smithi001 ceph-mon[30934]: pgmap v775: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:42:33.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:33 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:42:34.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:33 smithi019 ceph-mon[35186]: pgmap v775: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:42:34.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:33 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:42:34.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:33 smithi153 ceph-mon[35538]: pgmap v775: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:42:34.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:33 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:42:34.453 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '93c3a287', quiesce took 3.2 seconds. Will release after: 23.3 2024-09-16T00:42:36.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:35 smithi019 ceph-mon[35186]: pgmap v776: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-16T00:42:36.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:35 smithi001 ceph-mon[30934]: pgmap v776: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-16T00:42:36.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:35 smithi153 ceph-mon[35538]: pgmap v776: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-16T00:42:38.013 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:37 smithi153 ceph-mon[35538]: pgmap v777: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:42:38.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:37 smithi019 ceph-mon[35186]: pgmap v777: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:42:38.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:37 smithi001 ceph-mon[30934]: pgmap v777: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:42:40.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:39 smithi019 ceph-mon[35186]: pgmap v778: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:42:40.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:39 smithi001 ceph-mon[30934]: pgmap v778: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:42:40.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:39 smithi153 ceph-mon[35538]: pgmap v778: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:42:42.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:41 smithi019 ceph-mon[35186]: pgmap v779: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:42:42.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:41 smithi001 ceph-mon[30934]: pgmap v779: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:42:42.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:41 smithi153 ceph-mon[35538]: pgmap v779: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:42:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:43 smithi019 ceph-mon[35186]: pgmap v780: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-16T00:42:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:43 smithi001 ceph-mon[30934]: pgmap v780: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-16T00:42:44.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:43 smithi153 ceph-mon[35538]: pgmap v780: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-16T00:42:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:45 smithi019 ceph-mon[35186]: pgmap v781: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-16T00:42:46.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:45 smithi001 ceph-mon[30934]: pgmap v781: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-16T00:42:46.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:45 smithi153 ceph-mon[35538]: pgmap v781: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-16T00:42:48.004 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:47 smithi001 ceph-mon[30934]: pgmap v782: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-16T00:42:48.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:47 smithi019 ceph-mon[35186]: pgmap v782: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-16T00:42:48.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:47 smithi153 ceph-mon[35538]: pgmap v782: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-16T00:42:50.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:49 smithi019 ceph-mon[35186]: pgmap v783: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:42:50.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:49 smithi001 ceph-mon[30934]: pgmap v783: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:42:50.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:49 smithi153 ceph-mon[35538]: pgmap v783: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:42:52.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:51 smithi019 ceph-mon[35186]: pgmap v784: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:42:52.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:51 smithi001 ceph-mon[30934]: pgmap v784: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:42:52.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:51 smithi153 ceph-mon[35538]: pgmap v784: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:42:54.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:53 smithi019 ceph-mon[35186]: pgmap v785: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:42:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:53 smithi001 ceph-mon[30934]: pgmap v785: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:42:54.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:53 smithi153 ceph-mon[35538]: pgmap v785: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:42:56.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:55 smithi019 ceph-mon[35186]: pgmap v786: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T00:42:56.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:55 smithi001 ceph-mon[30934]: pgmap v786: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T00:42:56.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:55 smithi153 ceph-mon[35538]: pgmap v786: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T00:42:57.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:42:58.028 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:58.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:57 smithi019 ceph-mon[35186]: pgmap v787: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:42:58.151 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:57 smithi001 ceph-mon[30934]: pgmap v787: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:42:58.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:57 smithi153 ceph-mon[35538]: pgmap v787: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:42:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 -- 172.21.15.1:0/454230672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 msgr2=0x7efd0810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 --2- 172.21.15.1:0/454230672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 0x7efd0810ec40 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7efd00009930 tx=0x7efd0002a420 comp rx=0 tx=0).stop 2024-09-16T00:42:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 -- 172.21.15.1:0/454230672 shutdown_connections 2024-09-16T00:42:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 --2- 172.21.15.1:0/454230672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 0x7efd0810ec40 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 --2- 172.21.15.1:0/454230672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd081036e0 0x7efd08107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 --2- 172.21.15.1:0/454230672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd08102d30 0x7efd08103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 -- 172.21.15.1:0/454230672 >> 172.21.15.1:0/454230672 conn(0x7efd080fc7d0 msgr2=0x7efd080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 -- 172.21.15.1:0/454230672 shutdown_connections 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 -- 172.21.15.1:0/454230672 wait complete. 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 Processor -- start 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.559+0000 7efd0f568640 1 -- start start 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0f568640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd08102d30 0x7efd08100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0f568640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd081036e0 0x7efd08100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0f568640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 0x7efd08101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0f568640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efd081120a0 con 0x7efd081082a0 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0f568640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd08111f20 con 0x7efd081036e0 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0f568640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd08112220 con 0x7efd08102d30 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0db05640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 0x7efd08101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0db05640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 0x7efd08101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45034/0 (socket says 172.21.15.1:45034) 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0db05640 1 -- 172.21.15.1:0/1575811752 learned_addr learned my addr 172.21.15.1:0/1575811752 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0cb03640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd081036e0 0x7efd08100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0d304640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd08102d30 0x7efd08100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0db05640 1 -- 172.21.15.1:0/1575811752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd08102d30 msgr2=0x7efd08100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0db05640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd08102d30 0x7efd08100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0db05640 1 -- 172.21.15.1:0/1575811752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd081036e0 msgr2=0x7efd08100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0db05640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd081036e0 0x7efd08100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0db05640 1 -- 172.21.15.1:0/1575811752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd081a6980 con 0x7efd081082a0 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0d304640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd08102d30 0x7efd08100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd0db05640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 0x7efd08101410 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7efd00005ab0 tx=0x7efd000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.560+0000 7efd067fc640 1 -- 172.21.15.1:0/1575811752 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0003f030 con 0x7efd081082a0 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.561+0000 7efd067fc640 1 -- 172.21.15.1:0/1575811752 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd00003ce0 con 0x7efd081082a0 2024-09-16T00:42:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.561+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd081a6c10 con 0x7efd081082a0 2024-09-16T00:42:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.561+0000 7efd067fc640 1 -- 172.21.15.1:0/1575811752 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00035070 con 0x7efd081082a0 2024-09-16T00:42:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.561+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd081a7140 con 0x7efd081082a0 2024-09-16T00:42:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.562+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd08103110 con 0x7efd081082a0 2024-09-16T00:42:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.563+0000 7efd067fc640 1 -- 172.21.15.1:0/1575811752 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7efd00042030 con 0x7efd081082a0 2024-09-16T00:42:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.563+0000 7efd067fc640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efcf00a32c0 0x7efcf00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.564+0000 7efd0d304640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efcf00a32c0 0x7efcf00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.564+0000 7efd067fc640 1 -- 172.21.15.1:0/1575811752 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7efd000f04b0 con 0x7efd081082a0 2024-09-16T00:42:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.564+0000 7efd0d304640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efcf00a32c0 0x7efcf00a5780 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efcf8005fd0 tx=0x7efcf8009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.568+0000 7efd067fc640 1 -- 172.21.15.1:0/1575811752 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd000bcdd0 con 0x7efd081082a0 2024-09-16T00:42:58.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.779+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd0810a2b0 con 0x7efd081082a0 2024-09-16T00:42:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.780+0000 7efd067fc640 1 -- 172.21.15.1:0/1575811752 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7efd000bc6d0 con 0x7efd081082a0 2024-09-16T00:42:58.780 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:42:58.781 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:42:58.785 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.782+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efcf00a32c0 msgr2=0x7efcf00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efcf00a32c0 0x7efcf00a5780 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efcf8005fd0 tx=0x7efcf8009520 comp rx=0 tx=0).stop 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 msgr2=0x7efd08101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 0x7efd08101410 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7efd00005ab0 tx=0x7efd000042f0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 shutdown_connections 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd081082a0 0x7efd08101410 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd081036e0 0x7efd08100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7efcf00a32c0 0x7efcf00a5780 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 --2- 172.21.15.1:0/1575811752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd08102d30 0x7efd08100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 >> 172.21.15.1:0/1575811752 conn(0x7efd080fc7d0 msgr2=0x7efd08106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:58.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 shutdown_connections 2024-09-16T00:42:58.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:42:58.783+0000 7efd0f568640 1 -- 172.21.15.1:0/1575811752 wait complete. 2024-09-16T00:42:59.597 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 93c3a287 --release --await' 2024-09-16T00:42:59.598 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 93c3a287 --release --await 2024-09-16T00:42:59.859 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:59.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:59 smithi001 ceph-mon[30934]: pgmap v788: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:42:59.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:42:59 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1575811752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:00.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:59 smithi019 ceph-mon[35186]: pgmap v788: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:43:00.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:42:59 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1575811752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:00.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:59 smithi153 ceph-mon[35538]: pgmap v788: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:43:00.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:42:59 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1575811752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:00.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 -- 172.21.15.1:0/3172445222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 msgr2=0x7f287c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 --2- 172.21.15.1:0/3172445222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 0x7f287c10d950 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f2874009cd0 tx=0x7f287402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 -- 172.21.15.1:0/3172445222 shutdown_connections 2024-09-16T00:43:00.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 --2- 172.21.15.1:0/3172445222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 0x7f287c10d950 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 --2- 172.21.15.1:0/3172445222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f287c100a80 0x7f287c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 --2- 172.21.15.1:0/3172445222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c106a80 0x7f287c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 -- 172.21.15.1:0/3172445222 >> 172.21.15.1:0/3172445222 conn(0x7f287c0fc7a0 msgr2=0x7f287c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 -- 172.21.15.1:0/3172445222 shutdown_connections 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 -- 172.21.15.1:0/3172445222 wait complete. 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 Processor -- start 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.324+0000 7f2883780640 1 -- start start 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2883780640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c100a80 0x7f287c0fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2883780640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 0x7f287c0fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2883780640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f287c106a80 0x7f287c0fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2883780640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f287c110ea0 con 0x7f287c101420 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2883780640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f287c110d20 con 0x7f287c106a80 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2883780640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f287c111020 con 0x7f287c100a80 2024-09-16T00:43:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2880d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 0x7f287c0fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2880d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 0x7f287c0fd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42498/0 (socket says 172.21.15.1:42498) 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2880d1b640 1 -- 172.21.15.1:0/4092967001 learned_addr learned my addr 172.21.15.1:0/4092967001 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2881d1d640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f287c106a80 0x7f287c0fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f288151c640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c100a80 0x7f287c0fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2880d1b640 1 -- 172.21.15.1:0/4092967001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c100a80 msgr2=0x7f287c0fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2880d1b640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c100a80 0x7f287c0fcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2880d1b640 1 -- 172.21.15.1:0/4092967001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f287c106a80 msgr2=0x7f287c0fd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2880d1b640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f287c106a80 0x7f287c0fd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2880d1b640 1 -- 172.21.15.1:0/4092967001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c1a6900 con 0x7f287c101420 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f288151c640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c100a80 0x7f287c0fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.325+0000 7f2880d1b640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 0x7f287c0fd2a0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f286c00d950 tx=0x7f286c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.326+0000 7f28727fc640 1 -- 172.21.15.1:0/4092967001 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c017070 con 0x7f287c101420 2024-09-16T00:43:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.326+0000 7f28727fc640 1 -- 172.21.15.1:0/4092967001 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c009c30 con 0x7f287c101420 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.326+0000 7f2883780640 1 -- 172.21.15.1:0/4092967001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c1a6bf0 con 0x7f287c101420 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.326+0000 7f28727fc640 1 -- 172.21.15.1:0/4092967001 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c00f6b0 con 0x7f287c101420 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.326+0000 7f2883780640 1 -- 172.21.15.1:0/4092967001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c1a7130 con 0x7f287c101420 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.328+0000 7f28727fc640 1 -- 172.21.15.1:0/4092967001 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f286c0109d0 con 0x7f287c101420 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.328+0000 7f2885b59740 1 Processor -- start 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2885b59740 1 -- start start 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2885b59740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de16b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2885b59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de175d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2885b59740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de178630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2885b59740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5630de0d0620 con 0x5630de176240 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2885b59740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630de077e50 con 0x5630de16b060 2024-09-16T00:43:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2885b59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5630de0cc5e0 con 0x5630de16b980 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2880d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de178630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2880d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de178630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42500/0 (socket says 172.21.15.1:42500) 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2880d1b640 1 -- 172.21.15.1:0/883271460 learned_addr learned my addr 172.21.15.1:0/883271460 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2881d1d640 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de16b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f288151c640 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de175d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f28727fc640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f28680a3240 0x7f28680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2880d1b640 1 -- 172.21.15.1:0/883271460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 msgr2=0x5630de175d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2880d1b640 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de175d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2880d1b640 1 -- 172.21.15.1:0/883271460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 msgr2=0x5630de16b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2880d1b640 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de16b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2880d1b640 1 -- 172.21.15.1:0/883271460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630de068d40 con 0x5630de176240 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.329+0000 7f2881d1d640 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de16b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.330+0000 7f2881d1d640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f28680a3240 0x7f28680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.330+0000 7f28727fc640 1 -- 172.21.15.1:0/4092967001 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f286c0cf9c0 con 0x7f287c101420 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.330+0000 7f2880d1b640 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de178630 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f286c079ff0 tx=0x7f286c091060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e209f0f5ed310175 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.330+0000 7f28677fe640 1 -- 172.21.15.1:0/883271460 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c09b6c0 con 0x5630de176240 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.330+0000 7f28677fe640 1 -- 172.21.15.1:0/883271460 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c009de0 con 0x5630de176240 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.330+0000 7f28677fe640 1 -- 172.21.15.1:0/883271460 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c09b4e0 con 0x5630de176240 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 -- 172.21.15.1:0/883271460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 msgr2=0x5630de178630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2881d1d640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f28680a3240 0x7f28680a5700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2874006010 tx=0x7f28740095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de178630 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f286c079ff0 tx=0x7f286c091060 comp rx=0 tx=0).stop 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 -- 172.21.15.1:0/883271460 shutdown_connections 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de178630 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de175d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 --2- 172.21.15.1:0/883271460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de16b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 -- 172.21.15.1:0/883271460 >> 172.21.15.1:0/883271460 conn(0x5630de1675f0 msgr2=0x5630de1679d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 -- 172.21.15.1:0/883271460 shutdown_connections 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 -- 172.21.15.1:0/883271460 wait complete. 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 -- start start 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.331+0000 7f2885b59740 1 Processor -- start 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2885b59740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de1730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2885b59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de173610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2885b59740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de173b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2885b59740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5630de178e10 con 0x5630de176240 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2885b59740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630de058f10 con 0x5630de16b060 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2885b59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5630de178f90 con 0x5630de16b980 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2885b59740 10 client.0 _ll_register_callbacks cb 0x5630de07cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2881d1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de173b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2881d1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de173b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42502/0 (socket says 172.21.15.1:42502) 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2881d1d640 1 -- 172.21.15.1:0/2455088339 learned_addr learned my addr 172.21.15.1:0/2455088339 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f288151c640 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de1730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2880d1b640 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de173610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2881d1d640 1 -- 172.21.15.1:0/2455088339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 msgr2=0x5630de173610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2881d1d640 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de173610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2881d1d640 1 -- 172.21.15.1:0/2455088339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 msgr2=0x5630de1730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2881d1d640 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de1730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2881d1d640 1 -- 172.21.15.1:0/2455088339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ddddf580 con 0x5630de176240 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2880d1b640 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de173610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2881d1d640 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de173b50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f2874035000 tx=0x7f287402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb572e4c43e754f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2864ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2864ff9640 1 -- 172.21.15.1:0/2455088339 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874004750 con 0x5630de176240 2024-09-16T00:43:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2864ff9640 1 -- 172.21.15.1:0/2455088339 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f28740305a0 con 0x5630de176240 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.332+0000 7f2885b59740 1 -- 172.21.15.1:0/2455088339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f286c009de0 con 0x5630de176240 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2864ff9640 1 -- 172.21.15.1:0/2455088339 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28740307a0 con 0x5630de176240 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2885b59740 10 client.18825 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2885b59740 1 -- 172.21.15.1:0/2455088339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630ddea9960 con 0x5630de176240 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2864ff9640 1 -- 172.21.15.1:0/2455088339 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c009de0 con 0x5630de176240 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2864ff9640 1 -- 172.21.15.1:0/2455088339 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2874030070 con 0x5630de176240 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2885b59740 10 client.18825 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2885b59740 10 client.18825 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2885b59740 20 client.18825 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2885b59740 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5630de185dc0 0x5630de188240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2885b59740 4 client.18825 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f2885b59740 1 -- 172.21.15.1:0/2455088339 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630de0ddb90 con 0x5630de185dc0 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.333+0000 7f288151c640 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5630de185dc0 0x5630de188240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.334+0000 7f2864ff9640 10 client.18825 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.334+0000 7f288151c640 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5630de185dc0 0x5630de188240 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa25ee3897f9e4c server_cookie=3462892b10ad6076 in_seq=0 out_seq=0 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2864ff9640 1 -- 172.21.15.1:0/2455088339 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630de0ddb90 con 0x5630de185dc0 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2864ff9640 10 client.18825 handle_command_reply: tid=0 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 client.18825 shutdown 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 -- 172.21.15.1:0/2455088339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 msgr2=0x5630de173b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de173b50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f2874035000 tx=0x7f287402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 -- 172.21.15.1:0/2455088339 shutdown_connections 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5630de185dc0 0x5630de188240 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de173b50 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de173610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 --2- 172.21.15.1:0/2455088339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de1730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2864ff9640 0 client.18825 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 -- 172.21.15.1:0/2455088339 >> 172.21.15.1:0/2455088339 conn(0x5630de1675f0 msgr2=0x5630de16d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 -- 172.21.15.1:0/2455088339 shutdown_connections 2024-09-16T00:43:00.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 1 -- 172.21.15.1:0/2455088339 wait complete. 2024-09-16T00:43:00.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.335+0000 7f2885b59740 20 client.18825 trim_cache size 0 max 16384 2024-09-16T00:43:00.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.352+0000 7f2885b59740 1 Processor -- start 2024-09-16T00:43:00.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.352+0000 7f2885b59740 1 -- start start 2024-09-16T00:43:00.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.352+0000 7f2885b59740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de18db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.352+0000 7f2885b59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de18e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.352+0000 7f2885b59740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de170350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.352+0000 7f2885b59740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f287c1b3890 con 0x5630de176240 2024-09-16T00:43:00.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.352+0000 7f2885b59740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f287c1b1160 con 0x5630de16b060 2024-09-16T00:43:00.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.352+0000 7f2885b59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f287c1aefb0 con 0x5630de16b980 2024-09-16T00:43:00.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.352+0000 7f2881d1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de170350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2881d1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de170350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42508/0 (socket says 172.21.15.1:42508) 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2881d1d640 1 -- 172.21.15.1:0/1780026551 learned_addr learned my addr 172.21.15.1:0/1780026551 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f288151c640 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de18db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2880d1b640 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de18e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2881d1d640 1 -- 172.21.15.1:0/1780026551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 msgr2=0x5630de18e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2881d1d640 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de18e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2881d1d640 1 -- 172.21.15.1:0/1780026551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 msgr2=0x5630de18db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2881d1d640 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de18db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2881d1d640 1 -- 172.21.15.1:0/1780026551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630de167420 con 0x5630de176240 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f288151c640 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de18db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:00.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2881d1d640 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de170350 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f287402ae00 tx=0x7f2874005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295114156eb77c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2866ffd640 1 -- 172.21.15.1:0/1780026551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874004750 con 0x5630de176240 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2866ffd640 1 -- 172.21.15.1:0/1780026551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2874036040 con 0x5630de176240 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.353+0000 7f2866ffd640 1 -- 172.21.15.1:0/1780026551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874050500 con 0x5630de176240 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 -- 172.21.15.1:0/1780026551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 msgr2=0x5630de170350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de170350 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f287402ae00 tx=0x7f2874005fa0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 -- 172.21.15.1:0/1780026551 shutdown_connections 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de176240 0x5630de170350 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b980 0x5630de18e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 --2- 172.21.15.1:0/1780026551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de16b060 0x5630de18db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 -- 172.21.15.1:0/1780026551 >> 172.21.15.1:0/1780026551 conn(0x5630de1675f0 msgr2=0x5630de175150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 -- 172.21.15.1:0/1780026551 shutdown_connections 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 -- 172.21.15.1:0/1780026551 wait complete. 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 -- start start 2024-09-16T00:43:00.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.354+0000 7f2885b59740 1 Processor -- start 2024-09-16T00:43:00.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2885b59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b060 0x5630de17e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2885b59740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de16b980 0x5630de17e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2885b59740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de176240 0x5630de17b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2885b59740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f287c1af620 con 0x5630de16b980 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2885b59740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f287c1b0a90 con 0x5630de176240 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2885b59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f287c1afcf0 con 0x5630de16b060 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2885b59740 10 client.0 _ll_register_callbacks cb 0x7f2874031df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2880d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de16b980 0x5630de17e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2880d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de16b980 0x5630de17e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42512/0 (socket says 172.21.15.1:42512) 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2880d1b640 1 -- 172.21.15.1:0/2159497184 learned_addr learned my addr 172.21.15.1:0/2159497184 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f288151c640 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b060 0x5630de17e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2881d1d640 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de176240 0x5630de17b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2880d1b640 1 -- 172.21.15.1:0/2159497184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b060 msgr2=0x5630de17e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2880d1b640 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b060 0x5630de17e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2880d1b640 1 -- 172.21.15.1:0/2159497184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de176240 msgr2=0x5630de17b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2880d1b640 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de176240 0x5630de17b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2880d1b640 1 -- 172.21.15.1:0/2159497184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630de17b8f0 con 0x5630de16b980 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.355+0000 7f2880d1b640 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de16b980 0x5630de17e930 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f286c010fd0 tx=0x7f286c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5a4999d1c684fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2864ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2864ff9640 1 -- 172.21.15.1:0/2159497184 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c020030 con 0x5630de16b980 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2864ff9640 1 -- 172.21.15.1:0/2159497184 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f286c099e80 con 0x5630de16b980 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2885b59740 1 -- 172.21.15.1:0/2159497184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2874036040 con 0x5630de16b980 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2864ff9640 1 -- 172.21.15.1:0/2159497184 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f286c09b6c0 con 0x5630de16b980 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2885b59740 10 client.18831 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2885b59740 1 -- 172.21.15.1:0/2159497184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630de1f1580 con 0x5630de16b980 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2864ff9640 1 -- 172.21.15.1:0/2159497184 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2874036040 con 0x5630de16b980 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2864ff9640 1 -- 172.21.15.1:0/2159497184 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f286c0cd0e0 con 0x5630de16b980 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2885b59740 10 client.18831 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2885b59740 10 client.18831 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:43:00.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2885b59740 20 client.18831 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2885b59740 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5630de1f1800 0x5630de1f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2885b59740 4 client.18831 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "93c3a287", "release": true, "await": true}] 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.356+0000 7f2885b59740 1 -- 172.21.15.1:0/2159497184 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "93c3a287", "release": true, "await": true}) -- 0x5630de0d80a0 con 0x5630de1f1800 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.357+0000 7f288151c640 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5630de1f1800 0x5630de1f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.357+0000 7f288151c640 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5630de1f1800 0x5630de1f3bf0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe815a65c2cd4c9 server_cookie=51b8e8c7be92fc3d in_seq=0 out_seq=0 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.357+0000 7f2864ff9640 10 client.18831 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2864ff9640 1 -- 172.21.15.1:0/2159497184 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5630de0d80a0 con 0x5630de1f1800 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2864ff9640 10 client.18831 handle_command_reply: tid=0 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 client.18831 shutdown 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 -- 172.21.15.1:0/2159497184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de16b980 msgr2=0x5630de17e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de16b980 0x5630de17e930 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f286c010fd0 tx=0x7f286c010d70 comp rx=0 tx=0).stop 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 -- 172.21.15.1:0/2159497184 shutdown_connections 2024-09-16T00:43:00.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5630de1f1800 0x5630de1f3bf0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630de176240 0x5630de17b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5630de16b980 0x5630de17e930 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 --2- 172.21.15.1:0/2159497184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630de16b060 0x5630de17e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2864ff9640 0 client.18831 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 -- 172.21.15.1:0/2159497184 >> 172.21.15.1:0/2159497184 conn(0x5630de1675f0 msgr2=0x5630de173090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 -- 172.21.15.1:0/2159497184 shutdown_connections 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 1 -- 172.21.15.1:0/2159497184 wait complete. 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.360+0000 7f2885b59740 20 client.18831 trim_cache size 0 max 16384 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:43:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 52, 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "93c3a287": { 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 52, 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:43:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 146.5, 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:00.367 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 -- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f28680a3240 msgr2=0x7f28680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f28680a3240 0x7f28680a5700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2874006010 tx=0x7f28740095c0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 -- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 msgr2=0x7f287c0fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 0x7f287c0fd2a0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f286c00d950 tx=0x7f286c00de10 comp rx=0 tx=0).stop 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 -- 172.21.15.1:0/4092967001 shutdown_connections 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f287c106a80 0x7f287c0fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f287c101420 0x7f287c0fd2a0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f28680a3240 0x7f28680a5700 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 --2- 172.21.15.1:0/4092967001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c100a80 0x7f287c0fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 -- 172.21.15.1:0/4092967001 >> 172.21.15.1:0/4092967001 conn(0x7f287c0fc7a0 msgr2=0x7f287c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 -- 172.21.15.1:0/4092967001 shutdown_connections 2024-09-16T00:43:00.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:00.361+0000 7f2883780640 1 -- 172.21.15.1:0/4092967001 wait complete. 2024-09-16T00:43:00.782 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:43:01.220 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '93c3a287', total seconds elapsed: 30.0 2024-09-16T00:43:01.253 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:01.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 -- 172.21.15.1:0/229304783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c100ca0 msgr2=0x7f085c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 --2- 172.21.15.1:0/229304783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c100ca0 0x7f085c106840 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0844009930 tx=0x7f084402a420 comp rx=0 tx=0).stop 2024-09-16T00:43:01.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 -- 172.21.15.1:0/229304783 shutdown_connections 2024-09-16T00:43:01.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 --2- 172.21.15.1:0/229304783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f085c106f50 0x7f085c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 --2- 172.21.15.1:0/229304783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c100ca0 0x7f085c106840 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 --2- 172.21.15.1:0/229304783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c1002f0 0x7f085c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 -- 172.21.15.1:0/229304783 >> 172.21.15.1:0/229304783 conn(0x7f085c077210 msgr2=0x7f085c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:01.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 -- 172.21.15.1:0/229304783 shutdown_connections 2024-09-16T00:43:01.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 -- 172.21.15.1:0/229304783 wait complete. 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 Processor -- start 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.748+0000 7f0862474640 1 -- start start 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f0862474640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c1002f0 0x7f085c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f0862474640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c100ca0 0x7f085c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f0862474640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f085c106f50 0x7f085c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f0862474640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f085c111ff0 con 0x7f085c1002f0 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f0862474640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f085c111e70 con 0x7f085c100ca0 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f0862474640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f085c112170 con 0x7f085c106f50 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f085bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c1002f0 0x7f085c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f085bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c1002f0 0x7f085c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42526/0 (socket says 172.21.15.1:42526) 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f085bfff640 1 -- 172.21.15.1:0/4092986256 learned_addr learned my addr 172.21.15.1:0/4092986256 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f0853fff640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c100ca0 0x7f085c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f0860a11640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f085c106f50 0x7f085c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f085bfff640 1 -- 172.21.15.1:0/4092986256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f085c106f50 msgr2=0x7f085c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f085bfff640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f085c106f50 0x7f085c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f085bfff640 1 -- 172.21.15.1:0/4092986256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c100ca0 msgr2=0x7f085c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f085bfff640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c100ca0 0x7f085c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f085bfff640 1 -- 172.21.15.1:0/4092986256 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c19d3d0 con 0x7f085c1002f0 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.749+0000 7f085bfff640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c1002f0 0x7f085c198420 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f084c00c920 tx=0x7f084c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.750+0000 7f0859ffb640 1 -- 172.21.15.1:0/4092986256 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c007c70 con 0x7f085c1002f0 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.750+0000 7f0859ffb640 1 -- 172.21.15.1:0/4092986256 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c007e10 con 0x7f085c1002f0 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.750+0000 7f0862474640 1 -- 172.21.15.1:0/4092986256 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c19d6c0 con 0x7f085c1002f0 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.750+0000 7f0859ffb640 1 -- 172.21.15.1:0/4092986256 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c018890 con 0x7f085c1002f0 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.750+0000 7f0862474640 1 -- 172.21.15.1:0/4092986256 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c1a4fa0 con 0x7f085c1002f0 2024-09-16T00:43:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.752+0000 7f0859ffb640 1 -- 172.21.15.1:0/4092986256 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f084c018a70 con 0x7f085c1002f0 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.752+0000 7f086484d740 1 Processor -- start 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.752+0000 7f086484d740 1 -- start start 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f086484d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b729740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f086484d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b738550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f086484d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b73ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f086484d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dd2b68f2e0 con 0x55dd2b729c80 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f086484d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd2b636ce0 con 0x55dd2b729360 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f086484d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dd2b68b2a0 con 0x55dd2b738a90 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f085bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b738550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f085bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b738550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42530/0 (socket says 172.21.15.1:42530) 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f085bfff640 1 -- 172.21.15.1:0/272599273 learned_addr learned my addr 172.21.15.1:0/272599273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f0853fff640 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b73ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f0860a11640 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b729740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f0859ffb640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08480a32c0 0x7f08480a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f0860a11640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08480a32c0 0x7f08480a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f085bfff640 1 -- 172.21.15.1:0/272599273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 msgr2=0x55dd2b73ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f085bfff640 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b73ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f085bfff640 1 -- 172.21.15.1:0/272599273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 msgr2=0x55dd2b729740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f085bfff640 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b729740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f0859ffb640 1 -- 172.21.15.1:0/4092986256 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f084c0cfd50 con 0x7f085c1002f0 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f085bfff640 1 -- 172.21.15.1:0/272599273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd2b61b440 con 0x55dd2b729c80 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f0853fff640 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b73ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.753+0000 7f085bfff640 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b738550 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f084c0cd450 tx=0x7f084c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372217aee1fa2170 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f0860a11640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08480a32c0 0x7f08480a5780 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0854009fd0 tx=0x7f0854004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f08527fc640 1 -- 172.21.15.1:0/272599273 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c09bab0 con 0x55dd2b729c80 2024-09-16T00:43:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f08527fc640 1 -- 172.21.15.1:0/272599273 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c09c670 con 0x55dd2b729c80 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f08527fc640 1 -- 172.21.15.1:0/272599273 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0bc070 con 0x55dd2b729c80 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f086484d740 1 -- 172.21.15.1:0/272599273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 msgr2=0x55dd2b738550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f086484d740 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b738550 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f084c0cd450 tx=0x7f084c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f086484d740 1 -- 172.21.15.1:0/272599273 shutdown_connections 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f086484d740 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b73ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f086484d740 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b738550 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f086484d740 1 --2- 172.21.15.1:0/272599273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b729740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f086484d740 1 -- 172.21.15.1:0/272599273 >> 172.21.15.1:0/272599273 conn(0x55dd2b725a30 msgr2=0x55dd2b725e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f086484d740 1 -- 172.21.15.1:0/272599273 shutdown_connections 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.754+0000 7f086484d740 1 -- 172.21.15.1:0/272599273 wait complete. 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f086484d740 1 -- start start 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f086484d740 1 Processor -- start 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f086484d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b738400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f086484d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f086484d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b735c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f086484d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dd2b73b600 con 0x55dd2b729c80 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f086484d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd2b64a050 con 0x55dd2b729360 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f086484d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dd2b73b780 con 0x55dd2b738a90 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f0853fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f086484d740 10 client.0 _ll_register_callbacks cb 0x55dd2b68a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.755+0000 7f0853fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42532/0 (socket says 172.21.15.1:42532) 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f0853fff640 1 -- 172.21.15.1:0/3661135192 learned_addr learned my addr 172.21.15.1:0/3661135192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f0860a11640 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b735c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f085bfff640 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b738400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f0853fff640 1 -- 172.21.15.1:0/3661135192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 msgr2=0x55dd2b735c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f0853fff640 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b735c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f0853fff640 1 -- 172.21.15.1:0/3661135192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 msgr2=0x55dd2b738400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f0853fff640 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b738400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f0853fff640 1 -- 172.21.15.1:0/3661135192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd2b4003c0 con 0x55dd2b729c80 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f085bfff640 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b738400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f0853fff640 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7356b0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f085c06a100 tx=0x7f08440045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e638ff4fb72339c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f083b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f083b7fe640 1 -- 172.21.15.1:0/3661135192 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844035040 con 0x55dd2b729c80 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f083b7fe640 1 -- 172.21.15.1:0/3661135192 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0844005550 con 0x55dd2b729c80 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.756+0000 7f086484d740 1 -- 172.21.15.1:0/3661135192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084c09c670 con 0x55dd2b729c80 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f083b7fe640 1 -- 172.21.15.1:0/3661135192 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f08440046c0 con 0x55dd2b729c80 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f086484d740 10 client.18843 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f086484d740 1 -- 172.21.15.1:0/3661135192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dd2b747010 con 0x55dd2b729c80 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f083b7fe640 1 -- 172.21.15.1:0/3661135192 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c09c670 con 0x55dd2b729c80 2024-09-16T00:43:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f083b7fe640 1 -- 172.21.15.1:0/3661135192 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0844003ca0 con 0x55dd2b729c80 2024-09-16T00:43:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f086484d740 10 client.18843 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f086484d740 10 client.18843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f086484d740 20 client.18843 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f086484d740 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dd2b7484d0 0x55dd2b74a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f086484d740 4 client.18843 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.757+0000 7f086484d740 1 -- 172.21.15.1:0/3661135192 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd2b69c850 con 0x55dd2b7484d0 2024-09-16T00:43:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.758+0000 7f085bfff640 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dd2b7484d0 0x55dd2b74a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.758+0000 7f085bfff640 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dd2b7484d0 0x55dd2b74a910 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=844e5e0958c81cdb server_cookie=684afe8685ee199a in_seq=0 out_seq=0 2024-09-16T00:43:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.758+0000 7f083b7fe640 10 client.18843 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f083b7fe640 1 -- 172.21.15.1:0/3661135192 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd2b69c850 con 0x55dd2b7484d0 2024-09-16T00:43:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f083b7fe640 10 client.18843 handle_command_reply: tid=0 2024-09-16T00:43:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 client.18843 shutdown 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 -- 172.21.15.1:0/3661135192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 msgr2=0x55dd2b7356b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7356b0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f085c06a100 tx=0x7f08440045d0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 -- 172.21.15.1:0/3661135192 shutdown_connections 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dd2b7484d0 0x55dd2b74a910 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b735c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7356b0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f083b7fe640 0 client.18843 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 --2- 172.21.15.1:0/3661135192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b738400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 -- 172.21.15.1:0/3661135192 >> 172.21.15.1:0/3661135192 conn(0x55dd2b725a30 msgr2=0x55dd2b72b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 -- 172.21.15.1:0/3661135192 shutdown_connections 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 1 -- 172.21.15.1:0/3661135192 wait complete. 2024-09-16T00:43:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.759+0000 7f086484d740 20 client.18843 trim_cache size 0 max 16384 2024-09-16T00:43:01.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f086484d740 1 Processor -- start 2024-09-16T00:43:01.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f086484d740 1 -- start start 2024-09-16T00:43:01.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f086484d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729360 0x55dd2b7448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f086484d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729c80 0x55dd2b7501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f086484d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b750720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f086484d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1af4d0 con 0x55dd2b729360 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f086484d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1acda0 con 0x55dd2b729c80 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f086484d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1aabf0 con 0x55dd2b738a90 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f085bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729360 0x55dd2b7448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f085bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729360 0x55dd2b7448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42546/0 (socket says 172.21.15.1:42546) 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f085bfff640 1 -- 172.21.15.1:0/83915699 learned_addr learned my addr 172.21.15.1:0/83915699 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f0853fff640 1 --2- 172.21.15.1:0/83915699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729c80 0x55dd2b7501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f0860a11640 1 --2- 172.21.15.1:0/83915699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b750720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f085bfff640 1 -- 172.21.15.1:0/83915699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 msgr2=0x55dd2b750720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f085bfff640 1 --2- 172.21.15.1:0/83915699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b750720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f085bfff640 1 -- 172.21.15.1:0/83915699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729c80 msgr2=0x55dd2b7501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f085bfff640 1 --2- 172.21.15.1:0/83915699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729c80 0x55dd2b7501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.777+0000 7f085bfff640 1 -- 172.21.15.1:0/83915699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd2b735f80 con 0x55dd2b729360 2024-09-16T00:43:01.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.778+0000 7f085bfff640 1 --2- 172.21.15.1:0/83915699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729360 0x55dd2b7448a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f084c0cd4a0 tx=0x7f084c0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48935924e13704bd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:01.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.778+0000 7f0851ffb640 1 -- 172.21.15.1:0/83915699 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0b6020 con 0x55dd2b729360 2024-09-16T00:43:01.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.778+0000 7f0851ffb640 1 -- 172.21.15.1:0/83915699 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c0a0320 con 0x55dd2b729360 2024-09-16T00:43:01.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.778+0000 7f0851ffb640 1 -- 172.21.15.1:0/83915699 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c048a50 con 0x55dd2b729360 2024-09-16T00:43:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 -- 172.21.15.1:0/83915699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729360 msgr2=0x55dd2b7448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 --2- 172.21.15.1:0/83915699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729360 0x55dd2b7448a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f084c0cd4a0 tx=0x7f084c0bc040 comp rx=0 tx=0).stop 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 -- 172.21.15.1:0/83915699 shutdown_connections 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 --2- 172.21.15.1:0/83915699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b750720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 --2- 172.21.15.1:0/83915699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729c80 0x55dd2b7501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 --2- 172.21.15.1:0/83915699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729360 0x55dd2b7448a0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 -- 172.21.15.1:0/83915699 >> 172.21.15.1:0/83915699 conn(0x55dd2b725a30 msgr2=0x55dd2b730d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 -- 172.21.15.1:0/83915699 shutdown_connections 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 -- 172.21.15.1:0/83915699 wait complete. 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 -- start start 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 Processor -- start 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.779+0000 7f086484d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b7ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f086484d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f086484d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b73bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f086484d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1ab260 con 0x55dd2b729c80 2024-09-16T00:43:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f086484d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1ac6d0 con 0x55dd2b729360 2024-09-16T00:43:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f086484d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1ab930 con 0x55dd2b738a90 2024-09-16T00:43:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f086484d740 10 client.0 _ll_register_callbacks cb 0x7f0844009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0853fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0853fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7ab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42560/0 (socket says 172.21.15.1:42560) 2024-09-16T00:43:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0853fff640 1 -- 172.21.15.1:0/2724954434 learned_addr learned my addr 172.21.15.1:0/2724954434 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:01.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:01 smithi001 ceph-mon[30934]: pgmap v789: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f085bfff640 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b7ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0860a11640 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b73bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0853fff640 1 -- 172.21.15.1:0/2724954434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 msgr2=0x55dd2b73bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0853fff640 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b73bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0853fff640 1 -- 172.21.15.1:0/2724954434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 msgr2=0x55dd2b7ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0853fff640 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b7ab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0853fff640 1 -- 172.21.15.1:0/2724954434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd2b73c7a0 con 0x55dd2b729c80 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f0853fff640 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7ab9e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f0844009850 tx=0x7f0844005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580a887556e0f35f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.780+0000 7f083b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f083b7fe640 1 -- 172.21.15.1:0/2724954434 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844057070 con 0x55dd2b729c80 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f083b7fe640 1 -- 172.21.15.1:0/2724954434 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f0844005550 con 0x55dd2b729c80 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f086484d740 1 -- 172.21.15.1:0/2724954434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084c0a0320 con 0x55dd2b729c80 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f083b7fe640 1 -- 172.21.15.1:0/2724954434 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0844035070 con 0x55dd2b729c80 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f086484d740 10 client.18849 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f086484d740 1 -- 172.21.15.1:0/2724954434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dd2b7b2cc0 con 0x55dd2b729c80 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f083b7fe640 1 -- 172.21.15.1:0/2724954434 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c0a0320 con 0x55dd2b729c80 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f083b7fe640 1 -- 172.21.15.1:0/2724954434 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f084405a020 con 0x55dd2b729c80 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f086484d740 10 client.18849 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f086484d740 10 client.18849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f086484d740 20 client.18849 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f086484d740 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dd2b7484d0 0x55dd2b7b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f086484d740 4 client.18849 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:43:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.781+0000 7f086484d740 1 -- 172.21.15.1:0/2724954434 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd2b696d60 con 0x55dd2b7484d0 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.782+0000 7f085bfff640 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dd2b7484d0 0x55dd2b7b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.782+0000 7f085bfff640 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dd2b7484d0 0x55dd2b7b54a0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7cb0db008dfdcf server_cookie=6d52b5126c92873b in_seq=0 out_seq=0 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.782+0000 7f083b7fe640 10 client.18849 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.782+0000 7f083b7fe640 1 -- 172.21.15.1:0/2724954434 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dd2b696d60 con 0x55dd2b7484d0 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.782+0000 7f083b7fe640 10 client.18849 handle_command_reply: tid=0 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.782+0000 7f086484d740 1 client.18849 shutdown 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 -- 172.21.15.1:0/2724954434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 msgr2=0x55dd2b7ab9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7ab9e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f0844009850 tx=0x7f0844005b50 comp rx=0 tx=0).stop 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 -- 172.21.15.1:0/2724954434 shutdown_connections 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55dd2b7484d0 0x55dd2b7b54a0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd2b738a90 0x55dd2b73bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd2b729c80 0x55dd2b7ab9e0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f083b7fe640 0 client.18849 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 --2- 172.21.15.1:0/2724954434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2b729360 0x55dd2b7ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 -- 172.21.15.1:0/2724954434 >> 172.21.15.1:0/2724954434 conn(0x55dd2b725a30 msgr2=0x55dd2b733f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 -- 172.21.15.1:0/2724954434 shutdown_connections 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 1 -- 172.21.15.1:0/2724954434 wait complete. 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f086484d740 20 client.18849 trim_cache size 0 max 16384 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:43:01.789 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:43:01.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f0862474640 1 -- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08480a32c0 msgr2=0x7f08480a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f0862474640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08480a32c0 0x7f08480a5780 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0854009fd0 tx=0x7f0854004ab0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f0862474640 1 -- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c1002f0 msgr2=0x7f085c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.783+0000 7f0862474640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c1002f0 0x7f085c198420 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f084c00c920 tx=0x7f084c00cde0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.784+0000 7f0862474640 1 -- 172.21.15.1:0/4092986256 shutdown_connections 2024-09-16T00:43:01.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.784+0000 7f0862474640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f08480a32c0 0x7f08480a5780 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.784+0000 7f0862474640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f085c106f50 0x7f085c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.784+0000 7f0862474640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c100ca0 0x7f085c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.784+0000 7f0862474640 1 --2- 172.21.15.1:0/4092986256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f085c1002f0 0x7f085c198420 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.784+0000 7f0862474640 1 -- 172.21.15.1:0/4092986256 >> 172.21.15.1:0/4092986256 conn(0x7f085c077210 msgr2=0x7f085c105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:01.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.784+0000 7f0862474640 1 -- 172.21.15.1:0/4092986256 shutdown_connections 2024-09-16T00:43:01.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:01.784+0000 7f0862474640 1 -- 172.21.15.1:0/4092986256 wait complete. 2024-09-16T00:43:02.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:01 smithi019 ceph-mon[35186]: pgmap v789: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:43:02.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:01 smithi153 ceph-mon[35538]: pgmap v789: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T00:43:02.530 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:43:02.530 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:43:02.836 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:03.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.305+0000 7f364b8d5640 1 -- 172.21.15.1:0/2069784542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644106a80 msgr2=0x7f3644106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:03.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.305+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2069784542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644106a80 0x7f3644106e60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f3640009930 tx=0x7f364002a420 comp rx=0 tx=0).stop 2024-09-16T00:43:03.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.305+0000 7f364b8d5640 1 -- 172.21.15.1:0/2069784542 shutdown_connections 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.305+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2069784542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3644101420 0x7f364410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.305+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2069784542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3644100a80 0x7f3644100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.305+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2069784542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644106a80 0x7f3644106e60 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.305+0000 7f364b8d5640 1 -- 172.21.15.1:0/2069784542 >> 172.21.15.1:0/2069784542 conn(0x7f36440fc7a0 msgr2=0x7f36440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.305+0000 7f364b8d5640 1 -- 172.21.15.1:0/2069784542 shutdown_connections 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.305+0000 7f364b8d5640 1 -- 172.21.15.1:0/2069784542 wait complete. 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f364b8d5640 1 Processor -- start 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f364b8d5640 1 -- start start 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f364b8d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3644100a80 0x7f364419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f364b8d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644101420 0x7f364419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f364b8d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3644106a80 0x7f364419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:03.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f364b8d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3644110ec0 con 0x7f3644101420 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f364b8d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3644110d40 con 0x7f3644106a80 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f364b8d5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3644111040 con 0x7f3644100a80 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f3648e70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644101420 0x7f364419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f3648e70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644101420 0x7f364419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42586/0 (socket says 172.21.15.1:42586) 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.306+0000 7f3648e70640 1 -- 172.21.15.1:0/2622074624 learned_addr learned my addr 172.21.15.1:0/2622074624 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3649e72640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3644106a80 0x7f364419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3649671640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3644100a80 0x7f364419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3648e70640 1 -- 172.21.15.1:0/2622074624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3644100a80 msgr2=0x7f364419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3648e70640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3644100a80 0x7f364419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3648e70640 1 -- 172.21.15.1:0/2622074624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3644106a80 msgr2=0x7f364419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3648e70640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3644106a80 0x7f364419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3648e70640 1 -- 172.21.15.1:0/2622074624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364419f6c0 con 0x7f3644101420 2024-09-16T00:43:03.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3649e72640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3644106a80 0x7f364419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3649671640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3644100a80 0x7f364419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f3648e70640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644101420 0x7f364419ac50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f363400c970 tx=0x7f363400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f363a7fc640 1 -- 172.21.15.1:0/2622074624 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634007bb0 con 0x7f3644101420 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364419f9b0 con 0x7f3644101420 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f363a7fc640 1 -- 172.21.15.1:0/2622074624 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3634007d50 con 0x7f3644101420 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f363a7fc640 1 -- 172.21.15.1:0/2622074624 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363400f450 con 0x7f3644101420 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.307+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36441a7290 con 0x7f3644101420 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.308+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3644100ee0 con 0x7f3644101420 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.309+0000 7f363a7fc640 1 -- 172.21.15.1:0/2622074624 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3634004fe0 con 0x7f3644101420 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.310+0000 7f363a7fc640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36300a3240 0x7f36300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.310+0000 7f3649671640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36300a3240 0x7f36300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.310+0000 7f363a7fc640 1 -- 172.21.15.1:0/2622074624 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f36340cff90 con 0x7f3644101420 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.311+0000 7f3649671640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36300a3240 0x7f36300a5700 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3640000c00 tx=0x7f3640032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:03.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.313+0000 7f363a7fc640 1 -- 172.21.15.1:0/2622074624 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3634014030 con 0x7f3644101420 2024-09-16T00:43:03.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.527+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f364419ba70 con 0x7f3644101420 2024-09-16T00:43:03.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.527+0000 7f363a7fc640 1 -- 172.21.15.1:0/2622074624 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f363409c8b0 con 0x7f3644101420 2024-09-16T00:43:03.528 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:43:03.528 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:43:03.528 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:43:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.530+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36300a3240 msgr2=0x7f36300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.530+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36300a3240 0x7f36300a5700 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3640000c00 tx=0x7f3640032d50 comp rx=0 tx=0).stop 2024-09-16T00:43:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.530+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644101420 msgr2=0x7f364419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.530+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644101420 0x7f364419ac50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f363400c970 tx=0x7f363400ce30 comp rx=0 tx=0).stop 2024-09-16T00:43:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.531+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 shutdown_connections 2024-09-16T00:43:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.531+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3644106a80 0x7f364419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.531+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644101420 0x7f364419ac50 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.531+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f36300a3240 0x7f36300a5700 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.531+0000 7f364b8d5640 1 --2- 172.21.15.1:0/2622074624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3644100a80 0x7f364419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:03.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.531+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 >> 172.21.15.1:0/2622074624 conn(0x7f36440fc7a0 msgr2=0x7f364406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:03.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.531+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 shutdown_connections 2024-09-16T00:43:03.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:03.531+0000 7f364b8d5640 1 -- 172.21.15.1:0/2622074624 wait complete. 2024-09-16T00:43:03.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:03 smithi001 ceph-mon[30934]: pgmap v790: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:43:03.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:03 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2622074624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:03 smithi019 ceph-mon[35186]: pgmap v790: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:43:04.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:03 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2622074624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:04.169 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:43:04.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:03 smithi153 ceph-mon[35538]: pgmap v790: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:43:04.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:03 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2622074624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:04.431 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:04.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.966+0000 7f3dbf07c640 1 -- 172.21.15.1:0/2301279180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8101de0 msgr2=0x7f3db81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:04.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.966+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/2301279180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8101de0 0x7f3db81021c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009900 tx=0x7f3db402a410 comp rx=0 tx=0).stop 2024-09-16T00:43:04.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.966+0000 7f3dbf07c640 1 -- 172.21.15.1:0/2301279180 shutdown_connections 2024-09-16T00:43:04.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.966+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/2301279180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db8106f10 0x7f3db810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.966+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/2301279180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db8102790 0x7f3db81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.966+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/2301279180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8101de0 0x7f3db81021c0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.966+0000 7f3dbf07c640 1 -- 172.21.15.1:0/2301279180 >> 172.21.15.1:0/2301279180 conn(0x7f3db80fd8c0 msgr2=0x7f3db80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:04.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.966+0000 7f3dbf07c640 1 -- 172.21.15.1:0/2301279180 shutdown_connections 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.966+0000 7f3dbf07c640 1 -- 172.21.15.1:0/2301279180 wait complete. 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbf07c640 1 Processor -- start 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbf07c640 1 -- start start 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbf07c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db8101de0 0x7f3db81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbf07c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db8102790 0x7f3db8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbf07c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8106f10 0x7f3db819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbf07c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8111f70 con 0x7f3db8106f10 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbf07c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8111df0 con 0x7f3db8101de0 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbf07c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81120f0 con 0x7f3db8102790 2024-09-16T00:43:04.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbd619640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8106f10 0x7f3db819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbd619640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8106f10 0x7f3db819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42606/0 (socket says 172.21.15.1:42606) 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbd619640 1 -- 172.21.15.1:0/3531408856 learned_addr learned my addr 172.21.15.1:0/3531408856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3daffff640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db8102790 0x7f3db8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.967+0000 7f3dbce18640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db8101de0 0x7f3db81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dbd619640 1 -- 172.21.15.1:0/3531408856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db8102790 msgr2=0x7f3db8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dbd619640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db8102790 0x7f3db8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dbd619640 1 -- 172.21.15.1:0/3531408856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db8101de0 msgr2=0x7f3db81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dbd619640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db8101de0 0x7f3db81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dbd619640 1 -- 172.21.15.1:0/3531408856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db819d380 con 0x7f3db8106f10 2024-09-16T00:43:04.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dbce18640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db8101de0 0x7f3db81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dbd619640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8106f10 0x7f3db819cca0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f3db000cce0 tx=0x7f3db000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dadffb640 1 -- 172.21.15.1:0/3531408856 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db000f450 con 0x7f3db8106f10 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dadffb640 1 -- 172.21.15.1:0/3531408856 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db000fad0 con 0x7f3db8106f10 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dbf07c640 1 -- 172.21.15.1:0/3531408856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db819d670 con 0x7f3db8106f10 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dadffb640 1 -- 172.21.15.1:0/3531408856 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0018c10 con 0x7f3db8106f10 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.968+0000 7f3dbf07c640 1 -- 172.21.15.1:0/3531408856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db81a4f50 con 0x7f3db8106f10 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.970+0000 7f3dadffb640 1 -- 172.21.15.1:0/3531408856 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3db000f5f0 con 0x7f3db8106f10 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dc1455740 1 Processor -- start 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dc1455740 1 -- start start 2024-09-16T00:43:04.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dadffb640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32c0 0x7f3da00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57741090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e577415d0 0x562e57741a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5774c370 0x562e5774e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dadffb640 1 -- 172.21.15.1:0/3531408856 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3db00d02c0 con 0x7f3db8106f10 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dc1455740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562e576a72f0 con 0x562e57742b40 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dc1455740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e5764ecf0 con 0x562e5774c370 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dc1455740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e576a32b0 con 0x562e577415d0 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3daffff640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32c0 0x7f3da00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dbce18640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57741090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dbce18640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57741090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42610/0 (socket says 172.21.15.1:42610) 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dbce18640 1 -- 172.21.15.1:0/2165501267 learned_addr learned my addr 172.21.15.1:0/2165501267 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:04.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dbd619640 1 --2- 172.21.15.1:0/2165501267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5774c370 0x562e5774e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dbce18640 1 -- 172.21.15.1:0/2165501267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e577415d0 msgr2=0x562e57741a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dbce18640 1 --2- 172.21.15.1:0/2165501267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e577415d0 0x562e57741a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dbce18640 1 -- 172.21.15.1:0/2165501267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5774c370 msgr2=0x562e5774e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dbce18640 1 --2- 172.21.15.1:0/2165501267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5774c370 0x562e5774e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.971+0000 7f3dbce18640 1 -- 172.21.15.1:0/2165501267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e576333a0 con 0x562e57742b40 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.972+0000 7f3daffff640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32c0 0x7f3da00a5780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3db80fdce0 tx=0x7f3da800a6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.972+0000 7f3dbce18640 1 --2- 172.21.15.1:0/2165501267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57741090 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f3db4000c00 tx=0x7f3db4032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbf13590025e8bb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.972+0000 7f3d9effd640 1 -- 172.21.15.1:0/2165501267 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db403f070 con 0x562e57742b40 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.972+0000 7f3d9effd640 1 -- 172.21.15.1:0/2165501267 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db40317b0 con 0x562e57742b40 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.972+0000 7f3d9effd640 1 -- 172.21.15.1:0/2165501267 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4035070 con 0x562e57742b40 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.972+0000 7f3dc1455740 1 -- 172.21.15.1:0/2165501267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 msgr2=0x562e57741090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.972+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2165501267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57741090 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f3db4000c00 tx=0x7f3db4032d70 comp rx=0 tx=0).stop 2024-09-16T00:43:04.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 -- 172.21.15.1:0/2165501267 shutdown_connections 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2165501267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5774c370 0x562e5774e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2165501267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e577415d0 0x562e57741a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2165501267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57741090 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 -- 172.21.15.1:0/2165501267 >> 172.21.15.1:0/2165501267 conn(0x562e5770a310 msgr2=0x562e5773dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 -- 172.21.15.1:0/2165501267 shutdown_connections 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 -- 172.21.15.1:0/2165501267 wait complete. 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 -- start start 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 Processor -- start 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e577415d0 0x562e5774bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e57742b40 0x562e57748f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e577494d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562e5774f010 con 0x562e5774c370 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e57662060 con 0x562e57742b40 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e5774f190 con 0x562e577415d0 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.973+0000 7f3dc1455740 10 client.0 _ll_register_callbacks cb 0x562e573b58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbd619640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e577494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3daffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e57742b40 0x562e57748f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbd619640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e577494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42614/0 (socket says 172.21.15.1:42614) 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbd619640 1 -- 172.21.15.1:0/2692148060 learned_addr learned my addr 172.21.15.1:0/2692148060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbce18640 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e577415d0 0x562e5774bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbd619640 1 -- 172.21.15.1:0/2692148060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e577415d0 msgr2=0x562e5774bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbd619640 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e577415d0 0x562e5774bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbd619640 1 -- 172.21.15.1:0/2692148060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e57742b40 msgr2=0x562e57748f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:04.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbd619640 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e57742b40 0x562e57748f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbd619640 1 -- 172.21.15.1:0/2692148060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e574183c0 con 0x562e5774c370 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dbd619640 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e577494d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3db00a4c20 tx=0x7f3db009c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2755ead95603995f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3d97fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3d97fff640 1 -- 172.21.15.1:0/2692148060 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db009c4e0 con 0x562e5774c370 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3d97fff640 1 -- 172.21.15.1:0/2692148060 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3db00a4cc0 con 0x562e5774c370 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3dc1455740 1 -- 172.21.15.1:0/2692148060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db40317b0 con 0x562e5774c370 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.974+0000 7f3d97fff640 1 -- 172.21.15.1:0/2692148060 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3db00a0890 con 0x562e5774c370 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3dc1455740 10 client.18867 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3dc1455740 1 -- 172.21.15.1:0/2692148060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e5775aab0 con 0x562e5774c370 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3d97fff640 1 -- 172.21.15.1:0/2692148060 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db40317b0 con 0x562e5774c370 2024-09-16T00:43:04.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3d97fff640 1 -- 172.21.15.1:0/2692148060 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3db00b52a0 con 0x562e5774c370 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3dc1455740 10 client.18867 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3dc1455740 10 client.18867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3dc1455740 20 client.18867 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562e5775bfc0 0x562e5775e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3dc1455740 4 client.18867 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3dc1455740 1 -- 172.21.15.1:0/2692148060 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e576b4860 con 0x562e5775bfc0 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.975+0000 7f3dbce18640 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562e5775bfc0 0x562e5775e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.976+0000 7f3dbce18640 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562e5775bfc0 0x562e5775e440 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b20b225da9d08b58 server_cookie=6f2a952f6ca82881 in_seq=0 out_seq=0 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.976+0000 7f3d97fff640 10 client.18867 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.976+0000 7f3d97fff640 1 -- 172.21.15.1:0/2692148060 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e576b4860 con 0x562e5775bfc0 2024-09-16T00:43:04.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.976+0000 7f3d97fff640 10 client.18867 handle_command_reply: tid=0 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.976+0000 7f3dc1455740 1 client.18867 shutdown 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 -- 172.21.15.1:0/2692148060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 msgr2=0x562e577494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e577494d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3db00a4c20 tx=0x7f3db009c0e0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 -- 172.21.15.1:0/2692148060 shutdown_connections 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e577494d0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562e5775bfc0 0x562e5775e440 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e57742b40 0x562e57748f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2692148060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e577415d0 0x562e5774bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 -- 172.21.15.1:0/2692148060 >> 172.21.15.1:0/2692148060 conn(0x562e5770a310 msgr2=0x562e57743930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3d97fff640 0 client.18867 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 -- 172.21.15.1:0/2692148060 shutdown_connections 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 1 -- 172.21.15.1:0/2692148060 wait complete. 2024-09-16T00:43:04.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.977+0000 7f3dc1455740 20 client.18867 trim_cache size 0 max 16384 2024-09-16T00:43:04.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dc1455740 1 Processor -- start 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dc1455740 1 -- start start 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 0x562e577582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57744120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e5774c370 0x562e57744660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dc1455740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81b1690 con 0x562e57742b40 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dc1455740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81aef60 con 0x562e577415d0 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dc1455740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81acdb0 con 0x562e5774c370 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3daffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57744120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3daffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57744120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42624/0 (socket says 172.21.15.1:42624) 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3daffff640 1 -- 172.21.15.1:0/2160836011 learned_addr learned my addr 172.21.15.1:0/2160836011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:05.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dbd619640 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e5774c370 0x562e57744660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.990+0000 7f3dbce18640 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 0x562e577582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3daffff640 1 -- 172.21.15.1:0/2160836011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e5774c370 msgr2=0x562e57744660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3daffff640 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e5774c370 0x562e57744660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3daffff640 1 -- 172.21.15.1:0/2160836011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 msgr2=0x562e577582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3daffff640 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 0x562e577582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3daffff640 1 -- 172.21.15.1:0/2160836011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5774f1b0 con 0x562e57742b40 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3dbce18640 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 0x562e577582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3daffff640 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57744120 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f3da800afb0 tx=0x7f3da8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce180084db379956 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3d9e7fc640 1 -- 172.21.15.1:0/2160836011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8018070 con 0x562e57742b40 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3d9e7fc640 1 -- 172.21.15.1:0/2160836011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da8016070 con 0x562e57742b40 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.991+0000 7f3d9e7fc640 1 -- 172.21.15.1:0/2160836011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da801f660 con 0x562e57742b40 2024-09-16T00:43:05.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 -- 172.21.15.1:0/2160836011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 msgr2=0x562e57744120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57744120 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f3da800afb0 tx=0x7f3da8014000 comp rx=0 tx=0).stop 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 -- 172.21.15.1:0/2160836011 shutdown_connections 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e5774c370 0x562e57744660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e57742b40 0x562e57744120 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 --2- 172.21.15.1:0/2160836011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 0x562e577582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 -- 172.21.15.1:0/2160836011 >> 172.21.15.1:0/2160836011 conn(0x562e5770a310 msgr2=0x562e57748f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 -- 172.21.15.1:0/2160836011 shutdown_connections 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 -- 172.21.15.1:0/2160836011 wait complete. 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 -- start start 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.992+0000 7f3dc1455740 1 Processor -- start 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 0x562e57754c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e57742b40 0x562e5774ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dc1455740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e57750470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:05.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dc1455740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81ad420 con 0x562e5774c370 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dc1455740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81ae890 con 0x562e577415d0 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dc1455740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81adaf0 con 0x562e57742b40 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dc1455740 10 client.0 _ll_register_callbacks cb 0x7f3db000cc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbd619640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e57750470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbd619640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e57750470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42640/0 (socket says 172.21.15.1:42640) 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbd619640 1 -- 172.21.15.1:0/76169962 learned_addr learned my addr 172.21.15.1:0/76169962 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3daffff640 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e57742b40 0x562e5774ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbce18640 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 0x562e57754c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbd619640 1 -- 172.21.15.1:0/76169962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e57742b40 msgr2=0x562e5774ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbd619640 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e57742b40 0x562e5774ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbd619640 1 -- 172.21.15.1:0/76169962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 msgr2=0x562e57754c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbd619640 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 0x562e57754c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbd619640 1 -- 172.21.15.1:0/76169962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e57750ca0 con 0x562e5774c370 2024-09-16T00:43:05.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dbd619640 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e57750470 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f3db009a790 tx=0x7f3db00b5580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf90ecc4892b406 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3d97fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3d97fff640 1 -- 172.21.15.1:0/76169962 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0084060 con 0x562e5774c370 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3dc1455740 1 -- 172.21.15.1:0/76169962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da8016070 con 0x562e5774c370 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.993+0000 7f3d97fff640 1 -- 172.21.15.1:0/76169962 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f3db00a4ec0 con 0x562e5774c370 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3d97fff640 1 -- 172.21.15.1:0/76169962 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3db009c4e0 con 0x562e5774c370 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3dc1455740 10 client.18873 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3dc1455740 1 -- 172.21.15.1:0/76169962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e577c6d40 con 0x562e5774c370 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3d97fff640 1 -- 172.21.15.1:0/76169962 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da8016070 con 0x562e5774c370 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3d97fff640 1 -- 172.21.15.1:0/76169962 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3db00983c0 con 0x562e5774c370 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3dc1455740 10 client.18873 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3dc1455740 10 client.18873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3dc1455740 20 client.18873 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:05.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3dc1455740 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562e577c7320 0x562e577c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3dc1455740 4 client.18873 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3dc1455740 1 -- 172.21.15.1:0/76169962 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e576aed70 con 0x562e577c7320 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.994+0000 7f3dbce18640 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562e577c7320 0x562e577c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dbce18640 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562e577c7320 0x562e577c9710 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50c3fcc523013963 server_cookie=c5e470e099cedddf in_seq=0 out_seq=0 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3d97fff640 10 client.18873 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3d97fff640 1 -- 172.21.15.1:0/76169962 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e576aed70 con 0x562e577c7320 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3d97fff640 10 client.18873 handle_command_reply: tid=0 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 client.18873 shutdown 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 -- 172.21.15.1:0/76169962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 msgr2=0x562e57750470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e57750470 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f3db009a790 tx=0x7f3db00b5580 comp rx=0 tx=0).stop 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 -- 172.21.15.1:0/76169962 shutdown_connections 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e5774c370 0x562e57750470 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e57742b40 0x562e5774ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x562e577c7320 0x562e577c9710 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 --2- 172.21.15.1:0/76169962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e577415d0 0x562e57754c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 -- 172.21.15.1:0/76169962 >> 172.21.15.1:0/76169962 conn(0x562e5770a310 msgr2=0x562e5774a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3d97fff640 0 client.18873 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:43:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.995+0000 7f3dc1455740 1 -- 172.21.15.1:0/76169962 shutdown_connections 2024-09-16T00:43:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dc1455740 1 -- 172.21.15.1:0/76169962 wait complete. 2024-09-16T00:43:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dc1455740 20 client.18873 trim_cache size 0 max 16384 2024-09-16T00:43:05.006 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:43:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 -- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32c0 msgr2=0x7f3da00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32c0 0x7f3da00a5780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3db80fdce0 tx=0x7f3da800a6f0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 -- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8106f10 msgr2=0x7f3db819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8106f10 0x7f3db819cca0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f3db000cce0 tx=0x7f3db000a620 comp rx=0 tx=0).stop 2024-09-16T00:43:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 -- 172.21.15.1:0/3531408856 shutdown_connections 2024-09-16T00:43:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3db8106f10 0x7f3db819cca0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f3da00a32c0 0x7f3da00a5780 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3db8102790 0x7f3db8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 --2- 172.21.15.1:0/3531408856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db8101de0 0x7f3db81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 -- 172.21.15.1:0/3531408856 >> 172.21.15.1:0/3531408856 conn(0x7f3db80fd8c0 msgr2=0x7f3db80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 -- 172.21.15.1:0/3531408856 shutdown_connections 2024-09-16T00:43:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:04.996+0000 7f3dbf07c640 1 -- 172.21.15.1:0/3531408856 wait complete. 2024-09-16T00:43:05.684 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:43:05.939 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:05.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:05 smithi001 ceph-mon[30934]: pgmap v791: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:43:06.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:05 smithi019 ceph-mon[35186]: pgmap v791: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:43:06.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:05 smithi153 ceph-mon[35538]: pgmap v791: 129 pgs: 129 active+clean; 6.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:43:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.450+0000 7f4c6c574640 1 -- 172.21.15.1:0/3369723874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64100e20 msgr2=0x7f4c64101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.450+0000 7f4c6c574640 1 --2- 172.21.15.1:0/3369723874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64100e20 0x7f4c64101200 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009900 tx=0x7f4c6002a480 comp rx=0 tx=0).stop 2024-09-16T00:43:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.450+0000 7f4c6c574640 1 -- 172.21.15.1:0/3369723874 shutdown_connections 2024-09-16T00:43:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.450+0000 7f4c6c574640 1 --2- 172.21.15.1:0/3369723874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c64102ef0 0x7f4c6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.450+0000 7f4c6c574640 1 --2- 172.21.15.1:0/3369723874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c64102570 0x7f4c641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.450+0000 7f4c6c574640 1 --2- 172.21.15.1:0/3369723874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64100e20 0x7f4c64101200 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.450+0000 7f4c6c574640 1 -- 172.21.15.1:0/3369723874 >> 172.21.15.1:0/3369723874 conn(0x7f4c640fc7d0 msgr2=0x7f4c640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.450+0000 7f4c6c574640 1 -- 172.21.15.1:0/3369723874 shutdown_connections 2024-09-16T00:43:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.450+0000 7f4c6c574640 1 -- 172.21.15.1:0/3369723874 wait complete. 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6c574640 1 Processor -- start 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6c574640 1 -- start start 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6c574640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c64100e20 0x7f4c64198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6c574640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64102570 0x7f4c64198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6c574640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c64102ef0 0x7f4c6419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6c574640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64110fa0 con 0x7f4c64102570 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6c574640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64110e20 con 0x7f4c64100e20 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6c574640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64111120 con 0x7f4c64102ef0 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c69b0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64102570 0x7f4c64198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c69b0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64102570 0x7f4c64198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42668/0 (socket says 172.21.15.1:42668) 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c69b0f640 1 -- 172.21.15.1:0/23310737 learned_addr learned my addr 172.21.15.1:0/23310737 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6a310640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c64100e20 0x7f4c64198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c64102ef0 0x7f4c6419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c69b0f640 1 -- 172.21.15.1:0/23310737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c64102ef0 msgr2=0x7f4c6419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c64102ef0 0x7f4c6419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c69b0f640 1 -- 172.21.15.1:0/23310737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c64100e20 msgr2=0x7f4c64198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c64100e20 0x7f4c64198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.451+0000 7f4c69b0f640 1 -- 172.21.15.1:0/23310737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6419d4c0 con 0x7f4c64102570 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.452+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64102570 0x7f4c64198a50 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400b4a0 tx=0x7f4c5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.452+0000 7f4c5b7fe640 1 -- 172.21.15.1:0/23310737 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54014070 con 0x7f4c64102570 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.452+0000 7f4c5b7fe640 1 -- 172.21.15.1:0/23310737 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c54010af0 con 0x7f4c64102570 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.452+0000 7f4c6c574640 1 -- 172.21.15.1:0/23310737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6419d7b0 con 0x7f4c64102570 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.452+0000 7f4c5b7fe640 1 -- 172.21.15.1:0/23310737 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54017d10 con 0x7f4c64102570 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.452+0000 7f4c6c574640 1 -- 172.21.15.1:0/23310737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c641a5090 con 0x7f4c64102570 2024-09-16T00:43:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.454+0000 7f4c5b7fe640 1 -- 172.21.15.1:0/23310737 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4c54017070 con 0x7f4c64102570 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.454+0000 7f4c6e94d740 1 Processor -- start 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6e94d740 1 -- start start 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed63b40 0x555b6ed62090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 0x555b6ed62a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed6d370 0x555b6ed6f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555b6ecc82f0 con 0x555b6ed63b40 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555b6ec6fcf0 con 0x555b6ed6d370 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b6ecc42b0 con 0x555b6ed625d0 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6ab11640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed63b40 0x555b6ed62090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6ab11640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed63b40 0x555b6ed62090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42680/0 (socket says 172.21.15.1:42680) 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6ab11640 1 -- 172.21.15.1:0/1512556481 learned_addr learned my addr 172.21.15.1:0/1512556481 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c5b7fe640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4c500a3240 0x7f4c500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6a310640 1 --2- 172.21.15.1:0/1512556481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 0x555b6ed62a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/1512556481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed6d370 0x555b6ed6f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6ab11640 1 -- 172.21.15.1:0/1512556481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 msgr2=0x555b6ed62a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/1512556481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 0x555b6ed62a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6ab11640 1 -- 172.21.15.1:0/1512556481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed6d370 msgr2=0x555b6ed6f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/1512556481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed6d370 0x555b6ed6f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.455+0000 7f4c6ab11640 1 -- 172.21.15.1:0/1512556481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6ec543a0 con 0x555b6ed63b40 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.456+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4c500a3240 0x7f4c500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.456+0000 7f4c5b7fe640 1 -- 172.21.15.1:0/23310737 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4c540cf200 con 0x7f4c64102570 2024-09-16T00:43:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.456+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/1512556481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed63b40 0x555b6ed62090 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c007fd0 tx=0x7f4c5c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7627339c36f7a99 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.456+0000 7f4c58ff9640 1 -- 172.21.15.1:0/1512556481 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c022070 con 0x555b6ed63b40 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.456+0000 7f4c58ff9640 1 -- 172.21.15.1:0/1512556481 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c011c30 con 0x555b6ed63b40 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.456+0000 7f4c58ff9640 1 -- 172.21.15.1:0/1512556481 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c01a400 con 0x555b6ed63b40 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.456+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4c500a3240 0x7f4c500a5700 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c01f000 tx=0x7f4c5c020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1512556481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed63b40 msgr2=0x555b6ed62090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1512556481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed63b40 0x555b6ed62090 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c007fd0 tx=0x7f4c5c00e7e0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1512556481 shutdown_connections 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1512556481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed6d370 0x555b6ed6f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1512556481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 0x555b6ed62a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1512556481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed63b40 0x555b6ed62090 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1512556481 >> 172.21.15.1:0/1512556481 conn(0x555b6ed2b310 msgr2=0x555b6ed5ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1512556481 shutdown_connections 2024-09-16T00:43:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1512556481 wait complete. 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 -- start start 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.457+0000 7f4c6e94d740 1 Processor -- start 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 0x555b6ed6cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 0x555b6ed6a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed6a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555b6ed70070 con 0x555b6ed6d370 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555b6ec83060 con 0x555b6ed63b40 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b6ed701f0 con 0x555b6ed625d0 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6e94d740 10 client.0 _ll_register_callbacks cb 0x555b6e9d68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6ab11640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed6a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6ab11640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed6a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42684/0 (socket says 172.21.15.1:42684) 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6ab11640 1 -- 172.21.15.1:0/1830798775 learned_addr learned my addr 172.21.15.1:0/1830798775 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 0x555b6ed6a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6a310640 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 0x555b6ed6cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6ab11640 1 -- 172.21.15.1:0/1830798775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 msgr2=0x555b6ed6cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 0x555b6ed6cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6ab11640 1 -- 172.21.15.1:0/1830798775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 msgr2=0x555b6ed6a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 0x555b6ed6a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6ab11640 1 -- 172.21.15.1:0/1830798775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6ea393c0 con 0x555b6ed6d370 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed6a600 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c012480 tx=0x7f4c5c012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5319c81a3713f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.458+0000 7f4c4dffb640 1 -- 172.21.15.1:0/1830798775 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c034060 con 0x555b6ed6d370 2024-09-16T00:43:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c4dffb640 1 -- 172.21.15.1:0/1830798775 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4c5c011a30 con 0x555b6ed6d370 2024-09-16T00:43:06.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1830798775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5c011c30 con 0x555b6ed6d370 2024-09-16T00:43:06.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c4dffb640 1 -- 172.21.15.1:0/1830798775 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4c5c03b050 con 0x555b6ed6d370 2024-09-16T00:43:06.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c6e94d740 10 client.18885 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:06.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1830798775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b6ed7bb10 con 0x555b6ed6d370 2024-09-16T00:43:06.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c4dffb640 1 -- 172.21.15.1:0/1830798775 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c011c30 con 0x555b6ed6d370 2024-09-16T00:43:06.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c4dffb640 1 -- 172.21.15.1:0/1830798775 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4c5c019040 con 0x555b6ed6d370 2024-09-16T00:43:06.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c6e94d740 10 client.18885 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:06.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c6e94d740 10 client.18885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:43:06.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c6e94d740 20 client.18885 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555b6ed7cf10 0x555b6ed7f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c6e94d740 4 client.18885 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.459+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1830798775 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b6ecd5860 con 0x555b6ed7cf10 2024-09-16T00:43:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.460+0000 7f4c6a310640 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555b6ed7cf10 0x555b6ed7f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.460+0000 7f4c6a310640 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555b6ed7cf10 0x555b6ed7f300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52b08dbebf9311a1 server_cookie=9d9675a40bb39c3e in_seq=0 out_seq=0 2024-09-16T00:43:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.460+0000 7f4c4dffb640 10 client.18885 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:43:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c4dffb640 1 -- 172.21.15.1:0/1830798775 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b6ecd5860 con 0x555b6ed7cf10 2024-09-16T00:43:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c4dffb640 10 client.18885 handle_command_reply: tid=0 2024-09-16T00:43:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 client.18885 shutdown 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1830798775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 msgr2=0x555b6ed6a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed6a600 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c012480 tx=0x7f4c5c012670 comp rx=0 tx=0).stop 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1830798775 shutdown_connections 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555b6ed7cf10 0x555b6ed7f300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed6a600 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c4dffb640 0 client.18885 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 0x555b6ed6a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/1830798775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed625d0 0x555b6ed6cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1830798775 >> 172.21.15.1:0/1830798775 conn(0x555b6ed2b310 msgr2=0x555b6ed64960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1830798775 shutdown_connections 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 1 -- 172.21.15.1:0/1830798775 wait complete. 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.461+0000 7f4c6e94d740 20 client.18885 trim_cache size 0 max 16384 2024-09-16T00:43:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6e94d740 1 Processor -- start 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6e94d740 1 -- start start 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed625d0 0x555b6ed5ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed63b40 0x555b6ed79310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed65120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641b16d0 con 0x555b6ed6d370 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641aefa0 con 0x555b6ed625d0 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641acdf0 con 0x555b6ed63b40 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6ab11640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed65120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6ab11640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed65120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42692/0 (socket says 172.21.15.1:42692) 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.474+0000 7f4c6ab11640 1 -- 172.21.15.1:0/3766458625 learned_addr learned my addr 172.21.15.1:0/3766458625 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed63b40 0x555b6ed79310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c6a310640 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed625d0 0x555b6ed5ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c6ab11640 1 -- 172.21.15.1:0/3766458625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed63b40 msgr2=0x555b6ed79310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed63b40 0x555b6ed79310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c6ab11640 1 -- 172.21.15.1:0/3766458625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed625d0 msgr2=0x555b6ed5ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed625d0 0x555b6ed5ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c6ab11640 1 -- 172.21.15.1:0/3766458625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6ed59ab0 con 0x555b6ed6d370 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c6a310640 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed625d0 0x555b6ed5ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed65120 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0077a0 tx=0x7f4c5c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af665fc2969af6d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c4ffff640 1 -- 172.21.15.1:0/3766458625 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c034040 con 0x555b6ed6d370 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c4ffff640 1 -- 172.21.15.1:0/3766458625 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c007ad0 con 0x555b6ed6d370 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.475+0000 7f4c4ffff640 1 -- 172.21.15.1:0/3766458625 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c022040 con 0x555b6ed6d370 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3766458625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 msgr2=0x555b6ed65120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed65120 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0077a0 tx=0x7f4c5c00e7e0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3766458625 shutdown_connections 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed6d370 0x555b6ed65120 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed63b40 0x555b6ed79310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3766458625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed625d0 0x555b6ed5ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3766458625 >> 172.21.15.1:0/3766458625 conn(0x555b6ed2b310 msgr2=0x555b6ed69160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3766458625 shutdown_connections 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3766458625 wait complete. 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 -- start start 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.476+0000 7f4c6e94d740 1 Processor -- start 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed625d0 0x555b6ed75660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 0x555b6ed75ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6e94d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed6d370 0x555b6ed66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641ad460 con 0x555b6ed625d0 2024-09-16T00:43:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641ae8d0 con 0x555b6ed63b40 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6e94d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641adb30 con 0x555b6ed6d370 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6e94d740 10 client.0 _ll_register_callbacks cb 0x7f4c5c020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6a310640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed625d0 0x555b6ed75660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6a310640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed625d0 0x555b6ed75660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42698/0 (socket says 172.21.15.1:42698) 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6a310640 1 -- 172.21.15.1:0/3362375894 learned_addr learned my addr 172.21.15.1:0/3362375894 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6ab11640 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed6d370 0x555b6ed66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 0x555b6ed75ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6a310640 1 -- 172.21.15.1:0/3362375894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed6d370 msgr2=0x555b6ed66880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6a310640 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed6d370 0x555b6ed66880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6a310640 1 -- 172.21.15.1:0/3362375894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 msgr2=0x555b6ed75ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6a310640 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 0x555b6ed75ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6a310640 1 -- 172.21.15.1:0/3362375894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6ed670b0 con 0x555b6ed625d0 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 0x555b6ed75ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c6a310640 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed625d0 0x555b6ed75660 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4c6002a4c0 tx=0x7f4c60004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33462565376a77bc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.477+0000 7f4c4dffb640 1 -- 172.21.15.1:0/3362375894 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60035070 con 0x555b6ed625d0 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c4dffb640 1 -- 172.21.15.1:0/3362375894 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4c60005450 con 0x555b6ed625d0 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3362375894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5c007ad0 con 0x555b6ed625d0 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c4dffb640 1 -- 172.21.15.1:0/3362375894 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4c60052050 con 0x555b6ed625d0 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c6e94d740 10 client.18891 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3362375894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b6ede74e0 con 0x555b6ed625d0 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c4dffb640 1 -- 172.21.15.1:0/3362375894 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c007ad0 con 0x555b6ed625d0 2024-09-16T00:43:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c4dffb640 1 -- 172.21.15.1:0/3362375894 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4c60003890 con 0x555b6ed625d0 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c6e94d740 10 client.18891 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c6e94d740 10 client.18891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c6e94d740 20 client.18891 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555b6ed7cf10 0x555b6ede9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c6e94d740 4 client.18891 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3362375894 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b6eccfd70 con 0x555b6ed7cf10 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.478+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555b6ed7cf10 0x555b6ede9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c4dffb640 10 client.18891 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c69b0f640 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555b6ed7cf10 0x555b6ede9c70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43463497df3605e5 server_cookie=bdc1958ae97e2b03 in_seq=0 out_seq=0 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c4dffb640 1 -- 172.21.15.1:0/3362375894 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b6eccfd70 con 0x555b6ed7cf10 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c4dffb640 10 client.18891 handle_command_reply: tid=0 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 client.18891 shutdown 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3362375894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed625d0 msgr2=0x555b6ed75660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed625d0 0x555b6ed75660 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4c6002a4c0 tx=0x7f4c60004410 comp rx=0 tx=0).stop 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3362375894 shutdown_connections 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6ed6d370 0x555b6ed66880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6ed63b40 0x555b6ed75ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x555b6ed7cf10 0x555b6ede9c70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 --2- 172.21.15.1:0/3362375894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6ed625d0 0x555b6ed75660 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3362375894 >> 172.21.15.1:0/3362375894 conn(0x555b6ed2b310 msgr2=0x555b6ed5d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c4dffb640 0 client.18891 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3362375894 shutdown_connections 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 1 -- 172.21.15.1:0/3362375894 wait complete. 2024-09-16T00:43:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.479+0000 7f4c6e94d740 20 client.18891 trim_cache size 0 max 16384 2024-09-16T00:43:06.499 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:43:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 -- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4c500a3240 msgr2=0x7f4c500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4c500a3240 0x7f4c500a5700 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c01f000 tx=0x7f4c5c020000 comp rx=0 tx=0).stop 2024-09-16T00:43:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 -- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64102570 msgr2=0x7f4c64198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64102570 0x7f4c64198a50 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400b4a0 tx=0x7f4c5400b960 comp rx=0 tx=0).stop 2024-09-16T00:43:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 -- 172.21.15.1:0/23310737 shutdown_connections 2024-09-16T00:43:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4c500a3240 0x7f4c500a5700 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c64102ef0 0x7f4c6419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c64102570 0x7f4c64198a50 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 --2- 172.21.15.1:0/23310737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c64100e20 0x7f4c64198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:06.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 -- 172.21.15.1:0/23310737 >> 172.21.15.1:0/23310737 conn(0x7f4c640fc7d0 msgr2=0x7f4c6406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:06.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 -- 172.21.15.1:0/23310737 shutdown_connections 2024-09-16T00:43:06.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:06.480+0000 7f4c6c574640 1 -- 172.21.15.1:0/23310737 wait complete. 2024-09-16T00:43:07.217 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:43:07.472 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:07.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:07 smithi001 ceph-mon[30934]: pgmap v792: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-09-16T00:43:08.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.986+0000 7f85fc6d2640 1 -- 172.21.15.1:0/285933105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102570 msgr2=0x7f85f41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:08.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.986+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/285933105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102570 0x7f85f41029b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009900 tx=0x7f85e402a480 comp rx=0 tx=0).stop 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.986+0000 7f85fc6d2640 1 -- 172.21.15.1:0/285933105 shutdown_connections 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.986+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/285933105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f4102ef0 0x7f85f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.986+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/285933105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102570 0x7f85f41029b0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.986+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/285933105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4100e20 0x7f85f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.986+0000 7f85fc6d2640 1 -- 172.21.15.1:0/285933105 >> 172.21.15.1:0/285933105 conn(0x7f85f40fc7d0 msgr2=0x7f85f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.986+0000 7f85fc6d2640 1 -- 172.21.15.1:0/285933105 shutdown_connections 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.986+0000 7f85fc6d2640 1 -- 172.21.15.1:0/285933105 wait complete. 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fc6d2640 1 Processor -- start 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fc6d2640 1 -- start start 2024-09-16T00:43:08.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fc6d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4100e20 0x7f85f4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fc6d2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f4102570 0x7f85f4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fc6d2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102ef0 0x7f85f419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fc6d2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4110fa0 con 0x7f85f4102ef0 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fc6d2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4110e20 con 0x7f85f4100e20 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fc6d2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4111120 con 0x7f85f4102570 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fac6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102ef0 0x7f85f419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fac6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102ef0 0x7f85f419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42718/0 (socket says 172.21.15.1:42718) 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fac6f640 1 -- 172.21.15.1:0/325826229 learned_addr learned my addr 172.21.15.1:0/325826229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fa46e640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4100e20 0x7f85f4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f4102570 0x7f85f4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fac6f640 1 -- 172.21.15.1:0/325826229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f4102570 msgr2=0x7f85f4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fac6f640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f4102570 0x7f85f4198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fac6f640 1 -- 172.21.15.1:0/325826229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4100e20 msgr2=0x7f85f4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fac6f640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4100e20 0x7f85f4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.987+0000 7f85fac6f640 1 -- 172.21.15.1:0/325826229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f419d4d0 con 0x7f85f4102ef0 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.988+0000 7f85fac6f640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102ef0 0x7f85f419cdf0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00b970 tx=0x7f85ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.988+0000 7f85eb7fe640 1 -- 172.21.15.1:0/325826229 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec019070 con 0x7f85f4102ef0 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.988+0000 7f85eb7fe640 1 -- 172.21.15.1:0/325826229 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ec00cb40 con 0x7f85f4102ef0 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.988+0000 7f85eb7fe640 1 -- 172.21.15.1:0/325826229 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec007650 con 0x7f85f4102ef0 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.988+0000 7f85fc6d2640 1 -- 172.21.15.1:0/325826229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f419d7c0 con 0x7f85f4102ef0 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.988+0000 7f85fc6d2640 1 -- 172.21.15.1:0/325826229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f41a50a0 con 0x7f85f4102ef0 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.990+0000 7f85eb7fe640 1 -- 172.21.15.1:0/325826229 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f85ec01c070 con 0x7f85f4102ef0 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.990+0000 7f85feaab740 1 Processor -- start 2024-09-16T00:43:08.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.990+0000 7f85feaab740 1 -- start start 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85feaab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 0x556379d76090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85feaab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d765d0 0x556379d76a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85feaab740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d87b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85feaab740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556379cdc2f0 con 0x556379d85780 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85feaab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556379c83cf0 con 0x556379d77b40 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85feaab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556379cd82b0 con 0x556379d765d0 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d87b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85eb7fe640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f85e00a32c0 0x7f85e00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d87b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42722/0 (socket says 172.21.15.1:42722) 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 -- 172.21.15.1:0/1100237350 learned_addr learned my addr 172.21.15.1:0/1100237350 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85fa46e640 1 --2- 172.21.15.1:0/1100237350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d765d0 0x556379d76a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/1100237350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 0x556379d76090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85fac6f640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f85e00a32c0 0x7f85e00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 -- 172.21.15.1:0/1100237350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d765d0 msgr2=0x556379d76a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/1100237350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d765d0 0x556379d76a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 -- 172.21.15.1:0/1100237350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 msgr2=0x556379d76090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/1100237350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 0x556379d76090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 -- 172.21.15.1:0/1100237350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556379c683a0 con 0x556379d85780 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85eb7fe640 1 -- 172.21.15.1:0/325826229 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f85ec0cf290 con 0x7f85f4102ef0 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.991+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/1100237350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d87b70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f85e40314e0 tx=0x7f85e40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6282eafb787ac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.992+0000 7f85fac6f640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f85e00a32c0 0x7f85e00a5780 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f85ec09b260 tx=0x7f85ec00f300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.992+0000 7f85e8ff9640 1 -- 172.21.15.1:0/1100237350 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4035040 con 0x556379d85780 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.992+0000 7f85e8ff9640 1 -- 172.21.15.1:0/1100237350 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e4003ad0 con 0x556379d85780 2024-09-16T00:43:08.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.992+0000 7f85e8ff9640 1 -- 172.21.15.1:0/1100237350 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4031cb0 con 0x556379d85780 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.992+0000 7f85feaab740 1 -- 172.21.15.1:0/1100237350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 msgr2=0x556379d87b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.992+0000 7f85feaab740 1 --2- 172.21.15.1:0/1100237350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d87b70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f85e40314e0 tx=0x7f85e40056d0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 -- 172.21.15.1:0/1100237350 shutdown_connections 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 --2- 172.21.15.1:0/1100237350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d87b70 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 --2- 172.21.15.1:0/1100237350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d765d0 0x556379d76a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 --2- 172.21.15.1:0/1100237350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 0x556379d76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 -- 172.21.15.1:0/1100237350 >> 172.21.15.1:0/1100237350 conn(0x556379d3f310 msgr2=0x556379d72c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 -- 172.21.15.1:0/1100237350 shutdown_connections 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 -- 172.21.15.1:0/1100237350 wait complete. 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 -- start start 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 Processor -- start 2024-09-16T00:43:08.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d85180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 0x556379d82430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d85780 0x556379d82a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556379d88420 con 0x556379d765d0 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556379c97060 con 0x556379d85780 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556379d885a0 con 0x556379d77b40 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85feaab740 10 client.0 _ll_register_callbacks cb 0x5563799ea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85f9c6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d85180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85f9c6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d85180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42726/0 (socket says 172.21.15.1:42726) 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.993+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2299833815 learned_addr learned my addr 172.21.15.1:0/2299833815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d85780 0x556379d82a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85fa46e640 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 0x556379d82430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2299833815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 msgr2=0x556379d82430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 0x556379d82430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2299833815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d85780 msgr2=0x556379d82a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d85780 0x556379d82a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2299833815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556379a4d3c0 con 0x556379d765d0 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d85180 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f85e402a4c0 tx=0x7f85e4031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d70de25e44ede4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85ddffb640 1 -- 172.21.15.1:0/2299833815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4035030 con 0x556379d765d0 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85ddffb640 1 -- 172.21.15.1:0/2299833815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f85e4031c40 con 0x556379d765d0 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85feaab740 1 -- 172.21.15.1:0/2299833815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e4003ad0 con 0x556379d765d0 2024-09-16T00:43:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85ddffb640 1 -- 172.21.15.1:0/2299833815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85e4059050 con 0x556379d765d0 2024-09-16T00:43:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85feaab740 10 client.18903 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85feaab740 1 -- 172.21.15.1:0/2299833815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556379d93ec0 con 0x556379d765d0 2024-09-16T00:43:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.994+0000 7f85ddffb640 1 -- 172.21.15.1:0/2299833815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e4003ad0 con 0x556379d765d0 2024-09-16T00:43:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85ddffb640 1 -- 172.21.15.1:0/2299833815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f85e403a040 con 0x556379d765d0 2024-09-16T00:43:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85feaab740 10 client.18903 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85feaab740 10 client.18903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85feaab740 20 client.18903 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85feaab740 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556379d95210 0x556379d97600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85feaab740 4 client.18903 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85feaab740 1 -- 172.21.15.1:0/2299833815 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556379ce9860 con 0x556379d95210 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85fa46e640 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556379d95210 0x556379d97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85fa46e640 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556379d95210 0x556379d97600 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b4d202a7bc1bb80 server_cookie=a163827792898fc7 in_seq=0 out_seq=0 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.995+0000 7f85ddffb640 10 client.18903 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85ddffb640 1 -- 172.21.15.1:0/2299833815 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556379ce9860 con 0x556379d95210 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85ddffb640 10 client.18903 handle_command_reply: tid=0 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 client.18903 shutdown 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 -- 172.21.15.1:0/2299833815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 msgr2=0x556379d85180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d85180 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f85e402a4c0 tx=0x7f85e4031950 comp rx=0 tx=0).stop 2024-09-16T00:43:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 -- 172.21.15.1:0/2299833815 shutdown_connections 2024-09-16T00:43:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556379d95210 0x556379d97600 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d85780 0x556379d82a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 0x556379d82430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 --2- 172.21.15.1:0/2299833815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d85180 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 -- 172.21.15.1:0/2299833815 >> 172.21.15.1:0/2299833815 conn(0x556379d3f310 msgr2=0x556379d78960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85ddffb640 0 client.18903 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 -- 172.21.15.1:0/2299833815 shutdown_connections 2024-09-16T00:43:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.996+0000 7f85feaab740 1 -- 172.21.15.1:0/2299833815 wait complete. 2024-09-16T00:43:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:07.997+0000 7f85feaab740 20 client.18903 trim_cache size 0 max 16384 2024-09-16T00:43:08.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.009+0000 7f85feaab740 1 Processor -- start 2024-09-16T00:43:08.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85feaab740 1 -- start start 2024-09-16T00:43:08.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85feaab740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85feaab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 0x556379d7f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85feaab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d85780 0x556379d7fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85feaab740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f85f41af4b0 con 0x556379d765d0 2024-09-16T00:43:08.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85feaab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85f41acd80 con 0x556379d77b40 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85feaab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85f41aabd0 con 0x556379d85780 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85f9c6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85f9c6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d843a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42740/0 (socket says 172.21.15.1:42740) 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2274411514 learned_addr learned my addr 172.21.15.1:0/2274411514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2274411514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d85780 msgr2=0x556379d7fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2274411514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d85780 0x556379d7fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85fa46e640 1 --2- 172.21.15.1:0/2274411514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 0x556379d7f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2274411514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 msgr2=0x556379d7f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2274411514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 0x556379d7f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2274411514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556379d84da0 con 0x556379d765d0 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2274411514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d843a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f85e4031920 tx=0x7f85e40319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2566cf856be9f6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.010+0000 7f85dffff640 1 -- 172.21.15.1:0/2274411514 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4035040 con 0x556379d765d0 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85dffff640 1 -- 172.21.15.1:0/2274411514 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e403cce0 con 0x556379d765d0 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85dffff640 1 -- 172.21.15.1:0/2274411514 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4005030 con 0x556379d765d0 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 -- 172.21.15.1:0/2274411514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 msgr2=0x556379d843a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 --2- 172.21.15.1:0/2274411514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d843a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f85e4031920 tx=0x7f85e40319c0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 -- 172.21.15.1:0/2274411514 shutdown_connections 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 --2- 172.21.15.1:0/2274411514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d85780 0x556379d7fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 --2- 172.21.15.1:0/2274411514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d77b40 0x556379d7f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 --2- 172.21.15.1:0/2274411514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d765d0 0x556379d843a0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 -- 172.21.15.1:0/2274411514 >> 172.21.15.1:0/2274411514 conn(0x556379d3f310 msgr2=0x556379d83c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 -- 172.21.15.1:0/2274411514 shutdown_connections 2024-09-16T00:43:08.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 -- 172.21.15.1:0/2274411514 wait complete. 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.011+0000 7f85feaab740 1 -- start start 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85feaab740 1 Processor -- start 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85feaab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d765d0 0x556379d9a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85feaab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 0x556379d97c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85feaab740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d98180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85feaab740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f85f41ab240 con 0x556379d85780 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85feaab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85f41ac6b0 con 0x556379d765d0 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85feaab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85f41ab910 con 0x556379d77b40 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85feaab740 10 client.0 _ll_register_callbacks cb 0x7f85e40043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85f9c6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d98180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85f9c6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d98180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42744/0 (socket says 172.21.15.1:42744) 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2805540058 learned_addr learned my addr 172.21.15.1:0/2805540058 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d765d0 0x556379d9a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85fa46e640 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 0x556379d97c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2805540058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 msgr2=0x556379d97c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 0x556379d97c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2805540058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d765d0 msgr2=0x556379d9a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d765d0 0x556379d9a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85f9c6d640 1 -- 172.21.15.1:0/2805540058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556379d989b0 con 0x556379d85780 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.012+0000 7f85fa46e640 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 0x556379d97c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85f9c6d640 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d98180 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f85e403c480 tx=0x7f85e4004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2062283796f4ab0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85ddffb640 1 -- 172.21.15.1:0/2805540058 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4035040 con 0x556379d85780 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85feaab740 1 -- 172.21.15.1:0/2805540058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e403cce0 con 0x556379d85780 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85ddffb640 1 -- 172.21.15.1:0/2805540058 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f85e40315e0 con 0x556379d85780 2024-09-16T00:43:08.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85ddffb640 1 -- 172.21.15.1:0/2805540058 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85e403c800 con 0x556379d85780 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85feaab740 10 client.18909 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85feaab740 1 -- 172.21.15.1:0/2805540058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556379dffac0 con 0x556379d85780 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85ddffb640 1 -- 172.21.15.1:0/2805540058 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e403cce0 con 0x556379d85780 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85ddffb640 1 -- 172.21.15.1:0/2805540058 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f85e403d9c0 con 0x556379d85780 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85feaab740 10 client.18909 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85feaab740 10 client.18909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85feaab740 20 client.18909 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85feaab740 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556379e00130 0x556379e02520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85feaab740 4 client.18909 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.013+0000 7f85feaab740 1 -- 172.21.15.1:0/2805540058 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556379ce3d70 con 0x556379e00130 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85fa46e640 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556379e00130 0x556379e02520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85ddffb640 10 client.18909 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85fa46e640 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556379e00130 0x556379e02520 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5cdf9b7f4561edd server_cookie=b93f4cbb7092ebe8 in_seq=0 out_seq=0 2024-09-16T00:43:08.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85ddffb640 1 -- 172.21.15.1:0/2805540058 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556379ce3d70 con 0x556379e00130 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85ddffb640 10 client.18909 handle_command_reply: tid=0 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85feaab740 1 client.18909 shutdown 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85feaab740 1 -- 172.21.15.1:0/2805540058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 msgr2=0x556379d98180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85feaab740 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d98180 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f85e403c480 tx=0x7f85e4004260 comp rx=0 tx=0).stop 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85feaab740 1 -- 172.21.15.1:0/2805540058 shutdown_connections 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85feaab740 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x556379e00130 0x556379e02520 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85feaab740 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556379d85780 0x556379d98180 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85feaab740 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556379d77b40 0x556379d97c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85feaab740 1 --2- 172.21.15.1:0/2805540058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556379d765d0 0x556379d9a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85feaab740 1 -- 172.21.15.1:0/2805540058 >> 172.21.15.1:0/2805540058 conn(0x556379d3f310 msgr2=0x556379d844b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:08.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.014+0000 7f85ddffb640 0 client.18909 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:08.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85feaab740 1 -- 172.21.15.1:0/2805540058 shutdown_connections 2024-09-16T00:43:08.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85feaab740 1 -- 172.21.15.1:0/2805540058 wait complete. 2024-09-16T00:43:08.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85feaab740 20 client.18909 trim_cache size 0 max 16384 2024-09-16T00:43:08.026 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:43:08.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 -- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f85e00a32c0 msgr2=0x7f85e00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:08.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f85e00a32c0 0x7f85e00a5780 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f85ec09b260 tx=0x7f85ec00f300 comp rx=0 tx=0).stop 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 -- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102ef0 msgr2=0x7f85f419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102ef0 0x7f85f419cdf0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00b970 tx=0x7f85ec00be30 comp rx=0 tx=0).stop 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 -- 172.21.15.1:0/325826229 shutdown_connections 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85f4102ef0 0x7f85f419cdf0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f85e00a32c0 0x7f85e00a5780 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f4102570 0x7f85f4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 --2- 172.21.15.1:0/325826229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4100e20 0x7f85f4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 -- 172.21.15.1:0/325826229 >> 172.21.15.1:0/325826229 conn(0x7f85f40fc7d0 msgr2=0x7f85f406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 -- 172.21.15.1:0/325826229 shutdown_connections 2024-09-16T00:43:08.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:08.015+0000 7f85fc6d2640 1 -- 172.21.15.1:0/325826229 wait complete. 2024-09-16T00:43:08.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:07 smithi019 ceph-mon[35186]: pgmap v792: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-09-16T00:43:08.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:07 smithi153 ceph-mon[35538]: pgmap v792: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-09-16T00:43:09.711 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:43:09.712 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:43:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:10.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:09 smithi001 ceph-mon[30934]: pgmap v793: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-09-16T00:43:10.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:09 smithi019 ceph-mon[35186]: pgmap v793: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-09-16T00:43:10.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:09 smithi153 ceph-mon[35538]: pgmap v793: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-09-16T00:43:10.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.498+0000 7f8418f09640 1 -- 172.21.15.1:0/2448691183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 msgr2=0x7f841410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:10.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.498+0000 7f8418f09640 1 --2- 172.21.15.1:0/2448691183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 0x7f841410d950 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f840c009cd0 tx=0x7f840c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.498+0000 7f8418f09640 1 -- 172.21.15.1:0/2448691183 shutdown_connections 2024-09-16T00:43:10.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.498+0000 7f8418f09640 1 --2- 172.21.15.1:0/2448691183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 0x7f841410d950 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.498+0000 7f8418f09640 1 --2- 172.21.15.1:0/2448691183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8414100a80 0x7f8414100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.498+0000 7f8418f09640 1 --2- 172.21.15.1:0/2448691183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8414106a80 0x7f8414106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.498+0000 7f8418f09640 1 -- 172.21.15.1:0/2448691183 >> 172.21.15.1:0/2448691183 conn(0x7f84140fc7a0 msgr2=0x7f84140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:10.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.498+0000 7f8418f09640 1 -- 172.21.15.1:0/2448691183 shutdown_connections 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.498+0000 7f8418f09640 1 -- 172.21.15.1:0/2448691183 wait complete. 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8418f09640 1 Processor -- start 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8418f09640 1 -- start start 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8418f09640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8414100a80 0x7f8414196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8418f09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 0x7f8414197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8418f09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8414106a80 0x7f8414197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8418f09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8414110e80 con 0x7f8414101420 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8418f09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8414110d00 con 0x7f8414106a80 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8418f09640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111000 con 0x7f8414100a80 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8409d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 0x7f8414197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8409d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 0x7f8414197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44118/0 (socket says 172.21.15.1:44118) 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8409d9b640 1 -- 172.21.15.1:0/1569378588 learned_addr learned my addr 172.21.15.1:0/1569378588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:10.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f841259c640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8414100a80 0x7f8414196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.499+0000 7f8412d9d640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8414106a80 0x7f8414197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f8409d9b640 1 -- 172.21.15.1:0/1569378588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8414100a80 msgr2=0x7f8414196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f8409d9b640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8414100a80 0x7f8414196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f8409d9b640 1 -- 172.21.15.1:0/1569378588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8414106a80 msgr2=0x7f8414197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f8409d9b640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8414106a80 0x7f8414197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f8409d9b640 1 -- 172.21.15.1:0/1569378588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84141a6850 con 0x7f8414101420 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f8412d9d640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8414106a80 0x7f8414197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f8409d9b640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 0x7f8414197520 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c890 tx=0x7f83fc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f840bfff640 1 -- 172.21.15.1:0/1569378588 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc004220 con 0x7f8414101420 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f8418f09640 1 -- 172.21.15.1:0/1569378588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84141a6b40 con 0x7f8414101420 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f840bfff640 1 -- 172.21.15.1:0/1569378588 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc0043c0 con 0x7f8414101420 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f840bfff640 1 -- 172.21.15.1:0/1569378588 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc004a00 con 0x7f8414101420 2024-09-16T00:43:10.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.500+0000 7f8418f09640 1 -- 172.21.15.1:0/1569378588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84141a70d0 con 0x7f8414101420 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.502+0000 7f840bfff640 1 -- 172.21.15.1:0/1569378588 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f83fc0079b0 con 0x7f8414101420 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f841b2e2740 1 Processor -- start 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f841b2e2740 1 -- start start 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f840bfff640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f84000a3240 0x7f84000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 0x5577e5345f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e5350800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5350d40 0x5577e5353130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f841b2e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5577e52ab280 con 0x5577e5346480 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f841b2e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577e5252b10 con 0x5577e5345b60 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f841b2e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5577e52a7240 con 0x5577e5350d40 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f840bfff640 1 -- 172.21.15.1:0/1569378588 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f83fc0cf9e0 con 0x7f8414101420 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f8412d9d640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f84000a3240 0x7f84000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f8409d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e5350800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f8409d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e5350800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44122/0 (socket says 172.21.15.1:44122) 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.503+0000 7f8409d9b640 1 -- 172.21.15.1:0/3709637293 learned_addr learned my addr 172.21.15.1:0/3709637293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f841259c640 1 --2- 172.21.15.1:0/3709637293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 0x5577e5345f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8409d9b640 1 -- 172.21.15.1:0/3709637293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5350d40 msgr2=0x5577e5353130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8409d9b640 1 --2- 172.21.15.1:0/3709637293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5350d40 0x5577e5353130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8409d9b640 1 -- 172.21.15.1:0/3709637293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 msgr2=0x5577e5345f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8409d9b640 1 --2- 172.21.15.1:0/3709637293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 0x5577e5345f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8409d9b640 1 -- 172.21.15.1:0/3709637293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e525f1f0 con 0x5577e5346480 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8412d9d640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f84000a3240 0x7f84000a5700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f840c009a60 tx=0x7f840c031530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8409d9b640 1 --2- 172.21.15.1:0/3709637293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e5350800 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0a4340 tx=0x7f83fc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688ee51e588e482a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8408d99640 1 -- 172.21.15.1:0/3709637293 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc01d040 con 0x5577e5346480 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8408d99640 1 -- 172.21.15.1:0/3709637293 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc09bc00 con 0x5577e5346480 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.504+0000 7f8408d99640 1 -- 172.21.15.1:0/3709637293 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc09b050 con 0x5577e5346480 2024-09-16T00:43:10.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 -- 172.21.15.1:0/3709637293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 msgr2=0x5577e5350800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 --2- 172.21.15.1:0/3709637293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e5350800 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0a4340 tx=0x7f83fc007f90 comp rx=0 tx=0).stop 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 -- 172.21.15.1:0/3709637293 shutdown_connections 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 --2- 172.21.15.1:0/3709637293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5350d40 0x5577e5353130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 --2- 172.21.15.1:0/3709637293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e5350800 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 --2- 172.21.15.1:0/3709637293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 0x5577e5345f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 -- 172.21.15.1:0/3709637293 >> 172.21.15.1:0/3709637293 conn(0x5577e53420f0 msgr2=0x5577e53424d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 -- 172.21.15.1:0/3709637293 shutdown_connections 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 -- 172.21.15.1:0/3709637293 wait complete. 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 -- start start 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.505+0000 7f841b2e2740 1 Processor -- start 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5345b60 0x5577e534dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5346480 0x5577e534e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5350d40 0x5577e534e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841b2e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5577e5353910 con 0x5577e5350d40 2024-09-16T00:43:10.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841b2e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577e5265f10 con 0x5577e5346480 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841b2e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5577e5353a90 con 0x5577e5345b60 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841b2e2740 10 client.0 _ll_register_callbacks cb 0x5577e522ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8412d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5350d40 0x5577e534e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8412d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5350d40 0x5577e534e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44136/0 (socket says 172.21.15.1:44136) 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8412d9d640 1 -- 172.21.15.1:0/861697566 learned_addr learned my addr 172.21.15.1:0/861697566 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841259c640 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5345b60 0x5577e534dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8409d9b640 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5346480 0x5577e534e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8412d9d640 1 -- 172.21.15.1:0/861697566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5345b60 msgr2=0x5577e534dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8412d9d640 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5345b60 0x5577e534dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8412d9d640 1 -- 172.21.15.1:0/861697566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5346480 msgr2=0x5577e534e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8412d9d640 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5346480 0x5577e534e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8412d9d640 1 -- 172.21.15.1:0/861697566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e501c6a0 con 0x5577e5350d40 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841259c640 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5345b60 0x5577e534dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:10.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f8412d9d640 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5350d40 0x5577e534e640 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f840c031860 tx=0x7f840c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc73fe89d4d22e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f83f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f83f9ffb640 1 -- 172.21.15.1:0/861697566 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c04c070 con 0x5577e5350d40 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f841b2e2740 1 -- 172.21.15.1:0/861697566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83fc09bc00 con 0x5577e5350d40 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.506+0000 7f83f9ffb640 1 -- 172.21.15.1:0/861697566 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f840c0305a0 con 0x5577e5350d40 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f83f9ffb640 1 -- 172.21.15.1:0/861697566 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f840c058050 con 0x5577e5350d40 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f841b2e2740 10 client.18921 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f841b2e2740 1 -- 172.21.15.1:0/861697566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577e535f4c0 con 0x5577e5350d40 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f83f9ffb640 1 -- 172.21.15.1:0/861697566 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc09bc00 con 0x5577e5350d40 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f83f9ffb640 1 -- 172.21.15.1:0/861697566 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f840c030070 con 0x5577e5350d40 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f841b2e2740 10 client.18921 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f841b2e2740 10 client.18921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f841b2e2740 20 client.18921 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:10.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f841b2e2740 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5577e5360920 0x5577e5362d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f841b2e2740 4 client.18921 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f841b2e2740 1 -- 172.21.15.1:0/861697566 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e52b87f0 con 0x5577e5360920 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.507+0000 7f841259c640 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5577e5360920 0x5577e5362d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.508+0000 7f83f9ffb640 10 client.18921 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.508+0000 7f841259c640 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5577e5360920 0x5577e5362d70 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad923d5a81adcae4 server_cookie=435af253024bd0ae in_seq=0 out_seq=0 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.508+0000 7f83f9ffb640 1 -- 172.21.15.1:0/861697566 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577e52b87f0 con 0x5577e5360920 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.508+0000 7f83f9ffb640 10 client.18921 handle_command_reply: tid=0 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.508+0000 7f841b2e2740 1 client.18921 shutdown 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 -- 172.21.15.1:0/861697566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5350d40 msgr2=0x5577e534e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5350d40 0x5577e534e640 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f840c031860 tx=0x7f840c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 -- 172.21.15.1:0/861697566 shutdown_connections 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5577e5360920 0x5577e5362d70 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5350d40 0x5577e534e640 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5346480 0x5577e534e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f83f9ffb640 0 client.18921 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 --2- 172.21.15.1:0/861697566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5345b60 0x5577e534dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 -- 172.21.15.1:0/861697566 >> 172.21.15.1:0/861697566 conn(0x5577e53420f0 msgr2=0x5577e5347e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 -- 172.21.15.1:0/861697566 shutdown_connections 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 1 -- 172.21.15.1:0/861697566 wait complete. 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.509+0000 7f841b2e2740 20 client.18921 trim_cache size 0 max 16384 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f841b2e2740 1 Processor -- start 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f841b2e2740 1 -- start start 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 0x5577e53485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e534fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5350d40 0x5577e5368640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f841b2e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f84141b38c0 con 0x5577e5346480 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f841b2e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84141b1190 con 0x5577e5345b60 2024-09-16T00:43:10.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f841b2e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84141aefe0 con 0x5577e5350d40 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f8409d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e534fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f8409d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e534fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44146/0 (socket says 172.21.15.1:44146) 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.526+0000 7f8409d9b640 1 -- 172.21.15.1:0/2714138081 learned_addr learned my addr 172.21.15.1:0/2714138081 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f841259c640 1 --2- 172.21.15.1:0/2714138081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 0x5577e53485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f8409d9b640 1 -- 172.21.15.1:0/2714138081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5350d40 msgr2=0x5577e5368640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f8409d9b640 1 --2- 172.21.15.1:0/2714138081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5350d40 0x5577e5368640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f8409d9b640 1 -- 172.21.15.1:0/2714138081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 msgr2=0x5577e53485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f8409d9b640 1 --2- 172.21.15.1:0/2714138081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 0x5577e53485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f8409d9b640 1 -- 172.21.15.1:0/2714138081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e53462b0 con 0x5577e5346480 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f841259c640 1 --2- 172.21.15.1:0/2714138081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 0x5577e53485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f8409d9b640 1 --2- 172.21.15.1:0/2714138081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e534fa20 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0058c0 tx=0x7f83fc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2737e6b2f2118 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f83fbfff640 1 -- 172.21.15.1:0/2714138081 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc01d040 con 0x5577e5346480 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f83fbfff640 1 -- 172.21.15.1:0/2714138081 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc09bc00 con 0x5577e5346480 2024-09-16T00:43:10.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.527+0000 7f83fbfff640 1 -- 172.21.15.1:0/2714138081 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc09b050 con 0x5577e5346480 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 -- 172.21.15.1:0/2714138081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 msgr2=0x5577e534fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 --2- 172.21.15.1:0/2714138081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e534fa20 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0058c0 tx=0x7f83fc00c810 comp rx=0 tx=0).stop 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 -- 172.21.15.1:0/2714138081 shutdown_connections 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 --2- 172.21.15.1:0/2714138081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5350d40 0x5577e5368640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 --2- 172.21.15.1:0/2714138081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5346480 0x5577e534fa20 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 --2- 172.21.15.1:0/2714138081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5345b60 0x5577e53485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 -- 172.21.15.1:0/2714138081 >> 172.21.15.1:0/2714138081 conn(0x5577e53420f0 msgr2=0x5577e534f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 -- 172.21.15.1:0/2714138081 shutdown_connections 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 -- 172.21.15.1:0/2714138081 wait complete. 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 -- start start 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.528+0000 7f841b2e2740 1 Processor -- start 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5345b60 0x5577e53595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5346480 0x5577e5355cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841b2e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5350d40 0x5577e5356440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841b2e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f84141af650 con 0x5577e5345b60 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841b2e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84141b0ac0 con 0x5577e5350d40 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841b2e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84141afd20 con 0x5577e5346480 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841b2e2740 10 client.0 _ll_register_callbacks cb 0x7f840c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5345b60 0x5577e53595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5345b60 0x5577e53595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44148/0 (socket says 172.21.15.1:44148) 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841259c640 1 -- 172.21.15.1:0/652839114 learned_addr learned my addr 172.21.15.1:0/652839114 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f8409d9b640 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5346480 0x5577e5355cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841259c640 1 -- 172.21.15.1:0/652839114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5346480 msgr2=0x5577e5355cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841259c640 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5346480 0x5577e5355cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f8412d9d640 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5350d40 0x5577e5356440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841259c640 1 -- 172.21.15.1:0/652839114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5350d40 msgr2=0x5577e5356440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841259c640 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5350d40 0x5577e5356440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841259c640 1 -- 172.21.15.1:0/652839114 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e53cb620 con 0x5577e5345b60 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f8409d9b640 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5346480 0x5577e5355cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f8412d9d640 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5350d40 0x5577e5356440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841259c640 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5345b60 0x5577e53595b0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x5577e5363580 tx=0x7f8404013b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebab3c34bf75ef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f83f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f83f9ffb640 1 -- 172.21.15.1:0/652839114 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840400faf0 con 0x5577e5345b60 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f83f9ffb640 1 -- 172.21.15.1:0/652839114 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f840400fc90 con 0x5577e5345b60 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.529+0000 7f841b2e2740 1 -- 172.21.15.1:0/652839114 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83fc09bc00 con 0x5577e5345b60 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f83f9ffb640 1 -- 172.21.15.1:0/652839114 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f840402c050 con 0x5577e5345b60 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f841b2e2740 10 client.18927 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f841b2e2740 1 -- 172.21.15.1:0/652839114 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577e53cc850 con 0x5577e5345b60 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f83f9ffb640 1 -- 172.21.15.1:0/652839114 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc09bc00 con 0x5577e5345b60 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f83f9ffb640 1 -- 172.21.15.1:0/652839114 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8404011600 con 0x5577e5345b60 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f841b2e2740 10 client.18927 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:10.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f841b2e2740 10 client.18927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:10.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f841b2e2740 20 client.18927 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:10.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f841b2e2740 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5577e53cd050 0x5577e53cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:10.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f841b2e2740 4 client.18927 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:43:10.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f841b2e2740 1 -- 172.21.15.1:0/652839114 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5577e52b2d00 con 0x5577e53cd050 2024-09-16T00:43:10.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.530+0000 7f8409d9b640 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5577e53cd050 0x5577e53cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:10.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.531+0000 7f8409d9b640 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5577e53cd050 0x5577e53cf470 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c4756e3ce4faa4 server_cookie=c09062c18c48e672 in_seq=0 out_seq=0 2024-09-16T00:43:10.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.531+0000 7f83f9ffb640 10 client.18927 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f83f9ffb640 1 -- 172.21.15.1:0/652839114 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5577e52b2d00 con 0x5577e53cd050 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f83f9ffb640 10 client.18927 handle_command_reply: tid=0 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 client.18927 shutdown 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 -- 172.21.15.1:0/652839114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5345b60 msgr2=0x5577e53595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5345b60 0x5577e53595b0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x5577e5363580 tx=0x7f8404013b20 comp rx=0 tx=0).stop 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 -- 172.21.15.1:0/652839114 shutdown_connections 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x5577e53cd050 0x5577e53cf470 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577e5350d40 0x5577e5356440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577e5346480 0x5577e5355cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 --2- 172.21.15.1:0/652839114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577e5345b60 0x5577e53595b0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 -- 172.21.15.1:0/652839114 >> 172.21.15.1:0/652839114 conn(0x5577e53420f0 msgr2=0x5577e534e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:10.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f83f9ffb640 0 client.18927 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:10.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 -- 172.21.15.1:0/652839114 shutdown_connections 2024-09-16T00:43:10.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 1 -- 172.21.15.1:0/652839114 wait complete. 2024-09-16T00:43:10.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.532+0000 7f841b2e2740 20 client.18927 trim_cache size 0 max 16384 2024-09-16T00:43:10.547 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:43:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:43:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "42384e8b-e098-4237-affc-12e416df2151", 2024-09-16T00:43:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:43:10.547 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 -- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f84000a3240 msgr2=0x7f84000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f84000a3240 0x7f84000a5700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f840c009a60 tx=0x7f840c031530 comp rx=0 tx=0).stop 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 -- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 msgr2=0x7f8414197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 0x7f8414197520 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c890 tx=0x7f83fc00cc90 comp rx=0 tx=0).stop 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 -- 172.21.15.1:0/1569378588 shutdown_connections 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8414106a80 0x7f8414197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8414101420 0x7f8414197520 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f84000a3240 0x7f84000a5700 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 --2- 172.21.15.1:0/1569378588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8414100a80 0x7f8414196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 -- 172.21.15.1:0/1569378588 >> 172.21.15.1:0/1569378588 conn(0x7f84140fc7a0 msgr2=0x7f84140fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:10.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 -- 172.21.15.1:0/1569378588 shutdown_connections 2024-09-16T00:43:10.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:10.533+0000 7f8418f09640 1 -- 172.21.15.1:0/1569378588 wait complete. 2024-09-16T00:43:11.274 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:43:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:11.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:11 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:43:11.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:11 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:11.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:11 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:43:11.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:11 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:11.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:11 smithi001 ceph-mon[30934]: pgmap v794: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:12.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.038+0000 7fa076f73640 1 -- 172.21.15.1:0/2587032249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070106f10 msgr2=0x7fa07010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:12.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.038+0000 7fa076f73640 1 --2- 172.21.15.1:0/2587032249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070106f10 0x7fa07010ea90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa068009900 tx=0x7fa06802a410 comp rx=0 tx=0).stop 2024-09-16T00:43:12.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.038+0000 7fa076f73640 1 -- 172.21.15.1:0/2587032249 shutdown_connections 2024-09-16T00:43:12.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.038+0000 7fa076f73640 1 --2- 172.21.15.1:0/2587032249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070106f10 0x7fa07010ea90 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.038+0000 7fa076f73640 1 --2- 172.21.15.1:0/2587032249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa070102790 0x7fa0701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.038+0000 7fa076f73640 1 --2- 172.21.15.1:0/2587032249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa070101de0 0x7fa0701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.038+0000 7fa076f73640 1 -- 172.21.15.1:0/2587032249 >> 172.21.15.1:0/2587032249 conn(0x7fa0700fd8c0 msgr2=0x7fa0700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.038+0000 7fa076f73640 1 -- 172.21.15.1:0/2587032249 shutdown_connections 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.038+0000 7fa076f73640 1 -- 172.21.15.1:0/2587032249 wait complete. 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa076f73640 1 Processor -- start 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa076f73640 1 -- start start 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa076f73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070101de0 0x7fa070198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa076f73640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa070102790 0x7fa070198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa076f73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa070106f10 0x7fa07019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa076f73640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa070111f70 con 0x7fa070101de0 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa076f73640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa070111df0 con 0x7fa070106f10 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa076f73640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701120f0 con 0x7fa070102790 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa074d0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070101de0 0x7fa070198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa074d0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070101de0 0x7fa070198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44162/0 (socket says 172.21.15.1:44162) 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa074d0f640 1 -- 172.21.15.1:0/3748108813 learned_addr learned my addr 172.21.15.1:0/3748108813 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa06ffff640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa070102790 0x7fa070198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa075510640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa070106f10 0x7fa07019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa074d0f640 1 -- 172.21.15.1:0/3748108813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa070102790 msgr2=0x7fa070198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa074d0f640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa070102790 0x7fa070198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa074d0f640 1 -- 172.21.15.1:0/3748108813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa070106f10 msgr2=0x7fa07019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa074d0f640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa070106f10 0x7fa07019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.039+0000 7fa074d0f640 1 -- 172.21.15.1:0/3748108813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07019d3c0 con 0x7fa070101de0 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.040+0000 7fa074d0f640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070101de0 0x7fa070198410 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa05800c890 tx=0x7fa05800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.040+0000 7fa06dffb640 1 -- 172.21.15.1:0/3748108813 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058004220 con 0x7fa070101de0 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.040+0000 7fa076f73640 1 -- 172.21.15.1:0/3748108813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07019d6b0 con 0x7fa070101de0 2024-09-16T00:43:12.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.040+0000 7fa06dffb640 1 -- 172.21.15.1:0/3748108813 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0580043c0 con 0x7fa070101de0 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.040+0000 7fa06dffb640 1 -- 172.21.15.1:0/3748108813 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058004a00 con 0x7fa070101de0 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.040+0000 7fa076f73640 1 -- 172.21.15.1:0/3748108813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0701a4f90 con 0x7fa070101de0 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.043+0000 7fa06dffb640 1 -- 172.21.15.1:0/3748108813 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa0580079b0 con 0x7fa070101de0 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.043+0000 7fa07934c740 1 Processor -- start 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.043+0000 7fa07934c740 1 -- start start 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.043+0000 7fa07934c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 0x55cd323ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa07934c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323cedc0 0x55cd323d9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa07934c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323d9680 0x55cd323dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa07934c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd323342e0 con 0x55cd323cedc0 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa07934c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd322dbce0 con 0x55cd323ce4a0 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa07934c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd323302a0 con 0x55cd323d9680 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa06dffb640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa05c0a3240 0x7fa05c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa074d0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323cedc0 0x55cd323d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa074d0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323cedc0 0x55cd323d9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44166/0 (socket says 172.21.15.1:44166) 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa074d0f640 1 -- 172.21.15.1:0/2549291831 learned_addr learned my addr 172.21.15.1:0/2549291831 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa075510640 1 --2- 172.21.15.1:0/2549291831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 0x55cd323ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa06ffff640 1 --2- 172.21.15.1:0/2549291831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323d9680 0x55cd323dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa075510640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa05c0a3240 0x7fa05c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa06dffb640 1 -- 172.21.15.1:0/3748108813 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa0580cfb20 con 0x7fa070101de0 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa074d0f640 1 -- 172.21.15.1:0/2549291831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323d9680 msgr2=0x55cd323dba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa074d0f640 1 --2- 172.21.15.1:0/2549291831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323d9680 0x55cd323dba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa074d0f640 1 -- 172.21.15.1:0/2549291831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 msgr2=0x55cd323ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa074d0f640 1 --2- 172.21.15.1:0/2549291831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 0x55cd323ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa074d0f640 1 -- 172.21.15.1:0/2549291831 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd322c0440 con 0x55cd323cedc0 2024-09-16T00:43:12.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa074d0f640 1 --2- 172.21.15.1:0/2549291831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323cedc0 0x55cd323d9140 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fa058005fd0 tx=0x7fa058000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddfc9fbe70d199c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa075510640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa05c0a3240 0x7fa05c0a5700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa068032fd0 tx=0x7fa068031460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.044+0000 7fa066ffd640 1 -- 172.21.15.1:0/2549291831 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05801d070 con 0x55cd323cedc0 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa066ffd640 1 -- 172.21.15.1:0/2549291831 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05809c3e0 con 0x55cd323cedc0 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa066ffd640 1 -- 172.21.15.1:0/2549291831 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0580cd2d0 con 0x55cd323cedc0 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa07934c740 1 -- 172.21.15.1:0/2549291831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323cedc0 msgr2=0x55cd323d9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa07934c740 1 --2- 172.21.15.1:0/2549291831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323cedc0 0x55cd323d9140 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fa058005fd0 tx=0x7fa058000bc0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa07934c740 1 -- 172.21.15.1:0/2549291831 shutdown_connections 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa07934c740 1 --2- 172.21.15.1:0/2549291831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323d9680 0x55cd323dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa07934c740 1 --2- 172.21.15.1:0/2549291831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323cedc0 0x55cd323d9140 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa07934c740 1 --2- 172.21.15.1:0/2549291831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 0x55cd323ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa07934c740 1 -- 172.21.15.1:0/2549291831 >> 172.21.15.1:0/2549291831 conn(0x55cd323caa30 msgr2=0x55cd323cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa07934c740 1 -- 172.21.15.1:0/2549291831 shutdown_connections 2024-09-16T00:43:12.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.045+0000 7fa07934c740 1 -- 172.21.15.1:0/2549291831 wait complete. 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa07934c740 1 -- start start 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa07934c740 1 Processor -- start 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa07934c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323d8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa07934c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323cedc0 0x55cd323d6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa07934c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323d9680 0x55cd323d6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa07934c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd323dc250 con 0x55cd323ce4a0 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa07934c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd322ef050 con 0x55cd323cedc0 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa07934c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd323dc3d0 con 0x55cd323d9680 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa07934c740 10 client.0 _ll_register_callbacks cb 0x55cd3232f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa074d0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323d8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa074d0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323d8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44174/0 (socket says 172.21.15.1:44174) 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.046+0000 7fa074d0f640 1 -- 172.21.15.1:0/3861796789 learned_addr learned my addr 172.21.15.1:0/3861796789 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa06ffff640 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323cedc0 0x55cd323d6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa074d0f640 1 -- 172.21.15.1:0/3861796789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323d9680 msgr2=0x55cd323d6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa074d0f640 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323d9680 0x55cd323d6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa074d0f640 1 -- 172.21.15.1:0/3861796789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323cedc0 msgr2=0x55cd323d6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa074d0f640 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323cedc0 0x55cd323d6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa074d0f640 1 -- 172.21.15.1:0/3861796789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd320a53c0 con 0x55cd323ce4a0 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa06ffff640 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323cedc0 0x55cd323d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa074d0f640 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323d8fa0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fa0580986b0 tx=0x7fa0580046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e518411fa27d124 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa04ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa04ffff640 1 -- 172.21.15.1:0/3861796789 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05801d040 con 0x55cd323ce4a0 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa04ffff640 1 -- 172.21.15.1:0/3861796789 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa0580ba5f0 con 0x55cd323ce4a0 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa07934c740 1 -- 172.21.15.1:0/3861796789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05809c3e0 con 0x55cd323ce4a0 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa04ffff640 1 -- 172.21.15.1:0/3861796789 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa05809bce0 con 0x55cd323ce4a0 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa07934c740 10 client.18939 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa07934c740 1 -- 172.21.15.1:0/3861796789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd323dc250 con 0x55cd323ce4a0 2024-09-16T00:43:12.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa04ffff640 1 -- 172.21.15.1:0/3861796789 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05809c3e0 con 0x55cd323ce4a0 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.047+0000 7fa04ffff640 1 -- 172.21.15.1:0/3861796789 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa0580a44c0 con 0x55cd323ce4a0 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.048+0000 7fa07934c740 10 client.18939 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.048+0000 7fa07934c740 10 client.18939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.048+0000 7fa07934c740 20 client.18939 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.048+0000 7fa07934c740 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd323e8d80 0x55cd323eb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.048+0000 7fa07934c740 4 client.18939 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.048+0000 7fa07934c740 1 -- 172.21.15.1:0/3861796789 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd32341850 con 0x55cd323e8d80 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.048+0000 7fa06ffff640 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd323e8d80 0x55cd323eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.048+0000 7fa04ffff640 10 client.18939 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.048+0000 7fa06ffff640 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd323e8d80 0x55cd323eb170 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b22533f4886a0e0 server_cookie=5a2734cb1f5ac866 in_seq=0 out_seq=0 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa04ffff640 1 -- 172.21.15.1:0/3861796789 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd32341850 con 0x55cd323e8d80 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa04ffff640 10 client.18939 handle_command_reply: tid=0 2024-09-16T00:43:12.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 client.18939 shutdown 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 -- 172.21.15.1:0/3861796789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 msgr2=0x55cd323d8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323d8fa0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fa0580986b0 tx=0x7fa0580046e0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 -- 172.21.15.1:0/3861796789 shutdown_connections 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd323e8d80 0x55cd323eb170 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323d9680 0x55cd323d6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323cedc0 0x55cd323d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa04ffff640 0 client.18939 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 --2- 172.21.15.1:0/3861796789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323d8fa0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 -- 172.21.15.1:0/3861796789 >> 172.21.15.1:0/3861796789 conn(0x55cd323caa30 msgr2=0x55cd323d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 -- 172.21.15.1:0/3861796789 shutdown_connections 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.049+0000 7fa07934c740 1 -- 172.21.15.1:0/3861796789 wait complete. 2024-09-16T00:43:12.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.050+0000 7fa07934c740 20 client.18939 trim_cache size 0 max 16384 2024-09-16T00:43:12.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa07934c740 1 Processor -- start 2024-09-16T00:43:12.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa07934c740 1 -- start start 2024-09-16T00:43:12.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa07934c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323e54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa07934c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa07934c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323d9680 0x55cd323f0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa07934c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701af5e0 con 0x55cd323ce4a0 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa07934c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701aceb0 con 0x55cd323d9680 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa07934c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701aad00 con 0x55cd323cedc0 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa074d0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323e54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa074d0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323e54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44190/0 (socket says 172.21.15.1:44190) 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa074d0f640 1 -- 172.21.15.1:0/3682617689 learned_addr learned my addr 172.21.15.1:0/3682617689 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa075510640 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323d9680 0x55cd323f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa06ffff640 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa074d0f640 1 -- 172.21.15.1:0/3682617689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 msgr2=0x55cd323d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa074d0f640 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323d5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa074d0f640 1 -- 172.21.15.1:0/3682617689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323d9680 msgr2=0x55cd323f0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa074d0f640 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323d9680 0x55cd323f0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa074d0f640 1 -- 172.21.15.1:0/3682617689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd323ca860 con 0x55cd323ce4a0 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa06ffff640 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:12.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.067+0000 7fa074d0f640 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323e54e0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fa05809bea0 tx=0x7fa05809a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3b5dbede25ad20 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa0667fc640 1 -- 172.21.15.1:0/3682617689 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05801d070 con 0x55cd323ce4a0 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa0667fc640 1 -- 172.21.15.1:0/3682617689 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0580a0090 con 0x55cd323ce4a0 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa0667fc640 1 -- 172.21.15.1:0/3682617689 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05809b410 con 0x55cd323ce4a0 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa07934c740 1 -- 172.21.15.1:0/3682617689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 msgr2=0x55cd323e54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa07934c740 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323e54e0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fa05809bea0 tx=0x7fa05809a1f0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa07934c740 1 -- 172.21.15.1:0/3682617689 shutdown_connections 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa07934c740 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323d9680 0x55cd323f0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa07934c740 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa07934c740 1 --2- 172.21.15.1:0/3682617689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323ce4a0 0x55cd323e54e0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa07934c740 1 -- 172.21.15.1:0/3682617689 >> 172.21.15.1:0/3682617689 conn(0x55cd323caa30 msgr2=0x55cd323d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa07934c740 1 -- 172.21.15.1:0/3682617689 shutdown_connections 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.068+0000 7fa07934c740 1 -- 172.21.15.1:0/3682617689 wait complete. 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa07934c740 1 -- start start 2024-09-16T00:43:12.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa07934c740 1 Processor -- start 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa07934c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 0x55cd323e1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa07934c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323dd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa07934c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323d9680 0x55cd323dd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa07934c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701ab370 con 0x55cd323d9680 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa07934c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701ac7e0 con 0x55cd323ce4a0 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa07934c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701aba40 con 0x55cd323cedc0 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa07934c740 10 client.0 _ll_register_callbacks cb 0x7fa068009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa075510640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323d9680 0x55cd323dd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa075510640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323d9680 0x55cd323dd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44198/0 (socket says 172.21.15.1:44198) 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa075510640 1 -- 172.21.15.1:0/33103993 learned_addr learned my addr 172.21.15.1:0/33103993 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.069+0000 7fa06ffff640 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323dd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa074d0f640 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 0x55cd323e1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa075510640 1 -- 172.21.15.1:0/33103993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 msgr2=0x55cd323dd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa075510640 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323dd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa075510640 1 -- 172.21.15.1:0/33103993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 msgr2=0x55cd323e1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa075510640 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 0x55cd323e1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa075510640 1 -- 172.21.15.1:0/33103993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd323ddf70 con 0x55cd323d9680 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa06ffff640 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323dd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa075510640 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323d9680 0x55cd323dd740 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa068009900 tx=0x7fa0680056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cafb81235fe1285 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa04ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa04ffff640 1 -- 172.21.15.1:0/33103993 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068035040 con 0x55cd323d9680 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa07934c740 1 -- 172.21.15.1:0/33103993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0580a0090 con 0x55cd323d9680 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa04ffff640 1 -- 172.21.15.1:0/33103993 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa06803baf0 con 0x55cd323d9680 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa04ffff640 1 -- 172.21.15.1:0/33103993 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa06805f050 con 0x55cd323d9680 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa07934c740 10 client.18945 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa07934c740 1 -- 172.21.15.1:0/33103993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd32453970 con 0x55cd323d9680 2024-09-16T00:43:12.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.070+0000 7fa04ffff640 1 -- 172.21.15.1:0/33103993 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0580a0090 con 0x55cd323d9680 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa04ffff640 1 -- 172.21.15.1:0/33103993 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa06804c450 con 0x55cd323d9680 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa07934c740 10 client.18945 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa07934c740 10 client.18945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa07934c740 20 client.18945 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa07934c740 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd32453ff0 0x55cd324563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa07934c740 4 client.18945 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa07934c740 1 -- 172.21.15.1:0/33103993 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd3233bd60 con 0x55cd32453ff0 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa074d0f640 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd32453ff0 0x55cd324563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa074d0f640 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd32453ff0 0x55cd324563e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae358111bc9cd63 server_cookie=15bb1996862dc0f2 in_seq=0 out_seq=0 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa04ffff640 10 client.18945 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa04ffff640 1 -- 172.21.15.1:0/33103993 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cd3233bd60 con 0x55cd32453ff0 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa04ffff640 10 client.18945 handle_command_reply: tid=0 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.071+0000 7fa07934c740 1 client.18945 shutdown 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 -- 172.21.15.1:0/33103993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323d9680 msgr2=0x55cd323dd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:12.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323d9680 0x55cd323dd740 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa068009900 tx=0x7fa0680056d0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 -- 172.21.15.1:0/33103993 shutdown_connections 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55cd32453ff0 0x55cd324563e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd323d9680 0x55cd323dd740 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd323cedc0 0x55cd323dd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 --2- 172.21.15.1:0/33103993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd323ce4a0 0x55cd323e1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 -- 172.21.15.1:0/33103993 >> 172.21.15.1:0/33103993 conn(0x55cd323caa30 msgr2=0x55cd323c9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa04ffff640 0 client.18945 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 -- 172.21.15.1:0/33103993 shutdown_connections 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 1 -- 172.21.15.1:0/33103993 wait complete. 2024-09-16T00:43:12.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa07934c740 20 client.18945 trim_cache size 0 max 16384 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: "42384e8b-e098-4237-affc-12e416df2151": { 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "42384e8b-e098-4237-affc-12e416df2151", 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:12.085 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:43:12.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa076f73640 1 -- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa05c0a3240 msgr2=0x7fa05c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:12.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa076f73640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa05c0a3240 0x7fa05c0a5700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa068032fd0 tx=0x7fa068031460 comp rx=0 tx=0).stop 2024-09-16T00:43:12.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa076f73640 1 -- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070101de0 msgr2=0x7fa070198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:12.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.072+0000 7fa076f73640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070101de0 0x7fa070198410 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa05800c890 tx=0x7fa05800cd50 comp rx=0 tx=0).stop 2024-09-16T00:43:12.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.073+0000 7fa076f73640 1 -- 172.21.15.1:0/3748108813 shutdown_connections 2024-09-16T00:43:12.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.073+0000 7fa076f73640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa070106f10 0x7fa07019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.073+0000 7fa076f73640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa05c0a3240 0x7fa05c0a5700 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.073+0000 7fa076f73640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa070102790 0x7fa070198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.073+0000 7fa076f73640 1 --2- 172.21.15.1:0/3748108813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa070101de0 0x7fa070198410 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:12.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.073+0000 7fa076f73640 1 -- 172.21.15.1:0/3748108813 >> 172.21.15.1:0/3748108813 conn(0x7fa0700fd8c0 msgr2=0x7fa0700feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:12.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.073+0000 7fa076f73640 1 -- 172.21.15.1:0/3748108813 shutdown_connections 2024-09-16T00:43:12.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:12.073+0000 7fa076f73640 1 -- 172.21.15.1:0/3748108813 wait complete. 2024-09-16T00:43:12.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:11 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:43:12.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:11 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:12.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:11 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:43:12.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:11 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:12.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:11 smithi019 ceph-mon[35186]: pgmap v794: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:12.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:11 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:43:12.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:11 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:12.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:11 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:43:12.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:11 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:12.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:11 smithi153 ceph-mon[35538]: pgmap v794: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:12.849 INFO:tasks.cephfs.filesystem:scrub status for tag:42384e8b-e098-4237-affc-12e416df2151 - {'path': '/', 'tag': '42384e8b-e098-4237-affc-12e416df2151', 'options': 'recursive,force'} 2024-09-16T00:43:14.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:13 smithi019 ceph-mon[35186]: pgmap v795: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:14.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:13 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:43:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:13 smithi001 ceph-mon[30934]: pgmap v795: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:13 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:43:14.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:13 smithi153 ceph-mon[35538]: pgmap v795: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:14.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:13 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:43:16.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:15 smithi019 ceph-mon[35186]: pgmap v796: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:16.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:15 smithi001 ceph-mon[30934]: pgmap v796: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:16.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:15 smithi153 ceph-mon[35538]: pgmap v796: 129 pgs: 129 active+clean; 6.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:18.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:17 smithi019 ceph-mon[35186]: pgmap v797: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:43:18.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:17 smithi001 ceph-mon[30934]: pgmap v797: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:43:18.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:17 smithi153 ceph-mon[35538]: pgmap v797: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:43:20.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:19 smithi019 ceph-mon[35186]: pgmap v798: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:20.204 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:19 smithi001 ceph-mon[30934]: pgmap v798: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:20.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:19 smithi153 ceph-mon[35538]: pgmap v798: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:43:22.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:21 smithi019 ceph-mon[35186]: pgmap v799: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:22.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:21 smithi001 ceph-mon[30934]: pgmap v799: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:22.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:21 smithi153 ceph-mon[35538]: pgmap v799: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:24.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:23 smithi019 ceph-mon[35186]: pgmap v800: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:43:24.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:23 smithi001 ceph-mon[30934]: pgmap v800: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:43:24.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:23 smithi153 ceph-mon[35538]: pgmap v800: 129 pgs: 129 active+clean; 6.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:43:26.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:25 smithi019 ceph-mon[35186]: pgmap v801: 129 pgs: 129 active+clean; 6.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:25 smithi001 ceph-mon[30934]: pgmap v801: 129 pgs: 129 active+clean; 6.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:26.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:25 smithi153 ceph-mon[35538]: pgmap v801: 129 pgs: 129 active+clean; 6.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:28.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:27 smithi019 ceph-mon[35186]: pgmap v802: 129 pgs: 129 active+clean; 7.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:28.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:27 smithi001 ceph-mon[30934]: pgmap v802: 129 pgs: 129 active+clean; 7.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:28.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:27 smithi153 ceph-mon[35538]: pgmap v802: 129 pgs: 129 active+clean; 7.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:30.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:29 smithi019 ceph-mon[35186]: pgmap v803: 129 pgs: 129 active+clean; 7.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:30.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:29 smithi001 ceph-mon[30934]: pgmap v803: 129 pgs: 129 active+clean; 7.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:30.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:29 smithi153 ceph-mon[35538]: pgmap v803: 129 pgs: 129 active+clean; 7.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:31.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:30 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:43:31.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:30 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:43:31.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:30 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:43:32.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:31 smithi019 ceph-mon[35186]: pgmap v804: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:32.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:43:32.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:43:32.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:43:32.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:31 smithi001 ceph-mon[30934]: pgmap v804: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:32.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:43:32.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:43:32.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:43:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:31 smithi153 ceph-mon[35538]: pgmap v804: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:43:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:43:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:43:34.109 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:33 smithi019 ceph-mon[35186]: pgmap v805: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:34.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:33 smithi001 ceph-mon[30934]: pgmap v805: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:34.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:33 smithi153 ceph-mon[35538]: pgmap v805: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:35.636 INFO:tasks.workunit.client.0.smithi001.stdout:499/600: open fd = 3 2024-09-16T00:43:35.636 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.636 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.636 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.636 INFO:tasks.workunit.client.0.smithi001.stdout:500/600: open fd = 3 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:501/600: open fd = 3 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:502/600: open fd = 3 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:503/600: open fd = 3 2024-09-16T00:43:35.637 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:504/600: open fd = 3 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:505/600: open fd = 3 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:506/600: open fd = 3 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.638 INFO:tasks.workunit.client.0.smithi001.stdout:507/600: open fd = 3 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:508/600: open fd = 3 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:509/600: open fd = 3 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:510/600: open fd = 3 2024-09-16T00:43:35.639 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:511/600: open fd = 3 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:512/600: open fd = 3 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:513/600: open fd = 3 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.640 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:514/600: open fd = 3 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:515/600: open fd = 3 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.641 INFO:tasks.workunit.client.0.smithi001.stdout:516/600: open fd = 3 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:517/600: open fd = 3 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:518/600: open fd = 3 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:519/600: open fd = 3 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.642 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:520/600: open fd = 3 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:521/600: open fd = 3 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:522/600: open fd = 3 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.643 INFO:tasks.workunit.client.0.smithi001.stdout:523/600: open fd = 3 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:524/600: open fd = 3 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:525/600: open fd = 3 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.644 INFO:tasks.workunit.client.0.smithi001.stdout:526/600: open fd = 3 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:527/600: open fd = 3 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:528/600: open fd = 3 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.645 INFO:tasks.workunit.client.0.smithi001.stdout:529/600: open fd = 3 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:530/600: open fd = 3 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:531/600: open fd = 3 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:532/600: open fd = 3 2024-09-16T00:43:35.646 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:533/600: open fd = 3 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:534/600: open fd = 3 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:535/600: open fd = 3 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.647 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.648 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.648 INFO:tasks.workunit.client.0.smithi001.stdout:536/600: open fd = 3 2024-09-16T00:43:35.648 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.648 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:537/600: open fd = 3 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:538/600: open fd = 3 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.649 INFO:tasks.workunit.client.0.smithi001.stdout:539/600: open fd = 3 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:540/600: open fd = 3 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:541/600: open fd = 3 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.650 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:542/600: open fd = 3 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:543/600: open fd = 3 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:544/600: open fd = 3 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.651 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:545/600: open fd = 3 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:546/600: open fd = 3 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:547/600: open fd = 3 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.652 INFO:tasks.workunit.client.0.smithi001.stdout:548/600: open fd = 3 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:549/600: open fd = 3 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:550/600: open fd = 3 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:551/600: open fd = 3 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.653 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:552/600: open fd = 3 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:553/600: open fd = 3 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:554/600: open fd = 3 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.654 INFO:tasks.workunit.client.0.smithi001.stdout:555/600: open fd = 3 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:556/600: open fd = 3 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:557/600: open fd = 3 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:558/600: open fd = 3 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.655 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.656 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.656 INFO:tasks.workunit.client.0.smithi001.stdout:559/600: open fd = 3 2024-09-16T00:43:35.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.656 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:35.656 INFO:tasks.workunit.client.0.smithi001.stdout:560/600: open fd = 3 2024-09-16T00:43:35.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.656 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:43:35.656 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:43:36.106 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:35 smithi019 ceph-mon[35186]: pgmap v806: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:36.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:35 smithi001 ceph-mon[30934]: pgmap v806: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:36.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:35 smithi153 ceph-mon[35538]: pgmap v806: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:38.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:37 smithi019 ceph-mon[35186]: pgmap v807: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:43:38.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:37 smithi001 ceph-mon[30934]: pgmap v807: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:43:38.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:37 smithi153 ceph-mon[35538]: pgmap v807: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-16T00:43:40.111 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:39 smithi019 ceph-mon[35186]: pgmap v808: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:40.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:39 smithi001 ceph-mon[30934]: pgmap v808: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:40.213 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:39 smithi153 ceph-mon[35538]: pgmap v808: 129 pgs: 129 active+clean; 7.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:42.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:41 smithi019 ceph-mon[35186]: pgmap v809: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:42.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:41 smithi001 ceph-mon[30934]: pgmap v809: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:42.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:41 smithi153 ceph-mon[35538]: pgmap v809: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:42.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:43:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:43.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.687+0000 7f99497f9640 1 -- 172.21.15.1:0/4002849141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994410c1a0 msgr2=0x7f994410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:43.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.687+0000 7f99497f9640 1 --2- 172.21.15.1:0/4002849141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994410c1a0 0x7f994410e610 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f993c009ca0 tx=0x7f993c02a950 comp rx=0 tx=0).stop 2024-09-16T00:43:43.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.687+0000 7f99497f9640 1 -- 172.21.15.1:0/4002849141 shutdown_connections 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.687+0000 7f99497f9640 1 --2- 172.21.15.1:0/4002849141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994410c1a0 0x7f994410e610 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.687+0000 7f99497f9640 1 --2- 172.21.15.1:0/4002849141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9944103720 0x7f994410bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.687+0000 7f99497f9640 1 --2- 172.21.15.1:0/4002849141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9944102e00 0x7f99441031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.687+0000 7f99497f9640 1 -- 172.21.15.1:0/4002849141 >> 172.21.15.1:0/4002849141 conn(0x7f99440fc7a0 msgr2=0x7f99440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.687+0000 7f99497f9640 1 -- 172.21.15.1:0/4002849141 shutdown_connections 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.687+0000 7f99497f9640 1 -- 172.21.15.1:0/4002849141 wait complete. 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f99497f9640 1 Processor -- start 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f99497f9640 1 -- start start 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f99497f9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9944102e00 0x7f9944198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f99497f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9944103720 0x7f9944198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f99497f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994410c1a0 0x7f994419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f99497f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9944111f70 con 0x7f9944103720 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f99497f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9944111df0 con 0x7f994410c1a0 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f99497f9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99441120f0 con 0x7f9944102e00 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f993bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9944103720 0x7f9944198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f993bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9944103720 0x7f9944198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35184/0 (socket says 172.21.15.1:35184) 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f993bfff640 1 -- 172.21.15.1:0/64301935 learned_addr learned my addr 172.21.15.1:0/64301935 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.688+0000 7f99437fe640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994410c1a0 0x7f994419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f9942ffd640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9944102e00 0x7f9944198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f993bfff640 1 -- 172.21.15.1:0/64301935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9944102e00 msgr2=0x7f9944198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f993bfff640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9944102e00 0x7f9944198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f993bfff640 1 -- 172.21.15.1:0/64301935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994410c1a0 msgr2=0x7f994419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f993bfff640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994410c1a0 0x7f994419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f993bfff640 1 -- 172.21.15.1:0/64301935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994419d300 con 0x7f9944103720 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f993bfff640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9944103720 0x7f9944198890 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c920 tx=0x7f992c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f9940ff9640 1 -- 172.21.15.1:0/64301935 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c007bb0 con 0x7f9944103720 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f99497f9640 1 -- 172.21.15.1:0/64301935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994419d5f0 con 0x7f9944103720 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f9940ff9640 1 -- 172.21.15.1:0/64301935 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992c007d50 con 0x7f9944103720 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f9940ff9640 1 -- 172.21.15.1:0/64301935 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c018780 con 0x7f9944103720 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.689+0000 7f99497f9640 1 -- 172.21.15.1:0/64301935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99441a4ed0 con 0x7f9944103720 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.692+0000 7f994bbd2740 1 Processor -- start 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.692+0000 7f9940ff9640 1 -- 172.21.15.1:0/64301935 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f992c00f450 con 0x7f9944103720 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.692+0000 7f994bbd2740 1 -- start start 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.692+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b384760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.692+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b393550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.692+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 0x563a3b395e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.692+0000 7f994bbd2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563a3b2ea2e0 con 0x563a3b393a90 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.692+0000 7f994bbd2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a3b291ce0 con 0x563a3b384380 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.692+0000 7f994bbd2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563a3b2e62a0 con 0x563a3b384ca0 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f993bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 0x563a3b395e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f99437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b384760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f9942ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b393550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f99437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b384760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:32794/0 (socket says 172.21.15.1:32794) 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f99437fe640 1 -- 172.21.15.1:0/1770973664 learned_addr learned my addr 172.21.15.1:0/1770973664 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f993bfff640 1 -- 172.21.15.1:0/1770973664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 msgr2=0x563a3b393550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f993bfff640 1 --2- 172.21.15.1:0/1770973664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b393550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f9940ff9640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99300a3240 0x7f99300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f993bfff640 1 -- 172.21.15.1:0/1770973664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 msgr2=0x563a3b384760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f993bfff640 1 --2- 172.21.15.1:0/1770973664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b384760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f993bfff640 1 -- 172.21.15.1:0/1770973664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a3b276440 con 0x563a3b393a90 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f99437fe640 1 --2- 172.21.15.1:0/1770973664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b384760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f99437fe640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99300a3240 0x7f99300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f9940ff9640 1 -- 172.21.15.1:0/64301935 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f992c0cfd30 con 0x7f9944103720 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f993bfff640 1 --2- 172.21.15.1:0/1770973664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 0x563a3b395e80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f992c0cd400 tx=0x7f992c098020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6437c13f803860d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.693+0000 7f99397fa640 1 -- 172.21.15.1:0/1770973664 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c09bd20 con 0x563a3b393a90 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f99397fa640 1 -- 172.21.15.1:0/1770973664 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992c0d3e50 con 0x563a3b393a90 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f99397fa640 1 -- 172.21.15.1:0/1770973664 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c09b8b0 con 0x563a3b393a90 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f99437fe640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99300a3240 0x7f99300a5700 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f993c005fe0 tx=0x7f993c03c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f994bbd2740 1 -- 172.21.15.1:0/1770973664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 msgr2=0x563a3b395e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:43.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f994bbd2740 1 --2- 172.21.15.1:0/1770973664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 0x563a3b395e80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f992c0cd400 tx=0x7f992c098020 comp rx=0 tx=0).stop 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f994bbd2740 1 -- 172.21.15.1:0/1770973664 shutdown_connections 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f994bbd2740 1 --2- 172.21.15.1:0/1770973664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 0x563a3b395e80 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f994bbd2740 1 --2- 172.21.15.1:0/1770973664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b393550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f994bbd2740 1 --2- 172.21.15.1:0/1770973664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b384760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f994bbd2740 1 -- 172.21.15.1:0/1770973664 >> 172.21.15.1:0/1770973664 conn(0x563a3b380a30 msgr2=0x563a3b380e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f994bbd2740 1 -- 172.21.15.1:0/1770973664 shutdown_connections 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.694+0000 7f994bbd2740 1 -- 172.21.15.1:0/1770973664 wait complete. 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f994bbd2740 1 -- start start 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f994bbd2740 1 Processor -- start 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b393420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384ca0 0x563a3b3906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b393a90 0x563a3b390c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f994bbd2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563a3b396600 con 0x563a3b384380 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f994bbd2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a3b2a5050 con 0x563a3b384ca0 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f994bbd2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563a3b396780 con 0x563a3b393a90 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f994bbd2740 10 client.0 _ll_register_callbacks cb 0x563a3b2e5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f9942ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b393420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f9942ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b393420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35202/0 (socket says 172.21.15.1:35202) 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f9942ffd640 1 -- 172.21.15.1:0/3546695282 learned_addr learned my addr 172.21.15.1:0/3546695282 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f993bfff640 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384ca0 0x563a3b3906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.695+0000 7f99437fe640 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b393a90 0x563a3b390c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f9942ffd640 1 -- 172.21.15.1:0/3546695282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b393a90 msgr2=0x563a3b390c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f9942ffd640 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b393a90 0x563a3b390c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f9942ffd640 1 -- 172.21.15.1:0/3546695282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384ca0 msgr2=0x563a3b3906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f9942ffd640 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384ca0 0x563a3b3906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f9942ffd640 1 -- 172.21.15.1:0/3546695282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3b05b3c0 con 0x563a3b384380 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f993bfff640 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384ca0 0x563a3b3906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f9942ffd640 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b393420 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9934010a50 tx=0x7f9934010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce721da0125b9af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f992a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f992a7fc640 1 -- 172.21.15.1:0/3546695282 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993401e060 con 0x563a3b384380 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f992a7fc640 1 -- 172.21.15.1:0/3546695282 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f993400fbd0 con 0x563a3b384380 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f994bbd2740 1 -- 172.21.15.1:0/3546695282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f992c0d3e50 con 0x563a3b384380 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f992a7fc640 1 -- 172.21.15.1:0/3546695282 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9934025050 con 0x563a3b384380 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f994bbd2740 10 client.18957 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f994bbd2740 1 -- 172.21.15.1:0/3546695282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a3b3a2090 con 0x563a3b384380 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.696+0000 7f992a7fc640 1 -- 172.21.15.1:0/3546695282 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992c0d3e50 con 0x563a3b384380 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f992a7fc640 1 -- 172.21.15.1:0/3546695282 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9934018370 con 0x563a3b384380 2024-09-16T00:43:43.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f994bbd2740 10 client.18957 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f994bbd2740 10 client.18957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f994bbd2740 20 client.18957 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f994bbd2740 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563a3b3a3550 0x563a3b3a5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f994bbd2740 4 client.18957 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f994bbd2740 1 -- 172.21.15.1:0/3546695282 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a3b2f7850 con 0x563a3b3a3550 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f993bfff640 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563a3b3a3550 0x563a3b3a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f993bfff640 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563a3b3a3550 0x563a3b3a5980 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5a100efc4417f6 server_cookie=dc26a7006aa4f9f4 in_seq=0 out_seq=0 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.697+0000 7f992a7fc640 10 client.18957 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f992a7fc640 1 -- 172.21.15.1:0/3546695282 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a3b2f7850 con 0x563a3b3a3550 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f992a7fc640 10 client.18957 handle_command_reply: tid=0 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 client.18957 shutdown 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 -- 172.21.15.1:0/3546695282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 msgr2=0x563a3b393420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b393420 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9934010a50 tx=0x7f9934010f10 comp rx=0 tx=0).stop 2024-09-16T00:43:43.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 -- 172.21.15.1:0/3546695282 shutdown_connections 2024-09-16T00:43:43.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563a3b3a3550 0x563a3b3a5980 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b393a90 0x563a3b390c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384ca0 0x563a3b3906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f992a7fc640 0 client.18957 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:43.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 --2- 172.21.15.1:0/3546695282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b393420 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 -- 172.21.15.1:0/3546695282 >> 172.21.15.1:0/3546695282 conn(0x563a3b380a30 msgr2=0x563a3b386790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:43.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 -- 172.21.15.1:0/3546695282 shutdown_connections 2024-09-16T00:43:43.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 1 -- 172.21.15.1:0/3546695282 wait complete. 2024-09-16T00:43:43.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.699+0000 7f994bbd2740 20 client.18957 trim_cache size 0 max 16384 2024-09-16T00:43:43.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f994bbd2740 1 Processor -- start 2024-09-16T00:43:43.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f994bbd2740 1 -- start start 2024-09-16T00:43:43.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b39f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b38efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b393a90 0x563a3b3ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f994bbd2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99441af410 con 0x563a3b384380 2024-09-16T00:43:43.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f994bbd2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99441acce0 con 0x563a3b393a90 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f994bbd2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99441aab30 con 0x563a3b384ca0 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f9942ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b39f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f9942ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b39f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35208/0 (socket says 172.21.15.1:35208) 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f9942ffd640 1 -- 172.21.15.1:0/1738685797 learned_addr learned my addr 172.21.15.1:0/1738685797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f99437fe640 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b393a90 0x563a3b3ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f9942ffd640 1 -- 172.21.15.1:0/1738685797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 msgr2=0x563a3b38efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.717+0000 7f993bfff640 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b38efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.718+0000 7f9942ffd640 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b38efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.718+0000 7f9942ffd640 1 -- 172.21.15.1:0/1738685797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b393a90 msgr2=0x563a3b3ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.718+0000 7f9942ffd640 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b393a90 0x563a3b3ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.718+0000 7f9942ffd640 1 -- 172.21.15.1:0/1738685797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a3b38f680 con 0x563a3b384380 2024-09-16T00:43:43.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.718+0000 7f99437fe640 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b393a90 0x563a3b3ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.718+0000 7f9942ffd640 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b39f8a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9934010a20 tx=0x7f993400a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c381e4a9fb92702c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.718+0000 7f9938ff9640 1 -- 172.21.15.1:0/1738685797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993402e070 con 0x563a3b384380 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.718+0000 7f9938ff9640 1 -- 172.21.15.1:0/1738685797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9934005510 con 0x563a3b384380 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.718+0000 7f9938ff9640 1 -- 172.21.15.1:0/1738685797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934004660 con 0x563a3b384380 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 -- 172.21.15.1:0/1738685797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 msgr2=0x563a3b39f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b39f8a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9934010a20 tx=0x7f993400a930 comp rx=0 tx=0).stop 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 -- 172.21.15.1:0/1738685797 shutdown_connections 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b393a90 0x563a3b3ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b38efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 --2- 172.21.15.1:0/1738685797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b384380 0x563a3b39f8a0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 -- 172.21.15.1:0/1738685797 >> 172.21.15.1:0/1738685797 conn(0x563a3b380a30 msgr2=0x563a3b38bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 -- 172.21.15.1:0/1738685797 shutdown_connections 2024-09-16T00:43:43.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 -- 172.21.15.1:0/1738685797 wait complete. 2024-09-16T00:43:43.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 -- start start 2024-09-16T00:43:43.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 Processor -- start 2024-09-16T00:43:43.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b39c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b397440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 0x563a3b397980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99441ab1a0 con 0x563a3b393a90 2024-09-16T00:43:43.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99441ac610 con 0x563a3b384380 2024-09-16T00:43:43.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.719+0000 7f994bbd2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99441ab870 con 0x563a3b384ca0 2024-09-16T00:43:43.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f994bbd2740 10 client.0 _ll_register_callbacks cb 0x7f993400a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:43.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f9942ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b39c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f9942ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b39c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:32820/0 (socket says 172.21.15.1:32820) 2024-09-16T00:43:43.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f9942ffd640 1 -- 172.21.15.1:0/2570890009 learned_addr learned my addr 172.21.15.1:0/2570890009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:43.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f99437fe640 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 0x563a3b397980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f993bfff640 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b397440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f9942ffd640 1 -- 172.21.15.1:0/2570890009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 msgr2=0x563a3b397440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:43.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f9942ffd640 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b397440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f9942ffd640 1 -- 172.21.15.1:0/2570890009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 msgr2=0x563a3b397980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f9942ffd640 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 0x563a3b397980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f9942ffd640 1 -- 172.21.15.1:0/2570890009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3b390fc0 con 0x563a3b384380 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f993bfff640 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b397440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f9942ffd640 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b39c110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f993402c040 tx=0x7f993400f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52090ffa59cef44 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f992a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f992a7fc640 1 -- 172.21.15.1:0/2570890009 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993402e070 con 0x563a3b384380 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f992a7fc640 1 -- 172.21.15.1:0/2570890009 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f993400fcb0 con 0x563a3b384380 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.720+0000 7f994bbd2740 1 -- 172.21.15.1:0/2570890009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9934005510 con 0x563a3b384380 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f992a7fc640 1 -- 172.21.15.1:0/2570890009 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f993401e040 con 0x563a3b384380 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f994bbd2740 10 client.27788 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f994bbd2740 1 -- 172.21.15.1:0/2570890009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a3b40df90 con 0x563a3b384380 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f992a7fc640 1 -- 172.21.15.1:0/2570890009 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9934005510 con 0x563a3b384380 2024-09-16T00:43:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f992a7fc640 1 -- 172.21.15.1:0/2570890009 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9934032400 con 0x563a3b384380 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f994bbd2740 10 client.27788 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f994bbd2740 10 client.27788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f994bbd2740 20 client.27788 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f994bbd2740 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563a3b40e5f0 0x563a3b4109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f994bbd2740 4 client.27788 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f994bbd2740 1 -- 172.21.15.1:0/2570890009 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a3b2f1d60 con 0x563a3b40e5f0 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.721+0000 7f993bfff640 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563a3b40e5f0 0x563a3b4109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f993bfff640 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563a3b40e5f0 0x563a3b4109e0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe67a0bc9162d17 server_cookie=7042ab7791057270 in_seq=0 out_seq=0 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f992a7fc640 10 client.27788 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f992a7fc640 1 -- 172.21.15.1:0/2570890009 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563a3b2f1d60 con 0x563a3b40e5f0 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f992a7fc640 10 client.27788 handle_command_reply: tid=0 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 client.27788 shutdown 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 -- 172.21.15.1:0/2570890009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 msgr2=0x563a3b39c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b39c110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f993402c040 tx=0x7f993400f830 comp rx=0 tx=0).stop 2024-09-16T00:43:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 -- 172.21.15.1:0/2570890009 shutdown_connections 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x563a3b40e5f0 0x563a3b4109e0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3b393a90 0x563a3b397980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563a3b384ca0 0x563a3b397440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f992a7fc640 0 client.27788 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 --2- 172.21.15.1:0/2570890009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a3b384380 0x563a3b39c110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 -- 172.21.15.1:0/2570890009 >> 172.21.15.1:0/2570890009 conn(0x563a3b380a30 msgr2=0x563a3b38bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 -- 172.21.15.1:0/2570890009 shutdown_connections 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 1 -- 172.21.15.1:0/2570890009 wait complete. 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.722+0000 7f994bbd2740 20 client.27788 trim_cache size 0 max 16384 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:43:43.731 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:43:43.732 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:43:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 -- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99300a3240 msgr2=0x7f99300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99300a3240 0x7f99300a5700 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f993c005fe0 tx=0x7f993c03c040 comp rx=0 tx=0).stop 2024-09-16T00:43:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 -- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9944103720 msgr2=0x7f9944198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9944103720 0x7f9944198890 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c920 tx=0x7f992c00cd20 comp rx=0 tx=0).stop 2024-09-16T00:43:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 -- 172.21.15.1:0/64301935 shutdown_connections 2024-09-16T00:43:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994410c1a0 0x7f994419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9944103720 0x7f9944198890 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f99300a3240 0x7f99300a5700 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 --2- 172.21.15.1:0/64301935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9944102e00 0x7f9944198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:43.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 -- 172.21.15.1:0/64301935 >> 172.21.15.1:0/64301935 conn(0x7f99440fc7a0 msgr2=0x7f99440fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:43.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 -- 172.21.15.1:0/64301935 shutdown_connections 2024-09-16T00:43:43.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:43.723+0000 7f99497f9640 1 -- 172.21.15.1:0/64301935 wait complete. 2024-09-16T00:43:43.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:43 smithi001 ceph-mon[30934]: pgmap v810: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:44.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:43 smithi019 ceph-mon[35186]: pgmap v810: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:44.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:43 smithi153 ceph-mon[35538]: pgmap v810: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:44.560 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:43:44.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:43:44.826 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:45.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.379+0000 7f1e8198d640 1 -- 172.21.15.1:0/279870783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c100a80 msgr2=0x7f1e7c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:45.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.379+0000 7f1e8198d640 1 --2- 172.21.15.1:0/279870783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c100a80 0x7f1e7c100ee0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009930 tx=0x7f1e6402a420 comp rx=0 tx=0).stop 2024-09-16T00:43:45.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.379+0000 7f1e8198d640 1 -- 172.21.15.1:0/279870783 shutdown_connections 2024-09-16T00:43:45.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.379+0000 7f1e8198d640 1 --2- 172.21.15.1:0/279870783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e7c101420 0x7f1e7c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:45.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.379+0000 7f1e8198d640 1 --2- 172.21.15.1:0/279870783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c100a80 0x7f1e7c100ee0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:45.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.379+0000 7f1e8198d640 1 --2- 172.21.15.1:0/279870783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e7c106a80 0x7f1e7c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:45.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.379+0000 7f1e8198d640 1 -- 172.21.15.1:0/279870783 >> 172.21.15.1:0/279870783 conn(0x7f1e7c0fc7a0 msgr2=0x7f1e7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:45.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.379+0000 7f1e8198d640 1 -- 172.21.15.1:0/279870783 shutdown_connections 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.379+0000 7f1e8198d640 1 -- 172.21.15.1:0/279870783 wait complete. 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e8198d640 1 Processor -- start 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e8198d640 1 -- start start 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e8198d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e7c100a80 0x7f1e7c0fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e8198d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c101420 0x7f1e7c0fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e8198d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e7c106a80 0x7f1e7c0fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e8198d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c110e80 con 0x7f1e7c101420 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e8198d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c110d00 con 0x7f1e7c106a80 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e8198d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c111000 con 0x7f1e7c100a80 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e7a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c101420 0x7f1e7c0fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e7a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c101420 0x7f1e7c0fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35224/0 (socket says 172.21.15.1:35224) 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.380+0000 7f1e7a7fc640 1 -- 172.21.15.1:0/3544380074 learned_addr learned my addr 172.21.15.1:0/3544380074 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e7b7fe640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e7c106a80 0x7f1e7c0fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e7affd640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e7c100a80 0x7f1e7c0fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e7a7fc640 1 -- 172.21.15.1:0/3544380074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e7c100a80 msgr2=0x7f1e7c0fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e7a7fc640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e7c100a80 0x7f1e7c0fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e7a7fc640 1 -- 172.21.15.1:0/3544380074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e7c106a80 msgr2=0x7f1e7c0fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e7a7fc640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e7c106a80 0x7f1e7c0fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e7a7fc640 1 -- 172.21.15.1:0/3544380074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7c1a6910 con 0x7f1e7c101420 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e7affd640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e7c100a80 0x7f1e7c0fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e7a7fc640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c101420 0x7f1e7c0fd320 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009230 tx=0x7f1e64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e8098b640 1 -- 172.21.15.1:0/3544380074 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6403f030 con 0x7f1e7c101420 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e8098b640 1 -- 172.21.15.1:0/3544380074 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e64003ad0 con 0x7f1e7c101420 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7c1a6ba0 con 0x7f1e7c101420 2024-09-16T00:43:45.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e8098b640 1 -- 172.21.15.1:0/3544380074 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64035070 con 0x7f1e7c101420 2024-09-16T00:43:45.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.381+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7c1a7080 con 0x7f1e7c101420 2024-09-16T00:43:45.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.382+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e7c100ee0 con 0x7f1e7c101420 2024-09-16T00:43:45.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.383+0000 7f1e8098b640 1 -- 172.21.15.1:0/3544380074 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f1e64042030 con 0x7f1e7c101420 2024-09-16T00:43:45.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.384+0000 7f1e8098b640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1e680a3240 0x7f1e680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:45.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.384+0000 7f1e7affd640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1e680a3240 0x7f1e680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:45.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.384+0000 7f1e8098b640 1 -- 172.21.15.1:0/3544380074 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f1e640f0270 con 0x7f1e7c101420 2024-09-16T00:43:45.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.385+0000 7f1e7affd640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1e680a3240 0x7f1e680a5700 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005cc0 tx=0x7f1e6c005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:45.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.388+0000 7f1e8098b640 1 -- 172.21.15.1:0/3544380074 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e640bcb30 con 0x7f1e7c101420 2024-09-16T00:43:45.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.599+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e7c10a020 con 0x7f1e7c101420 2024-09-16T00:43:45.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.600+0000 7f1e8098b640 1 -- 172.21.15.1:0/3544380074 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f1e640bc430 con 0x7f1e7c101420 2024-09-16T00:43:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:43:45.601 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:43:45.602 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:43:45.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1e680a3240 msgr2=0x7f1e680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:45.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1e680a3240 0x7f1e680a5700 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005cc0 tx=0x7f1e6c005c30 comp rx=0 tx=0).stop 2024-09-16T00:43:45.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c101420 msgr2=0x7f1e7c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:45.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c101420 0x7f1e7c0fd320 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009230 tx=0x7f1e64004270 comp rx=0 tx=0).stop 2024-09-16T00:43:45.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 shutdown_connections 2024-09-16T00:43:45.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e7c106a80 0x7f1e7c0fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:45.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e7c101420 0x7f1e7c0fd320 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:45.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f1e680a3240 0x7f1e680a5700 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:45.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 --2- 172.21.15.1:0/3544380074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e7c100a80 0x7f1e7c0fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:45.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 >> 172.21.15.1:0/3544380074 conn(0x7f1e7c0fc7a0 msgr2=0x7f1e7c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:45.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 shutdown_connections 2024-09-16T00:43:45.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:45.603+0000 7f1e8198d640 1 -- 172.21.15.1:0/3544380074 wait complete. 2024-09-16T00:43:45.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:45 smithi001 ceph-mon[30934]: pgmap v811: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:45.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:45 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3544380074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:45 smithi019 ceph-mon[35186]: pgmap v811: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:46.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:45 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3544380074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:46.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:45 smithi153 ceph-mon[35538]: pgmap v811: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:46.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:45 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3544380074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:46.394 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:43:46.658 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:47.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 -- 172.21.15.1:0/111183759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c106f10 msgr2=0x7f354c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 --2- 172.21.15.1:0/111183759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c106f10 0x7f354c10ea90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f3544009ca0 tx=0x7f354402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 -- 172.21.15.1:0/111183759 shutdown_connections 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 --2- 172.21.15.1:0/111183759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c106f10 0x7f354c10ea90 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 --2- 172.21.15.1:0/111183759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f354c102790 0x7f354c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 --2- 172.21.15.1:0/111183759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f354c101de0 0x7f354c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 -- 172.21.15.1:0/111183759 >> 172.21.15.1:0/111183759 conn(0x7f354c0fd8c0 msgr2=0x7f354c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 -- 172.21.15.1:0/111183759 shutdown_connections 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 -- 172.21.15.1:0/111183759 wait complete. 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 Processor -- start 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.206+0000 7f35523a0640 1 -- start start 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f35523a0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f354c101de0 0x7f354c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f35523a0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c102790 0x7f354c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f35523a0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f354c106f10 0x7f354c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f35523a0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f354c111f70 con 0x7f354c102790 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f35523a0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f354c111df0 con 0x7f354c101de0 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f35523a0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1120f0 con 0x7f354c106f10 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c102790 0x7f354c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c102790 0x7f354c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35240/0 (socket says 172.21.15.1:35240) 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354b7fe640 1 -- 172.21.15.1:0/518836404 learned_addr learned my addr 172.21.15.1:0/518836404 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354bfff640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f354c101de0 0x7f354c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f355093d640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f354c106f10 0x7f354c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354b7fe640 1 -- 172.21.15.1:0/518836404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f354c106f10 msgr2=0x7f354c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354b7fe640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f354c106f10 0x7f354c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354b7fe640 1 -- 172.21.15.1:0/518836404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f354c101de0 msgr2=0x7f354c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354b7fe640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f354c101de0 0x7f354c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354b7fe640 1 -- 172.21.15.1:0/518836404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c19f500 con 0x7f354c102790 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.207+0000 7f354b7fe640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c102790 0x7f354c19aa90 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f354000c920 tx=0x7f354000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.208+0000 7f35497fa640 1 -- 172.21.15.1:0/518836404 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540007c70 con 0x7f354c102790 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.208+0000 7f35523a0640 1 -- 172.21.15.1:0/518836404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c19f7f0 con 0x7f354c102790 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.208+0000 7f35497fa640 1 -- 172.21.15.1:0/518836404 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3540007e10 con 0x7f354c102790 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.208+0000 7f35497fa640 1 -- 172.21.15.1:0/518836404 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540018890 con 0x7f354c102790 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.208+0000 7f35523a0640 1 -- 172.21.15.1:0/518836404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c1a70d0 con 0x7f354c102790 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.210+0000 7f35497fa640 1 -- 172.21.15.1:0/518836404 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3540018a70 con 0x7f354c102790 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.210+0000 7f3554779740 1 Processor -- start 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.210+0000 7f3554779740 1 -- start start 2024-09-16T00:43:47.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.210+0000 7f3554779740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdbef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.210+0000 7f3554779740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbefa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.210+0000 7f3554779740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbfc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.210+0000 7f3554779740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b9bdb552f0 con 0x55b9bdbef5d0 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.210+0000 7f3554779740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b9bdafccf0 con 0x55b9bdbf0b40 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.210+0000 7f3554779740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b9bdb512b0 con 0x55b9bdbfa370 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbefa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbefa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35244/0 (socket says 172.21.15.1:35244) 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354bfff640 1 -- 172.21.15.1:0/81602735 learned_addr learned my addr 172.21.15.1:0/81602735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f355093d640 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdbef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354b7fe640 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbfc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f35497fa640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35340a3240 0x7f35340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354bfff640 1 -- 172.21.15.1:0/81602735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 msgr2=0x55b9bdbfc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354bfff640 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbfc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354bfff640 1 -- 172.21.15.1:0/81602735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 msgr2=0x55b9bdbef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354bfff640 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdbef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354bfff640 1 -- 172.21.15.1:0/81602735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9bdae13a0 con 0x55b9bdbef5d0 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f355093d640 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdbef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f355093d640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35340a3240 0x7f35340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f35497fa640 1 -- 172.21.15.1:0/518836404 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f35400cfe90 con 0x7f354c102790 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f354bfff640 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbefa10 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f353c00bcf0 tx=0x7f353c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255a67790f157d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f35327fc640 1 -- 172.21.15.1:0/81602735 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c015070 con 0x55b9bdbef5d0 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f35327fc640 1 -- 172.21.15.1:0/81602735 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f353c01a940 con 0x55b9bdbef5d0 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.211+0000 7f35327fc640 1 -- 172.21.15.1:0/81602735 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0119d0 con 0x55b9bdbef5d0 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 -- 172.21.15.1:0/81602735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 msgr2=0x55b9bdbefa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbefa10 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f353c00bcf0 tx=0x7f353c00f730 comp rx=0 tx=0).stop 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f355093d640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35340a3240 0x7f35340a5700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3544005cc0 tx=0x7f35440095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 -- 172.21.15.1:0/81602735 shutdown_connections 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbfc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbefa10 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 --2- 172.21.15.1:0/81602735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdbef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 -- 172.21.15.1:0/81602735 >> 172.21.15.1:0/81602735 conn(0x55b9bdbb8310 msgr2=0x55b9bdbebc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 -- 172.21.15.1:0/81602735 shutdown_connections 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 -- 172.21.15.1:0/81602735 wait complete. 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.212+0000 7f3554779740 1 -- start start 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f3554779740 1 Processor -- start 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f3554779740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbf9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f3554779740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbf0b40 0x55b9bdbf6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f3554779740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f3554779740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b9bdbfd070 con 0x55b9bdbfa370 2024-09-16T00:43:47.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f3554779740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b9bdb10060 con 0x55b9bdbef5d0 2024-09-16T00:43:47.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f3554779740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b9bdbfd1f0 con 0x55b9bdbf0b40 2024-09-16T00:43:47.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f3554779740 10 client.0 _ll_register_callbacks cb 0x55b9bd8638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:47.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f355093d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f354bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbf9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f354bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbf9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:32878/0 (socket says 172.21.15.1:32878) 2024-09-16T00:43:47.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f354bfff640 1 -- 172.21.15.1:0/2777118141 learned_addr learned my addr 172.21.15.1:0/2777118141 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:47.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f355093d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35246/0 (socket says 172.21.15.1:35246) 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.213+0000 7f354b7fe640 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbf0b40 0x55b9bdbf6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f355093d640 1 -- 172.21.15.1:0/2777118141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbf0b40 msgr2=0x55b9bdbf6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f355093d640 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbf0b40 0x55b9bdbf6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f355093d640 1 -- 172.21.15.1:0/2777118141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbef5d0 msgr2=0x55b9bdbf9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f355093d640 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbf9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f355093d640 1 -- 172.21.15.1:0/2777118141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9bd8c63c0 con 0x55b9bdbfa370 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f355093d640 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf7530 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3544009bf0 tx=0x7f354402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe1a0932c654e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f352b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f352b7fe640 1 -- 172.21.15.1:0/2777118141 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544004240 con 0x55b9bdbfa370 2024-09-16T00:43:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f352b7fe640 1 -- 172.21.15.1:0/2777118141 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f35440043e0 con 0x55b9bdbfa370 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f3554779740 1 -- 172.21.15.1:0/2777118141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f353c01a940 con 0x55b9bdbfa370 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f352b7fe640 1 -- 172.21.15.1:0/2777118141 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35440045e0 con 0x55b9bdbfa370 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f3554779740 10 client.18981 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f3554779740 1 -- 172.21.15.1:0/2777118141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b9bdc08aa0 con 0x55b9bdbfa370 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.214+0000 7f352b7fe640 1 -- 172.21.15.1:0/2777118141 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f353c01a940 con 0x55b9bdbfa370 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f352b7fe640 1 -- 172.21.15.1:0/2777118141 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f35440305c0 con 0x55b9bdbfa370 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f3554779740 10 client.18981 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f3554779740 10 client.18981 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f3554779740 20 client.18981 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f3554779740 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9bdc09f50 0x55b9bdc0c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f3554779740 4 client.18981 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:47.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f3554779740 1 -- 172.21.15.1:0/2777118141 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9bdb62860 con 0x55b9bdc09f50 2024-09-16T00:43:47.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f354bfff640 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9bdc09f50 0x55b9bdc0c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f354bfff640 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9bdc09f50 0x55b9bdc0c3c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7c882023a7fa5ee server_cookie=9413b105e149a12b in_seq=0 out_seq=0 2024-09-16T00:43:47.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.215+0000 7f352b7fe640 10 client.18981 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:43:47.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f352b7fe640 1 -- 172.21.15.1:0/2777118141 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9bdb62860 con 0x55b9bdc09f50 2024-09-16T00:43:47.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f352b7fe640 10 client.18981 handle_command_reply: tid=0 2024-09-16T00:43:47.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 client.18981 shutdown 2024-09-16T00:43:47.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 -- 172.21.15.1:0/2777118141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbfa370 msgr2=0x55b9bdbf7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf7530 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3544009bf0 tx=0x7f354402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 -- 172.21.15.1:0/2777118141 shutdown_connections 2024-09-16T00:43:47.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf7530 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbf0b40 0x55b9bdbf6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9bdc09f50 0x55b9bdc0c3c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 --2- 172.21.15.1:0/2777118141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbf9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 -- 172.21.15.1:0/2777118141 >> 172.21.15.1:0/2777118141 conn(0x55b9bdbb8310 msgr2=0x55b9bdbf1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f352b7fe640 0 client.18981 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:43:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 -- 172.21.15.1:0/2777118141 shutdown_connections 2024-09-16T00:43:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.216+0000 7f3554779740 1 -- 172.21.15.1:0/2777118141 wait complete. 2024-09-16T00:43:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.217+0000 7f3554779740 20 client.18981 trim_cache size 0 max 16384 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.229+0000 7f3554779740 1 Processor -- start 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f3554779740 1 -- start start 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f3554779740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f3554779740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdc06310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f3554779740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f3554779740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1b37c0 con 0x55b9bdbef5d0 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f3554779740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1b1090 con 0x55b9bdbf0b40 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f3554779740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1aeee0 con 0x55b9bdbfa370 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbebe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35258/0 (socket says 172.21.15.1:35258) 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354bfff640 1 -- 172.21.15.1:0/187581463 learned_addr learned my addr 172.21.15.1:0/187581463 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:47.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354b7fe640 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdc06310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f355093d640 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354bfff640 1 -- 172.21.15.1:0/187581463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 msgr2=0x55b9bdbf2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354bfff640 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354bfff640 1 -- 172.21.15.1:0/187581463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 msgr2=0x55b9bdc06310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354bfff640 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdc06310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354bfff640 1 -- 172.21.15.1:0/187581463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9bdbf9920 con 0x55b9bdbef5d0 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f355093d640 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f354bfff640 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbebe10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f353c0115c0 tx=0x7f353c00b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc3d2048f171ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.230+0000 7f3531ffb640 1 -- 172.21.15.1:0/187581463 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c015050 con 0x55b9bdbef5d0 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3531ffb640 1 -- 172.21.15.1:0/187581463 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f353c00be60 con 0x55b9bdbef5d0 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3531ffb640 1 -- 172.21.15.1:0/187581463 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0048e0 con 0x55b9bdbef5d0 2024-09-16T00:43:47.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3554779740 1 -- 172.21.15.1:0/187581463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 msgr2=0x55b9bdbebe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3554779740 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbebe10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f353c0115c0 tx=0x7f353c00b5e0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3554779740 1 -- 172.21.15.1:0/187581463 shutdown_connections 2024-09-16T00:43:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3554779740 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbf2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3554779740 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdc06310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3554779740 1 --2- 172.21.15.1:0/187581463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdbebe10 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3554779740 1 -- 172.21.15.1:0/187581463 >> 172.21.15.1:0/187581463 conn(0x55b9bdbb8310 msgr2=0x55b9bdbeb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3554779740 1 -- 172.21.15.1:0/187581463 shutdown_connections 2024-09-16T00:43:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.231+0000 7f3554779740 1 -- 172.21.15.1:0/187581463 wait complete. 2024-09-16T00:43:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f3554779740 1 -- start start 2024-09-16T00:43:47.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f3554779740 1 Processor -- start 2024-09-16T00:43:47.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f3554779740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdc6cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f3554779740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdc6d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f3554779740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbfdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f3554779740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1af550 con 0x55b9bdbef5d0 2024-09-16T00:43:47.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f3554779740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1b09c0 con 0x55b9bdbf0b40 2024-09-16T00:43:47.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f3554779740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1afc20 con 0x55b9bdbfa370 2024-09-16T00:43:47.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f3554779740 10 client.0 _ll_register_callbacks cb 0x7f3544009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:47.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f354bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdc6cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f354bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdc6cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35262/0 (socket says 172.21.15.1:35262) 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f354bfff640 1 -- 172.21.15.1:0/69418234 learned_addr learned my addr 172.21.15.1:0/69418234 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.232+0000 7f354b7fe640 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdc6d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f355093d640 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbfdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f354bfff640 1 -- 172.21.15.1:0/69418234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 msgr2=0x55b9bdbfdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f354bfff640 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbfdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f354bfff640 1 -- 172.21.15.1:0/69418234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 msgr2=0x55b9bdc6d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f354bfff640 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdc6d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f354bfff640 1 -- 172.21.15.1:0/69418234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9bdbfe190 con 0x55b9bdbef5d0 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f355093d640 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbfdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f354bfff640 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdc6cf50 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f353c011590 tx=0x7f353c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba8760e131da4d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:47.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f352b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f352b7fe640 1 -- 172.21.15.1:0/69418234 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c034070 con 0x55b9bdbef5d0 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f352b7fe640 1 -- 172.21.15.1:0/69418234 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f353c011a50 con 0x55b9bdbef5d0 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f3554779740 1 -- 172.21.15.1:0/69418234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f353c00be60 con 0x55b9bdbef5d0 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f352b7fe640 1 -- 172.21.15.1:0/69418234 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f353c015020 con 0x55b9bdbef5d0 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f3554779740 10 client.18987 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f3554779740 1 -- 172.21.15.1:0/69418234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b9bdc74720 con 0x55b9bdbef5d0 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.233+0000 7f352b7fe640 1 -- 172.21.15.1:0/69418234 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f353c00be60 con 0x55b9bdbef5d0 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f352b7fe640 1 -- 172.21.15.1:0/69418234 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f353c004b40 con 0x55b9bdbef5d0 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f3554779740 10 client.18987 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f3554779740 10 client.18987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f3554779740 20 client.18987 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f3554779740 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9bdc74bf0 0x55b9bdc76fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:47.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f3554779740 4 client.18987 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f3554779740 1 -- 172.21.15.1:0/69418234 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9bdb5cd70 con 0x55b9bdc74bf0 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f354b7fe640 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9bdc74bf0 0x55b9bdc76fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f354b7fe640 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9bdc74bf0 0x55b9bdc76fe0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb6d77c4d1f18d74 server_cookie=d5133f1670de79f in_seq=0 out_seq=0 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.234+0000 7f352b7fe640 10 client.18987 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f352b7fe640 1 -- 172.21.15.1:0/69418234 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9bdb5cd70 con 0x55b9bdc74bf0 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f352b7fe640 10 client.18987 handle_command_reply: tid=0 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 client.18987 shutdown 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 -- 172.21.15.1:0/69418234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 msgr2=0x55b9bdc6cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdc6cf50 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f353c011590 tx=0x7f353c00b870 comp rx=0 tx=0).stop 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 -- 172.21.15.1:0/69418234 shutdown_connections 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bdbfa370 0x55b9bdbfdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x55b9bdc74bf0 0x55b9bdc76fe0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9bdbf0b40 0x55b9bdc6d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 --2- 172.21.15.1:0/69418234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b9bdbef5d0 0x55b9bdc6cf50 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 -- 172.21.15.1:0/69418234 >> 172.21.15.1:0/69418234 conn(0x55b9bdbb8310 msgr2=0x55b9bdbf9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:47.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f352b7fe640 0 client.18987 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:43:47.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 -- 172.21.15.1:0/69418234 shutdown_connections 2024-09-16T00:43:47.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 1 -- 172.21.15.1:0/69418234 wait complete. 2024-09-16T00:43:47.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.235+0000 7f3554779740 20 client.18987 trim_cache size 0 max 16384 2024-09-16T00:43:47.242 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:43:47.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 -- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35340a3240 msgr2=0x7f35340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35340a3240 0x7f35340a5700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3544005cc0 tx=0x7f35440095c0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 -- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c102790 msgr2=0x7f354c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c102790 0x7f354c19aa90 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f354000c920 tx=0x7f354000cde0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 -- 172.21.15.1:0/518836404 shutdown_connections 2024-09-16T00:43:47.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f35340a3240 0x7f35340a5700 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f354c106f10 0x7f354c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f354c102790 0x7f354c19aa90 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 --2- 172.21.15.1:0/518836404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f354c101de0 0x7f354c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 -- 172.21.15.1:0/518836404 >> 172.21.15.1:0/518836404 conn(0x7f354c0fd8c0 msgr2=0x7f354c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:47.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 -- 172.21.15.1:0/518836404 shutdown_connections 2024-09-16T00:43:47.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:47.236+0000 7f35523a0640 1 -- 172.21.15.1:0/518836404 wait complete. 2024-09-16T00:43:48.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:43:48.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:47 smithi019 ceph-mon[35186]: pgmap v812: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:43:48.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:47 smithi001 ceph-mon[30934]: pgmap v812: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:43:48.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:47 smithi153 ceph-mon[35538]: pgmap v812: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:43:48.290 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:48.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.836+0000 7f45e7318640 1 -- 172.21.15.1:0/3783010866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e0102d40 msgr2=0x7f45e0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:48.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.836+0000 7f45e7318640 1 --2- 172.21.15.1:0/3783010866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e0102d40 0x7f45e0103120 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009900 tx=0x7f45d802a480 comp rx=0 tx=0).stop 2024-09-16T00:43:48.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.836+0000 7f45e7318640 1 -- 172.21.15.1:0/3783010866 shutdown_connections 2024-09-16T00:43:48.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.836+0000 7f45e7318640 1 --2- 172.21.15.1:0/3783010866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e010c170 0x7f45e010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.836+0000 7f45e7318640 1 --2- 172.21.15.1:0/3783010866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e0103660 0x7f45e010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.836+0000 7f45e7318640 1 --2- 172.21.15.1:0/3783010866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e0102d40 0x7f45e0103120 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.836+0000 7f45e7318640 1 -- 172.21.15.1:0/3783010866 >> 172.21.15.1:0/3783010866 conn(0x7f45e00fc7a0 msgr2=0x7f45e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:48.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.837+0000 7f45e7318640 1 -- 172.21.15.1:0/3783010866 shutdown_connections 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.837+0000 7f45e7318640 1 -- 172.21.15.1:0/3783010866 wait complete. 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.837+0000 7f45e7318640 1 Processor -- start 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.837+0000 7f45e7318640 1 -- start start 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e7318640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e0102d40 0x7f45e019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e7318640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e0103660 0x7f45e019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e7318640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e010c170 0x7f45e019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e7318640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0111fc0 con 0x7f45e010c170 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e7318640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0111e40 con 0x7f45e0103660 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e7318640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0112140 con 0x7f45e0102d40 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e58b5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e010c170 0x7f45e019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e58b5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e010c170 0x7f45e019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35278/0 (socket says 172.21.15.1:35278) 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e58b5640 1 -- 172.21.15.1:0/2156987787 learned_addr learned my addr 172.21.15.1:0/2156987787 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e50b4640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e0102d40 0x7f45e019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e48b3640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e0103660 0x7f45e019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e58b5640 1 -- 172.21.15.1:0/2156987787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e0102d40 msgr2=0x7f45e019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e58b5640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e0102d40 0x7f45e019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e58b5640 1 -- 172.21.15.1:0/2156987787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e0103660 msgr2=0x7f45e019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e58b5640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e0103660 0x7f45e019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e58b5640 1 -- 172.21.15.1:0/2156987787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e019f5d0 con 0x7f45e010c170 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e58b5640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e010c170 0x7f45e019eef0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f45d000eb10 tx=0x7f45d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45de7fc640 1 -- 172.21.15.1:0/2156987787 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004b20 con 0x7f45e010c170 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45de7fc640 1 -- 172.21.15.1:0/2156987787 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d0004cc0 con 0x7f45e010c170 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.838+0000 7f45e7318640 1 -- 172.21.15.1:0/2156987787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e019f8c0 con 0x7f45e010c170 2024-09-16T00:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.839+0000 7f45de7fc640 1 -- 172.21.15.1:0/2156987787 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0010760 con 0x7f45e010c170 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.839+0000 7f45e7318640 1 -- 172.21.15.1:0/2156987787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e01a71a0 con 0x7f45e010c170 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.841+0000 7f45de7fc640 1 -- 172.21.15.1:0/2156987787 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f45d0005020 con 0x7f45e010c170 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.841+0000 7f45e96f1740 1 Processor -- start 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.841+0000 7f45e96f1740 1 -- start start 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.841+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645376090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 0x55a645376a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 0x55a645383760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e96f1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a6452dc2f0 con 0x55a645377b40 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e96f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a645283cf0 con 0x55a6453765d0 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e96f1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a6452d82b0 con 0x55a645381370 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45de7fc640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45c80a3240 0x7f45c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645376090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645376090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35282/0 (socket says 172.21.15.1:35282) 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 -- 172.21.15.1:0/3188858111 learned_addr learned my addr 172.21.15.1:0/3188858111 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e50b4640 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 0x55a645376a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45de7fc640 1 -- 172.21.15.1:0/2156987787 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f45d00d0260 con 0x7f45e010c170 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e58b5640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45c80a3240 0x7f45c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 0x55a645383760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 -- 172.21.15.1:0/3188858111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 msgr2=0x55a645383760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 0x55a645383760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 -- 172.21.15.1:0/3188858111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 msgr2=0x55a645376a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 0x55a645376a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 -- 172.21.15.1:0/3188858111 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6452683a0 con 0x55a645377b40 2024-09-16T00:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e50b4640 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 0x55a645376a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e48b3640 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645376090 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f45d40058a0 tx=0x7f45d400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba75670a79acf4fc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.842+0000 7f45e58b5640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45c80a3240 0x7f45c80a5700 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f45d00a4b60 tx=0x7f45d000efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45c77fe640 1 -- 172.21.15.1:0/3188858111 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4004030 con 0x55a645377b40 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45c77fe640 1 -- 172.21.15.1:0/3188858111 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d40041d0 con 0x55a645377b40 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45c77fe640 1 -- 172.21.15.1:0/3188858111 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d400f6e0 con 0x55a645377b40 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45e96f1740 1 -- 172.21.15.1:0/3188858111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 msgr2=0x55a645376090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645376090 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f45d40058a0 tx=0x7f45d400b830 comp rx=0 tx=0).stop 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45e96f1740 1 -- 172.21.15.1:0/3188858111 shutdown_connections 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 0x55a645383760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 0x55a645376a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3188858111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645376090 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45e96f1740 1 -- 172.21.15.1:0/3188858111 >> 172.21.15.1:0/3188858111 conn(0x55a64533f310 msgr2=0x55a645372c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45e96f1740 1 -- 172.21.15.1:0/3188858111 shutdown_connections 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.843+0000 7f45e96f1740 1 -- 172.21.15.1:0/3188858111 wait complete. 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e96f1740 1 -- start start 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e96f1740 1 Processor -- start 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 0x55a645380d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a64537dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 0x55a64537e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e96f1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a645384070 con 0x55a645377b40 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e96f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a645297060 con 0x55a6453765d0 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e96f1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a6453841f0 con 0x55a645381370 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e96f1740 10 client.0 _ll_register_callbacks cb 0x55a644fea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e50b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a64537dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e50b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a64537dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35290/0 (socket says 172.21.15.1:35290) 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.844+0000 7f45e50b4640 1 -- 172.21.15.1:0/3084686022 learned_addr learned my addr 172.21.15.1:0/3084686022 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e48b3640 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 0x55a645380d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e48b3640 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 0x55a64537e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e50b4640 1 -- 172.21.15.1:0/3084686022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 msgr2=0x55a64537e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e50b4640 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 0x55a64537e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e50b4640 1 -- 172.21.15.1:0/3084686022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 msgr2=0x55a645380d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e50b4640 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 0x55a645380d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e50b4640 1 -- 172.21.15.1:0/3084686022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64504d3c0 con 0x55a645377b40 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e48b3640 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 0x55a64537e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e50b4640 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a64537dfe0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009230 tx=0x7f45d80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f31e5328cb7c895 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45c4ff9640 1 -- 172.21.15.1:0/3084686022 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8035040 con 0x55a645377b40 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45c4ff9640 1 -- 172.21.15.1:0/3084686022 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f45d8005450 con 0x55a645377b40 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e96f1740 1 -- 172.21.15.1:0/3084686022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d40041d0 con 0x55a645377b40 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45c4ff9640 1 -- 172.21.15.1:0/3084686022 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45d804f050 con 0x55a645377b40 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e96f1740 10 client.18999 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45e96f1740 1 -- 172.21.15.1:0/3084686022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a64538fad0 con 0x55a645377b40 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.845+0000 7f45c4ff9640 1 -- 172.21.15.1:0/3084686022 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d40041d0 con 0x55a645377b40 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45c4ff9640 1 -- 172.21.15.1:0/3084686022 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f45d80038c0 con 0x55a645377b40 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45e96f1740 10 client.18999 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45e96f1740 10 client.18999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45e96f1740 20 client.18999 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a645390f90 0x55a6453933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45e96f1740 4 client.18999 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45e96f1740 1 -- 172.21.15.1:0/3084686022 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6452e9860 con 0x55a645390f90 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45e48b3640 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a645390f90 0x55a6453933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45c4ff9640 10 client.18999 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.846+0000 7f45e48b3640 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a645390f90 0x55a6453933f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17033638cbad30bc server_cookie=422e5562d7176074 in_seq=0 out_seq=0 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45c4ff9640 1 -- 172.21.15.1:0/3084686022 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6452e9860 con 0x55a645390f90 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45c4ff9640 10 client.18999 handle_command_reply: tid=0 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 client.18999 shutdown 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 -- 172.21.15.1:0/3084686022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 msgr2=0x55a64537dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a64537dfe0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009230 tx=0x7f45d80043b0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 -- 172.21.15.1:0/3084686022 shutdown_connections 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a645381370 0x55a64537e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a645390f90 0x55a6453933f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a64537dfe0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 --2- 172.21.15.1:0/3084686022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6453765d0 0x55a645380d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 -- 172.21.15.1:0/3084686022 >> 172.21.15.1:0/3084686022 conn(0x55a64533f310 msgr2=0x55a645378960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45c4ff9640 0 client.18999 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 -- 172.21.15.1:0/3084686022 shutdown_connections 2024-09-16T00:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.847+0000 7f45e96f1740 1 -- 172.21.15.1:0/3084686022 wait complete. 2024-09-16T00:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.848+0000 7f45e96f1740 20 client.18999 trim_cache size 0 max 16384 2024-09-16T00:43:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.865+0000 7f45e96f1740 1 Processor -- start 2024-09-16T00:43:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.865+0000 7f45e96f1740 1 -- start start 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.865+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 0x55a64537ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.865+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645379120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.865+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 0x55a645379660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.865+0000 7f45e96f1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f45e01b3860 con 0x55a645377b40 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.865+0000 7f45e96f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45e01b1130 con 0x55a645381370 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.865+0000 7f45e96f1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45e01aef80 con 0x55a6453765d0 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e50b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645379120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e50b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645379120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35304/0 (socket says 172.21.15.1:35304) 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e50b4640 1 -- 172.21.15.1:0/2554959053 learned_addr learned my addr 172.21.15.1:0/2554959053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e48b3640 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 0x55a64537ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e48b3640 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 0x55a645379660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e50b4640 1 -- 172.21.15.1:0/2554959053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 msgr2=0x55a64537ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e50b4640 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 0x55a64537ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e50b4640 1 -- 172.21.15.1:0/2554959053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 msgr2=0x55a645379660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e50b4640 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 0x55a645379660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e50b4640 1 -- 172.21.15.1:0/2554959053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64537e940 con 0x55a645377b40 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e48b3640 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 0x55a645379660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45e50b4640 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645379120 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f45d802aa70 tx=0x7f45d802ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44dec21be5f1217 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45c6ffd640 1 -- 172.21.15.1:0/2554959053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8035070 con 0x55a645377b40 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45c6ffd640 1 -- 172.21.15.1:0/2554959053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d8041070 con 0x55a645377b40 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.866+0000 7f45c6ffd640 1 -- 172.21.15.1:0/2554959053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8003a30 con 0x55a645377b40 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 -- 172.21.15.1:0/2554959053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 msgr2=0x55a645379120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645379120 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f45d802aa70 tx=0x7f45d802ac00 comp rx=0 tx=0).stop 2024-09-16T00:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 -- 172.21.15.1:0/2554959053 shutdown_connections 2024-09-16T00:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 0x55a645379660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645379120 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2554959053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 0x55a64537ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 -- 172.21.15.1:0/2554959053 >> 172.21.15.1:0/2554959053 conn(0x55a64533f310 msgr2=0x55a64537f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 -- 172.21.15.1:0/2554959053 shutdown_connections 2024-09-16T00:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 -- 172.21.15.1:0/2554959053 wait complete. 2024-09-16T00:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 -- start start 2024-09-16T00:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.867+0000 7f45e96f1740 1 Processor -- start 2024-09-16T00:43:48.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 0x55a645385130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645385670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e96f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 0x55a645385bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e96f1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f45e01af5f0 con 0x55a645377b40 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e96f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45e01b0a60 con 0x55a645381370 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e96f1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45e01afcc0 con 0x55a6453765d0 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e96f1740 10 client.0 _ll_register_callbacks cb 0x7f45d8005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e50b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645385670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e50b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645385670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35312/0 (socket says 172.21.15.1:35312) 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e50b4640 1 -- 172.21.15.1:0/2513341873 learned_addr learned my addr 172.21.15.1:0/2513341873 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e48b3640 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 0x55a645385130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e48b3640 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 0x55a645385bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e50b4640 1 -- 172.21.15.1:0/2513341873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 msgr2=0x55a645385130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e50b4640 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 0x55a645385130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e50b4640 1 -- 172.21.15.1:0/2513341873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 msgr2=0x55a645385bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e50b4640 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 0x55a645385bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e50b4640 1 -- 172.21.15.1:0/2513341873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6453fac90 con 0x55a645377b40 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.868+0000 7f45e48b3640 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 0x55a645385130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e50b4640 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645385670 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f45d802a9a0 tx=0x7f45d8031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba0ec523575a710 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45c4ff9640 1 -- 172.21.15.1:0/2513341873 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8035070 con 0x55a645377b40 2024-09-16T00:43:48.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45c4ff9640 1 -- 172.21.15.1:0/2513341873 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f45d804c040 con 0x55a645377b40 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e96f1740 1 -- 172.21.15.1:0/2513341873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d8041070 con 0x55a645377b40 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45c4ff9640 1 -- 172.21.15.1:0/2513341873 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45d8005250 con 0x55a645377b40 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e96f1740 10 client.19005 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e96f1740 1 -- 172.21.15.1:0/2513341873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6453fbe40 con 0x55a645377b40 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45c4ff9640 1 -- 172.21.15.1:0/2513341873 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d8041070 con 0x55a645377b40 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45c4ff9640 1 -- 172.21.15.1:0/2513341873 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f45d80315b0 con 0x55a645377b40 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e96f1740 10 client.19005 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e96f1740 10 client.19005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e96f1740 20 client.19005 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a6453fc520 0x55a6453fe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e96f1740 4 client.19005 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.869+0000 7f45e96f1740 1 -- 172.21.15.1:0/2513341873 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6452e3d70 con 0x55a6453fc520 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e48b3640 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a6453fc520 0x55a6453fe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45c4ff9640 10 client.19005 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e48b3640 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a6453fc520 0x55a6453fe940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ea58710eff8152f server_cookie=28110683cb41d8e8 in_seq=0 out_seq=0 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45c4ff9640 1 -- 172.21.15.1:0/2513341873 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6452e3d70 con 0x55a6453fc520 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45c4ff9640 10 client.19005 handle_command_reply: tid=0 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e96f1740 1 client.19005 shutdown 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e96f1740 1 -- 172.21.15.1:0/2513341873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 msgr2=0x55a645385670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645385670 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f45d802a9a0 tx=0x7f45d8031480 comp rx=0 tx=0).stop 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e96f1740 1 -- 172.21.15.1:0/2513341873 shutdown_connections 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a645381370 0x55a645385bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55a6453fc520 0x55a6453fe940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a645377b40 0x55a645385670 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e96f1740 1 --2- 172.21.15.1:0/2513341873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6453765d0 0x55a645385130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45e96f1740 1 -- 172.21.15.1:0/2513341873 >> 172.21.15.1:0/2513341873 conn(0x55a64533f310 msgr2=0x55a645380060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.870+0000 7f45c4ff9640 0 client.19005 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:43:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e96f1740 1 -- 172.21.15.1:0/2513341873 shutdown_connections 2024-09-16T00:43:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e96f1740 1 -- 172.21.15.1:0/2513341873 wait complete. 2024-09-16T00:43:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e96f1740 20 client.19005 trim_cache size 0 max 16384 2024-09-16T00:43:48.878 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:43:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 -- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45c80a3240 msgr2=0x7f45c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45c80a3240 0x7f45c80a5700 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f45d00a4b60 tx=0x7f45d000efd0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 -- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e010c170 msgr2=0x7f45e019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e010c170 0x7f45e019eef0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f45d000eb10 tx=0x7f45d000c2d0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 -- 172.21.15.1:0/2156987787 shutdown_connections 2024-09-16T00:43:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45e010c170 0x7f45e019eef0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e0103660 0x7f45e019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f45c80a3240 0x7f45c80a5700 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 --2- 172.21.15.1:0/2156987787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e0102d40 0x7f45e019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 -- 172.21.15.1:0/2156987787 >> 172.21.15.1:0/2156987787 conn(0x7f45e00fc7a0 msgr2=0x7f45e010d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 -- 172.21.15.1:0/2156987787 shutdown_connections 2024-09-16T00:43:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:48.871+0000 7f45e7318640 1 -- 172.21.15.1:0/2156987787 wait complete. 2024-09-16T00:43:49.694 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:43:49.951 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:49 smithi001 ceph-mon[30934]: pgmap v813: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:49.956 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:50.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:49 smithi019 ceph-mon[35186]: pgmap v813: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:50.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:49 smithi153 ceph-mon[35538]: pgmap v813: 129 pgs: 129 active+clean; 7.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:50.435 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:43:50.435 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-16T00:43:50.436 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:43:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.494+0000 7f688f975640 1 -- 172.21.15.1:0/3413368519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888102570 msgr2=0x7f68881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.494+0000 7f688f975640 1 --2- 172.21.15.1:0/3413368519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888102570 0x7f68881029b0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f687c009900 tx=0x7f687c02a480 comp rx=0 tx=0).stop 2024-09-16T00:43:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.494+0000 7f688f975640 1 -- 172.21.15.1:0/3413368519 shutdown_connections 2024-09-16T00:43:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.494+0000 7f688f975640 1 --2- 172.21.15.1:0/3413368519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6888102ef0 0x7f688810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.494+0000 7f688f975640 1 --2- 172.21.15.1:0/3413368519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888102570 0x7f68881029b0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.494+0000 7f688f975640 1 --2- 172.21.15.1:0/3413368519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6888100e20 0x7f6888101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.494+0000 7f688f975640 1 -- 172.21.15.1:0/3413368519 >> 172.21.15.1:0/3413368519 conn(0x7f68880fc7d0 msgr2=0x7f68880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.494+0000 7f688f975640 1 -- 172.21.15.1:0/3413368519 shutdown_connections 2024-09-16T00:43:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.494+0000 7f688f975640 1 -- 172.21.15.1:0/3413368519 wait complete. 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688f975640 1 Processor -- start 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688f975640 1 -- start start 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688f975640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888100e20 0x7f6888199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688f975640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6888102570 0x7f6888199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688f975640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6888102ef0 0x7f68881a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688f975640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6888110f80 con 0x7f6888100e20 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688f975640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6888110e00 con 0x7f6888102570 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688f975640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6888111100 con 0x7f6888102ef0 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688d711640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888100e20 0x7f6888199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688d711640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888100e20 0x7f6888199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50674/0 (socket says 172.21.15.1:50674) 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688d711640 1 -- 172.21.15.1:0/1316453963 learned_addr learned my addr 172.21.15.1:0/1316453963 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.495+0000 7f688cf10640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6888102570 0x7f6888199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688df12640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6888102ef0 0x7f68881a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688d711640 1 -- 172.21.15.1:0/1316453963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6888102ef0 msgr2=0x7f68881a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688d711640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6888102ef0 0x7f68881a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688d711640 1 -- 172.21.15.1:0/1316453963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6888102570 msgr2=0x7f6888199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688d711640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6888102570 0x7f6888199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688d711640 1 -- 172.21.15.1:0/1316453963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68881a4910 con 0x7f6888100e20 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688df12640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6888102ef0 0x7f68881a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688d711640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888100e20 0x7f6888199350 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f687800eab0 tx=0x7f687800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f68867fc640 1 -- 172.21.15.1:0/1316453963 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687800ce40 con 0x7f6888100e20 2024-09-16T00:43:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f68867fc640 1 -- 172.21.15.1:0/1316453963 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6878011950 con 0x7f6888100e20 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f68867fc640 1 -- 172.21.15.1:0/1316453963 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878010ae0 con 0x7f6888100e20 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688f975640 1 -- 172.21.15.1:0/1316453963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68881a4ae0 con 0x7f6888100e20 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.496+0000 7f688f975640 1 -- 172.21.15.1:0/1316453963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68881a4fa0 con 0x7f6888100e20 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.499+0000 7f68867fc640 1 -- 172.21.15.1:0/1316453963 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f6878011b00 con 0x7f6888100e20 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.499+0000 7f6891d4e740 1 Processor -- start 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.499+0000 7f6891d4e740 1 -- start start 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.499+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 0x55b2500ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.499+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.499+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500b9370 0x55b2500bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.499+0000 7f6891d4e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b2500142f0 con 0x55b2500ae5d0 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.499+0000 7f6891d4e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b24ffbbcf0 con 0x55b2500b9370 2024-09-16T00:43:50.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.499+0000 7f6891d4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b2500102b0 con 0x55b2500afb40 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f68867fc640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68700a3240 0x7f68700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688d711640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688d711640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50678/0 (socket says 172.21.15.1:50678) 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688d711640 1 -- 172.21.15.1:0/2804987280 learned_addr learned my addr 172.21.15.1:0/2804987280 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688df12640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68700a3240 0x7f68700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688cf10640 1 --2- 172.21.15.1:0/2804987280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500b9370 0x55b2500bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688df12640 1 --2- 172.21.15.1:0/2804987280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 0x55b2500ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f68867fc640 1 -- 172.21.15.1:0/1316453963 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f68780d03e0 con 0x7f6888100e20 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688d711640 1 -- 172.21.15.1:0/2804987280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 msgr2=0x55b2500ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688d711640 1 --2- 172.21.15.1:0/2804987280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 0x55b2500ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688d711640 1 -- 172.21.15.1:0/2804987280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500b9370 msgr2=0x55b2500bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:50.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688d711640 1 --2- 172.21.15.1:0/2804987280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500b9370 0x55b2500bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688d711640 1 -- 172.21.15.1:0/2804987280 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b24ffa03a0 con 0x55b2500ae5d0 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688d711640 1 --2- 172.21.15.1:0/2804987280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500aea10 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f687809c7c0 tx=0x7f6878010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476ba54b3ab47889 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f688df12640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68700a3240 0x7f68700a5700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f688000d830 tx=0x7f6880005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.500+0000 7f686f7fe640 1 -- 172.21.15.1:0/2804987280 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878098040 con 0x55b2500ae5d0 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f686f7fe640 1 -- 172.21.15.1:0/2804987280 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687809cd00 con 0x55b2500ae5d0 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f686f7fe640 1 -- 172.21.15.1:0/2804987280 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687809a9b0 con 0x55b2500ae5d0 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f6891d4e740 1 -- 172.21.15.1:0/2804987280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 msgr2=0x55b2500aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2804987280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500aea10 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f687809c7c0 tx=0x7f6878010540 comp rx=0 tx=0).stop 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f6891d4e740 1 -- 172.21.15.1:0/2804987280 shutdown_connections 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2804987280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500b9370 0x55b2500bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2804987280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500aea10 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2804987280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 0x55b2500ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f6891d4e740 1 -- 172.21.15.1:0/2804987280 >> 172.21.15.1:0/2804987280 conn(0x55b250077310 msgr2=0x55b2500aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f6891d4e740 1 -- 172.21.15.1:0/2804987280 shutdown_connections 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.501+0000 7f6891d4e740 1 -- 172.21.15.1:0/2804987280 wait complete. 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f6891d4e740 1 -- start start 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f6891d4e740 1 Processor -- start 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 0x55b2500b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 0x55b2500b6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f6891d4e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b2500bbfb0 con 0x55b2500ae5d0 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f6891d4e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b24ffcf060 con 0x55b2500afb40 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f6891d4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b2500bc130 con 0x55b2500b9370 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f6891d4e740 10 client.0 _ll_register_callbacks cb 0x55b24fd228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f688d711640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f688d711640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500b8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50690/0 (socket says 172.21.15.1:50690) 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.502+0000 7f688d711640 1 -- 172.21.15.1:0/2031861504 learned_addr learned my addr 172.21.15.1:0/2031861504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f688cf10640 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 0x55b2500b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f688df12640 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 0x55b2500b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f688d711640 1 -- 172.21.15.1:0/2031861504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 msgr2=0x55b2500b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f688d711640 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 0x55b2500b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f688d711640 1 -- 172.21.15.1:0/2031861504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 msgr2=0x55b2500b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f688d711640 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 0x55b2500b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f688d711640 1 -- 172.21.15.1:0/2031861504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b24fd853c0 con 0x55b2500ae5d0 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f688df12640 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 0x55b2500b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f688d711640 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500b8dd0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6878098fd0 tx=0x7f68780cda50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fad69e8e557969 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:50.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f686cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f686cff9640 1 -- 172.21.15.1:0/2031861504 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780c0070 con 0x55b2500ae5d0 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f6891d4e740 1 -- 172.21.15.1:0/2031861504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687809cd00 con 0x55b2500ae5d0 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f686cff9640 1 -- 172.21.15.1:0/2031861504 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f687809c860 con 0x55b2500ae5d0 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f686cff9640 1 -- 172.21.15.1:0/2031861504 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6878098040 con 0x55b2500ae5d0 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f6891d4e740 10 client.19017 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f6891d4e740 1 -- 172.21.15.1:0/2031861504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b2500c7ad0 con 0x55b2500ae5d0 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.503+0000 7f686cff9640 1 -- 172.21.15.1:0/2031861504 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687809cd00 con 0x55b2500ae5d0 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f686cff9640 1 -- 172.21.15.1:0/2031861504 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f687800f040 con 0x55b2500ae5d0 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f6891d4e740 10 client.19017 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f6891d4e740 10 client.19017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f6891d4e740 20 client.19017 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b2500c8e20 0x55b2500cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f6891d4e740 4 client.19017 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f6891d4e740 1 -- 172.21.15.1:0/2031861504 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b250021860 con 0x55b2500c8e20 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f688cf10640 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b2500c8e20 0x55b2500cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f688cf10640 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b2500c8e20 0x55b2500cb210 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c03668b169e8dd server_cookie=5cc269ec4c68099c in_seq=0 out_seq=0 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.504+0000 7f686cff9640 10 client.19017 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f686cff9640 1 -- 172.21.15.1:0/2031861504 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b250021860 con 0x55b2500c8e20 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f686cff9640 10 client.19017 handle_command_reply: tid=0 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 client.19017 shutdown 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 -- 172.21.15.1:0/2031861504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 msgr2=0x55b2500b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500b8dd0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6878098fd0 tx=0x7f68780cda50 comp rx=0 tx=0).stop 2024-09-16T00:43:50.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 -- 172.21.15.1:0/2031861504 shutdown_connections 2024-09-16T00:43:50.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b2500c8e20 0x55b2500cb210 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 0x55b2500b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 0x55b2500b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2031861504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500b8dd0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 -- 172.21.15.1:0/2031861504 >> 172.21.15.1:0/2031861504 conn(0x55b250077310 msgr2=0x55b2500b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:50.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f686cff9640 0 client.19017 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:50.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 -- 172.21.15.1:0/2031861504 shutdown_connections 2024-09-16T00:43:50.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.505+0000 7f6891d4e740 1 -- 172.21.15.1:0/2031861504 wait complete. 2024-09-16T00:43:50.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.506+0000 7f6891d4e740 20 client.19017 trim_cache size 0 max 16384 2024-09-16T00:43:50.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.519+0000 7f6891d4e740 1 Processor -- start 2024-09-16T00:43:50.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.519+0000 7f6891d4e740 1 -- start start 2024-09-16T00:43:50.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500ae5d0 0x55b2500b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 0x55b2500b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500b9370 0x55b2500cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f6891d4e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68881af590 con 0x55b2500b9370 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f6891d4e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68881ace60 con 0x55b2500ae5d0 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f6891d4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68881aacb0 con 0x55b2500afb40 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688df12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500b9370 0x55b2500cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688df12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500b9370 0x55b2500cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50692/0 (socket says 172.21.15.1:50692) 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688d711640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500ae5d0 0x55b2500b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688df12640 1 -- 172.21.15.1:0/2258132669 learned_addr learned my addr 172.21.15.1:0/2258132669 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688cf10640 1 --2- 172.21.15.1:0/2258132669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 0x55b2500b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688df12640 1 -- 172.21.15.1:0/2258132669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 msgr2=0x55b2500b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688df12640 1 --2- 172.21.15.1:0/2258132669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 0x55b2500b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688df12640 1 -- 172.21.15.1:0/2258132669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500ae5d0 msgr2=0x55b2500b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688df12640 1 --2- 172.21.15.1:0/2258132669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500ae5d0 0x55b2500b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688df12640 1 -- 172.21.15.1:0/2258132669 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2500b63e0 con 0x55b2500b9370 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.520+0000 7f688df12640 1 --2- 172.21.15.1:0/2258132669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500b9370 0x55b2500cde50 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f688000dee0 tx=0x7f6880005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48035a6e083fbe43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f686effd640 1 -- 172.21.15.1:0/2258132669 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880016660 con 0x55b2500b9370 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f686effd640 1 -- 172.21.15.1:0/2258132669 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880026070 con 0x55b2500b9370 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f686effd640 1 -- 172.21.15.1:0/2258132669 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688002ba60 con 0x55b2500b9370 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f6891d4e740 1 -- 172.21.15.1:0/2258132669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500b9370 msgr2=0x55b2500cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2258132669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500b9370 0x55b2500cde50 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f688000dee0 tx=0x7f6880005900 comp rx=0 tx=0).stop 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f6891d4e740 1 -- 172.21.15.1:0/2258132669 shutdown_connections 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2258132669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500b9370 0x55b2500cde50 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2258132669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500afb40 0x55b2500b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f6891d4e740 1 --2- 172.21.15.1:0/2258132669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500ae5d0 0x55b2500b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f6891d4e740 1 -- 172.21.15.1:0/2258132669 >> 172.21.15.1:0/2258132669 conn(0x55b250077310 msgr2=0x55b2500c4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.521+0000 7f6891d4e740 1 -- 172.21.15.1:0/2258132669 shutdown_connections 2024-09-16T00:43:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 1 -- 172.21.15.1:0/2258132669 wait complete. 2024-09-16T00:43:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 1 -- start start 2024-09-16T00:43:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 1 Processor -- start 2024-09-16T00:43:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 0x55b2500cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 0x55b2500cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68881ab320 con 0x55b2500ae5d0 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68881ac790 con 0x55b2500afb40 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68881ab9f0 con 0x55b2500b9370 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.522+0000 7f6891d4e740 10 client.0 _ll_register_callbacks cb 0x7f687c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688d711640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688d711640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50700/0 (socket says 172.21.15.1:50700) 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688d711640 1 -- 172.21.15.1:0/779154524 learned_addr learned my addr 172.21.15.1:0/779154524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688cf10640 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 0x55b2500cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688df12640 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 0x55b2500cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688d711640 1 -- 172.21.15.1:0/779154524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 msgr2=0x55b2500cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688d711640 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 0x55b2500cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688d711640 1 -- 172.21.15.1:0/779154524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 msgr2=0x55b2500cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688d711640 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 0x55b2500cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688d711640 1 -- 172.21.15.1:0/779154524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b250132760 con 0x55b2500ae5d0 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688cf10640 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 0x55b2500cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f688d711640 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500ce780 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f68780d0790 tx=0x7f6878098900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1310da1522a5821 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f686cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f686cff9640 1 -- 172.21.15.1:0/779154524 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780c0070 con 0x55b2500ae5d0 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f686cff9640 1 -- 172.21.15.1:0/779154524 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f687809c860 con 0x55b2500ae5d0 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.523+0000 7f6891d4e740 1 -- 172.21.15.1:0/779154524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6880026070 con 0x55b2500ae5d0 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f686cff9640 1 -- 172.21.15.1:0/779154524 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f687809c3d0 con 0x55b2500ae5d0 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f6891d4e740 10 client.19023 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f6891d4e740 1 -- 172.21.15.1:0/779154524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b250133940 con 0x55b2500ae5d0 2024-09-16T00:43:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f686cff9640 1 -- 172.21.15.1:0/779154524 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880026070 con 0x55b2500ae5d0 2024-09-16T00:43:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f686cff9640 1 -- 172.21.15.1:0/779154524 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f68780671b0 con 0x55b2500ae5d0 2024-09-16T00:43:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f6891d4e740 10 client.19023 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f6891d4e740 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f6891d4e740 20 client.19023 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f6891d4e740 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b250133e10 0x55b250136200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f6891d4e740 4 client.19023 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:43:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f6891d4e740 1 -- 172.21.15.1:0/779154524 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b25001bd70 con 0x55b250133e10 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.524+0000 7f688cf10640 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b250133e10 0x55b250136200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f688cf10640 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b250133e10 0x55b250136200 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3085e9d38d5d86b5 server_cookie=af8c73ce9ddee673 in_seq=0 out_seq=0 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f686cff9640 10 client.19023 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f686cff9640 1 -- 172.21.15.1:0/779154524 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b25001bd70 con 0x55b250133e10 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f686cff9640 10 client.19023 handle_command_reply: tid=0 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f6891d4e740 1 client.19023 shutdown 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f6891d4e740 1 -- 172.21.15.1:0/779154524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 msgr2=0x55b2500ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f6891d4e740 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500ce780 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f68780d0790 tx=0x7f6878098900 comp rx=0 tx=0).stop 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f6891d4e740 1 -- 172.21.15.1:0/779154524 shutdown_connections 2024-09-16T00:43:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f6891d4e740 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55b250133e10 0x55b250136200 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f686cff9640 0 client.19023 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f6891d4e740 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2500b9370 0x55b2500cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f6891d4e740 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2500afb40 0x55b2500cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f6891d4e740 1 --2- 172.21.15.1:0/779154524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2500ae5d0 0x55b2500ce780 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.525+0000 7f6891d4e740 1 -- 172.21.15.1:0/779154524 >> 172.21.15.1:0/779154524 conn(0x55b250077310 msgr2=0x55b2500b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.526+0000 7f6891d4e740 1 -- 172.21.15.1:0/779154524 shutdown_connections 2024-09-16T00:43:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.526+0000 7f6891d4e740 1 -- 172.21.15.1:0/779154524 wait complete. 2024-09-16T00:43:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.526+0000 7f6891d4e740 20 client.19023 trim_cache size 0 max 16384 2024-09-16T00:43:50.533 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:43:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.526+0000 7f688f975640 1 -- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68700a3240 msgr2=0x7f68700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.526+0000 7f688f975640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68700a3240 0x7f68700a5700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f688000d830 tx=0x7f6880005d10 comp rx=0 tx=0).stop 2024-09-16T00:43:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.526+0000 7f688f975640 1 -- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888100e20 msgr2=0x7f6888199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.526+0000 7f688f975640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888100e20 0x7f6888199350 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f687800eab0 tx=0x7f687800ef70 comp rx=0 tx=0).stop 2024-09-16T00:43:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.527+0000 7f688f975640 1 -- 172.21.15.1:0/1316453963 shutdown_connections 2024-09-16T00:43:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.527+0000 7f688f975640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68700a3240 0x7f68700a5700 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.527+0000 7f688f975640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6888102ef0 0x7f68881a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.527+0000 7f688f975640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6888102570 0x7f6888199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.527+0000 7f688f975640 1 --2- 172.21.15.1:0/1316453963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6888100e20 0x7f6888199350 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.527+0000 7f688f975640 1 -- 172.21.15.1:0/1316453963 >> 172.21.15.1:0/1316453963 conn(0x7f68880fc7d0 msgr2=0x7f68880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.527+0000 7f688f975640 1 -- 172.21.15.1:0/1316453963 shutdown_connections 2024-09-16T00:43:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:50.527+0000 7f688f975640 1 -- 172.21.15.1:0/1316453963 wait complete. 2024-09-16T00:43:51.470 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:51.903 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:51 smithi001 ceph-mon[30934]: pgmap v814: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.986+0000 7f470b07a640 1 -- 172.21.15.1:0/595346893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704100a80 msgr2=0x7f4704100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.986+0000 7f470b07a640 1 --2- 172.21.15.1:0/595346893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704100a80 0x7f4704100ee0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009930 tx=0x7f46f402a420 comp rx=0 tx=0).stop 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.986+0000 7f470b07a640 1 -- 172.21.15.1:0/595346893 shutdown_connections 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.986+0000 7f470b07a640 1 --2- 172.21.15.1:0/595346893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4704101420 0x7f470410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.986+0000 7f470b07a640 1 --2- 172.21.15.1:0/595346893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704100a80 0x7f4704100ee0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.986+0000 7f470b07a640 1 --2- 172.21.15.1:0/595346893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4704106a80 0x7f4704106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.986+0000 7f470b07a640 1 -- 172.21.15.1:0/595346893 >> 172.21.15.1:0/595346893 conn(0x7f47040fc7a0 msgr2=0x7f47040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.986+0000 7f470b07a640 1 -- 172.21.15.1:0/595346893 shutdown_connections 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.986+0000 7f470b07a640 1 -- 172.21.15.1:0/595346893 wait complete. 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f470b07a640 1 Processor -- start 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f470b07a640 1 -- start start 2024-09-16T00:43:52.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f470b07a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4704100a80 0x7f47040fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f470b07a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704101420 0x7f47040fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f470b07a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4704106a80 0x7f47040fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f470b07a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4704110e80 con 0x7f4704101420 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f470b07a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4704110d00 con 0x7f4704100a80 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f470b07a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4704111000 con 0x7f4704106a80 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4703fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704101420 0x7f47040fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4703fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704101420 0x7f47040fd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50716/0 (socket says 172.21.15.1:50716) 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4703fff640 1 -- 172.21.15.1:0/2989010188 learned_addr learned my addr 172.21.15.1:0/2989010188 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4708e16640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4704100a80 0x7f47040fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4709617640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4704106a80 0x7f47040fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4703fff640 1 -- 172.21.15.1:0/2989010188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4704106a80 msgr2=0x7f47040fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4703fff640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4704106a80 0x7f47040fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4703fff640 1 -- 172.21.15.1:0/2989010188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4704100a80 msgr2=0x7f47040fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4703fff640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4704100a80 0x7f47040fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.987+0000 7f4703fff640 1 -- 172.21.15.1:0/2989010188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47041a6960 con 0x7f4704101420 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.988+0000 7f4703fff640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704101420 0x7f47040fd390 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009230 tx=0x7f46f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.988+0000 7f4701ffb640 1 -- 172.21.15.1:0/2989010188 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f403f030 con 0x7f4704101420 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.988+0000 7f4701ffb640 1 -- 172.21.15.1:0/2989010188 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f4003ad0 con 0x7f4704101420 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.988+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47041a6bf0 con 0x7f4704101420 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.988+0000 7f4701ffb640 1 -- 172.21.15.1:0/2989010188 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4035070 con 0x7f4704101420 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.988+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47041a70d0 con 0x7f4704101420 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.989+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4704100ee0 con 0x7f4704101420 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.990+0000 7f4701ffb640 1 -- 172.21.15.1:0/2989010188 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f46f4042030 con 0x7f4704101420 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.991+0000 7f4701ffb640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46f00a3240 0x7f46f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:52.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.991+0000 7f4701ffb640 1 -- 172.21.15.1:0/2989010188 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f46f40f0290 con 0x7f4704101420 2024-09-16T00:43:52.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.991+0000 7f4708e16640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46f00a3240 0x7f46f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:52.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.992+0000 7f4708e16640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46f00a3240 0x7f46f00a5700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f46ec005d90 tx=0x7f46ec005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:52.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:51.995+0000 7f4701ffb640 1 -- 172.21.15.1:0/2989010188 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46f40bcbb0 con 0x7f4704101420 2024-09-16T00:43:52.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:51 smithi019 ceph-mon[35186]: pgmap v814: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:52.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.207+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4704109a70 con 0x7f4704101420 2024-09-16T00:43:52.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.207+0000 7f4701ffb640 1 -- 172.21.15.1:0/2989010188 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f46f40bc4b0 con 0x7f4704101420 2024-09-16T00:43:52.208 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:43:52.208 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:43:52.208 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:43:52.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46f00a3240 msgr2=0x7f46f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:52.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46f00a3240 0x7f46f00a5700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f46ec005d90 tx=0x7f46ec005d20 comp rx=0 tx=0).stop 2024-09-16T00:43:52.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704101420 msgr2=0x7f47040fd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:52.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704101420 0x7f47040fd390 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009230 tx=0x7f46f4004270 comp rx=0 tx=0).stop 2024-09-16T00:43:52.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 shutdown_connections 2024-09-16T00:43:52.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46f00a3240 0x7f46f00a5700 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:52.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4704106a80 0x7f47040fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:52.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4704101420 0x7f47040fd390 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:52.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 --2- 172.21.15.1:0/2989010188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4704100a80 0x7f47040fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:52.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 >> 172.21.15.1:0/2989010188 conn(0x7f47040fc7a0 msgr2=0x7f470406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:52.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 shutdown_connections 2024-09-16T00:43:52.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:52.210+0000 7f470b07a640 1 -- 172.21.15.1:0/2989010188 wait complete. 2024-09-16T00:43:52.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:51 smithi153 ceph-mon[35538]: pgmap v814: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:52.364 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:43:52.365 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:43:52.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:52 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2989010188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:52.919 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-16T00:43:52.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-16T00:43:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:53.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:52 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2989010188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:53.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:52 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2989010188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:53.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.456+0000 7f61eba70640 1 -- 172.21.15.1:0/2916533712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e41027a0 msgr2=0x7f61e41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:53.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.456+0000 7f61eba70640 1 --2- 172.21.15.1:0/2916533712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e41027a0 0x7f61e41067f0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009900 tx=0x7f61d402a410 comp rx=0 tx=0).stop 2024-09-16T00:43:53.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.456+0000 7f61eba70640 1 -- 172.21.15.1:0/2916533712 shutdown_connections 2024-09-16T00:43:53.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.456+0000 7f61eba70640 1 --2- 172.21.15.1:0/2916533712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e4106f20 0x7f61e410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.456+0000 7f61eba70640 1 --2- 172.21.15.1:0/2916533712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e41027a0 0x7f61e41067f0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.456+0000 7f61eba70640 1 --2- 172.21.15.1:0/2916533712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e4101df0 0x7f61e41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.456+0000 7f61eba70640 1 -- 172.21.15.1:0/2916533712 >> 172.21.15.1:0/2916533712 conn(0x7f61e40fd8b0 msgr2=0x7f61e40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:53.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.456+0000 7f61eba70640 1 -- 172.21.15.1:0/2916533712 shutdown_connections 2024-09-16T00:43:53.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.456+0000 7f61eba70640 1 -- 172.21.15.1:0/2916533712 wait complete. 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.457+0000 7f61eba70640 1 Processor -- start 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.457+0000 7f61eba70640 1 -- start start 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.457+0000 7f61eba70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e4101df0 0x7f61e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.457+0000 7f61eba70640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e41027a0 0x7f61e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.457+0000 7f61eba70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e4106f20 0x7f61e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.457+0000 7f61eba70640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4111f80 con 0x7f61e4106f20 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.457+0000 7f61eba70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4111e00 con 0x7f61e4101df0 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.457+0000 7f61eba70640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4112100 con 0x7f61e41027a0 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61ea00d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e4106f20 0x7f61e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61ea00d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e4106f20 0x7f61e4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50744/0 (socket says 172.21.15.1:50744) 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61ea00d640 1 -- 172.21.15.1:0/1272132859 learned_addr learned my addr 172.21.15.1:0/1272132859 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:53.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61e980c640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e4101df0 0x7f61e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61e900b640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e41027a0 0x7f61e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61ea00d640 1 -- 172.21.15.1:0/1272132859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e41027a0 msgr2=0x7f61e4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61ea00d640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e41027a0 0x7f61e4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61ea00d640 1 -- 172.21.15.1:0/1272132859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e4101df0 msgr2=0x7f61e40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61ea00d640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e4101df0 0x7f61e40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61ea00d640 1 -- 172.21.15.1:0/1272132859 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e41a68a0 con 0x7f61e4106f20 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61ea00d640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e4106f20 0x7f61e4074f70 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00eaa0 tx=0x7f61dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61daffd640 1 -- 172.21.15.1:0/1272132859 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00cdb0 con 0x7f61e4106f20 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61daffd640 1 -- 172.21.15.1:0/1272132859 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61dc005020 con 0x7f61e4106f20 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61eba70640 1 -- 172.21.15.1:0/1272132859 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e41a6b90 con 0x7f61e4106f20 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61daffd640 1 -- 172.21.15.1:0/1272132859 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0116b0 con 0x7f61e4106f20 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.458+0000 7f61eba70640 1 -- 172.21.15.1:0/1272132859 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e41a70a0 con 0x7f61e4106f20 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.460+0000 7f61daffd640 1 -- 172.21.15.1:0/1272132859 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f61dc005560 con 0x7f61e4106f20 2024-09-16T00:43:53.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61ede49740 1 Processor -- start 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61ede49740 1 -- start start 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61ede49740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61ede49740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 0x557a32dff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61ede49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32dffd40 0x557a32e02130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61ede49740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557a32d5a280 con 0x557a32df4b60 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61ede49740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a32d01b10 con 0x557a32df5480 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61ede49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a32d56240 con 0x557a32dffd40 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61e900b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61e900b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50750/0 (socket says 172.21.15.1:50750) 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.461+0000 7f61e900b640 1 -- 172.21.15.1:0/1632482876 learned_addr learned my addr 172.21.15.1:0/1632482876 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61daffd640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61d00a3240 0x7f61d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61e980c640 1 --2- 172.21.15.1:0/1632482876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 0x557a32dff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61e900b640 1 -- 172.21.15.1:0/1632482876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32dffd40 msgr2=0x557a32e02130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61e900b640 1 --2- 172.21.15.1:0/1632482876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32dffd40 0x557a32e02130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61e900b640 1 -- 172.21.15.1:0/1632482876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 msgr2=0x557a32dff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61e900b640 1 --2- 172.21.15.1:0/1632482876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 0x557a32dff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61e900b640 1 -- 172.21.15.1:0/1632482876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a32d0e1f0 con 0x557a32df4b60 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61e980c640 1 --2- 172.21.15.1:0/1632482876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 0x557a32dff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61ea00d640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61d00a3240 0x7f61d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61daffd640 1 -- 172.21.15.1:0/1272132859 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f61dc0cfe40 con 0x7f61e4106f20 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61e900b640 1 --2- 172.21.15.1:0/1632482876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df4f40 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f61d402a930 tx=0x7f61d40315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e968a34403de63d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61cffff640 1 -- 172.21.15.1:0/1632482876 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4040070 con 0x557a32df4b60 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61cffff640 1 -- 172.21.15.1:0/1632482876 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4005020 con 0x557a32df4b60 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.462+0000 7f61cffff640 1 -- 172.21.15.1:0/1632482876 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4035040 con 0x557a32df4b60 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 -- 172.21.15.1:0/1632482876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 msgr2=0x557a32df4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:53.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 --2- 172.21.15.1:0/1632482876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df4f40 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f61d402a930 tx=0x7f61d40315b0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ea00d640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61d00a3240 0x7f61d00a5700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f61dc09c0a0 tx=0x7f61dc0054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 -- 172.21.15.1:0/1632482876 shutdown_connections 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 --2- 172.21.15.1:0/1632482876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32dffd40 0x557a32e02130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 --2- 172.21.15.1:0/1632482876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 0x557a32dff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 --2- 172.21.15.1:0/1632482876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df4f40 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 -- 172.21.15.1:0/1632482876 >> 172.21.15.1:0/1632482876 conn(0x557a32df10f0 msgr2=0x557a32df14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 -- 172.21.15.1:0/1632482876 shutdown_connections 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 -- 172.21.15.1:0/1632482876 wait complete. 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.463+0000 7f61ede49740 1 -- start start 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61ede49740 1 Processor -- start 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61ede49740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df4b60 0x557a32dfcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61ede49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 0x557a32dfd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61ede49740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32dffd40 0x557a32dfd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61ede49740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557a32e02910 con 0x557a32dffd40 2024-09-16T00:43:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61ede49740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a32d14f10 con 0x557a32df4b60 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61ede49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a32e02a90 con 0x557a32df5480 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61ede49740 10 client.0 _ll_register_callbacks cb 0x557a32cdda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e900b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32dffd40 0x557a32dfd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e900b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32dffd40 0x557a32dfd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50756/0 (socket says 172.21.15.1:50756) 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e900b640 1 -- 172.21.15.1:0/3525138297 learned_addr learned my addr 172.21.15.1:0/3525138297 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e980c640 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 0x557a32dfd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e900b640 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df4b60 0x557a32dfcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e900b640 1 -- 172.21.15.1:0/3525138297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 msgr2=0x557a32dfd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e900b640 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 0x557a32dfd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e900b640 1 -- 172.21.15.1:0/3525138297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df4b60 msgr2=0x557a32dfcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e900b640 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df4b60 0x557a32dfcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.464+0000 7f61e900b640 1 -- 172.21.15.1:0/3525138297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a32acb6a0 con 0x557a32dffd40 2024-09-16T00:43:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61e900b640 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32dffd40 0x557a32dfd640 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f61d4031aa0 tx=0x7f61d4031b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe0cbe4ff51e404 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3525138297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4048070 con 0x557a32dffd40 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3525138297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f61d40055f0 con 0x557a32dffd40 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61ede49740 1 -- 172.21.15.1:0/3525138297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4005020 con 0x557a32dffd40 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3525138297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61d4035030 con 0x557a32dffd40 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61ede49740 10 client.19041 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61ede49740 1 -- 172.21.15.1:0/3525138297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a32e0e4f0 con 0x557a32dffd40 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3525138297 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4005020 con 0x557a32dffd40 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3525138297 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f61d403c930 con 0x557a32dffd40 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61ede49740 10 client.19041 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61ede49740 10 client.19041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61ede49740 20 client.19041 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61ede49740 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557a32e0f840 0x557a32e11c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61ede49740 4 client.19041 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.465+0000 7f61ede49740 1 -- 172.21.15.1:0/3525138297 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a32d677f0 con 0x557a32e0f840 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.466+0000 7f61e980c640 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557a32e0f840 0x557a32e11c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.466+0000 7f61e980c640 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557a32e0f840 0x557a32e11c30 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc46be6c9ff267 server_cookie=ebfe5a21dfa93809 in_seq=0 out_seq=0 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.466+0000 7f61cd7fa640 10 client.19041 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3525138297 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a32d677f0 con 0x557a32e0f840 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61cd7fa640 10 client.19041 handle_command_reply: tid=0 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 client.19041 shutdown 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 -- 172.21.15.1:0/3525138297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32dffd40 msgr2=0x557a32dfd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32dffd40 0x557a32dfd640 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f61d4031aa0 tx=0x7f61d4031b00 comp rx=0 tx=0).stop 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 -- 172.21.15.1:0/3525138297 shutdown_connections 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557a32e0f840 0x557a32e11c30 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32dffd40 0x557a32dfd640 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 0x557a32dfd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 --2- 172.21.15.1:0/3525138297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df4b60 0x557a32dfcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61cd7fa640 0 client.19041 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:53.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 -- 172.21.15.1:0/3525138297 >> 172.21.15.1:0/3525138297 conn(0x557a32df10f0 msgr2=0x557a32df6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:53.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 -- 172.21.15.1:0/3525138297 shutdown_connections 2024-09-16T00:43:53.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 1 -- 172.21.15.1:0/3525138297 wait complete. 2024-09-16T00:43:53.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.467+0000 7f61ede49740 20 client.19041 trim_cache size 0 max 16384 2024-09-16T00:43:53.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.485+0000 7f61ede49740 1 Processor -- start 2024-09-16T00:43:53.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.485+0000 7f61ede49740 1 -- start start 2024-09-16T00:43:53.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.485+0000 7f61ede49740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.485+0000 7f61ede49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 0x557a32dfea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.485+0000 7f61ede49740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32dffd40 0x557a32e17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.485+0000 7f61ede49740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b3860 con 0x557a32df4b60 2024-09-16T00:43:53.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.485+0000 7f61ede49740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b1130 con 0x557a32dffd40 2024-09-16T00:43:53.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.485+0000 7f61ede49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41aef80 con 0x557a32df5480 2024-09-16T00:43:53.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50770/0 (socket says 172.21.15.1:50770) 2024-09-16T00:43:53.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 -- 172.21.15.1:0/1330305704 learned_addr learned my addr 172.21.15.1:0/1330305704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:53.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 --2- 172.21.15.1:0/1330305704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32dffd40 0x557a32e17230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e980c640 1 --2- 172.21.15.1:0/1330305704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 0x557a32dfea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 -- 172.21.15.1:0/1330305704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 msgr2=0x557a32dfea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:53.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 --2- 172.21.15.1:0/1330305704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 0x557a32dfea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 -- 172.21.15.1:0/1330305704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32dffd40 msgr2=0x557a32e17230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:53.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 --2- 172.21.15.1:0/1330305704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32dffd40 0x557a32e17230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 -- 172.21.15.1:0/1330305704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a32df0f20 con 0x557a32df4b60 2024-09-16T00:43:53.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.486+0000 7f61e900b640 1 --2- 172.21.15.1:0/1330305704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df75c0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f61d40096c0 tx=0x7f61d4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fafc849dc795b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:53.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.487+0000 7f61cf7fe640 1 -- 172.21.15.1:0/1330305704 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4048040 con 0x557a32df4b60 2024-09-16T00:43:53.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.487+0000 7f61cf7fe640 1 -- 172.21.15.1:0/1330305704 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4031860 con 0x557a32df4b60 2024-09-16T00:43:53.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.487+0000 7f61cf7fe640 1 -- 172.21.15.1:0/1330305704 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4040040 con 0x557a32df4b60 2024-09-16T00:43:53.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 -- 172.21.15.1:0/1330305704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 msgr2=0x557a32df75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:53.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 --2- 172.21.15.1:0/1330305704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df75c0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f61d40096c0 tx=0x7f61d4031400 comp rx=0 tx=0).stop 2024-09-16T00:43:53.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 -- 172.21.15.1:0/1330305704 shutdown_connections 2024-09-16T00:43:53.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 --2- 172.21.15.1:0/1330305704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32dffd40 0x557a32e17230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 --2- 172.21.15.1:0/1330305704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32df5480 0x557a32dfea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 --2- 172.21.15.1:0/1330305704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df75c0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 -- 172.21.15.1:0/1330305704 >> 172.21.15.1:0/1330305704 conn(0x557a32df10f0 msgr2=0x557a32dfe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:53.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 -- 172.21.15.1:0/1330305704 shutdown_connections 2024-09-16T00:43:53.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 -- 172.21.15.1:0/1330305704 wait complete. 2024-09-16T00:43:53.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.488+0000 7f61ede49740 1 -- start start 2024-09-16T00:43:53.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.489+0000 7f61ede49740 1 Processor -- start 2024-09-16T00:43:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.489+0000 7f61ede49740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.489+0000 7f61ede49740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 0x557a32dfa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.489+0000 7f61ede49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32dffd40 0x557a32dfa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.489+0000 7f61ede49740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41af5f0 con 0x557a32df4b60 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.489+0000 7f61ede49740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b0a60 con 0x557a32df5480 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.489+0000 7f61ede49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41afcc0 con 0x557a32dffd40 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.489+0000 7f61ede49740 10 client.0 _ll_register_callbacks cb 0x7f61d4005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50772/0 (socket says 172.21.15.1:50772) 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 -- 172.21.15.1:0/3387869913 learned_addr learned my addr 172.21.15.1:0/3387869913 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e980c640 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 0x557a32dfa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32dffd40 0x557a32dfa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 -- 172.21.15.1:0/3387869913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32dffd40 msgr2=0x557a32dfa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32dffd40 0x557a32dfa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 -- 172.21.15.1:0/3387869913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 msgr2=0x557a32dfa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 0x557a32dfa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 -- 172.21.15.1:0/3387869913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a32e79200 con 0x557a32df4b60 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61e900b640 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df9be0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009900 tx=0x7f61d40314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789d491959e0f1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.490+0000 7f61cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3387869913 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4040070 con 0x557a32df4b60 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3387869913 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f61d403c440 con 0x557a32df4b60 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61ede49740 1 -- 172.21.15.1:0/3387869913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4031860 con 0x557a32df4b60 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3387869913 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61d4048040 con 0x557a32df4b60 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61ede49740 10 client.19047 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61ede49740 1 -- 172.21.15.1:0/3387869913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a32e82d30 con 0x557a32df4b60 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3387869913 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4031860 con 0x557a32df4b60 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3387869913 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f61d4069070 con 0x557a32df4b60 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61ede49740 10 client.19047 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61ede49740 10 client.19047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61ede49740 20 client.19047 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61ede49740 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557a32e83290 0x557a32e856b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.491+0000 7f61ede49740 4 client.19047 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:43:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.492+0000 7f61ede49740 1 -- 172.21.15.1:0/3387869913 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557a32d61d00 con 0x557a32e83290 2024-09-16T00:43:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.492+0000 7f61e980c640 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557a32e83290 0x557a32e856b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.492+0000 7f61e980c640 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557a32e83290 0x557a32e856b0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3f3718e17819ee server_cookie=26e64e279891791e in_seq=0 out_seq=0 2024-09-16T00:43:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.492+0000 7f61cd7fa640 10 client.19047 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.493+0000 7f61cd7fa640 1 -- 172.21.15.1:0/3387869913 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a32d61d00 con 0x557a32e83290 2024-09-16T00:43:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.493+0000 7f61cd7fa640 10 client.19047 handle_command_reply: tid=0 2024-09-16T00:43:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.493+0000 7f61ede49740 1 client.19047 shutdown 2024-09-16T00:43:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.493+0000 7f61ede49740 1 -- 172.21.15.1:0/3387869913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 msgr2=0x557a32df9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.493+0000 7f61ede49740 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df9be0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009900 tx=0x7f61d40314e0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61ede49740 1 -- 172.21.15.1:0/3387869913 shutdown_connections 2024-09-16T00:43:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61ede49740 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x557a32e83290 0x557a32e856b0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61ede49740 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a32dffd40 0x557a32dfa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61ede49740 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32df5480 0x557a32dfa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61ede49740 1 --2- 172.21.15.1:0/3387869913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a32df4b60 0x557a32df9be0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61cd7fa640 0 client.19047 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61ede49740 1 -- 172.21.15.1:0/3387869913 >> 172.21.15.1:0/3387869913 conn(0x557a32df10f0 msgr2=0x557a32df0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61ede49740 1 -- 172.21.15.1:0/3387869913 shutdown_connections 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61ede49740 1 -- 172.21.15.1:0/3387869913 wait complete. 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61ede49740 20 client.19047 trim_cache size 0 max 16384 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:43:53.497 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:43:53.498 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "11cedabf-c707-48cf-89b2-9939ee6f623c", 2024-09-16T00:43:53.498 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:43:53.498 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:43:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61eba70640 1 -- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61d00a3240 msgr2=0x7f61d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61eba70640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61d00a3240 0x7f61d00a5700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f61dc09c0a0 tx=0x7f61dc0054d0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61eba70640 1 -- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e4106f20 msgr2=0x7f61e4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.494+0000 7f61eba70640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e4106f20 0x7f61e4074f70 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00eaa0 tx=0x7f61dc00ef60 comp rx=0 tx=0).stop 2024-09-16T00:43:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.495+0000 7f61eba70640 1 -- 172.21.15.1:0/1272132859 shutdown_connections 2024-09-16T00:43:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.495+0000 7f61eba70640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61e4106f20 0x7f61e4074f70 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.495+0000 7f61eba70640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f61d00a3240 0x7f61d00a5700 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.495+0000 7f61eba70640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e41027a0 0x7f61e4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.495+0000 7f61eba70640 1 --2- 172.21.15.1:0/1272132859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e4101df0 0x7f61e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.495+0000 7f61eba70640 1 -- 172.21.15.1:0/1272132859 >> 172.21.15.1:0/1272132859 conn(0x7f61e40fd8b0 msgr2=0x7f61e40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.495+0000 7f61eba70640 1 -- 172.21.15.1:0/1272132859 shutdown_connections 2024-09-16T00:43:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:53.495+0000 7f61eba70640 1 -- 172.21.15.1:0/1272132859 wait complete. 2024-09-16T00:43:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:54.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:53 smithi019 ceph-mon[35186]: pgmap v815: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:43:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:53 smithi001 ceph-mon[30934]: pgmap v815: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:43:54.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:53 smithi153 ceph-mon[35538]: pgmap v815: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-16T00:43:54.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:43:54.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.755+0000 7fde3fcd7640 1 -- 172.21.15.1:0/2018547860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38100a90 msgr2=0x7fde38100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:54.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.755+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/2018547860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38100a90 0x7fde38100ef0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fde28009930 tx=0x7fde2802a420 comp rx=0 tx=0).stop 2024-09-16T00:43:54.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.755+0000 7fde3fcd7640 1 -- 172.21.15.1:0/2018547860 shutdown_connections 2024-09-16T00:43:54.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.755+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/2018547860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde38101430 0x7fde3810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.755+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/2018547860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38100a90 0x7fde38100ef0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.755+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/2018547860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde38106a90 0x7fde38106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.755+0000 7fde3fcd7640 1 -- 172.21.15.1:0/2018547860 >> 172.21.15.1:0/2018547860 conn(0x7fde380fc7d0 msgr2=0x7fde380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.756+0000 7fde3fcd7640 1 -- 172.21.15.1:0/2018547860 shutdown_connections 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.756+0000 7fde3fcd7640 1 -- 172.21.15.1:0/2018547860 wait complete. 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.756+0000 7fde3fcd7640 1 Processor -- start 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.756+0000 7fde3fcd7640 1 -- start start 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.756+0000 7fde3fcd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde38100a90 0x7fde381984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3fcd7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde38101430 0x7fde381989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3fcd7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38106a90 0x7fde3819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3fcd7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde38110e90 con 0x7fde38106a90 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3fcd7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde38110d10 con 0x7fde38100a90 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3fcd7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde38111010 con 0x7fde38101430 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3da73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde38100a90 0x7fde381984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3e274640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38106a90 0x7fde3819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3da73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde38100a90 0x7fde381984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:47278/0 (socket says 172.21.15.1:47278) 2024-09-16T00:43:54.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3e274640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38106a90 0x7fde3819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50780/0 (socket says 172.21.15.1:50780) 2024-09-16T00:43:54.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3da73640 1 -- 172.21.15.1:0/1310360561 learned_addr learned my addr 172.21.15.1:0/1310360561 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:54.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3d272640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde38101430 0x7fde381989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3da73640 1 -- 172.21.15.1:0/1310360561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde38101430 msgr2=0x7fde381989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3da73640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde38101430 0x7fde381989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3da73640 1 -- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38106a90 msgr2=0x7fde3819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3da73640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38106a90 0x7fde3819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3da73640 1 -- 172.21.15.1:0/1310360561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3819d450 con 0x7fde38100a90 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3d272640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde38101430 0x7fde381989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.757+0000 7fde3e274640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38106a90 0x7fde3819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.758+0000 7fde3da73640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde38100a90 0x7fde381984a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fde3400e9f0 tx=0x7fde3400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.758+0000 7fde2effd640 1 -- 172.21.15.1:0/1310360561 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3400cd80 con 0x7fde38100a90 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.758+0000 7fde2effd640 1 -- 172.21.15.1:0/1310360561 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde34005500 con 0x7fde38100a90 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.758+0000 7fde3fcd7640 1 -- 172.21.15.1:0/1310360561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3819d740 con 0x7fde38100a90 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.758+0000 7fde2effd640 1 -- 172.21.15.1:0/1310360561 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34010690 con 0x7fde38100a90 2024-09-16T00:43:54.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.758+0000 7fde3fcd7640 1 -- 172.21.15.1:0/1310360561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde381a5020 con 0x7fde38100a90 2024-09-16T00:43:54.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde420b0740 1 Processor -- start 2024-09-16T00:43:54.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde420b0740 1 -- start start 2024-09-16T00:43:54.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde2effd640 1 -- 172.21.15.1:0/1310360561 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fde340046c0 con 0x7fde38100a90 2024-09-16T00:43:54.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde420b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66640 0x56192ed66a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde420b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66f60 0x56192ed75810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde420b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 0x56192ed78140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde420b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56192eccc2a0 con 0x56192ed66f60 2024-09-16T00:43:54.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde420b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56192ec73b10 con 0x56192ed66640 2024-09-16T00:43:54.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde420b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56192ecc8260 con 0x56192ed75d50 2024-09-16T00:43:54.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.761+0000 7fde3da73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66f60 0x56192ed75810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3e274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66640 0x56192ed66a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3e274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66640 0x56192ed66a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:47282/0 (socket says 172.21.15.1:47282) 2024-09-16T00:43:54.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3e274640 1 -- 172.21.15.1:0/1777523246 learned_addr learned my addr 172.21.15.1:0/1777523246 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:54.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3d272640 1 --2- 172.21.15.1:0/1777523246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 0x56192ed78140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3e274640 1 -- 172.21.15.1:0/1777523246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 msgr2=0x56192ed78140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:54.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3e274640 1 --2- 172.21.15.1:0/1777523246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 0x56192ed78140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3e274640 1 -- 172.21.15.1:0/1777523246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66f60 msgr2=0x56192ed75810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:54.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3e274640 1 --2- 172.21.15.1:0/1777523246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66f60 0x56192ed75810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3e274640 1 -- 172.21.15.1:0/1777523246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192ea3db90 con 0x56192ed66640 2024-09-16T00:43:54.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3d272640 1 --2- 172.21.15.1:0/1777523246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 0x56192ed78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:54.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde3e274640 1 --2- 172.21.15.1:0/1777523246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66640 0x56192ed66a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde30010750 tx=0x7fde30010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e36c0b14070f5a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde2effd640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde240a32c0 0x7fde240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde23fff640 1 -- 172.21.15.1:0/1777523246 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3000ead0 con 0x56192ed66640 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.762+0000 7fde23fff640 1 -- 172.21.15.1:0/1777523246 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde30010e60 con 0x56192ed66640 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde23fff640 1 -- 172.21.15.1:0/1777523246 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30011640 con 0x56192ed66640 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde2effd640 1 -- 172.21.15.1:0/1310360561 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fde340d0110 con 0x7fde38100a90 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde3da73640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde240a32c0 0x7fde240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde420b0740 1 -- 172.21.15.1:0/1777523246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66640 msgr2=0x56192ed66a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde420b0740 1 --2- 172.21.15.1:0/1777523246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66640 0x56192ed66a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde30010750 tx=0x7fde30010c10 comp rx=0 tx=0).stop 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde420b0740 1 -- 172.21.15.1:0/1777523246 shutdown_connections 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde420b0740 1 --2- 172.21.15.1:0/1777523246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 0x56192ed78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde420b0740 1 --2- 172.21.15.1:0/1777523246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66f60 0x56192ed75810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde420b0740 1 --2- 172.21.15.1:0/1777523246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66640 0x56192ed66a20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.763+0000 7fde420b0740 1 -- 172.21.15.1:0/1777523246 >> 172.21.15.1:0/1777523246 conn(0x56192ed62cf0 msgr2=0x56192ed630d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.764+0000 7fde3da73640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde240a32c0 0x7fde240a5780 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fde340cd7b0 tx=0x7fde340d3b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.764+0000 7fde420b0740 1 -- 172.21.15.1:0/1777523246 shutdown_connections 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.764+0000 7fde420b0740 1 -- 172.21.15.1:0/1777523246 wait complete. 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.764+0000 7fde420b0740 1 -- start start 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.764+0000 7fde420b0740 1 Processor -- start 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde420b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed72c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde420b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66f60 0x56192ed73150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde420b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed75d50 0x56192ed73690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde420b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56192ed78920 con 0x56192ed66640 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde420b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56192ec546e0 con 0x56192ed75d50 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde420b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56192ed78aa0 con 0x56192ed66f60 2024-09-16T00:43:54.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde420b0740 10 client.0 _ll_register_callbacks cb 0x56192ec78a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde3e274640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed72c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde3e274640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed72c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50804/0 (socket says 172.21.15.1:50804) 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde3d272640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66f60 0x56192ed73150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde3e274640 1 -- 172.21.15.1:0/911642880 learned_addr learned my addr 172.21.15.1:0/911642880 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde3e274640 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed75d50 0x56192ed73690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde3d272640 1 -- 172.21.15.1:0/911642880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed75d50 msgr2=0x56192ed73690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde3d272640 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed75d50 0x56192ed73690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.765+0000 7fde3d272640 1 -- 172.21.15.1:0/911642880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 msgr2=0x56192ed72c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde3d272640 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed72c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde3d272640 1 -- 172.21.15.1:0/911642880 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192ed81e20 con 0x56192ed66f60 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde3e274640 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed72c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde3d272640 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66f60 0x56192ed73150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde28009900 tx=0x7fde28031860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd7b7e56b75c7dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:54.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde217fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-16T00:43:54.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde217fa640 1 -- 172.21.15.1:0/911642880 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28035070 con 0x56192ed66f60 2024-09-16T00:43:54.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde420b0740 1 -- 172.21.15.1:0/911642880 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde30010e60 con 0x56192ed66f60 2024-09-16T00:43:54.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde217fa640 1 -- 172.21.15.1:0/911642880 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fde28004240 con 0x56192ed66f60 2024-09-16T00:43:54.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde217fa640 1 -- 172.21.15.1:0/911642880 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde2804d050 con 0x56192ed66f60 2024-09-16T00:43:54.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde420b0740 10 client.27532 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:54.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.766+0000 7fde420b0740 1 -- 172.21.15.1:0/911642880 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56192ed84800 con 0x56192ed66f60 2024-09-16T00:43:54.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.767+0000 7fde217fa640 1 -- 172.21.15.1:0/911642880 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde30010e60 con 0x56192ed66f60 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.767+0000 7fde217fa640 1 -- 172.21.15.1:0/911642880 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fde28004a20 con 0x56192ed66f60 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.767+0000 7fde420b0740 10 client.27532 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.767+0000 7fde420b0740 10 client.27532 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.767+0000 7fde420b0740 20 client.27532 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.767+0000 7fde420b0740 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56192ed85c10 0x56192ed88000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.767+0000 7fde420b0740 4 client.27532 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.767+0000 7fde420b0740 1 -- 172.21.15.1:0/911642880 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56192ecd99d0 con 0x56192ed85c10 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.767+0000 7fde3e274640 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56192ed85c10 0x56192ed88000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.768+0000 7fde217fa640 10 client.27532 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.768+0000 7fde3e274640 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56192ed85c10 0x56192ed88000 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc3228c151652b5 server_cookie=e21382b7558b27ee in_seq=0 out_seq=0 2024-09-16T00:43:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.768+0000 7fde217fa640 1 -- 172.21.15.1:0/911642880 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56192ecd99d0 con 0x56192ed85c10 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.768+0000 7fde217fa640 10 client.27532 handle_command_reply: tid=0 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.768+0000 7fde420b0740 1 client.27532 shutdown 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 -- 172.21.15.1:0/911642880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66f60 msgr2=0x56192ed73150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66f60 0x56192ed73150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde28009900 tx=0x7fde28031860 comp rx=0 tx=0).stop 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 -- 172.21.15.1:0/911642880 shutdown_connections 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56192ed85c10 0x56192ed88000 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed75d50 0x56192ed73690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66f60 0x56192ed73150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde217fa640 0 client.27532 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 --2- 172.21.15.1:0/911642880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed72c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 -- 172.21.15.1:0/911642880 >> 172.21.15.1:0/911642880 conn(0x56192ed62cf0 msgr2=0x56192ed68a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 -- 172.21.15.1:0/911642880 shutdown_connections 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 1 -- 172.21.15.1:0/911642880 wait complete. 2024-09-16T00:43:54.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.769+0000 7fde420b0740 20 client.27532 trim_cache size 0 max 16384 2024-09-16T00:43:54.780 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:54.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde420b0740 1 Processor -- start 2024-09-16T00:43:54.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde420b0740 1 -- start start 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde420b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed61d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde420b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed8d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde420b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 0x56192ed8ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde420b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde381af580 con 0x56192ed66640 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde420b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde381ace50 con 0x56192ed66f60 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde420b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde381aaca0 con 0x56192ed75d50 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3e274640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed61d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3e274640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed61d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50814/0 (socket says 172.21.15.1:50814) 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3d272640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed8d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3e274640 1 -- 172.21.15.1:0/1198290227 learned_addr learned my addr 172.21.15.1:0/1198290227 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:54.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3e274640 1 --2- 172.21.15.1:0/1198290227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 0x56192ed8ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3d272640 1 -- 172.21.15.1:0/1198290227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 msgr2=0x56192ed8ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3d272640 1 --2- 172.21.15.1:0/1198290227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 0x56192ed8ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3d272640 1 -- 172.21.15.1:0/1198290227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 msgr2=0x56192ed61d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3d272640 1 --2- 172.21.15.1:0/1198290227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed61d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3d272640 1 -- 172.21.15.1:0/1198290227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192ed5e260 con 0x56192ed66f60 2024-09-16T00:43:54.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.788+0000 7fde3e274640 1 --2- 172.21.15.1:0/1198290227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed61d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:54.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.789+0000 7fde3d272640 1 --2- 172.21.15.1:0/1198290227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed8d890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fde28009930 tx=0x7fde280317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52130ecf7a17d144 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.789+0000 7fde237fe640 1 -- 172.21.15.1:0/1198290227 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28000e60 con 0x56192ed66f60 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.789+0000 7fde237fe640 1 -- 172.21.15.1:0/1198290227 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde28005000 con 0x56192ed66f60 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.789+0000 7fde237fe640 1 -- 172.21.15.1:0/1198290227 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28035070 con 0x56192ed66f60 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 -- 172.21.15.1:0/1198290227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 msgr2=0x56192ed8d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 --2- 172.21.15.1:0/1198290227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed8d890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fde28009930 tx=0x7fde280317c0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 -- 172.21.15.1:0/1198290227 shutdown_connections 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 --2- 172.21.15.1:0/1198290227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed75d50 0x56192ed8ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 --2- 172.21.15.1:0/1198290227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed8d890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 --2- 172.21.15.1:0/1198290227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed66640 0x56192ed61d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 -- 172.21.15.1:0/1198290227 >> 172.21.15.1:0/1198290227 conn(0x56192ed62cf0 msgr2=0x56192ed61650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 -- 172.21.15.1:0/1198290227 shutdown_connections 2024-09-16T00:43:54.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 -- 172.21.15.1:0/1198290227 wait complete. 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 -- start start 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.790+0000 7fde420b0740 1 Processor -- start 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde420b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66640 0x56192ed7a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde420b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed7a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde420b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed75d50 0x56192edef860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde420b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde381ab310 con 0x56192ed75d50 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde420b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde381ac780 con 0x56192ed66f60 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde420b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde381ab9e0 con 0x56192ed66640 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde420b0740 10 client.0 _ll_register_callbacks cb 0x7fde28005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3d272640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed7a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3e274640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed75d50 0x56192edef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3d272640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed7a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:47304/0 (socket says 172.21.15.1:47304) 2024-09-16T00:43:54.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3d272640 1 -- 172.21.15.1:0/3841157921 learned_addr learned my addr 172.21.15.1:0/3841157921 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3e274640 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed75d50 0x56192edef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50820/0 (socket says 172.21.15.1:50820) 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3e274640 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66640 0x56192ed7a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3d272640 1 -- 172.21.15.1:0/3841157921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66640 msgr2=0x56192ed7a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3d272640 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66640 0x56192ed7a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3d272640 1 -- 172.21.15.1:0/3841157921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed75d50 msgr2=0x56192edef860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3d272640 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed75d50 0x56192edef860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3d272640 1 -- 172.21.15.1:0/3841157921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192edefda0 con 0x56192ed66f60 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.791+0000 7fde3e274640 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed75d50 0x56192edef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde3d272640 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed7a650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde2802a460 tx=0x7fde280053c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348d3bd176703843 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde217fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde217fa640 1 -- 172.21.15.1:0/3841157921 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28035070 con 0x56192ed66f60 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde420b0740 1 -- 172.21.15.1:0/3841157921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde28005000 con 0x56192ed66f60 2024-09-16T00:43:54.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde217fa640 1 -- 172.21.15.1:0/3841157921 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fde280055b0 con 0x56192ed66f60 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde217fa640 1 -- 172.21.15.1:0/3841157921 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde28064050 con 0x56192ed66f60 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde420b0740 10 client.27875 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde420b0740 1 -- 172.21.15.1:0/3841157921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56192edf0fc0 con 0x56192ed66f60 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde217fa640 1 -- 172.21.15.1:0/3841157921 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde28005000 con 0x56192ed66f60 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde217fa640 1 -- 172.21.15.1:0/3841157921 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fde28059020 con 0x56192ed66f60 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde420b0740 10 client.27875 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.792+0000 7fde420b0740 10 client.27875 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.793+0000 7fde420b0740 20 client.27875 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.793+0000 7fde420b0740 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56192edf16c0 0x56192edf3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.793+0000 7fde420b0740 4 client.27875 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-16T00:43:54.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.793+0000 7fde420b0740 1 -- 172.21.15.1:0/3841157921 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x56192ecd3d20 con 0x56192edf16c0 2024-09-16T00:43:54.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.793+0000 7fde3e274640 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56192edf16c0 0x56192edf3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:54.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.793+0000 7fde3e274640 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56192edf16c0 0x56192edf3ae0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d656fa70774f625e server_cookie=cdf302fee2cabe74 in_seq=0 out_seq=0 2024-09-16T00:43:54.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.793+0000 7fde217fa640 10 client.27875 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:54.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde217fa640 1 -- 172.21.15.1:0/3841157921 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56192ecd3d20 con 0x56192edf16c0 2024-09-16T00:43:54.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde217fa640 10 client.27875 handle_command_reply: tid=0 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 client.27875 shutdown 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 -- 172.21.15.1:0/3841157921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 msgr2=0x56192ed7a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed7a650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde2802a460 tx=0x7fde280053c0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 -- 172.21.15.1:0/3841157921 shutdown_connections 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56192edf16c0 0x56192edf3ae0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56192ed75d50 0x56192edef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192ed66f60 0x56192ed7a650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 --2- 172.21.15.1:0/3841157921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56192ed66640 0x56192ed7a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde217fa640 0 client.27875 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 -- 172.21.15.1:0/3841157921 >> 172.21.15.1:0/3841157921 conn(0x56192ed62cf0 msgr2=0x56192ed733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 -- 172.21.15.1:0/3841157921 shutdown_connections 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 1 -- 172.21.15.1:0/3841157921 wait complete. 2024-09-16T00:43:54.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.803+0000 7fde420b0740 20 client.27875 trim_cache size 0 max 16384 2024-09-16T00:43:54.805 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:43:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:43:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:43:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 54, 2024-09-16T00:43:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:43:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "9bedaf13": { 2024-09-16T00:43:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 54, 2024-09-16T00:43:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 137.7, 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:54.807 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 -- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde240a32c0 msgr2=0x7fde240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde240a32c0 0x7fde240a5780 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fde340cd7b0 tx=0x7fde340d3b50 comp rx=0 tx=0).stop 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 -- 172.21.15.1:0/1310360561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde38100a90 msgr2=0x7fde381984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde38100a90 0x7fde381984a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fde3400e9f0 tx=0x7fde3400edf0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 -- 172.21.15.1:0/1310360561 shutdown_connections 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde38106a90 0x7fde3819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde240a32c0 0x7fde240a5780 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde38101430 0x7fde381989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 --2- 172.21.15.1:0/1310360561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde38100a90 0x7fde381984a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.804+0000 7fde3fcd7640 1 -- 172.21.15.1:0/1310360561 >> 172.21.15.1:0/1310360561 conn(0x7fde380fc7d0 msgr2=0x7fde3806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.805+0000 7fde3fcd7640 1 -- 172.21.15.1:0/1310360561 shutdown_connections 2024-09-16T00:43:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:54.805+0000 7fde3fcd7640 1 -- 172.21.15.1:0/1310360561 wait complete. 2024-09-16T00:43:55.016 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:54 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:43:55.016 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:54 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:55.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:54 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:43:55.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:54 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:55.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:54 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:43:55.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:54 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:55.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:54 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:43:55.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:54 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:55.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:54 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:43:55.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:54 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:55.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:54 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:43:55.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:54 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:55.830 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9bedaf13', quiesce took 5.4 seconds. Will release after: 12.299999999999999 2024-09-16T00:43:55.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:55 smithi001 ceph-mon[30934]: pgmap v816: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.072+0000 7fbd7542d640 1 -- 172.21.15.1:0/2451255967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 msgr2=0x7fbd7010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.072+0000 7fbd7542d640 1 --2- 172.21.15.1:0/2451255967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 0x7fbd7010d910 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009ca0 tx=0x7fbd6802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.073+0000 7fbd7542d640 1 -- 172.21.15.1:0/2451255967 shutdown_connections 2024-09-16T00:43:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.073+0000 7fbd7542d640 1 --2- 172.21.15.1:0/2451255967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 0x7fbd7010d910 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.073+0000 7fbd7542d640 1 --2- 172.21.15.1:0/2451255967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70101240 0x7fbd70101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.073+0000 7fbd7542d640 1 --2- 172.21.15.1:0/2451255967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd70105a90 0x7fbd70105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.073+0000 7fbd7542d640 1 -- 172.21.15.1:0/2451255967 >> 172.21.15.1:0/2451255967 conn(0x7fbd70077210 msgr2=0x7fbd700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.073+0000 7fbd7542d640 1 -- 172.21.15.1:0/2451255967 shutdown_connections 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.073+0000 7fbd7542d640 1 -- 172.21.15.1:0/2451255967 wait complete. 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.073+0000 7fbd7542d640 1 Processor -- start 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.073+0000 7fbd7542d640 1 -- start start 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd7542d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70101240 0x7fbd70198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd7542d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 0x7fbd701989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd7542d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd70105a90 0x7fbd7019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd7542d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70112090 con 0x7fbd70101bc0 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd7542d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70111f10 con 0x7fbd70105a90 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd7542d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70112210 con 0x7fbd70101240 2024-09-16T00:43:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 0x7fbd701989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 0x7fbd701989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50850/0 (socket says 172.21.15.1:50850) 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6e7fc640 1 -- 172.21.15.1:0/989381738 learned_addr learned my addr 172.21.15.1:0/989381738 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6effd640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70101240 0x7fbd70198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd70105a90 0x7fbd7019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6e7fc640 1 -- 172.21.15.1:0/989381738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70101240 msgr2=0x7fbd70198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70101240 0x7fbd70198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6e7fc640 1 -- 172.21.15.1:0/989381738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd70105a90 msgr2=0x7fbd7019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd70105a90 0x7fbd7019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6e7fc640 1 -- 172.21.15.1:0/989381738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7019d440 con 0x7fbd70101bc0 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.074+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 0x7fbd701989d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400d770 tx=0x7fbd6400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.075+0000 7fbd57fff640 1 -- 172.21.15.1:0/989381738 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64017070 con 0x7fbd70101bc0 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.075+0000 7fbd57fff640 1 -- 172.21.15.1:0/989381738 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd64005500 con 0x7fbd70101bc0 2024-09-16T00:43:56.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.075+0000 7fbd7542d640 1 -- 172.21.15.1:0/989381738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7019d730 con 0x7fbd70101bc0 2024-09-16T00:43:56.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.075+0000 7fbd57fff640 1 -- 172.21.15.1:0/989381738 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64004e00 con 0x7fbd70101bc0 2024-09-16T00:43:56.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.075+0000 7fbd7542d640 1 -- 172.21.15.1:0/989381738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd701a5010 con 0x7fbd70101bc0 2024-09-16T00:43:56.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.077+0000 7fbd57fff640 1 -- 172.21.15.1:0/989381738 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fbd64004fa0 con 0x7fbd70101bc0 2024-09-16T00:43:56.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.077+0000 7fbd77806740 1 Processor -- start 2024-09-16T00:43:56.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd77806740 1 -- start start 2024-09-16T00:43:56.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd77806740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 0x55eebe5b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd57fff640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd580a32c0 0x7fbd580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd77806740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b6dc0 0x55eebe5c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd77806740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5c1680 0x55eebe5c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd77806740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eebe51c2e0 con 0x55eebe5c1680 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd77806740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eebe4c3ce0 con 0x55eebe5b6dc0 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd77806740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eebe5182a0 con 0x55eebe5b64a0 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5c1680 0x55eebe5c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5c1680 0x55eebe5c3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50866/0 (socket says 172.21.15.1:50866) 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6f7fe640 1 -- 172.21.15.1:0/2937090407 learned_addr learned my addr 172.21.15.1:0/2937090407 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd57fff640 1 -- 172.21.15.1:0/989381738 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbd640cf690 con 0x7fbd70101bc0 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/2937090407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 0x55eebe5b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/2937090407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b6dc0 0x55eebe5c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6effd640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd580a32c0 0x7fbd580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6f7fe640 1 -- 172.21.15.1:0/2937090407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 msgr2=0x55eebe5b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:56.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/2937090407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 0x55eebe5b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6f7fe640 1 -- 172.21.15.1:0/2937090407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b6dc0 msgr2=0x55eebe5c1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:56.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/2937090407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b6dc0 0x55eebe5c1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.078+0000 7fbd6f7fe640 1 -- 172.21.15.1:0/2937090407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eebe4a8440 con 0x55eebe5c1680 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/2937090407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5c1680 0x55eebe5c3a70 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fbd68035000 tx=0x7fbd680093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc0d54772e1fdc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd6effd640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd580a32c0 0x7fbd580a5780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbd60005d00 tx=0x7fbd6000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd557fa640 1 -- 172.21.15.1:0/2937090407 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003970 con 0x55eebe5c1680 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd557fa640 1 -- 172.21.15.1:0/2937090407 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd68036070 con 0x55eebe5c1680 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd557fa640 1 -- 172.21.15.1:0/2937090407 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68004250 con 0x55eebe5c1680 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd77806740 1 -- 172.21.15.1:0/2937090407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5c1680 msgr2=0x55eebe5c3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd77806740 1 --2- 172.21.15.1:0/2937090407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5c1680 0x55eebe5c3a70 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fbd68035000 tx=0x7fbd680093d0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd77806740 1 -- 172.21.15.1:0/2937090407 shutdown_connections 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd77806740 1 --2- 172.21.15.1:0/2937090407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5c1680 0x55eebe5c3a70 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd77806740 1 --2- 172.21.15.1:0/2937090407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b6dc0 0x55eebe5c1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.079+0000 7fbd77806740 1 --2- 172.21.15.1:0/2937090407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 0x55eebe5b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 -- 172.21.15.1:0/2937090407 >> 172.21.15.1:0/2937090407 conn(0x55eebe5b2a30 msgr2=0x55eebe5b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:56.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 -- 172.21.15.1:0/2937090407 shutdown_connections 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 -- 172.21.15.1:0/2937090407 wait complete. 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 -- start start 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 Processor -- start 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b64a0 0x55eebe5c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 0x55eebe5be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5c1680 0x55eebe5be780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eebe5c40d0 con 0x55eebe5b64a0 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eebe4d7050 con 0x55eebe5c1680 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eebe5c4250 con 0x55eebe5b6dc0 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.080+0000 7fbd77806740 10 client.0 _ll_register_callbacks cb 0x55eebe517160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b64a0 0x55eebe5c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b64a0 0x55eebe5c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50868/0 (socket says 172.21.15.1:50868) 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6effd640 1 -- 172.21.15.1:0/17048916 learned_addr learned my addr 172.21.15.1:0/17048916 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 0x55eebe5be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5c1680 0x55eebe5be780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6effd640 1 -- 172.21.15.1:0/17048916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 msgr2=0x55eebe5be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6effd640 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 0x55eebe5be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6effd640 1 -- 172.21.15.1:0/17048916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5c1680 msgr2=0x55eebe5be780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6effd640 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5c1680 0x55eebe5be780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6effd640 1 -- 172.21.15.1:0/17048916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eebe28d3c0 con 0x55eebe5b64a0 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5c1680 0x55eebe5be780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 0x55eebe5be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd6effd640 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b64a0 0x55eebe5c0f90 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fbd600116c0 tx=0x7fbd60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69300ee9ffdfb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd52ff5640 1 -- 172.21.15.1:0/17048916 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6001e8f0 con 0x55eebe5b64a0 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd52ff5640 1 -- 172.21.15.1:0/17048916 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbd6001ea90 con 0x55eebe5b64a0 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd77806740 1 -- 172.21.15.1:0/17048916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd68036070 con 0x55eebe5b64a0 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.081+0000 7fbd52ff5640 1 -- 172.21.15.1:0/17048916 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd6002f050 con 0x55eebe5b64a0 2024-09-16T00:43:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd77806740 10 client.19077 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd77806740 1 -- 172.21.15.1:0/17048916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eebe5cfb00 con 0x55eebe5b64a0 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd52ff5640 1 -- 172.21.15.1:0/17048916 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd68036070 con 0x55eebe5b64a0 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd52ff5640 1 -- 172.21.15.1:0/17048916 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbd6001d3b0 con 0x55eebe5b64a0 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd77806740 10 client.19077 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd77806740 10 client.19077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd77806740 20 client.19077 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd77806740 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55eebe5d0fc0 0x55eebe5d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd77806740 4 client.19077 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd77806740 1 -- 172.21.15.1:0/17048916 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eebe529850 con 0x55eebe5d0fc0 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.082+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55eebe5d0fc0 0x55eebe5d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.083+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55eebe5d0fc0 0x55eebe5d33d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88f0f490e315c3 server_cookie=5c7e082f3aa011b4 in_seq=0 out_seq=0 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.083+0000 7fbd52ff5640 10 client.19077 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.083+0000 7fbd52ff5640 1 -- 172.21.15.1:0/17048916 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eebe529850 con 0x55eebe5d0fc0 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.083+0000 7fbd52ff5640 10 client.19077 handle_command_reply: tid=0 2024-09-16T00:43:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.083+0000 7fbd77806740 1 client.19077 shutdown 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.083+0000 7fbd77806740 1 -- 172.21.15.1:0/17048916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b64a0 msgr2=0x55eebe5c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.083+0000 7fbd77806740 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b64a0 0x55eebe5c0f90 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fbd600116c0 tx=0x7fbd60004730 comp rx=0 tx=0).stop 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd77806740 1 -- 172.21.15.1:0/17048916 shutdown_connections 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd77806740 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55eebe5d0fc0 0x55eebe5d33d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd77806740 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5c1680 0x55eebe5be780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd77806740 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 0x55eebe5be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd77806740 1 --2- 172.21.15.1:0/17048916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b64a0 0x55eebe5c0f90 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd77806740 1 -- 172.21.15.1:0/17048916 >> 172.21.15.1:0/17048916 conn(0x55eebe5b2a30 msgr2=0x55eebe5b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd52ff5640 0 client.19077 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd77806740 1 -- 172.21.15.1:0/17048916 shutdown_connections 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd77806740 1 -- 172.21.15.1:0/17048916 wait complete. 2024-09-16T00:43:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.084+0000 7fbd77806740 20 client.19077 trim_cache size 0 max 16384 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.101+0000 7fbd77806740 1 Processor -- start 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.101+0000 7fbd77806740 1 -- start start 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd77806740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b2a30 0x55eebe5cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd77806740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 0x55eebe5d8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd77806740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b6dc0 0x55eebe5d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd77806740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbd701b1690 con 0x55eebe5b6dc0 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd77806740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd701aef60 con 0x55eebe5b2a30 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd77806740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd701acdb0 con 0x55eebe5b64a0 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b6dc0 0x55eebe5d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b6dc0 0x55eebe5d91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50878/0 (socket says 172.21.15.1:50878) 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6f7fe640 1 -- 172.21.15.1:0/335104977 learned_addr learned my addr 172.21.15.1:0/335104977 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6effd640 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b2a30 0x55eebe5cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 0x55eebe5d8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6f7fe640 1 -- 172.21.15.1:0/335104977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 msgr2=0x55eebe5d8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 0x55eebe5d8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6f7fe640 1 -- 172.21.15.1:0/335104977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b2a30 msgr2=0x55eebe5cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b2a30 0x55eebe5cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6f7fe640 1 -- 172.21.15.1:0/335104977 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eebe5b24d0 con 0x55eebe5b6dc0 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 0x55eebe5d8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b6dc0 0x55eebe5d91e0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fbd68035000 tx=0x7fbd6802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0ebf72863d9ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.102+0000 7fbd54ff9640 1 -- 172.21.15.1:0/335104977 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68004d40 con 0x55eebe5b6dc0 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd54ff9640 1 -- 172.21.15.1:0/335104977 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6804f030 con 0x55eebe5b6dc0 2024-09-16T00:43:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd54ff9640 1 -- 172.21.15.1:0/335104977 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6802fc70 con 0x55eebe5b6dc0 2024-09-16T00:43:56.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd77806740 1 -- 172.21.15.1:0/335104977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b6dc0 msgr2=0x55eebe5d91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:56.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd77806740 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b6dc0 0x55eebe5d91e0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fbd68035000 tx=0x7fbd6802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd77806740 1 -- 172.21.15.1:0/335104977 shutdown_connections 2024-09-16T00:43:56.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd77806740 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b6dc0 0x55eebe5d91e0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd77806740 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b64a0 0x55eebe5d8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd77806740 1 --2- 172.21.15.1:0/335104977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b2a30 0x55eebe5cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd77806740 1 -- 172.21.15.1:0/335104977 >> 172.21.15.1:0/335104977 conn(0x55eebe5be8e0 msgr2=0x55eebe5bcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd77806740 1 -- 172.21.15.1:0/335104977 shutdown_connections 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.103+0000 7fbd77806740 1 -- 172.21.15.1:0/335104977 wait complete. 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd77806740 1 -- start start 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd77806740 1 Processor -- start 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd77806740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b2a30 0x55eebe5c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd77806740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b64a0 0x55eebe5c5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd77806740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 0x55eebe5c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd77806740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbd701ad420 con 0x55eebe5b2a30 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd77806740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd701ae890 con 0x55eebe5b64a0 2024-09-16T00:43:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd77806740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd701adaf0 con 0x55eebe5b6dc0 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd77806740 10 client.0 _ll_register_callbacks cb 0x7fbd6000b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd6effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b2a30 0x55eebe5c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd6effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b2a30 0x55eebe5c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50888/0 (socket says 172.21.15.1:50888) 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd6effd640 1 -- 172.21.15.1:0/2871652914 learned_addr learned my addr 172.21.15.1:0/2871652914 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b64a0 0x55eebe5c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.104+0000 7fbd6effd640 1 -- 172.21.15.1:0/2871652914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 msgr2=0x55eebe5c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd6f7fe640 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 0x55eebe5c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd6effd640 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 0x55eebe5c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd6effd640 1 -- 172.21.15.1:0/2871652914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b64a0 msgr2=0x55eebe5c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd6effd640 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b64a0 0x55eebe5c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd6effd640 1 -- 172.21.15.1:0/2871652914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eebe63acc0 con 0x55eebe5b2a30 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b64a0 0x55eebe5c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd6effd640 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b2a30 0x55eebe5c5230 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fbd600116c0 tx=0x7fbd60010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2f6bc155da8d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd52ff5640 1 -- 172.21.15.1:0/2871652914 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6001dbb0 con 0x55eebe5b2a30 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd52ff5640 1 -- 172.21.15.1:0/2871652914 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbd6001dd50 con 0x55eebe5b2a30 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd77806740 1 -- 172.21.15.1:0/2871652914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6804f030 con 0x55eebe5b2a30 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd52ff5640 1 -- 172.21.15.1:0/2871652914 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd6001e6f0 con 0x55eebe5b2a30 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd77806740 10 client.19083 fetch_fsmap learned FSMap version 25 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd77806740 1 -- 172.21.15.1:0/2871652914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eebe63bf00 con 0x55eebe5b2a30 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.105+0000 7fbd52ff5640 1 -- 172.21.15.1:0/2871652914 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6804f030 con 0x55eebe5b2a30 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd52ff5640 1 -- 172.21.15.1:0/2871652914 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbd6003b070 con 0x55eebe5b2a30 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd77806740 10 client.19083 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd77806740 10 client.19083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd77806740 20 client.19083 populate_metadata read hostname 'smithi001' 2024-09-16T00:43:56.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd77806740 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55eebe5c1680 0x55eebe63e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd77806740 4 client.19083 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd77806740 1 -- 172.21.15.1:0/2871652914 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eebe523d60 con 0x55eebe5c1680 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55eebe5c1680 0x55eebe63e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd52ff5640 10 client.19083 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd6e7fc640 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55eebe5c1680 0x55eebe63e700 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d8db22dcd4a86b server_cookie=859bf32c6c8664f5 in_seq=0 out_seq=0 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd52ff5640 1 -- 172.21.15.1:0/2871652914 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eebe523d60 con 0x55eebe5c1680 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd52ff5640 10 client.19083 handle_command_reply: tid=0 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.106+0000 7fbd77806740 1 client.19083 shutdown 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 -- 172.21.15.1:0/2871652914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b2a30 msgr2=0x55eebe5c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b2a30 0x55eebe5c5230 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fbd600116c0 tx=0x7fbd60010dc0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 -- 172.21.15.1:0/2871652914 shutdown_connections 2024-09-16T00:43:56.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55eebe5c1680 0x55eebe63e700 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eebe5b6dc0 0x55eebe5c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebe5b64a0 0x55eebe5c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 --2- 172.21.15.1:0/2871652914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eebe5b2a30 0x55eebe5c5230 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd52ff5640 0 client.19083 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 -- 172.21.15.1:0/2871652914 >> 172.21.15.1:0/2871652914 conn(0x55eebe5be8e0 msgr2=0x55eebe5b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 -- 172.21.15.1:0/2871652914 shutdown_connections 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 1 -- 172.21.15.1:0/2871652914 wait complete. 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd77806740 20 client.19083 trim_cache size 0 max 16384 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:43:56.114 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:43:56.115 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:43:56.115 INFO:teuthology.orchestra.run.smithi001.stdout: "11cedabf-c707-48cf-89b2-9939ee6f623c": { 2024-09-16T00:43:56.115 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:43:56.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "11cedabf-c707-48cf-89b2-9939ee6f623c", 2024-09-16T00:43:56.115 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:43:56.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:56.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:43:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd7542d640 1 -- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd580a32c0 msgr2=0x7fbd580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd7542d640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd580a32c0 0x7fbd580a5780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbd60005d00 tx=0x7fbd6000b410 comp rx=0 tx=0).stop 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd7542d640 1 -- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 msgr2=0x7fbd701989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd7542d640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 0x7fbd701989d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400d770 tx=0x7fbd6400dc30 comp rx=0 tx=0).stop 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.107+0000 7fbd7542d640 1 -- 172.21.15.1:0/989381738 shutdown_connections 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.108+0000 7fbd7542d640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd70105a90 0x7fbd7019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.108+0000 7fbd7542d640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd70101bc0 0x7fbd701989d0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.108+0000 7fbd7542d640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbd580a32c0 0x7fbd580a5780 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.108+0000 7fbd7542d640 1 --2- 172.21.15.1:0/989381738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70101240 0x7fbd70198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.108+0000 7fbd7542d640 1 -- 172.21.15.1:0/989381738 >> 172.21.15.1:0/989381738 conn(0x7fbd70077210 msgr2=0x7fbd701039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.108+0000 7fbd7542d640 1 -- 172.21.15.1:0/989381738 shutdown_connections 2024-09-16T00:43:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:43:56.108+0000 7fbd7542d640 1 -- 172.21.15.1:0/989381738 wait complete. 2024-09-16T00:43:56.117 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:55 smithi019 ceph-mon[35186]: pgmap v816: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:56.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:55 smithi153 ceph-mon[35538]: pgmap v816: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:43:56.900 INFO:tasks.cephfs.filesystem:scrub status for tag:11cedabf-c707-48cf-89b2-9939ee6f623c - {'path': '/', 'tag': '11cedabf-c707-48cf-89b2-9939ee6f623c', 'options': 'recursive,force'} 2024-09-16T00:43:58.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:57 smithi019 ceph-mon[35186]: pgmap v817: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:58.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:57 smithi001 ceph-mon[30934]: pgmap v817: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:58.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:57 smithi153 ceph-mon[35538]: pgmap v817: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-16T00:43:59.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:58 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:43:59.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:58 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:43:59.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:58 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:44:00.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:43:59 smithi001 ceph-mon[30934]: pgmap v818: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:44:00.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:43:59 smithi153 ceph-mon[35538]: pgmap v818: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:44:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:43:59 smithi019 ceph-mon[35186]: pgmap v818: 129 pgs: 129 active+clean; 7.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-16T00:44:02.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:01 smithi001 ceph-mon[30934]: pgmap v819: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:44:02.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:01 smithi153 ceph-mon[35538]: pgmap v819: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:44:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:01 smithi019 ceph-mon[35186]: pgmap v819: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-16T00:44:04.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:03 smithi001 ceph-mon[30934]: pgmap v820: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-16T00:44:04.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:03 smithi153 ceph-mon[35538]: pgmap v820: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-16T00:44:04.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:03 smithi019 ceph-mon[35186]: pgmap v820: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-16T00:44:06.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:05 smithi001 ceph-mon[30934]: pgmap v821: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-16T00:44:06.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:05 smithi153 ceph-mon[35538]: pgmap v821: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-16T00:44:06.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:05 smithi019 ceph-mon[35186]: pgmap v821: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-16T00:44:08.132 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:44:08.161 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:07 smithi001 ceph-mon[30934]: pgmap v822: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-16T00:44:08.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:07 smithi153 ceph-mon[35538]: pgmap v822: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-16T00:44:08.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:07 smithi019 ceph-mon[35186]: pgmap v822: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-16T00:44:08.405 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:08.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.923+0000 7f73d2f48640 1 -- 172.21.15.1:0/2591599216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106a80 msgr2=0x7f73cc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.923+0000 7f73d2f48640 1 --2- 172.21.15.1:0/2591599216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106a80 0x7f73cc106e60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009930 tx=0x7f73c002a420 comp rx=0 tx=0).stop 2024-09-16T00:44:08.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.923+0000 7f73d2f48640 1 -- 172.21.15.1:0/2591599216 shutdown_connections 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.923+0000 7f73d2f48640 1 --2- 172.21.15.1:0/2591599216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73cc101420 0x7f73cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.923+0000 7f73d2f48640 1 --2- 172.21.15.1:0/2591599216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73cc100a80 0x7f73cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.923+0000 7f73d2f48640 1 --2- 172.21.15.1:0/2591599216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106a80 0x7f73cc106e60 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.923+0000 7f73d2f48640 1 -- 172.21.15.1:0/2591599216 >> 172.21.15.1:0/2591599216 conn(0x7f73cc0fc7a0 msgr2=0x7f73cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.923+0000 7f73d2f48640 1 -- 172.21.15.1:0/2591599216 shutdown_connections 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.923+0000 7f73d2f48640 1 -- 172.21.15.1:0/2591599216 wait complete. 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73d2f48640 1 Processor -- start 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73d2f48640 1 -- start start 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73d2f48640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73cc100a80 0x7f73cc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73d2f48640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc101420 0x7f73cc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73d2f48640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73cc106a80 0x7f73cc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73d2f48640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc110ea0 con 0x7f73cc101420 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73d2f48640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc110d20 con 0x7f73cc106a80 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73d2f48640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc111020 con 0x7f73cc100a80 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc101420 0x7f73cc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc101420 0x7f73cc1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45438/0 (socket says 172.21.15.1:45438) 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.924+0000 7f73cbfff640 1 -- 172.21.15.1:0/1269721606 learned_addr learned my addr 172.21.15.1:0/1269721606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73d14e5640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73cc106a80 0x7f73cc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73d0ce4640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73cc100a80 0x7f73cc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73cbfff640 1 -- 172.21.15.1:0/1269721606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73cc100a80 msgr2=0x7f73cc198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73cbfff640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73cc100a80 0x7f73cc198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73cbfff640 1 -- 172.21.15.1:0/1269721606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73cc106a80 msgr2=0x7f73cc19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73cbfff640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73cc106a80 0x7f73cc19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73cbfff640 1 -- 172.21.15.1:0/1269721606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc19d430 con 0x7f73cc101420 2024-09-16T00:44:08.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73d0ce4640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73cc100a80 0x7f73cc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73cbfff640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc101420 0x7f73cc1989c0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f73b800d950 tx=0x7f73b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73c9ffb640 1 -- 172.21.15.1:0/1269721606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8017070 con 0x7f73cc101420 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73c9ffb640 1 -- 172.21.15.1:0/1269721606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73b8009c30 con 0x7f73cc101420 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc19d720 con 0x7f73cc101420 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73c9ffb640 1 -- 172.21.15.1:0/1269721606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b800f6b0 con 0x7f73cc101420 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.925+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc1a5000 con 0x7f73cc101420 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.926+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73cc100ee0 con 0x7f73cc101420 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.927+0000 7f73c9ffb640 1 -- 172.21.15.1:0/1269721606 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f73b80109d0 con 0x7f73cc101420 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.928+0000 7f73c9ffb640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73bc0a3240 0x7f73bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.928+0000 7f73d0ce4640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73bc0a3240 0x7f73bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.928+0000 7f73c9ffb640 1 -- 172.21.15.1:0/1269721606 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f73b80cfa50 con 0x7f73cc101420 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.929+0000 7f73d0ce4640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73bc0a3240 0x7f73bc0a5700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f73c0000c00 tx=0x7f73c0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:08.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:08.933+0000 7f73c9ffb640 1 -- 172.21.15.1:0/1269721606 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73b8009de0 con 0x7f73cc101420 2024-09-16T00:44:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.144+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73cc02d7c0 con 0x7f73cc101420 2024-09-16T00:44:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.145+0000 7f73c9ffb640 1 -- 172.21.15.1:0/1269721606 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f73b809c370 con 0x7f73cc101420 2024-09-16T00:44:09.146 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:44:09.146 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:44:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:44:09.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73bc0a3240 msgr2=0x7f73bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:09.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73bc0a3240 0x7f73bc0a5700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f73c0000c00 tx=0x7f73c0032c70 comp rx=0 tx=0).stop 2024-09-16T00:44:09.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc101420 msgr2=0x7f73cc1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:09.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc101420 0x7f73cc1989c0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f73b800d950 tx=0x7f73b800de10 comp rx=0 tx=0).stop 2024-09-16T00:44:09.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 shutdown_connections 2024-09-16T00:44:09.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73cc106a80 0x7f73cc19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:09.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc101420 0x7f73cc1989c0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:09.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f73bc0a3240 0x7f73bc0a5700 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:09.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 --2- 172.21.15.1:0/1269721606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73cc100a80 0x7f73cc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:09.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 >> 172.21.15.1:0/1269721606 conn(0x7f73cc0fc7a0 msgr2=0x7f73cc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:09.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 shutdown_connections 2024-09-16T00:44:09.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:09.147+0000 7f73d2f48640 1 -- 172.21.15.1:0/1269721606 wait complete. 2024-09-16T00:44:09.843 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 9bedaf13 --release --await' 2024-09-16T00:44:09.843 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 9bedaf13 --release --await 2024-09-16T00:44:10.142 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:10.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:09 smithi001 ceph-mon[30934]: pgmap v823: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T00:44:10.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:09 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1269721606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:10.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:09 smithi153 ceph-mon[35538]: pgmap v823: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T00:44:10.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:09 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1269721606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:09 smithi019 ceph-mon[35186]: pgmap v823: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T00:44:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:09 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1269721606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:10.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 -- 172.21.15.1:0/1721656702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c106a80 msgr2=0x7fa95c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 --2- 172.21.15.1:0/1721656702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c106a80 0x7fa95c106e60 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009930 tx=0x7fa94c02a420 comp rx=0 tx=0).stop 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 -- 172.21.15.1:0/1721656702 shutdown_connections 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 --2- 172.21.15.1:0/1721656702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa95c101420 0x7fa95c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 --2- 172.21.15.1:0/1721656702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c100a80 0x7fa95c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 --2- 172.21.15.1:0/1721656702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c106a80 0x7fa95c106e60 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 -- 172.21.15.1:0/1721656702 >> 172.21.15.1:0/1721656702 conn(0x7fa95c0fc7a0 msgr2=0x7fa95c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 -- 172.21.15.1:0/1721656702 shutdown_connections 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 -- 172.21.15.1:0/1721656702 wait complete. 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 Processor -- start 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.639+0000 7fa96223f640 1 -- start start 2024-09-16T00:44:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa96223f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c100a80 0x7fa95c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa96223f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c101420 0x7fa95c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa96223f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa95c106a80 0x7fa95c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa96223f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c110ea0 con 0x7fa95c100a80 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa96223f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c110d20 con 0x7fa95c101420 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa96223f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c111020 con 0x7fa95c106a80 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c100a80 0x7fa95c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c100a80 0x7fa95c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38598/0 (socket says 172.21.15.1:38598) 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95b7fe640 1 -- 172.21.15.1:0/956647694 learned_addr learned my addr 172.21.15.1:0/956647694 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95affd640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c101420 0x7fa95c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95bfff640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa95c106a80 0x7fa95c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95b7fe640 1 -- 172.21.15.1:0/956647694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa95c106a80 msgr2=0x7fa95c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa95c106a80 0x7fa95c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95b7fe640 1 -- 172.21.15.1:0/956647694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c101420 msgr2=0x7fa95c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c101420 0x7fa95c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95b7fe640 1 -- 172.21.15.1:0/956647694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c19d450 con 0x7fa95c100a80 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.640+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c100a80 0x7fa95c1984a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009230 tx=0x7fa94c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.641+0000 7fa958ff9640 1 -- 172.21.15.1:0/956647694 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c03f030 con 0x7fa95c100a80 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.641+0000 7fa958ff9640 1 -- 172.21.15.1:0/956647694 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94c003ad0 con 0x7fa95c100a80 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.641+0000 7fa96223f640 1 -- 172.21.15.1:0/956647694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95c19d6e0 con 0x7fa95c100a80 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.641+0000 7fa958ff9640 1 -- 172.21.15.1:0/956647694 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c035070 con 0x7fa95c100a80 2024-09-16T00:44:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.641+0000 7fa96223f640 1 -- 172.21.15.1:0/956647694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95c1a5020 con 0x7fa95c100a80 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.643+0000 7fa958ff9640 1 -- 172.21.15.1:0/956647694 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fa94c042030 con 0x7fa95c100a80 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.643+0000 7fa964618740 1 Processor -- start 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.643+0000 7fa964618740 1 -- start start 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa964618740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 0x564e3d16e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa964618740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 0x564e3d178d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa964618740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d179240 0x564e3d17b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa964618740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564e3d0d3620 con 0x564e3d179240 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa964618740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564e3d07ae50 con 0x564e3d16e980 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa964618740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e3d0cf5e0 con 0x564e3d16e060 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 0x564e3d16e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d179240 0x564e3d17b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 0x564e3d16e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:33558/0 (socket says 172.21.15.1:33558) 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95bfff640 1 -- 172.21.15.1:0/804222398 learned_addr learned my addr 172.21.15.1:0/804222398 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/804222398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 0x564e3d178d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa958ff9640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa9440a32c0 0x7fa9440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95affd640 1 -- 172.21.15.1:0/804222398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 msgr2=0x564e3d16e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95affd640 1 --2- 172.21.15.1:0/804222398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 0x564e3d16e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95affd640 1 -- 172.21.15.1:0/804222398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 msgr2=0x564e3d178d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95affd640 1 --2- 172.21.15.1:0/804222398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 0x564e3d178d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95affd640 1 -- 172.21.15.1:0/804222398 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e3d06bd40 con 0x564e3d179240 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/804222398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 0x564e3d178d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95bfff640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa9440a32c0 0x7fa9440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa958ff9640 1 -- 172.21.15.1:0/956647694 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa94c0f0200 con 0x7fa95c100a80 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.644+0000 7fa95affd640 1 --2- 172.21.15.1:0/804222398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d179240 0x564e3d17b630 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fa95000bbc0 tx=0x7fa9500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beac39d33f62c269 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa941ffb640 1 -- 172.21.15.1:0/804222398 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950004030 con 0x564e3d179240 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa941ffb640 1 -- 172.21.15.1:0/804222398 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9500041d0 con 0x564e3d179240 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa941ffb640 1 -- 172.21.15.1:0/804222398 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9500116a0 con 0x564e3d179240 2024-09-16T00:44:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa95bfff640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa9440a32c0 0x7fa9440a5780 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa95400a5a0 tx=0x7fa95400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa964618740 1 -- 172.21.15.1:0/804222398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d179240 msgr2=0x564e3d17b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa964618740 1 --2- 172.21.15.1:0/804222398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d179240 0x564e3d17b630 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fa95000bbc0 tx=0x7fa9500107a0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa964618740 1 -- 172.21.15.1:0/804222398 shutdown_connections 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa964618740 1 --2- 172.21.15.1:0/804222398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d179240 0x564e3d17b630 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa964618740 1 --2- 172.21.15.1:0/804222398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 0x564e3d178d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa964618740 1 --2- 172.21.15.1:0/804222398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 0x564e3d16e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa964618740 1 -- 172.21.15.1:0/804222398 >> 172.21.15.1:0/804222398 conn(0x564e3d16a5f0 msgr2=0x564e3d16a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa964618740 1 -- 172.21.15.1:0/804222398 shutdown_connections 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.645+0000 7fa964618740 1 -- 172.21.15.1:0/804222398 wait complete. 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa964618740 1 -- start start 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa964618740 1 Processor -- start 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa964618740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 0x564e3d1760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa964618740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e980 0x564e3d176630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa964618740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 0x564e3d176b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa964618740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564e3d17be10 con 0x564e3d16e980 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa964618740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564e3d05bf10 con 0x564e3d179240 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa964618740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e3d17bf90 con 0x564e3d16e060 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa964618740 10 client.0 _ll_register_callbacks cb 0x564e3d07fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa95affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e980 0x564e3d176630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa95affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e980 0x564e3d176630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38616/0 (socket says 172.21.15.1:38616) 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.646+0000 7fa95affd640 1 -- 172.21.15.1:0/3915232953 learned_addr learned my addr 172.21.15.1:0/3915232953 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa95bfff640 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 0x564e3d176b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 0x564e3d1760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa95affd640 1 -- 172.21.15.1:0/3915232953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 msgr2=0x564e3d1760f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:10.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa95affd640 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 0x564e3d1760f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa95affd640 1 -- 172.21.15.1:0/3915232953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 msgr2=0x564e3d176b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa95affd640 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 0x564e3d176b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa95affd640 1 -- 172.21.15.1:0/3915232953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e3cde2580 con 0x564e3d16e980 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa95affd640 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e980 0x564e3d176630 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa950005890 tx=0x7fa95000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69b95f5e6ba0a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa93affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa93affd640 1 -- 172.21.15.1:0/3915232953 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95002a060 con 0x564e3d16e980 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa93affd640 1 -- 172.21.15.1:0/3915232953 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa9500035f0 con 0x564e3d16e980 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa964618740 1 -- 172.21.15.1:0/3915232953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9500041d0 con 0x564e3d16e980 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa93affd640 1 -- 172.21.15.1:0/3915232953 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa950031050 con 0x564e3d16e980 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa964618740 10 client.19101 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa964618740 1 -- 172.21.15.1:0/3915232953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e3ceac960 con 0x564e3d16e980 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.647+0000 7fa93affd640 1 -- 172.21.15.1:0/3915232953 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9500041d0 con 0x564e3d16e980 2024-09-16T00:44:10.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa93affd640 1 -- 172.21.15.1:0/3915232953 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa9500047e0 con 0x564e3d16e980 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa964618740 10 client.19101 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa964618740 10 client.19101 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa964618740 20 client.19101 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa964618740 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e3d188de0 0x564e3d18b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa964618740 4 client.19101 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa964618740 1 -- 172.21.15.1:0/3915232953 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e3d0e0b90 con 0x564e3d188de0 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e3d188de0 0x564e3d18b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa93affd640 10 client.19101 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.648+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e3d188de0 0x564e3d18b1d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd18fee4f3bfeea server_cookie=27678cd0e0fe117c in_seq=0 out_seq=0 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa93affd640 1 -- 172.21.15.1:0/3915232953 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e3d0e0b90 con 0x564e3d188de0 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa93affd640 10 client.19101 handle_command_reply: tid=0 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 client.19101 shutdown 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 -- 172.21.15.1:0/3915232953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e980 msgr2=0x564e3d176630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:10.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e980 0x564e3d176630 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa950005890 tx=0x7fa95000bfa0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 -- 172.21.15.1:0/3915232953 shutdown_connections 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e3d188de0 0x564e3d18b1d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 0x564e3d176b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e980 0x564e3d176630 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 --2- 172.21.15.1:0/3915232953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e060 0x564e3d1760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa93affd640 0 client.19101 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 -- 172.21.15.1:0/3915232953 >> 172.21.15.1:0/3915232953 conn(0x564e3d16a5f0 msgr2=0x564e3d1702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 -- 172.21.15.1:0/3915232953 shutdown_connections 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 1 -- 172.21.15.1:0/3915232953 wait complete. 2024-09-16T00:44:10.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.649+0000 7fa964618740 20 client.19101 trim_cache size 0 max 16384 2024-09-16T00:44:10.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.666+0000 7fa964618740 1 Processor -- start 2024-09-16T00:44:10.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.666+0000 7fa964618740 1 -- start start 2024-09-16T00:44:10.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.666+0000 7fa964618740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d190800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.666+0000 7fa964618740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 0x564e3d190d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.666+0000 7fa964618740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d179240 0x564e3d173370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.666+0000 7fa964618740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1b1680 con 0x564e3d16e060 2024-09-16T00:44:10.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.666+0000 7fa964618740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1aef50 con 0x564e3d16e980 2024-09-16T00:44:10.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.666+0000 7fa964618740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1acda0 con 0x564e3d179240 2024-09-16T00:44:10.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d190800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d190800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38632/0 (socket says 172.21.15.1:38632) 2024-09-16T00:44:10.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95b7fe640 1 -- 172.21.15.1:0/1306782117 learned_addr learned my addr 172.21.15.1:0/1306782117 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:10.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95affd640 1 --2- 172.21.15.1:0/1306782117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 0x564e3d190d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95b7fe640 1 -- 172.21.15.1:0/1306782117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d179240 msgr2=0x564e3d173370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:10.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/1306782117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d179240 0x564e3d173370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95b7fe640 1 -- 172.21.15.1:0/1306782117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 msgr2=0x564e3d190d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:10.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/1306782117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 0x564e3d190d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95b7fe640 1 -- 172.21.15.1:0/1306782117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e3d16e7b0 con 0x564e3d16e060 2024-09-16T00:44:10.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/1306782117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d190800 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0bc350 tx=0x7fa94c0f2650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf685e9a5c97cd66 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:10.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa9417fa640 1 -- 172.21.15.1:0/1306782117 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c049040 con 0x564e3d16e060 2024-09-16T00:44:10.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa9417fa640 1 -- 172.21.15.1:0/1306782117 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94c0ed9b0 con 0x564e3d16e060 2024-09-16T00:44:10.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.667+0000 7fa9417fa640 1 -- 172.21.15.1:0/1306782117 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c003e10 con 0x564e3d16e060 2024-09-16T00:44:10.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 -- 172.21.15.1:0/1306782117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 msgr2=0x564e3d190800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 --2- 172.21.15.1:0/1306782117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d190800 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0bc350 tx=0x7fa94c0f2650 comp rx=0 tx=0).stop 2024-09-16T00:44:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 -- 172.21.15.1:0/1306782117 shutdown_connections 2024-09-16T00:44:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 --2- 172.21.15.1:0/1306782117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d179240 0x564e3d173370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 --2- 172.21.15.1:0/1306782117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d16e980 0x564e3d190d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 --2- 172.21.15.1:0/1306782117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d190800 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 -- 172.21.15.1:0/1306782117 >> 172.21.15.1:0/1306782117 conn(0x564e3d16a5f0 msgr2=0x564e3d1749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 -- 172.21.15.1:0/1306782117 shutdown_connections 2024-09-16T00:44:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 -- 172.21.15.1:0/1306782117 wait complete. 2024-09-16T00:44:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.668+0000 7fa964618740 1 -- start start 2024-09-16T00:44:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa964618740 1 Processor -- start 2024-09-16T00:44:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa964618740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d1811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa964618740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e980 0x564e3d181720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa964618740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 0x564e3d17e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa964618740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1ad410 con 0x564e3d16e060 2024-09-16T00:44:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa964618740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1ae880 con 0x564e3d179240 2024-09-16T00:44:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa964618740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1adae0 con 0x564e3d16e980 2024-09-16T00:44:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa964618740 10 client.0 _ll_register_callbacks cb 0x7fa95000b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:10.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d1811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d1811e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38640/0 (socket says 172.21.15.1:38640) 2024-09-16T00:44:10.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95b7fe640 1 -- 172.21.15.1:0/2717115189 learned_addr learned my addr 172.21.15.1:0/2717115189 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:10.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95bfff640 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 0x564e3d17e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95affd640 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e980 0x564e3d181720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95b7fe640 1 -- 172.21.15.1:0/2717115189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e980 msgr2=0x564e3d181720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:10.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e980 0x564e3d181720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95b7fe640 1 -- 172.21.15.1:0/2717115189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 msgr2=0x564e3d17e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:10.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 0x564e3d17e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95b7fe640 1 -- 172.21.15.1:0/2717115189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e3d17e6f0 con 0x564e3d16e060 2024-09-16T00:44:10.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95bfff640 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 0x564e3d17e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:10.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa95b7fe640 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d1811e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0b8d90 tx=0x7fa94c0bc540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4144afb303c4ac4a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:10.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.669+0000 7fa93affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:10.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa93affd640 1 -- 172.21.15.1:0/2717115189 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c049040 con 0x564e3d16e060 2024-09-16T00:44:10.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa93affd640 1 -- 172.21.15.1:0/2717115189 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fa94c0b8a30 con 0x564e3d16e060 2024-09-16T00:44:10.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa964618740 1 -- 172.21.15.1:0/2717115189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa94c0ed9b0 con 0x564e3d16e060 2024-09-16T00:44:10.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa93affd640 1 -- 172.21.15.1:0/2717115189 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa94c0c0770 con 0x564e3d16e060 2024-09-16T00:44:10.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa964618740 10 client.19107 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa964618740 1 -- 172.21.15.1:0/2717115189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e3d1f3f30 con 0x564e3d16e060 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa93affd640 1 -- 172.21.15.1:0/2717115189 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94c0ed9b0 con 0x564e3d16e060 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa93affd640 1 -- 172.21.15.1:0/2717115189 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa94c0bc620 con 0x564e3d16e060 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa964618740 10 client.19107 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa964618740 10 client.19107 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa964618740 20 client.19107 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa964618740 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e3d1f45f0 0x564e3d1f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa964618740 4 client.19107 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9bedaf13", "release": true, "await": true}] 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa964618740 1 -- 172.21.15.1:0/2717115189 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9bedaf13", "release": true, "await": true}) -- 0x564e3d0db0a0 con 0x564e3d1f45f0 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.670+0000 7fa95affd640 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e3d1f45f0 0x564e3d1f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.671+0000 7fa95affd640 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e3d1f45f0 0x564e3d1f6a10 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c62c1c9fddb8dc server_cookie=90ebd46f91e2d6ed in_seq=0 out_seq=0 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.671+0000 7fa93affd640 10 client.19107 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:10.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.673+0000 7fa93affd640 1 -- 172.21.15.1:0/2717115189 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564e3d0db0a0 con 0x564e3d1f45f0 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.673+0000 7fa93affd640 10 client.19107 handle_command_reply: tid=0 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.673+0000 7fa964618740 1 client.19107 shutdown 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.673+0000 7fa964618740 1 -- 172.21.15.1:0/2717115189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 msgr2=0x564e3d1811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d1811e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0b8d90 tx=0x7fa94c0bc540 comp rx=0 tx=0).stop 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 1 -- 172.21.15.1:0/2717115189 shutdown_connections 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564e3d1f45f0 0x564e3d1f6a10 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e3d179240 0x564e3d17e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e3d16e980 0x564e3d181720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 1 --2- 172.21.15.1:0/2717115189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e3d16e060 0x564e3d1811e0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 1 -- 172.21.15.1:0/2717115189 >> 172.21.15.1:0/2717115189 conn(0x564e3d16a5f0 msgr2=0x564e3d168f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa93affd640 0 client.19107 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 1 -- 172.21.15.1:0/2717115189 shutdown_connections 2024-09-16T00:44:10.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 1 -- 172.21.15.1:0/2717115189 wait complete. 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa964618740 20 client.19107 trim_cache size 0 max 16384 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 56, 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "9bedaf13": { 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 56, 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:44:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 137.7, 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:44:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:10.688 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 -- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa9440a32c0 msgr2=0x7fa9440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa9440a32c0 0x7fa9440a5780 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa95400a5a0 tx=0x7fa95400c040 comp rx=0 tx=0).stop 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 -- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c100a80 msgr2=0x7fa95c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c100a80 0x7fa95c1984a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009230 tx=0x7fa94c004270 comp rx=0 tx=0).stop 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 -- 172.21.15.1:0/956647694 shutdown_connections 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fa9440a32c0 0x7fa9440a5780 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa95c106a80 0x7fa95c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c101420 0x7fa95c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 --2- 172.21.15.1:0/956647694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa95c100a80 0x7fa95c1984a0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.674+0000 7fa96223f640 1 -- 172.21.15.1:0/956647694 >> 172.21.15.1:0/956647694 conn(0x7fa95c0fc7a0 msgr2=0x7fa95c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.675+0000 7fa96223f640 1 -- 172.21.15.1:0/956647694 shutdown_connections 2024-09-16T00:44:10.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:10.675+0000 7fa96223f640 1 -- 172.21.15.1:0/956647694 wait complete. 2024-09-16T00:44:11.461 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9bedaf13', total seconds elapsed: 21.0 2024-09-16T00:44:11.462 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-16T00:44:11.462 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:44:11.719 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:12.158 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:11 smithi019 ceph-mon[35186]: pgmap v824: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:44:12.170 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:11 smithi001 ceph-mon[30934]: pgmap v824: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:44:12.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:11 smithi153 ceph-mon[35538]: pgmap v824: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:44:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.251+0000 7f9af4fd9640 1 -- 172.21.15.1:0/271941244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 msgr2=0x7f9af01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.251+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/271941244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 0x7f9af01029b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009900 tx=0x7f9ad802a480 comp rx=0 tx=0).stop 2024-09-16T00:44:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.251+0000 7f9af4fd9640 1 -- 172.21.15.1:0/271941244 shutdown_connections 2024-09-16T00:44:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.251+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/271941244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af0102ef0 0x7f9af010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.251+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/271941244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 0x7f9af01029b0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.251+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/271941244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af0100e20 0x7f9af0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.251+0000 7f9af4fd9640 1 -- 172.21.15.1:0/271941244 >> 172.21.15.1:0/271941244 conn(0x7f9af00fc7d0 msgr2=0x7f9af00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.251+0000 7f9af4fd9640 1 -- 172.21.15.1:0/271941244 shutdown_connections 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.251+0000 7f9af4fd9640 1 -- 172.21.15.1:0/271941244 wait complete. 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9af4fd9640 1 Processor -- start 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9af4fd9640 1 -- start start 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9af4fd9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af0100e20 0x7f9af01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9af4fd9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 0x7f9af01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9af4fd9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af0102ef0 0x7f9af01a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9af4fd9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0110f80 con 0x7f9af0102570 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9af4fd9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0110e00 con 0x7f9af0100e20 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9af4fd9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0111100 con 0x7f9af0102ef0 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9ae7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 0x7f9af01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9ae7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 0x7f9af01998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38658/0 (socket says 172.21.15.1:38658) 2024-09-16T00:44:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9ae7fff640 1 -- 172.21.15.1:0/1674022576 learned_addr learned my addr 172.21.15.1:0/1674022576 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9aee59c640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af0100e20 0x7f9af01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.252+0000 7f9aeed9d640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af0102ef0 0x7f9af01a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.253+0000 7f9ae7fff640 1 -- 172.21.15.1:0/1674022576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af0102ef0 msgr2=0x7f9af01a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.253+0000 7f9ae7fff640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af0102ef0 0x7f9af01a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.253+0000 7f9ae7fff640 1 -- 172.21.15.1:0/1674022576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af0100e20 msgr2=0x7f9af01993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.253+0000 7f9ae7fff640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af0100e20 0x7f9af01993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.253+0000 7f9ae7fff640 1 -- 172.21.15.1:0/1674022576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af01a4970 con 0x7f9af0102570 2024-09-16T00:44:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.253+0000 7f9ae7fff640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 0x7f9af01998f0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006010 tx=0x7f9ad8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.254+0000 7f9ae77fe640 1 -- 172.21.15.1:0/1674022576 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad803f070 con 0x7f9af0102570 2024-09-16T00:44:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.254+0000 7f9ae77fe640 1 -- 172.21.15.1:0/1674022576 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ad8031c30 con 0x7f9af0102570 2024-09-16T00:44:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.254+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af01a4b40 con 0x7f9af0102570 2024-09-16T00:44:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.254+0000 7f9ae77fe640 1 -- 172.21.15.1:0/1674022576 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8035030 con 0x7f9af0102570 2024-09-16T00:44:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.254+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af01a4f90 con 0x7f9af0102570 2024-09-16T00:44:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.255+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af0101200 con 0x7f9af0102570 2024-09-16T00:44:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.256+0000 7f9ae77fe640 1 -- 172.21.15.1:0/1674022576 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9ad8042070 con 0x7f9af0102570 2024-09-16T00:44:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.257+0000 7f9ae77fe640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9adc0a3240 0x7f9adc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.257+0000 7f9aee59c640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9adc0a3240 0x7f9adc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.257+0000 7f9ae77fe640 1 -- 172.21.15.1:0/1674022576 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9ad80f0950 con 0x7f9af0102570 2024-09-16T00:44:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.257+0000 7f9aee59c640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9adc0a3240 0x7f9adc0a5700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009bd0 tx=0x7f9ae0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:12.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.260+0000 7f9ae77fe640 1 -- 172.21.15.1:0/1674022576 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ad80bd210 con 0x7f9af0102570 2024-09-16T00:44:12.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.471+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9af0109090 con 0x7f9af0102570 2024-09-16T00:44:12.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.472+0000 7f9ae77fe640 1 -- 172.21.15.1:0/1674022576 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f9ad80bcb10 con 0x7f9af0102570 2024-09-16T00:44:12.472 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:44:12.473 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:44:12.473 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:44:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9adc0a3240 msgr2=0x7f9adc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9adc0a3240 0x7f9adc0a5700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009bd0 tx=0x7f9ae0005f90 comp rx=0 tx=0).stop 2024-09-16T00:44:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 msgr2=0x7f9af01998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 0x7f9af01998f0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006010 tx=0x7f9ad8003c10 comp rx=0 tx=0).stop 2024-09-16T00:44:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 shutdown_connections 2024-09-16T00:44:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f9adc0a3240 0x7f9adc0a5700 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af0102ef0 0x7f9af01a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9af0102570 0x7f9af01998f0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 --2- 172.21.15.1:0/1674022576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af0100e20 0x7f9af01993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 >> 172.21.15.1:0/1674022576 conn(0x7f9af00fc7d0 msgr2=0x7f9af00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 shutdown_connections 2024-09-16T00:44:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:12.474+0000 7f9af4fd9640 1 -- 172.21.15.1:0/1674022576 wait complete. 2024-09-16T00:44:13.150 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:12 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1674022576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:13.179 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-16T00:44:13.179 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-16T00:44:13.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:12 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1674022576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:13.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:12 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1674022576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 -- 172.21.15.1:0/2398037683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 msgr2=0x7fe4b01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/2398037683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 0x7fe4b01067f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009900 tx=0x7fe4a402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 -- 172.21.15.1:0/2398037683 shutdown_connections 2024-09-16T00:44:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/2398037683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4b0106f20 0x7fe4b010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/2398037683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 0x7fe4b01067f0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/2398037683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0101df0 0x7fe4b01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 -- 172.21.15.1:0/2398037683 >> 172.21.15.1:0/2398037683 conn(0x7fe4b00fd8f0 msgr2=0x7fe4b00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 -- 172.21.15.1:0/2398037683 shutdown_connections 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 -- 172.21.15.1:0/2398037683 wait complete. 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.968+0000 7fe4b5d58640 1 Processor -- start 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4b5d58640 1 -- start start 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4b5d58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0101df0 0x7fe4b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4b5d58640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 0x7fe4b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4b5d58640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4b0106f20 0x7fe4b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4b5d58640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b0111f40 con 0x7fe4b01027a0 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4b5d58640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b0111dc0 con 0x7fe4b0101df0 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4b5d58640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b01120c0 con 0x7fe4b0106f20 2024-09-16T00:44:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 0x7fe4b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 0x7fe4b0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38678/0 (socket says 172.21.15.1:38678) 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4aeffd640 1 -- 172.21.15.1:0/3790235890 learned_addr learned my addr 172.21.15.1:0/3790235890 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4affff640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4b0106f20 0x7fe4b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0101df0 0x7fe4b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4aeffd640 1 -- 172.21.15.1:0/3790235890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4b0106f20 msgr2=0x7fe4b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4b0106f20 0x7fe4b0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4aeffd640 1 -- 172.21.15.1:0/3790235890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0101df0 msgr2=0x7fe4b00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0101df0 0x7fe4b00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4aeffd640 1 -- 172.21.15.1:0/3790235890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b01a67f0 con 0x7fe4b01027a0 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.969+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0101df0 0x7fe4b00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.970+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 0x7fe4b0074a30 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4005ab0 tx=0x7fe4a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.970+0000 7fe4acff9640 1 -- 172.21.15.1:0/3790235890 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a403f030 con 0x7fe4b01027a0 2024-09-16T00:44:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.970+0000 7fe4b5d58640 1 -- 172.21.15.1:0/3790235890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b01a6a80 con 0x7fe4b01027a0 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.970+0000 7fe4acff9640 1 -- 172.21.15.1:0/3790235890 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4003ce0 con 0x7fe4b01027a0 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.970+0000 7fe4acff9640 1 -- 172.21.15.1:0/3790235890 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4035070 con 0x7fe4b01027a0 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.970+0000 7fe4b5d58640 1 -- 172.21.15.1:0/3790235890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b01a6fb0 con 0x7fe4b01027a0 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4acff9640 1 -- 172.21.15.1:0/3790235890 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe4a4042030 con 0x7fe4b01027a0 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4b8131740 1 Processor -- start 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4b8131740 1 -- start start 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 0x555b6bf32b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf3d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bf3fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4b8131740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555b6be982a0 con 0x555b6bf3d940 2024-09-16T00:44:13.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4b8131740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555b6be3fb10 con 0x555b6bf32760 2024-09-16T00:44:13.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4b8131740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b6be94260 con 0x555b6bf33080 2024-09-16T00:44:13.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bf3fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bf3fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38694/0 (socket says 172.21.15.1:38694) 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4aeffd640 1 -- 172.21.15.1:0/605265880 learned_addr learned my addr 172.21.15.1:0/605265880 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4affff640 1 --2- 172.21.15.1:0/605265880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 0x555b6bf32b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4acff9640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4980a3240 0x7fe4980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/605265880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf3d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4aeffd640 1 -- 172.21.15.1:0/605265880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 msgr2=0x555b6bf3d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/605265880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf3d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4aeffd640 1 -- 172.21.15.1:0/605265880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 msgr2=0x555b6bf32b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.973+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/605265880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 0x555b6bf32b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.974+0000 7fe4aeffd640 1 -- 172.21.15.1:0/605265880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6bc09b90 con 0x555b6bf3d940 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.974+0000 7fe4affff640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4980a3240 0x7fe4980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.974+0000 7fe4acff9640 1 -- 172.21.15.1:0/3790235890 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe4a40f0600 con 0x7fe4b01027a0 2024-09-16T00:44:13.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.974+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/605265880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bf3fd30 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40ac250 tx=0x7fe4a4087260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85b4e93439d9f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.974+0000 7fe4affff640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4980a3240 0x7fe4980a5700 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8007fc0 tx=0x7fe4a8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.974+0000 7fe495ffb640 1 -- 172.21.15.1:0/605265880 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40bc180 con 0x555b6bf3d940 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.974+0000 7fe495ffb640 1 -- 172.21.15.1:0/605265880 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a40bcf20 con 0x555b6bf3d940 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.974+0000 7fe495ffb640 1 -- 172.21.15.1:0/605265880 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a408fba0 con 0x555b6bf3d940 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 -- 172.21.15.1:0/605265880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 msgr2=0x555b6bf3fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 --2- 172.21.15.1:0/605265880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bf3fd30 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40ac250 tx=0x7fe4a4087260 comp rx=0 tx=0).stop 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 -- 172.21.15.1:0/605265880 shutdown_connections 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 --2- 172.21.15.1:0/605265880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bf3fd30 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 --2- 172.21.15.1:0/605265880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf3d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 --2- 172.21.15.1:0/605265880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 0x555b6bf32b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 -- 172.21.15.1:0/605265880 >> 172.21.15.1:0/605265880 conn(0x555b6bf2ecf0 msgr2=0x555b6bf2f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 -- 172.21.15.1:0/605265880 shutdown_connections 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 -- 172.21.15.1:0/605265880 wait complete. 2024-09-16T00:44:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 -- start start 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.975+0000 7fe4b8131740 1 Processor -- start 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf32760 0x555b6bf3a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf33080 0x555b6bf3ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 0x555b6bf3b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4b8131740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555b6bf402d0 con 0x555b6bf33080 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4b8131740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555b6be206e0 con 0x555b6bf3d940 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4b8131740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b6bf40450 con 0x555b6bf32760 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4b8131740 10 client.0 _ll_register_callbacks cb 0x555b6be44a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf33080 0x555b6bf3ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf33080 0x555b6bf3ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38704/0 (socket says 172.21.15.1:38704) 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4aeffd640 1 -- 172.21.15.1:0/4159633927 learned_addr learned my addr 172.21.15.1:0/4159633927 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf32760 0x555b6bf3a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4affff640 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 0x555b6bf3b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4aeffd640 1 -- 172.21.15.1:0/4159633927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf32760 msgr2=0x555b6bf3a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf32760 0x555b6bf3a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4aeffd640 1 -- 172.21.15.1:0/4159633927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 msgr2=0x555b6bf3b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 0x555b6bf3b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4aeffd640 1 -- 172.21.15.1:0/4159633927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6bf496b0 con 0x555b6bf33080 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf33080 0x555b6bf3ace0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006010 tx=0x7fe4a403b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca654d925819b7a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.976+0000 7fe48effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe48effd640 1 -- 172.21.15.1:0/4159633927 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4049040 con 0x555b6bf33080 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe48effd640 1 -- 172.21.15.1:0/4159633927 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe4a403bd70 con 0x555b6bf33080 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe4b8131740 1 -- 172.21.15.1:0/4159633927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a40bcf20 con 0x555b6bf33080 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe48effd640 1 -- 172.21.15.1:0/4159633927 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe4a40bc180 con 0x555b6bf33080 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe4b8131740 10 client.19125 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe4b8131740 1 -- 172.21.15.1:0/4159633927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b6bf402d0 con 0x555b6bf33080 2024-09-16T00:44:13.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe48effd640 1 -- 172.21.15.1:0/4159633927 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a40bcf20 con 0x555b6bf33080 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe48effd640 1 -- 172.21.15.1:0/4159633927 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe4a4089020 con 0x555b6bf33080 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe4b8131740 10 client.19125 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe4b8131740 10 client.19125 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe4b8131740 20 client.19125 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe4b8131740 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555b6bf4d370 0x555b6bf4f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe4b8131740 4 client.19125 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.977+0000 7fe4b8131740 1 -- 172.21.15.1:0/4159633927 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b6bea59d0 con 0x555b6bf4d370 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.978+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555b6bf4d370 0x555b6bf4f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.978+0000 7fe48effd640 10 client.19125 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.978+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555b6bf4d370 0x555b6bf4f760 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46060170507a08cc server_cookie=afd8aa4a33c7d8f4 in_seq=0 out_seq=0 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe48effd640 1 -- 172.21.15.1:0/4159633927 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b6bea59d0 con 0x555b6bf4d370 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe48effd640 10 client.19125 handle_command_reply: tid=0 2024-09-16T00:44:13.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 client.19125 shutdown 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 -- 172.21.15.1:0/4159633927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf33080 msgr2=0x555b6bf3ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf33080 0x555b6bf3ace0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006010 tx=0x7fe4a403b800 comp rx=0 tx=0).stop 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 -- 172.21.15.1:0/4159633927 shutdown_connections 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555b6bf4d370 0x555b6bf4f760 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 0x555b6bf3b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf33080 0x555b6bf3ace0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 --2- 172.21.15.1:0/4159633927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf32760 0x555b6bf3a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 -- 172.21.15.1:0/4159633927 >> 172.21.15.1:0/4159633927 conn(0x555b6bf2ecf0 msgr2=0x555b6bf34a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe48effd640 0 client.19125 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 -- 172.21.15.1:0/4159633927 shutdown_connections 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 1 -- 172.21.15.1:0/4159633927 wait complete. 2024-09-16T00:44:13.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.979+0000 7fe4b8131740 20 client.19125 trim_cache size 0 max 16384 2024-09-16T00:44:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.996+0000 7fe4b8131740 1 Processor -- start 2024-09-16T00:44:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.996+0000 7fe4b8131740 1 -- start start 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.996+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf32760 0x555b6bf3b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.996+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf55020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.996+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 0x555b6bf55560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.996+0000 7fe4b8131740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b01b3800 con 0x555b6bf32760 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.996+0000 7fe4b8131740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b01b10d0 con 0x555b6bf3d940 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.996+0000 7fe4b8131740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b01aef20 con 0x555b6bf33080 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4af7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf32760 0x555b6bf3b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4af7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf32760 0x555b6bf3b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38714/0 (socket says 172.21.15.1:38714) 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4af7fe640 1 -- 172.21.15.1:0/671234194 learned_addr learned my addr 172.21.15.1:0/671234194 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf55020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4affff640 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 0x555b6bf55560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:13.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4af7fe640 1 -- 172.21.15.1:0/671234194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 msgr2=0x555b6bf55020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf55020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4af7fe640 1 -- 172.21.15.1:0/671234194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 msgr2=0x555b6bf55560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 0x555b6bf55560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4af7fe640 1 -- 172.21.15.1:0/671234194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6bf2a260 con 0x555b6bf32760 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4affff640 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 0x555b6bf55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf32760 0x555b6bf3b070 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0005940 tx=0x7fe4a0013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ed4555b91277e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4957fa640 1 -- 172.21.15.1:0/671234194 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00130d0 con 0x555b6bf32760 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4957fa640 1 -- 172.21.15.1:0/671234194 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a0013270 con 0x555b6bf32760 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.997+0000 7fe4957fa640 1 -- 172.21.15.1:0/671234194 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0011440 con 0x555b6bf32760 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 -- 172.21.15.1:0/671234194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf32760 msgr2=0x555b6bf3b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf32760 0x555b6bf3b070 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0005940 tx=0x7fe4a0013830 comp rx=0 tx=0).stop 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 -- 172.21.15.1:0/671234194 shutdown_connections 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf3d940 0x555b6bf55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf55020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 --2- 172.21.15.1:0/671234194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf32760 0x555b6bf3b070 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 -- 172.21.15.1:0/671234194 >> 172.21.15.1:0/671234194 conn(0x555b6bf2ecf0 msgr2=0x555b6bf3c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 -- 172.21.15.1:0/671234194 shutdown_connections 2024-09-16T00:44:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 -- 172.21.15.1:0/671234194 wait complete. 2024-09-16T00:44:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.998+0000 7fe4b8131740 1 -- start start 2024-09-16T00:44:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4b8131740 1 Processor -- start 2024-09-16T00:44:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 0x555b6bf41f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4b8131740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bfaffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4b8131740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b01af590 con 0x555b6bf3d940 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4b8131740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b01b0a00 con 0x555b6bf32760 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4b8131740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b01afc60 con 0x555b6bf33080 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4b8131740 10 client.0 _ll_register_callbacks cb 0x7fe4a40f0900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bfaffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bfaffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38726/0 (socket says 172.21.15.1:38726) 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4affff640 1 -- 172.21.15.1:0/1439003275 learned_addr learned my addr 172.21.15.1:0/1439003275 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 0x555b6bf41f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4affff640 1 -- 172.21.15.1:0/1439003275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 msgr2=0x555b6bf424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4affff640 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4affff640 1 -- 172.21.15.1:0/1439003275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 msgr2=0x555b6bf41f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4affff640 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 0x555b6bf41f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4affff640 1 -- 172.21.15.1:0/1439003275 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6bf355d0 con 0x555b6bf3d940 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:13.999+0000 7fe4aeffd640 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4affff640 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bfaffb0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80079c0 tx=0x7fe4a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bb210a18020ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe48effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe48effd640 1 -- 172.21.15.1:0/1439003275 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8010bb0 con 0x555b6bf3d940 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe48effd640 1 -- 172.21.15.1:0/1439003275 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe4a8010d50 con 0x555b6bf3d940 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4b8131740 1 -- 172.21.15.1:0/1439003275 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a0013270 con 0x555b6bf3d940 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe48effd640 1 -- 172.21.15.1:0/1439003275 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe4a8020410 con 0x555b6bf3d940 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4b8131740 10 client.19131 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4b8131740 1 -- 172.21.15.1:0/1439003275 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b6bfb88c0 con 0x555b6bf3d940 2024-09-16T00:44:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe48effd640 1 -- 172.21.15.1:0/1439003275 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a0013270 con 0x555b6bf3d940 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe48effd640 1 -- 172.21.15.1:0/1439003275 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe4a8029060 con 0x555b6bf3d940 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4b8131740 10 client.19131 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4b8131740 10 client.19131 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4b8131740 20 client.19131 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4b8131740 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555b6bfb90a0 0x555b6bfbb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4b8131740 4 client.19131 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.000+0000 7fe4b8131740 1 -- 172.21.15.1:0/1439003275 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x555b6be9fd20 con 0x555b6bfb90a0 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.001+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555b6bfb90a0 0x555b6bfbb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.001+0000 7fe4af7fe640 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555b6bfb90a0 0x555b6bfbb490 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0d8a03bf8d6257 server_cookie=dc61a51aabb9fd5a in_seq=0 out_seq=0 2024-09-16T00:44:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.001+0000 7fe48effd640 10 client.19131 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe48effd640 1 -- 172.21.15.1:0/1439003275 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555b6be9fd20 con 0x555b6bfb90a0 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe48effd640 10 client.19131 handle_command_reply: tid=0 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 client.19131 shutdown 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 -- 172.21.15.1:0/1439003275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 msgr2=0x555b6bfaffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bfaffb0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80079c0 tx=0x7fe4a8004ab0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 -- 172.21.15.1:0/1439003275 shutdown_connections 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x555b6bfb90a0 0x555b6bfbb490 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b6bf3d940 0x555b6bfaffb0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6bf33080 0x555b6bf424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 --2- 172.21.15.1:0/1439003275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b6bf32760 0x555b6bf41f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 -- 172.21.15.1:0/1439003275 >> 172.21.15.1:0/1439003275 conn(0x555b6bf2ecf0 msgr2=0x555b6bf2d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe48effd640 0 client.19131 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 -- 172.21.15.1:0/1439003275 shutdown_connections 2024-09-16T00:44:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 1 -- 172.21.15.1:0/1439003275 wait complete. 2024-09-16T00:44:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.012+0000 7fe4b8131740 20 client.19131 trim_cache size 0 max 16384 2024-09-16T00:44:14.014 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:44:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:44:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:44:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 58, 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "44509eca": { 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 58, 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 137.7, 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:44:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:44:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-16T00:44:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:44:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:14.016 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 -- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4980a3240 msgr2=0x7fe4980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4980a3240 0x7fe4980a5700 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8007fc0 tx=0x7fe4a8004a40 comp rx=0 tx=0).stop 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 -- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 msgr2=0x7fe4b0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 0x7fe4b0074a30 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4005ab0 tx=0x7fe4a40042f0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 -- 172.21.15.1:0/3790235890 shutdown_connections 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4980a3240 0x7fe4980a5700 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4b0106f20 0x7fe4b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4b01027a0 0x7fe4b0074a30 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 --2- 172.21.15.1:0/3790235890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0101df0 0x7fe4b00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 -- 172.21.15.1:0/3790235890 >> 172.21.15.1:0/3790235890 conn(0x7fe4b00fd8f0 msgr2=0x7fe4b00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 -- 172.21.15.1:0/3790235890 shutdown_connections 2024-09-16T00:44:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:14.013+0000 7fe4b5d58640 1 -- 172.21.15.1:0/3790235890 wait complete. 2024-09-16T00:44:14.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:13 smithi001 ceph-mon[30934]: pgmap v825: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:44:14.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:13 smithi153 ceph-mon[35538]: pgmap v825: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:44:14.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:13 smithi019 ceph-mon[35186]: pgmap v825: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:44:14.707 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '44509eca', quiesce took 3.2 seconds. Will release after: 14.5 2024-09-16T00:44:16.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:15 smithi001 ceph-mon[30934]: pgmap v826: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:44:16.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:15 smithi153 ceph-mon[35538]: pgmap v826: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:44:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:15 smithi019 ceph-mon[35186]: pgmap v826: 129 pgs: 129 active+clean; 7.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:44:18.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:17 smithi001 ceph-mon[30934]: pgmap v827: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-16T00:44:18.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:17 smithi153 ceph-mon[35538]: pgmap v827: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-16T00:44:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:17 smithi019 ceph-mon[35186]: pgmap v827: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-16T00:44:20.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:19 smithi001 ceph-mon[30934]: pgmap v828: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-16T00:44:20.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:19 smithi153 ceph-mon[35538]: pgmap v828: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-16T00:44:20.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:19 smithi019 ceph-mon[35186]: pgmap v828: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-16T00:44:22.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:21 smithi001 ceph-mon[30934]: pgmap v829: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-09-16T00:44:22.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:21 smithi153 ceph-mon[35538]: pgmap v829: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-09-16T00:44:22.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:21 smithi019 ceph-mon[35186]: pgmap v829: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-09-16T00:44:24.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:23 smithi001 ceph-mon[30934]: pgmap v830: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-16T00:44:24.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:23 smithi153 ceph-mon[35538]: pgmap v830: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-16T00:44:24.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:23 smithi019 ceph-mon[35186]: pgmap v830: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-16T00:44:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:25 smithi001 ceph-mon[30934]: pgmap v831: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-09-16T00:44:26.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:25 smithi153 ceph-mon[35538]: pgmap v831: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-09-16T00:44:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:25 smithi019 ceph-mon[35186]: pgmap v831: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-09-16T00:44:26.901 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:44:27.177 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:27.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 -- 172.21.15.1:0/2650242377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 msgr2=0x7f8a6810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:27.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 --2- 172.21.15.1:0/2650242377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 0x7f8a6810ec70 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009930 tx=0x7f8a6002a420 comp rx=0 tx=0).stop 2024-09-16T00:44:27.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 -- 172.21.15.1:0/2650242377 shutdown_connections 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 --2- 172.21.15.1:0/2650242377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 0x7f8a6810ec70 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 --2- 172.21.15.1:0/2650242377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68103780 0x7f8a68107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 --2- 172.21.15.1:0/2650242377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a68102dd0 0x7f8a681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 -- 172.21.15.1:0/2650242377 >> 172.21.15.1:0/2650242377 conn(0x7f8a680fc7d0 msgr2=0x7f8a680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 -- 172.21.15.1:0/2650242377 shutdown_connections 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 -- 172.21.15.1:0/2650242377 wait complete. 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 Processor -- start 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.718+0000 7f8a6d757640 1 -- start start 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a6d757640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68102dd0 0x7f8a68198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a6d757640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a68103780 0x7f8a68198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a6d757640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 0x7f8a6819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a6d757640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681120d0 con 0x7f8a68108300 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a6d757640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68111f50 con 0x7f8a68102dd0 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a6d757640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68112250 con 0x7f8a68103780 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 0x7f8a6819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 0x7f8a6819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35406/0 (socket says 172.21.15.1:35406) 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a677fe640 1 -- 172.21.15.1:0/1314179458 learned_addr learned my addr 172.21.15.1:0/1314179458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a66ffd640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68102dd0 0x7f8a68198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a667fc640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a68103780 0x7f8a68198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a677fe640 1 -- 172.21.15.1:0/1314179458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a68103780 msgr2=0x7f8a68198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a677fe640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a68103780 0x7f8a68198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a677fe640 1 -- 172.21.15.1:0/1314179458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68102dd0 msgr2=0x7f8a68198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a677fe640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68102dd0 0x7f8a68198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a677fe640 1 -- 172.21.15.1:0/1314179458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6819d540 con 0x7f8a68108300 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a677fe640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 0x7f8a6819ce60 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f8a60005ab0 tx=0x7f8a600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a4ffff640 1 -- 172.21.15.1:0/1314179458 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6003f030 con 0x7f8a68108300 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.719+0000 7f8a6d757640 1 -- 172.21.15.1:0/1314179458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6819d7d0 con 0x7f8a68108300 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.720+0000 7f8a4ffff640 1 -- 172.21.15.1:0/1314179458 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a60003ce0 con 0x7f8a68108300 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.720+0000 7f8a4ffff640 1 -- 172.21.15.1:0/1314179458 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60035070 con 0x7f8a68108300 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.720+0000 7f8a6d757640 1 -- 172.21.15.1:0/1314179458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a681a5110 con 0x7f8a68108300 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a4ffff640 1 -- 172.21.15.1:0/1314179458 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8a60042030 con 0x7f8a68108300 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a6fb30740 1 Processor -- start 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a6fb30740 1 -- start start 2024-09-16T00:44:27.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 0x564a7af25880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af30140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af32a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564a7ae8b2e0 con 0x564a7af30680 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a7ae32ce0 con 0x564a7af254a0 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564a7ae872a0 con 0x564a7af25dc0 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.722+0000 7f8a667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af32a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af32a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35422/0 (socket says 172.21.15.1:35422) 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a667fc640 1 -- 172.21.15.1:0/2941742424 learned_addr learned my addr 172.21.15.1:0/2941742424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a667fc640 1 --2- 172.21.15.1:0/2941742424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 0x564a7af25880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a4ffff640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a500a3240 0x7f8a500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a66ffd640 1 --2- 172.21.15.1:0/2941742424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af30140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a677fe640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a500a3240 0x7f8a500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a667fc640 1 -- 172.21.15.1:0/2941742424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 msgr2=0x564a7af30140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a667fc640 1 --2- 172.21.15.1:0/2941742424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af30140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a667fc640 1 -- 172.21.15.1:0/2941742424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 msgr2=0x564a7af25880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a667fc640 1 --2- 172.21.15.1:0/2941742424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 0x564a7af25880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a667fc640 1 -- 172.21.15.1:0/2941742424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a7ae17440 con 0x564a7af30680 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a4ffff640 1 -- 172.21.15.1:0/1314179458 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8a600f0100 con 0x7f8a68108300 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a667fc640 1 --2- 172.21.15.1:0/2941742424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af32a70 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c0126c0 tx=0x7f8a5c00b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e506cfd6b0569490 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/2941742424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c012c40 con 0x564a7af30680 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a677fe640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a500a3240 0x7f8a500a5700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8a600b8830 tx=0x7f8a600872b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/2941742424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c012de0 con 0x564a7af30680 2024-09-16T00:44:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.723+0000 7f8a4d7fa640 1 -- 172.21.15.1:0/2941742424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c004a00 con 0x564a7af30680 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2941742424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 msgr2=0x564a7af32a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/2941742424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af32a70 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c0126c0 tx=0x7f8a5c00b6e0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2941742424 shutdown_connections 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/2941742424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af32a70 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/2941742424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af30140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/2941742424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 0x564a7af25880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2941742424 >> 172.21.15.1:0/2941742424 conn(0x564a7af21a30 msgr2=0x564a7af21e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2941742424 shutdown_connections 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2941742424 wait complete. 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 -- start start 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.724+0000 7f8a6fb30740 1 Processor -- start 2024-09-16T00:44:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 0x564a7af30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af2d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af2d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564a7af33250 con 0x564a7af30680 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a7ae46050 con 0x564a7af254a0 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564a7af333d0 con 0x564a7af25dc0 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a6fb30740 10 client.0 _ll_register_callbacks cb 0x564a7ae86160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af2d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af2d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35436/0 (socket says 172.21.15.1:35436) 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 -- 172.21.15.1:0/3891413881 learned_addr learned my addr 172.21.15.1:0/3891413881 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a66ffd640 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af2d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 0x564a7af30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 -- 172.21.15.1:0/3891413881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 msgr2=0x564a7af2d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af2d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 -- 172.21.15.1:0/3891413881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 msgr2=0x564a7af30090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 0x564a7af30090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 -- 172.21.15.1:0/3891413881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7abfc3c0 con 0x564a7af30680 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a66ffd640 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af2d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a667fc640 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af2d950 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001eb0 tx=0x7f8a5c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba17d1a9fb565fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a4aff5640 1 -- 172.21.15.1:0/3891413881 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c012c40 con 0x564a7af30680 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a6fb30740 1 -- 172.21.15.1:0/3891413881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a5c012de0 con 0x564a7af30680 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.725+0000 7f8a4aff5640 1 -- 172.21.15.1:0/3891413881 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8a5c010d80 con 0x564a7af30680 2024-09-16T00:44:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a4aff5640 1 -- 172.21.15.1:0/3891413881 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a5c031050 con 0x564a7af30680 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a6fb30740 10 client.19143 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a6fb30740 1 -- 172.21.15.1:0/3891413881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a7af3ece0 con 0x564a7af30680 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a4aff5640 1 -- 172.21.15.1:0/3891413881 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c012de0 con 0x564a7af30680 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a4aff5640 1 -- 172.21.15.1:0/3891413881 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8a5c01e5d0 con 0x564a7af30680 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a6fb30740 10 client.19143 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a6fb30740 10 client.19143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a6fb30740 20 client.19143 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a7af40030 0x564a7af42420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a6fb30740 4 client.19143 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a6fb30740 1 -- 172.21.15.1:0/3891413881 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a7ae98850 con 0x564a7af40030 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.726+0000 7f8a66ffd640 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a7af40030 0x564a7af42420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.727+0000 7f8a66ffd640 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a7af40030 0x564a7af42420 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4faef5759198c0 server_cookie=8729334a272a0793 in_seq=0 out_seq=0 2024-09-16T00:44:27.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.727+0000 7f8a4aff5640 10 client.19143 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.727+0000 7f8a4aff5640 1 -- 172.21.15.1:0/3891413881 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a7ae98850 con 0x564a7af40030 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.727+0000 7f8a4aff5640 10 client.19143 handle_command_reply: tid=0 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 client.19143 shutdown 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 -- 172.21.15.1:0/3891413881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 msgr2=0x564a7af2d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af2d950 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001eb0 tx=0x7f8a5c014040 comp rx=0 tx=0).stop 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 -- 172.21.15.1:0/3891413881 shutdown_connections 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a7af40030 0x564a7af42420 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af30680 0x564a7af2d950 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af2d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a4aff5640 0 client.19143 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/3891413881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af254a0 0x564a7af30090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 -- 172.21.15.1:0/3891413881 >> 172.21.15.1:0/3891413881 conn(0x564a7af21a30 msgr2=0x564a7af27710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 -- 172.21.15.1:0/3891413881 shutdown_connections 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 1 -- 172.21.15.1:0/3891413881 wait complete. 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.728+0000 7f8a6fb30740 20 client.19143 trim_cache size 0 max 16384 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a6fb30740 1 Processor -- start 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a6fb30740 1 -- start start 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af3c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af25dc0 0x564a7af28090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af30680 0x564a7af47bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681b1750 con 0x564a7af254a0 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681af020 con 0x564a7af25dc0 2024-09-16T00:44:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681ace70 con 0x564a7af30680 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af3c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af3c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35444/0 (socket says 172.21.15.1:35444) 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a667fc640 1 -- 172.21.15.1:0/2175722760 learned_addr learned my addr 172.21.15.1:0/2175722760 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a66ffd640 1 --2- 172.21.15.1:0/2175722760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af25dc0 0x564a7af28090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a667fc640 1 -- 172.21.15.1:0/2175722760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af30680 msgr2=0x564a7af47bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a667fc640 1 --2- 172.21.15.1:0/2175722760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af30680 0x564a7af47bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a667fc640 1 -- 172.21.15.1:0/2175722760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af25dc0 msgr2=0x564a7af28090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a667fc640 1 --2- 172.21.15.1:0/2175722760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af25dc0 0x564a7af28090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a667fc640 1 -- 172.21.15.1:0/2175722760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a7af28f40 con 0x564a7af254a0 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.746+0000 7f8a667fc640 1 --2- 172.21.15.1:0/2175722760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af3c4b0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c01ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418061c4a1251a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a4cff9640 1 -- 172.21.15.1:0/2175722760 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c02d070 con 0x564a7af254a0 2024-09-16T00:44:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a4cff9640 1 -- 172.21.15.1:0/2175722760 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c016040 con 0x564a7af254a0 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a4cff9640 1 -- 172.21.15.1:0/2175722760 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c0035f0 con 0x564a7af254a0 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2175722760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 msgr2=0x564a7af3c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/2175722760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af3c4b0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c01ff30 comp rx=0 tx=0).stop 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2175722760 shutdown_connections 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/2175722760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af30680 0x564a7af47bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/2175722760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af25dc0 0x564a7af28090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/2175722760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af3c4b0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2175722760 >> 172.21.15.1:0/2175722760 conn(0x564a7af21a30 msgr2=0x564a7af2f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2175722760 shutdown_connections 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.747+0000 7f8a6fb30740 1 -- 172.21.15.1:0/2175722760 wait complete. 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a6fb30740 1 -- start start 2024-09-16T00:44:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a6fb30740 1 Processor -- start 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af34930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a6fb30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af30680 0x564a7af34e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681ad4e0 con 0x564a7af254a0 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681ae950 con 0x564a7af30680 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a6fb30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681adbb0 con 0x564a7af25dc0 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a6fb30740 10 client.0 _ll_register_callbacks cb 0x7f8a5c00bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af343f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35446/0 (socket says 172.21.15.1:35446) 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a667fc640 1 -- 172.21.15.1:0/1643272548 learned_addr learned my addr 172.21.15.1:0/1643272548 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a667fc640 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af30680 0x564a7af34e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a667fc640 1 -- 172.21.15.1:0/1643272548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 msgr2=0x564a7af34930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a667fc640 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af34930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a667fc640 1 -- 172.21.15.1:0/1643272548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af30680 msgr2=0x564a7af34e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a667fc640 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af30680 0x564a7af34e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.748+0000 7f8a667fc640 1 -- 172.21.15.1:0/1643272548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7afa9bb0 con 0x564a7af254a0 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a667fc640 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af343f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c015040 tx=0x7f8a5c01ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8e036c60cfe0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a4aff5640 1 -- 172.21.15.1:0/1643272548 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c03a040 con 0x564a7af254a0 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a4aff5640 1 -- 172.21.15.1:0/1643272548 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8a5c010c80 con 0x564a7af254a0 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a6fb30740 1 -- 172.21.15.1:0/1643272548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a5c016040 con 0x564a7af254a0 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a4aff5640 1 -- 172.21.15.1:0/1643272548 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a5c02d040 con 0x564a7af254a0 2024-09-16T00:44:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a6fb30740 10 client.19149 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a6fb30740 1 -- 172.21.15.1:0/1643272548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a7afaae80 con 0x564a7af254a0 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a4aff5640 1 -- 172.21.15.1:0/1643272548 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c016040 con 0x564a7af254a0 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a4aff5640 1 -- 172.21.15.1:0/1643272548 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8a5c040070 con 0x564a7af254a0 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a6fb30740 10 client.19149 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a6fb30740 10 client.19149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a6fb30740 20 client.19149 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a7afab520 0x564a7afad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a6fb30740 4 client.19149 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.749+0000 7f8a6fb30740 1 -- 172.21.15.1:0/1643272548 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a7ae92d60 con 0x564a7afab520 2024-09-16T00:44:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a66ffd640 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a7afab520 0x564a7afad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a4aff5640 10 client.19149 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a66ffd640 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a7afab520 0x564a7afad910 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431775f96627d064 server_cookie=a2ebe893503727cc in_seq=0 out_seq=0 2024-09-16T00:44:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a4aff5640 1 -- 172.21.15.1:0/1643272548 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564a7ae92d60 con 0x564a7afab520 2024-09-16T00:44:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a4aff5640 10 client.19149 handle_command_reply: tid=0 2024-09-16T00:44:27.783 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:44:27.783 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:44:27.783 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:44:27.784 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:44:27.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a6fb30740 1 client.19149 shutdown 2024-09-16T00:44:27.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a6fb30740 1 -- 172.21.15.1:0/1643272548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 msgr2=0x564a7af343f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:27.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af343f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c015040 tx=0x7f8a5c01ffa0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a6fb30740 1 -- 172.21.15.1:0/1643272548 shutdown_connections 2024-09-16T00:44:27.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x564a7afab520 0x564a7afad910 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a7af30680 0x564a7af34e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a4aff5640 0 client.19149 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a7af25dc0 0x564a7af34930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a6fb30740 1 --2- 172.21.15.1:0/1643272548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a7af254a0 0x564a7af343f0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.750+0000 7f8a6fb30740 1 -- 172.21.15.1:0/1643272548 >> 172.21.15.1:0/1643272548 conn(0x564a7af21a30 msgr2=0x564a7af2f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6fb30740 1 -- 172.21.15.1:0/1643272548 shutdown_connections 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6fb30740 1 -- 172.21.15.1:0/1643272548 wait complete. 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6fb30740 20 client.19149 trim_cache size 0 max 16384 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 -- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a500a3240 msgr2=0x7f8a500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a500a3240 0x7f8a500a5700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8a600b8830 tx=0x7f8a600872b0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 -- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 msgr2=0x7f8a6819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 0x7f8a6819ce60 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f8a60005ab0 tx=0x7f8a600042f0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 -- 172.21.15.1:0/1314179458 shutdown_connections 2024-09-16T00:44:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a68108300 0x7f8a6819ce60 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8a500a3240 0x7f8a500a5700 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a68103780 0x7f8a68198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 --2- 172.21.15.1:0/1314179458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68102dd0 0x7f8a68198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 -- 172.21.15.1:0/1314179458 >> 172.21.15.1:0/1314179458 conn(0x7f8a680fc7d0 msgr2=0x7f8a68105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 -- 172.21.15.1:0/1314179458 shutdown_connections 2024-09-16T00:44:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:27.751+0000 7f8a6d757640 1 -- 172.21.15.1:0/1314179458 wait complete. 2024-09-16T00:44:28.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:27 smithi001 ceph-mon[30934]: pgmap v832: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-16T00:44:28.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:27 smithi153 ceph-mon[35538]: pgmap v832: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-16T00:44:28.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:27 smithi019 ceph-mon[35186]: pgmap v832: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-16T00:44:28.560 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:44:28.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:44:28.823 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:29.208 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:44:29.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.342+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/1168737696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 msgr2=0x7f8bb010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:29.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.342+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/1168737696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 0x7f8bb010e790 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009ca0 tx=0x7f8ba802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.343+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/1168737696 shutdown_connections 2024-09-16T00:44:29.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.343+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/1168737696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 0x7f8bb010e790 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.343+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/1168737696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bb01037a0 0x7f8bb010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.343+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/1168737696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bb0102df0 0x7f8bb01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.343+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/1168737696 >> 172.21.15.1:0/1168737696 conn(0x7f8bb00fc7d0 msgr2=0x7f8bb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:29.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.343+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/1168737696 shutdown_connections 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.343+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/1168737696 wait complete. 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.343+0000 7f8bb7c5a640 1 Processor -- start 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb7c5a640 1 -- start start 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb7c5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bb0102df0 0x7f8bb0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb7c5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bb01037a0 0x7f8bb0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb7c5a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 0x7f8bb019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb7c5a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01120f0 con 0x7f8bb010c2f0 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb7c5a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0111f70 con 0x7f8bb0102df0 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb7c5a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0112270 con 0x7f8bb01037a0 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb61f7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 0x7f8bb019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb61f7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 0x7f8bb019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35476/0 (socket says 172.21.15.1:35476) 2024-09-16T00:44:29.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb61f7640 1 -- 172.21.15.1:0/173398208 learned_addr learned my addr 172.21.15.1:0/173398208 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb51f5640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bb01037a0 0x7f8bb0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb59f6640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bb0102df0 0x7f8bb0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb61f7640 1 -- 172.21.15.1:0/173398208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bb01037a0 msgr2=0x7f8bb0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb61f7640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bb01037a0 0x7f8bb0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb61f7640 1 -- 172.21.15.1:0/173398208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bb0102df0 msgr2=0x7f8bb0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb61f7640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bb0102df0 0x7f8bb0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb61f7640 1 -- 172.21.15.1:0/173398208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb019d4c0 con 0x7f8bb010c2f0 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb59f6640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bb0102df0 0x7f8bb0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.344+0000 7f8bb51f5640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bb01037a0 0x7f8bb0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.345+0000 7f8bb61f7640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 0x7f8bb019cde0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8031a20 tx=0x7f8ba8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.345+0000 7f8ba6ffd640 1 -- 172.21.15.1:0/173398208 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8003c30 con 0x7f8bb010c2f0 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.345+0000 7f8ba6ffd640 1 -- 172.21.15.1:0/173398208 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba8003dd0 con 0x7f8bb010c2f0 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.345+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb019d750 con 0x7f8bb010c2f0 2024-09-16T00:44:29.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.345+0000 7f8ba6ffd640 1 -- 172.21.15.1:0/173398208 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba802fcf0 con 0x7f8bb010c2f0 2024-09-16T00:44:29.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.345+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb01a5090 con 0x7f8bb010c2f0 2024-09-16T00:44:29.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.346+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb01031d0 con 0x7f8bb010c2f0 2024-09-16T00:44:29.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.347+0000 7f8ba6ffd640 1 -- 172.21.15.1:0/173398208 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8ba8004050 con 0x7f8bb010c2f0 2024-09-16T00:44:29.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.348+0000 7f8ba6ffd640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8b9c0a32c0 0x7f8b9c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:29.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.348+0000 7f8bb59f6640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8b9c0a32c0 0x7f8b9c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:29.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.348+0000 7f8ba6ffd640 1 -- 172.21.15.1:0/173398208 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8ba80f1250 con 0x7f8bb010c2f0 2024-09-16T00:44:29.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.348+0000 7f8bb59f6640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8b9c0a32c0 0x7f8b9c0a5780 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009b40 tx=0x7f8bac005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:29.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.351+0000 7f8ba6ffd640 1 -- 172.21.15.1:0/173398208 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ba80bdc00 con 0x7f8bb010c2f0 2024-09-16T00:44:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:29.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.584+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bb01a5260 con 0x7f8bb010c2f0 2024-09-16T00:44:29.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.585+0000 7f8ba6ffd640 1 -- 172.21.15.1:0/173398208 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f8ba802fe90 con 0x7f8bb010c2f0 2024-09-16T00:44:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:44:29.585 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:44:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8b9c0a32c0 msgr2=0x7f8b9c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8b9c0a32c0 0x7f8b9c0a5780 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009b40 tx=0x7f8bac005e50 comp rx=0 tx=0).stop 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 msgr2=0x7f8bb019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 0x7f8bb019cde0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8031a20 tx=0x7f8ba8031a50 comp rx=0 tx=0).stop 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 shutdown_connections 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb010c2f0 0x7f8bb019cde0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8b9c0a32c0 0x7f8b9c0a5780 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bb01037a0 0x7f8bb0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 --2- 172.21.15.1:0/173398208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bb0102df0 0x7f8bb0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 >> 172.21.15.1:0/173398208 conn(0x7f8bb00fc7d0 msgr2=0x7f8bb00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 shutdown_connections 2024-09-16T00:44:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:29.587+0000 7f8bb7c5a640 1 -- 172.21.15.1:0/173398208 wait complete. 2024-09-16T00:44:30.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:29 smithi001 ceph-mon[30934]: pgmap v833: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:44:30.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:29 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/173398208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:30.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:29 smithi153 ceph-mon[35538]: pgmap v833: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:44:30.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:29 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/173398208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:29 smithi019 ceph-mon[35186]: pgmap v833: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:44:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:29 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/173398208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:30.527 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 -- 172.21.15.1:0/3929230251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8102570 msgr2=0x7f96a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3929230251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8102570 0x7f96a81029b0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009900 tx=0x7f96a402a480 comp rx=0 tx=0).stop 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 -- 172.21.15.1:0/3929230251 shutdown_connections 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3929230251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8102ef0 0x7f96a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3929230251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8102570 0x7f96a81029b0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3929230251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a8100e20 0x7f96a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 -- 172.21.15.1:0/3929230251 >> 172.21.15.1:0/3929230251 conn(0x7f96a80fc7d0 msgr2=0x7f96a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 -- 172.21.15.1:0/3929230251 shutdown_connections 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 -- 172.21.15.1:0/3929230251 wait complete. 2024-09-16T00:44:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.702+0000 7f96af4c2640 1 Processor -- start 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96af4c2640 1 -- start start 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96af4c2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8100e20 0x7f96a81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96af4c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a8102570 0x7f96a8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96af4c2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8102ef0 0x7f96a819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96af4c2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8110f80 con 0x7f96a8100e20 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96af4c2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8110e00 con 0x7f96a8102570 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96af4c2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8111100 con 0x7f96a8102ef0 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ad25e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8100e20 0x7f96a81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ad25e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8100e20 0x7f96a81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36688/0 (socket says 172.21.15.1:36688) 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ad25e640 1 -- 172.21.15.1:0/3641361599 learned_addr learned my addr 172.21.15.1:0/3641361599 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96aca5d640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a8102570 0x7f96a8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ada5f640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8102ef0 0x7f96a819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ad25e640 1 -- 172.21.15.1:0/3641361599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8102ef0 msgr2=0x7f96a819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ad25e640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8102ef0 0x7f96a819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ad25e640 1 -- 172.21.15.1:0/3641361599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a8102570 msgr2=0x7f96a8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ad25e640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a8102570 0x7f96a8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ad25e640 1 -- 172.21.15.1:0/3641361599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a819d480 con 0x7f96a8100e20 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.703+0000 7f96ad25e640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8100e20 0x7f96a81984d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f969c00d950 tx=0x7f969c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.704+0000 7f969a7fc640 1 -- 172.21.15.1:0/3641361599 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c017030 con 0x7f96a8100e20 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.704+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a819d770 con 0x7f96a8100e20 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.704+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a81a5050 con 0x7f96a8100e20 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.704+0000 7f969a7fc640 1 -- 172.21.15.1:0/3641361599 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969c004020 con 0x7f96a8100e20 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.704+0000 7f969a7fc640 1 -- 172.21.15.1:0/3641361599 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0047a0 con 0x7f96a8100e20 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.705+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a8101200 con 0x7f96a8100e20 2024-09-16T00:44:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.706+0000 7f969a7fc640 1 -- 172.21.15.1:0/3641361599 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f969c0041d0 con 0x7f96a8100e20 2024-09-16T00:44:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.707+0000 7f969a7fc640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f96940a32c0 0x7f96940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.707+0000 7f969a7fc640 1 -- 172.21.15.1:0/3641361599 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f969c0cfdd0 con 0x7f96a8100e20 2024-09-16T00:44:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.707+0000 7f96aca5d640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f96940a32c0 0x7f96940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.707+0000 7f96aca5d640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f96940a32c0 0x7f96940a5780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009230 tx=0x7f96a40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.710+0000 7f969a7fc640 1 -- 172.21.15.1:0/3641361599 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f969c09c770 con 0x7f96a8100e20 2024-09-16T00:44:30.797 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:30.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.939+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96a8199830 con 0x7f96a8100e20 2024-09-16T00:44:30.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.939+0000 7f969a7fc640 1 -- 172.21.15.1:0/3641361599 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f96a8199830 con 0x7f96a8100e20 2024-09-16T00:44:30.940 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:44:30.940 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:44:30.940 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f96940a32c0 msgr2=0x7f96940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f96940a32c0 0x7f96940a5780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009230 tx=0x7f96a40058d0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8100e20 msgr2=0x7f96a81984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8100e20 0x7f96a81984d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f969c00d950 tx=0x7f969c00de10 comp rx=0 tx=0).stop 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 shutdown_connections 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f96940a32c0 0x7f96940a5780 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8102ef0 0x7f96a819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a8102570 0x7f96a8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 --2- 172.21.15.1:0/3641361599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8100e20 0x7f96a81984d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 >> 172.21.15.1:0/3641361599 conn(0x7f96a80fc7d0 msgr2=0x7f96a806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.941+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 shutdown_connections 2024-09-16T00:44:30.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:30.942+0000 7f96af4c2640 1 -- 172.21.15.1:0/3641361599 wait complete. 2024-09-16T00:44:31.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.495+0000 7f78241e6640 1 -- 172.21.15.1:0/840894427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c101de0 msgr2=0x7f781c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:31.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.495+0000 7f78241e6640 1 --2- 172.21.15.1:0/840894427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c101de0 0x7f781c1021c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7818009900 tx=0x7f781802a410 comp rx=0 tx=0).stop 2024-09-16T00:44:31.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.495+0000 7f78241e6640 1 -- 172.21.15.1:0/840894427 shutdown_connections 2024-09-16T00:44:31.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.495+0000 7f78241e6640 1 --2- 172.21.15.1:0/840894427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c106f10 0x7f781c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.495+0000 7f78241e6640 1 --2- 172.21.15.1:0/840894427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f781c102790 0x7f781c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.495+0000 7f78241e6640 1 --2- 172.21.15.1:0/840894427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c101de0 0x7f781c1021c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.495+0000 7f78241e6640 1 -- 172.21.15.1:0/840894427 >> 172.21.15.1:0/840894427 conn(0x7f781c0fd8c0 msgr2=0x7f781c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:31.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 -- 172.21.15.1:0/840894427 shutdown_connections 2024-09-16T00:44:31.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 -- 172.21.15.1:0/840894427 wait complete. 2024-09-16T00:44:31.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 Processor -- start 2024-09-16T00:44:31.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 -- start start 2024-09-16T00:44:31.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101de0 0x7f781c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f781c102790 0x7f781c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c106f10 0x7f781c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f781c111f70 con 0x7f781c106f10 2024-09-16T00:44:31.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f781c111df0 con 0x7f781c102790 2024-09-16T00:44:31.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.496+0000 7f78241e6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1120f0 con 0x7f781c101de0 2024-09-16T00:44:31.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821781640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f781c102790 0x7f781c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821781640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f781c102790 0x7f781c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:39532/0 (socket says 172.21.15.1:39532) 2024-09-16T00:44:31.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821781640 1 -- 172.21.15.1:0/3222098708 learned_addr learned my addr 172.21.15.1:0/3222098708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:31.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821f82640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101de0 0x7f781c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7822783640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c106f10 0x7f781c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821781640 1 -- 172.21.15.1:0/3222098708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101de0 msgr2=0x7f781c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:31.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821781640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101de0 0x7f781c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821781640 1 -- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c106f10 msgr2=0x7f781c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:31.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821781640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c106f10 0x7f781c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821781640 1 -- 172.21.15.1:0/3222098708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c19f6a0 con 0x7f781c102790 2024-09-16T00:44:31.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7821781640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f781c102790 0x7f781c19ac30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f780c00e9f0 tx=0x7f780c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:31.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7812ffd640 1 -- 172.21.15.1:0/3222098708 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c00cd80 con 0x7f781c102790 2024-09-16T00:44:31.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7812ffd640 1 -- 172.21.15.1:0/3222098708 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c010910 con 0x7f781c102790 2024-09-16T00:44:31.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f78241e6640 1 -- 172.21.15.1:0/3222098708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c19f990 con 0x7f781c102790 2024-09-16T00:44:31.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f7812ffd640 1 -- 172.21.15.1:0/3222098708 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c018990 con 0x7f781c102790 2024-09-16T00:44:31.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.497+0000 7f78241e6640 1 -- 172.21.15.1:0/3222098708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c1a7270 con 0x7f781c102790 2024-09-16T00:44:31.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.500+0000 7f7812ffd640 1 -- 172.21.15.1:0/3222098708 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f780c018b30 con 0x7f781c102790 2024-09-16T00:44:31.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.500+0000 7f7812ffd640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f78080a3240 0x7f78080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.500+0000 7f78265bf740 1 Processor -- start 2024-09-16T00:44:31.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.500+0000 7f7821f82640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f78080a3240 0x7f78080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.500+0000 7f7812ffd640 1 -- 172.21.15.1:0/3222098708 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f780c0d0070 con 0x7f781c102790 2024-09-16T00:44:31.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f78265bf740 1 -- start start 2024-09-16T00:44:31.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f78265bf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0ccb40 0x560d4c0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f78265bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0cb5d0 0x560d4c0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f78265bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 0x560d4c0d8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f78265bf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d4c0312f0 con 0x560d4c0ccb40 2024-09-16T00:44:31.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f78265bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560d4bfd8cf0 con 0x560d4c0d6370 2024-09-16T00:44:31.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f78265bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560d4c02d2b0 con 0x560d4c0cb5d0 2024-09-16T00:44:31.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821f82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0ccb40 0x560d4c0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821f82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0ccb40 0x560d4c0cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36704/0 (socket says 172.21.15.1:36704) 2024-09-16T00:44:31.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821f82640 1 -- 172.21.15.1:0/698067378 learned_addr learned my addr 172.21.15.1:0/698067378 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:31.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821f82640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f78080a3240 0x7f78080a5700 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7818009230 tx=0x7f78180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:31.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7822783640 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 0x560d4c0d8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821781640 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0cb5d0 0x560d4c0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821f82640 1 -- 172.21.15.1:0/698067378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0cb5d0 msgr2=0x560d4c0cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:31.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821f82640 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0cb5d0 0x560d4c0cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821f82640 1 -- 172.21.15.1:0/698067378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 msgr2=0x560d4c0d8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:31.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821f82640 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 0x560d4c0d8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7821f82640 1 -- 172.21.15.1:0/698067378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4bfbd3a0 con 0x560d4c0ccb40 2024-09-16T00:44:31.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.501+0000 7f7822783640 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 0x560d4c0d8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:31.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.502+0000 7f7821f82640 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0ccb40 0x560d4c0cb090 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f7818033000 tx=0x7f781803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6207bdaa4c4c7df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:31.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.502+0000 7f7807fff640 1 -- 172.21.15.1:0/698067378 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781804d070 con 0x560d4c0ccb40 2024-09-16T00:44:31.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.502+0000 7f7807fff640 1 -- 172.21.15.1:0/698067378 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78180043c0 con 0x560d4c0ccb40 2024-09-16T00:44:31.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.502+0000 7f7807fff640 1 -- 172.21.15.1:0/698067378 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818035040 con 0x560d4c0ccb40 2024-09-16T00:44:31.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 -- 172.21.15.1:0/698067378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0ccb40 msgr2=0x560d4c0cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:31.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0ccb40 0x560d4c0cb090 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f7818033000 tx=0x7f781803d000 comp rx=0 tx=0).stop 2024-09-16T00:44:31.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 -- 172.21.15.1:0/698067378 shutdown_connections 2024-09-16T00:44:31.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 0x560d4c0d8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0cb5d0 0x560d4c0cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 --2- 172.21.15.1:0/698067378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0ccb40 0x560d4c0cb090 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 -- 172.21.15.1:0/698067378 >> 172.21.15.1:0/698067378 conn(0x560d4c094310 msgr2=0x560d4c0c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:31.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 -- 172.21.15.1:0/698067378 shutdown_connections 2024-09-16T00:44:31.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 -- 172.21.15.1:0/698067378 wait complete. 2024-09-16T00:44:31.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 -- start start 2024-09-16T00:44:31.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.503+0000 7f78265bf740 1 Processor -- start 2024-09-16T00:44:31.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f78265bf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f78265bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0ccb40 0x560d4c0d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f78265bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 0x560d4c0d34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f78265bf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d4c0d9010 con 0x560d4c0cb5d0 2024-09-16T00:44:31.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f78265bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560d4bfec060 con 0x560d4c0d6370 2024-09-16T00:44:31.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f78265bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560d4c0d9190 con 0x560d4c0ccb40 2024-09-16T00:44:31.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f78265bf740 10 client.0 _ll_register_callbacks cb 0x560d4bd3f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:31.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821f82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821f82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0d5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36706/0 (socket says 172.21.15.1:36706) 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821f82640 1 -- 172.21.15.1:0/3342174781 learned_addr learned my addr 172.21.15.1:0/3342174781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7822783640 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 0x560d4c0d34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821781640 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0ccb40 0x560d4c0d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821f82640 1 -- 172.21.15.1:0/3342174781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0ccb40 msgr2=0x560d4c0d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821f82640 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0ccb40 0x560d4c0d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821f82640 1 -- 172.21.15.1:0/3342174781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 msgr2=0x560d4c0d34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821f82640 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 0x560d4c0d34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821f82640 1 -- 172.21.15.1:0/3342174781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4bda23c0 con 0x560d4c0cb5d0 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f7821f82640 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0d5ce0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f7818004770 tx=0x7f7818049870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931305283f25999d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.504+0000 7f78057fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:31.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78057fa640 1 -- 172.21.15.1:0/3342174781 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781805a020 con 0x560d4c0cb5d0 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78057fa640 1 -- 172.21.15.1:0/3342174781 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f7818049c70 con 0x560d4c0cb5d0 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78265bf740 1 -- 172.21.15.1:0/3342174781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78180043c0 con 0x560d4c0cb5d0 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78057fa640 1 -- 172.21.15.1:0/3342174781 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7818065050 con 0x560d4c0cb5d0 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78265bf740 10 client.19173 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78265bf740 1 -- 172.21.15.1:0/3342174781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d4c0e4a20 con 0x560d4c0cb5d0 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78057fa640 1 -- 172.21.15.1:0/3342174781 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78180043c0 con 0x560d4c0cb5d0 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78057fa640 1 -- 172.21.15.1:0/3342174781 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f78180527b0 con 0x560d4c0cb5d0 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78265bf740 10 client.19173 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78265bf740 10 client.19173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78265bf740 20 client.19173 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78265bf740 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x560d4c0e5d70 0x560d4c0e8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78265bf740 4 client.19173 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.505+0000 7f78265bf740 1 -- 172.21.15.1:0/3342174781 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d4c03e860 con 0x560d4c0e5d70 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.506+0000 7f7821781640 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x560d4c0e5d70 0x560d4c0e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.506+0000 7f7821781640 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x560d4c0e5d70 0x560d4c0e8160 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a82e65a15b6d7d3d server_cookie=4fc8aea741638b5b in_seq=0 out_seq=0 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.506+0000 7f78057fa640 10 client.19173 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78057fa640 1 -- 172.21.15.1:0/3342174781 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d4c03e860 con 0x560d4c0e5d70 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78057fa640 10 client.19173 handle_command_reply: tid=0 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 client.19173 shutdown 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 -- 172.21.15.1:0/3342174781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 msgr2=0x560d4c0d5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0d5ce0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f7818004770 tx=0x7f7818049870 comp rx=0 tx=0).stop 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 -- 172.21.15.1:0/3342174781 shutdown_connections 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x560d4c0e5d70 0x560d4c0e8160 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0d6370 0x560d4c0d34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0ccb40 0x560d4c0d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 --2- 172.21.15.1:0/3342174781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0d5ce0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78057fa640 0 client.19173 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:44:31.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 -- 172.21.15.1:0/3342174781 >> 172.21.15.1:0/3342174781 conn(0x560d4c094310 msgr2=0x560d4c0cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:31.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 -- 172.21.15.1:0/3342174781 shutdown_connections 2024-09-16T00:44:31.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 1 -- 172.21.15.1:0/3342174781 wait complete. 2024-09-16T00:44:31.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.507+0000 7f78265bf740 20 client.19173 trim_cache size 0 max 16384 2024-09-16T00:44:31.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.520+0000 7f78265bf740 1 Processor -- start 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f78265bf740 1 -- start start 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f78265bf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f78265bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f78265bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0d6370 0x560d4c0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f78265bf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1b3a50 con 0x560d4c0cb5d0 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f78265bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1b1320 con 0x560d4c0ccb40 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f78265bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1af170 con 0x560d4c0d6370 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7821f82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7821781640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7821f82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0e22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36718/0 (socket says 172.21.15.1:36718) 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7821f82640 1 -- 172.21.15.1:0/1398323471 learned_addr learned my addr 172.21.15.1:0/1398323471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:31.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7822783640 1 --2- 172.21.15.1:0/1398323471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0d6370 0x560d4c0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7821781640 1 -- 172.21.15.1:0/1398323471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0d6370 msgr2=0x560d4c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7821781640 1 --2- 172.21.15.1:0/1398323471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0d6370 0x560d4c0ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7821781640 1 -- 172.21.15.1:0/1398323471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 msgr2=0x560d4c0e22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7821781640 1 --2- 172.21.15.1:0/1398323471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0e22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7821781640 1 -- 172.21.15.1:0/1398323471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4c0d3860 con 0x560d4c0ccb40 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.521+0000 7f7822783640 1 --2- 172.21.15.1:0/1398323471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0d6370 0x560d4c0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f7821781640 1 --2- 172.21.15.1:0/1398323471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0ce120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f780c0cd770 tx=0x7f780c012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c60c32417d422c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78077fe640 1 -- 172.21.15.1:0/1398323471 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c09a640 con 0x560d4c0ccb40 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78077fe640 1 -- 172.21.15.1:0/1398323471 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c010430 con 0x560d4c0ccb40 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78077fe640 1 -- 172.21.15.1:0/1398323471 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c0a4cd0 con 0x560d4c0ccb40 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78265bf740 1 -- 172.21.15.1:0/1398323471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 msgr2=0x560d4c0ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78265bf740 1 --2- 172.21.15.1:0/1398323471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0ce120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f780c0cd770 tx=0x7f780c012040 comp rx=0 tx=0).stop 2024-09-16T00:44:31.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78265bf740 1 -- 172.21.15.1:0/1398323471 shutdown_connections 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78265bf740 1 --2- 172.21.15.1:0/1398323471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0d6370 0x560d4c0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78265bf740 1 --2- 172.21.15.1:0/1398323471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0ce120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78265bf740 1 --2- 172.21.15.1:0/1398323471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0cb5d0 0x560d4c0e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78265bf740 1 -- 172.21.15.1:0/1398323471 >> 172.21.15.1:0/1398323471 conn(0x560d4c094310 msgr2=0x560d4c0d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.522+0000 7f78265bf740 1 -- 172.21.15.1:0/1398323471 shutdown_connections 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 1 -- 172.21.15.1:0/1398323471 wait complete. 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 1 -- start start 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 1 Processor -- start 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0cb5d0 0x560d4c0cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0d6370 0x560d4c0d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1af7e0 con 0x560d4c0d6370 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1b0c50 con 0x560d4c0ccb40 2024-09-16T00:44:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1afeb0 con 0x560d4c0cb5d0 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f78265bf740 10 client.0 _ll_register_callbacks cb 0x7f7818009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f7822783640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0d6370 0x560d4c0d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f7821781640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.523+0000 7f7822783640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0d6370 0x560d4c0d09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36720/0 (socket says 172.21.15.1:36720) 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f7822783640 1 -- 172.21.15.1:0/3325582325 learned_addr learned my addr 172.21.15.1:0/3325582325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f7821781640 1 -- 172.21.15.1:0/3325582325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0cb5d0 msgr2=0x560d4c0cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f7821781640 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0cb5d0 0x560d4c0cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f7821781640 1 -- 172.21.15.1:0/3325582325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0d6370 msgr2=0x560d4c0d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f7821781640 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0d6370 0x560d4c0d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f7821781640 1 -- 172.21.15.1:0/3325582325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4c14f6d0 con 0x560d4c0ccb40 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f7822783640 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0d6370 0x560d4c0d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f7821781640 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0d0480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f780c0cd770 tx=0x7f780c0105e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eb8fe768bb5b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f78057fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f78057fa640 1 -- 172.21.15.1:0/3325582325 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c0a4a70 con 0x560d4c0ccb40 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f78057fa640 1 -- 172.21.15.1:0/3325582325 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f780c0a4c10 con 0x560d4c0ccb40 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f78265bf740 1 -- 172.21.15.1:0/3325582325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780c010430 con 0x560d4c0ccb40 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f78057fa640 1 -- 172.21.15.1:0/3325582325 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f780c0a0170 con 0x560d4c0ccb40 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f78265bf740 10 client.27977 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.524+0000 7f78265bf740 1 -- 172.21.15.1:0/3325582325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d4c150960 con 0x560d4c0ccb40 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.525+0000 7f78057fa640 1 -- 172.21.15.1:0/3325582325 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c010430 con 0x560d4c0ccb40 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.525+0000 7f78057fa640 1 -- 172.21.15.1:0/3325582325 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f780c09b790 con 0x560d4c0ccb40 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.525+0000 7f78265bf740 10 client.27977 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.525+0000 7f78265bf740 10 client.27977 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.525+0000 7f78265bf740 20 client.27977 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.525+0000 7f78265bf740 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x560d4c150fc0 0x560d4c1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.525+0000 7f78265bf740 4 client.27977 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:44:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.525+0000 7f78265bf740 1 -- 172.21.15.1:0/3325582325 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d4c038d70 con 0x560d4c150fc0 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.525+0000 7f7821f82640 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x560d4c150fc0 0x560d4c1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78057fa640 10 client.27977 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f7821f82640 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x560d4c150fc0 0x560d4c1533e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=287f8bc6f965e95d server_cookie=17b6f6917b101489 in_seq=0 out_seq=0 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78057fa640 1 -- 172.21.15.1:0/3325582325 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d4c038d70 con 0x560d4c150fc0 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78057fa640 10 client.27977 handle_command_reply: tid=0 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 client.27977 shutdown 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 -- 172.21.15.1:0/3325582325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 msgr2=0x560d4c0d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0d0480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f780c0cd770 tx=0x7f780c0105e0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 -- 172.21.15.1:0/3325582325 shutdown_connections 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d4c0d6370 0x560d4c0d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x560d4c150fc0 0x560d4c1533e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d4c0ccb40 0x560d4c0d0480 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 --2- 172.21.15.1:0/3325582325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c0cb5d0 0x560d4c0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78057fa640 0 client.27977 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:44:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 -- 172.21.15.1:0/3325582325 >> 172.21.15.1:0/3325582325 conn(0x560d4c094310 msgr2=0x560d4c0d47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 -- 172.21.15.1:0/3325582325 shutdown_connections 2024-09-16T00:44:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 1 -- 172.21.15.1:0/3325582325 wait complete. 2024-09-16T00:44:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.526+0000 7f78265bf740 20 client.27977 trim_cache size 0 max 16384 2024-09-16T00:44:31.536 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:44:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 -- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f78080a3240 msgr2=0x7f78080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f78080a3240 0x7f78080a5700 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7818009230 tx=0x7f78180058d0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 -- 172.21.15.1:0/3222098708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f781c102790 msgr2=0x7f781c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f781c102790 0x7f781c19ac30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f780c00e9f0 tx=0x7f780c00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 -- 172.21.15.1:0/3222098708 shutdown_connections 2024-09-16T00:44:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c106f10 0x7f781c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f781c102790 0x7f781c19ac30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f78080a3240 0x7f78080a5700 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 --2- 172.21.15.1:0/3222098708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101de0 0x7f781c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:31.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 -- 172.21.15.1:0/3222098708 >> 172.21.15.1:0/3222098708 conn(0x7f781c0fd8c0 msgr2=0x7f781c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:31.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 -- 172.21.15.1:0/3222098708 shutdown_connections 2024-09-16T00:44:31.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:31.527+0000 7f78241e6640 1 -- 172.21.15.1:0/3222098708 wait complete. 2024-09-16T00:44:32.150 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 44509eca --release --await' 2024-09-16T00:44:32.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --set-id 44509eca --release --await 2024-09-16T00:44:32.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:31 smithi001 ceph-mon[30934]: pgmap v834: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:44:32.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:31 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3641361599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:32.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:44:32.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:44:32.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:44:32.166 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:31 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:44:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:31 smithi153 ceph-mon[35538]: pgmap v834: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:44:32.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:31 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3641361599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:32.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:44:32.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:44:32.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:44:32.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:31 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:44:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:31 smithi019 ceph-mon[35186]: pgmap v834: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:44:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:31 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3641361599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:44:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:44:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:44:32.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:31 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:44:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:32.913 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:44:33.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.438+0000 7ff6690f0640 1 -- 172.21.15.1:0/2535675571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664102790 msgr2=0x7ff6641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:33.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.438+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2535675571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664102790 0x7ff6641067e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009900 tx=0x7ff64c02a410 comp rx=0 tx=0).stop 2024-09-16T00:44:33.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.438+0000 7ff6690f0640 1 -- 172.21.15.1:0/2535675571 shutdown_connections 2024-09-16T00:44:33.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.438+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2535675571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff664106f10 0x7ff66410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.438+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2535675571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664102790 0x7ff6641067e0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.438+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2535675571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff664101de0 0x7ff6641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.438+0000 7ff6690f0640 1 -- 172.21.15.1:0/2535675571 >> 172.21.15.1:0/2535675571 conn(0x7ff6640fd8c0 msgr2=0x7ff6640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.438+0000 7ff6690f0640 1 -- 172.21.15.1:0/2535675571 shutdown_connections 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.438+0000 7ff6690f0640 1 -- 172.21.15.1:0/2535675571 wait complete. 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff6690f0640 1 Processor -- start 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff6690f0640 1 -- start start 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff6690f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664101de0 0x7ff664198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff6690f0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff664102790 0x7ff6641989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff6690f0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff664106f10 0x7ff66419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff6690f0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff664111f90 con 0x7ff664101de0 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff6690f0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff664111e10 con 0x7ff664102790 2024-09-16T00:44:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff6690f0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff664112110 con 0x7ff664106f10 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff662d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664101de0 0x7ff664198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff662d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664101de0 0x7ff664198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36748/0 (socket says 172.21.15.1:36748) 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff662d9d640 1 -- 172.21.15.1:0/2687525600 learned_addr learned my addr 172.21.15.1:0/2687525600 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff65bfff640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff664102790 0x7ff6641989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff662d9d640 1 -- 172.21.15.1:0/2687525600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff664106f10 msgr2=0x7ff66419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff662d9d640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff664106f10 0x7ff66419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff662d9d640 1 -- 172.21.15.1:0/2687525600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff664102790 msgr2=0x7ff6641989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff662d9d640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff664102790 0x7ff6641989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.439+0000 7ff662d9d640 1 -- 172.21.15.1:0/2687525600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66419d440 con 0x7ff664101de0 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.440+0000 7ff662d9d640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664101de0 0x7ff664198490 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c890 tx=0x7ff65400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.440+0000 7ff660d99640 1 -- 172.21.15.1:0/2687525600 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654004220 con 0x7ff664101de0 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.440+0000 7ff6690f0640 1 -- 172.21.15.1:0/2687525600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66419d730 con 0x7ff664101de0 2024-09-16T00:44:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.440+0000 7ff660d99640 1 -- 172.21.15.1:0/2687525600 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6540043c0 con 0x7ff664101de0 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.440+0000 7ff660d99640 1 -- 172.21.15.1:0/2687525600 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65400f450 con 0x7ff664101de0 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.440+0000 7ff6690f0640 1 -- 172.21.15.1:0/2687525600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6641a5010 con 0x7ff664101de0 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.442+0000 7ff660d99640 1 -- 172.21.15.1:0/2687525600 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff6540079b0 con 0x7ff664101de0 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.442+0000 7ff66b4c9740 1 Processor -- start 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.442+0000 7ff66b4c9740 1 -- start start 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 0x558a38212440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 0x558a3821cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a3821f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558a38177620 con 0x558a3821d240 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a3811ee50 con 0x558a38212980 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a381735e0 con 0x558a38212060 2024-09-16T00:44:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a3821f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff662d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 0x558a3821cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a3821f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36764/0 (socket says 172.21.15.1:36764) 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff65bfff640 1 -- 172.21.15.1:0/91580701 learned_addr learned my addr 172.21.15.1:0/91580701 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff66359e640 1 --2- 172.21.15.1:0/91580701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 0x558a38212440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff660d99640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff6500a3240 0x7ff6500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff65bfff640 1 -- 172.21.15.1:0/91580701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 msgr2=0x558a38212440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff65bfff640 1 --2- 172.21.15.1:0/91580701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 0x558a38212440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff65bfff640 1 -- 172.21.15.1:0/91580701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 msgr2=0x558a3821cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff65bfff640 1 --2- 172.21.15.1:0/91580701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 0x558a3821cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff65bfff640 1 -- 172.21.15.1:0/91580701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a3810fd40 con 0x558a3821d240 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff66359e640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff6500a3240 0x7ff6500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.443+0000 7ff660d99640 1 -- 172.21.15.1:0/2687525600 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff6540cfae0 con 0x7ff664101de0 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.444+0000 7ff65bfff640 1 --2- 172.21.15.1:0/91580701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a3821f630 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff64c005ab0 tx=0x7ff64c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4181828e35da95 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.444+0000 7ff6597fa640 1 -- 172.21.15.1:0/91580701 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c03f070 con 0x558a3821d240 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.444+0000 7ff6597fa640 1 -- 172.21.15.1:0/91580701 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64c031a20 con 0x558a3821d240 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.444+0000 7ff6597fa640 1 -- 172.21.15.1:0/91580701 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c035070 con 0x558a3821d240 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 -- 172.21.15.1:0/91580701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 msgr2=0x558a3821f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/91580701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a3821f630 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff64c005ab0 tx=0x7ff64c031630 comp rx=0 tx=0).stop 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66359e640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff6500a3240 0x7ff6500a5700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009e80 tx=0x7ff65c004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 -- 172.21.15.1:0/91580701 shutdown_connections 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/91580701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a3821f630 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/91580701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 0x558a3821cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/91580701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 0x558a38212440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 -- 172.21.15.1:0/91580701 >> 172.21.15.1:0/91580701 conn(0x558a3820e5f0 msgr2=0x558a3820e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 -- 172.21.15.1:0/91580701 shutdown_connections 2024-09-16T00:44:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 -- 172.21.15.1:0/91580701 wait complete. 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 -- start start 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 Processor -- start 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212060 0x558a3821a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 0x558a3821a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a3821d240 0x558a3821ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558a3821fe10 con 0x558a38212060 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a380fff10 con 0x558a38212980 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a3821ff90 con 0x558a3821d240 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff66b4c9740 10 client.0 _ll_register_callbacks cb 0x558a38123db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.445+0000 7ff662d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212060 0x558a3821a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff662d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212060 0x558a3821a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36778/0 (socket says 172.21.15.1:36778) 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff662d9d640 1 -- 172.21.15.1:0/1186098322 learned_addr learned my addr 172.21.15.1:0/1186098322 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff65bfff640 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 0x558a3821a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff66359e640 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a3821d240 0x558a3821ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff662d9d640 1 -- 172.21.15.1:0/1186098322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a3821d240 msgr2=0x558a3821ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff662d9d640 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a3821d240 0x558a3821ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff662d9d640 1 -- 172.21.15.1:0/1186098322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 msgr2=0x558a3821a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff662d9d640 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 0x558a3821a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff662d9d640 1 -- 172.21.15.1:0/1186098322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a37e86580 con 0x558a38212060 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff662d9d640 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212060 0x558a3821a0a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff654007590 tx=0x7ff65400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8d2d681d0e6b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff64a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff64a7fc640 1 -- 172.21.15.1:0/1186098322 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65409b660 con 0x558a38212060 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff64a7fc640 1 -- 172.21.15.1:0/1186098322 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff65409a000 con 0x558a38212060 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff66b4c9740 1 -- 172.21.15.1:0/1186098322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64c031a20 con 0x558a38212060 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff64a7fc640 1 -- 172.21.15.1:0/1186098322 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff65409c400 con 0x558a38212060 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff66b4c9740 10 client.19191 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff66b4c9740 1 -- 172.21.15.1:0/1186098322 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a37f50960 con 0x558a38212060 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.446+0000 7ff64a7fc640 1 -- 172.21.15.1:0/1186098322 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64c031a20 con 0x558a38212060 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff64a7fc640 1 -- 172.21.15.1:0/1186098322 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff6540b3080 con 0x558a38212060 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff66b4c9740 10 client.19191 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff66b4c9740 10 client.19191 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff66b4c9740 20 client.19191 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a3822cd80 0x558a3822f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff66b4c9740 4 client.19191 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff66b4c9740 1 -- 172.21.15.1:0/1186098322 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a38184b90 con 0x558a3822cd80 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff65bfff640 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a3822cd80 0x558a3822f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff65bfff640 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a3822cd80 0x558a3822f210 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e299d74c39b25e2 server_cookie=394953b79509d79a in_seq=0 out_seq=0 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.447+0000 7ff64a7fc640 10 client.19191 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff64a7fc640 1 -- 172.21.15.1:0/1186098322 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a38184b90 con 0x558a3822cd80 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff64a7fc640 10 client.19191 handle_command_reply: tid=0 2024-09-16T00:44:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff66b4c9740 1 client.19191 shutdown 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff66b4c9740 1 -- 172.21.15.1:0/1186098322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212060 msgr2=0x558a3821a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212060 0x558a3821a0a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff654007590 tx=0x7ff65400a5f0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff66b4c9740 1 -- 172.21.15.1:0/1186098322 shutdown_connections 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a3822cd80 0x558a3822f210 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a3821d240 0x558a3821ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212980 0x558a3821a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/1186098322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212060 0x558a3821a0a0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.448+0000 7ff66b4c9740 1 -- 172.21.15.1:0/1186098322 >> 172.21.15.1:0/1186098322 conn(0x558a3820e5f0 msgr2=0x558a382142d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.449+0000 7ff64a7fc640 0 client.19191 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.449+0000 7ff66b4c9740 1 -- 172.21.15.1:0/1186098322 shutdown_connections 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.449+0000 7ff66b4c9740 1 -- 172.21.15.1:0/1186098322 wait complete. 2024-09-16T00:44:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.449+0000 7ff66b4c9740 20 client.19191 trim_cache size 0 max 16384 2024-09-16T00:44:33.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.469+0000 7ff66b4c9740 1 Processor -- start 2024-09-16T00:44:33.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.469+0000 7ff66b4c9740 1 -- start start 2024-09-16T00:44:33.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.469+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212060 0x558a382141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.469+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212980 0x558a38234a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.469+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a38234fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641b17f0 con 0x558a3821d240 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641af0c0 con 0x558a38212060 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641acf10 con 0x558a38212980 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff66359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a38234fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff66359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a38234fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36780/0 (socket says 172.21.15.1:36780) 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff66359e640 1 -- 172.21.15.1:0/989047245 learned_addr learned my addr 172.21.15.1:0/989047245 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff662d9d640 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212060 0x558a382141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff65bfff640 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212980 0x558a38234a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff662d9d640 1 -- 172.21.15.1:0/989047245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212980 msgr2=0x558a38234a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff662d9d640 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212980 0x558a38234a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff662d9d640 1 -- 172.21.15.1:0/989047245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 msgr2=0x558a38234fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:33.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff662d9d640 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a38234fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff662d9d640 1 -- 172.21.15.1:0/989047245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a38219070 con 0x558a38212060 2024-09-16T00:44:33.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.470+0000 7ff66359e640 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a38234fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:33.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.471+0000 7ff662d9d640 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212060 0x558a382141c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff654098160 tx=0x7ff6540056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b77605b46d0bcd0d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:33.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.471+0000 7ff658ff9640 1 -- 172.21.15.1:0/989047245 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6540d3020 con 0x558a38212060 2024-09-16T00:44:33.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.471+0000 7ff658ff9640 1 -- 172.21.15.1:0/989047245 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6540d32b0 con 0x558a38212060 2024-09-16T00:44:33.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.471+0000 7ff658ff9640 1 -- 172.21.15.1:0/989047245 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65409a000 con 0x558a38212060 2024-09-16T00:44:33.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.471+0000 7ff66b4c9740 1 -- 172.21.15.1:0/989047245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212060 msgr2=0x558a382141c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:33.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212060 0x558a382141c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff654098160 tx=0x7ff6540056d0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 -- 172.21.15.1:0/989047245 shutdown_connections 2024-09-16T00:44:33.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a3821d240 0x558a38234fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212980 0x558a38234a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/989047245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a38212060 0x558a382141c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 -- 172.21.15.1:0/989047245 >> 172.21.15.1:0/989047245 conn(0x558a3820e5f0 msgr2=0x558a38213ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:33.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 -- 172.21.15.1:0/989047245 shutdown_connections 2024-09-16T00:44:33.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 -- 172.21.15.1:0/989047245 wait complete. 2024-09-16T00:44:33.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 -- start start 2024-09-16T00:44:33.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.472+0000 7ff66b4c9740 1 Processor -- start 2024-09-16T00:44:33.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 0x558a38225790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212980 0x558a382221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff66b4c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3821d240 0x558a382226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212980 0x558a382221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641ad580 con 0x558a38212980 2024-09-16T00:44:33.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff662d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 0x558a38225790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212980 0x558a382221b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36786/0 (socket says 172.21.15.1:36786) 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff66b4c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641ae9f0 con 0x558a3821d240 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff65bfff640 1 -- 172.21.15.1:0/4203740044 learned_addr learned my addr 172.21.15.1:0/4203740044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff66b4c9740 1 -- 172.21.15.1:0/4203740044 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641adc50 con 0x558a38212060 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff66b4c9740 10 client.0 _ll_register_callbacks cb 0x7ff654097fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff66359e640 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3821d240 0x558a382226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff65bfff640 1 -- 172.21.15.1:0/4203740044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 msgr2=0x558a38225790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff65bfff640 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 0x558a38225790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff65bfff640 1 -- 172.21.15.1:0/4203740044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3821d240 msgr2=0x558a382226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff65bfff640 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3821d240 0x558a382226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.473+0000 7ff65bfff640 1 -- 172.21.15.1:0/4203740044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a38222de0 con 0x558a38212980 2024-09-16T00:44:33.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.474+0000 7ff65bfff640 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212980 0x558a382221b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0096c0 tx=0x7ff64c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142e75991b9b9f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.474+0000 7ff64a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.474+0000 7ff64a7fc640 1 -- 172.21.15.1:0/4203740044 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c035050 con 0x558a38212980 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.474+0000 7ff64a7fc640 1 -- 172.21.15.1:0/4203740044 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff64c004240 con 0x558a38212980 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.474+0000 7ff66b4c9740 1 -- 172.21.15.1:0/4203740044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6540d32b0 con 0x558a38212980 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.474+0000 7ff64a7fc640 1 -- 172.21.15.1:0/4203740044 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff64c05e050 con 0x558a38212980 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.474+0000 7ff66b4c9740 10 client.19197 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.474+0000 7ff66b4c9740 1 -- 172.21.15.1:0/4203740044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a38298950 con 0x558a38212980 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.475+0000 7ff64a7fc640 1 -- 172.21.15.1:0/4203740044 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6540d32b0 con 0x558a38212980 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.475+0000 7ff64a7fc640 1 -- 172.21.15.1:0/4203740044 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff64c003970 con 0x558a38212980 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.475+0000 7ff66b4c9740 10 client.19197 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.475+0000 7ff66b4c9740 10 client.19197 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.475+0000 7ff66b4c9740 20 client.19197 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.475+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a38298fc0 0x558a3829b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.475+0000 7ff66b4c9740 4 client.19197 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "44509eca", "release": true, "await": true}] 2024-09-16T00:44:33.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.475+0000 7ff66b4c9740 1 -- 172.21.15.1:0/4203740044 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "set_id": "44509eca", "release": true, "await": true}) -- 0x558a3817f0a0 con 0x558a38298fc0 2024-09-16T00:44:33.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.475+0000 7ff662d9d640 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a38298fc0 0x558a3829b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:33.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.476+0000 7ff662d9d640 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a38298fc0 0x558a3829b3e0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9239803fb16e92 server_cookie=4794a05e3f07c531 in_seq=0 out_seq=0 2024-09-16T00:44:33.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.476+0000 7ff64a7fc640 10 client.19197 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:33.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff64a7fc640 1 -- 172.21.15.1:0/4203740044 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558a3817f0a0 con 0x558a38298fc0 2024-09-16T00:44:33.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff64a7fc640 10 client.19197 handle_command_reply: tid=0 2024-09-16T00:44:33.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff66b4c9740 1 client.19197 shutdown 2024-09-16T00:44:33.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff66b4c9740 1 -- 172.21.15.1:0/4203740044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212980 msgr2=0x558a382221b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:33.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212980 0x558a382221b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0096c0 tx=0x7ff64c02a410 comp rx=0 tx=0).stop 2024-09-16T00:44:33.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff66b4c9740 1 -- 172.21.15.1:0/4203740044 shutdown_connections 2024-09-16T00:44:33.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x558a38298fc0 0x558a3829b3e0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3821d240 0x558a382226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a38212980 0x558a382221b0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff66b4c9740 1 --2- 172.21.15.1:0/4203740044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a38212060 0x558a38225790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff66b4c9740 1 -- 172.21.15.1:0/4203740044 >> 172.21.15.1:0/4203740044 conn(0x558a3820e5f0 msgr2=0x558a3820d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.478+0000 7ff64a7fc640 0 client.19197 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.479+0000 7ff66b4c9740 1 -- 172.21.15.1:0/4203740044 shutdown_connections 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.479+0000 7ff66b4c9740 1 -- 172.21.15.1:0/4203740044 wait complete. 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.479+0000 7ff66b4c9740 20 client.19197 trim_cache size 0 max 16384 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 60, 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-16T00:44:33.480 INFO:teuthology.orchestra.run.smithi001.stdout: "44509eca": { 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 60, 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 137.7, 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-16T00:44:33.481 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-16T00:44:33.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:33.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:33.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:33.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:33.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:33.482 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.479+0000 7ff6690f0640 1 -- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff6500a3240 msgr2=0x7ff6500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.479+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff6500a3240 0x7ff6500a5700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009e80 tx=0x7ff65c004b50 comp rx=0 tx=0).stop 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.479+0000 7ff6690f0640 1 -- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664101de0 msgr2=0x7ff664198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.479+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664101de0 0x7ff664198490 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c890 tx=0x7ff65400cd50 comp rx=0 tx=0).stop 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.479+0000 7ff6690f0640 1 -- 172.21.15.1:0/2687525600 shutdown_connections 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.479+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff6500a3240 0x7ff6500a5700 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.480+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff664106f10 0x7ff66419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.480+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff664102790 0x7ff6641989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.480+0000 7ff6690f0640 1 --2- 172.21.15.1:0/2687525600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff664101de0 0x7ff664198490 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.480+0000 7ff6690f0640 1 -- 172.21.15.1:0/2687525600 >> 172.21.15.1:0/2687525600 conn(0x7ff6640fd8c0 msgr2=0x7ff6640feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.480+0000 7ff6690f0640 1 -- 172.21.15.1:0/2687525600 shutdown_connections 2024-09-16T00:44:33.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:33.480+0000 7ff6690f0640 1 -- 172.21.15.1:0/2687525600 wait complete. 2024-09-16T00:44:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:34.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:33 smithi001 ceph-mon[30934]: pgmap v835: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:44:34.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:33 smithi153 ceph-mon[35538]: pgmap v835: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:44:34.239 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '44509eca', total seconds elapsed: 22.8 2024-09-16T00:44:34.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:33 smithi019 ceph-mon[35186]: pgmap v835: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:44:34.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.761+0000 7f976eedd640 1 -- 172.21.15.1:0/3017025176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9768101420 msgr2=0x7f976810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:34.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.761+0000 7f976eedd640 1 --2- 172.21.15.1:0/3017025176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9768101420 0x7f976810d950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9760009c10 tx=0x7f976002a6e0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 -- 172.21.15.1:0/3017025176 shutdown_connections 2024-09-16T00:44:34.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 --2- 172.21.15.1:0/3017025176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9768101420 0x7f976810d950 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 --2- 172.21.15.1:0/3017025176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9768100a80 0x7f9768100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 --2- 172.21.15.1:0/3017025176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768106a80 0x7f9768106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 -- 172.21.15.1:0/3017025176 >> 172.21.15.1:0/3017025176 conn(0x7f97680fc7a0 msgr2=0x7f97680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:34.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 -- 172.21.15.1:0/3017025176 shutdown_connections 2024-09-16T00:44:34.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 -- 172.21.15.1:0/3017025176 wait complete. 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 Processor -- start 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 -- start start 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9768100a80 0x7f976819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.762+0000 7f976eedd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768101420 0x7f976819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f976eedd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9768106a80 0x7f976819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f976eedd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9768110e30 con 0x7f9768101420 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f976eedd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9768110cb0 con 0x7f9768100a80 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f976eedd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9768110fb0 con 0x7f9768106a80 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768101420 0x7f976819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768101420 0x7f976819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36806/0 (socket says 172.21.15.1:36806) 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975ffff640 1 -- 172.21.15.1:0/1965357417 learned_addr learned my addr 172.21.15.1:0/1965357417 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f976d47a640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9768106a80 0x7f976819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f976cc79640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9768100a80 0x7f976819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975ffff640 1 -- 172.21.15.1:0/1965357417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9768106a80 msgr2=0x7f976819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975ffff640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9768106a80 0x7f976819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975ffff640 1 -- 172.21.15.1:0/1965357417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9768100a80 msgr2=0x7f976819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975ffff640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9768100a80 0x7f976819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975ffff640 1 -- 172.21.15.1:0/1965357417 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976819f5d0 con 0x7f9768101420 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975ffff640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768101420 0x7f976819ab60 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f975800e9f0 tx=0x7f975800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975dffb640 1 -- 172.21.15.1:0/1965357417 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975800ccc0 con 0x7f9768101420 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f976eedd640 1 -- 172.21.15.1:0/1965357417 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976819f8c0 con 0x7f9768101420 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975dffb640 1 -- 172.21.15.1:0/1965357417 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758010910 con 0x7f9768101420 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.763+0000 7f975dffb640 1 -- 172.21.15.1:0/1965357417 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580188a0 con 0x7f9768101420 2024-09-16T00:44:34.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.764+0000 7f976eedd640 1 -- 172.21.15.1:0/1965357417 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97681a71a0 con 0x7f9768101420 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.766+0000 7f975dffb640 1 -- 172.21.15.1:0/1965357417 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f9758010ac0 con 0x7f9768101420 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.766+0000 7f97712b6740 1 Processor -- start 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.766+0000 7f97712b6740 1 -- start start 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.766+0000 7f97712b6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afaf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.766+0000 7f97712b6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 0x55800afafa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f97712b6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afbc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f97712b6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55800af152f0 con 0x55800afba370 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f97712b6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55800aebccf0 con 0x55800afaf5d0 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f97712b6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55800af112b0 con 0x55800afb0b40 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afbc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afbc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36814/0 (socket says 172.21.15.1:36814) 2024-09-16T00:44:34.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f976cc79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 0x55800afafa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975ffff640 1 -- 172.21.15.1:0/4122305984 learned_addr learned my addr 172.21.15.1:0/4122305984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975dffb640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97500a3240 0x7f97500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f976d47a640 1 --2- 172.21.15.1:0/4122305984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afaf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f976d47a640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97500a3240 0x7f97500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975ffff640 1 -- 172.21.15.1:0/4122305984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 msgr2=0x55800afaf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975ffff640 1 --2- 172.21.15.1:0/4122305984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afaf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975ffff640 1 -- 172.21.15.1:0/4122305984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 msgr2=0x55800afafa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975ffff640 1 --2- 172.21.15.1:0/4122305984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 0x55800afafa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975ffff640 1 -- 172.21.15.1:0/4122305984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800aea13a0 con 0x55800afba370 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f976cc79640 1 --2- 172.21.15.1:0/4122305984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 0x55800afafa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.767+0000 7f975ffff640 1 --2- 172.21.15.1:0/4122305984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afbc760 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f97580d8320 tx=0x7f97580d87e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d872d850d5e768ed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.768+0000 7f975dffb640 1 -- 172.21.15.1:0/1965357417 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f97580d01a0 con 0x7f9768101420 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.768+0000 7f974effd640 1 -- 172.21.15.1:0/4122305984 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758026030 con 0x55800afba370 2024-09-16T00:44:34.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.768+0000 7f976d47a640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97500a3240 0x7f97500a5700 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9760004af0 tx=0x7f9760009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.768+0000 7f974effd640 1 -- 172.21.15.1:0/4122305984 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975809ca60 con 0x55800afba370 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.768+0000 7f974effd640 1 -- 172.21.15.1:0/4122305984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975809bcc0 con 0x55800afba370 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.768+0000 7f97712b6740 1 -- 172.21.15.1:0/4122305984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 msgr2=0x55800afbc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.768+0000 7f97712b6740 1 --2- 172.21.15.1:0/4122305984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afbc760 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f97580d8320 tx=0x7f97580d87e0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 -- 172.21.15.1:0/4122305984 shutdown_connections 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 --2- 172.21.15.1:0/4122305984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afbc760 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 --2- 172.21.15.1:0/4122305984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 0x55800afafa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 --2- 172.21.15.1:0/4122305984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afaf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 -- 172.21.15.1:0/4122305984 >> 172.21.15.1:0/4122305984 conn(0x55800af78310 msgr2=0x55800afabc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 -- 172.21.15.1:0/4122305984 shutdown_connections 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 -- 172.21.15.1:0/4122305984 wait complete. 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 -- start start 2024-09-16T00:44:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 Processor -- start 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afaf5d0 0x55800afb9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afb0b40 0x55800afb7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afba370 0x55800afb7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55800afbd070 con 0x55800afb0b40 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55800aed0060 con 0x55800afba370 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.769+0000 7f97712b6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55800afbd1f0 con 0x55800afaf5d0 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f97712b6740 10 client.0 _ll_register_callbacks cb 0x55800ac238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f975ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afb0b40 0x55800afb7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f975ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afb0b40 0x55800afb7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36824/0 (socket says 172.21.15.1:36824) 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f975ffff640 1 -- 172.21.15.1:0/2061621933 learned_addr learned my addr 172.21.15.1:0/2061621933 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f976d47a640 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afba370 0x55800afb7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f976cc79640 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afaf5d0 0x55800afb9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f975ffff640 1 -- 172.21.15.1:0/2061621933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afaf5d0 msgr2=0x55800afb9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f975ffff640 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afaf5d0 0x55800afb9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f975ffff640 1 -- 172.21.15.1:0/2061621933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afba370 msgr2=0x55800afb7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f975ffff640 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afba370 0x55800afb7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f975ffff640 1 -- 172.21.15.1:0/2061621933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55800ac863c0 con 0x55800afb0b40 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f975ffff640 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afb0b40 0x55800afb7070 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f975809a610 tx=0x7f975800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876c2306fb310b7a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f9747fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f9747fff640 1 -- 172.21.15.1:0/2061621933 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580d3cd0 con 0x55800afb0b40 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f9747fff640 1 -- 172.21.15.1:0/2061621933 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f97580a04a0 con 0x55800afb0b40 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.770+0000 7f97712b6740 1 -- 172.21.15.1:0/2061621933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975809ca60 con 0x55800afb0b40 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f9747fff640 1 -- 172.21.15.1:0/2061621933 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f975809bcc0 con 0x55800afb0b40 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f97712b6740 10 client.19209 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f97712b6740 1 -- 172.21.15.1:0/2061621933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55800afbd070 con 0x55800afb0b40 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f9747fff640 1 -- 172.21.15.1:0/2061621933 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975809ca60 con 0x55800afb0b40 2024-09-16T00:44:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f9747fff640 1 -- 172.21.15.1:0/2061621933 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f97580a4b40 con 0x55800afb0b40 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f97712b6740 10 client.19209 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f97712b6740 10 client.19209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f97712b6740 20 client.19209 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f97712b6740 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55800afc9d00 0x55800afcc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f97712b6740 4 client.19209 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f97712b6740 1 -- 172.21.15.1:0/2061621933 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55800af22860 con 0x55800afc9d00 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.771+0000 7f976cc79640 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55800afc9d00 0x55800afcc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.772+0000 7f976cc79640 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55800afc9d00 0x55800afcc0f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=42641f18ecdbd147 server_cookie=5dcd4fa32d1b1694 in_seq=0 out_seq=0 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.772+0000 7f9747fff640 10 client.19209 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.772+0000 7f9747fff640 1 -- 172.21.15.1:0/2061621933 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55800af22860 con 0x55800afc9d00 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.772+0000 7f9747fff640 10 client.19209 handle_command_reply: tid=0 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.772+0000 7f97712b6740 1 client.19209 shutdown 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.772+0000 7f97712b6740 1 -- 172.21.15.1:0/2061621933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afb0b40 msgr2=0x55800afb7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afb0b40 0x55800afb7070 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f975809a610 tx=0x7f975800ce60 comp rx=0 tx=0).stop 2024-09-16T00:44:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 1 -- 172.21.15.1:0/2061621933 shutdown_connections 2024-09-16T00:44:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afba370 0x55800afb7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55800afc9d00 0x55800afcc0f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f9747fff640 0 client.19209 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:44:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afb0b40 0x55800afb7070 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 1 --2- 172.21.15.1:0/2061621933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afaf5d0 0x55800afb9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 1 -- 172.21.15.1:0/2061621933 >> 172.21.15.1:0/2061621933 conn(0x55800af78310 msgr2=0x55800afb1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 1 -- 172.21.15.1:0/2061621933 shutdown_connections 2024-09-16T00:44:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 1 -- 172.21.15.1:0/2061621933 wait complete. 2024-09-16T00:44:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.773+0000 7f97712b6740 20 client.19209 trim_cache size 0 max 16384 2024-09-16T00:44:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.786+0000 7f97712b6740 1 Processor -- start 2024-09-16T00:44:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.786+0000 7f97712b6740 1 -- start start 2024-09-16T00:44:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.786+0000 7f97712b6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afaf5d0 0x55800afabca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.786+0000 7f97712b6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afb0b40 0x55800afd16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.786+0000 7f97712b6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afba370 0x55800afd1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.786+0000 7f97712b6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b39b0 con 0x55800afaf5d0 2024-09-16T00:44:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.786+0000 7f97712b6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b1280 con 0x55800afb0b40 2024-09-16T00:44:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.786+0000 7f97712b6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f97681af0d0 con 0x55800afba370 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976cc79640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afaf5d0 0x55800afabca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976cc79640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afaf5d0 0x55800afabca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36828/0 (socket says 172.21.15.1:36828) 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976cc79640 1 -- 172.21.15.1:0/848643185 learned_addr learned my addr 172.21.15.1:0/848643185 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f975ffff640 1 --2- 172.21.15.1:0/848643185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afb0b40 0x55800afd16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976d47a640 1 --2- 172.21.15.1:0/848643185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afba370 0x55800afd1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976cc79640 1 -- 172.21.15.1:0/848643185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afba370 msgr2=0x55800afd1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976cc79640 1 --2- 172.21.15.1:0/848643185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afba370 0x55800afd1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976cc79640 1 -- 172.21.15.1:0/848643185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afb0b40 msgr2=0x55800afd16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976cc79640 1 --2- 172.21.15.1:0/848643185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afb0b40 0x55800afd16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976cc79640 1 -- 172.21.15.1:0/848643185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800afb7b70 con 0x55800afaf5d0 2024-09-16T00:44:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f976cc79640 1 --2- 172.21.15.1:0/848643185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afaf5d0 0x55800afabca0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9764009af0 tx=0x7f9764009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdd52a6d748befb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f974e7fc640 1 -- 172.21.15.1:0/848643185 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764005020 con 0x55800afaf5d0 2024-09-16T00:44:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.787+0000 7f974e7fc640 1 -- 172.21.15.1:0/848643185 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97640047a0 con 0x55800afaf5d0 2024-09-16T00:44:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f974e7fc640 1 -- 172.21.15.1:0/848643185 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640107b0 con 0x55800afaf5d0 2024-09-16T00:44:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f97712b6740 1 -- 172.21.15.1:0/848643185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afaf5d0 msgr2=0x55800afabca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f97712b6740 1 --2- 172.21.15.1:0/848643185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afaf5d0 0x55800afabca0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9764009af0 tx=0x7f9764009260 comp rx=0 tx=0).stop 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f97712b6740 1 -- 172.21.15.1:0/848643185 shutdown_connections 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f97712b6740 1 --2- 172.21.15.1:0/848643185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afba370 0x55800afd1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f97712b6740 1 --2- 172.21.15.1:0/848643185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afb0b40 0x55800afd16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f97712b6740 1 --2- 172.21.15.1:0/848643185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afaf5d0 0x55800afabca0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f97712b6740 1 -- 172.21.15.1:0/848643185 >> 172.21.15.1:0/848643185 conn(0x55800af78310 msgr2=0x55800afb6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f97712b6740 1 -- 172.21.15.1:0/848643185 shutdown_connections 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.788+0000 7f97712b6740 1 -- 172.21.15.1:0/848643185 wait complete. 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f97712b6740 1 -- start start 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f97712b6740 1 Processor -- start 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f97712b6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 0x55800afc2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f97712b6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afb3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f97712b6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afb4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f97712b6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f97681af740 con 0x55800afba370 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f97712b6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b0bb0 con 0x55800afaf5d0 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f97712b6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f97681afe10 con 0x55800afb0b40 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f976d47a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afb4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f97712b6740 10 client.0 _ll_register_callbacks cb 0x7f97640097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f975ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afb3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f976d47a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afb4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36836/0 (socket says 172.21.15.1:36836) 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f975ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afb3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.1:40166/0 (socket says 172.21.15.1:40166) 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.789+0000 7f976d47a640 1 -- 172.21.15.1:0/1400308986 learned_addr learned my addr 172.21.15.1:0/1400308986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f976cc79640 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 0x55800afc2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f976d47a640 1 -- 172.21.15.1:0/1400308986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 msgr2=0x55800afb3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:34.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f976d47a640 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afb3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f976d47a640 1 -- 172.21.15.1:0/1400308986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 msgr2=0x55800afc2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f976d47a640 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 0x55800afc2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f976d47a640 1 -- 172.21.15.1:0/1400308986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55800afb4850 con 0x55800afba370 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f975ffff640 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afb3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f976d47a640 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afb4160 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f9760005f00 tx=0x7f9760009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c219cd8d4fa7e958 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f9747fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f9747fff640 1 -- 172.21.15.1:0/1400308986 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760045040 con 0x55800afba370 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f9747fff640 1 -- 172.21.15.1:0/1400308986 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f9760003b70 con 0x55800afba370 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.790+0000 7f97712b6740 1 -- 172.21.15.1:0/1400308986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97640047a0 con 0x55800afba370 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f9747fff640 1 -- 172.21.15.1:0/1400308986 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9760036040 con 0x55800afba370 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f97712b6740 10 client.19215 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f97712b6740 1 -- 172.21.15.1:0/1400308986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55800b03cda0 con 0x55800afba370 2024-09-16T00:44:34.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f9747fff640 1 -- 172.21.15.1:0/1400308986 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97640047a0 con 0x55800afba370 2024-09-16T00:44:34.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f9747fff640 1 -- 172.21.15.1:0/1400308986 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f976004b070 con 0x55800afba370 2024-09-16T00:44:34.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f97712b6740 10 client.19215 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:34.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f97712b6740 10 client.19215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:44:34.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f97712b6740 20 client.19215 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:34.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f97712b6740 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55800afc9d00 0x55800b03f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:34.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f97712b6740 4 client.19215 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:44:34.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f97712b6740 1 -- 172.21.15.1:0/1400308986 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55800af1cd70 con 0x55800afc9d00 2024-09-16T00:44:34.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.791+0000 7f976cc79640 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55800afc9d00 0x55800b03f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:34.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f9747fff640 10 client.19215 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:44:34.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f976cc79640 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55800afc9d00 0x55800b03f560 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e78b861f540c96b server_cookie=3abacb378ffce415 in_seq=0 out_seq=0 2024-09-16T00:44:34.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f9747fff640 1 -- 172.21.15.1:0/1400308986 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55800af1cd70 con 0x55800afc9d00 2024-09-16T00:44:34.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f9747fff640 10 client.19215 handle_command_reply: tid=0 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 client.19215 shutdown 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 -- 172.21.15.1:0/1400308986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 msgr2=0x55800afb4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afb4160 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f9760005f00 tx=0x7f9760009500 comp rx=0 tx=0).stop 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 -- 172.21.15.1:0/1400308986 shutdown_connections 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x55800afc9d00 0x55800b03f560 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55800afba370 0x55800afb4160 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55800afb0b40 0x55800afb3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 --2- 172.21.15.1:0/1400308986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55800afaf5d0 0x55800afc2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 -- 172.21.15.1:0/1400308986 >> 172.21.15.1:0/1400308986 conn(0x55800af78310 msgr2=0x55800afaa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f9747fff640 0 client.19215 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 -- 172.21.15.1:0/1400308986 shutdown_connections 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.792+0000 7f97712b6740 1 -- 172.21.15.1:0/1400308986 wait complete. 2024-09-16T00:44:34.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f97712b6740 20 client.19215 trim_cache size 0 max 16384 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 -- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97500a3240 msgr2=0x7f97500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97500a3240 0x7f97500a5700 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9760004af0 tx=0x7f9760009280 comp rx=0 tx=0).stop 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 -- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768101420 msgr2=0x7f976819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768101420 0x7f976819ab60 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f975800e9f0 tx=0x7f975800edf0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 -- 172.21.15.1:0/1965357417 shutdown_connections 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f97500a3240 0x7f97500a5700 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9768106a80 0x7f976819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768101420 0x7f976819ab60 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 --2- 172.21.15.1:0/1965357417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9768100a80 0x7f976819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:34.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 -- 172.21.15.1:0/1965357417 >> 172.21.15.1:0/1965357417 conn(0x7f97680fc7a0 msgr2=0x7f976806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:34.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 -- 172.21.15.1:0/1965357417 shutdown_connections 2024-09-16T00:44:34.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:34.793+0000 7f976eedd640 1 -- 172.21.15.1:0/1965357417 wait complete. 2024-09-16T00:44:35.623 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:44:35.885 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:36.169 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:35 smithi001 ceph-mon[30934]: pgmap v836: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:44:36.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:35 smithi153 ceph-mon[35538]: pgmap v836: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:44:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:35 smithi019 ceph-mon[35186]: pgmap v836: 129 pgs: 129 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:44:36.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.360+0000 7fdb78b45640 1 -- 172.21.15.1:0/1182102180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74101420 msgr2=0x7fdb7410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.360+0000 7fdb78b45640 1 --2- 172.21.15.1:0/1182102180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74101420 0x7fdb7410d950 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009930 tx=0x7fdb6c02a420 comp rx=0 tx=0).stop 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.360+0000 7fdb78b45640 1 -- 172.21.15.1:0/1182102180 shutdown_connections 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.360+0000 7fdb78b45640 1 --2- 172.21.15.1:0/1182102180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74101420 0x7fdb7410d950 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.360+0000 7fdb78b45640 1 --2- 172.21.15.1:0/1182102180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74100a80 0x7fdb74100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.360+0000 7fdb78b45640 1 --2- 172.21.15.1:0/1182102180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb74106a80 0x7fdb74106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.360+0000 7fdb78b45640 1 -- 172.21.15.1:0/1182102180 >> 172.21.15.1:0/1182102180 conn(0x7fdb740fc7a0 msgr2=0x7fdb740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.360+0000 7fdb78b45640 1 -- 172.21.15.1:0/1182102180 shutdown_connections 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.360+0000 7fdb78b45640 1 -- 172.21.15.1:0/1182102180 wait complete. 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb78b45640 1 Processor -- start 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb78b45640 1 -- start start 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb78b45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb74100a80 0x7fdb74199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb78b45640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74101420 0x7fdb741997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb78b45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74106a80 0x7fdb741a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb78b45640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74110e80 con 0x7fdb74106a80 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb78b45640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74110d00 con 0x7fdb74100a80 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb78b45640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74111000 con 0x7fdb74101420 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb72d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74106a80 0x7fdb741a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb72d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74106a80 0x7fdb741a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36854/0 (socket says 172.21.15.1:36854) 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb72d9d640 1 -- 172.21.15.1:0/638047267 learned_addr learned my addr 172.21.15.1:0/638047267 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb7259c640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb74100a80 0x7fdb74199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.361+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74101420 0x7fdb741997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb72d9d640 1 -- 172.21.15.1:0/638047267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74101420 msgr2=0x7fdb741997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb72d9d640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74101420 0x7fdb741997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb72d9d640 1 -- 172.21.15.1:0/638047267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb74100a80 msgr2=0x7fdb74199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb72d9d640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb74100a80 0x7fdb74199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb72d9d640 1 -- 172.21.15.1:0/638047267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb741a4830 con 0x7fdb74106a80 2024-09-16T00:44:36.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb72d9d640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74106a80 0x7fdb741a42f0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c005ab0 tx=0x7fdb6c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb637fe640 1 -- 172.21.15.1:0/638047267 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c03f030 con 0x7fdb74106a80 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb78b45640 1 -- 172.21.15.1:0/638047267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb741a4a00 con 0x7fdb74106a80 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb637fe640 1 -- 172.21.15.1:0/638047267 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6c0044c0 con 0x7fdb74106a80 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb637fe640 1 -- 172.21.15.1:0/638047267 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c035070 con 0x7fdb74106a80 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.362+0000 7fdb78b45640 1 -- 172.21.15.1:0/638047267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb741a4e50 con 0x7fdb74106a80 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb637fe640 1 -- 172.21.15.1:0/638047267 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fdb6c042030 con 0x7fdb74106a80 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7af1e740 1 Processor -- start 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7af1e740 1 -- start start 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e354b40 0x56056e353090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e353a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 0x56056e364b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56056e2b92f0 con 0x56056e3535d0 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56056e260cf0 con 0x56056e354b40 2024-09-16T00:44:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56056e2b52b0 con 0x56056e362780 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e353a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.365+0000 7fdb7259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e353a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36862/0 (socket says 172.21.15.1:36862) 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb7259c640 1 -- 172.21.15.1:0/1719330407 learned_addr learned my addr 172.21.15.1:0/1719330407 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb637fe640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdb5c0a3240 0x7fdb5c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e354b40 0x56056e353090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 0x56056e364b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb72d9d640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdb5c0a3240 0x7fdb5c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb7259c640 1 -- 172.21.15.1:0/1719330407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 msgr2=0x56056e364b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb7259c640 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 0x56056e364b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb7259c640 1 -- 172.21.15.1:0/1719330407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e354b40 msgr2=0x56056e353090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb7259c640 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e354b40 0x56056e353090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb7259c640 1 -- 172.21.15.1:0/1719330407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56056e2453a0 con 0x56056e3535d0 2024-09-16T00:44:36.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 0x56056e364b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb637fe640 1 -- 172.21.15.1:0/638047267 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdb6c0f00e0 con 0x7fdb74106a80 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb7259c640 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e353a10 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009fd0 tx=0x7fdb640118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82112d5e63308821 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb60ff9640 1 -- 172.21.15.1:0/1719330407 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6401e070 con 0x56056e3535d0 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb72d9d640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdb5c0a3240 0x7fdb5c0a5700 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009900 tx=0x7fdb6c086d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb60ff9640 1 -- 172.21.15.1:0/1719330407 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb64003c30 con 0x56056e3535d0 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.366+0000 7fdb60ff9640 1 -- 172.21.15.1:0/1719330407 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6400f9f0 con 0x56056e3535d0 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.367+0000 7fdb7af1e740 1 -- 172.21.15.1:0/1719330407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 msgr2=0x56056e353a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.367+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e353a10 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009fd0 tx=0x7fdb640118d0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.367+0000 7fdb7af1e740 1 -- 172.21.15.1:0/1719330407 shutdown_connections 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.367+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 0x56056e364b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.367+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e353a10 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.367+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/1719330407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e354b40 0x56056e353090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.367+0000 7fdb7af1e740 1 -- 172.21.15.1:0/1719330407 >> 172.21.15.1:0/1719330407 conn(0x56056e31c310 msgr2=0x56056e34fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.367+0000 7fdb7af1e740 1 -- 172.21.15.1:0/1719330407 shutdown_connections 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.367+0000 7fdb7af1e740 1 -- 172.21.15.1:0/1719330407 wait complete. 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7af1e740 1 -- start start 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7af1e740 1 Processor -- start 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e362150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 0x56056e35f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e362780 0x56056e35f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56056e365480 con 0x56056e3535d0 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56056e274060 con 0x56056e362780 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56056e365600 con 0x56056e354b40 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7af1e740 10 client.0 _ll_register_callbacks cb 0x56056dfc78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb71d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e362150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb71d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e362150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36874/0 (socket says 172.21.15.1:36874) 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb71d9b640 1 -- 172.21.15.1:0/683078951 learned_addr learned my addr 172.21.15.1:0/683078951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb7259c640 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 0x56056e35f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb71d9b640 1 -- 172.21.15.1:0/683078951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 msgr2=0x56056e35f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 0x56056e35f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb71d9b640 1 -- 172.21.15.1:0/683078951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e362780 msgr2=0x56056e35f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e362780 0x56056e35f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.368+0000 7fdb71d9b640 1 -- 172.21.15.1:0/683078951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56056e02a3c0 con 0x56056e3535d0 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e362150 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdb68014510 tx=0x7fdb680149d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13af9e568f0766b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb59ffb640 1 -- 172.21.15.1:0/683078951 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68004220 con 0x56056e3535d0 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb59ffb640 1 -- 172.21.15.1:0/683078951 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdb680043c0 con 0x56056e3535d0 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb7af1e740 1 -- 172.21.15.1:0/683078951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb64003c30 con 0x56056e3535d0 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb59ffb640 1 -- 172.21.15.1:0/683078951 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb68025050 con 0x56056e3535d0 2024-09-16T00:44:36.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb7af1e740 10 client.19227 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb7af1e740 1 -- 172.21.15.1:0/683078951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56056e370ef0 con 0x56056e3535d0 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb59ffb640 1 -- 172.21.15.1:0/683078951 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb64003c30 con 0x56056e3535d0 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb59ffb640 1 -- 172.21.15.1:0/683078951 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdb68011910 con 0x56056e3535d0 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb7af1e740 10 client.19227 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb7af1e740 10 client.19227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb7af1e740 20 client.19227 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56056e3723b0 0x56056e3747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb7af1e740 4 client.19227 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.369+0000 7fdb7af1e740 1 -- 172.21.15.1:0/683078951 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56056e2c6860 con 0x56056e3723b0 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.370+0000 7fdb7259c640 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56056e3723b0 0x56056e3747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.370+0000 7fdb7259c640 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56056e3723b0 0x56056e3747a0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee06ffda62f68af4 server_cookie=e372a0c7e76c4b9b in_seq=0 out_seq=0 2024-09-16T00:44:36.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.370+0000 7fdb59ffb640 10 client.19227 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:36.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb59ffb640 1 -- 172.21.15.1:0/683078951 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56056e2c6860 con 0x56056e3723b0 2024-09-16T00:44:36.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb59ffb640 10 client.19227 handle_command_reply: tid=0 2024-09-16T00:44:36.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 client.19227 shutdown 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 -- 172.21.15.1:0/683078951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 msgr2=0x56056e362150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e362150 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdb68014510 tx=0x7fdb680149d0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 -- 172.21.15.1:0/683078951 shutdown_connections 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56056e3723b0 0x56056e3747a0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb59ffb640 0 client.19227 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e362780 0x56056e35f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 0x56056e35f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/683078951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e362150 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 -- 172.21.15.1:0/683078951 >> 172.21.15.1:0/683078951 conn(0x56056e31c310 msgr2=0x56056e355930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 -- 172.21.15.1:0/683078951 shutdown_connections 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 1 -- 172.21.15.1:0/683078951 wait complete. 2024-09-16T00:44:36.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.371+0000 7fdb7af1e740 20 client.19227 trim_cache size 0 max 16384 2024-09-16T00:44:36.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.384+0000 7fdb7af1e740 1 Processor -- start 2024-09-16T00:44:36.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.384+0000 7fdb7af1e740 1 -- start start 2024-09-16T00:44:36.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.384+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e3535d0 0x56056e34fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.384+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e354b40 0x56056e36e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.384+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 0x56056e35ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.384+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741af530 con 0x56056e354b40 2024-09-16T00:44:36.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.384+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741ace00 con 0x56056e3535d0 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.384+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741aac50 con 0x56056e362780 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb7259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e354b40 0x56056e36e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb7259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e354b40 0x56056e36e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36880/0 (socket says 172.21.15.1:36880) 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb7259c640 1 -- 172.21.15.1:0/100811825 learned_addr learned my addr 172.21.15.1:0/100811825 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/100811825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e3535d0 0x56056e34fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/100811825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 0x56056e35ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb7259c640 1 -- 172.21.15.1:0/100811825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 msgr2=0x56056e35ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb7259c640 1 --2- 172.21.15.1:0/100811825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 0x56056e35ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb7259c640 1 -- 172.21.15.1:0/100811825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e3535d0 msgr2=0x56056e34fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb7259c640 1 --2- 172.21.15.1:0/100811825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e3535d0 0x56056e34fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb7259c640 1 -- 172.21.15.1:0/100811825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56056e35fd60 con 0x56056e354b40 2024-09-16T00:44:36.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb7259c640 1 --2- 172.21.15.1:0/100811825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e354b40 0x56056e36e750 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x56056e350ac0 tx=0x7fdb64009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebaee2209e5e53e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb5bfff640 1 -- 172.21.15.1:0/100811825 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6401e040 con 0x56056e354b40 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb5bfff640 1 -- 172.21.15.1:0/100811825 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb64004bb0 con 0x56056e354b40 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.385+0000 7fdb5bfff640 1 -- 172.21.15.1:0/100811825 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64005330 con 0x56056e354b40 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 -- 172.21.15.1:0/100811825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e354b40 msgr2=0x56056e36e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/100811825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e354b40 0x56056e36e750 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x56056e350ac0 tx=0x7fdb64009ce0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 -- 172.21.15.1:0/100811825 shutdown_connections 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/100811825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e362780 0x56056e35ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/100811825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e354b40 0x56056e36e750 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/100811825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e3535d0 0x56056e34fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 -- 172.21.15.1:0/100811825 >> 172.21.15.1:0/100811825 conn(0x56056e31c310 msgr2=0x56056e34f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 -- 172.21.15.1:0/100811825 shutdown_connections 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 -- 172.21.15.1:0/100811825 wait complete. 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 -- start start 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.386+0000 7fdb7af1e740 1 Processor -- start 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e36aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 0x56056e36b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb7af1e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e362780 0x56056e35be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741ab2c0 con 0x56056e3535d0 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741ac730 con 0x56056e362780 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb7af1e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741ab990 con 0x56056e354b40 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb7af1e740 10 client.0 _ll_register_callbacks cb 0x7fdb6800bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e36aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e36aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36890/0 (socket says 172.21.15.1:36890) 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 -- 172.21.15.1:0/4148336682 learned_addr learned my addr 172.21.15.1:0/4148336682 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e362780 0x56056e35be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb7259c640 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 0x56056e36b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 -- 172.21.15.1:0/4148336682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 msgr2=0x56056e36b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 0x56056e36b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 -- 172.21.15.1:0/4148336682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e362780 msgr2=0x56056e35be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e362780 0x56056e35be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 -- 172.21.15.1:0/4148336682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56056e35c670 con 0x56056e3535d0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb7259c640 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 0x56056e36b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb71d9b640 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e36aba0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005d00 tx=0x7fdb6800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3392f5191c22dae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.387+0000 7fdb59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb59ffb640 1 -- 172.21.15.1:0/4148336682 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68005550 con 0x56056e3535d0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb59ffb640 1 -- 172.21.15.1:0/4148336682 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fdb680114a0 con 0x56056e3535d0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb7af1e740 1 -- 172.21.15.1:0/4148336682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb64004bb0 con 0x56056e3535d0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb59ffb640 1 -- 172.21.15.1:0/4148336682 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb680116c0 con 0x56056e3535d0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb7af1e740 10 client.19233 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb7af1e740 1 -- 172.21.15.1:0/4148336682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56056e3dcbb0 con 0x56056e3535d0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb59ffb640 1 -- 172.21.15.1:0/4148336682 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb64004bb0 con 0x56056e3535d0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb59ffb640 1 -- 172.21.15.1:0/4148336682 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdb68029430 con 0x56056e3535d0 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb7af1e740 10 client.19233 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb7af1e740 10 client.19233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb7af1e740 20 client.19233 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56056e3723b0 0x56056e3df2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb7af1e740 4 client.19233 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.388+0000 7fdb7af1e740 1 -- 172.21.15.1:0/4148336682 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56056e2c0d70 con 0x56056e3723b0 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7259c640 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56056e3723b0 0x56056e3df2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7259c640 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56056e3723b0 0x56056e3df2c0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82004b1dbabdfa2 server_cookie=ce9b635f3cc7bae0 in_seq=0 out_seq=0 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb59ffb640 10 client.19233 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb59ffb640 1 -- 172.21.15.1:0/4148336682 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56056e2c0d70 con 0x56056e3723b0 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb59ffb640 10 client.19233 handle_command_reply: tid=0 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 client.19233 shutdown 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 -- 172.21.15.1:0/4148336682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 msgr2=0x56056e36aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e36aba0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005d00 tx=0x7fdb6800bb00 comp rx=0 tx=0).stop 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 -- 172.21.15.1:0/4148336682 shutdown_connections 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x56056e3723b0 0x56056e3df2c0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056e362780 0x56056e35be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56056e354b40 0x56056e36b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 --2- 172.21.15.1:0/4148336682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56056e3535d0 0x56056e36aba0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 -- 172.21.15.1:0/4148336682 >> 172.21.15.1:0/4148336682 conn(0x56056e31c310 msgr2=0x56056e34e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb59ffb640 0 client.19233 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 -- 172.21.15.1:0/4148336682 shutdown_connections 2024-09-16T00:44:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.389+0000 7fdb7af1e740 1 -- 172.21.15.1:0/4148336682 wait complete. 2024-09-16T00:44:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb7af1e740 20 client.19233 trim_cache size 0 max 16384 2024-09-16T00:44:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 -- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdb5c0a3240 msgr2=0x7fdb5c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdb5c0a3240 0x7fdb5c0a5700 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009900 tx=0x7fdb6c086d90 comp rx=0 tx=0).stop 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 -- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74106a80 msgr2=0x7fdb741a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74106a80 0x7fdb741a42f0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c005ab0 tx=0x7fdb6c0040c0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 -- 172.21.15.1:0/638047267 shutdown_connections 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb74106a80 0x7fdb741a42f0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fdb5c0a3240 0x7fdb5c0a5700 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74101420 0x7fdb741997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 --2- 172.21.15.1:0/638047267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb74100a80 0x7fdb74199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 -- 172.21.15.1:0/638047267 >> 172.21.15.1:0/638047267 conn(0x7fdb740fc7a0 msgr2=0x7fdb740fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 -- 172.21.15.1:0/638047267 shutdown_connections 2024-09-16T00:44:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:36.390+0000 7fdb78b45640 1 -- 172.21.15.1:0/638047267 wait complete. 2024-09-16T00:44:38.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:37 smithi001 ceph-mon[30934]: pgmap v837: 129 pgs: 129 active+clean; 7.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:44:38.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:37 smithi153 ceph-mon[35538]: pgmap v837: 129 pgs: 129 active+clean; 7.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:44:38.222 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:44:38.223 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:44:38.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:37 smithi019 ceph-mon[35186]: pgmap v837: 129 pgs: 129 active+clean; 7.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-16T00:44:38.494 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.982+0000 7ff9b86ba640 1 -- 172.21.15.1:0/102443045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9b0106f10 msgr2=0x7ff9b010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.982+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/102443045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9b0106f10 0x7ff9b010ea90 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009ca0 tx=0x7ff9a802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.982+0000 7ff9b86ba640 1 -- 172.21.15.1:0/102443045 shutdown_connections 2024-09-16T00:44:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.982+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/102443045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9b0106f10 0x7ff9b010ea90 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.982+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/102443045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102790 0x7ff9b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.982+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/102443045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9b0101de0 0x7ff9b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.982+0000 7ff9b86ba640 1 -- 172.21.15.1:0/102443045 >> 172.21.15.1:0/102443045 conn(0x7ff9b00fd8c0 msgr2=0x7ff9b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.982+0000 7ff9b86ba640 1 -- 172.21.15.1:0/102443045 shutdown_connections 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.983+0000 7ff9b86ba640 1 -- 172.21.15.1:0/102443045 wait complete. 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.983+0000 7ff9b86ba640 1 Processor -- start 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.983+0000 7ff9b86ba640 1 -- start start 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b86ba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0101de0 0x7ff9b019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b86ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9b0102790 0x7ff9b019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b86ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9b0106f10 0x7ff9b019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b86ba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b0111f90 con 0x7ff9b0106f10 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b86ba640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b0111e10 con 0x7ff9b0101de0 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b86ba640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b0112110 con 0x7ff9b0102790 2024-09-16T00:44:39.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6456640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0101de0 0x7ff9b019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6456640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0101de0 0x7ff9b019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.1:39732/0 (socket says 172.21.15.1:39732) 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6456640 1 -- 172.21.15.1:0/479785192 learned_addr learned my addr 172.21.15.1:0/479785192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9b0102790 0x7ff9b019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9b0106f10 0x7ff9b019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6456640 1 -- 172.21.15.1:0/479785192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9b0102790 msgr2=0x7ff9b019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6456640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9b0102790 0x7ff9b019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6456640 1 -- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9b0106f10 msgr2=0x7ff9b019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6456640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9b0106f10 0x7ff9b019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6456640 1 -- 172.21.15.1:0/479785192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b019f610 con 0x7ff9b0101de0 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9b0106f10 0x7ff9b019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.984+0000 7ff9b6456640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0101de0 0x7ff9b019a660 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00b4a0 tx=0x7ff9ac00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.985+0000 7ff9a77fe640 1 -- 172.21.15.1:0/479785192 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac014070 con 0x7ff9b0101de0 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.985+0000 7ff9b86ba640 1 -- 172.21.15.1:0/479785192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b019f900 con 0x7ff9b0101de0 2024-09-16T00:44:39.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.985+0000 7ff9a77fe640 1 -- 172.21.15.1:0/479785192 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ac010af0 con 0x7ff9b0101de0 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.985+0000 7ff9a77fe640 1 -- 172.21.15.1:0/479785192 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac017d10 con 0x7ff9b0101de0 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.985+0000 7ff9b86ba640 1 -- 172.21.15.1:0/479785192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b01a71e0 con 0x7ff9b0101de0 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.987+0000 7ff9a77fe640 1 -- 172.21.15.1:0/479785192 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff9ac017070 con 0x7ff9b0101de0 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.987+0000 7ff9baa93740 1 Processor -- start 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.987+0000 7ff9baa93740 1 -- start start 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d746ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746f480 0x55e9d7479800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d7479d40 0x55e9d747c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9baa93740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e9d73d4280 con 0x55e9d746eb60 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9baa93740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e9d737bb10 con 0x55e9d746f480 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9baa93740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9d73d0240 con 0x55e9d7479d40 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d746ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6456640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746f480 0x55e9d7479800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d746ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36924/0 (socket says 172.21.15.1:36924) 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 -- 172.21.15.1:0/326990285 learned_addr learned my addr 172.21.15.1:0/326990285 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/326990285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d7479d40 0x55e9d747c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9a77fe640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff99c0a3240 0x7ff99c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 -- 172.21.15.1:0/326990285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d7479d40 msgr2=0x55e9d747c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/326990285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d7479d40 0x55e9d747c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 -- 172.21.15.1:0/326990285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746f480 msgr2=0x55e9d7479800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/326990285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746f480 0x55e9d7479800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 -- 172.21.15.1:0/326990285 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9d73881f0 con 0x55e9d746eb60 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9a77fe640 1 -- 172.21.15.1:0/479785192 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff9ac0cf0c0 con 0x7ff9b0101de0 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff99c0a3240 0x7ff99c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.988+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/326990285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d746ef40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8035000 tx=0x7ff9a80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8db9492ec838a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:39.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9a4ff9640 1 -- 172.21.15.1:0/326990285 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8042040 con 0x55e9d746eb60 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9a4ff9640 1 -- 172.21.15.1:0/326990285 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a8036040 con 0x55e9d746eb60 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9a4ff9640 1 -- 172.21.15.1:0/326990285 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80305c0 con 0x55e9d746eb60 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff99c0a3240 0x7ff99c0a5700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009ca0 tx=0x7ff9a804e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9baa93740 1 -- 172.21.15.1:0/326990285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 msgr2=0x55e9d746ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9baa93740 1 --2- 172.21.15.1:0/326990285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d746ef40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8035000 tx=0x7ff9a80039c0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9baa93740 1 -- 172.21.15.1:0/326990285 shutdown_connections 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9baa93740 1 --2- 172.21.15.1:0/326990285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d7479d40 0x55e9d747c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9baa93740 1 --2- 172.21.15.1:0/326990285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746f480 0x55e9d7479800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9baa93740 1 --2- 172.21.15.1:0/326990285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d746ef40 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.989+0000 7ff9baa93740 1 -- 172.21.15.1:0/326990285 >> 172.21.15.1:0/326990285 conn(0x55e9d746b0f0 msgr2=0x55e9d746b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 -- 172.21.15.1:0/326990285 shutdown_connections 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 -- 172.21.15.1:0/326990285 wait complete. 2024-09-16T00:44:39.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 -- start start 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 Processor -- start 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d7476bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 0x55e9d74770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 0x55e9d7477630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e9d747c910 con 0x55e9d746eb60 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e9d738ef10 con 0x55e9d7479d40 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9d747ca90 con 0x55e9d746f480 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9baa93740 10 client.0 _ll_register_callbacks cb 0x55e9d7357a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9b6456640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d7476bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9b6456640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d7476bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36928/0 (socket says 172.21.15.1:36928) 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.990+0000 7ff9b6456640 1 -- 172.21.15.1:0/1925144837 learned_addr learned my addr 172.21.15.1:0/1925144837 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:39.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 0x55e9d7477630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 0x55e9d74770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9b6456640 1 -- 172.21.15.1:0/1925144837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 msgr2=0x55e9d74770f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9b6456640 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 0x55e9d74770f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9b6456640 1 -- 172.21.15.1:0/1925144837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 msgr2=0x55e9d7477630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9b6456640 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 0x55e9d7477630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9b6456640 1 -- 172.21.15.1:0/1925144837 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9d71456a0 con 0x55e9d746eb60 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 0x55e9d7477630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9b6456640 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d7476bb0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0cc7a0 tx=0x7ff9ac0092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717b1f1fce861be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff999ffb640 1 -- 172.21.15.1:0/1925144837 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac01e050 con 0x55e9d746eb60 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff999ffb640 1 -- 172.21.15.1:0/1925144837 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff9ac010ca0 con 0x55e9d746eb60 2024-09-16T00:44:39.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9baa93740 1 -- 172.21.15.1:0/1925144837 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a8036040 con 0x55e9d746eb60 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff999ffb640 1 -- 172.21.15.1:0/1925144837 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9ac09acd0 con 0x55e9d746eb60 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9baa93740 10 client.19245 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff9baa93740 1 -- 172.21.15.1:0/1925144837 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9d74884a0 con 0x55e9d746eb60 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.991+0000 7ff999ffb640 1 -- 172.21.15.1:0/1925144837 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a8036040 con 0x55e9d746eb60 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff999ffb640 1 -- 172.21.15.1:0/1925144837 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff9ac0a8020 con 0x55e9d746eb60 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff9baa93740 10 client.19245 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff9baa93740 10 client.19245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff9baa93740 20 client.19245 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e9d74898b0 0x55e9d748bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff9baa93740 4 client.19245 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff9baa93740 1 -- 172.21.15.1:0/1925144837 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9d73e17f0 con 0x55e9d74898b0 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e9d74898b0 0x55e9d748bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e9d74898b0 0x55e9d748bcc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f9b2f0c8bf0a96d server_cookie=b117737230dd77cb in_seq=0 out_seq=0 2024-09-16T00:44:39.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.992+0000 7ff999ffb640 10 client.19245 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff999ffb640 1 -- 172.21.15.1:0/1925144837 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9d73e17f0 con 0x55e9d74898b0 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff999ffb640 10 client.19245 handle_command_reply: tid=0 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 client.19245 shutdown 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 -- 172.21.15.1:0/1925144837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 msgr2=0x55e9d7476bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d7476bb0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0cc7a0 tx=0x7ff9ac0092b0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 -- 172.21.15.1:0/1925144837 shutdown_connections 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e9d74898b0 0x55e9d748bcc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 0x55e9d7477630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 0x55e9d74770f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1925144837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746eb60 0x55e9d7476bb0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 -- 172.21.15.1:0/1925144837 >> 172.21.15.1:0/1925144837 conn(0x55e9d746b0f0 msgr2=0x55e9d746a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff999ffb640 0 client.19245 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 -- 172.21.15.1:0/1925144837 shutdown_connections 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 1 -- 172.21.15.1:0/1925144837 wait complete. 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:38.993+0000 7ff9baa93740 20 client.19245 trim_cache size 0 max 16384 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9baa93740 1 Processor -- start 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9baa93740 1 -- start start 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746eb60 0x55e9d7470cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 0x55e9d7491590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d7479d40 0x55e9d7491ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9baa93740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01b3950 con 0x55e9d7479d40 2024-09-16T00:44:39.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9baa93740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01b1220 con 0x55e9d746eb60 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9baa93740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01af070 con 0x55e9d746f480 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9b6c57640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d7479d40 0x55e9d7491ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9b6c57640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d7479d40 0x55e9d7491ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36938/0 (socket says 172.21.15.1:36938) 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9b6c57640 1 -- 172.21.15.1:0/1120623932 learned_addr learned my addr 172.21.15.1:0/1120623932 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.011+0000 7ff9b6456640 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746eb60 0x55e9d7470cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 0x55e9d7491590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff9b6c57640 1 -- 172.21.15.1:0/1120623932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 msgr2=0x55e9d7491590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 0x55e9d7491590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff9b6c57640 1 -- 172.21.15.1:0/1120623932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746eb60 msgr2=0x55e9d7470cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746eb60 0x55e9d7470cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff9b6c57640 1 -- 172.21.15.1:0/1120623932 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9d746af20 con 0x55e9d7479d40 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 0x55e9d7491590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d7479d40 0x55e9d7491ad0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8003a30 tx=0x7ff9a8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa050d4cdb82481 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff99bfff640 1 -- 172.21.15.1:0/1120623932 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8042040 con 0x55e9d7479d40 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff99bfff640 1 -- 172.21.15.1:0/1120623932 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a8036030 con 0x55e9d7479d40 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.012+0000 7ff99bfff640 1 -- 172.21.15.1:0/1120623932 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8030c20 con 0x55e9d7479d40 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 -- 172.21.15.1:0/1120623932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d7479d40 msgr2=0x55e9d7491ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d7479d40 0x55e9d7491ad0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8003a30 tx=0x7ff9a8005ef0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 -- 172.21.15.1:0/1120623932 shutdown_connections 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d7479d40 0x55e9d7491ad0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746f480 0x55e9d7491590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1120623932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d746eb60 0x55e9d7470cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 -- 172.21.15.1:0/1120623932 >> 172.21.15.1:0/1120623932 conn(0x55e9d746b0f0 msgr2=0x55e9d74705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 -- 172.21.15.1:0/1120623932 shutdown_connections 2024-09-16T00:44:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 -- 172.21.15.1:0/1120623932 wait complete. 2024-09-16T00:44:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 -- start start 2024-09-16T00:44:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 Processor -- start 2024-09-16T00:44:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746eb60 0x55e9d747dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746f480 0x55e9d747e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 0x55e9d74f3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01af6e0 con 0x55e9d746f480 2024-09-16T00:44:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.013+0000 7ff9baa93740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01b0b50 con 0x55e9d7479d40 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9baa93740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01afdb0 con 0x55e9d746eb60 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9baa93740 10 client.0 _ll_register_callbacks cb 0x7ff9ac00b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b5c55640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746f480 0x55e9d747e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b5c55640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746f480 0x55e9d747e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36946/0 (socket says 172.21.15.1:36946) 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b5c55640 1 -- 172.21.15.1:0/1427417509 learned_addr learned my addr 172.21.15.1:0/1427417509 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b6456640 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746eb60 0x55e9d747dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b6c57640 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 0x55e9d74f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b5c55640 1 -- 172.21.15.1:0/1427417509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746eb60 msgr2=0x55e9d747dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746eb60 0x55e9d747dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b5c55640 1 -- 172.21.15.1:0/1427417509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 msgr2=0x55e9d74f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 0x55e9d74f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b5c55640 1 -- 172.21.15.1:0/1427417509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9d74f3aa0 con 0x55e9d746f480 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9b5c55640 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746f480 0x55e9d747e400 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0076480 tx=0x7ff9a00108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f28e982315b885 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff999ffb640 1 -- 172.21.15.1:0/1427417509 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0004220 con 0x55e9d746f480 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff999ffb640 1 -- 172.21.15.1:0/1427417509 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff9a00043c0 con 0x55e9d746f480 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.014+0000 7ff9baa93740 1 -- 172.21.15.1:0/1427417509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a8036030 con 0x55e9d746f480 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff999ffb640 1 -- 172.21.15.1:0/1427417509 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9a0030050 con 0x55e9d746f480 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff9baa93740 10 client.19251 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff9baa93740 1 -- 172.21.15.1:0/1427417509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9d74f4cc0 con 0x55e9d746f480 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff999ffb640 1 -- 172.21.15.1:0/1427417509 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a8036030 con 0x55e9d746f480 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff999ffb640 1 -- 172.21.15.1:0/1427417509 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff9a0003ad0 con 0x55e9d746f480 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff9baa93740 10 client.19251 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff9baa93740 10 client.19251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff9baa93740 20 client.19251 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e9d74f5470 0x55e9d74f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff9baa93740 4 client.19251 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:44:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff9baa93740 1 -- 172.21.15.1:0/1427417509 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e9d73dbd00 con 0x55e9d74f5470 2024-09-16T00:44:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.015+0000 7ff9b6456640 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e9d74f5470 0x55e9d74f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.016+0000 7ff9b6456640 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e9d74f5470 0x55e9d74f7890 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e9f5ef967e0555 server_cookie=2d9ddc48f75bba31 in_seq=0 out_seq=0 2024-09-16T00:44:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.016+0000 7ff999ffb640 10 client.19251 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.016+0000 7ff999ffb640 1 -- 172.21.15.1:0/1427417509 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e9d73dbd00 con 0x55e9d74f5470 2024-09-16T00:44:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.016+0000 7ff999ffb640 10 client.19251 handle_command_reply: tid=0 2024-09-16T00:44:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.016+0000 7ff9baa93740 1 client.19251 shutdown 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "cf0eb06f-3f1e-44bb-b7a9-5f10547a52ec", 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 -- 172.21.15.1:0/1427417509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746f480 msgr2=0x55e9d747e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746f480 0x55e9d747e400 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0076480 tx=0x7ff9a00108c0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 -- 172.21.15.1:0/1427417509 shutdown_connections 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e9d74f5470 0x55e9d74f7890 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9d7479d40 0x55e9d74f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff999ffb640 0 client.19251 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e9d746f480 0x55e9d747e400 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 --2- 172.21.15.1:0/1427417509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9d746eb60 0x55e9d747dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 -- 172.21.15.1:0/1427417509 >> 172.21.15.1:0/1427417509 conn(0x55e9d746b0f0 msgr2=0x55e9d7470dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 -- 172.21.15.1:0/1427417509 shutdown_connections 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 1 -- 172.21.15.1:0/1427417509 wait complete. 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9baa93740 20 client.19251 trim_cache size 0 max 16384 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9b86ba640 1 -- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff99c0a3240 msgr2=0x7ff99c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff99c0a3240 0x7ff99c0a5700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009ca0 tx=0x7ff9a804e000 comp rx=0 tx=0).stop 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9b86ba640 1 -- 172.21.15.1:0/479785192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0101de0 msgr2=0x7ff9b019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.017+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0101de0 0x7ff9b019a660 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00b4a0 tx=0x7ff9ac00b960 comp rx=0 tx=0).stop 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.018+0000 7ff9b86ba640 1 -- 172.21.15.1:0/479785192 shutdown_connections 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.018+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9b0106f10 0x7ff9b019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.018+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff99c0a3240 0x7ff99c0a5700 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.018+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9b0102790 0x7ff9b019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.018+0000 7ff9b86ba640 1 --2- 172.21.15.1:0/479785192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0101de0 0x7ff9b019a660 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:39.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.018+0000 7ff9b86ba640 1 -- 172.21.15.1:0/479785192 >> 172.21.15.1:0/479785192 conn(0x7ff9b00fd8c0 msgr2=0x7ff9b00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:39.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.018+0000 7ff9b86ba640 1 -- 172.21.15.1:0/479785192 shutdown_connections 2024-09-16T00:44:39.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:39.018+0000 7ff9b86ba640 1 -- 172.21.15.1:0/479785192 wait complete. 2024-09-16T00:44:39.782 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:44:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:40.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:39 smithi001 ceph-mon[30934]: pgmap v838: 129 pgs: 129 active+clean; 7.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-16T00:44:40.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:39 smithi001 ceph-mon[30934]: scrub queued for path: / 2024-09-16T00:44:40.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:39 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:40.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:39 smithi001 ceph-mon[30934]: scrub summary: active paths [/] 2024-09-16T00:44:40.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:39 smithi001 ceph-mon[30934]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:40.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:39 smithi153 ceph-mon[35538]: pgmap v838: 129 pgs: 129 active+clean; 7.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-16T00:44:40.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:39 smithi153 ceph-mon[35538]: scrub queued for path: / 2024-09-16T00:44:40.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:39 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:39 smithi153 ceph-mon[35538]: scrub summary: active paths [/] 2024-09-16T00:44:40.212 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:39 smithi153 ceph-mon[35538]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:39 smithi019 ceph-mon[35186]: pgmap v838: 129 pgs: 129 active+clean; 7.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-16T00:44:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:39 smithi019 ceph-mon[35186]: scrub queued for path: / 2024-09-16T00:44:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:39 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:39 smithi019 ceph-mon[35186]: scrub summary: active paths [/] 2024-09-16T00:44:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:39 smithi019 ceph-mon[35186]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.555+0000 7ff88cc4a640 1 -- 172.21.15.1:0/1474302144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888102790 msgr2=0x7ff8881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.555+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/1474302144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888102790 0x7ff8881067e0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff870009900 tx=0x7ff87002a410 comp rx=0 tx=0).stop 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.555+0000 7ff88cc4a640 1 -- 172.21.15.1:0/1474302144 shutdown_connections 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.555+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/1474302144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff888106f10 0x7ff88810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.555+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/1474302144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888102790 0x7ff8881067e0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.555+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/1474302144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff888101de0 0x7ff8881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.555+0000 7ff88cc4a640 1 -- 172.21.15.1:0/1474302144 >> 172.21.15.1:0/1474302144 conn(0x7ff8880fd8c0 msgr2=0x7ff8880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.555+0000 7ff88cc4a640 1 -- 172.21.15.1:0/1474302144 shutdown_connections 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.555+0000 7ff88cc4a640 1 -- 172.21.15.1:0/1474302144 wait complete. 2024-09-16T00:44:40.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff88cc4a640 1 Processor -- start 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff88cc4a640 1 -- start start 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff88cc4a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff888101de0 0x7ff888198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff88cc4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff888102790 0x7ff888198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff88cc4a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888106f10 0x7ff88819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff88cc4a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111f90 con 0x7ff888106f10 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff88cc4a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111e10 con 0x7ff888102790 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff88cc4a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff888112110 con 0x7ff888101de0 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff886d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888106f10 0x7ff88819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff886d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888106f10 0x7ff88819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58582/0 (socket says 172.21.15.1:58582) 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.556+0000 7ff886d9d640 1 -- 172.21.15.1:0/90292504 learned_addr learned my addr 172.21.15.1:0/90292504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff888102790 0x7ff888198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff88659c640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff888101de0 0x7ff888198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff886d9d640 1 -- 172.21.15.1:0/90292504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff888101de0 msgr2=0x7ff888198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff886d9d640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff888101de0 0x7ff888198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff886d9d640 1 -- 172.21.15.1:0/90292504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff888102790 msgr2=0x7ff888198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff886d9d640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff888102790 0x7ff888198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff886d9d640 1 -- 172.21.15.1:0/90292504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88819d4c0 con 0x7ff888106f10 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff888102790 0x7ff888198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff886d9d640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888106f10 0x7ff88819cde0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7ff880009fd0 tx=0x7ff88000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff87ffff640 1 -- 172.21.15.1:0/90292504 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880017070 con 0x7ff888106f10 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff88cc4a640 1 -- 172.21.15.1:0/90292504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88819d7b0 con 0x7ff888106f10 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff87ffff640 1 -- 172.21.15.1:0/90292504 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff880003e50 con 0x7ff888106f10 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff87ffff640 1 -- 172.21.15.1:0/90292504 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8800106c0 con 0x7ff888106f10 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.557+0000 7ff88cc4a640 1 -- 172.21.15.1:0/90292504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881a5090 con 0x7ff888106f10 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.560+0000 7ff87ffff640 1 -- 172.21.15.1:0/90292504 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7ff880003970 con 0x7ff888106f10 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.560+0000 7ff88f023740 1 Processor -- start 2024-09-16T00:44:40.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88f023740 1 -- start start 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88f023740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88f023740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88f023740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 0x561b8b6dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88f023740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561b8b6342e0 con 0x561b8b6cedc0 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88f023740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561b8b5dbce0 con 0x561b8b6d9680 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88f023740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561b8b6302a0 con 0x561b8b6ce4a0 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58584/0 (socket says 172.21.15.1:58584) 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88659c640 1 -- 172.21.15.1:0/2087771251 learned_addr learned my addr 172.21.15.1:0/2087771251 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88659c640 1 -- 172.21.15.1:0/2087771251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 msgr2=0x561b8b6ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88659c640 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 0x561b8b6dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88659c640 1 -- 172.21.15.1:0/2087771251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 msgr2=0x561b8b6dba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff87ffff640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff8740a3240 0x7ff8740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88659c640 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 0x561b8b6dba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff88659c640 1 -- 172.21.15.1:0/2087771251 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8b5c0440 con 0x561b8b6cedc0 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.561+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 0x561b8b6dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.562+0000 7ff87ffff640 1 -- 172.21.15.1:0/90292504 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff8800cfb80 con 0x7ff888106f10 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.562+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff8740a3240 0x7ff8740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.562+0000 7ff88659c640 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d9140 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7ff87800b820 tx=0x7ff87800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfeb41e61cf101b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.562+0000 7ff87cd99640 1 -- 172.21.15.1:0/2087771251 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87800d950 con 0x561b8b6cedc0 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.562+0000 7ff87cd99640 1 -- 172.21.15.1:0/2087771251 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff878019070 con 0x561b8b6cedc0 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.562+0000 7ff87cd99640 1 -- 172.21.15.1:0/2087771251 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780123e0 con 0x561b8b6cedc0 2024-09-16T00:44:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff8740a3240 0x7ff8740a5700 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff8881087c0 tx=0x7ff87002a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 -- 172.21.15.1:0/2087771251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 msgr2=0x561b8b6d9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d9140 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7ff87800b820 tx=0x7ff87800bce0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 -- 172.21.15.1:0/2087771251 shutdown_connections 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 0x561b8b6dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d9140 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 --2- 172.21.15.1:0/2087771251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 -- 172.21.15.1:0/2087771251 >> 172.21.15.1:0/2087771251 conn(0x561b8b6caa30 msgr2=0x561b8b6cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 -- 172.21.15.1:0/2087771251 shutdown_connections 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 -- 172.21.15.1:0/2087771251 wait complete. 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 -- start start 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 Processor -- start 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6ce4a0 0x561b8b6d8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 0x561b8b6d6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6d9680 0x561b8b6d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.563+0000 7ff88f023740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561b8b6dc250 con 0x561b8b6ce4a0 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88f023740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561b8b5ef050 con 0x561b8b6cedc0 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88f023740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561b8b6dc3d0 con 0x561b8b6d9680 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88f023740 10 client.0 _ll_register_callbacks cb 0x561b8b62f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6ce4a0 0x561b8b6d8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6ce4a0 0x561b8b6d8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58592/0 (socket says 172.21.15.1:58592) 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88659c640 1 -- 172.21.15.1:0/1056874054 learned_addr learned my addr 172.21.15.1:0/1056874054 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff886d9d640 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6d9680 0x561b8b6d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 0x561b8b6d6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88659c640 1 -- 172.21.15.1:0/1056874054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6d9680 msgr2=0x561b8b6d6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88659c640 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6d9680 0x561b8b6d6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88659c640 1 -- 172.21.15.1:0/1056874054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 msgr2=0x561b8b6d6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88659c640 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 0x561b8b6d6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88659c640 1 -- 172.21.15.1:0/1056874054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b8b3a53c0 con 0x561b8b6ce4a0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88659c640 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6ce4a0 0x561b8b6d8f50 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7ff878014b80 tx=0x7ff878014660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c52b62b832149c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff86dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff86dffb640 1 -- 172.21.15.1:0/1056874054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878023070 con 0x561b8b6ce4a0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff86dffb640 1 -- 172.21.15.1:0/1056874054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff87801f440 con 0x561b8b6ce4a0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.564+0000 7ff88f023740 1 -- 172.21.15.1:0/1056874054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff878019070 con 0x561b8b6ce4a0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff86dffb640 1 -- 172.21.15.1:0/1056874054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff87802f050 con 0x561b8b6ce4a0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff88f023740 10 client.19263 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff88f023740 1 -- 172.21.15.1:0/1056874054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561b8b6e7b10 con 0x561b8b6ce4a0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff86dffb640 1 -- 172.21.15.1:0/1056874054 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff878019070 con 0x561b8b6ce4a0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff86dffb640 1 -- 172.21.15.1:0/1056874054 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff878014ce0 con 0x561b8b6ce4a0 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff88f023740 10 client.19263 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff88f023740 10 client.19263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff88f023740 20 client.19263 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff88f023740 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561b8b6e8ec0 0x561b8b6eb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff88f023740 4 client.19263 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff88f023740 1 -- 172.21.15.1:0/1056874054 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b8b641850 con 0x561b8b6e8ec0 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.565+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561b8b6e8ec0 0x561b8b6eb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff86dffb640 10 client.19263 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561b8b6e8ec0 0x561b8b6eb2b0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ea92be8ce898d1 server_cookie=162f4d3b9e321e1b in_seq=0 out_seq=0 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff86dffb640 1 -- 172.21.15.1:0/1056874054 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b8b641850 con 0x561b8b6e8ec0 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff86dffb640 10 client.19263 handle_command_reply: tid=0 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff88f023740 1 client.19263 shutdown 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff88f023740 1 -- 172.21.15.1:0/1056874054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6ce4a0 msgr2=0x561b8b6d8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff88f023740 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6ce4a0 0x561b8b6d8f50 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7ff878014b80 tx=0x7ff878014660 comp rx=0 tx=0).stop 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff88f023740 1 -- 172.21.15.1:0/1056874054 shutdown_connections 2024-09-16T00:44:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff88f023740 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561b8b6e8ec0 0x561b8b6eb2b0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff88f023740 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6d9680 0x561b8b6d6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff88f023740 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 0x561b8b6d6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff86dffb640 0 client.19263 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:40.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.566+0000 7ff88f023740 1 --2- 172.21.15.1:0/1056874054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6ce4a0 0x561b8b6d8f50 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.567+0000 7ff88f023740 1 -- 172.21.15.1:0/1056874054 >> 172.21.15.1:0/1056874054 conn(0x561b8b6caa30 msgr2=0x561b8b6d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:40.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.567+0000 7ff88f023740 1 -- 172.21.15.1:0/1056874054 shutdown_connections 2024-09-16T00:44:40.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.567+0000 7ff88f023740 1 -- 172.21.15.1:0/1056874054 wait complete. 2024-09-16T00:44:40.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.567+0000 7ff88f023740 20 client.19263 trim_cache size 0 max 16384 2024-09-16T00:44:40.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff88f023740 1 Processor -- start 2024-09-16T00:44:40.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff88f023740 1 -- start start 2024-09-16T00:44:40.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff88f023740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff88f023740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff88f023740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 0x561b8b6d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff88f023740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b1750 con 0x561b8b6cedc0 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff88f023740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881af020 con 0x561b8b6d9680 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff88f023740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881ace70 con 0x561b8b6ce4a0 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff87dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff87dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58598/0 (socket says 172.21.15.1:58598) 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.584+0000 7ff87dd9b640 1 -- 172.21.15.1:0/3920070293 learned_addr learned my addr 172.21.15.1:0/3920070293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.585+0000 7ff87dd9b640 1 -- 172.21.15.1:0/3920070293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 msgr2=0x561b8b6e54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.585+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/3920070293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.585+0000 7ff87dd9b640 1 -- 172.21.15.1:0/3920070293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 msgr2=0x561b8b6d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.585+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/3920070293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 0x561b8b6d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.585+0000 7ff87dd9b640 1 -- 172.21.15.1:0/3920070293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8b6ca860 con 0x561b8b6cedc0 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.585+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/3920070293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d1030 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7ff870009900 tx=0x7ff87003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74207e4d3207ea44 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:40.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.585+0000 7ff86ffff640 1 -- 172.21.15.1:0/3920070293 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870035040 con 0x561b8b6cedc0 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.585+0000 7ff86ffff640 1 -- 172.21.15.1:0/3920070293 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8700044b0 con 0x561b8b6cedc0 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.585+0000 7ff86ffff640 1 -- 172.21.15.1:0/3920070293 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87003daa0 con 0x561b8b6cedc0 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 -- 172.21.15.1:0/3920070293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 msgr2=0x561b8b6d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 --2- 172.21.15.1:0/3920070293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d1030 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7ff870009900 tx=0x7ff87003f000 comp rx=0 tx=0).stop 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 -- 172.21.15.1:0/3920070293 shutdown_connections 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 --2- 172.21.15.1:0/3920070293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6d9680 0x561b8b6d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 --2- 172.21.15.1:0/3920070293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6cedc0 0x561b8b6d1030 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 --2- 172.21.15.1:0/3920070293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6e54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 -- 172.21.15.1:0/3920070293 >> 172.21.15.1:0/3920070293 conn(0x561b8b6caa30 msgr2=0x561b8b6d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 -- 172.21.15.1:0/3920070293 shutdown_connections 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 -- 172.21.15.1:0/3920070293 wait complete. 2024-09-16T00:44:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 -- start start 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 Processor -- start 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6e1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 0x561b8b6dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6d9680 0x561b8b6dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881ad4e0 con 0x561b8b6d9680 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881ae950 con 0x561b8b6cedc0 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881adbb0 con 0x561b8b6ce4a0 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.586+0000 7ff88f023740 10 client.0 _ll_register_callbacks cb 0x7ff87800b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff886d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6d9680 0x561b8b6dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff886d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6d9680 0x561b8b6dd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58608/0 (socket says 172.21.15.1:58608) 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff886d9d640 1 -- 172.21.15.1:0/2702034079 learned_addr learned my addr 172.21.15.1:0/2702034079 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff88659c640 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6e1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff87dd9b640 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 0x561b8b6dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff886d9d640 1 -- 172.21.15.1:0/2702034079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 msgr2=0x561b8b6e1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff886d9d640 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6e1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff886d9d640 1 -- 172.21.15.1:0/2702034079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 msgr2=0x561b8b6dcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff886d9d640 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 0x561b8b6dcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff886d9d640 1 -- 172.21.15.1:0/2702034079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b8b6dd830 con 0x561b8b6d9680 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff88659c640 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6e1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff886d9d640 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6d9680 0x561b8b6dd2f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7ff880005c40 tx=0x7ff88009b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb2a374134b5320 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff86dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff86dffb640 1 -- 172.21.15.1:0/2702034079 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88009f030 con 0x561b8b6d9680 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff86dffb640 1 -- 172.21.15.1:0/2702034079 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7ff880098410 con 0x561b8b6d9680 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff88f023740 1 -- 172.21.15.1:0/2702034079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8700044b0 con 0x561b8b6d9680 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff86dffb640 1 -- 172.21.15.1:0/2702034079 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff880020040 con 0x561b8b6d9680 2024-09-16T00:44:40.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff88f023740 10 client.19269 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.587+0000 7ff88f023740 1 -- 172.21.15.1:0/2702034079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561b8b7535c0 con 0x561b8b6d9680 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff86dffb640 1 -- 172.21.15.1:0/2702034079 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff880064020 con 0x561b8b6d9680 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff86dffb640 1 -- 172.21.15.1:0/2702034079 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff88009b1f0 con 0x561b8b6d9680 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff88f023740 10 client.19269 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff88f023740 10 client.19269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff88f023740 20 client.19269 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff88f023740 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561b8b753d80 0x561b8b7561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff88f023740 4 client.19269 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff88f023740 1 -- 172.21.15.1:0/2702034079 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b8b63bd60 con 0x561b8b753d80 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff88659c640 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561b8b753d80 0x561b8b7561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff88659c640 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561b8b753d80 0x561b8b7561a0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fbce76157032f7c server_cookie=dca41757bf643a40 in_seq=0 out_seq=0 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.588+0000 7ff86dffb640 10 client.19269 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:40.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff86dffb640 1 -- 172.21.15.1:0/2702034079 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561b8b63bd60 con 0x561b8b753d80 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff86dffb640 10 client.19269 handle_command_reply: tid=0 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 client.19269 shutdown 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 -- 172.21.15.1:0/2702034079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6d9680 msgr2=0x561b8b6dd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6d9680 0x561b8b6dd2f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7ff880005c40 tx=0x7ff88009b7c0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 -- 172.21.15.1:0/2702034079 shutdown_connections 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x561b8b753d80 0x561b8b7561a0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8b6d9680 0x561b8b6dd2f0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561b8b6cedc0 0x561b8b6dcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 --2- 172.21.15.1:0/2702034079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b8b6ce4a0 0x561b8b6e1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff86dffb640 0 client.19269 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 -- 172.21.15.1:0/2702034079 >> 172.21.15.1:0/2702034079 conn(0x561b8b6caa30 msgr2=0x561b8b6d7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 -- 172.21.15.1:0/2702034079 shutdown_connections 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 1 -- 172.21.15.1:0/2702034079 wait complete. 2024-09-16T00:44:40.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88f023740 20 client.19269 trim_cache size 0 max 16384 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: "cf0eb06f-3f1e-44bb-b7a9-5f10547a52ec": { 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "cf0eb06f-3f1e-44bb-b7a9-5f10547a52ec", 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:44:40.601 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:44:40.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88cc4a640 1 -- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff8740a3240 msgr2=0x7ff8740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:40.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff8740a3240 0x7ff8740a5700 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff8881087c0 tx=0x7ff87002a410 comp rx=0 tx=0).stop 2024-09-16T00:44:40.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88cc4a640 1 -- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888106f10 msgr2=0x7ff88819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:40.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.589+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888106f10 0x7ff88819cde0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7ff880009fd0 tx=0x7ff88000b260 comp rx=0 tx=0).stop 2024-09-16T00:44:40.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.590+0000 7ff88cc4a640 1 -- 172.21.15.1:0/90292504 shutdown_connections 2024-09-16T00:44:40.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.590+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff888106f10 0x7ff88819cde0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.590+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff888102790 0x7ff888198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.590+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7ff8740a3240 0x7ff8740a5700 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.590+0000 7ff88cc4a640 1 --2- 172.21.15.1:0/90292504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff888101de0 0x7ff888198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:40.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.590+0000 7ff88cc4a640 1 -- 172.21.15.1:0/90292504 >> 172.21.15.1:0/90292504 conn(0x7ff8880fd8c0 msgr2=0x7ff8880fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:40.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.590+0000 7ff88cc4a640 1 -- 172.21.15.1:0/90292504 shutdown_connections 2024-09-16T00:44:40.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:40.590+0000 7ff88cc4a640 1 -- 172.21.15.1:0/90292504 wait complete. 2024-09-16T00:44:41.298 INFO:tasks.cephfs.filesystem:scrub status for tag:cf0eb06f-3f1e-44bb-b7a9-5f10547a52ec - {'path': '/', 'tag': 'cf0eb06f-3f1e-44bb-b7a9-5f10547a52ec', 'options': 'recursive,force'} 2024-09-16T00:44:42.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:41 smithi001 ceph-mon[30934]: pgmap v839: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:44:42.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:41 smithi153 ceph-mon[35538]: pgmap v839: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:44:42.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:41 smithi019 ceph-mon[35186]: pgmap v839: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:44:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:43 smithi001 ceph-mon[30934]: pgmap v840: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:44:44.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:43 smithi001 ceph-mon[30934]: scrub summary: idle 2024-09-16T00:44:44.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:43 smithi153 ceph-mon[35538]: pgmap v840: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:44:44.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:43 smithi153 ceph-mon[35538]: scrub summary: idle 2024-09-16T00:44:44.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:43 smithi019 ceph-mon[35186]: pgmap v840: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-16T00:44:44.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:43 smithi019 ceph-mon[35186]: scrub summary: idle 2024-09-16T00:44:46.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:45 smithi001 ceph-mon[30934]: pgmap v841: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2024-09-16T00:44:46.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:45 smithi153 ceph-mon[35538]: pgmap v841: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2024-09-16T00:44:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:45 smithi019 ceph-mon[35186]: pgmap v841: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2024-09-16T00:44:48.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:47 smithi001 ceph-mon[30934]: pgmap v842: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:44:48.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:47 smithi153 ceph-mon[35538]: pgmap v842: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:44:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:47 smithi019 ceph-mon[35186]: pgmap v842: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-16T00:44:49.598 INFO:tasks.workunit.client.0.smithi001.stdout:561/600: open fd = 3 2024-09-16T00:44:49.598 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.598 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.598 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.598 INFO:tasks.workunit.client.0.smithi001.stdout:562/600: open fd = 3 2024-09-16T00:44:49.598 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.598 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.598 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:563/600: open fd = 3 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:564/600: open fd = 3 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:565/600: open fd = 3 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:566/600: open fd = 3 2024-09-16T00:44:49.599 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:567/600: open fd = 3 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:568/600: open fd = 3 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:569/600: open fd = 3 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.600 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:570/600: open fd = 3 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:571/600: open fd = 3 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:572/600: open fd = 3 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.601 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:573/600: open fd = 3 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:574/600: open fd = 3 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:575/600: open fd = 3 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:576/600: open fd = 3 2024-09-16T00:44:49.602 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:577/600: open fd = 3 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:578/600: open fd = 3 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:579/600: open fd = 3 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.603 INFO:tasks.workunit.client.0.smithi001.stdout:580/600: open fd = 3 2024-09-16T00:44:49.604 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.604 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.604 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.604 INFO:tasks.workunit.client.0.smithi001.stdout:581/600: open fd = 3 2024-09-16T00:44:49.604 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.604 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.604 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.604 INFO:tasks.workunit.client.0.smithi001.stdout:582/600: open fd = 3 2024-09-16T00:44:49.604 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:583/600: open fd = 3 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:584/600: open fd = 3 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:585/600: open fd = 3 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.605 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:586/600: open fd = 3 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:587/600: open fd = 3 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:588/600: open fd = 3 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.606 INFO:tasks.workunit.client.0.smithi001.stdout:589/600: open fd = 3 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:590/600: open fd = 3 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:591/600: open fd = 3 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:592/600: open fd = 3 2024-09-16T00:44:49.607 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:593/600: open fd = 3 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:594/600: open fd = 3 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:595/600: open fd = 3 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.608 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:596/600: open fd = 3 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:597/600: open fd = 3 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:598/600: open fd = 3 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.609 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.610 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.610 INFO:tasks.workunit.client.0.smithi001.stdout:599/600: open fd = 3 2024-09-16T00:44:49.610 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.610 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.610 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.610 INFO:tasks.workunit.client.0.smithi001.stdout:600/600: open fd = 3 2024-09-16T00:44:49.610 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.610 INFO:tasks.workunit.client.0.smithi001.stdout:write ret = 32 2024-09-16T00:44:49.611 INFO:tasks.workunit.client.0.smithi001.stdout:pread ret = 64 2024-09-16T00:44:49.611 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo OK 2024-09-16T00:44:49.612 INFO:tasks.workunit.client.0.smithi001.stdout:OK 2024-09-16T00:44:49.612 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:44:49.612 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:44:49.655 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-16T00:44:49.655 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T00:44:50.018 DEBUG:teuthology.parallel:result is None 2024-09-16T00:44:50.019 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:44:50.054 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:44:50.054 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-16T00:44:50.065 INFO:tasks.quiescer:joining Quiescers 2024-09-16T00:44:50.065 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-16T00:44:50.065 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:44:50.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:49 smithi001 ceph-mon[30934]: pgmap v843: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:44:50.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:49 smithi153 ceph-mon[35538]: pgmap v843: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:44:50.355 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:50.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:49 smithi019 ceph-mon[35186]: pgmap v843: 129 pgs: 129 active+clean; 7.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:44:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.918+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2090047847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db8101280 msgr2=0x7f2db810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.918+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2090047847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db8101280 0x7f2db810df40 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009900 tx=0x7f2da802a410 comp rx=0 tx=0).stop 2024-09-16T00:44:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.918+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2090047847 shutdown_connections 2024-09-16T00:44:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.918+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2090047847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db810e480 0x7f2db8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:50.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.918+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2090047847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db8101280 0x7f2db810df40 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.918+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2090047847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db81008d0 0x7f2db8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.918+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2090047847 >> 172.21.15.1:0/2090047847 conn(0x7f2db80fc7a0 msgr2=0x7f2db80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.918+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2090047847 shutdown_connections 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.918+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2090047847 wait complete. 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbfa13640 1 Processor -- start 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbfa13640 1 -- start start 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbfa13640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db81008d0 0x7f2db819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbfa13640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db8101280 0x7f2db819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbfa13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db810e480 0x7f2db819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbfa13640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8114250 con 0x7f2db81008d0 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbfa13640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81140d0 con 0x7f2db810e480 2024-09-16T00:44:50.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbfa13640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81143d0 con 0x7f2db8101280 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbd7af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db81008d0 0x7f2db819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbd7af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db81008d0 0x7f2db819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35756/0 (socket says 172.21.15.1:35756) 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbd7af640 1 -- 172.21.15.1:0/2529241308 learned_addr learned my addr 172.21.15.1:0/2529241308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbdfb0640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db810e480 0x7f2db819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbcfae640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db8101280 0x7f2db819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbd7af640 1 -- 172.21.15.1:0/2529241308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db8101280 msgr2=0x7f2db819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbd7af640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db8101280 0x7f2db819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbd7af640 1 -- 172.21.15.1:0/2529241308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db810e480 msgr2=0x7f2db819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbd7af640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db810e480 0x7f2db819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbd7af640 1 -- 172.21.15.1:0/2529241308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db819f6a0 con 0x7f2db81008d0 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.919+0000 7f2dbcfae640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db8101280 0x7f2db819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.920+0000 7f2dbd7af640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db81008d0 0x7f2db819a6f0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2db400b4a0 tx=0x7f2db400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.920+0000 7f2dae7fc640 1 -- 172.21.15.1:0/2529241308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4014070 con 0x7f2db81008d0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.920+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db819f990 con 0x7f2db81008d0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.920+0000 7f2dae7fc640 1 -- 172.21.15.1:0/2529241308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db4010af0 con 0x7f2db81008d0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.920+0000 7f2dae7fc640 1 -- 172.21.15.1:0/2529241308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4017d10 con 0x7f2db81008d0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.920+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db81a7270 con 0x7f2db81008d0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.921+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db8100cb0 con 0x7f2db81008d0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.922+0000 7f2dae7fc640 1 -- 172.21.15.1:0/2529241308 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2db4017070 con 0x7f2db81008d0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.923+0000 7f2dae7fc640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2da40a3240 0x7f2da40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.923+0000 7f2dbcfae640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2da40a3240 0x7f2da40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.923+0000 7f2dae7fc640 1 -- 172.21.15.1:0/2529241308 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2db40cf3d0 con 0x7f2db81008d0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.923+0000 7f2dbcfae640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2da40a3240 0x7f2da40a5700 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2da8000c00 tx=0x7f2da8005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:50.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:50.926+0000 7f2dae7fc640 1 -- 172.21.15.1:0/2529241308 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2db409bd90 con 0x7f2db81008d0 2024-09-16T00:44:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.138+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2db8109610 con 0x7f2db81008d0 2024-09-16T00:44:51.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.139+0000 7f2dae7fc640 1 -- 172.21.15.1:0/2529241308 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f2db409b690 con 0x7f2db81008d0 2024-09-16T00:44:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:44:51.139 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:44:51.139 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:44:51.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2da40a3240 msgr2=0x7f2da40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:51.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2da40a3240 0x7f2da40a5700 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2da8000c00 tx=0x7f2da8005a40 comp rx=0 tx=0).stop 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db81008d0 msgr2=0x7f2db819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db81008d0 0x7f2db819a6f0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2db400b4a0 tx=0x7f2db400b960 comp rx=0 tx=0).stop 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 shutdown_connections 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db810e480 0x7f2db819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2da40a3240 0x7f2da40a5700 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2db8101280 0x7f2db819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 --2- 172.21.15.1:0/2529241308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2db81008d0 0x7f2db819a6f0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 >> 172.21.15.1:0/2529241308 conn(0x7f2db80fc7a0 msgr2=0x7f2db80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 shutdown_connections 2024-09-16T00:44:51.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:51.141+0000 7f2dbfa13640 1 -- 172.21.15.1:0/2529241308 wait complete. 2024-09-16T00:44:51.903 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --cancel --all' 2024-09-16T00:44:51.904 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24506 quiesce db --cancel --all 2024-09-16T00:44:52.163 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:52.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:51 smithi001 ceph-mon[30934]: pgmap v844: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:44:52.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:51 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2529241308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:52.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:51 smithi153 ceph-mon[35538]: pgmap v844: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:44:52.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:51 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2529241308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:51 smithi019 ceph-mon[35186]: pgmap v844: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-16T00:44:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:51 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2529241308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 -- 172.21.15.1:0/2825661242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 msgr2=0x7f4e28104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/2825661242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 0x7f4e28104e80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009900 tx=0x7f4e1c02a410 comp rx=0 tx=0).stop 2024-09-16T00:44:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 -- 172.21.15.1:0/2825661242 shutdown_connections 2024-09-16T00:44:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/2825661242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e281055b0 0x7f4e2810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/2825661242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 0x7f4e28104e80 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/2825661242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e28100480 0x7f4e28100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 -- 172.21.15.1:0/2825661242 >> 172.21.15.1:0/2825661242 conn(0x7f4e28077210 msgr2=0x7f4e280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 -- 172.21.15.1:0/2825661242 shutdown_connections 2024-09-16T00:44:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 -- 172.21.15.1:0/2825661242 wait complete. 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 Processor -- start 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 -- start start 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e28100480 0x7f4e28199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 0x7f4e281997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e281055b0 0x7f4e281a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28110e90 con 0x7f4e28100e30 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28110d10 con 0x7f4e281055b0 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.673+0000 7f4e2ed85640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28111010 con 0x7f4e28100480 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 0x7f4e281997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 0x7f4e281997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35764/0 (socket says 172.21.15.1:35764) 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e27fff640 1 -- 172.21.15.1:0/3525729686 learned_addr learned my addr 172.21.15.1:0/3525729686 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e2d322640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e281055b0 0x7f4e281a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e28100480 0x7f4e28199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e27fff640 1 -- 172.21.15.1:0/3525729686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e28100480 msgr2=0x7f4e28199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e27fff640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e28100480 0x7f4e28199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e27fff640 1 -- 172.21.15.1:0/3525729686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e281055b0 msgr2=0x7f4e281a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e27fff640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e281055b0 0x7f4e281a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e27fff640 1 -- 172.21.15.1:0/3525729686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e281a4840 con 0x7f4e28100e30 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e27fff640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 0x7f4e281997c0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c006010 tx=0x7f4e1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e25ffb640 1 -- 172.21.15.1:0/3525729686 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c03f030 con 0x7f4e28100e30 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e25ffb640 1 -- 172.21.15.1:0/3525729686 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e1c003ce0 con 0x7f4e28100e30 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e2ed85640 1 -- 172.21.15.1:0/3525729686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e281a4a10 con 0x7f4e28100e30 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.674+0000 7f4e25ffb640 1 -- 172.21.15.1:0/3525729686 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c035070 con 0x7f4e28100e30 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.675+0000 7f4e2ed85640 1 -- 172.21.15.1:0/3525729686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e281a4e70 con 0x7f4e28100e30 2024-09-16T00:44:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e25ffb640 1 -- 172.21.15.1:0/3525729686 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f4e1c042030 con 0x7f4e28100e30 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e3115e740 1 Processor -- start 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e3115e740 1 -- start start 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 0x562a38de6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38df5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38df83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e3115e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562a38d4b200 con 0x562a38df5ff0 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e3115e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562a38cf1dc0 con 0x562a38de68e0 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e3115e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a38d471c0 con 0x562a38de7200 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.677+0000 7f4e25ffb640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e100a32c0 0x7f4e100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38df83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38df83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35770/0 (socket says 172.21.15.1:35770) 2024-09-16T00:44:52.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e27fff640 1 -- 172.21.15.1:0/3949774613 learned_addr learned my addr 172.21.15.1:0/3949774613 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e2d322640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 0x562a38de6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e2d322640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e100a32c0 0x7f4e100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/3949774613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38df5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e25ffb640 1 -- 172.21.15.1:0/3525729686 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4e1c0f02a0 con 0x7f4e28100e30 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e27fff640 1 -- 172.21.15.1:0/3949774613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 msgr2=0x562a38df5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e27fff640 1 --2- 172.21.15.1:0/3949774613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38df5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e27fff640 1 -- 172.21.15.1:0/3949774613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 msgr2=0x562a38de6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e27fff640 1 --2- 172.21.15.1:0/3949774613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 0x562a38de6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e27fff640 1 -- 172.21.15.1:0/3949774613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a38abc620 con 0x562a38df5ff0 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e2d322640 1 --2- 172.21.15.1:0/3949774613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 0x562a38de6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e27fff640 1 --2- 172.21.15.1:0/3949774613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38df83e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005ab0 tx=0x7f4e1c003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3d6172269eb3d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e2d322640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e100a32c0 0x7f4e100a5780 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4e2810ead0 tx=0x7f4e20009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e0effd640 1 -- 172.21.15.1:0/3949774613 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0ede60 con 0x562a38df5ff0 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e0effd640 1 -- 172.21.15.1:0/3949774613 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e1c03b580 con 0x562a38df5ff0 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.678+0000 7f4e0effd640 1 -- 172.21.15.1:0/3949774613 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c049070 con 0x562a38df5ff0 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.679+0000 7f4e3115e740 1 -- 172.21.15.1:0/3949774613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 msgr2=0x562a38df83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.679+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3949774613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38df83e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005ab0 tx=0x7f4e1c003e90 comp rx=0 tx=0).stop 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.679+0000 7f4e3115e740 1 -- 172.21.15.1:0/3949774613 shutdown_connections 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.679+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3949774613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38df83e0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.679+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3949774613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38df5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.679+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3949774613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 0x562a38de6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.679+0000 7f4e3115e740 1 -- 172.21.15.1:0/3949774613 >> 172.21.15.1:0/3949774613 conn(0x562a38de2f90 msgr2=0x562a38de3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.679+0000 7f4e3115e740 1 -- 172.21.15.1:0/3949774613 shutdown_connections 2024-09-16T00:44:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.679+0000 7f4e3115e740 1 -- 172.21.15.1:0/3949774613 wait complete. 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e3115e740 1 -- start start 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e3115e740 1 Processor -- start 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de68e0 0x562a38df5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de7200 0x562a38df2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 0x562a38df3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e3115e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562a38df8980 con 0x562a38de7200 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e3115e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562a38ce5c50 con 0x562a38df5ff0 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e3115e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a38df8b00 con 0x562a38de68e0 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e3115e740 10 client.0 _ll_register_callbacks cb 0x562a38cf7ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de7200 0x562a38df2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de7200 0x562a38df2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35784/0 (socket says 172.21.15.1:35784) 2024-09-16T00:44:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e27fff640 1 -- 172.21.15.1:0/1641515937 learned_addr learned my addr 172.21.15.1:0/1641515937 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de68e0 0x562a38df5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e27fff640 1 -- 172.21.15.1:0/1641515937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de68e0 msgr2=0x562a38df5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e27fff640 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de68e0 0x562a38df5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.680+0000 7f4e27fff640 1 -- 172.21.15.1:0/1641515937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 msgr2=0x562a38df3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e2d322640 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 0x562a38df3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e27fff640 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 0x562a38df3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e27fff640 1 -- 172.21.15.1:0/1641515937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a38df3950 con 0x562a38de7200 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de68e0 0x562a38df5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e27fff640 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de7200 0x562a38df2be0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0b8f50 tx=0x7f4e1c03bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5546361793fb075c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e07fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e07fff640 1 -- 172.21.15.1:0/1641515937 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0ede60 con 0x562a38de7200 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e07fff640 1 -- 172.21.15.1:0/1641515937 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4e1c0b8bf0 con 0x562a38de7200 2024-09-16T00:44:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e3115e740 1 -- 172.21.15.1:0/1641515937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a38d58770 con 0x562a38de7200 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e07fff640 1 -- 172.21.15.1:0/1641515937 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e1c049040 con 0x562a38de7200 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e3115e740 10 client.19287 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e3115e740 1 -- 172.21.15.1:0/1641515937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a38df8980 con 0x562a38de7200 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e07fff640 1 -- 172.21.15.1:0/1641515937 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562a38d58770 con 0x562a38de7200 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e07fff640 1 -- 172.21.15.1:0/1641515937 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4e1c0bc7e0 con 0x562a38de7200 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e3115e740 10 client.19287 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e3115e740 10 client.19287 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e3115e740 20 client.19287 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.681+0000 7f4e3115e740 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562a38e056c0 0x562a38e07ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.682+0000 7f4e3115e740 4 client.19287 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.682+0000 7f4e3115e740 1 -- 172.21.15.1:0/1641515937 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a38d52c80 con 0x562a38e056c0 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.682+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562a38e056c0 0x562a38e07ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.682+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562a38e056c0 0x562a38e07ae0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a70c185679f6050b server_cookie=40fcf7e5bbfb6219 in_seq=0 out_seq=0 2024-09-16T00:44:52.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.682+0000 7f4e07fff640 10 client.19287 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e07fff640 1 -- 172.21.15.1:0/1641515937 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a38d52c80 con 0x562a38e056c0 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e07fff640 10 client.19287 handle_command_reply: tid=0 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 client.19287 shutdown 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 -- 172.21.15.1:0/1641515937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de7200 msgr2=0x562a38df2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de7200 0x562a38df2be0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0b8f50 tx=0x7f4e1c03bcb0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 -- 172.21.15.1:0/1641515937 shutdown_connections 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562a38e056c0 0x562a38e07ae0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 0x562a38df3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de7200 0x562a38df2be0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e07fff640 0 client.19287 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 --2- 172.21.15.1:0/1641515937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de68e0 0x562a38df5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 -- 172.21.15.1:0/1641515937 >> 172.21.15.1:0/1641515937 conn(0x562a38de2f90 msgr2=0x562a38de8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 -- 172.21.15.1:0/1641515937 shutdown_connections 2024-09-16T00:44:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 1 -- 172.21.15.1:0/1641515937 wait complete. 2024-09-16T00:44:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.683+0000 7f4e3115e740 20 client.19287 trim_cache size 0 max 16384 2024-09-16T00:44:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.700+0000 7f4e3115e740 1 Processor -- start 2024-09-16T00:44:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.700+0000 7f4e3115e740 1 -- start start 2024-09-16T00:44:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de68e0 0x562a38e01be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38de20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 0x562a38e0d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e3115e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281af4f0 con 0x562a38de68e0 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e3115e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281acdc0 con 0x562a38df5ff0 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e3115e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281aac10 con 0x562a38de7200 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2cb21640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de68e0 0x562a38e01be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2cb21640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de68e0 0x562a38e01be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35792/0 (socket says 172.21.15.1:35792) 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2cb21640 1 -- 172.21.15.1:0/4099790344 learned_addr learned my addr 172.21.15.1:0/4099790344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2cb21640 1 -- 172.21.15.1:0/4099790344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 msgr2=0x562a38de20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/4099790344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38de20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2cb21640 1 -- 172.21.15.1:0/4099790344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 msgr2=0x562a38e0d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2d322640 1 --2- 172.21.15.1:0/4099790344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 0x562a38e0d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/4099790344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 0x562a38e0d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2cb21640 1 -- 172.21.15.1:0/4099790344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a38de2a30 con 0x562a38de68e0 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2d322640 1 --2- 172.21.15.1:0/4099790344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 0x562a38e0d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/4099790344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de68e0 0x562a38e01be0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f4e18004650 tx=0x7f4e1800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae2af1d65236651 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e0e7fc640 1 -- 172.21.15.1:0/4099790344 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18003ea0 con 0x562a38de68e0 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e0e7fc640 1 -- 172.21.15.1:0/4099790344 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e18005500 con 0x562a38de68e0 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.701+0000 7f4e0e7fc640 1 -- 172.21.15.1:0/4099790344 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18010690 con 0x562a38de68e0 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 -- 172.21.15.1:0/4099790344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de68e0 msgr2=0x562a38e01be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 --2- 172.21.15.1:0/4099790344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de68e0 0x562a38e01be0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f4e18004650 tx=0x7f4e1800cde0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 -- 172.21.15.1:0/4099790344 shutdown_connections 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 --2- 172.21.15.1:0/4099790344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38df5ff0 0x562a38e0d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 --2- 172.21.15.1:0/4099790344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38de20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 --2- 172.21.15.1:0/4099790344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38de68e0 0x562a38e01be0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 -- 172.21.15.1:0/4099790344 >> 172.21.15.1:0/4099790344 conn(0x562a38de2f90 msgr2=0x562a38de19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 -- 172.21.15.1:0/4099790344 shutdown_connections 2024-09-16T00:44:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 -- 172.21.15.1:0/4099790344 wait complete. 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 -- start start 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.702+0000 7f4e3115e740 1 Processor -- start 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 0x562a38df9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38df99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e3115e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38e68340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e3115e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281ab280 con 0x562a38df5ff0 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e3115e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281ac6f0 con 0x562a38de68e0 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e3115e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281ab950 con 0x562a38de7200 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e3115e740 10 client.0 _ll_register_callbacks cb 0x7f4e1c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2d322640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38e68340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2d322640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38e68340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35794/0 (socket says 172.21.15.1:35794) 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2d322640 1 -- 172.21.15.1:0/3308460973 learned_addr learned my addr 172.21.15.1:0/3308460973 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:44:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 0x562a38df9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e27fff640 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38df99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2d322640 1 -- 172.21.15.1:0/3308460973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 msgr2=0x562a38df99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2d322640 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38df99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2d322640 1 -- 172.21.15.1:0/3308460973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 msgr2=0x562a38df9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2d322640 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 0x562a38df9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2d322640 1 -- 172.21.15.1:0/3308460973 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a38e68980 con 0x562a38df5ff0 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e2d322640 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38e68340 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f4e20003bf0 tx=0x7f4e20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9907d2d33d28178 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.703+0000 7f4e07fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e07fff640 1 -- 172.21.15.1:0/3308460973 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20004b20 con 0x562a38df5ff0 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e07fff640 1 -- 172.21.15.1:0/3308460973 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f4e20004cc0 con 0x562a38df5ff0 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e3115e740 1 -- 172.21.15.1:0/3308460973 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e18005500 con 0x562a38df5ff0 2024-09-16T00:44:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e07fff640 1 -- 172.21.15.1:0/3308460973 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e20032050 con 0x562a38df5ff0 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e3115e740 10 client.19293 fetch_fsmap learned FSMap version 25 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e3115e740 1 -- 172.21.15.1:0/3308460973 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a38e6fce0 con 0x562a38df5ff0 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e07fff640 1 -- 172.21.15.1:0/3308460973 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e18005500 con 0x562a38df5ff0 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e07fff640 1 -- 172.21.15.1:0/3308460973 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4e2000f040 con 0x562a38df5ff0 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e3115e740 10 client.19293 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e3115e740 10 client.19293 resolve_mds: validated gid 24506 aka daemon mds.b 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e3115e740 20 client.19293 populate_metadata read hostname 'smithi001' 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562a38e70400 0x562a38e727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e3115e740 4 client.19293 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e3115e740 1 -- 172.21.15.1:0/3308460973 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x562a38ccf1e0 con 0x562a38e70400 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.704+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562a38e70400 0x562a38e727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e2cb21640 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562a38e70400 0x562a38e727f0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=453ab0f90b8a712 server_cookie=a99b81bf9a5b3b05 in_seq=0 out_seq=0 2024-09-16T00:44:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e07fff640 10 client.19293 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e07fff640 1 -- 172.21.15.1:0/3308460973 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x562a38ccf1e0 con 0x562a38e70400 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e07fff640 10 client.19293 handle_command_reply: tid=0 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e3115e740 1 client.19293 shutdown 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e3115e740 1 -- 172.21.15.1:0/3308460973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 msgr2=0x562a38e68340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38e68340 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f4e20003bf0 tx=0x7f4e20004ab0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e3115e740 1 -- 172.21.15.1:0/3308460973 shutdown_connections 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562a38e70400 0x562a38e727f0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e07fff640 0 client.19293 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562a38df5ff0 0x562a38e68340 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38de7200 0x562a38df99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e3115e740 1 --2- 172.21.15.1:0/3308460973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a38de68e0 0x562a38df9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.705+0000 7f4e3115e740 1 -- 172.21.15.1:0/3308460973 >> 172.21.15.1:0/3308460973 conn(0x562a38de2f90 msgr2=0x562a38de21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:52.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e3115e740 1 -- 172.21.15.1:0/3308460973 shutdown_connections 2024-09-16T00:44:52.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e3115e740 1 -- 172.21.15.1:0/3308460973 wait complete. 2024-09-16T00:44:52.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e3115e740 20 client.19293 trim_cache size 0 max 16384 2024-09-16T00:44:52.716 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:44:52.716 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2024-09-16T00:44:52.716 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24506, 2024-09-16T00:44:52.716 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 60, 2024-09-16T00:44:52.716 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": {} 2024-09-16T00:44:52.716 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:44:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 -- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e100a32c0 msgr2=0x7f4e100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e100a32c0 0x7f4e100a5780 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4e2810ead0 tx=0x7f4e20009600 comp rx=0 tx=0).stop 2024-09-16T00:44:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 -- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 msgr2=0x7f4e281997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 0x7f4e281997c0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c006010 tx=0x7f4e1c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 -- 172.21.15.1:0/3525729686 shutdown_connections 2024-09-16T00:44:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e281055b0 0x7f4e281a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e28100e30 0x7f4e281997c0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f4e100a32c0 0x7f4e100a5780 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 --2- 172.21.15.1:0/3525729686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e28100480 0x7f4e28199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 -- 172.21.15.1:0/3525729686 >> 172.21.15.1:0/3525729686 conn(0x7f4e28077210 msgr2=0x7f4e28103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 -- 172.21.15.1:0/3525729686 shutdown_connections 2024-09-16T00:44:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:44:52.706+0000 7f4e2ed85640 1 -- 172.21.15.1:0/3525729686 wait complete. 2024-09-16T00:44:53.405 INFO:tasks.quiescer:done joining Quiescers 2024-09-16T00:44:53.405 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-16T00:44:53.415 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-16T00:44:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:53 smithi001 ceph-mon[30934]: pgmap v845: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:44:54.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:53 smithi153 ceph-mon[35538]: pgmap v845: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:44:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:53 smithi019 ceph-mon[35186]: pgmap v845: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:44:56.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:55 smithi001 ceph-mon[30934]: pgmap v846: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:44:56.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:55 smithi153 ceph-mon[35538]: pgmap v846: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:44:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:55 smithi019 ceph-mon[35186]: pgmap v846: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-16T00:44:58.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:57 smithi001 ceph-mon[30934]: pgmap v847: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-16T00:44:58.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:57 smithi153 ceph-mon[35538]: pgmap v847: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-16T00:44:58.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:57 smithi019 ceph-mon[35186]: pgmap v847: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-16T00:45:00.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:44:59 smithi001 ceph-mon[30934]: pgmap v848: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-16T00:45:00.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:44:59 smithi153 ceph-mon[35538]: pgmap v848: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-16T00:45:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:44:59 smithi019 ceph-mon[35186]: pgmap v848: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-16T00:45:02.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:01 smithi153 ceph-mon[35538]: pgmap v849: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-09-16T00:45:02.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:01 smithi019 ceph-mon[35186]: pgmap v849: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-09-16T00:45:02.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:01 smithi001 ceph-mon[30934]: pgmap v849: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-09-16T00:45:04.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:03 smithi153 ceph-mon[35538]: pgmap v850: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-16T00:45:04.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:03 smithi019 ceph-mon[35186]: pgmap v850: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-16T00:45:04.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:03 smithi001 ceph-mon[30934]: pgmap v850: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-16T00:45:06.211 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:05 smithi153 ceph-mon[35538]: pgmap v851: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-16T00:45:06.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:05 smithi019 ceph-mon[35186]: pgmap v851: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-16T00:45:06.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:05 smithi001 ceph-mon[30934]: pgmap v851: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-16T00:45:08.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:07 smithi019 ceph-mon[35186]: pgmap v852: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-16T00:45:08.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:07 smithi001 ceph-mon[30934]: pgmap v852: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-16T00:45:08.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:07 smithi153 ceph-mon[35538]: pgmap v852: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-16T00:45:10.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:09 smithi019 ceph-mon[35186]: pgmap v853: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:45:10.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:09 smithi001 ceph-mon[30934]: pgmap v853: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:45:10.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:09 smithi153 ceph-mon[35538]: pgmap v853: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:45:11.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:45:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:12.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 -- 172.21.15.1:0/3917642241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 msgr2=0x7f247010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:12.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3917642241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 0x7f247010d950 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f2468009c10 tx=0x7f246802a6e0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 -- 172.21.15.1:0/3917642241 shutdown_connections 2024-09-16T00:45:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3917642241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 0x7f247010d950 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3917642241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2470100a80 0x7f2470100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3917642241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2470106a80 0x7f2470106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 -- 172.21.15.1:0/3917642241 >> 172.21.15.1:0/3917642241 conn(0x7f24700fc7a0 msgr2=0x7f24700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 -- 172.21.15.1:0/3917642241 shutdown_connections 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 -- 172.21.15.1:0/3917642241 wait complete. 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.097+0000 7f2474c8d640 1 Processor -- start 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f2474c8d640 1 -- start start 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f2474c8d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2470100a80 0x7f2470197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f2474c8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 0x7f2470197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f2474c8d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2470106a80 0x7f247019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f2474c8d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2470110e30 con 0x7f2470101420 2024-09-16T00:45:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f2474c8d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2470110cb0 con 0x7f2470100a80 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f2474c8d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2470110fb0 con 0x7f2470106a80 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 0x7f2470197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 0x7f2470197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32884/0 (socket says 172.21.15.1:32884) 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246dd9b640 1 -- 172.21.15.1:0/3625839625 learned_addr learned my addr 172.21.15.1:0/3625839625 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246e59c640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2470100a80 0x7f2470197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246ed9d640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2470106a80 0x7f247019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246dd9b640 1 -- 172.21.15.1:0/3625839625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2470106a80 msgr2=0x7f247019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246dd9b640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2470106a80 0x7f247019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246dd9b640 1 -- 172.21.15.1:0/3625839625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2470100a80 msgr2=0x7f2470197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246dd9b640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2470100a80 0x7f2470197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246dd9b640 1 -- 172.21.15.1:0/3625839625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247019c3e0 con 0x7f2470101420 2024-09-16T00:45:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.098+0000 7f246e59c640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2470100a80 0x7f2470197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.099+0000 7f246dd9b640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 0x7f2470197970 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f246400eb00 tx=0x7f246400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.099+0000 7f245f7fe640 1 -- 172.21.15.1:0/3625839625 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246401c070 con 0x7f2470101420 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.099+0000 7f2474c8d640 1 -- 172.21.15.1:0/3625839625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247019c6d0 con 0x7f2470101420 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.099+0000 7f245f7fe640 1 -- 172.21.15.1:0/3625839625 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f246400bb70 con 0x7f2470101420 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.099+0000 7f245f7fe640 1 -- 172.21.15.1:0/3625839625 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464016610 con 0x7f2470101420 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.099+0000 7f2474c8d640 1 -- 172.21.15.1:0/3625839625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247006a790 con 0x7f2470101420 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f245f7fe640 1 -- 172.21.15.1:0/3625839625 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f246401f070 con 0x7f2470101420 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f2477066740 1 Processor -- start 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f2477066740 1 -- start start 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f2477066740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 0x562e20bff880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f2477066740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 0x562e20c0a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f2477066740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c0ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f2477066740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562e20b652e0 con 0x562e20c0a680 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f2477066740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e20b0cce0 con 0x562e20bff4a0 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f2477066740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e20b612a0 con 0x562e20bffdc0 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.102+0000 7f246dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c0ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c0ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32888/0 (socket says 172.21.15.1:32888) 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246dd9b640 1 -- 172.21.15.1:0/2534557170 learned_addr learned my addr 172.21.15.1:0/2534557170 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246ed9d640 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 0x562e20bff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f245f7fe640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24580a3240 0x7f24580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246e59c640 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 0x562e20c0a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246dd9b640 1 -- 172.21.15.1:0/2534557170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 msgr2=0x562e20c0a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246dd9b640 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 0x562e20c0a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246dd9b640 1 -- 172.21.15.1:0/2534557170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 msgr2=0x562e20bff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246dd9b640 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 0x562e20bff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246dd9b640 1 -- 172.21.15.1:0/2534557170 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e20af1440 con 0x562e20c0a680 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246ed9d640 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 0x562e20bff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f246ed9d640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24580a3240 0x7f24580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.103+0000 7f245f7fe640 1 -- 172.21.15.1:0/3625839625 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f24640d1230 con 0x7f2470101420 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.104+0000 7f246dd9b640 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c0ca70 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2464054ad0 tx=0x7f2464024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a7f3f408642383 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.104+0000 7f245cff9640 1 -- 172.21.15.1:0/2534557170 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246409d040 con 0x562e20c0a680 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.104+0000 7f245cff9640 1 -- 172.21.15.1:0/2534557170 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2464055280 con 0x562e20c0a680 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.104+0000 7f246ed9d640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24580a3240 0x7f24580a5700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2468004af0 tx=0x7f2468009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.104+0000 7f245cff9640 1 -- 172.21.15.1:0/2534557170 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464055080 con 0x562e20c0a680 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.105+0000 7f2477066740 1 -- 172.21.15.1:0/2534557170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 msgr2=0x562e20c0ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.105+0000 7f2477066740 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c0ca70 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2464054ad0 tx=0x7f2464024000 comp rx=0 tx=0).stop 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.105+0000 7f2477066740 1 -- 172.21.15.1:0/2534557170 shutdown_connections 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.105+0000 7f2477066740 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c0ca70 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.105+0000 7f2477066740 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 0x562e20c0a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.105+0000 7f2477066740 1 --2- 172.21.15.1:0/2534557170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 0x562e20bff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.105+0000 7f2477066740 1 -- 172.21.15.1:0/2534557170 >> 172.21.15.1:0/2534557170 conn(0x562e20bfba30 msgr2=0x562e20bfbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.105+0000 7f2477066740 1 -- 172.21.15.1:0/2534557170 shutdown_connections 2024-09-16T00:45:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.105+0000 7f2477066740 1 -- 172.21.15.1:0/2534557170 wait complete. 2024-09-16T00:45:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f2477066740 1 -- start start 2024-09-16T00:45:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f2477066740 1 Processor -- start 2024-09-16T00:45:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f2477066740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 0x562e20c09f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f2477066740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bffdc0 0x562e20c07210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f2477066740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20c0a680 0x562e20c07750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f2477066740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562e20c0d250 con 0x562e20bffdc0 2024-09-16T00:45:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f2477066740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e20b20050 con 0x562e20bff4a0 2024-09-16T00:45:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f2477066740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e20c0d3d0 con 0x562e20c0a680 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f2477066740 10 client.0 _ll_register_callbacks cb 0x562e20b60160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f246dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bffdc0 0x562e20c07210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f246dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bffdc0 0x562e20c07210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32896/0 (socket says 172.21.15.1:32896) 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f246dd9b640 1 -- 172.21.15.1:0/1177617679 learned_addr learned my addr 172.21.15.1:0/1177617679 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f246e59c640 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 0x562e20c09f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.106+0000 7f246ed9d640 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20c0a680 0x562e20c07750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f246dd9b640 1 -- 172.21.15.1:0/1177617679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20c0a680 msgr2=0x562e20c07750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f246dd9b640 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20c0a680 0x562e20c07750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f246dd9b640 1 -- 172.21.15.1:0/1177617679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 msgr2=0x562e20c09f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f246dd9b640 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 0x562e20c09f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f246dd9b640 1 -- 172.21.15.1:0/1177617679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e208d63c0 con 0x562e20bffdc0 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f246dd9b640 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bffdc0 0x562e20c07210 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f24640ce980 tx=0x7f2464017c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf32cc74438dcbe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f2455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f2455ffb640 1 -- 172.21.15.1:0/1177617679 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464092840 con 0x562e20bffdc0 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f2477066740 1 -- 172.21.15.1:0/1177617679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e20b72850 con 0x562e20bffdc0 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f2455ffb640 1 -- 172.21.15.1:0/1177617679 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f246409b5a0 con 0x562e20bffdc0 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f2455ffb640 1 -- 172.21.15.1:0/1177617679 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2464092a50 con 0x562e20bffdc0 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f2477066740 10 client.19305 fetch_fsmap learned FSMap version 25 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f2477066740 1 -- 172.21.15.1:0/1177617679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e20c18bc0 con 0x562e20bffdc0 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.107+0000 7f2455ffb640 1 -- 172.21.15.1:0/1177617679 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562e20b72850 con 0x562e20bffdc0 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f2455ffb640 1 -- 172.21.15.1:0/1177617679 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f246409d760 con 0x562e20bffdc0 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f2477066740 10 client.19305 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f2477066740 10 client.19305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f2477066740 20 client.19305 populate_metadata read hostname 'smithi001' 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f2477066740 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562e20c19f10 0x562e20c1c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f2477066740 4 client.19305 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f2477066740 1 -- 172.21.15.1:0/1177617679 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e20b6cd60 con 0x562e20c19f10 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f246e59c640 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562e20c19f10 0x562e20c1c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f246e59c640 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562e20c19f10 0x562e20c1c300 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f14acc0d0627d63 server_cookie=d2e8db609f953439 in_seq=0 out_seq=0 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.108+0000 7f2455ffb640 10 client.19305 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2455ffb640 1 -- 172.21.15.1:0/1177617679 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e20b6cd60 con 0x562e20c19f10 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2455ffb640 10 client.19305 handle_command_reply: tid=0 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2477066740 1 client.19305 shutdown 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2477066740 1 -- 172.21.15.1:0/1177617679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bffdc0 msgr2=0x562e20c07210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2477066740 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bffdc0 0x562e20c07210 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f24640ce980 tx=0x7f2464017c80 comp rx=0 tx=0).stop 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2477066740 1 -- 172.21.15.1:0/1177617679 shutdown_connections 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2477066740 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562e20c19f10 0x562e20c1c300 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2477066740 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20c0a680 0x562e20c07750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2477066740 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bffdc0 0x562e20c07210 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2477066740 1 --2- 172.21.15.1:0/1177617679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bff4a0 0x562e20c09f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2455ffb640 0 client.19305 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:45:12.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.109+0000 7f2477066740 1 -- 172.21.15.1:0/1177617679 >> 172.21.15.1:0/1177617679 conn(0x562e20bfba30 msgr2=0x562e20c01760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:12.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.110+0000 7f2477066740 1 -- 172.21.15.1:0/1177617679 shutdown_connections 2024-09-16T00:45:12.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.110+0000 7f2477066740 1 -- 172.21.15.1:0/1177617679 wait complete. 2024-09-16T00:45:12.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.110+0000 7f2477066740 20 client.19305 trim_cache size 0 max 16384 2024-09-16T00:45:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f2477066740 1 Processor -- start 2024-09-16T00:45:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f2477066740 1 -- start start 2024-09-16T00:45:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f2477066740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bff4a0 0x562e20c04470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f2477066740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bffdc0 0x562e20c049b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f2477066740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f2477066740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24701af470 con 0x562e20c0a680 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f2477066740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24701acd40 con 0x562e20bffdc0 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f2477066740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24701aab90 con 0x562e20bff4a0 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32900/0 (socket says 172.21.15.1:32900) 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246ed9d640 1 -- 172.21.15.1:0/2592676326 learned_addr learned my addr 172.21.15.1:0/2592676326 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246e59c640 1 --2- 172.21.15.1:0/2592676326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bff4a0 0x562e20c04470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246dd9b640 1 --2- 172.21.15.1:0/2592676326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bffdc0 0x562e20c049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246ed9d640 1 -- 172.21.15.1:0/2592676326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bff4a0 msgr2=0x562e20c04470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246ed9d640 1 --2- 172.21.15.1:0/2592676326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bff4a0 0x562e20c04470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246ed9d640 1 -- 172.21.15.1:0/2592676326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bffdc0 msgr2=0x562e20c049b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246ed9d640 1 --2- 172.21.15.1:0/2592676326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bffdc0 0x562e20c049b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.127+0000 7f246ed9d640 1 -- 172.21.15.1:0/2592676326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e20bfb860 con 0x562e20c0a680 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f246ed9d640 1 --2- 172.21.15.1:0/2592676326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c059b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f2468009be0 tx=0x7f2468040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0b7a3066d9867b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2457fff640 1 -- 172.21.15.1:0/2592676326 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468036040 con 0x562e20c0a680 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2457fff640 1 -- 172.21.15.1:0/2592676326 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2468037040 con 0x562e20c0a680 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2457fff640 1 -- 172.21.15.1:0/2592676326 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24680317c0 con 0x562e20c0a680 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2477066740 1 -- 172.21.15.1:0/2592676326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 msgr2=0x562e20c059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2477066740 1 --2- 172.21.15.1:0/2592676326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c059b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f2468009be0 tx=0x7f2468040000 comp rx=0 tx=0).stop 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2477066740 1 -- 172.21.15.1:0/2592676326 shutdown_connections 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2477066740 1 --2- 172.21.15.1:0/2592676326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20c0a680 0x562e20c059b0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2477066740 1 --2- 172.21.15.1:0/2592676326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20bffdc0 0x562e20c049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2477066740 1 --2- 172.21.15.1:0/2592676326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bff4a0 0x562e20c04470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2477066740 1 -- 172.21.15.1:0/2592676326 >> 172.21.15.1:0/2592676326 conn(0x562e20bfba30 msgr2=0x562e20c028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2477066740 1 -- 172.21.15.1:0/2592676326 shutdown_connections 2024-09-16T00:45:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.128+0000 7f2477066740 1 -- 172.21.15.1:0/2592676326 wait complete. 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f2477066740 1 -- start start 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f2477066740 1 Processor -- start 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f2477066740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bff4a0 0x562e20c1f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f2477066740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 0x562e20c1f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f2477066740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20c0a680 0x562e20c1c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f2477066740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24701ab200 con 0x562e20bff4a0 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f2477066740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24701ac670 con 0x562e20c0a680 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f2477066740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24701ab8d0 con 0x562e20bffdc0 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f2477066740 10 client.0 _ll_register_callbacks cb 0x7f2464099660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f246e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bff4a0 0x562e20c1f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f246e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bff4a0 0x562e20c1f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32902/0 (socket says 172.21.15.1:32902) 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f246e59c640 1 -- 172.21.15.1:0/3457067344 learned_addr learned my addr 172.21.15.1:0/3457067344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f246dd9b640 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 0x562e20c1f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.129+0000 7f246ed9d640 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20c0a680 0x562e20c1c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f246e59c640 1 -- 172.21.15.1:0/3457067344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 msgr2=0x562e20c1f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f246e59c640 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 0x562e20c1f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f246e59c640 1 -- 172.21.15.1:0/3457067344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20c0a680 msgr2=0x562e20c1c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f246e59c640 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20c0a680 0x562e20c1c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f246e59c640 1 -- 172.21.15.1:0/3457067344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e20c1cce0 con 0x562e20bff4a0 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f246dd9b640 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 0x562e20c1f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f246e59c640 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bff4a0 0x562e20c1f010 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f2460004090 tx=0x7f2460004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dadc7b1a16d468e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2455ffb640 1 -- 172.21.15.1:0/3457067344 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24600124e0 con 0x562e20bff4a0 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2455ffb640 1 -- 172.21.15.1:0/3457067344 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f2460005020 con 0x562e20bff4a0 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2477066740 1 -- 172.21.15.1:0/3457067344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2468037040 con 0x562e20bff4a0 2024-09-16T00:45:12.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2455ffb640 1 -- 172.21.15.1:0/3457067344 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f246002e050 con 0x562e20bff4a0 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2477066740 10 client.19311 fetch_fsmap learned FSMap version 25 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2477066740 1 -- 172.21.15.1:0/3457067344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e20c843a0 con 0x562e20bff4a0 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2455ffb640 1 -- 172.21.15.1:0/3457067344 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2468037040 con 0x562e20bff4a0 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2455ffb640 1 -- 172.21.15.1:0/3457067344 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2460010690 con 0x562e20bff4a0 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2477066740 10 client.19311 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.130+0000 7f2477066740 10 client.19311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f2477066740 20 client.19311 populate_metadata read hostname 'smithi001' 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f2477066740 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562e20c84b50 0x562e20c86f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f2477066740 4 client.19311 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f2477066740 1 -- 172.21.15.1:0/3457067344 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e20ae9210 con 0x562e20c84b50 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f246dd9b640 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562e20c84b50 0x562e20c86f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f2455ffb640 10 client.19311 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f246dd9b640 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562e20c84b50 0x562e20c86f90 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d825a13d3c66827b server_cookie=ffb5f5470720559a in_seq=0 out_seq=0 2024-09-16T00:45:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f2455ffb640 1 -- 172.21.15.1:0/3457067344 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562e20ae9210 con 0x562e20c84b50 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f2455ffb640 10 client.19311 handle_command_reply: tid=0 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f2477066740 1 client.19311 shutdown 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.131+0000 7f2477066740 1 -- 172.21.15.1:0/3457067344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bff4a0 msgr2=0x562e20c1f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bff4a0 0x562e20c1f010 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f2460004090 tx=0x7f2460004550 comp rx=0 tx=0).stop 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 1 -- 172.21.15.1:0/3457067344 shutdown_connections 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x562e20c84b50 0x562e20c86f90 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e20c0a680 0x562e20c1c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e20bffdc0 0x562e20c1f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 1 --2- 172.21.15.1:0/3457067344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e20bff4a0 0x562e20c1f010 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 1 -- 172.21.15.1:0/3457067344 >> 172.21.15.1:0/3457067344 conn(0x562e20bfba30 msgr2=0x562e20bfa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2455ffb640 0 client.19311 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 1 -- 172.21.15.1:0/3457067344 shutdown_connections 2024-09-16T00:45:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 1 -- 172.21.15.1:0/3457067344 wait complete. 2024-09-16T00:45:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2477066740 20 client.19311 trim_cache size 0 max 16384 2024-09-16T00:45:12.139 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:45:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-16T00:45:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-16T00:45:12.139 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 -- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24580a3240 msgr2=0x7f24580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24580a3240 0x7f24580a5700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2468004af0 tx=0x7f2468009500 comp rx=0 tx=0).stop 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 -- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 msgr2=0x7f2470197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 0x7f2470197970 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f246400eb00 tx=0x7f246400ef00 comp rx=0 tx=0).stop 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 -- 172.21.15.1:0/3625839625 shutdown_connections 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f24580a3240 0x7f24580a5700 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2470106a80 0x7f247019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2470101420 0x7f2470197970 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 --2- 172.21.15.1:0/3625839625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2470100a80 0x7f2470197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 -- 172.21.15.1:0/3625839625 >> 172.21.15.1:0/3625839625 conn(0x7f24700fc7a0 msgr2=0x7f24700fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.132+0000 7f2474c8d640 1 -- 172.21.15.1:0/3625839625 shutdown_connections 2024-09-16T00:45:12.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:12.133+0000 7f2474c8d640 1 -- 172.21.15.1:0/3625839625 wait complete. 2024-09-16T00:45:12.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:11 smithi019 ceph-mon[35186]: pgmap v854: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-16T00:45:12.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:11 smithi001 ceph-mon[30934]: pgmap v854: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-16T00:45:12.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:11 smithi153 ceph-mon[35538]: pgmap v854: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-16T00:45:13.034 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:45:13.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:45:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.852+0000 7f0a0130a640 1 -- 172.21.15.1:0/3349694296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 msgr2=0x7f09fc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.852+0000 7f0a0130a640 1 --2- 172.21.15.1:0/3349694296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 0x7f09fc10ec40 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009930 tx=0x7f09ec02a420 comp rx=0 tx=0).stop 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.852+0000 7f0a0130a640 1 -- 172.21.15.1:0/3349694296 shutdown_connections 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.852+0000 7f0a0130a640 1 --2- 172.21.15.1:0/3349694296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 0x7f09fc10ec40 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.852+0000 7f0a0130a640 1 --2- 172.21.15.1:0/3349694296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09fc1036e0 0x7f09fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.852+0000 7f0a0130a640 1 --2- 172.21.15.1:0/3349694296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09fc102d30 0x7f09fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.852+0000 7f0a0130a640 1 -- 172.21.15.1:0/3349694296 >> 172.21.15.1:0/3349694296 conn(0x7f09fc0fc7d0 msgr2=0x7f09fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.852+0000 7f0a0130a640 1 -- 172.21.15.1:0/3349694296 shutdown_connections 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.852+0000 7f0a0130a640 1 -- 172.21.15.1:0/3349694296 wait complete. 2024-09-16T00:45:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.853+0000 7f0a0130a640 1 Processor -- start 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.853+0000 7f0a0130a640 1 -- start start 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.853+0000 7f0a0130a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09fc102d30 0x7f09fc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.853+0000 7f0a0130a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09fc1036e0 0x7f09fc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.853+0000 7f0a0130a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 0x7f09fc19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.853+0000 7f0a0130a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc112100 con 0x7f09fc1082a0 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.853+0000 7f0a0130a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc111f80 con 0x7f09fc102d30 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.853+0000 7f0a0130a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc112280 con 0x7f09fc1036e0 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.853+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 0x7f09fc19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 0x7f09fc19cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32922/0 (socket says 172.21.15.1:32922) 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fb7fe640 1 -- 172.21.15.1:0/1110526158 learned_addr learned my addr 172.21.15.1:0/1110526158 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09faffd640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09fc102d30 0x7f09fc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fa7fc640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09fc1036e0 0x7f09fc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fb7fe640 1 -- 172.21.15.1:0/1110526158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09fc1036e0 msgr2=0x7f09fc198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fb7fe640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09fc1036e0 0x7f09fc198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fb7fe640 1 -- 172.21.15.1:0/1110526158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09fc102d30 msgr2=0x7f09fc1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fb7fe640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09fc102d30 0x7f09fc1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fb7fe640 1 -- 172.21.15.1:0/1110526158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc19d580 con 0x7f09fc1082a0 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fa7fc640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09fc1036e0 0x7f09fc198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09fb7fe640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 0x7f09fc19cea0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f09ec005ab0 tx=0x7f09ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09f3fff640 1 -- 172.21.15.1:0/1110526158 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec03f030 con 0x7f09fc1082a0 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09f3fff640 1 -- 172.21.15.1:0/1110526158 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09ec003ce0 con 0x7f09fc1082a0 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc19d810 con 0x7f09fc1082a0 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f09f3fff640 1 -- 172.21.15.1:0/1110526158 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec035070 con 0x7f09fc1082a0 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.854+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc1a5150 con 0x7f09fc1082a0 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.855+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09fc06a850 con 0x7f09fc1082a0 2024-09-16T00:45:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.857+0000 7f09f3fff640 1 -- 172.21.15.1:0/1110526158 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f09ec042030 con 0x7f09fc1082a0 2024-09-16T00:45:13.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.857+0000 7f09f3fff640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09e80a3240 0x7f09e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:13.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.857+0000 7f09faffd640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09e80a3240 0x7f09e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:13.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.857+0000 7f09f3fff640 1 -- 172.21.15.1:0/1110526158 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f09ec0f0240 con 0x7f09fc1082a0 2024-09-16T00:45:13.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.858+0000 7f09faffd640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09e80a3240 0x7f09e80a5700 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009f20 tx=0x7f09f40094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:13.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:13.861+0000 7f09f3fff640 1 -- 172.21.15.1:0/1110526158 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09ec0bcb60 con 0x7f09fc1082a0 2024-09-16T00:45:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.076+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09fc10a2b0 con 0x7f09fc1082a0 2024-09-16T00:45:14.078 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:13 smithi001 ceph-mon[30934]: pgmap v855: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:45:14.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.077+0000 7f09f3fff640 1 -- 172.21.15.1:0/1110526158 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f09ec0bc460 con 0x7f09fc1082a0 2024-09-16T00:45:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:45:14.079 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:45:14.079 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:45:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09e80a3240 msgr2=0x7f09e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09e80a3240 0x7f09e80a5700 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009f20 tx=0x7f09f40094f0 comp rx=0 tx=0).stop 2024-09-16T00:45:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 msgr2=0x7f09fc19cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 0x7f09fc19cea0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f09ec005ab0 tx=0x7f09ec0042f0 comp rx=0 tx=0).stop 2024-09-16T00:45:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 shutdown_connections 2024-09-16T00:45:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09fc1082a0 0x7f09fc19cea0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f09e80a3240 0x7f09e80a5700 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09fc1036e0 0x7f09fc198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:14.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 --2- 172.21.15.1:0/1110526158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09fc102d30 0x7f09fc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:14.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 >> 172.21.15.1:0/1110526158 conn(0x7f09fc0fc7d0 msgr2=0x7f09fc105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:14.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.079+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 shutdown_connections 2024-09-16T00:45:14.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:14.080+0000 7f0a0130a640 1 -- 172.21.15.1:0/1110526158 wait complete. 2024-09-16T00:45:14.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:13 smithi019 ceph-mon[35186]: pgmap v855: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:45:14.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:13 smithi153 ceph-mon[35538]: pgmap v855: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:45:14.838 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:45:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:15.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:14 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1110526158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:15.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:14 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1110526158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:15.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:14 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1110526158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.615+0000 7fe789e88640 1 -- 172.21.15.1:0/1444337079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784102d40 msgr2=0x7fe784103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.615+0000 7fe789e88640 1 --2- 172.21.15.1:0/1444337079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784102d40 0x7fe784103120 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe774009900 tx=0x7fe77402a480 comp rx=0 tx=0).stop 2024-09-16T00:45:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.615+0000 7fe789e88640 1 -- 172.21.15.1:0/1444337079 shutdown_connections 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.615+0000 7fe789e88640 1 --2- 172.21.15.1:0/1444337079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78410c170 0x7fe78410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.615+0000 7fe789e88640 1 --2- 172.21.15.1:0/1444337079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe784103660 0x7fe78410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.615+0000 7fe789e88640 1 --2- 172.21.15.1:0/1444337079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784102d40 0x7fe784103120 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.615+0000 7fe789e88640 1 -- 172.21.15.1:0/1444337079 >> 172.21.15.1:0/1444337079 conn(0x7fe7840fc7a0 msgr2=0x7fe7840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.615+0000 7fe789e88640 1 -- 172.21.15.1:0/1444337079 shutdown_connections 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.615+0000 7fe789e88640 1 -- 172.21.15.1:0/1444337079 wait complete. 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.616+0000 7fe789e88640 1 Processor -- start 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.616+0000 7fe789e88640 1 -- start start 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.616+0000 7fe789e88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe784102d40 0x7fe784198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.616+0000 7fe789e88640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe784103660 0x7fe7841989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.616+0000 7fe789e88640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe78410c170 0x7fe78419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.616+0000 7fe789e88640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe784111fb0 con 0x7fe78410c170 2024-09-16T00:45:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.616+0000 7fe789e88640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe784111e30 con 0x7fe784102d40 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.616+0000 7fe789e88640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe784112130 con 0x7fe784103660 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe783fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe78410c170 0x7fe78419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe783fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe78410c170 0x7fe78419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32938/0 (socket says 172.21.15.1:32938) 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe783fff640 1 -- 172.21.15.1:0/1437122246 learned_addr learned my addr 172.21.15.1:0/1437122246 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe7837fe640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe784102d40 0x7fe784198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe784103660 0x7fe7841989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe783fff640 1 -- 172.21.15.1:0/1437122246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe784103660 msgr2=0x7fe7841989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe783fff640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe784103660 0x7fe7841989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe783fff640 1 -- 172.21.15.1:0/1437122246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe784102d40 msgr2=0x7fe784198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe783fff640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe784102d40 0x7fe784198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe783fff640 1 -- 172.21.15.1:0/1437122246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78419d440 con 0x7fe78410c170 2024-09-16T00:45:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe784103660 0x7fe7841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe783fff640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe78410c170 0x7fe78419cd60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00b970 tx=0x7fe77c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe780ff9640 1 -- 172.21.15.1:0/1437122246 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c019070 con 0x7fe78410c170 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe780ff9640 1 -- 172.21.15.1:0/1437122246 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77c00cb40 con 0x7fe78410c170 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe789e88640 1 -- 172.21.15.1:0/1437122246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78419d730 con 0x7fe78410c170 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe780ff9640 1 -- 172.21.15.1:0/1437122246 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c007650 con 0x7fe78410c170 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.617+0000 7fe789e88640 1 -- 172.21.15.1:0/1437122246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7841a5010 con 0x7fe78410c170 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe780ff9640 1 -- 172.21.15.1:0/1437122246 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe77c01c070 con 0x7fe78410c170 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe78c261740 1 Processor -- start 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe78c261740 1 -- start start 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe78c261740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe78c261740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 0x5558abc41a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe78c261740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe78c261740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5558abba72f0 con 0x5558abc4c370 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe78c261740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5558abb4ecf0 con 0x5558abc415d0 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe78c261740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5558abba32b0 con 0x5558abc42b40 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.620+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc4e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32952/0 (socket says 172.21.15.1:32952) 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe782ffd640 1 -- 172.21.15.1:0/1430654494 learned_addr learned my addr 172.21.15.1:0/1430654494 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe780ff9640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe76c0a3240 0x7fe76c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1430654494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe7837fe640 1 --2- 172.21.15.1:0/1430654494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 0x5558abc41a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe783fff640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe76c0a3240 0x7fe76c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe782ffd640 1 -- 172.21.15.1:0/1430654494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 msgr2=0x5558abc41090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1430654494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc41090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe782ffd640 1 -- 172.21.15.1:0/1430654494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 msgr2=0x5558abc41a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1430654494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 0x5558abc41a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe780ff9640 1 -- 172.21.15.1:0/1437122246 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe77c0cf320 con 0x7fe78410c170 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe782ffd640 1 -- 172.21.15.1:0/1430654494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558abb333a0 con 0x5558abc4c370 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1430654494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc4e760 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fe778013890 tx=0x7fe77800b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c2798f2c6bed08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe783fff640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe76c0a3240 0x7fe76c0a5700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe77c097220 tx=0x7fe77c06e3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe769ffb640 1 -- 172.21.15.1:0/1430654494 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77800e970 con 0x5558abc4c370 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe769ffb640 1 -- 172.21.15.1:0/1430654494 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77801e430 con 0x5558abc4c370 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.621+0000 7fe769ffb640 1 -- 172.21.15.1:0/1430654494 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778011620 con 0x5558abc4c370 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 -- 172.21.15.1:0/1430654494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 msgr2=0x5558abc4e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 --2- 172.21.15.1:0/1430654494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc4e760 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fe778013890 tx=0x7fe77800b550 comp rx=0 tx=0).stop 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 -- 172.21.15.1:0/1430654494 shutdown_connections 2024-09-16T00:45:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 --2- 172.21.15.1:0/1430654494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc4e760 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 --2- 172.21.15.1:0/1430654494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 0x5558abc41a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 --2- 172.21.15.1:0/1430654494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 -- 172.21.15.1:0/1430654494 >> 172.21.15.1:0/1430654494 conn(0x5558abc0a310 msgr2=0x5558abc3dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 -- 172.21.15.1:0/1430654494 shutdown_connections 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 -- 172.21.15.1:0/1430654494 wait complete. 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.622+0000 7fe78c261740 1 -- start start 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe78c261740 1 Processor -- start 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe78c261740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 0x5558abc4be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe78c261740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe78c261740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc49840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe78c261740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5558abc4f070 con 0x5558abc4c370 2024-09-16T00:45:15.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe78c261740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5558abb62060 con 0x5558abc415d0 2024-09-16T00:45:15.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe78c261740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5558abc4f1f0 con 0x5558abc42b40 2024-09-16T00:45:15.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe78c261740 10 client.0 _ll_register_callbacks cb 0x5558ab8b58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:15.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc49840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc49840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32962/0 (socket says 172.21.15.1:32962) 2024-09-16T00:45:15.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe782ffd640 1 -- 172.21.15.1:0/237545245 learned_addr learned my addr 172.21.15.1:0/237545245 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:15.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe782ffd640 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 0x5558abc4be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe7837fe640 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe782ffd640 1 -- 172.21.15.1:0/237545245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 msgr2=0x5558abc490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe782ffd640 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe782ffd640 1 -- 172.21.15.1:0/237545245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 msgr2=0x5558abc4be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe782ffd640 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 0x5558abc4be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.623+0000 7fe782ffd640 1 -- 172.21.15.1:0/237545245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558ab9183c0 con 0x5558abc4c370 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe782ffd640 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc49840 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe778005740 tx=0x7fe778022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b8f4b1b8aedbdf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe762ffd640 1 -- 172.21.15.1:0/237545245 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780134a0 con 0x5558abc4c370 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe762ffd640 1 -- 172.21.15.1:0/237545245 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe778029040 con 0x5558abc4c370 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe78c261740 1 -- 172.21.15.1:0/237545245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77801e430 con 0x5558abc4c370 2024-09-16T00:45:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe762ffd640 1 -- 172.21.15.1:0/237545245 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe77801e7f0 con 0x5558abc4c370 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe78c261740 10 client.19329 fetch_fsmap learned FSMap version 25 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe78c261740 1 -- 172.21.15.1:0/237545245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5558abc5abc0 con 0x5558abc4c370 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe762ffd640 1 -- 172.21.15.1:0/237545245 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77801e430 con 0x5558abc4c370 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe762ffd640 1 -- 172.21.15.1:0/237545245 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe77800f040 con 0x5558abc4c370 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe78c261740 10 client.19329 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe78c261740 10 client.19329 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.624+0000 7fe78c261740 20 client.19329 populate_metadata read hostname 'smithi001' 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.625+0000 7fe78c261740 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5558abc5bf10 0x5558abc5e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.625+0000 7fe78c261740 4 client.19329 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.625+0000 7fe78c261740 1 -- 172.21.15.1:0/237545245 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558abbb4860 con 0x5558abc5bf10 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.625+0000 7fe7837fe640 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5558abc5bf10 0x5558abc5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.625+0000 7fe762ffd640 10 client.19329 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.625+0000 7fe7837fe640 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5558abc5bf10 0x5558abc5e300 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=319424862383d6e3 server_cookie=1c0a5a4967321429 in_seq=0 out_seq=0 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe762ffd640 1 -- 172.21.15.1:0/237545245 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558abbb4860 con 0x5558abc5bf10 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe762ffd640 10 client.19329 handle_command_reply: tid=0 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 client.19329 shutdown 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 -- 172.21.15.1:0/237545245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 msgr2=0x5558abc49840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc49840 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe778005740 tx=0x7fe778022040 comp rx=0 tx=0).stop 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 -- 172.21.15.1:0/237545245 shutdown_connections 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc4c370 0x5558abc49840 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5558abc5bf10 0x5558abc5e300 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 --2- 172.21.15.1:0/237545245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc415d0 0x5558abc4be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe762ffd640 0 client.19329 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:45:15.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 -- 172.21.15.1:0/237545245 >> 172.21.15.1:0/237545245 conn(0x5558abc0a310 msgr2=0x5558abc43960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:15.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 -- 172.21.15.1:0/237545245 shutdown_connections 2024-09-16T00:45:15.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 1 -- 172.21.15.1:0/237545245 wait complete. 2024-09-16T00:45:15.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.626+0000 7fe78c261740 20 client.19329 trim_cache size 0 max 16384 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe78c261740 1 Processor -- start 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe78c261740 1 -- start start 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe78c261740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc3dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe78c261740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe78c261740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc4c370 0x5558abc48630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe78c261740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841b1830 con 0x5558abc415d0 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe78c261740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841af100 con 0x5558abc4c370 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe78c261740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841acf50 con 0x5558abc42b40 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc3dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc3dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32966/0 (socket says 172.21.15.1:32966) 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe782ffd640 1 -- 172.21.15.1:0/1778266537 learned_addr learned my addr 172.21.15.1:0/1778266537 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.640+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1778266537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc4c370 0x5558abc48630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.641+0000 7fe782ffd640 1 -- 172.21.15.1:0/1778266537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 msgr2=0x5558abc497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.641+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1778266537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.641+0000 7fe782ffd640 1 -- 172.21.15.1:0/1778266537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc4c370 msgr2=0x5558abc48630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.641+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1778266537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc4c370 0x5558abc48630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.641+0000 7fe782ffd640 1 -- 172.21.15.1:0/1778266537 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558abc3e1a0 con 0x5558abc415d0 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.641+0000 7fe782ffd640 1 --2- 172.21.15.1:0/1778266537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc3dbf0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x5558abc3e170 tx=0x7fe778010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53f22b53306b6aa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.641+0000 7fe7697fa640 1 -- 172.21.15.1:0/1778266537 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77801e430 con 0x5558abc415d0 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.641+0000 7fe7697fa640 1 -- 172.21.15.1:0/1778266537 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77800bc90 con 0x5558abc415d0 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.641+0000 7fe7697fa640 1 -- 172.21.15.1:0/1778266537 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778005020 con 0x5558abc415d0 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 -- 172.21.15.1:0/1778266537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 msgr2=0x5558abc3dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 --2- 172.21.15.1:0/1778266537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc3dbf0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x5558abc3e170 tx=0x7fe778010980 comp rx=0 tx=0).stop 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 -- 172.21.15.1:0/1778266537 shutdown_connections 2024-09-16T00:45:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 --2- 172.21.15.1:0/1778266537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc4c370 0x5558abc48630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 --2- 172.21.15.1:0/1778266537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc42b40 0x5558abc497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 --2- 172.21.15.1:0/1778266537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc3dbf0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 -- 172.21.15.1:0/1778266537 >> 172.21.15.1:0/1778266537 conn(0x5558abc0a310 msgr2=0x5558abc3cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 -- 172.21.15.1:0/1778266537 shutdown_connections 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 -- 172.21.15.1:0/1778266537 wait complete. 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 -- start start 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.642+0000 7fe78c261740 1 Processor -- start 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe78c261740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc54660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe78c261740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc42b40 0x5558abc54ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe78c261740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc4c370 0x5558abc45860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe78c261740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841ad5c0 con 0x5558abc415d0 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe78c261740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841aea30 con 0x5558abc42b40 2024-09-16T00:45:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe78c261740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841adc90 con 0x5558abc4c370 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe78c261740 10 client.0 _ll_register_callbacks cb 0x7fe774009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc54660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc54660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32968/0 (socket says 172.21.15.1:32968) 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe782ffd640 1 -- 172.21.15.1:0/2945735561 learned_addr learned my addr 172.21.15.1:0/2945735561 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe7837fe640 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc42b40 0x5558abc54ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe782ffd640 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc4c370 0x5558abc45860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe782ffd640 1 -- 172.21.15.1:0/2945735561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc4c370 msgr2=0x5558abc45860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe782ffd640 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc4c370 0x5558abc45860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe782ffd640 1 -- 172.21.15.1:0/2945735561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc42b40 msgr2=0x5558abc54ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe782ffd640 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc42b40 0x5558abc54ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe782ffd640 1 -- 172.21.15.1:0/2945735561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558abc46040 con 0x5558abc415d0 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.643+0000 7fe7837fe640 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc42b40 0x5558abc54ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe782ffd640 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc54660 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe77800c010 tx=0x7fe778005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fb51d4bf85ffcc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe762ffd640 1 -- 172.21.15.1:0/2945735561 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77801ea60 con 0x5558abc415d0 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe762ffd640 1 -- 172.21.15.1:0/2945735561 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe778026070 con 0x5558abc415d0 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe78c261740 1 -- 172.21.15.1:0/2945735561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77800bc90 con 0x5558abc415d0 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe762ffd640 1 -- 172.21.15.1:0/2945735561 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe77801ec00 con 0x5558abc415d0 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe78c261740 10 client.19335 fetch_fsmap learned FSMap version 25 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe78c261740 1 -- 172.21.15.1:0/2945735561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5558abcc63e0 con 0x5558abc415d0 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe762ffd640 1 -- 172.21.15.1:0/2945735561 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77800bc90 con 0x5558abc415d0 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe762ffd640 1 -- 172.21.15.1:0/2945735561 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe77800e850 con 0x5558abc415d0 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe78c261740 10 client.19335 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe78c261740 10 client.19335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe78c261740 20 client.19335 populate_metadata read hostname 'smithi001' 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe78c261740 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5558abcc67c0 0x5558abcc8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe78c261740 4 client.19335 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.644+0000 7fe78c261740 1 -- 172.21.15.1:0/2945735561 --> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558abbaed70 con 0x5558abcc67c0 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.645+0000 7fe7837fe640 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5558abcc67c0 0x5558abcc8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.645+0000 7fe7837fe640 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5558abcc67c0 0x5558abcc8bb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8eb03c606dd9334d server_cookie=9986411794204050 in_seq=0 out_seq=0 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.645+0000 7fe762ffd640 10 client.19335 ms_handle_connect on v2:172.21.15.153:6838/1065022363 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe762ffd640 1 -- 172.21.15.1:0/2945735561 <== mds.2 v2:172.21.15.153:6838/1065022363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558abbaed70 con 0x5558abcc67c0 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe762ffd640 10 client.19335 handle_command_reply: tid=0 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 client.19335 shutdown 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 -- 172.21.15.1:0/2945735561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 msgr2=0x5558abc54660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc54660 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe77800c010 tx=0x7fe778005e50 comp rx=0 tx=0).stop 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 -- 172.21.15.1:0/2945735561 shutdown_connections 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558abc4c370 0x5558abc45860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.153:6838/1065022363,v1:172.21.15.153:6839/1065022363] conn(0x5558abcc67c0 0x5558abcc8bb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5558abc42b40 0x5558abc54ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 --2- 172.21.15.1:0/2945735561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558abc415d0 0x5558abc54660 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 -- 172.21.15.1:0/2945735561 >> 172.21.15.1:0/2945735561 conn(0x5558abc0a310 msgr2=0x5558abc4a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe762ffd640 0 client.19335 ms_handle_reset on v2:172.21.15.153:6838/1065022363 2024-09-16T00:45:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 -- 172.21.15.1:0/2945735561 shutdown_connections 2024-09-16T00:45:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 1 -- 172.21.15.1:0/2945735561 wait complete. 2024-09-16T00:45:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe78c261740 20 client.19335 trim_cache size 0 max 16384 2024-09-16T00:45:15.656 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:45:15.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe789e88640 1 -- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe76c0a3240 msgr2=0x7fe76c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:15.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe789e88640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe76c0a3240 0x7fe76c0a5700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe77c097220 tx=0x7fe77c06e3c0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe789e88640 1 -- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe78410c170 msgr2=0x7fe78419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:15.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.646+0000 7fe789e88640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe78410c170 0x7fe78419cd60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00b970 tx=0x7fe77c00be30 comp rx=0 tx=0).stop 2024-09-16T00:45:15.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.647+0000 7fe789e88640 1 -- 172.21.15.1:0/1437122246 shutdown_connections 2024-09-16T00:45:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.647+0000 7fe789e88640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe78410c170 0x7fe78419cd60 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.647+0000 7fe789e88640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe76c0a3240 0x7fe76c0a5700 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.647+0000 7fe789e88640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe784103660 0x7fe7841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.647+0000 7fe789e88640 1 --2- 172.21.15.1:0/1437122246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe784102d40 0x7fe784198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.647+0000 7fe789e88640 1 -- 172.21.15.1:0/1437122246 >> 172.21.15.1:0/1437122246 conn(0x7fe7840fc7a0 msgr2=0x7fe7840fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.647+0000 7fe789e88640 1 -- 172.21.15.1:0/1437122246 shutdown_connections 2024-09-16T00:45:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:15.647+0000 7fe789e88640 1 -- 172.21.15.1:0/1437122246 wait complete. 2024-09-16T00:45:16.259 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:15 smithi001 ceph-mon[30934]: pgmap v856: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:45:16.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:15 smithi019 ceph-mon[35186]: pgmap v856: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:45:16.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:15 smithi153 ceph-mon[35538]: pgmap v856: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:45:16.536 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:45:16.801 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.299+0000 7fde4cfbd640 1 -- 172.21.15.1:0/1863330232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48101de0 msgr2=0x7fde481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.299+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/1863330232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48101de0 0x7fde481021c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fde38009900 tx=0x7fde3802a410 comp rx=0 tx=0).stop 2024-09-16T00:45:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.299+0000 7fde4cfbd640 1 -- 172.21.15.1:0/1863330232 shutdown_connections 2024-09-16T00:45:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.299+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/1863330232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde48106f10 0x7fde4810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.299+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/1863330232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde48102790 0x7fde481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.299+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/1863330232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48101de0 0x7fde481021c0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.299+0000 7fde4cfbd640 1 -- 172.21.15.1:0/1863330232 >> 172.21.15.1:0/1863330232 conn(0x7fde480fd8c0 msgr2=0x7fde480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.299+0000 7fde4cfbd640 1 -- 172.21.15.1:0/1863330232 shutdown_connections 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.299+0000 7fde4cfbd640 1 -- 172.21.15.1:0/1863330232 wait complete. 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde4cfbd640 1 Processor -- start 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde4cfbd640 1 -- start start 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde4cfbd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde48101de0 0x7fde48198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde4cfbd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde48102790 0x7fde481989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde4cfbd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48106f10 0x7fde4819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde4cfbd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde48111f90 con 0x7fde48106f10 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde4cfbd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde48111e10 con 0x7fde48102790 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde4cfbd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde48112110 con 0x7fde48101de0 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde46d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48106f10 0x7fde4819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde46d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48106f10 0x7fde4819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32982/0 (socket says 172.21.15.1:32982) 2024-09-16T00:45:17.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde46d9d640 1 -- 172.21.15.1:0/4234893979 learned_addr learned my addr 172.21.15.1:0/4234893979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde4659c640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde48101de0 0x7fde48198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.300+0000 7fde3ffff640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde48102790 0x7fde481989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde46d9d640 1 -- 172.21.15.1:0/4234893979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde48101de0 msgr2=0x7fde48198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde46d9d640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde48101de0 0x7fde48198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde46d9d640 1 -- 172.21.15.1:0/4234893979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde48102790 msgr2=0x7fde481989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde46d9d640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde48102790 0x7fde481989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde46d9d640 1 -- 172.21.15.1:0/4234893979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4819d440 con 0x7fde48106f10 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde3ffff640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde48102790 0x7fde481989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde46d9d640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48106f10 0x7fde4819cd60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fde4000eaa0 tx=0x7fde4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde3f7fe640 1 -- 172.21.15.1:0/4234893979 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4000cdb0 con 0x7fde48106f10 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde3f7fe640 1 -- 172.21.15.1:0/4234893979 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde40005020 con 0x7fde48106f10 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde4cfbd640 1 -- 172.21.15.1:0/4234893979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4819d730 con 0x7fde48106f10 2024-09-16T00:45:17.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde3f7fe640 1 -- 172.21.15.1:0/4234893979 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde400116b0 con 0x7fde48106f10 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.301+0000 7fde4cfbd640 1 -- 172.21.15.1:0/4234893979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde481a5010 con 0x7fde48106f10 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.303+0000 7fde3f7fe640 1 -- 172.21.15.1:0/4234893979 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fde40005560 con 0x7fde48106f10 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.304+0000 7fde4f396740 1 Processor -- start 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.304+0000 7fde4f396740 1 -- start start 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.304+0000 7fde3f7fe640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde340a3240 0x7fde340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.304+0000 7fde4f396740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca6f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.304+0000 7fde4f396740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f55d0 0x5614ca6f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4f396740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca700370 0x5614ca702760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4f396740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5614ca65b2f0 con 0x5614ca6f6b40 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4f396740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5614ca602cf0 con 0x5614ca700370 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4f396740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5614ca6572b0 con 0x5614ca6f55d0 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca6f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca6f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32992/0 (socket says 172.21.15.1:32992) 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4659c640 1 -- 172.21.15.1:0/4108158523 learned_addr learned my addr 172.21.15.1:0/4108158523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde3ffff640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde340a3240 0x7fde340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde3f7fe640 1 -- 172.21.15.1:0/4234893979 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fde400cfff0 con 0x7fde48106f10 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde3ffff640 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f55d0 0x5614ca6f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde46d9d640 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca700370 0x5614ca702760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4659c640 1 -- 172.21.15.1:0/4108158523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f55d0 msgr2=0x5614ca6f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4659c640 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f55d0 0x5614ca6f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4659c640 1 -- 172.21.15.1:0/4108158523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca700370 msgr2=0x5614ca702760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4659c640 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca700370 0x5614ca702760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4659c640 1 -- 172.21.15.1:0/4108158523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ca5e73a0 con 0x5614ca6f6b40 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde3ffff640 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f55d0 0x5614ca6f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde46d9d640 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca700370 0x5614ca702760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde4659c640 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca6f5090 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fde38009900 tx=0x7fde38031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6499cc97c82b6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde3ffff640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde340a3240 0x7fde340a5700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5614ca6f2ac0 tx=0x7fde30007240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.305+0000 7fde3cff9640 1 -- 172.21.15.1:0/4108158523 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38040070 con 0x5614ca6f6b40 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde3cff9640 1 -- 172.21.15.1:0/4108158523 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde38005020 con 0x5614ca6f6b40 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde3cff9640 1 -- 172.21.15.1:0/4108158523 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38035040 con 0x5614ca6f6b40 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde4f396740 1 -- 172.21.15.1:0/4108158523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 msgr2=0x5614ca6f5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde4f396740 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca6f5090 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fde38009900 tx=0x7fde38031630 comp rx=0 tx=0).stop 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde4f396740 1 -- 172.21.15.1:0/4108158523 shutdown_connections 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde4f396740 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca700370 0x5614ca702760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde4f396740 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f55d0 0x5614ca6f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde4f396740 1 --2- 172.21.15.1:0/4108158523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca6f5090 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde4f396740 1 -- 172.21.15.1:0/4108158523 >> 172.21.15.1:0/4108158523 conn(0x5614ca6be310 msgr2=0x5614ca6f1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde4f396740 1 -- 172.21.15.1:0/4108158523 shutdown_connections 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.306+0000 7fde4f396740 1 -- 172.21.15.1:0/4108158523 wait complete. 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4f396740 1 -- start start 2024-09-16T00:45:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4f396740 1 Processor -- start 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4f396740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f55d0 0x5614ca6ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4f396740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f6b40 0x5614ca6fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4f396740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca700370 0x5614ca6fd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4f396740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5614ca703010 con 0x5614ca6f55d0 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4f396740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5614ca616060 con 0x5614ca6f6b40 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4f396740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5614ca703190 con 0x5614ca700370 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4f396740 10 client.0 _ll_register_callbacks cb 0x5614ca3698c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f55d0 0x5614ca6ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f55d0 0x5614ca6ffc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33006/0 (socket says 172.21.15.1:33006) 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde4659c640 1 -- 172.21.15.1:0/2432821640 learned_addr learned my addr 172.21.15.1:0/2432821640 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.307+0000 7fde3ffff640 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f6b40 0x5614ca6fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4659c640 1 -- 172.21.15.1:0/2432821640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca700370 msgr2=0x5614ca6fd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4659c640 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca700370 0x5614ca6fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4659c640 1 -- 172.21.15.1:0/2432821640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f6b40 msgr2=0x5614ca6fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4659c640 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f6b40 0x5614ca6fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4659c640 1 -- 172.21.15.1:0/2432821640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ca3cc3c0 con 0x5614ca6f55d0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4659c640 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f55d0 0x5614ca6ffc50 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fde3802a930 tx=0x7fde38005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4835fec68138ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde2dffb640 1 -- 172.21.15.1:0/2432821640 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38040030 con 0x5614ca6f55d0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde2dffb640 1 -- 172.21.15.1:0/2432821640 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fde38004b50 con 0x5614ca6f55d0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4f396740 1 -- 172.21.15.1:0/2432821640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde38005020 con 0x5614ca6f55d0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde2dffb640 1 -- 172.21.15.1:0/2432821640 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde38035070 con 0x5614ca6f55d0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4f396740 10 client.19347 fetch_fsmap learned FSMap version 25 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4f396740 1 -- 172.21.15.1:0/2432821640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614ca70e930 con 0x5614ca6f55d0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde2dffb640 1 -- 172.21.15.1:0/2432821640 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde38005020 con 0x5614ca6f55d0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde2dffb640 1 -- 172.21.15.1:0/2432821640 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fde3804e420 con 0x5614ca6f55d0 2024-09-16T00:45:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4f396740 10 client.19347 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4f396740 10 client.19347 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4f396740 20 client.19347 populate_metadata read hostname 'smithi001' 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4f396740 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5614ca70fc80 0x5614ca712070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4f396740 4 client.19347 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.308+0000 7fde4f396740 1 -- 172.21.15.1:0/2432821640 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614ca668860 con 0x5614ca70fc80 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.309+0000 7fde3ffff640 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5614ca70fc80 0x5614ca712070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.309+0000 7fde3ffff640 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5614ca70fc80 0x5614ca712070 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70c3d2d1cd7e229e server_cookie=6aa417a94b8c077b in_seq=0 out_seq=0 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.309+0000 7fde2dffb640 10 client.19347 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde2dffb640 1 -- 172.21.15.1:0/2432821640 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614ca668860 con 0x5614ca70fc80 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde2dffb640 10 client.19347 handle_command_reply: tid=0 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 client.19347 shutdown 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 -- 172.21.15.1:0/2432821640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f55d0 msgr2=0x5614ca6ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f55d0 0x5614ca6ffc50 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fde3802a930 tx=0x7fde38005400 comp rx=0 tx=0).stop 2024-09-16T00:45:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 -- 172.21.15.1:0/2432821640 shutdown_connections 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca700370 0x5614ca6fd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f6b40 0x5614ca6fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5614ca70fc80 0x5614ca712070 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 --2- 172.21.15.1:0/2432821640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f55d0 0x5614ca6ffc50 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 -- 172.21.15.1:0/2432821640 >> 172.21.15.1:0/2432821640 conn(0x5614ca6be310 msgr2=0x5614ca6f7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde2dffb640 0 client.19347 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 -- 172.21.15.1:0/2432821640 shutdown_connections 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 1 -- 172.21.15.1:0/2432821640 wait complete. 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.310+0000 7fde4f396740 20 client.19347 trim_cache size 0 max 16384 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde4f396740 1 Processor -- start 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde4f396740 1 -- start start 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde4f396740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 0x5614ca70c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde4f396740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f6b40 0x5614ca717730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde4f396740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca700370 0x5614ca717c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde4f396740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde481b16d0 con 0x5614ca700370 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde4f396740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde481aefa0 con 0x5614ca6f55d0 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde4f396740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde481acdf0 con 0x5614ca6f6b40 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde46d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca700370 0x5614ca717c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde46d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca700370 0x5614ca717c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33020/0 (socket says 172.21.15.1:33020) 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.323+0000 7fde46d9d640 1 -- 172.21.15.1:0/726832083 learned_addr learned my addr 172.21.15.1:0/726832083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde4659c640 1 --2- 172.21.15.1:0/726832083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 0x5614ca70c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde46d9d640 1 -- 172.21.15.1:0/726832083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f6b40 msgr2=0x5614ca717730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde46d9d640 1 --2- 172.21.15.1:0/726832083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f6b40 0x5614ca717730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde46d9d640 1 -- 172.21.15.1:0/726832083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 msgr2=0x5614ca70c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde46d9d640 1 --2- 172.21.15.1:0/726832083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 0x5614ca70c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde46d9d640 1 -- 172.21.15.1:0/726832083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ca6f21a0 con 0x5614ca700370 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde4659c640 1 --2- 172.21.15.1:0/726832083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 0x5614ca70c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde46d9d640 1 --2- 172.21.15.1:0/726832083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca700370 0x5614ca717c70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fde4009beb0 tx=0x7fde4009c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88527d2b9de7c873 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde2ffff640 1 -- 172.21.15.1:0/726832083 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4001f040 con 0x5614ca700370 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde2ffff640 1 -- 172.21.15.1:0/726832083 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde400a4e20 con 0x5614ca700370 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde2ffff640 1 -- 172.21.15.1:0/726832083 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4007fe80 con 0x5614ca700370 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde4f396740 1 -- 172.21.15.1:0/726832083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca700370 msgr2=0x5614ca717c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.324+0000 7fde4f396740 1 --2- 172.21.15.1:0/726832083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca700370 0x5614ca717c70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fde4009beb0 tx=0x7fde4009c970 comp rx=0 tx=0).stop 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 -- 172.21.15.1:0/726832083 shutdown_connections 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 --2- 172.21.15.1:0/726832083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca700370 0x5614ca717c70 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 --2- 172.21.15.1:0/726832083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca6f6b40 0x5614ca717730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 --2- 172.21.15.1:0/726832083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 0x5614ca70c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 -- 172.21.15.1:0/726832083 >> 172.21.15.1:0/726832083 conn(0x5614ca6be310 msgr2=0x5614ca6fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 -- 172.21.15.1:0/726832083 shutdown_connections 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 -- 172.21.15.1:0/726832083 wait complete. 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 -- start start 2024-09-16T00:45:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 Processor -- start 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 0x5614ca7040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca704630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca700370 0x5614ca704b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde481ad460 con 0x5614ca6f6b40 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde481ae8d0 con 0x5614ca6f55d0 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde481adb30 con 0x5614ca700370 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4f396740 10 client.0 _ll_register_callbacks cb 0x7fde38009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde3ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca704630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde3ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca704630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33026/0 (socket says 172.21.15.1:33026) 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde3ffff640 1 -- 172.21.15.1:0/836821235 learned_addr learned my addr 172.21.15.1:0/836821235 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.325+0000 7fde4659c640 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 0x5614ca7040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde3ffff640 1 -- 172.21.15.1:0/836821235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca700370 msgr2=0x5614ca704b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde3ffff640 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca700370 0x5614ca704b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde3ffff640 1 -- 172.21.15.1:0/836821235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 msgr2=0x5614ca7040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde3ffff640 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 0x5614ca7040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde3ffff640 1 -- 172.21.15.1:0/836821235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ca6f0310 con 0x5614ca6f6b40 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde3ffff640 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca704630 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fde30006fd0 tx=0x7fde300074c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22318b0d3c7a7f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde2dffb640 1 -- 172.21.15.1:0/836821235 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3000f040 con 0x5614ca6f6b40 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde2dffb640 1 -- 172.21.15.1:0/836821235 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fde3001d040 con 0x5614ca6f6b40 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde4f396740 1 -- 172.21.15.1:0/836821235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde400a4e20 con 0x5614ca6f6b40 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde2dffb640 1 -- 172.21.15.1:0/836821235 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde30014a60 con 0x5614ca6f6b40 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde4f396740 10 client.19353 fetch_fsmap learned FSMap version 25 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde4f396740 1 -- 172.21.15.1:0/836821235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614ca77a790 con 0x5614ca6f6b40 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde2dffb640 1 -- 172.21.15.1:0/836821235 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde400a4e20 con 0x5614ca6f6b40 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde2dffb640 1 -- 172.21.15.1:0/836821235 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fde30022410 con 0x5614ca6f6b40 2024-09-16T00:45:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde4f396740 10 client.19353 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde4f396740 10 client.19353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde4f396740 20 client.19353 populate_metadata read hostname 'smithi001' 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde4f396740 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5614ca77afc0 0x5614ca77d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde4f396740 4 client.19353 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.326+0000 7fde4f396740 1 -- 172.21.15.1:0/836821235 --> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614ca662d70 con 0x5614ca77afc0 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4659c640 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5614ca77afc0 0x5614ca77d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde2dffb640 10 client.19353 ms_handle_connect on v2:172.21.15.19:6836/928170641 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4659c640 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5614ca77afc0 0x5614ca77d3e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6925214664fa403 server_cookie=b12be2b6abb5af2d in_seq=0 out_seq=0 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde2dffb640 1 -- 172.21.15.1:0/836821235 <== mds.1 v2:172.21.15.19:6836/928170641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614ca662d70 con 0x5614ca77afc0 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde2dffb640 10 client.19353 handle_command_reply: tid=0 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4f396740 1 client.19353 shutdown 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4f396740 1 -- 172.21.15.1:0/836821235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 msgr2=0x5614ca704630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4f396740 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca704630 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fde30006fd0 tx=0x7fde300074c0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4f396740 1 -- 172.21.15.1:0/836821235 shutdown_connections 2024-09-16T00:45:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4f396740 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614ca700370 0x5614ca704b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4f396740 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.19:6836/928170641,v1:172.21.15.19:6837/928170641] conn(0x5614ca77afc0 0x5614ca77d3e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4f396740 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614ca6f6b40 0x5614ca704630 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4f396740 1 --2- 172.21.15.1:0/836821235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614ca6f55d0 0x5614ca7040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.327+0000 7fde4f396740 1 -- 172.21.15.1:0/836821235 >> 172.21.15.1:0/836821235 conn(0x5614ca6be310 msgr2=0x5614ca6f7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde2dffb640 0 client.19353 ms_handle_reset on v2:172.21.15.19:6836/928170641 2024-09-16T00:45:17.360 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4f396740 1 -- 172.21.15.1:0/836821235 shutdown_connections 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4f396740 1 -- 172.21.15.1:0/836821235 wait complete. 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4f396740 20 client.19353 trim_cache size 0 max 16384 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 -- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde340a3240 msgr2=0x7fde340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde340a3240 0x7fde340a5700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5614ca6f2ac0 tx=0x7fde30007240 comp rx=0 tx=0).stop 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 -- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48106f10 msgr2=0x7fde4819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48106f10 0x7fde4819cd60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fde4000eaa0 tx=0x7fde4000ef60 comp rx=0 tx=0).stop 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 -- 172.21.15.1:0/4234893979 shutdown_connections 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde48106f10 0x7fde4819cd60 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde48102790 0x7fde481989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fde340a3240 0x7fde340a5700 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 --2- 172.21.15.1:0/4234893979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde48101de0 0x7fde48198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 -- 172.21.15.1:0/4234893979 >> 172.21.15.1:0/4234893979 conn(0x7fde480fd8c0 msgr2=0x7fde480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:17.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 -- 172.21.15.1:0/4234893979 shutdown_connections 2024-09-16T00:45:17.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:17.328+0000 7fde4cfbd640 1 -- 172.21.15.1:0/4234893979 wait complete. 2024-09-16T00:45:18.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:45:18.334 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:18.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:17 smithi019 ceph-mon[35186]: pgmap v857: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:17 smithi001 ceph-mon[30934]: pgmap v857: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:18.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:17 smithi153 ceph-mon[35538]: pgmap v857: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:18.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.854+0000 7f6387c3a640 1 -- 172.21.15.1:0/990308529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380101420 msgr2=0x7f638010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.854+0000 7f6387c3a640 1 --2- 172.21.15.1:0/990308529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380101420 0x7f638010d950 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6378009cd0 tx=0x7f637802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.854+0000 7f6387c3a640 1 -- 172.21.15.1:0/990308529 shutdown_connections 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.854+0000 7f6387c3a640 1 --2- 172.21.15.1:0/990308529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380101420 0x7f638010d950 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.854+0000 7f6387c3a640 1 --2- 172.21.15.1:0/990308529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6380100a80 0x7f6380100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.854+0000 7f6387c3a640 1 --2- 172.21.15.1:0/990308529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6380106a80 0x7f6380106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.854+0000 7f6387c3a640 1 -- 172.21.15.1:0/990308529 >> 172.21.15.1:0/990308529 conn(0x7f63800fc7a0 msgr2=0x7f63800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.854+0000 7f6387c3a640 1 -- 172.21.15.1:0/990308529 shutdown_connections 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.854+0000 7f6387c3a640 1 -- 172.21.15.1:0/990308529 wait complete. 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f6387c3a640 1 Processor -- start 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f6387c3a640 1 -- start start 2024-09-16T00:45:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f6387c3a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380100a80 0x7f63801992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f6387c3a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6380101420 0x7f6380199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f6387c3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6380106a80 0x7f63801a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f6387c3a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6380110ea0 con 0x7f6380100a80 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f6387c3a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6380110d20 con 0x7f6380106a80 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f6387c3a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6380111020 con 0x7f6380101420 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63859d6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380100a80 0x7f63801992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63859d6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380100a80 0x7f63801992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33052/0 (socket says 172.21.15.1:33052) 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63859d6640 1 -- 172.21.15.1:0/2801747226 learned_addr learned my addr 172.21.15.1:0/2801747226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63861d7640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6380106a80 0x7f63801a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63859d6640 1 -- 172.21.15.1:0/2801747226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6380101420 msgr2=0x7f6380199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63851d5640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6380101420 0x7f6380199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63859d6640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6380101420 0x7f6380199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63859d6640 1 -- 172.21.15.1:0/2801747226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6380106a80 msgr2=0x7f63801a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63859d6640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6380106a80 0x7f63801a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63859d6640 1 -- 172.21.15.1:0/2801747226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63801a4890 con 0x7f6380100a80 2024-09-16T00:45:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.855+0000 7f63861d7640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6380106a80 0x7f63801a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:45:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.856+0000 7f63859d6640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380100a80 0x7f63801992d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f637c00b4a0 tx=0x7f637c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.856+0000 7f6376ffd640 1 -- 172.21.15.1:0/2801747226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c014070 con 0x7f6380100a80 2024-09-16T00:45:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.856+0000 7f6387c3a640 1 -- 172.21.15.1:0/2801747226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63801a4a60 con 0x7f6380100a80 2024-09-16T00:45:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.856+0000 7f6376ffd640 1 -- 172.21.15.1:0/2801747226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f637c010af0 con 0x7f6380100a80 2024-09-16T00:45:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.856+0000 7f6376ffd640 1 -- 172.21.15.1:0/2801747226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c017d10 con 0x7f6380100a80 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.856+0000 7f6387c3a640 1 -- 172.21.15.1:0/2801747226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63801a4fa0 con 0x7f6380100a80 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.858+0000 7f6376ffd640 1 -- 172.21.15.1:0/2801747226 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f637c017070 con 0x7f6380100a80 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.858+0000 7f638a013740 1 Processor -- start 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f638a013740 1 -- start start 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f638a013740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a532b40 0x55e85a531090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f638a013740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a5315d0 0x55e85a531a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f638a013740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a53c370 0x55e85a53e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f638a013740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e85a4972f0 con 0x55e85a53c370 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f638a013740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e85a43ecf0 con 0x55e85a532b40 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f638a013740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e85a4932b0 con 0x55e85a5315d0 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63851d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a53c370 0x55e85a53e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63851d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a53c370 0x55e85a53e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33062/0 (socket says 172.21.15.1:33062) 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63851d5640 1 -- 172.21.15.1:0/3715884460 learned_addr learned my addr 172.21.15.1:0/3715884460 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63859d6640 1 --2- 172.21.15.1:0/3715884460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a5315d0 0x55e85a531a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63861d7640 1 --2- 172.21.15.1:0/3715884460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a532b40 0x55e85a531090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f6376ffd640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f636c0a3240 0x7f636c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63851d5640 1 -- 172.21.15.1:0/3715884460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a5315d0 msgr2=0x55e85a531a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63851d5640 1 --2- 172.21.15.1:0/3715884460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a5315d0 0x55e85a531a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63851d5640 1 -- 172.21.15.1:0/3715884460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a532b40 msgr2=0x55e85a531090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63851d5640 1 --2- 172.21.15.1:0/3715884460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a532b40 0x55e85a531090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63851d5640 1 -- 172.21.15.1:0/3715884460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85a4233a0 con 0x55e85a53c370 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.859+0000 7f63861d7640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f636c0a3240 0x7f636c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.860+0000 7f6376ffd640 1 -- 172.21.15.1:0/2801747226 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f637c0cf190 con 0x7f6380100a80 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.860+0000 7f63851d5640 1 --2- 172.21.15.1:0/3715884460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a53c370 0x55e85a53e760 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f637000ba70 tx=0x7f637000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a5632aad8f0ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.860+0000 7f636bfff640 1 -- 172.21.15.1:0/3715884460 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370017420 con 0x55e85a53c370 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.860+0000 7f636bfff640 1 -- 172.21.15.1:0/3715884460 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6370017aa0 con 0x55e85a53c370 2024-09-16T00:45:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.860+0000 7f636bfff640 1 -- 172.21.15.1:0/3715884460 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637000fa40 con 0x55e85a53c370 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 -- 172.21.15.1:0/3715884460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a53c370 msgr2=0x55e85a53e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 --2- 172.21.15.1:0/3715884460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a53c370 0x55e85a53e760 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f637000ba70 tx=0x7f637000bf30 comp rx=0 tx=0).stop 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f63861d7640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f636c0a3240 0x7f636c0a5700 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6378009430 tx=0x7f637803e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 -- 172.21.15.1:0/3715884460 shutdown_connections 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 --2- 172.21.15.1:0/3715884460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a53c370 0x55e85a53e760 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 --2- 172.21.15.1:0/3715884460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a5315d0 0x55e85a531a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 --2- 172.21.15.1:0/3715884460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a532b40 0x55e85a531090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 -- 172.21.15.1:0/3715884460 >> 172.21.15.1:0/3715884460 conn(0x55e85a4fa310 msgr2=0x55e85a52dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 -- 172.21.15.1:0/3715884460 shutdown_connections 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 -- 172.21.15.1:0/3715884460 wait complete. 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 -- start start 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.861+0000 7f638a013740 1 Processor -- start 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f638a013740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 0x55e85a53bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f638a013740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a538f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f638a013740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 0x55e85a5394d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f638a013740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e85a53f070 con 0x55e85a532b40 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f638a013740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e85a452060 con 0x55e85a5315d0 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f638a013740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e85a53f1f0 con 0x55e85a53c370 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f638a013740 10 client.0 _ll_register_callbacks cb 0x55e85a1a58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63851d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a538f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63851d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a538f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33072/0 (socket says 172.21.15.1:33072) 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63851d5640 1 -- 172.21.15.1:0/3324587959 learned_addr learned my addr 172.21.15.1:0/3324587959 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63859d6640 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 0x55e85a53bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63861d7640 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 0x55e85a5394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63851d5640 1 -- 172.21.15.1:0/3324587959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 msgr2=0x55e85a5394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63851d5640 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 0x55e85a5394d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63851d5640 1 -- 172.21.15.1:0/3324587959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 msgr2=0x55e85a53bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63851d5640 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 0x55e85a53bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63851d5640 1 -- 172.21.15.1:0/3324587959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85a2083c0 con 0x55e85a532b40 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63851d5640 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a538f70 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f63700058c0 tx=0x7f6370017cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b09eb0ce43d4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63697fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63697fa640 1 -- 172.21.15.1:0/3324587959 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370025070 con 0x55e85a532b40 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f63697fa640 1 -- 172.21.15.1:0/3324587959 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6370017830 con 0x55e85a532b40 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.862+0000 7f638a013740 1 -- 172.21.15.1:0/3324587959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6370017aa0 con 0x55e85a532b40 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f63697fa640 1 -- 172.21.15.1:0/3324587959 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6370031050 con 0x55e85a532b40 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f638a013740 10 client.19365 fetch_fsmap learned FSMap version 25 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f638a013740 1 -- 172.21.15.1:0/3324587959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e85a54aa80 con 0x55e85a532b40 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f63697fa640 1 -- 172.21.15.1:0/3324587959 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6370017aa0 con 0x55e85a532b40 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f63697fa640 1 -- 172.21.15.1:0/3324587959 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f63700046c0 con 0x55e85a532b40 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f638a013740 10 client.19365 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f638a013740 10 client.19365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f638a013740 20 client.19365 populate_metadata read hostname 'smithi001' 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f638a013740 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e85a54be30 0x55e85a54e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f638a013740 4 client.19365 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f638a013740 1 -- 172.21.15.1:0/3324587959 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e85a4a4860 con 0x55e85a54be30 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.863+0000 7f63859d6640 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e85a54be30 0x55e85a54e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.864+0000 7f63697fa640 10 client.19365 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.864+0000 7f63859d6640 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e85a54be30 0x55e85a54e220 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1afa01713b01dca2 server_cookie=5cb998aac6c70824 in_seq=0 out_seq=0 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f63697fa640 1 -- 172.21.15.1:0/3324587959 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e85a4a4860 con 0x55e85a54be30 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f63697fa640 10 client.19365 handle_command_reply: tid=0 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 client.19365 shutdown 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 -- 172.21.15.1:0/3324587959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 msgr2=0x55e85a538f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a538f70 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f63700058c0 tx=0x7f6370017cd0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 -- 172.21.15.1:0/3324587959 shutdown_connections 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e85a54be30 0x55e85a54e220 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 0x55e85a5394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a538f70 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 --2- 172.21.15.1:0/3324587959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 0x55e85a53bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f63697fa640 0 client.19365 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 -- 172.21.15.1:0/3324587959 >> 172.21.15.1:0/3324587959 conn(0x55e85a4fa310 msgr2=0x55e85a533960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 -- 172.21.15.1:0/3324587959 shutdown_connections 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 1 -- 172.21.15.1:0/3324587959 wait complete. 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.865+0000 7f638a013740 20 client.19365 trim_cache size 0 max 16384 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f638a013740 1 Processor -- start 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f638a013740 1 -- start start 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f638a013740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a5315d0 0x55e85a548310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f638a013740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a532b40 0x55e85a53aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f638a013740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a53c370 0x55e85a53d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f638a013740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f63801b1800 con 0x55e85a5315d0 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f638a013740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63801af0d0 con 0x55e85a53c370 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f638a013740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63801acf20 con 0x55e85a532b40 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f63859d6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a5315d0 0x55e85a548310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f63859d6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a5315d0 0x55e85a548310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33082/0 (socket says 172.21.15.1:33082) 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.878+0000 7f63859d6640 1 -- 172.21.15.1:0/746139783 learned_addr learned my addr 172.21.15.1:0/746139783 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f63861d7640 1 --2- 172.21.15.1:0/746139783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a53c370 0x55e85a53d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f63859d6640 1 -- 172.21.15.1:0/746139783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a532b40 msgr2=0x55e85a53aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f63859d6640 1 --2- 172.21.15.1:0/746139783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a532b40 0x55e85a53aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f63859d6640 1 -- 172.21.15.1:0/746139783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a53c370 msgr2=0x55e85a53d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f63859d6640 1 --2- 172.21.15.1:0/746139783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a53c370 0x55e85a53d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f63859d6640 1 -- 172.21.15.1:0/746139783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85a52e960 con 0x55e85a5315d0 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f63859d6640 1 --2- 172.21.15.1:0/746139783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a5315d0 0x55e85a548310 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f637c00be60 tx=0x7f637c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2057334bf95370 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f636b7fe640 1 -- 172.21.15.1:0/746139783 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c01e040 con 0x55e85a5315d0 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f636b7fe640 1 -- 172.21.15.1:0/746139783 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f637c099650 con 0x55e85a5315d0 2024-09-16T00:45:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.879+0000 7f636b7fe640 1 -- 172.21.15.1:0/746139783 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c0d1110 con 0x55e85a5315d0 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 -- 172.21.15.1:0/746139783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a5315d0 msgr2=0x55e85a548310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 --2- 172.21.15.1:0/746139783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a5315d0 0x55e85a548310 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f637c00be60 tx=0x7f637c0047d0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 -- 172.21.15.1:0/746139783 shutdown_connections 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 --2- 172.21.15.1:0/746139783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a53c370 0x55e85a53d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 --2- 172.21.15.1:0/746139783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a532b40 0x55e85a53aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 --2- 172.21.15.1:0/746139783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a5315d0 0x55e85a548310 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 -- 172.21.15.1:0/746139783 >> 172.21.15.1:0/746139783 conn(0x55e85a4fa310 msgr2=0x55e85a53a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 -- 172.21.15.1:0/746139783 shutdown_connections 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 -- 172.21.15.1:0/746139783 wait complete. 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 -- start start 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.880+0000 7f638a013740 1 Processor -- start 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f638a013740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 0x55e85a544890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f638a013740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a53d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f638a013740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 0x55e85a53d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f638a013740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f63801ad590 con 0x55e85a532b40 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f638a013740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63801aea00 con 0x55e85a5315d0 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f638a013740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63801adc60 con 0x55e85a53c370 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f638a013740 10 client.0 _ll_register_callbacks cb 0x7f637000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63851d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a53d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63859d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 0x55e85a544890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63851d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a53d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33098/0 (socket says 172.21.15.1:33098) 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63851d5640 1 -- 172.21.15.1:0/672974114 learned_addr learned my addr 172.21.15.1:0/672974114 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63861d7640 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 0x55e85a53d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63851d5640 1 -- 172.21.15.1:0/672974114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 msgr2=0x55e85a53d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63851d5640 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 0x55e85a53d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63851d5640 1 -- 172.21.15.1:0/672974114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 msgr2=0x55e85a544890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63851d5640 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 0x55e85a544890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63851d5640 1 -- 172.21.15.1:0/672974114 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85a53de20 con 0x55e85a532b40 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63851d5640 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a53d3a0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f63700058c0 tx=0x7f6370005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f5da1d05b1526b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.881+0000 7f63697fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f63697fa640 1 -- 172.21.15.1:0/672974114 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637003a060 con 0x55e85a532b40 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f63697fa640 1 -- 172.21.15.1:0/672974114 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f6370004860 con 0x55e85a532b40 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f638a013740 1 -- 172.21.15.1:0/672974114 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f637c099650 con 0x55e85a532b40 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f63697fa640 1 -- 172.21.15.1:0/672974114 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6370041050 con 0x55e85a532b40 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f638a013740 10 client.19371 fetch_fsmap learned FSMap version 25 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f638a013740 1 -- 172.21.15.1:0/672974114 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e85a543180 con 0x55e85a532b40 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f63697fa640 1 -- 172.21.15.1:0/672974114 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f637c099650 con 0x55e85a532b40 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f63697fa640 1 -- 172.21.15.1:0/672974114 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6370005000 con 0x55e85a532b40 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f638a013740 10 client.19371 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:45:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f638a013740 10 client.19371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f638a013740 20 client.19371 populate_metadata read hostname 'smithi001' 2024-09-16T00:45:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f638a013740 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e85a5b6ce0 0x55e85a5b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f638a013740 4 client.19371 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:45:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f638a013740 1 -- 172.21.15.1:0/672974114 --> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e85a49ed70 con 0x55e85a5b6ce0 2024-09-16T00:45:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.882+0000 7f63859d6640 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e85a5b6ce0 0x55e85a5b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f63859d6640 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e85a5b6ce0 0x55e85a5b9100 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f88df3407f073d server_cookie=55a1e7b39a5f32d2 in_seq=0 out_seq=0 2024-09-16T00:45:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f63697fa640 10 client.19371 ms_handle_connect on v2:172.21.15.19:6838/3056769092 2024-09-16T00:45:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f63697fa640 1 -- 172.21.15.1:0/672974114 <== mds.0 v2:172.21.15.19:6838/3056769092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e85a49ed70 con 0x55e85a5b6ce0 2024-09-16T00:45:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f63697fa640 10 client.19371 handle_command_reply: tid=0 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 client.19371 shutdown 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 -- 172.21.15.1:0/672974114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 msgr2=0x55e85a53d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a53d3a0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f63700058c0 tx=0x7f6370005ef0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 -- 172.21.15.1:0/672974114 shutdown_connections 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.19:6838/3056769092,v1:172.21.15.19:6839/3056769092] conn(0x55e85a5b6ce0 0x55e85a5b9100 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e85a53c370 0x55e85a53d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85a532b40 0x55e85a53d3a0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f63697fa640 0 client.19371 ms_handle_reset on v2:172.21.15.19:6838/3056769092 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 --2- 172.21.15.1:0/672974114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a5315d0 0x55e85a544890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 -- 172.21.15.1:0/672974114 >> 172.21.15.1:0/672974114 conn(0x55e85a4fa310 msgr2=0x55e85a537710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 -- 172.21.15.1:0/672974114 shutdown_connections 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.883+0000 7f638a013740 1 -- 172.21.15.1:0/672974114 wait complete. 2024-09-16T00:45:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f638a013740 20 client.19371 trim_cache size 0 max 16384 2024-09-16T00:45:18.901 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-16T00:45:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 -- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f636c0a3240 msgr2=0x7f636c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f636c0a3240 0x7f636c0a5700 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6378009430 tx=0x7f637803e040 comp rx=0 tx=0).stop 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 -- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380100a80 msgr2=0x7f63801992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380100a80 0x7f63801992d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f637c00b4a0 tx=0x7f637c00b960 comp rx=0 tx=0).stop 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 -- 172.21.15.1:0/2801747226 shutdown_connections 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6380106a80 0x7f63801a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f636c0a3240 0x7f636c0a5700 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6380101420 0x7f6380199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 --2- 172.21.15.1:0/2801747226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380100a80 0x7f63801992d0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 -- 172.21.15.1:0/2801747226 >> 172.21.15.1:0/2801747226 conn(0x7f63800fc7a0 msgr2=0x7f63800fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 -- 172.21.15.1:0/2801747226 shutdown_connections 2024-09-16T00:45:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:18.884+0000 7f6387c3a640 1 -- 172.21.15.1:0/2801747226 wait complete. 2024-09-16T00:45:19.793 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-16T00:45:19.794 INFO:tasks.fwd_scrub:done joining 2024-09-16T00:45:19.794 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-16T00:45:19.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.914+0000 7f495440e640 1 -- 172.21.15.1:0/4221874677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 msgr2=0x7f494c1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.914+0000 7f495440e640 1 --2- 172.21.15.1:0/4221874677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 0x7f494c1020c0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f493c009900 tx=0x7f493c02a410 comp rx=0 tx=0).stop 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 -- 172.21.15.1:0/4221874677 shutdown_connections 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 --2- 172.21.15.1:0/4221874677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f494c102600 0x7f494c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 --2- 172.21.15.1:0/4221874677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 0x7f494c1020c0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 --2- 172.21.15.1:0/4221874677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c100a50 0x7f494c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 -- 172.21.15.1:0/4221874677 >> 172.21.15.1:0/4221874677 conn(0x7f494c0fc1f0 msgr2=0x7f494c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 -- 172.21.15.1:0/4221874677 shutdown_connections 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 -- 172.21.15.1:0/4221874677 wait complete. 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 Processor -- start 2024-09-16T00:45:19.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 -- start start 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c100a50 0x7f494c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 0x7f494c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f494c102600 0x7f494c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f494c10c6d0 con 0x7f494c101c40 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f494c10c550 con 0x7f494c100a50 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.915+0000 7f495440e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f494c10c850 con 0x7f494c102600 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49519a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 0x7f494c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49519a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 0x7f494c197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33118/0 (socket says 172.21.15.1:33118) 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49519a9640 1 -- 172.21.15.1:0/1104607545 learned_addr learned my addr 172.21.15.1:0/1104607545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49521aa640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c100a50 0x7f494c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49529ab640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f494c102600 0x7f494c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:19.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49519a9640 1 -- 172.21.15.1:0/1104607545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f494c102600 msgr2=0x7f494c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49519a9640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f494c102600 0x7f494c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49519a9640 1 -- 172.21.15.1:0/1104607545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c100a50 msgr2=0x7f494c196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49519a9640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c100a50 0x7f494c196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49519a9640 1 -- 172.21.15.1:0/1104607545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494c19e960 con 0x7f494c101c40 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49529ab640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f494c102600 0x7f494c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49519a9640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 0x7f494c197290 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f493c009230 tx=0x7f493c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49437fe640 1 -- 172.21.15.1:0/1104607545 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c046070 con 0x7f494c101c40 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49437fe640 1 -- 172.21.15.1:0/1104607545 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f493c0044d0 con 0x7f494c101c40 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494c0694f0 con 0x7f494c101c40 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f49437fe640 1 -- 172.21.15.1:0/1104607545 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c035050 con 0x7f494c101c40 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.916+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494c0699e0 con 0x7f494c101c40 2024-09-16T00:45:19.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.917+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f494c100e50 con 0x7f494c101c40 2024-09-16T00:45:19.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.920+0000 7f49437fe640 1 -- 172.21.15.1:0/1104607545 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f493c049070 con 0x7f494c101c40 2024-09-16T00:45:19.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.920+0000 7f49437fe640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49380a3250 0x7f49380a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:19.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.920+0000 7f49437fe640 1 -- 172.21.15.1:0/1104607545 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f493c0f1c80 con 0x7f494c101c40 2024-09-16T00:45:19.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.920+0000 7f49521aa640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49380a3250 0x7f49380a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:19.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.921+0000 7f49521aa640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49380a3250 0x7f49380a5710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f494c198220 tx=0x7f4948009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:19.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:19.923+0000 7f49437fe640 1 -- 172.21.15.1:0/1104607545 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f493c050080 con 0x7f494c101c40 2024-09-16T00:45:20.073 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:19 smithi001 ceph-mon[30934]: pgmap v858: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-16T00:45:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.102+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f494c107900 con 0x7f494c101c40 2024-09-16T00:45:20.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.104+0000 7f49437fe640 1 -- 172.21.15.1:0/1104607545 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24902 (secure 0 0 0) 0x7f494c107900 con 0x7f494c101c40 2024-09-16T00:45:20.106 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:45:20.106 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:27:26.243912+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49380a3250 msgr2=0x7f49380a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49380a3250 0x7f49380a5710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f494c198220 tx=0x7f4948009670 comp rx=0 tx=0).stop 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 msgr2=0x7f494c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 0x7f494c197290 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f493c009230 tx=0x7f493c0042e0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 shutdown_connections 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f49380a3250 0x7f49380a5710 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f494c102600 0x7f494c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c101c40 0x7f494c197290 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 --2- 172.21.15.1:0/1104607545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c100a50 0x7f494c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 >> 172.21.15.1:0/1104607545 conn(0x7f494c0fc1f0 msgr2=0x7f494c105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:20.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 shutdown_connections 2024-09-16T00:45:20.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.106+0000 7f495440e640 1 -- 172.21.15.1:0/1104607545 wait complete. 2024-09-16T00:45:20.120 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:21:40.721155+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:26:14.641676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:26:17.242600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-16T00:26:23.721714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-16T00:45:20.120 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T00:45:20.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.228+0000 7f262f331640 1 -- 172.21.15.1:0/1425617190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628104f60 msgr2=0x7f2628107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.228+0000 7f262f331640 1 --2- 172.21.15.1:0/1425617190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628104f60 0x7f2628107350 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f2618009900 tx=0x7f261802a410 comp rx=0 tx=0).stop 2024-09-16T00:45:20.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.228+0000 7f262f331640 1 -- 172.21.15.1:0/1425617190 shutdown_connections 2024-09-16T00:45:20.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.228+0000 7f262f331640 1 --2- 172.21.15.1:0/1425617190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2628107890 0x7f2628109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.228+0000 7f262f331640 1 --2- 172.21.15.1:0/1425617190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628104f60 0x7f2628107350 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.228+0000 7f262f331640 1 --2- 172.21.15.1:0/1425617190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2628102630 0x7f2628104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.228+0000 7f262f331640 1 -- 172.21.15.1:0/1425617190 >> 172.21.15.1:0/1425617190 conn(0x7f26280fc190 msgr2=0x7f26280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:20.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.228+0000 7f262f331640 1 -- 172.21.15.1:0/1425617190 shutdown_connections 2024-09-16T00:45:20.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.228+0000 7f262f331640 1 -- 172.21.15.1:0/1425617190 wait complete. 2024-09-16T00:45:20.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.229+0000 7f262f331640 1 Processor -- start 2024-09-16T00:45:20.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.229+0000 7f262f331640 1 -- start start 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.229+0000 7f262f331640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628102630 0x7f2628197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.229+0000 7f262f331640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2628104f60 0x7f26281983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.229+0000 7f262f331640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2628107890 0x7f262819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.229+0000 7f262f331640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f262810d7c0 con 0x7f2628102630 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.229+0000 7f262f331640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f262810d640 con 0x7f2628107890 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.229+0000 7f262f331640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f262810d940 con 0x7f2628104f60 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.229+0000 7f262d0cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628102630 0x7f2628197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262d0cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628102630 0x7f2628197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33128/0 (socket says 172.21.15.1:33128) 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262d0cd640 1 -- 172.21.15.1:0/1561212317 learned_addr learned my addr 172.21.15.1:0/1561212317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:20.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262c8cc640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2628104f60 0x7f26281983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262d8ce640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2628107890 0x7f262819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262d0cd640 1 -- 172.21.15.1:0/1561212317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2628104f60 msgr2=0x7f26281983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262d0cd640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2628104f60 0x7f26281983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262d0cd640 1 -- 172.21.15.1:0/1561212317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2628107890 msgr2=0x7f262819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262d0cd640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2628107890 0x7f262819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262d0cd640 1 -- 172.21.15.1:0/1561212317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262819fac0 con 0x7f2628102630 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262d0cd640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628102630 0x7f2628197ea0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f262400d950 tx=0x7f262400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f261e7fc640 1 -- 172.21.15.1:0/1561212317 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624017070 con 0x7f2628102630 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f261e7fc640 1 -- 172.21.15.1:0/1561212317 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2624009c30 con 0x7f2628102630 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262819fdb0 con 0x7f2628102630 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f261e7fc640 1 -- 172.21.15.1:0/1561212317 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262400f6b0 con 0x7f2628102630 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.230+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26281a0210 con 0x7f2628102630 2024-09-16T00:45:20.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.232+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26280fdc80 con 0x7f2628102630 2024-09-16T00:45:20.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.233+0000 7f261e7fc640 1 -- 172.21.15.1:0/1561212317 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f26240109d0 con 0x7f2628102630 2024-09-16T00:45:20.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.233+0000 7f261e7fc640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26140a3250 0x7f26140a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.234+0000 7f261e7fc640 1 -- 172.21.15.1:0/1561212317 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f26240cf8b0 con 0x7f2628102630 2024-09-16T00:45:20.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.234+0000 7f262c8cc640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26140a3250 0x7f26140a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.234+0000 7f262c8cc640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26140a3250 0x7f26140a5710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2618005ab0 tx=0x7f2618000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:20.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.238+0000 7f261e7fc640 1 -- 172.21.15.1:0/1561212317 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2624009de0 con 0x7f2628102630 2024-09-16T00:45:20.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:19 smithi019 ceph-mon[35186]: pgmap v858: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-16T00:45:20.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.431+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2628062ef0 con 0x7f2628102630 2024-09-16T00:45:20.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.431+0000 7f261e7fc640 1 -- 172.21.15.1:0/1561212317 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f262409bb80 con 0x7f2628102630 2024-09-16T00:45:20.433 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-09-16T00:45:20.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26140a3250 msgr2=0x7f26140a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26140a3250 0x7f26140a5710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2618005ab0 tx=0x7f2618000bc0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628102630 msgr2=0x7f2628197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628102630 0x7f2628197ea0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f262400d950 tx=0x7f262400de10 comp rx=0 tx=0).stop 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 shutdown_connections 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2628107890 0x7f262819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f26140a3250 0x7f26140a5710 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2628104f60 0x7f26281983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 --2- 172.21.15.1:0/1561212317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2628102630 0x7f2628197ea0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 >> 172.21.15.1:0/1561212317 conn(0x7f26280fc190 msgr2=0x7f2628103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 shutdown_connections 2024-09-16T00:45:20.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.433+0000 7f262f331640 1 -- 172.21.15.1:0/1561212317 wait complete. 2024-09-16T00:45:20.446 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T00:45:20.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:19 smithi153 ceph-mon[35538]: pgmap v858: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-16T00:45:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.553+0000 7fb81c045640 1 -- 172.21.15.1:0/24206199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814101c40 msgr2=0x7fb8141020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.553+0000 7fb81c045640 1 --2- 172.21.15.1:0/24206199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814101c40 0x7fb8141020c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fb804009900 tx=0x7fb80402a410 comp rx=0 tx=0).stop 2024-09-16T00:45:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.553+0000 7fb81c045640 1 -- 172.21.15.1:0/24206199 shutdown_connections 2024-09-16T00:45:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.553+0000 7fb81c045640 1 --2- 172.21.15.1:0/24206199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb814102600 0x7fb814108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.553+0000 7fb81c045640 1 --2- 172.21.15.1:0/24206199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814101c40 0x7fb8141020c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.553+0000 7fb81c045640 1 --2- 172.21.15.1:0/24206199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb814100a50 0x7fb814100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.553+0000 7fb81c045640 1 -- 172.21.15.1:0/24206199 >> 172.21.15.1:0/24206199 conn(0x7fb8140fc190 msgr2=0x7fb8140fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.553+0000 7fb81c045640 1 -- 172.21.15.1:0/24206199 shutdown_connections 2024-09-16T00:45:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.553+0000 7fb81c045640 1 -- 172.21.15.1:0/24206199 wait complete. 2024-09-16T00:45:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81c045640 1 Processor -- start 2024-09-16T00:45:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81c045640 1 -- start start 2024-09-16T00:45:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81c045640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb814100a50 0x7fb8141049e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81c045640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb814101c40 0x7fb814103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81c045640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814102600 0x7fb814103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81c045640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb81410c7a0 con 0x7fb814102600 2024-09-16T00:45:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81c045640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb81410c620 con 0x7fb814100a50 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81c045640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb81410c920 con 0x7fb814101c40 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81a5e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814102600 0x7fb814103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81a5e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814102600 0x7fb814103560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32936/0 (socket says 172.21.15.1:32936) 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.554+0000 7fb81a5e2640 1 -- 172.21.15.1:0/2707719535 learned_addr learned my addr 172.21.15.1:0/2707719535 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb819de1640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb814100a50 0x7fb8141049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb8195e0640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb814101c40 0x7fb814103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb81a5e2640 1 -- 172.21.15.1:0/2707719535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb814101c40 msgr2=0x7fb814103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb81a5e2640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb814101c40 0x7fb814103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb81a5e2640 1 -- 172.21.15.1:0/2707719535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb814100a50 msgr2=0x7fb8141049e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb81a5e2640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb814100a50 0x7fb8141049e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb81a5e2640 1 -- 172.21.15.1:0/2707719535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb814103df0 con 0x7fb814102600 2024-09-16T00:45:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb8195e0640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb814101c40 0x7fb814103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb81a5e2640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814102600 0x7fb814103560 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00cce0 tx=0x7fb80c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb80affd640 1 -- 172.21.15.1:0/2707719535 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00f450 con 0x7fb814102600 2024-09-16T00:45:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8141a1fe0 con 0x7fb814102600 2024-09-16T00:45:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb80affd640 1 -- 172.21.15.1:0/2707719535 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb80c00fad0 con 0x7fb814102600 2024-09-16T00:45:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb80affd640 1 -- 172.21.15.1:0/2707719535 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c018c10 con 0x7fb814102600 2024-09-16T00:45:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.555+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8141a24d0 con 0x7fb814102600 2024-09-16T00:45:20.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.556+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8141a27f0 con 0x7fb814102600 2024-09-16T00:45:20.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.558+0000 7fb80affd640 1 -- 172.21.15.1:0/2707719535 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fb80c00f5f0 con 0x7fb814102600 2024-09-16T00:45:20.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.558+0000 7fb80affd640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8000a3250 0x7fb8000a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.558+0000 7fb80affd640 1 -- 172.21.15.1:0/2707719535 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fb80c0d0540 con 0x7fb814102600 2024-09-16T00:45:20.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.558+0000 7fb819de1640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8000a3250 0x7fb8000a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.559+0000 7fb819de1640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8000a3250 0x7fb8000a5710 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb814101aa0 tx=0x7fb810008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:20.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.561+0000 7fb80affd640 1 -- 172.21.15.1:0/2707719535 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb80c014030 con 0x7fb814102600 2024-09-16T00:45:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.739+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8141a2b80 con 0x7fb814102600 2024-09-16T00:45:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.739+0000 7fb80affd640 1 -- 172.21.15.1:0/2707719535 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fb80c09cef0 con 0x7fb814102600 2024-09-16T00:45:20.741 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-09-16T00:45:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8000a3250 msgr2=0x7fb8000a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8000a3250 0x7fb8000a5710 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb814101aa0 tx=0x7fb810008040 comp rx=0 tx=0).stop 2024-09-16T00:45:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814102600 msgr2=0x7fb814103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814102600 0x7fb814103560 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00cce0 tx=0x7fb80c00a620 comp rx=0 tx=0).stop 2024-09-16T00:45:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 shutdown_connections 2024-09-16T00:45:20.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb814102600 0x7fb814103560 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fb8000a3250 0x7fb8000a5710 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb814101c40 0x7fb814103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 --2- 172.21.15.1:0/2707719535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb814100a50 0x7fb8141049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 >> 172.21.15.1:0/2707719535 conn(0x7fb8140fc190 msgr2=0x7fb8140fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:20.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 shutdown_connections 2024-09-16T00:45:20.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.741+0000 7fb81c045640 1 -- 172.21.15.1:0/2707719535 wait complete. 2024-09-16T00:45:20.754 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T00:45:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.860+0000 7f690060a640 1 -- 172.21.15.1:0/1137760222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 msgr2=0x7f68f8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.860+0000 7f690060a640 1 --2- 172.21.15.1:0/1137760222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 0x7f68f8101f60 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009900 tx=0x7f68ec02a3f0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 -- 172.21.15.1:0/1137760222 shutdown_connections 2024-09-16T00:45:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 --2- 172.21.15.1:0/1137760222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f8103710 0x7f68f8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 --2- 172.21.15.1:0/1137760222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68f8102d50 0x7f68f81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 --2- 172.21.15.1:0/1137760222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 0x7f68f8101f60 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 -- 172.21.15.1:0/1137760222 >> 172.21.15.1:0/1137760222 conn(0x7f68f80fd2e0 msgr2=0x7f68f80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 -- 172.21.15.1:0/1137760222 shutdown_connections 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 -- 172.21.15.1:0/1137760222 wait complete. 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 Processor -- start 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 -- start start 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 0x7f68f8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68f8102d50 0x7f68f80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f8103710 0x7f68f8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68f810d7c0 con 0x7f68f8101b60 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68f810d640 con 0x7f68f8103710 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.861+0000 7f690060a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68f810d940 con 0x7f68f8102d50 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fe3a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 0x7f68f8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fe3a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 0x7f68f8075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32946/0 (socket says 172.21.15.1:32946) 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fe3a6640 1 -- 172.21.15.1:0/3942275914 learned_addr learned my addr 172.21.15.1:0/3942275914 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68feba7640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f8103710 0x7f68f8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fdba5640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68f8102d50 0x7f68f80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fe3a6640 1 -- 172.21.15.1:0/3942275914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68f8102d50 msgr2=0x7f68f80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fe3a6640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68f8102d50 0x7f68f80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fe3a6640 1 -- 172.21.15.1:0/3942275914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f8103710 msgr2=0x7f68f8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fe3a6640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f8103710 0x7f68f8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fe3a6640 1 -- 172.21.15.1:0/3942275914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f80753b0 con 0x7f68f8101b60 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fdba5640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68f8102d50 0x7f68f80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68fe3a6640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 0x7f68f8075fa0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f68ec031610 tx=0x7f68ec0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68eb7fe640 1 -- 172.21.15.1:0/3942275914 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec03f030 con 0x7f68f8101b60 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68eb7fe640 1 -- 172.21.15.1:0/3942275914 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ec003ce0 con 0x7f68f8101b60 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f81a1e80 con 0x7f68f8101b60 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f68eb7fe640 1 -- 172.21.15.1:0/3942275914 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec035070 con 0x7f68f8101b60 2024-09-16T00:45:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.862+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f81a2290 con 0x7f68f8101b60 2024-09-16T00:45:20.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.864+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f8108e80 con 0x7f68f8101b60 2024-09-16T00:45:20.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.865+0000 7f68eb7fe640 1 -- 172.21.15.1:0/3942275914 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f68ec042030 con 0x7f68f8101b60 2024-09-16T00:45:20.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.865+0000 7f68eb7fe640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68e40a3250 0x7f68e40a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:20.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.865+0000 7f68fdba5640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68e40a3250 0x7f68e40a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:20.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.865+0000 7f68eb7fe640 1 -- 172.21.15.1:0/3942275914 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f68ec0f0380 con 0x7f68f8101b60 2024-09-16T00:45:20.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.866+0000 7f68fdba5640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68e40a3250 0x7f68e40a5710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f68f8075d10 tx=0x7f68f4005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:20.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:20.873+0000 7f68eb7fe640 1 -- 172.21.15.1:0/3942275914 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68ec0bcca0 con 0x7f68f8101b60 2024-09-16T00:45:21.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.052+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f68f8062ef0 con 0x7f68f8101b60 2024-09-16T00:45:21.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.052+0000 7f68eb7fe640 1 -- 172.21.15.1:0/3942275914 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f68ec0bc5a0 con 0x7f68f8101b60 2024-09-16T00:45:21.054 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-09-16T00:45:21.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68e40a3250 msgr2=0x7f68e40a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68e40a3250 0x7f68e40a5710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f68f8075d10 tx=0x7f68f4005d90 comp rx=0 tx=0).stop 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 msgr2=0x7f68f8075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 0x7f68f8075fa0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f68ec031610 tx=0x7f68ec0043b0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 shutdown_connections 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f8103710 0x7f68f8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f68e40a3250 0x7f68e40a5710 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68f8102d50 0x7f68f80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 --2- 172.21.15.1:0/3942275914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68f8101b60 0x7f68f8075fa0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 >> 172.21.15.1:0/3942275914 conn(0x7f68f80fd2e0 msgr2=0x7f68f8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 shutdown_connections 2024-09-16T00:45:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.055+0000 7f690060a640 1 -- 172.21.15.1:0/3942275914 wait complete. 2024-09-16T00:45:21.070 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:20 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1104607545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:45:21.070 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:20 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1561212317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:45:21.070 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:20 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/2707719535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:45:21.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-16T00:45:21.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 -- 172.21.15.1:0/2462526195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b40ffba0 msgr2=0x7f65b4100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/2462526195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b40ffba0 0x7f65b4100020 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009930 tx=0x7f65a802a420 comp rx=0 tx=0).stop 2024-09-16T00:45:21.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 -- 172.21.15.1:0/2462526195 shutdown_connections 2024-09-16T00:45:21.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/2462526195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65b4100560 0x7f65b410a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/2462526195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b40ffba0 0x7f65b4100020 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/2462526195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65b4103130 0x7f65b4103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 -- 172.21.15.1:0/2462526195 >> 172.21.15.1:0/2462526195 conn(0x7f65b40fb8f0 msgr2=0x7f65b40fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:21.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 -- 172.21.15.1:0/2462526195 shutdown_connections 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 -- 172.21.15.1:0/2462526195 wait complete. 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.177+0000 7f65b9bbd640 1 Processor -- start 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b9bbd640 1 -- start start 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b9bbd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65b40ffba0 0x7f65b4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b9bbd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b4100560 0x7f65b4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b9bbd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65b4103130 0x7f65b419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b9bbd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65b410d7b0 con 0x7f65b4100560 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b9bbd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65b410d630 con 0x7f65b40ffba0 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b9bbd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65b410d930 con 0x7f65b4103130 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b4100560 0x7f65b4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b4100560 0x7f65b4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32958/0 (socket says 172.21.15.1:32958) 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b2ffd640 1 -- 172.21.15.1:0/1497755959 learned_addr learned my addr 172.21.15.1:0/1497755959 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:21.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b3fff640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65b4103130 0x7f65b419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b37fe640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65b40ffba0 0x7f65b4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b2ffd640 1 -- 172.21.15.1:0/1497755959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65b4103130 msgr2=0x7f65b419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b2ffd640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65b4103130 0x7f65b419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b2ffd640 1 -- 172.21.15.1:0/1497755959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65b40ffba0 msgr2=0x7f65b4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b2ffd640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65b40ffba0 0x7f65b4197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b2ffd640 1 -- 172.21.15.1:0/1497755959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b419fa20 con 0x7f65b4100560 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.178+0000 7f65b2ffd640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b4100560 0x7f65b4198340 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f65a8031610 tx=0x7f65a80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.179+0000 7f65b0ff9640 1 -- 172.21.15.1:0/1497755959 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a803f030 con 0x7f65b4100560 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.179+0000 7f65b0ff9640 1 -- 172.21.15.1:0/1497755959 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a8003ce0 con 0x7f65b4100560 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.179+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b419fcb0 con 0x7f65b4100560 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.179+0000 7f65b0ff9640 1 -- 172.21.15.1:0/1497755959 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8035070 con 0x7f65b4100560 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.179+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b41a01a0 con 0x7f65b4100560 2024-09-16T00:45:21.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.180+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b4109060 con 0x7f65b4100560 2024-09-16T00:45:21.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.181+0000 7f65b0ff9640 1 -- 172.21.15.1:0/1497755959 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f65a8042030 con 0x7f65b4100560 2024-09-16T00:45:21.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.181+0000 7f65b0ff9640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f659c0a3250 0x7f659c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:21.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.182+0000 7f65b0ff9640 1 -- 172.21.15.1:0/1497755959 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f65a80f07b0 con 0x7f65b4100560 2024-09-16T00:45:21.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.182+0000 7f65b37fe640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f659c0a3250 0x7f659c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:21.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.182+0000 7f65b37fe640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f659c0a3250 0x7f659c0a5710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005fd0 tx=0x7f65a4005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:21.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.185+0000 7f65b0ff9640 1 -- 172.21.15.1:0/1497755959 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65a80bd0d0 con 0x7f65b4100560 2024-09-16T00:45:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:20 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1104607545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:45:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:20 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1561212317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:45:21.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:20 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/2707719535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:45:21.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.364+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f65b402da30 con 0x7f65b4100560 2024-09-16T00:45:21.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.364+0000 7f65b0ff9640 1 -- 172.21.15.1:0/1497755959 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f65a80bc9d0 con 0x7f65b4100560 2024-09-16T00:45:21.366 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f659c0a3250 msgr2=0x7f659c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f659c0a3250 0x7f659c0a5710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005fd0 tx=0x7f65a4005ce0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b4100560 msgr2=0x7f65b4198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b4100560 0x7f65b4198340 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f65a8031610 tx=0x7f65a80043b0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 shutdown_connections 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f659c0a3250 0x7f659c0a5710 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65b4103130 0x7f65b419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65b4100560 0x7f65b4198340 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 --2- 172.21.15.1:0/1497755959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65b40ffba0 0x7f65b4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 >> 172.21.15.1:0/1497755959 conn(0x7f65b40fb8f0 msgr2=0x7f65b4106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 shutdown_connections 2024-09-16T00:45:21.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.366+0000 7f65b9bbd640 1 -- 172.21.15.1:0/1497755959 wait complete. 2024-09-16T00:45:21.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-16T00:45:21.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:20 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1104607545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:45:21.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:20 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1561212317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:45:21.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:20 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/2707719535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:45:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.481+0000 7fbb4ed11640 1 -- 172.21.15.1:0/4263376256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48107890 msgr2=0x7fbb48109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.481+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/4263376256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48107890 0x7fbb48109ca0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009ca0 tx=0x7fbb4002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.481+0000 7fbb4ed11640 1 -- 172.21.15.1:0/4263376256 shutdown_connections 2024-09-16T00:45:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.481+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/4263376256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48107890 0x7fbb48109ca0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.481+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/4263376256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48104f60 0x7fbb48107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.481+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/4263376256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48102630 0x7fbb48104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.481+0000 7fbb4ed11640 1 -- 172.21.15.1:0/4263376256 >> 172.21.15.1:0/4263376256 conn(0x7fbb480fc190 msgr2=0x7fbb480fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.481+0000 7fbb4ed11640 1 -- 172.21.15.1:0/4263376256 shutdown_connections 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.481+0000 7fbb4ed11640 1 -- 172.21.15.1:0/4263376256 wait complete. 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4ed11640 1 Processor -- start 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4ed11640 1 -- start start 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4ed11640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102630 0x7fbb48101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4ed11640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48104f60 0x7fbb481005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4ed11640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48107890 0x7fbb48100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4ed11640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4810d810 con 0x7fbb48104f60 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4ed11640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4810d690 con 0x7fbb48107890 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4ed11640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4810d990 con 0x7fbb48102630 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48104f60 0x7fbb481005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48104f60 0x7fbb481005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32984/0 (socket says 172.21.15.1:32984) 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb47fff640 1 -- 172.21.15.1:0/3436701642 learned_addr learned my addr 172.21.15.1:0/3436701642 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:21.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4d2ae640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48107890 0x7fbb48100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb4caad640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102630 0x7fbb48101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb47fff640 1 -- 172.21.15.1:0/3436701642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102630 msgr2=0x7fbb48101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb47fff640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102630 0x7fbb48101fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb47fff640 1 -- 172.21.15.1:0/3436701642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48107890 msgr2=0x7fbb48100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.482+0000 7fbb47fff640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48107890 0x7fbb48100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.483+0000 7fbb47fff640 1 -- 172.21.15.1:0/3436701642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb481013b0 con 0x7fbb48104f60 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.483+0000 7fbb47fff640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48104f60 0x7fbb481005e0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800d740 tx=0x7fbb3800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.483+0000 7fbb45ffb640 1 -- 172.21.15.1:0/3436701642 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38017070 con 0x7fbb48104f60 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.483+0000 7fbb45ffb640 1 -- 172.21.15.1:0/3436701642 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3800de50 con 0x7fbb48104f60 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.483+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb481a1fb0 con 0x7fbb48104f60 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.483+0000 7fbb45ffb640 1 -- 172.21.15.1:0/3436701642 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38004e00 con 0x7fbb48104f60 2024-09-16T00:45:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.483+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb481a23c0 con 0x7fbb48104f60 2024-09-16T00:45:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.484+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb480fdc80 con 0x7fbb48104f60 2024-09-16T00:45:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.485+0000 7fbb45ffb640 1 -- 172.21.15.1:0/3436701642 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fbb38004fa0 con 0x7fbb48104f60 2024-09-16T00:45:21.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.486+0000 7fbb45ffb640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb340a3250 0x7fbb340a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:21.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.486+0000 7fbb45ffb640 1 -- 172.21.15.1:0/3436701642 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fbb380cf730 con 0x7fbb48104f60 2024-09-16T00:45:21.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.486+0000 7fbb4caad640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb340a3250 0x7fbb340a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:21.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.487+0000 7fbb4caad640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb340a3250 0x7fbb340a5710 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb30005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:21.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.490+0000 7fbb45ffb640 1 -- 172.21.15.1:0/3436701642 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb3809c050 con 0x7fbb48104f60 2024-09-16T00:45:21.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.718+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fbb48062f30 con 0x7fbb48104f60 2024-09-16T00:45:21.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.726+0000 7fbb45ffb640 1 -- 172.21.15.1:0/3436701642 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+742333 (secure 0 0 0) 0x7fbb3809b950 con 0x7fbb48104f60 2024-09-16T00:45:21.741 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:45:21.741 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:45:21.741 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 18, 2024-09-16T00:45:21.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: "active_gid": 14150, 2024-09-16T00:45:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: "active_name": "x", 2024-09-16T00:45:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addrs": { 2024-09-16T00:45:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-16T00:45:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-16T00:45:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-16T00:45:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3146869121 2024-09-16T00:45:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-16T00:45:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-16T00:45:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3146869121 2024-09-16T00:45:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:45:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addr": "172.21.15.1:6801/3146869121", 2024-09-16T00:45:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: "active_change": "2024-09-16T00:19:24.226129+0000", 2024-09-16T00:45:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: "active_mgr_features": 4540701547738038271, 2024-09-16T00:45:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: "available": true, 2024-09-16T00:45:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: "standbys": [ 2024-09-16T00:45:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 24109, 2024-09-16T00:45:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "z", 2024-09-16T00:45:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:45:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-09-16T00:45:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-09-16T00:45:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:45:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.749 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.749 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-09-16T00:45:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:45:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-09-16T00:45:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-09-16T00:45:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:45:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-09-16T00:45:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-09-16T00:45:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-09-16T00:45:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-09-16T00:45:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-09-16T00:45:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:45:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-09-16T00:45:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:45:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:45:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-09-16T00:45:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-09-16T00:45:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-09-16T00:45:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-09-16T00:45:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:45:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:45:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:45:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:45:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:45:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:45:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-09-16T00:45:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:45:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-09-16T00:45:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-09-16T00:45:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:45:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-09-16T00:45:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-09-16T00:45:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:45:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-09-16T00:45:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-09-16T00:45:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-09-16T00:45:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-16T00:45:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-16T00:45:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-09-16T00:45:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-09-16T00:45:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-09-16T00:45:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:45:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-09-16T00:45:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-09-16T00:45:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-09-16T00:45:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:45:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-09-16T00:45:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:45:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:45:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:45:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-09-16T00:45:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-09-16T00:45:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:45:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-09-16T00:45:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-09-16T00:45:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:45:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-09-16T00:45:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-09-16T00:45:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:45:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-09-16T00:45:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-09-16T00:45:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:45:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-09-16T00:45:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:45:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:45:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-09-16T00:45:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-09-16T00:45:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-09-16T00:45:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:45:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-09-16T00:45:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-09-16T00:45:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:45:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-09-16T00:45:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-09-16T00:45:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:45:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-09-16T00:45:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-09-16T00:45:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:45:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-09-16T00:45:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:45:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:45:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-09-16T00:45:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:45:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:45:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:45:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:45:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:45:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:45:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:45:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:45:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:45:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:45:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:45:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:45:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:45:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:45:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:45:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.935 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:45:21.935 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:45:21.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.936 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:45:21.936 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.936 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.936 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.937 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:45:21.937 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:45:21.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.938 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.938 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:21.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.939 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.939 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.939 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.940 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:45:21.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:45:21.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-09-16T00:45:21.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.942 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.942 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.942 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:45:21.942 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:45:21.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-09-16T00:45:21.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.944 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.945 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:45:21.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:45:21.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.946 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.947 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.947 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.947 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.947 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:45:21.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:45:21.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.948 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.948 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.948 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-16T00:45:21.948 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.949 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.949 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.950 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:45:21.950 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:45:21.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.950 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.950 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.951 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.951 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.951 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.951 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.951 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.951 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.952 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.953 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.954 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:45:21.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:45:21.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.955 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.959 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.959 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-09-16T00:45:21.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-09-16T00:45:21.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.959 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.960 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.960 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.960 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:45:21.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.960 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.960 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-09-16T00:45:21.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-09-16T00:45:21.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.961 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.961 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-09-16T00:45:21.961 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.961 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.961 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.962 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.962 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.962 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.962 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-16T00:45:21.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-16T00:45:21.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.963 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.963 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.963 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.963 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.963 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.963 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.963 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.964 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.964 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:21.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:21.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.964 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.965 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.965 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.965 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.965 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.965 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.965 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.965 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.965 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.965 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.966 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.966 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.966 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.966 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.966 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:21.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:21.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.967 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.967 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.968 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.968 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:21.968 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:21.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.969 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.969 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:21.969 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.969 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.969 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.970 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.970 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.970 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.970 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.970 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.970 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.970 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.970 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.971 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:21.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:21.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.971 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.972 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.972 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.972 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.973 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.973 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-09-16T00:45:21.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-09-16T00:45:21.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.973 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.973 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.974 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.974 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.974 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.974 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-09-16T00:45:21.974 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.974 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.975 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:45:21.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:45:21.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.975 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.975 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.976 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.976 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.976 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.976 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.976 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.976 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.976 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-16T00:45:21.977 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-16T00:45:21.977 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.977 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.978 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-16T00:45:21.978 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.978 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.978 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.978 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.978 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.979 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.979 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-16T00:45:21.979 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-16T00:45:21.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.979 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.979 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-09-16T00:45:21.980 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.980 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.980 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.980 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.980 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.982 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:21.985 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-16T00:45:21.986 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-09-16T00:45:21.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.988 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:21.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.995 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.997 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.997 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:45:21.997 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.997 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.997 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-09-16T00:45:21.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-09-16T00:45:21.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:21.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:21.999 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.000 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.003 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.004 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-09-16T00:45:22.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-09-16T00:45:22.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.008 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-09-16T00:45:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-09-16T00:45:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-09-16T00:45:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-09-16T00:45:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-09-16T00:45:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-09-16T00:45:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-09-16T00:45:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:45:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-16T00:45:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:45:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:45:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-09-16T00:45:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-16T00:45:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:45:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-09-16T00:45:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-09-16T00:45:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-09-16T00:45:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-09-16T00:45:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-09-16T00:45:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-09-16T00:45:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-09-16T00:45:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-09-16T00:45:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-09-16T00:45:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-09-16T00:45:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-16T00:45:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-09-16T00:45:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-09-16T00:45:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-09-16T00:45:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-09-16T00:45:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-09-16T00:45:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 24113, 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "y", 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-09-16T00:45:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.275 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.276 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.277 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.279 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.280 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.281 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-09-16T00:45:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.283 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-16T00:45:22.288 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-09-16T00:45:22.290 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.292 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.293 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.294 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.298 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.299 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.300 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.301 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.303 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.306 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-09-16T00:45:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:22.309 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-09-16T00:45:22.310 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:22.311 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.312 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.313 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.314 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.315 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.316 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.317 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-09-16T00:45:22.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.319 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.323 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.324 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.325 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.327 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.328 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.329 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.330 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.335 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-09-16T00:45:22.337 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.338 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-09-16T00:45:22.339 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:45:22.340 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.341 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.342 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.347 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.349 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.350 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:45:22.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.352 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.354 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.356 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.358 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.359 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.360 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.362 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.363 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.365 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.367 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.368 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.369 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.370 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.371 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.372 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.373 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.374 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.375 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.377 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:45:22.381 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:45:22.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:45:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.384 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:45:22.387 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:45:22.388 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:45:22.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.391 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:45:22.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:45:22.393 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.394 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.397 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.400 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.401 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.402 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.403 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.404 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.405 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.406 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.407 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.408 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-09-16T00:45:22.409 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.410 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.411 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.414 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-16T00:45:22.418 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.419 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.421 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.422 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.424 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.427 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.428 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.433 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.434 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.435 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.436 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.437 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.438 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.439 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.440 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.442 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.443 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-16T00:45:22.444 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-09-16T00:45:22.445 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.446 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.447 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.449 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.450 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.451 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.452 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.456 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.460 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.462 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.463 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.464 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.466 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.469 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.472 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.473 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.474 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.475 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.476 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.478 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.480 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.481 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.483 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.484 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.485 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.486 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-09-16T00:45:22.487 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-09-16T00:45:22.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.491 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.493 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.494 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.495 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.496 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.497 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.499 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.500 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.503 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.504 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.506 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.507 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.510 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.511 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.514 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.515 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-09-16T00:45:22.516 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.517 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.518 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.519 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.520 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.523 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.524 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.526 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.527 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.528 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.529 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.530 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.531 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.532 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.533 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.534 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.536 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.537 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.538 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.540 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.541 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.543 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.544 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.545 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:45:22.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.548 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.549 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.553 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.557 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-16T00:45:22.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.559 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.562 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.563 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.564 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.565 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:45:22.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:45:22.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-09-16T00:45:22.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.570 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.573 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.575 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.576 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.577 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.580 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.582 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.583 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.586 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.587 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.590 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.591 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.592 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.597 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-09-16T00:45:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.603 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.608 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.609 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.610 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-09-16T00:45:22.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.613 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.614 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.615 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.617 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.618 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.622 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.623 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.624 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.626 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.629 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.630 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.633 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.634 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.635 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.638 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.640 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.641 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.642 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.643 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.644 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.646 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-09-16T00:45:22.647 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.648 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.650 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.651 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.652 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.654 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.656 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.657 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.658 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.660 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.662 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.663 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.665 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.668 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.669 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.674 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.675 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.678 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.679 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.681 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.682 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "modules": [ 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard", 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "iostat", 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "nfs", 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "restful" 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.691 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.692 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.693 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.694 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.695 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-09-16T00:45:22.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-09-16T00:45:22.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.701 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-09-16T00:45:22.702 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.703 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.705 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.706 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.707 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:45:22.709 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.710 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.711 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.712 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.714 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.716 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-09-16T00:45:22.717 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:22.721 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.724 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.726 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.730 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.732 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.733 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.734 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.737 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:45:22.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-09-16T00:45:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.740 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-09-16T00:45:22.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-09-16T00:45:22.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.744 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:45:22.746 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.747 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.748 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.749 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.750 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.752 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:45:22.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:45:22.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:45:22.757 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.760 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.766 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.770 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.771 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-09-16T00:45:22.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.780 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.782 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:45:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:45:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:45:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:45:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:45:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:45:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:45:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:45:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:45:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:45:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:45:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-09-16T00:45:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-16T00:45:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-09-16T00:45:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:45:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:45:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:45:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-09-16T00:45:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:45:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-09-16T00:45:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-16T00:45:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-09-16T00:45:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-09-16T00:45:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-09-16T00:45:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-09-16T00:45:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:45:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-16T00:45:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "services": { 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard": "https://172.21.15.1:8443/" 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "always_on_modules": { 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus": [ 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-16T00:45:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific": [ 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-16T00:45:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy": [ 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-16T00:45:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "reef": [ 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-16T00:45:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "squid": [ 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-16T00:45:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "last_failure_osd_epoch": 3, 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "active_clients": [ 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2593060091 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "libcephsqlite", 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4268092099 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-16T00:45:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1102762229 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1469515748 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:45:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:23.128 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-16T00:45:23.128 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:45:23.129 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:21 smithi153 ceph-mon[35538]: pgmap v859: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:45:23.129 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:21 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3942275914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:45:23.129 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:21 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1497755959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:45:23.129 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:21 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3436701642' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:45:23.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:21 smithi019 ceph-mon[35186]: pgmap v859: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:45:23.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:21 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3942275914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:45:23.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:21 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1497755959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:45:23.129 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:21 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3436701642' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:45:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb340a3250 msgr2=0x7fbb340a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb340a3250 0x7fbb340a5710 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb30005850 comp rx=0 tx=0).stop 2024-09-16T00:45:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48104f60 msgr2=0x7fbb481005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48104f60 0x7fbb481005e0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800d740 tx=0x7fbb3800dc00 comp rx=0 tx=0).stop 2024-09-16T00:45:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 shutdown_connections 2024-09-16T00:45:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48107890 0x7fbb48100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb48104f60 0x7fbb481005e0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fbb340a3250 0x7fbb340a5710 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 --2- 172.21.15.1:0/3436701642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102630 0x7fbb48101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 >> 172.21.15.1:0/3436701642 conn(0x7fbb480fc190 msgr2=0x7fbb48102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 shutdown_connections 2024-09-16T00:45:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:21.732+0000 7fbb4ed11640 1 -- 172.21.15.1:0/3436701642 wait complete. 2024-09-16T00:45:23.176 DEBUG:tasks.check_counter:Getting stats from a 2024-09-16T00:45:23.177 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-16T00:45:23.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:21 smithi001 ceph-mon[30934]: pgmap v859: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:45:23.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:21 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3942275914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:45:23.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:21 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1497755959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:45:23.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:21 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3436701642' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:45:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:23.998 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:45:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001732307, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000413049, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 6, 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 238, 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3344, 2024-09-16T00:45:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 199392, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.022271241, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.015461391, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.004199506, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 237, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.018109255, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000076410 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 199392 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 312, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 309, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 506560, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 190624, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-09-16T00:45:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.040768262, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.021418399, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.021736663, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 308, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.016179919, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000052532 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 506560, 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 190624 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-16T00:45:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 149752, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-09-16T00:45:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-16T00:45:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 314, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 462452, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 314, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 314, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 462452, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:24.010 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:45:24.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:23 smithi001 ceph-mon[30934]: pgmap v860: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:45:24.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:23 smithi019 ceph-mon[35186]: pgmap v860: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:45:24.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:23 smithi153 ceph-mon[35538]: pgmap v860: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:45:24.780 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-16T00:45:24.780 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-16T00:45:24.780 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-16T00:45:24.780 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-16T00:45:24.781 DEBUG:tasks.check_counter:Getting stats from d 2024-09-16T00:45:24.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-16T00:45:25.045 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:25.579 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:45:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-09-16T00:45:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-09-16T00:45:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3024, 2024-09-16T00:45:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-09-16T00:45:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.003458633, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000049551, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.002326048, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.004115252, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.001371750 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 240, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 199648, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.020287985, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.015851334, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001735040, 2024-09-16T00:45:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 238, 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.013835418, 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000058132 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 199648 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 318, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 315, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 508192, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 194464, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.043672264, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.023613171, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.022211213, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 314, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.017267048, 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000054990 2024-09-16T00:45:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 508192, 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 194464 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-16T00:45:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148761, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-16T00:45:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-16T00:45:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 320, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 463232, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 320, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 320, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 463232, 2024-09-16T00:45:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-16T00:45:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:25.593 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:45:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:25 smithi001 ceph-mon[30934]: pgmap v861: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:45:26.273 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-16T00:45:26.273 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-16T00:45:26.273 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-16T00:45:26.273 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-16T00:45:26.273 DEBUG:tasks.check_counter:Getting stats from g 2024-09-16T00:45:26.273 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-16T00:45:26.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:25 smithi019 ceph-mon[35186]: pgmap v861: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:45:26.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:25 smithi153 ceph-mon[35538]: pgmap v861: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:45:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:27.057 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001364436, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000166168, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 237, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 199120, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.019053390, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.014888418, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001511937, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 235, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.013063380, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000055588 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 199120 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 318, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 313, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 510128, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 192368, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-09-16T00:45:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.045928809, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.023898624, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.024260408, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 313, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.021242770, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000067868 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 510128, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 192368 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-16T00:45:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148704, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 29, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-09-16T00:45:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-16T00:45:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 316, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 462712, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 316, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 316, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 462712, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:27.069 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:45:27.787 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-16T00:45:27.787 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-16T00:45:27.787 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-16T00:45:27.787 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-16T00:45:27.787 DEBUG:tasks.check_counter:Getting stats from j 2024-09-16T00:45:27.788 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-16T00:45:28.047 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:28.168 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:28 smithi001 ceph-mon[30934]: pgmap v862: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:45:28.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:28 smithi019 ceph-mon[35186]: pgmap v862: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:45:28.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:28 smithi153 ceph-mon[35538]: pgmap v862: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:45:28.607 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-16T00:45:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-09-16T00:45:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-09-16T00:45:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3024, 2024-09-16T00:45:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-09-16T00:45:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-09-16T00:45:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.003321505, 2024-09-16T00:45:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000040296, 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.002314409, 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.004102273, 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.001367424 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 242, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 200000, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.021094321, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.016378728, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001735967, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 240, 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.015114919, 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000062978 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 200000 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 319, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 316, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 508464, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 195120, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.043980337, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.023096564, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.023101099, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 315, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.018045556, 2024-09-16T00:45:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000057287 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 508464, 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 195120 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148761, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-09-16T00:45:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 321, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 463362, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 321, 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 321, 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 463362, 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-16T00:45:28.620 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-16T00:45:29.311 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-16T00:45:29.311 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-16T00:45:29.311 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-16T00:45:29.311 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-16T00:45:29.311 DEBUG:tasks.check_counter:Getting stats from b 2024-09-16T00:45:29.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-16T00:45:29.578 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:45:30.037 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-16T00:45:30.037 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:30.037 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4601, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4794, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1045254, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 12220282, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 294, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.485464995, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.151857175, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.220582086, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.080533784, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4793, 2024-09-16T00:45:30.038 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.142749790, 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000029782 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2624, 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000566708, 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000215 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2832, 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4469, 2024-09-16T00:45:30.039 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1012025, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 2004159, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 12, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.362282674, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.129875810, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.159659067, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.073569243, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4467, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.154112559, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000034500 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.040 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 5250, 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.001224430, 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000233 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 52288, 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 535184 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1955, 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2136, 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1034122, 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 738163, 2024-09-16T00:45:30.041 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 12, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 6, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.206300837, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.097398140, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.120789148, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.009156583, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2135, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.282074511, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000132119 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2441, 2024-09-16T00:45:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000646731, 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000264 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 562960, 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 231968 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 5212, 2024-09-16T00:45:30.043 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.448603340, 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000277936 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 24, 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000755837, 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000031493 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-09-16T00:45:30.044 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 676, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 676, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 676, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.921245486, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.004321369 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 12, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-09-16T00:45:30.045 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 24, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 24, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 23, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 2, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 2, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 768, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 758, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:45:30.046 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 415, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 4, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 234, 2024-09-16T00:45:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 415, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 267, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 30, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 237, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 148, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 646, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 670, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 599, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:45:30.048 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 749, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 708, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 3, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 628 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 11, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 23, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:45:30.049 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 3, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 3, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 15, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 208, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 30, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:45:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 2173, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 2173, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 4, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 4, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-09-16T00:45:30.051 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 11570141, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2157, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 4.184721322, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001940065 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 23, 2024-09-16T00:45:30.052 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 56, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 33, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 18, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 20, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 2, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 24, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 29, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 5, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 2, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 18, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 16, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 58180, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 231712 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 1898, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 676, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 24, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 19, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.012798203, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000673589 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 28, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.005928872, 2024-09-16T00:45:30.054 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000211745 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-09-16T00:45:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.001247711, 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000415903 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000185007, 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000185007 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-09-16T00:45:30.057 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.058 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.058 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.058 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-09-16T00:45:30.058 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8, 2024-09-16T00:45:30.058 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.012621562, 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001577695 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000224832, 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000224832 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.004004497, 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002002248 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.002301338, 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001150669 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.060 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.011336937, 2024-09-16T00:45:30.061 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.005668468 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 601, 2024-09-16T00:45:30.062 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.847724081, 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.004738309 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-09-16T00:45:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8, 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.020717345, 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002589668 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.064 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.002155101, 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002155101 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:30.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 3, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 2, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 61, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 61, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 1007, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-16T00:45:30.066 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:30.067 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 302439, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 100, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:30.069 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 9160, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 174, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 95882, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 667, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:30.070 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 3866, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 7399107, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 35, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 3866, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3866, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 22.500749358, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.005820162 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.071 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3866, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 3908 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 3866, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 477, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 3389, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 11, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 32, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 2159, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 70, 2024-09-16T00:45:30.072 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 517, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 445, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 22, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-09-16T00:45:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 599, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-09-16T00:45:30.074 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 91, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 9, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 12, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 12, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-09-16T00:45:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 45, 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 8 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 223 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.076 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 3, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 2, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 4, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 3, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 9384, 2024-09-16T00:45:30.077 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 2217085, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 9384, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 9384, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2217085, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.078 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3866, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 7399418, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3866, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 7399418, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.079 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3866, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 3866, 2024-09-16T00:45:30.080 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3866, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 3866, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 311, 2024-09-16T00:45:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 311, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x562ba4964500": { 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 2173, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 7375837, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 2173, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2157, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 7375837, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:30.084 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-16T00:45:30.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:30 smithi019 ceph-mon[35186]: pgmap v863: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-16T00:45:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:30 smithi001 ceph-mon[30934]: pgmap v863: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-16T00:45:30.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:30 smithi153 ceph-mon[35538]: pgmap v863: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-16T00:45:31.099 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-16T00:45:31.100 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-16T00:45:31.100 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=11 2024-09-16T00:45:31.100 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-16T00:45:31.100 DEBUG:tasks.check_counter:Getting stats from e 2024-09-16T00:45:31.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-16T00:45:31.351 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:45:31.866 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-16T00:45:31.866 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:31.866 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1112, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1114, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 292555, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 353680, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.107775865, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.044185183, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.025166196, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.026514883, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1112, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.047542950, 2024-09-16T00:45:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000042754 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 334, 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 327, 2024-09-16T00:45:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 535842, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 197393, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 3, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.050196110, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.023335087, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.032219943, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000052542, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 327, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.024577670, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000075161 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.869 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 535328, 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 196752 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1110, 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1349, 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 372608, 2024-09-16T00:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 929480, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.152992866, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.052792894, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.038443141, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.045585055, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1347, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.077528355, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000057556 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 577136 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-16T00:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4437, 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.139653689, 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000031474 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-09-16T00:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-09-16T00:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:45:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-09-16T00:45:31.878 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-09-16T00:45:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-09-16T00:45:31.880 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 2, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4194892, 2024-09-16T00:45:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4194352, 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 2 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 3, 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 3, 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 1, 2024-09-16T00:45:31.882 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 0, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 0, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 48348, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215328 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-09-16T00:45:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-09-16T00:45:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.887 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-09-16T00:45:31.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.889 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.889 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-09-16T00:45:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-09-16T00:45:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-09-16T00:45:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-09-16T00:45:31.892 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.893 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.896 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-09-16T00:45:31.896 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.896 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.896 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.896 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-09-16T00:45:31.896 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.896 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.896 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 0, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 0, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 0, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 0, 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-09-16T00:45:31.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.899 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-16T00:45:31.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:31.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:31.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:31.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:31.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:31.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:31.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 182087, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 48, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:31.902 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 9160, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 174, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 8512, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13, 2024-09-16T00:45:31.903 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 2222, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 2222, 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-09-16T00:45:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2222, 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.156986860, 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000520696 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2222, 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2223 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 2222, 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 2222, 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-09-16T00:45:31.905 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 1108, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 1112, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-09-16T00:45:31.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-09-16T00:45:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-09-16T00:45:31.908 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 91, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 2, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 4, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 5, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 1, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:45:31.909 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-09-16T00:45:31.910 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 2552, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 951360, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 2552, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2552, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 951360, 2024-09-16T00:45:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2222, 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 593, 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2222, 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 593, 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.913 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2222, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2222, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2222, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2222, 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:31.914 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-16T00:45:31.915 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x559e16b286e0": { 2024-09-16T00:45:31.916 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:31.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-16T00:45:31.917 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:31.917 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-16T00:45:31.917 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-16T00:45:31.917 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:31.917 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:31.917 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:31.917 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:31.917 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-16T00:45:31.918 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-16T00:45:31.918 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:31.918 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:31.918 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:31.918 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:31.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:31.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:31.918 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-16T00:45:32.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:32 smithi019 ceph-mon[35186]: pgmap v864: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:45:32.107 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:32 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:45:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:32 smithi001 ceph-mon[30934]: pgmap v864: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:45:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:32 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:45:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:32 smithi153 ceph-mon[35538]: pgmap v864: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:45:32.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:32 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:45:32.625 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-16T00:45:32.625 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-16T00:45:32.626 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-16T00:45:32.626 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-16T00:45:32.626 DEBUG:tasks.check_counter:Getting stats from h 2024-09-16T00:45:32.626 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-16T00:45:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:45:33.260 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.260 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: Adjusting osd_memory_target on smithi019 to 177.0M 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: Unable to set osd_memory_target on smithi019 to 185700659: error parsing value: Value '185700659' is below minimum 939524096 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:45:33.261 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:33 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:45:33.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: Adjusting osd_memory_target on smithi019 to 177.0M 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: Unable to set osd_memory_target on smithi019 to 185700659: error parsing value: Value '185700659' is below minimum 939524096 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:45:33.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:45:33.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:33 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:45:33.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: Adjusting osd_memory_target on smithi019 to 177.0M 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: Unable to set osd_memory_target on smithi019 to 185700659: error parsing value: Value '185700659' is below minimum 939524096 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:45:33.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:33 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' 2024-09-16T00:45:33.504 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-16T00:45:33.504 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 462, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 456, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 560846, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 992903, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 75, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.091248678, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.029765321, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.060962341, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000424261, 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:33.505 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 453, 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.031730073, 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000070044 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 107, 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000013642, 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000127 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 545008, 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 201904 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1344, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1383, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 324245, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 863847, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 9, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.146097840, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.073364022, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.073409637, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.005811738, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1383, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.068085444, 2024-09-16T00:45:33.507 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000049230 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2410, 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000523311, 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000217 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 490240 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 97, 2024-09-16T00:45:33.508 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 97, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 33334, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 63517, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 10, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.012749249, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.003139054, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.009553333, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000572975, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 96, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.024575524, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000255995 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 146, 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000029072, 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000199 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.510 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 49, 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.001585274, 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000032352 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000013470, 2024-09-16T00:45:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000006735 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 4, 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 2, 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000337619, 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000168809 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 2, 2024-09-16T00:45:33.512 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 11, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 16, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 16, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 13, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 4, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 1, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 1, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 3, 2024-09-16T00:45:33.513 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 46, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 37, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 1, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 7, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-09-16T00:45:33.514 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 5, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 298, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 1, 2024-09-16T00:45:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 1, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 1 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 16, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 11, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 12, 2024-09-16T00:45:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 15, 2024-09-16T00:45:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 89, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 5, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 5, 2024-09-16T00:45:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 2, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4195734, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-09-16T00:45:33.519 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.007545629, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001886407 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 20, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 22, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 2, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 19, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 22, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 3, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 16, 2024-09-16T00:45:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 20, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 4, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 1, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 2, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 1, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 50084, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 223520 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 5, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 4, 2024-09-16T00:45:33.521 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 18, 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000192400, 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000192400 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.522 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000145219, 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000145219 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-09-16T00:45:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-09-16T00:45:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-09-16T00:45:33.530 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.531 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.533 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 2, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 1, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 1, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 1, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 1010, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:33.534 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:33.535 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 246887, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 81, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:33.537 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 9160, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 174, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 81357, 2024-09-16T00:45:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 332, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 40, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 11732, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 33, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 40, 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-09-16T00:45:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 40, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.245997308, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.006149932 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 40, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 62 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 40, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 0, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 40, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-09-16T00:45:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 10, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 4, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 7, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 6, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-09-16T00:45:33.541 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 20, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-09-16T00:45:33.542 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-09-16T00:45:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 91, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 2, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 9, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 13, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 4, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 14, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 1 2024-09-16T00:45:33.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 104 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:45:33.545 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 1899, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 725864, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 1899, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:33.546 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 1899, 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 725864, 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 40, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 11732, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 40, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 11732, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.548 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:33.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.549 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-09-16T00:45:33.549 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:33.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-09-16T00:45:33.549 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:33.549 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-16T00:45:33.549 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-16T00:45:33.550 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:33.550 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:33.550 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 40, 2024-09-16T00:45:33.550 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 40, 2024-09-16T00:45:33.550 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 40, 2024-09-16T00:45:33.550 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 40, 2024-09-16T00:45:33.551 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:33.551 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.551 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.551 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.551 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:33.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.551 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:45:33.551 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:33.552 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x5623b6cc2500": { 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 5, 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1430, 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 5, 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 4, 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1430, 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:33.554 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:33.555 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-16T00:45:34.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:34 smithi001 ceph-mon[30934]: pgmap v865: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:45:34.255 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-16T00:45:34.255 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-16T00:45:34.255 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-16T00:45:34.255 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=16 2024-09-16T00:45:34.255 DEBUG:tasks.check_counter:Getting stats from k 2024-09-16T00:45:34.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-16T00:45:34.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:34 smithi019 ceph-mon[35186]: pgmap v865: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:45:34.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:34 smithi153 ceph-mon[35538]: pgmap v865: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:45:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001245965, 2024-09-16T00:45:34.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000249490, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:45:34.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:45:34.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:34.995 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:34.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2, 2024-09-16T00:45:34.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 244, 2024-09-16T00:45:34.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 320, 2024-09-16T00:45:34.996 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 200352, 2024-09-16T00:45:34.996 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:34.996 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:34.996 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.016067480, 2024-09-16T00:45:34.996 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.011774820, 2024-09-16T00:45:34.996 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.001333818, 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 242, 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.014586612, 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000060275 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 200352 2024-09-16T00:45:34.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 326, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 321, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 512304, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 197488, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.048701558, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.022602629, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.030139882, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 321, 2024-09-16T00:45:34.998 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.023349827, 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000072740 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 512304, 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 197488 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-16T00:45:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:35.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148704, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 29, 2024-09-16T00:45:35.002 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-16T00:45:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 324, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 463752, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:35.004 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 324, 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 324, 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 463752, 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-16T00:45:35.005 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-16T00:45:35.673 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-16T00:45:35.673 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-16T00:45:35.673 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-16T00:45:35.673 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-16T00:45:35.673 DEBUG:tasks.check_counter:Getting stats from c 2024-09-16T00:45:35.674 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-16T00:45:35.932 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:45:36.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:36 smithi019 ceph-mon[35186]: pgmap v866: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:45:36.368 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:36 smithi153 ceph-mon[35538]: pgmap v866: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:45:36.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:36 smithi001 ceph-mon[30934]: pgmap v866: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:45:36.484 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-16T00:45:36.484 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:36.484 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 2521, 2024-09-16T00:45:36.484 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 2514, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 5379976, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 892378, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 1, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.268700364, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.092888508, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.092146225, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.067755144, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2514, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.111395550, 2024-09-16T00:45:36.485 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000044310 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 535216, 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 196272 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 412, 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 651, 2024-09-16T00:45:36.486 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 101852, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 709198, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 7, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.056518005, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.025698819, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.011163148, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.013419612, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 649, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.035209507, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000054251 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:36.487 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 578752 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1394, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1396, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 3531283, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 443299, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:36.488 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 3, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.130461870, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.049449848, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.031420060, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.037462982, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1394, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.052995519, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000038016 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.489 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 6745, 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.176700099, 2024-09-16T00:45:36.490 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000026197 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-16T00:45:36.491 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 17, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 0, 2024-09-16T00:45:36.492 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 17, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 17, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 2, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-16T00:45:36.493 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 4, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 234, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-16T00:45:36.494 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:45:36.495 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-09-16T00:45:36.496 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-16T00:45:36.497 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 2173, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 4, 2024-09-16T00:45:36.498 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 2, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 11570141, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 11565840, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 2173 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 14, 2024-09-16T00:45:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 17, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 3, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 9, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 11, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 2, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 17, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 19, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 2, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 49556, 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 215328 2024-09-16T00:45:36.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.501 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-16T00:45:36.502 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.504 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-16T00:45:36.505 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.507 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-16T00:45:36.508 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.510 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.511 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.512 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 3, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 2, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 2270, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:36.513 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:36.514 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:36.515 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 190935, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 53, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:36.516 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 9160, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 174, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 54978, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 261, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:36.517 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 3993, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 26, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 4, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 3993, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3993, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.820967383, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000456039 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.518 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3993, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 3994 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 3993, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 3993, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 2224, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 1766, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2024-09-16T00:45:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 2, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:45:36.520 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-16T00:45:36.521 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 91, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 2, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 5, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 6, 2024-09-16T00:45:36.522 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 1, 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-16T00:45:36.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:36.524 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 4323, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 8604082, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 4323, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4323, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 8604082, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:36.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 3993, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 7375847, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3993, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 7375847, 2024-09-16T00:45:36.526 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 3993, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 3993, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3993, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 3993, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:36.528 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:36.529 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x55f4dd74a6e0": { 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:36.530 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:36.531 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:36.531 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:36.531 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:36.531 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-16T00:45:37.469 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-16T00:45:37.469 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-16T00:45:37.469 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-16T00:45:37.469 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-16T00:45:37.469 DEBUG:tasks.check_counter:Getting stats from f 2024-09-16T00:45:37.470 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-16T00:45:37.717 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:45:38.138 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:38 smithi153 ceph-mon[35538]: pgmap v867: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:45:38.263 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 3270, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 3270, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 926597, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 1039043, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 6, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.293763649, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.098186643, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.075763615, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.097163305, 2024-09-16T00:45:38.264 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3269, 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.134956542, 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000041283 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.265 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 330, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 325, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 532860, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 197002, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 3, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.047302925, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.022658285, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.029459684, 2024-09-16T00:45:38.266 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000078108, 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 324, 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.021214432, 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000065476 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 532304, 2024-09-16T00:45:38.267 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 196304 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 3, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 243, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 588, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 579246, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 6, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 3, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.015468302, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.010569051, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.002144077, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000026791, 2024-09-16T00:45:38.268 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 241, 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.016658205, 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000069121 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 578928 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.269 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 5499, 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.151773759, 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000027600 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-16T00:45:38.270 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.271 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-16T00:45:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 1, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-16T00:45:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-16T00:45:38.274 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-09-16T00:45:38.275 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-09-16T00:45:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:45:38.277 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 5, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 2, 2024-09-16T00:45:38.278 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 2, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 4195734, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 4195426, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 5 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-16T00:45:38.279 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 3, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 3, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 0, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 1, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 1, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 0, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 0, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 0, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 0, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 47924, 2024-09-16T00:45:38.280 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 215328 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.281 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-16T00:45:38.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-16T00:45:38.283 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.285 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.286 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-16T00:45:38.287 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-16T00:45:38.288 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.290 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.291 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 2, 2024-09-16T00:45:38.292 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 1, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 2272, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:38.293 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:38.294 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:38.295 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 182104, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 49, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-16T00:45:38.296 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 9160, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 174, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 8512, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 13, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-16T00:45:38.297 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 3269, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 21, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 3, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 3269, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3269, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.524972690, 2024-09-16T00:45:38.298 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000466495 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3269, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 3270 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 3269, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 3269, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 2146, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 1121, 2024-09-16T00:45:38.299 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 1, 2024-09-16T00:45:38.300 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-16T00:45:38.301 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-16T00:45:38.302 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 91, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 2, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 4, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 5, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 1, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-16T00:45:38.303 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-09-16T00:45:38.304 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 3599, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1116170, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 3599, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:38.305 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3599, 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1116170, 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:38.306 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 3269, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 1435, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3269, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1435, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-16T00:45:38.307 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 3269, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 3269, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3269, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 3269, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.308 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:38.309 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x563723c246e0": { 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:38.310 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:38.311 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-16T00:45:38.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:38 smithi019 ceph-mon[35186]: pgmap v867: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:45:38.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:38 smithi001 ceph-mon[30934]: pgmap v867: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T00:45:38.964 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-16T00:45:38.964 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-16T00:45:38.964 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-16T00:45:38.964 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-16T00:45:38.964 DEBUG:tasks.check_counter:Getting stats from i 2024-09-16T00:45:38.964 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-16T00:45:39.220 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:45:39.772 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1345, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1153, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 329374, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 198481, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 10, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.120271204, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.044056019, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.072316533, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.007008927, 2024-09-16T00:45:39.773 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1154, 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.063509951, 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000055034 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2416, 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000623793, 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000258 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:45:39.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 458, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 450, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 570582, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 360325, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 16, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 3, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.059891557, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.026446703, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.038867830, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000550082, 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:39.775 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 451, 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.037912840, 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000084063 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 159, 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000033674, 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000211 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 539440, 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 200464 2024-09-16T00:45:39.776 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 63, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 306, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 9801, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 1266203, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 67, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.035523641, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.011404733, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.019552647, 2024-09-16T00:45:39.777 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000151253, 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 300, 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.022059925, 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000073533 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 56, 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000004148, 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000074 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:45:39.778 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 580368 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 35, 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.001214374, 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000034696 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:39.779 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.780 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-16T00:45:39.780 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-16T00:45:39.780 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-16T00:45:39.780 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2, 2024-09-16T00:45:39.780 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000010410, 2024-09-16T00:45:39.780 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000005205 2024-09-16T00:45:39.780 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:39.780 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.780 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 11, 2024-09-16T00:45:39.781 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 16, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 0, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 16, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 13, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 4, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 3, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 44, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 35, 2024-09-16T00:45:39.782 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 7, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 1, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 1000, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 1, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:45:39.783 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 5, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 298, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:45:39.784 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 19, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 14, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 11, 2024-09-16T00:45:39.785 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 15, 2024-09-16T00:45:39.786 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 89, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 2, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-16T00:45:39.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 2, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 2, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 2, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 2, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 4194892, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2024-09-16T00:45:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1, 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.001565123, 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001565123 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 20, 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 22, 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 2, 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 19, 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 22, 2024-09-16T00:45:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 3, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 16, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 20, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 4, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 50972, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 223520 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-09-16T00:45:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.791 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.792 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.793 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.794 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.795 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-16T00:45:39.796 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.797 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.798 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-16T00:45:39.799 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.800 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-16T00:45:39.801 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-16T00:45:39.802 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:39.805 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 222423, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 66, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-16T00:45:39.806 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 9160, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 174, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 80941, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 324, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 35, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 10710, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 33, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 35, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 35, 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.206996387, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.005914182 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 35, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 57 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 35, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 0, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 35, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 0, 2024-09-16T00:45:39.809 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 10, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 0, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 1, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 5, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 6, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 0, 2024-09-16T00:45:39.810 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 20, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-16T00:45:39.811 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-16T00:45:39.812 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 91, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 2, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 8, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 13, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 5, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 14, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 0, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 1 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:45:39.813 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 104 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.814 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 1862, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 720832, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 1862, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 1862, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 720832, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.815 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:39.816 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 35, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 10710, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 35, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 10710, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:39.817 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 35, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 35, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 35, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 35, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:39.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:39.819 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x55e82a9da500": { 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 2, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 588, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 2, 2024-09-16T00:45:39.820 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 1, 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 588, 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:39.821 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-16T00:45:40.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:40 smithi019 ceph-mon[35186]: pgmap v868: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-16T00:45:40.380 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:40 smithi153 ceph-mon[35538]: pgmap v868: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-16T00:45:40.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:40 smithi001 ceph-mon[30934]: pgmap v868: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-16T00:45:40.615 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-16T00:45:40.615 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-16T00:45:40.615 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-16T00:45:40.615 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=19 2024-09-16T00:45:40.615 DEBUG:tasks.check_counter:Getting stats from l 2024-09-16T00:45:40.616 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-16T00:45:40.866 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:45:41.343 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-16T00:45:41.343 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:45:41.343 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 324, 2024-09-16T00:45:41.343 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 321, 2024-09-16T00:45:41.343 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 509824, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 198320, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 0, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.049076870, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.023757652, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.029629612, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 320, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.021584532, 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000067451 2024-09-16T00:45:41.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 509824, 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 198320 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 2, 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 246, 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 320, 2024-09-16T00:45:41.345 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 200704, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 4, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 2, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.015545598, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.011522246, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.000968351, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 244, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.018488857, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000075774 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:41.346 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 200704 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 4, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 4, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 3024, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 448, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 2, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 1, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.002393957, 2024-09-16T00:45:41.347 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.000033296, 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.002330562, 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3, 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.004124884, 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001374961 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.348 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 0, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:45:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:45:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:45:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 148721, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 30, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 0, 2024-09-16T00:45:41.352 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 0, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 928, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 2, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-16T00:45:41.353 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 326, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 464012, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 326, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 326, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 464012, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-16T00:45:41.354 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-16T00:45:42.132 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-16T00:45:42.132 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-16T00:45:42.132 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-16T00:45:42.132 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-16T00:45:42.133 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T00:45:42.151 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T00:45:42.151 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:45:42.151 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:45:42.174 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-16T00:45:42.174 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:45:42.235 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:45:42.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:42 smithi001 ceph-mon[30934]: pgmap v869: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:45:42.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:42 smithi019 ceph-mon[35186]: pgmap v869: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:45:42.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:42 smithi153 ceph-mon[35538]: pgmap v869: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:45:42.546 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:43.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.104+0000 7fd6709e9640 1 -- 172.21.15.1:0/408222604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd66c102790 msgr2=0x7fd66c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:43.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.104+0000 7fd6709e9640 1 --2- 172.21.15.1:0/408222604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd66c102790 0x7fd66c1067e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd660009860 tx=0x7fd66002a340 comp rx=0 tx=0).stop 2024-09-16T00:45:43.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.104+0000 7fd6709e9640 1 -- 172.21.15.1:0/408222604 shutdown_connections 2024-09-16T00:45:43.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.104+0000 7fd6709e9640 1 --2- 172.21.15.1:0/408222604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd66c106f10 0x7fd66c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.104+0000 7fd6709e9640 1 --2- 172.21.15.1:0/408222604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd66c102790 0x7fd66c1067e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.104+0000 7fd6709e9640 1 --2- 172.21.15.1:0/408222604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd66c101de0 0x7fd66c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.105+0000 7fd6709e9640 1 -- 172.21.15.1:0/408222604 >> 172.21.15.1:0/408222604 conn(0x7fd66c0fd8c0 msgr2=0x7fd66c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.105+0000 7fd6709e9640 1 -- 172.21.15.1:0/408222604 shutdown_connections 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.105+0000 7fd6709e9640 1 -- 172.21.15.1:0/408222604 wait complete. 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.105+0000 7fd6709e9640 1 Processor -- start 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.105+0000 7fd6709e9640 1 -- start start 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd6709e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd66c101de0 0x7fd66c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd6709e9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd66c102790 0x7fd66c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd6709e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd66c106f10 0x7fd66c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd6709e9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c111f70 con 0x7fd66c101de0 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd6709e9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c111df0 con 0x7fd66c106f10 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd6709e9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c1120f0 con 0x7fd66c102790 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd66c101de0 0x7fd66c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd66c101de0 0x7fd66c1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58452/0 (socket says 172.21.15.1:58452) 2024-09-16T00:45:43.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66a59c640 1 -- 172.21.15.1:0/4055814127 learned_addr learned my addr 172.21.15.1:0/4055814127 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66ad9d640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd66c106f10 0x7fd66c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd669d9b640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd66c102790 0x7fd66c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66ad9d640 1 -- 172.21.15.1:0/4055814127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd66c102790 msgr2=0x7fd66c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66ad9d640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd66c102790 0x7fd66c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66ad9d640 1 -- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd66c101de0 msgr2=0x7fd66c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66ad9d640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd66c101de0 0x7fd66c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66ad9d640 1 -- 172.21.15.1:0/4055814127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c19d470 con 0x7fd66c106f10 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66a59c640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd66c101de0 0x7fd66c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd66ad9d640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd66c106f10 0x7fd66c19cd90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd66400efc0 tx=0x7fd66400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.106+0000 7fd65b7fe640 1 -- 172.21.15.1:0/4055814127 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664003970 con 0x7fd66c106f10 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.107+0000 7fd65b7fe640 1 -- 172.21.15.1:0/4055814127 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd664004020 con 0x7fd66c106f10 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.107+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c19d760 con 0x7fd66c106f10 2024-09-16T00:45:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.107+0000 7fd65b7fe640 1 -- 172.21.15.1:0/4055814127 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66400b9f0 con 0x7fd66c106f10 2024-09-16T00:45:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.107+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c1a5040 con 0x7fd66c106f10 2024-09-16T00:45:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.107+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66c1021c0 con 0x7fd66c106f10 2024-09-16T00:45:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.109+0000 7fd65b7fe640 1 -- 172.21.15.1:0/4055814127 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd66401d020 con 0x7fd66c106f10 2024-09-16T00:45:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.109+0000 7fd65b7fe640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6540a3240 0x7fd6540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.109+0000 7fd66a59c640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6540a3240 0x7fd6540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.109+0000 7fd65b7fe640 1 -- 172.21.15.1:0/4055814127 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fd6640cfbd0 con 0x7fd66c106f10 2024-09-16T00:45:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.110+0000 7fd66a59c640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6540a3240 0x7fd6540a5700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd65c005fd0 tx=0x7fd65c00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.114+0000 7fd65b7fe640 1 -- 172.21.15.1:0/4055814127 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd66409c4f0 con 0x7fd66c106f10 2024-09-16T00:45:43.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.304+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd66c10a150 con 0x7fd66c106f10 2024-09-16T00:45:43.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.305+0000 7fd65b7fe640 1 -- 172.21.15.1:0/4055814127 <== mon.2 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+576 (secure 0 0 0) 0x7fd66409bdf0 con 0x7fd66c106f10 2024-09-16T00:45:43.305 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/178572347 2024-09-17T00:19:24.226024+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3932831772 2024-09-17T00:19:24.226024+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3251079055 2024-09-17T00:19:24.226024+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3191294475 2024-09-17T00:19:24.226024+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/178572347 2024-09-17T00:19:24.226024+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1613718962 2024-09-17T00:19:00.598574+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/786826717 2024-09-17T00:19:00.598574+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/786826717 2024-09-17T00:19:00.598574+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/639100255 2024-09-17T00:19:00.598574+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/365027668 2024-09-17T00:19:00.598574+0000 2024-09-16T00:45:43.306 INFO:teuthology.orchestra.run.smithi001.stderr:listed 10 entries 2024-09-16T00:45:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6540a3240 msgr2=0x7fd6540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6540a3240 0x7fd6540a5700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd65c005fd0 tx=0x7fd65c00b480 comp rx=0 tx=0).stop 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd66c106f10 msgr2=0x7fd66c19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd66c106f10 0x7fd66c19cd90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd66400efc0 tx=0x7fd66400c800 comp rx=0 tx=0).stop 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 shutdown_connections 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd66c106f10 0x7fd66c19cd90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fd6540a3240 0x7fd6540a5700 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd66c102790 0x7fd66c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 --2- 172.21.15.1:0/4055814127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd66c101de0 0x7fd66c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.307+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 >> 172.21.15.1:0/4055814127 conn(0x7fd66c0fd8c0 msgr2=0x7fd66c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.308+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 shutdown_connections 2024-09-16T00:45:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:43.308+0000 7fd6709e9640 1 -- 172.21.15.1:0/4055814127 wait complete. 2024-09-16T00:45:44.135 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T00:45:44.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:45:44.135 DEBUG:teuthology.orchestra.run.smithi001:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T00:45:44.225 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-09-16T00:45:44.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:45:44.226 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T00:45:44.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:45:44.254 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-16T00:45:44.311 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T00:45:44.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:45:44.311 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:45:44.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:44 smithi019 ceph-mon[35186]: pgmap v870: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:45:44.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:44 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/4055814127' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:45:44.373 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:44 smithi001 ceph-mon[30934]: pgmap v870: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:45:44.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:44 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/4055814127' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:45:44.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:45:44.378 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-09-16T00:45:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:44 smithi153 ceph-mon[35538]: pgmap v870: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:45:44.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:44 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/4055814127' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:45:44.494 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T00:45:44.503 INFO:tasks.cephadm:Teardown begin 2024-09-16T00:45:44.503 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T00:45:44.504 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:45:44.762 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 -- 172.21.15.1:0/3324229918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0101de0 msgr2=0x7f8cf01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 --2- 172.21.15.1:0/3324229918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0101de0 0x7f8cf01021c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009900 tx=0x7f8ce002a410 comp rx=0 tx=0).stop 2024-09-16T00:45:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 -- 172.21.15.1:0/3324229918 shutdown_connections 2024-09-16T00:45:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 --2- 172.21.15.1:0/3324229918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf0106f10 0x7f8cf010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 --2- 172.21.15.1:0/3324229918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cf0102790 0x7f8cf01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 --2- 172.21.15.1:0/3324229918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0101de0 0x7f8cf01021c0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 -- 172.21.15.1:0/3324229918 >> 172.21.15.1:0/3324229918 conn(0x7f8cf00fd8c0 msgr2=0x7f8cf00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 -- 172.21.15.1:0/3324229918 shutdown_connections 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 -- 172.21.15.1:0/3324229918 wait complete. 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.265+0000 7f8cf6017640 1 Processor -- start 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8cf6017640 1 -- start start 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8cf6017640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cf0101de0 0x7f8cf0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8cf6017640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0102790 0x7f8cf01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8cf6017640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf0106f10 0x7f8cf019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8cf6017640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0111f90 con 0x7f8cf0102790 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8cf6017640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0111e10 con 0x7f8cf0106f10 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8cf6017640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0112110 con 0x7f8cf0101de0 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8ceeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0102790 0x7f8cf01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:45.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8ceeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0102790 0x7f8cf01989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58472/0 (socket says 172.21.15.1:58472) 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8ceeffd640 1 -- 172.21.15.1:0/1995227790 learned_addr learned my addr 172.21.15.1:0/1995227790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8ceffff640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf0106f10 0x7f8cf019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.266+0000 7f8cef7fe640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cf0101de0 0x7f8cf0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8ceeffd640 1 -- 172.21.15.1:0/1995227790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cf0101de0 msgr2=0x7f8cf0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8ceeffd640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cf0101de0 0x7f8cf0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8ceeffd640 1 -- 172.21.15.1:0/1995227790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf0106f10 msgr2=0x7f8cf019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8ceeffd640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf0106f10 0x7f8cf019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8ceeffd640 1 -- 172.21.15.1:0/1995227790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf019d440 con 0x7f8cf0102790 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8ceeffd640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0102790 0x7f8cf01989d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400b4a0 tx=0x7f8ce400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8cecff9640 1 -- 172.21.15.1:0/1995227790 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4014070 con 0x7f8cf0102790 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf019d730 con 0x7f8cf0102790 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8cecff9640 1 -- 172.21.15.1:0/1995227790 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ce4010af0 con 0x7f8cf0102790 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8cecff9640 1 -- 172.21.15.1:0/1995227790 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4017d10 con 0x7f8cf0102790 2024-09-16T00:45:45.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.267+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01a5010 con 0x7f8cf0102790 2024-09-16T00:45:45.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.268+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf01021c0 con 0x7f8cf0102790 2024-09-16T00:45:45.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.269+0000 7f8cecff9640 1 -- 172.21.15.1:0/1995227790 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8ce4017070 con 0x7f8cf0102790 2024-09-16T00:45:45.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.270+0000 7f8cecff9640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cd80a3240 0x7f8cd80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:45.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.270+0000 7f8cef7fe640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cd80a3240 0x7f8cd80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:45.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.270+0000 7f8cecff9640 1 -- 172.21.15.1:0/1995227790 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f8ce40cf0e0 con 0x7f8cf0102790 2024-09-16T00:45:45.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.270+0000 7f8cef7fe640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cd80a3240 0x7f8cd80a5700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005ab0 tx=0x7f8ce0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:45.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.274+0000 7f8cecff9640 1 -- 172.21.15.1:0/1995227790 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ce409ba00 con 0x7f8cf0102790 2024-09-16T00:45:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.491+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cf010a150 con 0x7f8cf0102790 2024-09-16T00:45:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.491+0000 7f8cecff9640 1 -- 172.21.15.1:0/1995227790 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7f8ce409b300 con 0x7f8cf0102790 2024-09-16T00:45:45.492 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:45:45.492 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:45:45.492 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cd80a3240 msgr2=0x7f8cd80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cd80a3240 0x7f8cd80a5700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005ab0 tx=0x7f8ce0005a20 comp rx=0 tx=0).stop 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0102790 msgr2=0x7f8cf01989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0102790 0x7f8cf01989d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400b4a0 tx=0x7f8ce400b960 comp rx=0 tx=0).stop 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 shutdown_connections 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf0106f10 0x7f8cf019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cf0102790 0x7f8cf01989d0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f8cd80a3240 0x7f8cd80a5700 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 --2- 172.21.15.1:0/1995227790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cf0101de0 0x7f8cf0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.493+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 >> 172.21.15.1:0/1995227790 conn(0x7f8cf00fd8c0 msgr2=0x7f8cf00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:45.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.494+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 shutdown_connections 2024-09-16T00:45:45.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:45.494+0000 7f8cf6017640 1 -- 172.21.15.1:0/1995227790 wait complete. 2024-09-16T00:45:46.164 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:46 smithi001 ceph-mon[30934]: pgmap v871: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:45:46.164 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:46 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1995227790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:46.166 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:45:46.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:46 smithi019 ceph-mon[35186]: pgmap v871: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:45:46.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:46 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1995227790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:46.427 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:46 smithi153 ceph-mon[35538]: pgmap v871: 129 pgs: 129 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:45:46.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:46 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1995227790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.922+0000 7fe4f40fe640 1 -- 172.21.15.1:0/749346476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 msgr2=0x7fe4ec10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.922+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/749346476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 0x7fe4ec10bb50 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009900 tx=0x7fe4dc02a480 comp rx=0 tx=0).stop 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.922+0000 7fe4f40fe640 1 -- 172.21.15.1:0/749346476 shutdown_connections 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.922+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/749346476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4ec10c170 0x7fe4ec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.922+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/749346476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 0x7fe4ec10bb50 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.922+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/749346476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ec102d40 0x7fe4ec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.922+0000 7fe4f40fe640 1 -- 172.21.15.1:0/749346476 >> 172.21.15.1:0/749346476 conn(0x7fe4ec0fc7a0 msgr2=0x7fe4ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.922+0000 7fe4f40fe640 1 -- 172.21.15.1:0/749346476 shutdown_connections 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.922+0000 7fe4f40fe640 1 -- 172.21.15.1:0/749346476 wait complete. 2024-09-16T00:45:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f40fe640 1 Processor -- start 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f40fe640 1 -- start start 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f40fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ec102d40 0x7fe4ec198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f40fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 0x7fe4ec198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f40fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4ec10c170 0x7fe4ec19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f40fe640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec112030 con 0x7fe4ec103660 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f40fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec111eb0 con 0x7fe4ec102d40 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f40fe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec1121b0 con 0x7fe4ec10c170 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f1699640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 0x7fe4ec198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f1699640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 0x7fe4ec198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58484/0 (socket says 172.21.15.1:58484) 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f1699640 1 -- 172.21.15.1:0/358365133 learned_addr learned my addr 172.21.15.1:0/358365133 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f1e9a640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ec102d40 0x7fe4ec198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f269b640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4ec10c170 0x7fe4ec19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f1699640 1 -- 172.21.15.1:0/358365133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4ec10c170 msgr2=0x7fe4ec19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f1699640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4ec10c170 0x7fe4ec19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f1699640 1 -- 172.21.15.1:0/358365133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ec102d40 msgr2=0x7fe4ec198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f1699640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ec102d40 0x7fe4ec198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.923+0000 7fe4f1699640 1 -- 172.21.15.1:0/358365133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ec19d530 con 0x7fe4ec103660 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.924+0000 7fe4f1699640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 0x7fe4ec198ac0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005ab0 tx=0x7fe4dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.924+0000 7fe4e2ffd640 1 -- 172.21.15.1:0/358365133 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc03f030 con 0x7fe4ec103660 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.924+0000 7fe4e2ffd640 1 -- 172.21.15.1:0/358365133 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc003ce0 con 0x7fe4ec103660 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.924+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ec19d7c0 con 0x7fe4ec103660 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.924+0000 7fe4e2ffd640 1 -- 172.21.15.1:0/358365133 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc035070 con 0x7fe4ec103660 2024-09-16T00:45:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.924+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ec1a5100 con 0x7fe4ec103660 2024-09-16T00:45:46.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.925+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ec1031a0 con 0x7fe4ec103660 2024-09-16T00:45:46.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.926+0000 7fe4e2ffd640 1 -- 172.21.15.1:0/358365133 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe4dc042030 con 0x7fe4ec103660 2024-09-16T00:45:46.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.927+0000 7fe4e2ffd640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4d80a32c0 0x7fe4d80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:46.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.927+0000 7fe4f1e9a640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4d80a32c0 0x7fe4d80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:46.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.927+0000 7fe4e2ffd640 1 -- 172.21.15.1:0/358365133 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7fe4dc0f0520 con 0x7fe4ec103660 2024-09-16T00:45:46.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.927+0000 7fe4f1e9a640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4d80a32c0 0x7fe4d80a5780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009990 tx=0x7fe4e8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:46.931+0000 7fe4e2ffd640 1 -- 172.21.15.1:0/358365133 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4dc0bcde0 con 0x7fe4ec103660 2024-09-16T00:45:47.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.146+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4ec1998e0 con 0x7fe4ec103660 2024-09-16T00:45:47.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.147+0000 7fe4e2ffd640 1 -- 172.21.15.1:0/358365133 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11894 (secure 0 0 0) 0x7fe4ec1998e0 con 0x7fe4ec103660 2024-09-16T00:45:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:45:47.148 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"btime":"2024-09-16T00:28:41:668286+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2811424948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2811424948},{"type":"v1","addr":"172.21.15.1:6835","nonce":2811424948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/436431990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":436431990},{"type":"v1","addr":"172.21.15.1:6837","nonce":436431990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4066219801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4066219801},{"type":"v1","addr":"172.21.15.1:6839","nonce":4066219801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/208912636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":208912636},{"type":"v1","addr":"172.21.15.1:6841","nonce":208912636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1581415887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1581415887},{"type":"v1","addr":"172.21.15.153:6833","nonce":1581415887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1257631294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1257631294},{"type":"v1","addr":"172.21.15.19:6833","nonce":1257631294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:26:18.890037+0000","modified":"2024-09-16T00:28:41.306996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24470,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/389176790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":389176790},{"type":"v1","addr":"172.21.15.19:6835","nonce":389176790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3078906320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3078906320},{"type":"v1","addr":"172.21.15.153:6835","nonce":3078906320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/3490951208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3490951208},{"type":"v1","addr":"172.21.15.153:6837","nonce":3490951208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6839/1065022363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":1065022363},{"type":"v1","addr":"172.21.15.153:6839","nonce":1065022363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.19:6837/928170641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":928170641},{"type":"v1","addr":"172.21.15.19:6837","nonce":928170641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6839/3056769092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":3056769092},{"type":"v1","addr":"172.21.15.19:6839","nonce":3056769092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24470,24506]},"id":1}]} 2024-09-16T00:45:47.148 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-09-16T00:45:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4d80a32c0 msgr2=0x7fe4d80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4d80a32c0 0x7fe4d80a5780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009990 tx=0x7fe4e8009110 comp rx=0 tx=0).stop 2024-09-16T00:45:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 msgr2=0x7fe4ec198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 0x7fe4ec198ac0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005ab0 tx=0x7fe4dc0042f0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 shutdown_connections 2024-09-16T00:45:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe4d80a32c0 0x7fe4d80a5780 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4ec10c170 0x7fe4ec19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4ec103660 0x7fe4ec198ac0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 --2- 172.21.15.1:0/358365133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ec102d40 0x7fe4ec198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.149+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 >> 172.21.15.1:0/358365133 conn(0x7fe4ec0fc7a0 msgr2=0x7fe4ec0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.150+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 shutdown_connections 2024-09-16T00:45:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:47.150+0000 7fe4f40fe640 1 -- 172.21.15.1:0/358365133 wait complete. 2024-09-16T00:45:47.842 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:45:48.103 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:48.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:48 smithi001 ceph-mon[30934]: pgmap v872: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T00:45:48.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:48 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/358365133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:48 smithi019 ceph-mon[35186]: pgmap v872: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T00:45:48.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:48 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/358365133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:48 smithi153 ceph-mon[35538]: pgmap v872: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T00:45:48.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:48 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/358365133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.617+0000 7faa1c653640 1 -- 172.21.15.1:0/2340488702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 msgr2=0x7faa141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.617+0000 7faa1c653640 1 --2- 172.21.15.1:0/2340488702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 0x7faa141021c0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7faa10009900 tx=0x7faa1002a410 comp rx=0 tx=0).stop 2024-09-16T00:45:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.617+0000 7faa1c653640 1 -- 172.21.15.1:0/2340488702 shutdown_connections 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.617+0000 7faa1c653640 1 --2- 172.21.15.1:0/2340488702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14106f10 0x7faa1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.617+0000 7faa1c653640 1 --2- 172.21.15.1:0/2340488702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa14102790 0x7faa141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.617+0000 7faa1c653640 1 --2- 172.21.15.1:0/2340488702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 0x7faa141021c0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.617+0000 7faa1c653640 1 -- 172.21.15.1:0/2340488702 >> 172.21.15.1:0/2340488702 conn(0x7faa140fd8c0 msgr2=0x7faa140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.617+0000 7faa1c653640 1 -- 172.21.15.1:0/2340488702 shutdown_connections 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.617+0000 7faa1c653640 1 -- 172.21.15.1:0/2340488702 wait complete. 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1c653640 1 Processor -- start 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1c653640 1 -- start start 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1c653640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 0x7faa141983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1c653640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14102790 0x7faa14198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1c653640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa14106f10 0x7faa1419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1c653640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faa14111f70 con 0x7faa14101de0 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1c653640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa14111df0 con 0x7faa14102790 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1c653640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faa141120f0 con 0x7faa14106f10 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1a3ef640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 0x7faa141983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1a3ef640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 0x7faa141983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58496/0 (socket says 172.21.15.1:58496) 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.618+0000 7faa1a3ef640 1 -- 172.21.15.1:0/1554139360 learned_addr learned my addr 172.21.15.1:0/1554139360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa1abf0640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa14106f10 0x7faa1419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa19bee640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14102790 0x7faa14198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa1a3ef640 1 -- 172.21.15.1:0/1554139360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa14106f10 msgr2=0x7faa1419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa1a3ef640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa14106f10 0x7faa1419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa1a3ef640 1 -- 172.21.15.1:0/1554139360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14102790 msgr2=0x7faa14198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa1a3ef640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14102790 0x7faa14198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa1a3ef640 1 -- 172.21.15.1:0/1554139360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1419d380 con 0x7faa14101de0 2024-09-16T00:45:48.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa19bee640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14102790 0x7faa14198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa1a3ef640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 0x7faa141983d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7faa10006010 tx=0x7faa100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa0b7fe640 1 -- 172.21.15.1:0/1554139360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1003f030 con 0x7faa14101de0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa0b7fe640 1 -- 172.21.15.1:0/1554139360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa10003ce0 con 0x7faa14101de0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1419d610 con 0x7faa14101de0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa0b7fe640 1 -- 172.21.15.1:0/1554139360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10035070 con 0x7faa14101de0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.619+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa141a4f50 con 0x7faa14101de0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.620+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa141032c0 con 0x7faa14101de0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.621+0000 7faa0b7fe640 1 -- 172.21.15.1:0/1554139360 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7faa10042030 con 0x7faa14101de0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.622+0000 7faa0b7fe640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa000a3240 0x7faa000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.622+0000 7faa19bee640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa000a3240 0x7faa000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.622+0000 7faa0b7fe640 1 -- 172.21.15.1:0/1554139360 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7faa100f03e0 con 0x7faa14101de0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.623+0000 7faa19bee640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa000a3240 0x7faa000a5700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faa140fdce0 tx=0x7faa04005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:48.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.626+0000 7faa0b7fe640 1 -- 172.21.15.1:0/1554139360 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa100bce60 con 0x7faa14101de0 2024-09-16T00:45:48.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.802+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa141021c0 con 0x7faa14101de0 2024-09-16T00:45:48.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.803+0000 7faa0b7fe640 1 -- 172.21.15.1:0/1554139360 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24902 (secure 0 0 0) 0x7faa100bc760 con 0x7faa14101de0 2024-09-16T00:45:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-16T00:45:48.804 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"1226ed86-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:18:28.249157+0000","modified":"2024-09-16T00:27:26.243912+0000","last_up_change":"2024-09-16T00:25:26.382820+0000","last_in_change":"2024-09-16T00:25:02.992692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:21:40.721155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:26:14.641676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:26:17.242600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:26:23.721714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff1bfb29-43e6-4e65-8d96-715b74e01491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6803","nonce":2752149463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6805","nonce":2752149463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6809","nonce":2752149463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2752149463},{"type":"v1","addr":"172.21.15.1:6807","nonce":2752149463}]},"public_addr":"172.21.15.1:6803/2752149463","cluster_addr":"172.21.15.1:6805/2752149463","heartbeat_back_addr":"172.21.15.1:6809/2752149463","heartbeat_front_addr":"172.21.15.1:6807/2752149463","state":["exists","up"]},{"osd":1,"uuid":"6449fa21-488a-46cb-9e6c-1391e84c0f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6801","nonce":3708883557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6803","nonce":3708883557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6807","nonce":3708883557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3708883557},{"type":"v1","addr":"172.21.15.19:6805","nonce":3708883557}]},"public_addr":"172.21.15.19:6801/3708883557","cluster_addr":"172.21.15.19:6803/3708883557","heartbeat_back_addr":"172.21.15.19:6807/3708883557","heartbeat_front_addr":"172.21.15.19:6805/3708883557","state":["exists","up"]},{"osd":2,"uuid":"1c725bbf-d8d7-4c9e-9d63-4b1a82481a8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6801","nonce":846294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6803","nonce":846294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6807","nonce":846294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":846294555},{"type":"v1","addr":"172.21.15.153:6805","nonce":846294555}]},"public_addr":"172.21.15.153:6801/846294555","cluster_addr":"172.21.15.153:6803/846294555","heartbeat_back_addr":"172.21.15.153:6807/846294555","heartbeat_front_addr":"172.21.15.153:6805/846294555","state":["exists","up"]},{"osd":3,"uuid":"926a23b3-e3d6-41bf-856e-226f69d8915b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6811","nonce":2589582331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6813","nonce":2589582331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6817","nonce":2589582331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2589582331},{"type":"v1","addr":"172.21.15.1:6815","nonce":2589582331}]},"public_addr":"172.21.15.1:6811/2589582331","cluster_addr":"172.21.15.1:6813/2589582331","heartbeat_back_addr":"172.21.15.1:6817/2589582331","heartbeat_front_addr":"172.21.15.1:6815/2589582331","state":["exists","up"]},{"osd":4,"uuid":"e8536388-50a8-4dd3-acd2-bd8a71f8cfad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6809","nonce":616138863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6811","nonce":616138863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6815","nonce":616138863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":616138863},{"type":"v1","addr":"172.21.15.19:6813","nonce":616138863}]},"public_addr":"172.21.15.19:6809/616138863","cluster_addr":"172.21.15.19:6811/616138863","heartbeat_back_addr":"172.21.15.19:6815/616138863","heartbeat_front_addr":"172.21.15.19:6813/616138863","state":["exists","up"]},{"osd":5,"uuid":"617ccae2-59c9-45bd-83f0-59b1f3555020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6809","nonce":394316446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6811","nonce":394316446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6815","nonce":394316446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":394316446},{"type":"v1","addr":"172.21.15.153:6813","nonce":394316446}]},"public_addr":"172.21.15.153:6809/394316446","cluster_addr":"172.21.15.153:6811/394316446","heartbeat_back_addr":"172.21.15.153:6815/394316446","heartbeat_front_addr":"172.21.15.153:6813/394316446","state":["exists","up"]},{"osd":6,"uuid":"b3e05a31-43c7-466f-ab8b-5c5cbf8eee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6819","nonce":2734761121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6821","nonce":2734761121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6825","nonce":2734761121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2734761121},{"type":"v1","addr":"172.21.15.1:6823","nonce":2734761121}]},"public_addr":"172.21.15.1:6819/2734761121","cluster_addr":"172.21.15.1:6821/2734761121","heartbeat_back_addr":"172.21.15.1:6825/2734761121","heartbeat_front_addr":"172.21.15.1:6823/2734761121","state":["exists","up"]},{"osd":7,"uuid":"9860f5c4-575b-402c-98a2-3436d37b4cfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6817","nonce":549185107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6819","nonce":549185107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6823","nonce":549185107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":549185107},{"type":"v1","addr":"172.21.15.19:6821","nonce":549185107}]},"public_addr":"172.21.15.19:6817/549185107","cluster_addr":"172.21.15.19:6819/549185107","heartbeat_back_addr":"172.21.15.19:6823/549185107","heartbeat_front_addr":"172.21.15.19:6821/549185107","state":["exists","up"]},{"osd":8,"uuid":"2371d5a7-e6c3-415f-b6d3-35df1a4c3a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6817","nonce":2981426956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6819","nonce":2981426956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6823","nonce":2981426956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2981426956},{"type":"v1","addr":"172.21.15.153:6821","nonce":2981426956}]},"public_addr":"172.21.15.153:6817/2981426956","cluster_addr":"172.21.15.153:6819/2981426956","heartbeat_back_addr":"172.21.15.153:6823/2981426956","heartbeat_front_addr":"172.21.15.153:6821/2981426956","state":["exists","up"]},{"osd":9,"uuid":"92377e46-86e8-49b1-91b3-8180013968bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6827","nonce":4278887169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6829","nonce":4278887169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6833","nonce":4278887169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":4278887169},{"type":"v1","addr":"172.21.15.1:6831","nonce":4278887169}]},"public_addr":"172.21.15.1:6827/4278887169","cluster_addr":"172.21.15.1:6829/4278887169","heartbeat_back_addr":"172.21.15.1:6833/4278887169","heartbeat_front_addr":"172.21.15.1:6831/4278887169","state":["exists","up"]},{"osd":10,"uuid":"74fcd28a-7ae8-4dca-a314-b22a8c871db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6825","nonce":1721081370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6827","nonce":1721081370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6831","nonce":1721081370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1721081370},{"type":"v1","addr":"172.21.15.19:6829","nonce":1721081370}]},"public_addr":"172.21.15.19:6825/1721081370","cluster_addr":"172.21.15.19:6827/1721081370","heartbeat_back_addr":"172.21.15.19:6831/1721081370","heartbeat_front_addr":"172.21.15.19:6829/1721081370","state":["exists","up"]},{"osd":11,"uuid":"cf7afb8e-9c11-460a-960d-57f1eba3ead6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6825","nonce":3642895016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6827","nonce":3642895016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6831","nonce":3642895016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3642895016},{"type":"v1","addr":"172.21.15.153:6829","nonce":3642895016}]},"public_addr":"172.21.15.153:6825/3642895016","cluster_addr":"172.21.15.153:6827/3642895016","heartbeat_back_addr":"172.21.15.153:6831/3642895016","heartbeat_front_addr":"172.21.15.153:6829/3642895016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:47.676567+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:12.065908+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:36.333179+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.826379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:27.191861+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:52.631888+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:18.096497+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:23:43.391141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:07.760631+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:34.118722+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:59.491292+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:24.475000+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3932831772":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3251079055":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/3191294475":"2024-09-17T00:19:24.226024+0000","172.21.15.1:6800/178572347":"2024-09-17T00:19:24.226024+0000","172.21.15.1:0/1613718962":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6801/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:6800/786826717":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/639100255":"2024-09-17T00:19:00.598574+0000","172.21.15.1:0/365027668":"2024-09-17T00:19:00.598574+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:45:48.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa000a3240 msgr2=0x7faa000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:48.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa000a3240 0x7faa000a5700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faa140fdce0 tx=0x7faa04005e50 comp rx=0 tx=0).stop 2024-09-16T00:45:48.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 msgr2=0x7faa141983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:48.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 0x7faa141983d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7faa10006010 tx=0x7faa100042f0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 shutdown_connections 2024-09-16T00:45:48.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faa000a3240 0x7faa000a5700 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa14106f10 0x7faa1419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14102790 0x7faa14198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 --2- 172.21.15.1:0/1554139360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa14101de0 0x7faa141983d0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 >> 172.21.15.1:0/1554139360 conn(0x7faa140fd8c0 msgr2=0x7faa140febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:48.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 shutdown_connections 2024-09-16T00:45:48.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:48.806+0000 7faa1c653640 1 -- 172.21.15.1:0/1554139360 wait complete. 2024-09-16T00:45:49.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:49 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1554139360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:45:49.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:49 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1554139360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:45:49.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:49 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1554139360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:45:49.526 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-16T00:45:49.803 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:50.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:50 smithi001 ceph-mon[30934]: pgmap v873: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:45:50.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.306+0000 7f830f8c4640 1 -- 172.21.15.1:0/66822756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83081008c0 msgr2=0x7f8308100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:50.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.306+0000 7f830f8c4640 1 --2- 172.21.15.1:0/66822756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83081008c0 0x7f8308100ca0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f8304009900 tx=0x7f830402a410 comp rx=0 tx=0).stop 2024-09-16T00:45:50.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.306+0000 7f830f8c4640 1 -- 172.21.15.1:0/66822756 shutdown_connections 2024-09-16T00:45:50.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.306+0000 7f830f8c4640 1 --2- 172.21.15.1:0/66822756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f830810e470 0x7f8308110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:50.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.306+0000 7f830f8c4640 1 --2- 172.21.15.1:0/66822756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8308101270 0x7f830810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:50.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.306+0000 7f830f8c4640 1 --2- 172.21.15.1:0/66822756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83081008c0 0x7f8308100ca0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:50.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.306+0000 7f830f8c4640 1 -- 172.21.15.1:0/66822756 >> 172.21.15.1:0/66822756 conn(0x7f83080fc7d0 msgr2=0x7f83080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:50.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.306+0000 7f830f8c4640 1 -- 172.21.15.1:0/66822756 shutdown_connections 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.306+0000 7f830f8c4640 1 -- 172.21.15.1:0/66822756 wait complete. 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830f8c4640 1 Processor -- start 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830f8c4640 1 -- start start 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830f8c4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83081008c0 0x7f830819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830f8c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8308101270 0x7f830819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830f8c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f830810e470 0x7f830819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830f8c4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83081141a0 con 0x7f8308101270 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830f8c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8308114020 con 0x7f830810e470 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830f8c4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8308114320 con 0x7f83081008c0 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830ce5f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8308101270 0x7f830819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:50.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830ce5f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8308101270 0x7f830819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41708/0 (socket says 172.21.15.1:41708) 2024-09-16T00:45:50.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830ce5f640 1 -- 172.21.15.1:0/546472110 learned_addr learned my addr 172.21.15.1:0/546472110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:50.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.307+0000 7f830de61640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f830810e470 0x7f830819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:50.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830d660640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83081008c0 0x7f830819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:50.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830ce5f640 1 -- 172.21.15.1:0/546472110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83081008c0 msgr2=0x7f830819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:50.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830ce5f640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83081008c0 0x7f830819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:50.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830ce5f640 1 -- 172.21.15.1:0/546472110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f830810e470 msgr2=0x7f830819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:50.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830ce5f640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f830810e470 0x7f830819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:50.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830ce5f640 1 -- 172.21.15.1:0/546472110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830819f650 con 0x7f8308101270 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830d660640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83081008c0 0x7f830819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830ce5f640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8308101270 0x7f830819abe0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f82f800d950 tx=0x7f82f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f82fe7fc640 1 -- 172.21.15.1:0/546472110 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8017030 con 0x7f8308101270 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f82fe7fc640 1 -- 172.21.15.1:0/546472110 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f8004020 con 0x7f8308101270 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830819f940 con 0x7f8308101270 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f82fe7fc640 1 -- 172.21.15.1:0/546472110 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f80047a0 con 0x7f8308101270 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.308+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83081a7220 con 0x7f8308101270 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.309+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8308100ca0 con 0x7f8308101270 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.311+0000 7f82fe7fc640 1 -- 172.21.15.1:0/546472110 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f82f80041d0 con 0x7f8308101270 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.311+0000 7f82fe7fc640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82f40a32c0 0x7f82f40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.311+0000 7f82fe7fc640 1 -- 172.21.15.1:0/546472110 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9253+0+0 (secure 0 0 0) 0x7f82f80cfcf0 con 0x7f8308101270 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.311+0000 7f830d660640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82f40a32c0 0x7f82f40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.312+0000 7f830d660640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82f40a32c0 0x7f82f40a5780 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8304005ab0 tx=0x7f8304005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:50.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.315+0000 7f82fe7fc640 1 -- 172.21.15.1:0/546472110 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82f809c690 con 0x7f8308101270 2024-09-16T00:45:50.356 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-mon[35186]: pgmap v873: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:45:50.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-mon[35538]: pgmap v873: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:45:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.525+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f830819ba00 con 0x7f8308101270 2024-09-16T00:45:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.528+0000 7f82fe7fc640 1 -- 172.21.15.1:0/546472110 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f830819ba00 con 0x7f8308101270 2024-09-16T00:45:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82f40a32c0 msgr2=0x7f82f40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82f40a32c0 0x7f82f40a5780 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8304005ab0 tx=0x7f8304005a20 comp rx=0 tx=0).stop 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8308101270 msgr2=0x7f830819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8308101270 0x7f830819abe0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f82f800d950 tx=0x7f82f800de10 comp rx=0 tx=0).stop 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 shutdown_connections 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f830810e470 0x7f830819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8308101270 0x7f830819abe0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f82f40a32c0 0x7f82f40a5780 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 --2- 172.21.15.1:0/546472110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83081008c0 0x7f830819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 >> 172.21.15.1:0/546472110 conn(0x7f83080fc7d0 msgr2=0x7f83080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 shutdown_connections 2024-09-16T00:45:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:50.531+0000 7f830f8c4640 1 -- 172.21.15.1:0/546472110 wait complete. 2024-09-16T00:45:50.857 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e[65194]: ignoring --setuser ceph since I am not root 2024-09-16T00:45:50.857 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e[65194]: ignoring --setgroup ceph since I am not root 2024-09-16T00:45:50.857 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-mds[65198]: -- 172.21.15.19:0/3590879717 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612607f5860 con 0x5612607d3400 2024-09-16T00:45:50.857 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e[65194]: starting mds.e at 2024-09-16T00:45:50.858 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h[65495]: ignoring --setuser ceph since I am not root 2024-09-16T00:45:50.858 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h[65495]: ignoring --setgroup ceph since I am not root 2024-09-16T00:45:50.858 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-mds[65499]: -- 172.21.15.19:0/2124388105 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dcbdc59c20 con 0x55dcbdc37000 2024-09-16T00:45:50.858 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h[65495]: starting mds.h at 2024-09-16T00:45:50.858 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b[65807]: ignoring --setuser ceph since I am not root 2024-09-16T00:45:50.858 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b[65807]: ignoring --setgroup ceph since I am not root 2024-09-16T00:45:50.858 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-mds[65811]: -- 172.21.15.19:0/1927140634 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3fe07d860 con 0x55b3fe05a800 2024-09-16T00:45:50.858 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:45:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b[65807]: starting mds.b at 2024-09-16T00:45:50.872 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:50 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a[30930]: 2024-09-16T00:45:50.526+0000 7f9ebe757640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:45:50.961 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c[65572]: ignoring --setuser ceph since I am not root 2024-09-16T00:45:50.961 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c[65572]: ignoring --setgroup ceph since I am not root 2024-09-16T00:45:50.962 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-mds[65576]: -- 172.21.15.153:0/4134647247 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9c8b69c20 con 0x55f9c8b47400 2024-09-16T00:45:50.962 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c[65572]: starting mds.c at 2024-09-16T00:45:50.962 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f[65267]: ignoring --setuser ceph since I am not root 2024-09-16T00:45:50.962 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f[65267]: ignoring --setgroup ceph since I am not root 2024-09-16T00:45:50.962 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f[65267]: starting mds.f at 2024-09-16T00:45:50.962 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i[65875]: ignoring --setuser ceph since I am not root 2024-09-16T00:45:50.963 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i[65875]: ignoring --setgroup ceph since I am not root 2024-09-16T00:45:50.963 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-mds[65879]: -- 172.21.15.153:0/3682599624 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561fc6ac1860 con 0x561fc6a9f000 2024-09-16T00:45:50.963 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:45:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i[65875]: starting mds.i at 2024-09-16T00:45:51.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:51 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/546472110' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:45:51.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:51 smithi001 ceph-mon[30934]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:45:51.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:51 smithi001 ceph-mon[30934]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:45:51.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:51 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/546472110' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:45:51.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:51 smithi001 ceph-mon[30934]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:45:51.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:51 smithi001 ceph-mon[30934]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-16T00:45:51.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T00:45:51.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:51 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/546472110' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:45:51.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:51 smithi019 ceph-mon[35186]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:45:51.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:51 smithi019 ceph-mon[35186]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:45:51.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:51 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/546472110' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:45:51.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:51 smithi019 ceph-mon[35186]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:45:51.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:51 smithi019 ceph-mon[35186]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-16T00:45:51.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:51 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/546472110' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:45:51.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:51 smithi153 ceph-mon[35538]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:45:51.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:51 smithi153 ceph-mon[35538]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:45:51.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:51 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/546472110' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:45:51.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:51 smithi153 ceph-mon[35538]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:45:51.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:51 smithi153 ceph-mon[35538]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-16T00:45:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:52.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.030+0000 7f46c913e640 1 -- 172.21.15.1:0/183284318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4106f10 msgr2=0x7f46c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.030+0000 7f46c913e640 1 --2- 172.21.15.1:0/183284318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4106f10 0x7f46c410ea90 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009ca0 tx=0x7f46bc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.030+0000 7f46c913e640 1 -- 172.21.15.1:0/183284318 shutdown_connections 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.030+0000 7f46c913e640 1 --2- 172.21.15.1:0/183284318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4106f10 0x7f46c410ea90 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.030+0000 7f46c913e640 1 --2- 172.21.15.1:0/183284318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4102790 0x7f46c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.030+0000 7f46c913e640 1 --2- 172.21.15.1:0/183284318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c4101de0 0x7f46c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.030+0000 7f46c913e640 1 -- 172.21.15.1:0/183284318 >> 172.21.15.1:0/183284318 conn(0x7f46c40fd8c0 msgr2=0x7f46c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.030+0000 7f46c913e640 1 -- 172.21.15.1:0/183284318 shutdown_connections 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.030+0000 7f46c913e640 1 -- 172.21.15.1:0/183284318 wait complete. 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c913e640 1 Processor -- start 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c913e640 1 -- start start 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c913e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c4101de0 0x7f46c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c913e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4102790 0x7f46c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c913e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4106f10 0x7f46c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c913e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4111fc0 con 0x7f46c4102790 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c913e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4111e40 con 0x7f46c4106f10 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c913e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4112140 con 0x7f46c4101de0 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46ba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4102790 0x7f46c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46ba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4102790 0x7f46c4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41734/0 (socket says 172.21.15.1:41734) 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46ba59c640 1 -- 172.21.15.1:0/1213822394 learned_addr learned my addr 172.21.15.1:0/1213822394 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c359e640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4106f10 0x7f46c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.031+0000 7f46c2d9d640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c4101de0 0x7f46c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46ba59c640 1 -- 172.21.15.1:0/1213822394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c4101de0 msgr2=0x7f46c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46ba59c640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c4101de0 0x7f46c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46ba59c640 1 -- 172.21.15.1:0/1213822394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4106f10 msgr2=0x7f46c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46ba59c640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4106f10 0x7f46c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46ba59c640 1 -- 172.21.15.1:0/1213822394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c41a6860 con 0x7f46c4102790 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46c2d9d640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c4101de0 0x7f46c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46ba59c640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4102790 0x7f46c4074a30 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00d950 tx=0x7f46ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46c0d99640 1 -- 172.21.15.1:0/1213822394 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac017030 con 0x7f46c4102790 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46c0d99640 1 -- 172.21.15.1:0/1213822394 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac004020 con 0x7f46c4102790 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c41a6b50 con 0x7f46c4102790 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46c0d99640 1 -- 172.21.15.1:0/1213822394 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0047a0 con 0x7f46c4102790 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.032+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c41a7090 con 0x7f46c4102790 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.033+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c41021c0 con 0x7f46c4102790 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.034+0000 7f46c0d99640 1 -- 172.21.15.1:0/1213822394 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f46ac0041d0 con 0x7f46c4102790 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.035+0000 7f46c0d99640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46b00a3240 0x7f46b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.035+0000 7f46c2d9d640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46b00a3240 0x7f46b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.035+0000 7f46c0d99640 1 -- 172.21.15.1:0/1213822394 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9511+0+0 (secure 0 0 0) 0x7f46ac0cfe50 con 0x7f46c4102790 2024-09-16T00:45:52.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.036+0000 7f46c2d9d640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46b00a3240 0x7f46b00a5700 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f46b40058c0 tx=0x7f46b4005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:52.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.038+0000 7f46c0d99640 1 -- 172.21.15.1:0/1213822394 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46ac09c610 con 0x7f46c4102790 2024-09-16T00:45:52.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: pgmap v875: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:52.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.19:6834/454710863,v1:172.21.15.19:6835/454710863] up:boot 2024-09-16T00:45:52.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.153:6834/302821512,v1:172.21.15.153:6835/302821512] up:boot 2024-09-16T00:45:52.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.19:6837/511439295,v1:172.21.15.19:6839/511439295] up:boot 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.19:6836/193780539,v1:172.21.15.19:6838/193780539] up:boot 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.153:6837/2802791411,v1:172.21.15.153:6839/2802791411] up:boot 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: mds.? [v2:172.21.15.153:6836/3430779643,v1:172.21.15.153:6838/3430779643] up:boot 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:45:52.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:52 smithi001 ceph-mon[30934]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:45:52.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:52.256+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f46c41a7390 con 0x7f46c4102790 2024-09-16T00:45:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: pgmap v875: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.19:6834/454710863,v1:172.21.15.19:6835/454710863] up:boot 2024-09-16T00:45:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.153:6834/302821512,v1:172.21.15.153:6835/302821512] up:boot 2024-09-16T00:45:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.19:6837/511439295,v1:172.21.15.19:6839/511439295] up:boot 2024-09-16T00:45:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.19:6836/193780539,v1:172.21.15.19:6838/193780539] up:boot 2024-09-16T00:45:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.153:6837/2802791411,v1:172.21.15.153:6839/2802791411] up:boot 2024-09-16T00:45:52.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: mds.? [v2:172.21.15.153:6836/3430779643,v1:172.21.15.153:6838/3430779643] up:boot 2024-09-16T00:45:52.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-16T00:45:52.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:45:52.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:45:52.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:45:52.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:45:52.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:45:52.358 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:52 smithi019 ceph-mon[35186]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:45:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: pgmap v875: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.19:6834/454710863,v1:172.21.15.19:6835/454710863] up:boot 2024-09-16T00:45:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.153:6834/302821512,v1:172.21.15.153:6835/302821512] up:boot 2024-09-16T00:45:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.19:6837/511439295,v1:172.21.15.19:6839/511439295] up:boot 2024-09-16T00:45:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.19:6836/193780539,v1:172.21.15.19:6838/193780539] up:boot 2024-09-16T00:45:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.153:6837/2802791411,v1:172.21.15.153:6839/2802791411] up:boot 2024-09-16T00:45:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: mds.? [v2:172.21.15.153:6836/3430779643,v1:172.21.15.153:6838/3430779643] up:boot 2024-09-16T00:45:52.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-16T00:45:52.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:45:52.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:45:52.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:45:52.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:45:52.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:45:52.463 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:52 smithi153 ceph-mon[35538]: from='mgr.14150 172.21.15.1:0/2465394223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:45:53.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.074+0000 7f46c0d99640 1 -- 172.21.15.1:0/1213822394 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f46ac09bf10 con 0x7f46c4102790 2024-09-16T00:45:53.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46b00a3240 msgr2=0x7f46b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:53.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46b00a3240 0x7f46b00a5700 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f46b40058c0 tx=0x7f46b4005ec0 comp rx=0 tx=0).stop 2024-09-16T00:45:53.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4102790 msgr2=0x7f46c4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:53.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4102790 0x7f46c4074a30 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00d950 tx=0x7f46ac00de10 comp rx=0 tx=0).stop 2024-09-16T00:45:53.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 shutdown_connections 2024-09-16T00:45:53.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4106f10 0x7f46c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:53.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c4102790 0x7f46c4074a30 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:53.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f46b00a3240 0x7f46b00a5700 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:53.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 --2- 172.21.15.1:0/1213822394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46c4101de0 0x7f46c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:53.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 >> 172.21.15.1:0/1213822394 conn(0x7f46c40fd8c0 msgr2=0x7f46c40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:53.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 shutdown_connections 2024-09-16T00:45:53.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:53.077+0000 7f46c913e640 1 -- 172.21.15.1:0/1213822394 wait complete. 2024-09-16T00:45:53.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:53 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1213822394' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:45:53.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:53 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1213822394' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:45:53.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:53 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1213822394' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:45:53.894 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T00:45:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:54 smithi001 ceph-mon[30934]: pgmap v876: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:54 smithi001 ceph-mon[30934]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:45:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:54 smithi001 ceph-mon[30934]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:45:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:54 smithi001 ceph-mon[30934]: Cluster is now healthy 2024-09-16T00:45:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:54 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1213822394' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:45:54.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:54 smithi001 ceph-mon[30934]: fsmap 12 up:standby 2024-09-16T00:45:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:54 smithi019 ceph-mon[35186]: pgmap v876: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:54 smithi019 ceph-mon[35186]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:45:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:54 smithi019 ceph-mon[35186]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:45:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:54 smithi019 ceph-mon[35186]: Cluster is now healthy 2024-09-16T00:45:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:54 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1213822394' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:45:54.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:54 smithi019 ceph-mon[35186]: fsmap 12 up:standby 2024-09-16T00:45:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:54 smithi153 ceph-mon[35538]: pgmap v876: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:54.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:54 smithi153 ceph-mon[35538]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:45:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:54 smithi153 ceph-mon[35538]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:45:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:54 smithi153 ceph-mon[35538]: Cluster is now healthy 2024-09-16T00:45:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:54 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1213822394' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:45:54.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:54 smithi153 ceph-mon[35538]: fsmap 12 up:standby 2024-09-16T00:45:54.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.684+0000 7faf6bea3640 1 -- 172.21.15.1:0/651652421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 msgr2=0x7faf641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:54.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.684+0000 7faf6bea3640 1 --2- 172.21.15.1:0/651652421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 0x7faf641021c0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7faf60009900 tx=0x7faf6002a410 comp rx=0 tx=0).stop 2024-09-16T00:45:54.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.684+0000 7faf6bea3640 1 -- 172.21.15.1:0/651652421 shutdown_connections 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.684+0000 7faf6bea3640 1 --2- 172.21.15.1:0/651652421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf64106f10 0x7faf6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.684+0000 7faf6bea3640 1 --2- 172.21.15.1:0/651652421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf64102790 0x7faf641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.684+0000 7faf6bea3640 1 --2- 172.21.15.1:0/651652421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 0x7faf641021c0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.684+0000 7faf6bea3640 1 -- 172.21.15.1:0/651652421 >> 172.21.15.1:0/651652421 conn(0x7faf640fd8c0 msgr2=0x7faf640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.684+0000 7faf6bea3640 1 -- 172.21.15.1:0/651652421 shutdown_connections 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.684+0000 7faf6bea3640 1 -- 172.21.15.1:0/651652421 wait complete. 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6bea3640 1 Processor -- start 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6bea3640 1 -- start start 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6bea3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 0x7faf641985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:54.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6bea3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf64102790 0x7faf64198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6bea3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf64106f10 0x7faf6419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6bea3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112010 con 0x7faf64101de0 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6bea3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf64111e90 con 0x7faf64102790 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6bea3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112190 con 0x7faf64106f10 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf69c3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 0x7faf641985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf69c3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 0x7faf641985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41748/0 (socket says 172.21.15.1:41748) 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf69c3f640 1 -- 172.21.15.1:0/1612442020 learned_addr learned my addr 172.21.15.1:0/1612442020 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6943e640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf64102790 0x7faf64198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf6a440640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf64106f10 0x7faf6419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf69c3f640 1 -- 172.21.15.1:0/1612442020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf64106f10 msgr2=0x7faf6419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf69c3f640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf64106f10 0x7faf6419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf69c3f640 1 -- 172.21.15.1:0/1612442020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf64102790 msgr2=0x7faf64198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf69c3f640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf64102790 0x7faf64198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.685+0000 7faf69c3f640 1 -- 172.21.15.1:0/1612442020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6419d560 con 0x7faf64101de0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.686+0000 7faf69c3f640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 0x7faf641985b0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7faf60005ab0 tx=0x7faf600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.686+0000 7faf5affd640 1 -- 172.21.15.1:0/1612442020 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6003f030 con 0x7faf64101de0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.686+0000 7faf5affd640 1 -- 172.21.15.1:0/1612442020 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf60003ce0 con 0x7faf64101de0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.686+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6419d7f0 con 0x7faf64101de0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.686+0000 7faf5affd640 1 -- 172.21.15.1:0/1612442020 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60035070 con 0x7faf64101de0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.686+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a5130 con 0x7faf64101de0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.687+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf641032c0 con 0x7faf64101de0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.688+0000 7faf5affd640 1 -- 172.21.15.1:0/1612442020 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7faf60042030 con 0x7faf64101de0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.689+0000 7faf5affd640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf500a3240 0x7faf500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.689+0000 7faf6943e640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf500a3240 0x7faf500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.689+0000 7faf5affd640 1 -- 172.21.15.1:0/1612442020 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9511+0+0 (secure 0 0 0) 0x7faf600f09d0 con 0x7faf64101de0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.689+0000 7faf6943e640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf500a3240 0x7faf500a5700 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faf640ffde0 tx=0x7faf54009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.692+0000 7faf5affd640 1 -- 172.21.15.1:0/1612442020 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf600bd1f0 con 0x7faf64101de0 2024-09-16T00:45:54.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:54.866+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faf6410a0f0 con 0x7faf64101de0 2024-09-16T00:45:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.076+0000 7faf5affd640 1 -- 172.21.15.1:0/1612442020 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7faf600bcaf0 con 0x7faf64101de0 2024-09-16T00:45:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-09-16T00:45:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.079+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf500a3240 msgr2=0x7faf500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.079+0000 7faf6bea3640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf500a3240 0x7faf500a5700 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faf640ffde0 tx=0x7faf54009110 comp rx=0 tx=0).stop 2024-09-16T00:45:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.079+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 msgr2=0x7faf641985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.079+0000 7faf6bea3640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 0x7faf641985b0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7faf60005ab0 tx=0x7faf600042f0 comp rx=0 tx=0).stop 2024-09-16T00:45:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.080+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 shutdown_connections 2024-09-16T00:45:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.080+0000 7faf6bea3640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7faf500a3240 0x7faf500a5700 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.080+0000 7faf6bea3640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf64106f10 0x7faf6419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.080+0000 7faf6bea3640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf64102790 0x7faf64198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.080+0000 7faf6bea3640 1 --2- 172.21.15.1:0/1612442020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf64101de0 0x7faf641985b0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.080+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 >> 172.21.15.1:0/1612442020 conn(0x7faf640fd8c0 msgr2=0x7faf640fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.080+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 shutdown_connections 2024-09-16T00:45:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:55.080+0000 7faf6bea3640 1 -- 172.21.15.1:0/1612442020 wait complete. 2024-09-16T00:45:55.120 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:55 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1612442020' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:45:55.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:55 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1612442020' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:45:55.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:55 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1612442020' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:45:55.908 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T00:45:56.177 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:56.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:56 smithi001 ceph-mon[30934]: pgmap v877: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:56.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:56 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/1612442020' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:45:56.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:56 smithi001 ceph-mon[30934]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:45:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:56 smithi019 ceph-mon[35186]: pgmap v877: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:56 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/1612442020' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:45:56.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:56 smithi019 ceph-mon[35186]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:45:56.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:56 smithi153 ceph-mon[35538]: pgmap v877: 129 pgs: 129 active+clean; 7.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T00:45:56.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:56 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/1612442020' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:45:56.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:56 smithi153 ceph-mon[35538]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:45:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.708+0000 7f2e2d93d640 1 -- 172.21.15.1:0/2671181596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 msgr2=0x7f2e2810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.708+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/2671181596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 0x7f2e2810d950 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009cd0 tx=0x7f2e2002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.709+0000 7f2e2d93d640 1 -- 172.21.15.1:0/2671181596 shutdown_connections 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.709+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/2671181596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 0x7f2e2810d950 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.709+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/2671181596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e28100a80 0x7f2e28100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.709+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/2671181596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e28106a80 0x7f2e28106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.709+0000 7f2e2d93d640 1 -- 172.21.15.1:0/2671181596 >> 172.21.15.1:0/2671181596 conn(0x7f2e280fc7a0 msgr2=0x7f2e280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.709+0000 7f2e2d93d640 1 -- 172.21.15.1:0/2671181596 shutdown_connections 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.709+0000 7f2e2d93d640 1 -- 172.21.15.1:0/2671181596 wait complete. 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.709+0000 7f2e2d93d640 1 Processor -- start 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.709+0000 7f2e2d93d640 1 -- start start 2024-09-16T00:45:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2d93d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e28100a80 0x7f2e2810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2d93d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 0x7f2e2810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2d93d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e28106a80 0x7f2e2810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2d93d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2e28111fa0 con 0x7f2e28101420 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2d93d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e28111e20 con 0x7f2e28100a80 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2d93d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e28112120 con 0x7f2e28106a80 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e1ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 0x7f2e2810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e1ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 0x7f2e2810e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41770/0 (socket says 172.21.15.1:41770) 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e1ffff640 1 -- 172.21.15.1:0/3048909708 learned_addr learned my addr 172.21.15.1:0/3048909708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e26ffd640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e28100a80 0x7f2e2810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e277fe640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e28106a80 0x7f2e2810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e1ffff640 1 -- 172.21.15.1:0/3048909708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e28106a80 msgr2=0x7f2e2810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e1ffff640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e28106a80 0x7f2e2810ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e1ffff640 1 -- 172.21.15.1:0/3048909708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e28100a80 msgr2=0x7f2e2810e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e1ffff640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e28100a80 0x7f2e2810e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e1ffff640 1 -- 172.21.15.1:0/3048909708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e281a6880 con 0x7f2e28101420 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e1ffff640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 0x7f2e2810e920 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000c920 tx=0x7f2e1000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2c93b640 1 -- 172.21.15.1:0/3048909708 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10007bb0 con 0x7f2e28101420 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2c93b640 1 -- 172.21.15.1:0/3048909708 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e10007d50 con 0x7f2e28101420 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2c93b640 1 -- 172.21.15.1:0/3048909708 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e100187d0 con 0x7f2e28101420 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e281a6b70 con 0x7f2e28101420 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.710+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e281a70b0 con 0x7f2e28101420 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.711+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e28100ee0 con 0x7f2e28101420 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.713+0000 7f2e2c93b640 1 -- 172.21.15.1:0/3048909708 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f2e1000f450 con 0x7f2e28101420 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.713+0000 7f2e2c93b640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2e140a3240 0x7f2e140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.714+0000 7f2e26ffd640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2e140a3240 0x7f2e140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.714+0000 7f2e2c93b640 1 -- 172.21.15.1:0/3048909708 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8986+0+0 (secure 0 0 0) 0x7f2e100cfc90 con 0x7f2e28101420 2024-09-16T00:45:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.714+0000 7f2e26ffd640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2e140a3240 0x7f2e140a5700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800ab60 tx=0x7f2e1800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.717+0000 7f2e2c93b640 1 -- 172.21.15.1:0/3048909708 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e10014030 con 0x7f2e28101420 2024-09-16T00:45:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:56.893+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e281a7310 con 0x7f2e28101420 2024-09-16T00:45:57.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.085+0000 7f2e2c93b640 1 -- 172.21.15.1:0/3048909708 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f2e1009c6c0 con 0x7f2e28101420 2024-09-16T00:45:57.086 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-09-16T00:45:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.089+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2e140a3240 msgr2=0x7f2e140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.089+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2e140a3240 0x7f2e140a5700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800ab60 tx=0x7f2e1800a2d0 comp rx=0 tx=0).stop 2024-09-16T00:45:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.089+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 msgr2=0x7f2e2810e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.089+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 0x7f2e2810e920 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000c920 tx=0x7f2e1000cd20 comp rx=0 tx=0).stop 2024-09-16T00:45:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.090+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 shutdown_connections 2024-09-16T00:45:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.090+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7f2e140a3240 0x7f2e140a5700 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.090+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e28106a80 0x7f2e2810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.090+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e28101420 0x7f2e2810e920 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.090+0000 7f2e2d93d640 1 --2- 172.21.15.1:0/3048909708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e28100a80 0x7f2e2810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.090+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 >> 172.21.15.1:0/3048909708 conn(0x7f2e280fc7a0 msgr2=0x7f2e280fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.090+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 shutdown_connections 2024-09-16T00:45:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:57.090+0000 7f2e2d93d640 1 -- 172.21.15.1:0/3048909708 wait complete. 2024-09-16T00:45:57.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:57 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3048909708' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:45:57.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:57 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3048909708' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:45:57.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:57 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3048909708' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:45:57.879 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-16T00:45:58.087 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:58 smithi001 ceph-mon[30934]: pgmap v879: 97 pgs: 97 active+clean; 577 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:45:58.087 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:58 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/3048909708' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:45:58.087 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:58 smithi001 ceph-mon[30934]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:45:58.144 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:58.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:58 smithi019 ceph-mon[35186]: pgmap v879: 97 pgs: 97 active+clean; 577 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:45:58.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:58 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/3048909708' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:45:58.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:58 smithi019 ceph-mon[35186]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:45:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:58 smithi153 ceph-mon[35538]: pgmap v879: 97 pgs: 97 active+clean; 577 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:45:58.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:58 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/3048909708' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:45:58.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:58 smithi153 ceph-mon[35538]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:45:58.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.686+0000 7fe5849a2640 1 -- 172.21.15.1:0/1473265065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580101de0 msgr2=0x7fe5801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:58.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.686+0000 7fe5849a2640 1 --2- 172.21.15.1:0/1473265065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580101de0 0x7fe5801021c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fe570009ca0 tx=0x7fe57002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:45:58.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.687+0000 7fe5849a2640 1 -- 172.21.15.1:0/1473265065 shutdown_connections 2024-09-16T00:45:58.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.687+0000 7fe5849a2640 1 --2- 172.21.15.1:0/1473265065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe580106f10 0x7fe58010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:58.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.687+0000 7fe5849a2640 1 --2- 172.21.15.1:0/1473265065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe580102790 0x7fe5801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:58.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.687+0000 7fe5849a2640 1 --2- 172.21.15.1:0/1473265065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580101de0 0x7fe5801021c0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:58.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.687+0000 7fe5849a2640 1 -- 172.21.15.1:0/1473265065 >> 172.21.15.1:0/1473265065 conn(0x7fe5800fd8c0 msgr2=0x7fe5800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:58.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.687+0000 7fe5849a2640 1 -- 172.21.15.1:0/1473265065 shutdown_connections 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.687+0000 7fe5849a2640 1 -- 172.21.15.1:0/1473265065 wait complete. 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe5849a2640 1 Processor -- start 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe5849a2640 1 -- start start 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe5849a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe580101de0 0x7fe580198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe5849a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580102790 0x7fe5801989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe5849a2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe580106f10 0x7fe58019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe5849a2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe580111f90 con 0x7fe580102790 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe5849a2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe580111e10 con 0x7fe580101de0 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe5849a2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe580112110 con 0x7fe580106f10 2024-09-16T00:45:58.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580102790 0x7fe5801989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580102790 0x7fe5801989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41776/0 (socket says 172.21.15.1:41776) 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57dd9b640 1 -- 172.21.15.1:0/423416346 learned_addr learned my addr 172.21.15.1:0/423416346 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57e59c640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe580101de0 0x7fe580198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57ed9d640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe580106f10 0x7fe58019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57dd9b640 1 -- 172.21.15.1:0/423416346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe580106f10 msgr2=0x7fe58019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57dd9b640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe580106f10 0x7fe58019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57dd9b640 1 -- 172.21.15.1:0/423416346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe580101de0 msgr2=0x7fe580198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57dd9b640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe580101de0 0x7fe580198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.688+0000 7fe57dd9b640 1 -- 172.21.15.1:0/423416346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58019d440 con 0x7fe580102790 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.689+0000 7fe57dd9b640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580102790 0x7fe5801989d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b4a0 tx=0x7fe56800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.689+0000 7fe5777fe640 1 -- 172.21.15.1:0/423416346 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568014070 con 0x7fe580102790 2024-09-16T00:45:58.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.689+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58019d730 con 0x7fe580102790 2024-09-16T00:45:58.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.689+0000 7fe5777fe640 1 -- 172.21.15.1:0/423416346 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe568010af0 con 0x7fe580102790 2024-09-16T00:45:58.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.689+0000 7fe5777fe640 1 -- 172.21.15.1:0/423416346 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568017d10 con 0x7fe580102790 2024-09-16T00:45:58.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.689+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5801a5010 con 0x7fe580102790 2024-09-16T00:45:58.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.690+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5801021c0 con 0x7fe580102790 2024-09-16T00:45:58.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.691+0000 7fe5777fe640 1 -- 172.21.15.1:0/423416346 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fe568017070 con 0x7fe580102790 2024-09-16T00:45:58.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.692+0000 7fe5777fe640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe56c0a3240 0x7fe56c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:58.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.692+0000 7fe5777fe640 1 -- 172.21.15.1:0/423416346 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8324+0+0 (secure 0 0 0) 0x7fe5680cedf0 con 0x7fe580102790 2024-09-16T00:45:58.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.692+0000 7fe57e59c640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe56c0a3240 0x7fe56c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:58.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.692+0000 7fe57e59c640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe56c0a3240 0x7fe56c0a5700 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe570000f80 tx=0x7fe5700093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:58.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.696+0000 7fe5777fe640 1 -- 172.21.15.1:0/423416346 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe56809bab0 con 0x7fe580102790 2024-09-16T00:45:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:58.875+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe580109b60 con 0x7fe580102790 2024-09-16T00:45:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.092+0000 7fe5777fe640 1 -- 172.21.15.1:0/423416346 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v95) ==== 178+0+0 (secure 0 0 0) 0x7fe56809b3b0 con 0x7fe580102790 2024-09-16T00:45:59.093 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' removed 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe56c0a3240 msgr2=0x7fe56c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe56c0a3240 0x7fe56c0a5700 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe570000f80 tx=0x7fe5700093f0 comp rx=0 tx=0).stop 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580102790 msgr2=0x7fe5801989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580102790 0x7fe5801989d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b4a0 tx=0x7fe56800b960 comp rx=0 tx=0).stop 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 shutdown_connections 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:6800/3146869121,v1:172.21.15.1:6801/3146869121] conn(0x7fe56c0a3240 0x7fe56c0a5700 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe580106f10 0x7fe58019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe580102790 0x7fe5801989d0 secure :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b4a0 tx=0x7fe56800b960 comp rx=0 tx=0).stop 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 --2- 172.21.15.1:0/423416346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe580101de0 0x7fe580198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 >> 172.21.15.1:0/423416346 conn(0x7fe5800fd8c0 msgr2=0x7fe5800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:59.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 shutdown_connections 2024-09-16T00:45:59.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-16T00:45:59.095+0000 7fe5849a2640 1 -- 172.21.15.1:0/423416346 wait complete. 2024-09-16T00:45:59.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:45:59 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/423416346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:45:59.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:45:59 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/423416346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:45:59.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:45:59 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/423416346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:45:59.932 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:45:59.962 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:45:59.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:46:00.031 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T00:46:00.032 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:46:00.051 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:46:00.069 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:46:00.088 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T00:46:00.089 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T00:46:00.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a 2024-09-16T00:46:00.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:00 smithi001 ceph-mon[30934]: pgmap v881: 33 pgs: 33 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:46:00.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:00 smithi001 ceph-mon[30934]: from='client.? 172.21.15.1:0/423416346' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:46:00.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:00 smithi001 ceph-mon[30934]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T00:46:00.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:00 smithi001 systemd[1]: Stopping Ceph mon.a for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:46:00 smithi019 ceph-mon[35186]: pgmap v881: 33 pgs: 33 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:46:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:46:00 smithi019 ceph-mon[35186]: from='client.? 172.21.15.1:0/423416346' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:46:00.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:46:00 smithi019 ceph-mon[35186]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T00:46:00.461 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:46:00 smithi153 ceph-mon[35538]: pgmap v881: 33 pgs: 33 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:46:00.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:46:00 smithi153 ceph-mon[35538]: from='client.? 172.21.15.1:0/423416346' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:46:00.462 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:46:00 smithi153 ceph-mon[35538]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T00:46:00.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:00 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a[30930]: 2024-09-16T00:46:00.409+0000 7f9ec3f62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:00.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:00 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a[30930]: 2024-09-16T00:46:00.409+0000 7f9ec3f62640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T00:46:00.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:00 smithi001 podman[123241]: 2024-09-16 00:46:00.643753595 +0000 UTC m=+0.346877930 container died 196e28c9c2a564bd3fb5c19349da2cadd22c4591b09986bd73004b1284b8e0c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T00:46:01.405 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:01 smithi001 podman[123241]: 2024-09-16 00:46:01.159932224 +0000 UTC m=+0.863056587 container cleanup 196e28c9c2a564bd3fb5c19349da2cadd22c4591b09986bd73004b1284b8e0c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:46:01.405 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:01 smithi001 bash[123241]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a 2024-09-16T00:46:01.649 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T00:46:01.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:01 smithi001 podman[123252]: 2024-09-16 00:46:01.405535763 +0000 UTC m=+0.758092918 container remove 196e28c9c2a564bd3fb5c19349da2cadd22c4591b09986bd73004b1284b8e0c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:46:01.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:01 smithi001 systemd[1]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T00:46:01.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:01 smithi001 systemd[1]: Stopped Ceph mon.a for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:46:01.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 16 00:46:01 smithi001 systemd[1]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.a.service: Consumed 25.589s CPU time. 2024-09-16T00:46:01.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:01.720 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T00:46:01.720 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-16T00:46:01.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.b 2024-09-16T00:46:02.057 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:46:01 smithi019 systemd[1]: Stopping Ceph mon.b for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:46:02 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-b[35182]: 2024-09-16T00:46:02.056+0000 7fae5f7b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:46:02 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-b[35182]: 2024-09-16T00:46:02.056+0000 7fae5f7b9640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:46:02.357 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:46:02 smithi019 podman[70870]: 2024-09-16 00:46:02.10817383 +0000 UTC m=+0.181346506 container died bffa471eec8ce3b504800365a2f5ffcc108819d2ce9621a6fc2f249c72f260ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T00:46:02.742 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:46:02 smithi019 podman[70870]: 2024-09-16 00:46:02.524136412 +0000 UTC m=+0.597309088 container cleanup bffa471eec8ce3b504800365a2f5ffcc108819d2ce9621a6fc2f249c72f260ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:46:02.742 INFO:journalctl@ceph.mon.b.smithi019.stdout:Sep 16 00:46:02 smithi019 bash[70870]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-b 2024-09-16T00:46:02.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T00:46:03.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:03.032 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-16T00:46:03.032 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-16T00:46:03.033 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.c 2024-09-16T00:46:03.358 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:46:03 smithi153 systemd[1]: Stopping Ceph mon.c for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:03.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:46:03 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-c[35532]: 2024-09-16T00:46:03.356+0000 7faaac892640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:03.711 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:46:03 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-c[35532]: 2024-09-16T00:46:03.356+0000 7faaac892640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:46:03.712 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:46:03 smithi153 podman[71659]: 2024-09-16 00:46:03.490819385 +0000 UTC m=+0.256996895 container died bad8b46e65d39658de276212c1ace58480a72d49f26c68d928edab5433031193 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T00:46:04.194 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:46:03 smithi153 podman[71659]: 2024-09-16 00:46:03.867453282 +0000 UTC m=+0.633630794 container cleanup bad8b46e65d39658de276212c1ace58480a72d49f26c68d928edab5433031193 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T00:46:04.194 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 16 00:46:03 smithi153 bash[71659]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mon-c 2024-09-16T00:46:04.348 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T00:46:04.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:04.397 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-16T00:46:04.397 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T00:46:04.398 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.x 2024-09-16T00:46:04.701 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:46:04 smithi001 systemd[1]: Stopping Ceph mgr.x for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:05.044 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:46:04 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:46:04.717+0000 7f6e178bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:05.044 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:46:04 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x[31163]: 2024-09-16T00:46:04.717+0000 7f6e178bf640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-16T00:46:05.044 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:46:04 smithi001 podman[123365]: 2024-09-16 00:46:04.748243447 +0000 UTC m=+0.146053640 container died 7c193014c9329b61f157c23e4b6e789a2db5013745fb73646953adbee40dc192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:46:05.451 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:46:05 smithi001 podman[123365]: 2024-09-16 00:46:05.259551497 +0000 UTC m=+0.657361690 container cleanup 7c193014c9329b61f157c23e4b6e789a2db5013745fb73646953adbee40dc192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T00:46:05.451 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 16 00:46:05 smithi001 bash[123365]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-x 2024-09-16T00:46:05.699 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T00:46:05.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:05.736 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T00:46:05.736 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T00:46:05.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.y 2024-09-16T00:46:06.083 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:46:05 smithi019 systemd[1]: Stopping Ceph mgr.y for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:06.356 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:46:06 smithi019 podman[70993]: 2024-09-16 00:46:06.083529203 +0000 UTC m=+0.143181170 container died cf88e9079cd1fd9fe310ef192d2d9d9d1863d42cb57b8349729335d56b399a6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T00:46:06.673 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:46:06 smithi019 podman[70993]: 2024-09-16 00:46:06.463308256 +0000 UTC m=+0.522960222 container cleanup cf88e9079cd1fd9fe310ef192d2d9d9d1863d42cb57b8349729335d56b399a6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:46:06.673 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Sep 16 00:46:06 smithi019 bash[70993]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-y 2024-09-16T00:46:06.895 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T00:46:06.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:06.929 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T00:46:06.930 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-16T00:46:06.930 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.z 2024-09-16T00:46:07.211 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:46:06 smithi153 systemd[1]: Stopping Ceph mgr.z for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:07.549 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:46:07 smithi153 podman[71785]: 2024-09-16 00:46:07.259812814 +0000 UTC m=+0.137604518 container died a677ab130ae587cab7201ddfeee07641340ad9046db951266e36e42f72f9b988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:46:07.924 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:46:07 smithi153 podman[71785]: 2024-09-16 00:46:07.648205213 +0000 UTC m=+0.525996917 container cleanup a677ab130ae587cab7201ddfeee07641340ad9046db951266e36e42f72f9b988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T00:46:07.924 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 16 00:46:07 smithi153 bash[71785]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mgr-z 2024-09-16T00:46:08.077 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-16T00:46:08.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:08.114 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-16T00:46:08.114 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T00:46:08.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.0 2024-09-16T00:46:08.440 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:08 smithi001 systemd[1]: Stopping Ceph osd.0 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:08.701 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:08 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0[41979]: 2024-09-16T00:46:08.439+0000 7fa012909640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:08.702 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:08 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0[41979]: 2024-09-16T00:46:08.439+0000 7fa012909640 -1 osd.0 95 *** Got signal Terminated *** 2024-09-16T00:46:08.702 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:08 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0[41979]: 2024-09-16T00:46:08.439+0000 7fa012909640 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:46:14.033 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:13 smithi001 podman[123487]: 2024-09-16 00:46:13.768591782 +0000 UTC m=+5.452147770 container died 4ee55519adc1eaea42b6b92daf880d058d013ed9cc84f9eabf546b168e717741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:46:14.441 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:14 smithi001 podman[123487]: 2024-09-16 00:46:14.132096367 +0000 UTC m=+5.815652352 container cleanup 4ee55519adc1eaea42b6b92daf880d058d013ed9cc84f9eabf546b168e717741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:46:14.442 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:14 smithi001 bash[123487]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0 2024-09-16T00:46:14.701 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:14 smithi001 podman[123502]: 2024-09-16 00:46:14.441879688 +0000 UTC m=+0.666052696 container remove 4ee55519adc1eaea42b6b92daf880d058d013ed9cc84f9eabf546b168e717741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:46:15.054 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:14 smithi001 podman[123590]: 2024-09-16 00:46:14.66864911 +0000 UTC m=+0.016516576 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:46:15.054 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:14 smithi001 podman[123590]: 2024-09-16 00:46:14.792846516 +0000 UTC m=+0.140713972 container create db54c8f900c07d6c7ae996118b296ed5632ad410bcc344b17b2c258ad59e1f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T00:46:15.451 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:15 smithi001 podman[123590]: 2024-09-16 00:46:15.05428648 +0000 UTC m=+0.402153937 container init db54c8f900c07d6c7ae996118b296ed5632ad410bcc344b17b2c258ad59e1f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T00:46:15.451 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:15 smithi001 podman[123590]: 2024-09-16 00:46:15.058146158 +0000 UTC m=+0.406013617 container start db54c8f900c07d6c7ae996118b296ed5632ad410bcc344b17b2c258ad59e1f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-16T00:46:15.452 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:15 smithi001 podman[123590]: 2024-09-16 00:46:15.137515004 +0000 UTC m=+0.485382476 container attach db54c8f900c07d6c7ae996118b296ed5632ad410bcc344b17b2c258ad59e1f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True) 2024-09-16T00:46:15.452 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 16 00:46:15 smithi001 podman[123590]: 2024-09-16 00:46:15.248836085 +0000 UTC m=+0.596703542 container died db54c8f900c07d6c7ae996118b296ed5632ad410bcc344b17b2c258ad59e1f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T00:46:15.929 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T00:46:15.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:15.971 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T00:46:15.971 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T00:46:15.971 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.1 2024-09-16T00:46:16.357 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:16 smithi019 systemd[1]: Stopping Ceph osd.1 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:16.857 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:16 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1[39671]: 2024-09-16T00:46:16.409+0000 7f22ece97640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:16.857 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:16 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1[39671]: 2024-09-16T00:46:16.409+0000 7f22ece97640 -1 osd.1 95 *** Got signal Terminated *** 2024-09-16T00:46:16.857 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:16 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1[39671]: 2024-09-16T00:46:16.409+0000 7f22ece97640 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:46:22.219 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:21 smithi019 podman[71128]: 2024-09-16 00:46:21.93415884 +0000 UTC m=+5.774258482 container died 049dbd9fb3dfc744f8823460e7b9fe47e199b074fa797357c54e0a9531d57ca4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:46:22.607 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:22 smithi019 podman[71128]: 2024-09-16 00:46:22.310514599 +0000 UTC m=+6.150614240 container cleanup 049dbd9fb3dfc744f8823460e7b9fe47e199b074fa797357c54e0a9531d57ca4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:46:22.607 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:22 smithi019 bash[71128]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1 2024-09-16T00:46:23.088 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:22 smithi019 podman[71144]: 2024-09-16 00:46:22.628535436 +0000 UTC m=+0.692418276 container remove 049dbd9fb3dfc744f8823460e7b9fe47e199b074fa797357c54e0a9531d57ca4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T00:46:23.357 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:23 smithi019 podman[71233]: 2024-09-16 00:46:22.991527725 +0000 UTC m=+0.016032294 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:46:23.357 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:23 smithi019 podman[71233]: 2024-09-16 00:46:23.112607255 +0000 UTC m=+0.137111813 container create d52144b729f7339f59ecae0386970a4d238593b253d112d53ae3908ab0cabb15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T00:46:23.857 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:23 smithi019 podman[71233]: 2024-09-16 00:46:23.390455538 +0000 UTC m=+0.414960098 container init d52144b729f7339f59ecae0386970a4d238593b253d112d53ae3908ab0cabb15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:46:23.857 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:23 smithi019 podman[71233]: 2024-09-16 00:46:23.393804701 +0000 UTC m=+0.418309268 container start d52144b729f7339f59ecae0386970a4d238593b253d112d53ae3908ab0cabb15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T00:46:23.857 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:23 smithi019 podman[71233]: 2024-09-16 00:46:23.490503292 +0000 UTC m=+0.515007866 container attach d52144b729f7339f59ecae0386970a4d238593b253d112d53ae3908ab0cabb15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T00:46:23.857 INFO:journalctl@ceph.osd.1.smithi019.stdout:Sep 16 00:46:23 smithi019 podman[71233]: 2024-09-16 00:46:23.580885688 +0000 UTC m=+0.605390247 container died d52144b729f7339f59ecae0386970a4d238593b253d112d53ae3908ab0cabb15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T00:46:24.341 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T00:46:24.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:24.384 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T00:46:24.384 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T00:46:24.384 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.2 2024-09-16T00:46:24.694 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:24 smithi153 systemd[1]: Stopping Ceph osd.2 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:24.961 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2[40494]: 2024-09-16T00:46:24.693+0000 7f3a714b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:24.962 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2[40494]: 2024-09-16T00:46:24.693+0000 7f3a714b3640 -1 osd.2 95 *** Got signal Terminated *** 2024-09-16T00:46:24.962 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2[40494]: 2024-09-16T00:46:24.693+0000 7f3a714b3640 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:46:30.453 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:30 smithi153 podman[71920]: 2024-09-16 00:46:30.193015812 +0000 UTC m=+5.612957852 container died 4aa8e5bda815481fa1e3ceb3b7bd81399500789b5a600d918f5a0a0fc2f71d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:46:30.711 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:30 smithi153 podman[71920]: 2024-09-16 00:46:30.544113635 +0000 UTC m=+5.964055664 container cleanup 4aa8e5bda815481fa1e3ceb3b7bd81399500789b5a600d918f5a0a0fc2f71d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T00:46:30.711 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:30 smithi153 bash[71920]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2 2024-09-16T00:46:31.155 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:30 smithi153 podman[71936]: 2024-09-16 00:46:30.837296894 +0000 UTC m=+0.646317361 container remove 4aa8e5bda815481fa1e3ceb3b7bd81399500789b5a600d918f5a0a0fc2f71d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T00:46:31.432 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:31 smithi153 podman[72023]: 2024-09-16 00:46:31.058563751 +0000 UTC m=+0.016326361 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:46:31.432 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:31 smithi153 podman[72023]: 2024-09-16 00:46:31.179970072 +0000 UTC m=+0.137732660 container create 38f62a6f76124295ff7383d43e292a833e9fd03771dcfacc0472ffd0b59e66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T00:46:31.712 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:31 smithi153 podman[72023]: 2024-09-16 00:46:31.432762374 +0000 UTC m=+0.390524972 container init 38f62a6f76124295ff7383d43e292a833e9fd03771dcfacc0472ffd0b59e66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:46:31.712 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:31 smithi153 podman[72023]: 2024-09-16 00:46:31.436585422 +0000 UTC m=+0.394348013 container start 38f62a6f76124295ff7383d43e292a833e9fd03771dcfacc0472ffd0b59e66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T00:46:31.712 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:31 smithi153 podman[72023]: 2024-09-16 00:46:31.558111609 +0000 UTC m=+0.515874199 container attach 38f62a6f76124295ff7383d43e292a833e9fd03771dcfacc0472ffd0b59e66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:46:31.712 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 16 00:46:31 smithi153 podman[72023]: 2024-09-16 00:46:31.632962342 +0000 UTC m=+0.590724940 container died 38f62a6f76124295ff7383d43e292a833e9fd03771dcfacc0472ffd0b59e66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T00:46:32.265 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T00:46:32.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:32.305 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T00:46:32.305 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T00:46:32.305 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.3 2024-09-16T00:46:32.619 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:32 smithi001 systemd[1]: Stopping Ceph osd.3 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:32.925 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:32 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3[48428]: 2024-09-16T00:46:32.617+0000 7fac2b32e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:32.926 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:32 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3[48428]: 2024-09-16T00:46:32.617+0000 7fac2b32e640 -1 osd.3 95 *** Got signal Terminated *** 2024-09-16T00:46:32.926 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:32 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3[48428]: 2024-09-16T00:46:32.617+0000 7fac2b32e640 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:46:34.961 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:34 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:34.636+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:35.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:34 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:34.736+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:35.961 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:35.615+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:36.106 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:35 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:35.753+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:36.961 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:36 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:36.598+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:37.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:36 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:36.730+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:37.961 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:37.598+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:37.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:37.785+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:38.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:37 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:37.730+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:38.253 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:37 smithi001 podman[123694]: 2024-09-16 00:46:37.99747125 +0000 UTC m=+5.485628772 container died e24288000a4a8da4f6760140c2312fb16d2859a7a7f313f8ca6691b0c070444c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:46:38.645 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:38 smithi001 podman[123694]: 2024-09-16 00:46:38.427333026 +0000 UTC m=+5.915490540 container cleanup e24288000a4a8da4f6760140c2312fb16d2859a7a7f313f8ca6691b0c070444c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T00:46:38.645 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:38 smithi001 bash[123694]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3 2024-09-16T00:46:38.831 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:38.572+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:38.832 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:38.681+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:38.951 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:38 smithi001 podman[123721]: 2024-09-16 00:46:38.645511944 +0000 UTC m=+0.640521281 container remove e24288000a4a8da4f6760140c2312fb16d2859a7a7f313f8ca6691b0c070444c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:46:39.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:38 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:38.749+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:39.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:38.831+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:39.415 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:39 smithi001 podman[123808]: 2024-09-16 00:46:39.000203624 +0000 UTC m=+0.016340576 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:46:39.415 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:39 smithi001 podman[123808]: 2024-09-16 00:46:39.121596305 +0000 UTC m=+0.137733242 container create ebd5075b0ef3e1468f665adde26a5d50f1fc2980f76f64e18a40720aac6ca256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:46:39.702 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:39 smithi001 podman[123808]: 2024-09-16 00:46:39.415986648 +0000 UTC m=+0.432123586 container init ebd5075b0ef3e1468f665adde26a5d50f1fc2980f76f64e18a40720aac6ca256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T00:46:39.702 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:39 smithi001 podman[123808]: 2024-09-16 00:46:39.419493336 +0000 UTC m=+0.435630271 container start ebd5075b0ef3e1468f665adde26a5d50f1fc2980f76f64e18a40720aac6ca256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:46:39.702 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:39 smithi001 podman[123808]: 2024-09-16 00:46:39.524380041 +0000 UTC m=+0.540516986 container attach ebd5075b0ef3e1468f665adde26a5d50f1fc2980f76f64e18a40720aac6ca256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-16T00:46:39.702 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 16 00:46:39 smithi001 podman[123808]: 2024-09-16 00:46:39.620388262 +0000 UTC m=+0.636525190 container died ebd5075b0ef3e1468f665adde26a5d50f1fc2980f76f64e18a40720aac6ca256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-16T00:46:39.814 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:39.543+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:39.814 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:39.715+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:40.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:39 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:39.797+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:40.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:39.813+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:40.346 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T00:46:40.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:40.395 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T00:46:40.395 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T00:46:40.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.4 2024-09-16T00:46:40.714 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:40 smithi019 systemd[1]: Stopping Ceph osd.4 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:40.961 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:40.544+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:40.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:40.792+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:40.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:40.677+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:41.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:40 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:40.824+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:41.107 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:40 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4[46243]: 2024-09-16T00:46:40.712+0000 7ff8920bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:41.107 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:40 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4[46243]: 2024-09-16T00:46:40.712+0000 7ff8920bf640 -1 osd.4 95 *** Got signal Terminated *** 2024-09-16T00:46:41.107 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:40 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4[46243]: 2024-09-16T00:46:40.712+0000 7ff8920bf640 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:46:41.961 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:41 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:41.574+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:41.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:41 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:41.773+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:41.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:41 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:41.634+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:42.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:41 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:41.785+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:42.962 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:42 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:42.549+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:42.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:42 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:42.774+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:42.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:42 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:42.592+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:43.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:42 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:42.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:43.607 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:43 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4[46243]: 2024-09-16T00:46:43.144+0000 7ff88e6d5640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:17.765248+0000 front 2024-09-16T00:46:17.765186+0000 (oldest deadline 2024-09-16T00:46:42.465410+0000) 2024-09-16T00:46:43.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:43 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:43.437+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:43.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:43 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:43.616+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:43.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:43 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:43.802+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:43.962 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:43 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:43.586+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:43.962 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:43 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:43.586+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:44.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:43 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:43.751+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:44.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:43 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:43.751+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:44.607 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:44 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4[46243]: 2024-09-16T00:46:44.140+0000 7ff88e6d5640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:17.765248+0000 front 2024-09-16T00:46:17.765186+0000 (oldest deadline 2024-09-16T00:46:42.465410+0000) 2024-09-16T00:46:44.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:44 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:44.392+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:44.832 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:44 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:44.552+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:44.832 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:44 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:44.552+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:44.832 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:44 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:44.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:44.832 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:44 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:44.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:45.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:44 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:44.720+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:45.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:44 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:44.720+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:45.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:44 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:44.832+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:45.607 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:45 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4[46243]: 2024-09-16T00:46:45.134+0000 7ff88e6d5640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:17.765248+0000 front 2024-09-16T00:46:17.765186+0000 (oldest deadline 2024-09-16T00:46:42.465410+0000) 2024-09-16T00:46:45.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:45 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:45.419+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:45.876 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:45 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:45.579+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:45.876 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:45 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:45.579+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:45.877 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:45 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:45.559+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:45.877 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:45 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:45.559+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:46.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:45 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:45.735+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:46.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:45 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:45.735+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:46.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:45 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:45.876+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:46.382 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:46 smithi019 podman[71339]: 2024-09-16 00:46:46.117782025 +0000 UTC m=+5.524984937 container died 396246fa8bd7c374b6a4f196d10ba9b78883ac38f688c1ac188f8a79bee206f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:46:46.690 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:46 smithi019 podman[71339]: 2024-09-16 00:46:46.481158558 +0000 UTC m=+5.888361470 container cleanup 396246fa8bd7c374b6a4f196d10ba9b78883ac38f688c1ac188f8a79bee206f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:46:46.690 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:46 smithi019 bash[71339]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4 2024-09-16T00:46:46.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:46 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:46.389+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:46.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:46 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:46.269+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:46.842 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:46 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:46.559+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:46.842 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:46 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:46.559+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:46.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:46 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:46.605+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:46.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:46 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:46.605+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:47.073 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:46 smithi019 podman[71355]: 2024-09-16 00:46:46.724239058 +0000 UTC m=+0.603094561 container remove 396246fa8bd7c374b6a4f196d10ba9b78883ac38f688c1ac188f8a79bee206f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T00:46:47.073 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:46 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:46.689+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:47.073 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:46 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:46.689+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:47.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:46 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:46.842+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:47.335 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:47 smithi019 podman[71452]: 2024-09-16 00:46:46.976218412 +0000 UTC m=+0.015910852 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:46:47.335 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:47 smithi019 podman[71452]: 2024-09-16 00:46:47.074945951 +0000 UTC m=+0.114638393 container create 28a7c9af5246601bb5de1a10e759923f32a852df44fd0e38179208398ed01845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:46:47.607 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:47 smithi019 podman[71452]: 2024-09-16 00:46:47.335819886 +0000 UTC m=+0.375512326 container init 28a7c9af5246601bb5de1a10e759923f32a852df44fd0e38179208398ed01845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:46:47.607 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:47 smithi019 podman[71452]: 2024-09-16 00:46:47.339466714 +0000 UTC m=+0.379159156 container start 28a7c9af5246601bb5de1a10e759923f32a852df44fd0e38179208398ed01845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T00:46:47.607 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:47 smithi019 podman[71452]: 2024-09-16 00:46:47.410906866 +0000 UTC m=+0.450599310 container attach 28a7c9af5246601bb5de1a10e759923f32a852df44fd0e38179208398ed01845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T00:46:47.607 INFO:journalctl@ceph.osd.4.smithi019.stdout:Sep 16 00:46:47 smithi019 podman[71452]: 2024-09-16 00:46:47.531614786 +0000 UTC m=+0.571307227 container died 28a7c9af5246601bb5de1a10e759923f32a852df44fd0e38179208398ed01845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T00:46:47.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:47 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:47.431+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:47.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:47 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:47.317+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:47.850 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:47 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:47.608+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:47.850 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:47 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:47.608+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:47.850 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:47 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:47.531+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:47.850 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:47 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:47.531+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:48.064 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:47 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:47.864+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:48.064 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:47 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:47.702+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:48.065 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:47 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:47.702+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:48.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T00:46:48.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:48.128 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T00:46:48.128 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T00:46:48.128 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.5 2024-09-16T00:46:48.155 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:47 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:47.849+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:48.448 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:48 smithi153 systemd[1]: Stopping Ceph osd.5 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:48.704 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:48.386+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:48.705 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:48 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:48.287+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:48.711 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:48 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:48.658+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:48.711 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:48 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:48.658+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:48.711 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:48 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:48.447+0000 7f0c28def640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:48.712 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:48 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:48.447+0000 7f0c28def640 -1 osd.5 95 *** Got signal Terminated *** 2024-09-16T00:46:48.712 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:48 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:48.447+0000 7f0c28def640 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:46:48.712 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:48 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:48.528+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:48.712 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:48 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:48.528+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:49.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:48 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:48.906+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:49.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:48 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:48.742+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:49.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:48 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:48.742+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:49.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:48 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:48.883+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:49.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:49 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:49.373+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:49.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:49 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:49.310+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:49.914 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:49 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:49.680+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:49.914 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:49 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:49.680+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:49.915 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:49 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:49.554+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:49.915 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:49 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:49.554+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:50.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:49 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:49.858+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:50.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:49 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:49.712+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:50.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:49 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:49.712+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:50.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:49 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:49.914+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:50.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:50 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:50.391+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:50.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:50 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:50.277+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:50.885 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:50.599+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:50.885 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:50.599+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:50.885 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:50.701+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:50.885 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:50.701+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:50.885 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:50.701+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:46:51.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:50.815+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:51.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:50.750+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:51.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:50 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:50.750+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:51.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:50.885+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:51.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:50 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:50.885+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:46:51.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:51 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:51.401+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:51.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:51 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:51.401+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:46:51.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:51 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:51.245+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:51.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:51 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:51.245+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:46:51.866 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:51 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:51.673+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:51.866 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:51 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:51.673+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:51.866 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:51 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:51.673+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:46:51.866 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:51 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:51.609+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:51.867 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:51 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:51.609+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:51.867 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:51 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:51.609+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:28.618671+0000 front 2024-09-16T00:46:28.618647+0000 (oldest deadline 2024-09-16T00:46:51.518761+0000) 2024-09-16T00:46:52.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:51 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:51.773+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:52.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:51 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:51.739+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:52.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:51 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:51.739+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:52.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:51 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:51.739+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:46:52.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:51 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:51.865+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:52.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:51 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:51.865+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:46:52.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:52 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:52.423+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:52.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:52 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:52.423+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:46:52.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:52 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:52.251+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:52.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:52 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:52.251+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:46:52.870 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:52 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:52.563+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.216619+0000 front 2024-09-16T00:46:12.216892+0000 (oldest deadline 2024-09-16T00:46:33.916709+0000) 2024-09-16T00:46:52.870 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:52 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:52.563+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.217791+0000 front 2024-09-16T00:46:19.217766+0000 (oldest deadline 2024-09-16T00:46:42.717822+0000) 2024-09-16T00:46:52.870 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:52 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5[47262]: 2024-09-16T00:46:52.563+0000 7f0c213fd640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:28.618671+0000 front 2024-09-16T00:46:28.618647+0000 (oldest deadline 2024-09-16T00:46:51.518761+0000) 2024-09-16T00:46:52.870 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:52 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:52.700+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:52.870 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:52 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:52.700+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:52.870 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:52 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:52.700+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:46:53.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:52 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:52.817+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:53.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:52 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:52.817+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:46:53.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:52 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:52.712+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:53.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:52 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:52.712+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:53.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:52 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:52.712+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:46:53.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:52 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:52.870+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:53.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:52 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:52.870+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:46:53.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:53 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:53.378+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:53.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:53 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:53.378+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:46:53.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:53 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:53.227+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:53.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:53 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:53.227+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:46:53.961 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:53 smithi153 podman[72138]: 2024-09-16 00:46:53.838555805 +0000 UTC m=+5.504162736 container died e9a1a8ac2eff58f65175889fb4aff27b238398f6099996a5647947df66338e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T00:46:53.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:53 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:53.865+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:53.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:53 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:53.865+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:46:53.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:53 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:53.668+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:53.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:53 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:53.668+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:53.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:53 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:53.668+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:46:54.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:53 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:53.821+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:54.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:53 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:53.821+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:46:54.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:53 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:53.722+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:54.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:53 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:53.722+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:54.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:53 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:53.722+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:46:54.461 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:54 smithi153 podman[72138]: 2024-09-16 00:46:54.222992966 +0000 UTC m=+5.888599889 container cleanup e9a1a8ac2eff58f65175889fb4aff27b238398f6099996a5647947df66338e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:46:54.462 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:54 smithi153 bash[72138]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5 2024-09-16T00:46:54.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:54 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:54.366+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:54.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:54 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:54.366+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:46:54.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:54 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:54.231+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:54.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:54 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:54.231+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:46:54.842 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:54 smithi153 podman[72154]: 2024-09-16 00:46:54.507974858 +0000 UTC m=+0.665676125 container remove e9a1a8ac2eff58f65175889fb4aff27b238398f6099996a5647947df66338e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T00:46:54.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:54 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:54.648+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:54.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:54 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:54.649+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:54.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:54 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:54.649+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:46:55.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:54 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:54.781+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:55.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:54 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:54.781+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:46:55.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:54 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:54.736+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:55.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:54 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:54.736+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:55.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:54 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:54.736+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:46:55.152 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:54 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:54.883+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:55.153 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:54 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:54.883+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:46:55.153 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:54 smithi153 podman[72240]: 2024-09-16 00:46:54.74565301 +0000 UTC m=+0.015798516 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:46:55.153 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:54 smithi153 podman[72240]: 2024-09-16 00:46:54.883945641 +0000 UTC m=+0.154091141 container create 54ce68e3a1724af5161405211a399f10c2a58f9591549526bb92d3fb283e7c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:46:55.153 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:55 smithi153 podman[72240]: 2024-09-16 00:46:55.153149255 +0000 UTC m=+0.423294765 container init 54ce68e3a1724af5161405211a399f10c2a58f9591549526bb92d3fb283e7c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T00:46:55.461 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:55 smithi153 podman[72240]: 2024-09-16 00:46:55.156449545 +0000 UTC m=+0.426595045 container start 54ce68e3a1724af5161405211a399f10c2a58f9591549526bb92d3fb283e7c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:46:55.462 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:55 smithi153 podman[72240]: 2024-09-16 00:46:55.244807251 +0000 UTC m=+0.514952750 container attach 54ce68e3a1724af5161405211a399f10c2a58f9591549526bb92d3fb283e7c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:46:55.462 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 16 00:46:55 smithi153 podman[72240]: 2024-09-16 00:46:55.345323039 +0000 UTC m=+0.615468548 container died 54ce68e3a1724af5161405211a399f10c2a58f9591549526bb92d3fb283e7c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T00:46:55.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:55.382+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:55.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:55.382+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:46:55.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:55.261+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:55.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:55.261+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:46:55.890 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:55 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:55.603+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:55.891 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:55 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:55.603+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:55.891 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:55 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:55.603+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:46:55.946 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T00:46:56.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:46:56.026 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T00:46:56.026 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T00:46:56.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.6 2024-09-16T00:46:56.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:55 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:55.784+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:56.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:55 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:55.784+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:46:56.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:55 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:55.700+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:56.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:55 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:55.700+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:56.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:55 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:55.700+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:46:56.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:55 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:55.889+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:56.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:55 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:55.889+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:46:56.339 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:56 smithi001 systemd[1]: Stopping Ceph osd.6 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:46:56.340 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:56.284+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:56.340 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:56.284+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:46:56.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:56.339+0000 7fd765edb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:46:56.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:56.339+0000 7fd765edb640 -1 osd.6 95 *** Got signal Terminated *** 2024-09-16T00:46:56.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:56.339+0000 7fd765edb640 -1 osd.6 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:46:56.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:56.414+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:56.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:56 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:56.414+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:46:56.864 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:56 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:56.588+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:56.864 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:56 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:56.588+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:56.864 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:56 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:56.588+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:46:57.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:56 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:56.811+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:57.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:56 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:56.811+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:46:57.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:56 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:56.659+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:57.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:56 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:56.659+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:57.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:56 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:56.659+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:46:57.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:56 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:56.860+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:57.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:56 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:56.860+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:46:57.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:57 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:57.408+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:57.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:57 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:57.408+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:46:57.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:57 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:57.285+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:57.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:57 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:57.285+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:46:57.779 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:57 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:57.644+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:57.779 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:57 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:57.644+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:57.779 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:57 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:57.644+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:46:57.857 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:57 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:57.578+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:57.858 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:57 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:57.578+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:57.858 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:57 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:57.578+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:46:58.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:57 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:57.778+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:58.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:57 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:57.778+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:46:58.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:57 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:57.856+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:58.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:57 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:57.856+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:46:58.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:58 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:58.429+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:58.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:58 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:58.429+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:46:58.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:58 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:58.290+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:58.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:58 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:58.290+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:46:58.829 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:58 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:58.554+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:58.829 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:58 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:58.554+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:58.829 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:58 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:58.554+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:46:59.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:58 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:58.756+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:46:59.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:58 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:58.756+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:46:59.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:58 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:58.633+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:46:59.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:58 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:58.633+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:46:59.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:58 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:58.633+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:46:59.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:58 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:58.828+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:46:59.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:58 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:58.828+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:46:59.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:59.411+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:46:59.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:46:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:46:59.411+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:46:59.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:59.325+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:46:59.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:46:59 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:46:59.325+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:46:59.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:59 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:59.530+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:46:59.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:59 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:59.530+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:46:59.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:46:59 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:46:59.530+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:00.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:59 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:59.762+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:00.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:46:59 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:46:59.762+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:00.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:59 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:59.626+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:00.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:59 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:59.626+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:00.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:46:59 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:46:59.626+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:00.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:59 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:59.842+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:00.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:59 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:59.842+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:00.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:46:59 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:46:59.842+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:00.701 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:00 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:47:00.452+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:18.378678+0000 front 2024-09-16T00:46:18.378745+0000 (oldest deadline 2024-09-16T00:46:43.078506+0000) 2024-09-16T00:47:00.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:00 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6[54914]: 2024-09-16T00:47:00.452+0000 7fd75e4e9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.180764+0000 front 2024-09-16T00:46:25.181054+0000 (oldest deadline 2024-09-16T00:46:51.080872+0000) 2024-09-16T00:47:00.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:00 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:00.282+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:00.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:00 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:00.282+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:00.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:00 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:00.800+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:00.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:00 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:00.800+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:00.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:00 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:00.800+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.820294+0000 front 2024-09-16T00:46:34.820452+0000 (oldest deadline 2024-09-16T00:47:00.120228+0000) 2024-09-16T00:47:00.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:00 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:00.583+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:00.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:00 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:00.583+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:00.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:00 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:00.583+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:00.879 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:00 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:00.520+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:00.879 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:00 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:00.520+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:00.879 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:00 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:00.520+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:01.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:00 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:00.878+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:01.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:00 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:00.878+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:01.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:00 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:00.878+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:01.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:01 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:01.268+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:01.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:01 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:01.268+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:01.836 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:01 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:01.561+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:01.836 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:01 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:01.561+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:01.837 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:01 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:01.561+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:02.106 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:01 smithi001 podman[123909]: 2024-09-16 00:47:01.825001878 +0000 UTC m=+5.596471433 container died 48718c9375da236acf1c718f718bb66625f47c5b910f5727fb18843c30bbb280 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-8921c8e) 2024-09-16T00:47:02.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:01 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:01.777+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:02.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:01 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:01.777+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:02.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:01 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:01.777+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.820294+0000 front 2024-09-16T00:46:34.820452+0000 (oldest deadline 2024-09-16T00:47:00.120228+0000) 2024-09-16T00:47:02.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:01 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:01.612+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:02.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:01 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:01.612+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:02.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:01 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:01.612+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:02.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:01 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:01.836+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:02.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:01 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:01.836+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:02.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:01 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:01.836+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:02.451 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:02 smithi001 podman[123909]: 2024-09-16 00:47:02.298224709 +0000 UTC m=+6.069694265 container cleanup 48718c9375da236acf1c718f718bb66625f47c5b910f5727fb18843c30bbb280 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T00:47:02.451 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:02 smithi001 bash[123909]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6 2024-09-16T00:47:02.452 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:02 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:02.243+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:02.452 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:02 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:02.243+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:02.842 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:02 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:02.524+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:02.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:02 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:02.524+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:02.843 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:02 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:02.524+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:02.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:02 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:02.799+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:02.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:02 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:02.799+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:02.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:02 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:02.799+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.820294+0000 front 2024-09-16T00:46:34.820452+0000 (oldest deadline 2024-09-16T00:47:00.120228+0000) 2024-09-16T00:47:02.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:02 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:02.594+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:02.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:02 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:02.594+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:02.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:02 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:02.594+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:02.858 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:02 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:02.594+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:02.929 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:02 smithi001 podman[123925]: 2024-09-16 00:47:02.566358376 +0000 UTC m=+0.741515018 container remove 48718c9375da236acf1c718f718bb66625f47c5b910f5727fb18843c30bbb280 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T00:47:03.201 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:02 smithi001 podman[124015]: 2024-09-16 00:47:02.832182882 +0000 UTC m=+0.016657847 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:47:03.202 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:02 smithi001 podman[124015]: 2024-09-16 00:47:02.95050594 +0000 UTC m=+0.134980890 container create e9d94591addf2c6d83a4392c600fc6c493e8eefaadfe68610c459a3a62ce50ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T00:47:03.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:02 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:02.842+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:03.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:02 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:02.842+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:03.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:02 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:02.842+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:03.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:03.254+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:03.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:03.254+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:03.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:03 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:03.254+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:03.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:03 smithi001 podman[124015]: 2024-09-16 00:47:03.227735749 +0000 UTC m=+0.412210700 container init e9d94591addf2c6d83a4392c600fc6c493e8eefaadfe68610c459a3a62ce50ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:47:03.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:03 smithi001 podman[124015]: 2024-09-16 00:47:03.23173737 +0000 UTC m=+0.416212330 container start e9d94591addf2c6d83a4392c600fc6c493e8eefaadfe68610c459a3a62ce50ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T00:47:03.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:03 smithi001 podman[124015]: 2024-09-16 00:47:03.294484816 +0000 UTC m=+0.478959767 container attach e9d94591addf2c6d83a4392c600fc6c493e8eefaadfe68610c459a3a62ce50ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T00:47:03.702 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 16 00:47:03 smithi001 podman[124015]: 2024-09-16 00:47:03.42972179 +0000 UTC m=+0.614196747 container died e9d94591addf2c6d83a4392c600fc6c493e8eefaadfe68610c459a3a62ce50ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T00:47:03.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:03.832+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:03.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:03.832+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:03.857 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:03.832+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.820294+0000 front 2024-09-16T00:46:34.820452+0000 (oldest deadline 2024-09-16T00:47:00.120228+0000) 2024-09-16T00:47:03.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:03.602+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:03.858 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:03.602+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:03.858 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:03.602+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:03.858 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:03.602+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:03.874 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:03 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:03.571+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:03.874 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:03 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:03.571+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:03.874 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:03 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:03.571+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:04.087 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T00:47:04.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:04.131 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T00:47:04.132 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T00:47:04.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.7 2024-09-16T00:47:04.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:03 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:03.873+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:04.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:03 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:03.873+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:04.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:03 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:03.873+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:04.356 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:04 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:04.266+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:04.357 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:04 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:04.266+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:04.357 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:04 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:04.266+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:04.450 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:04 smithi019 systemd[1]: Stopping Ceph osd.7 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:04.804 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:04.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:04.804 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:04.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:04.804 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:04.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:04.804 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:04.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:04.805 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:04.450+0000 7f9250333640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:04.805 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:04.450+0000 7f9250333640 -1 osd.7 95 *** Got signal Terminated *** 2024-09-16T00:47:04.805 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:04.450+0000 7f9250333640 -1 osd.7 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:47:04.805 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:04.803+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:04.805 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:04.803+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:04.805 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:04 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:04.803+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.820294+0000 front 2024-09-16T00:46:34.820452+0000 (oldest deadline 2024-09-16T00:47:00.120228+0000) 2024-09-16T00:47:04.865 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:04 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:04.525+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:04.865 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:04 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:04.525+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:04.866 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:04 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:04.525+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:05.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:04 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:04.865+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:05.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:04 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:04.865+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:05.212 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:04 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:04.865+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:05.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:05.246+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:05.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:05.246+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:05.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:05 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:05.246+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:05.840 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:05 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:05.574+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:05.840 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:05 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:05.574+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:05.840 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:05 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:05.574+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:06.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:05 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:05.804+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:06.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:05 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:05.804+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:06.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:05 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:05.804+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.820294+0000 front 2024-09-16T00:46:34.820452+0000 (oldest deadline 2024-09-16T00:47:00.120228+0000) 2024-09-16T00:47:06.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:05 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:05.611+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:06.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:05 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:05.611+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:06.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:05 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:05.611+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:06.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:05 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:05.611+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:06.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:05 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:05.839+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:06.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:05 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:05.839+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:06.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:05 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:05.839+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:06.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:06 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:06.204+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:06.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:06 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:06.204+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:06.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:06 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:06.204+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:06.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:06 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:06.790+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:06.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:06 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:06.790+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:06.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:06 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:06.790+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:06.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:06 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:06.588+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:06.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:06 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:06.588+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:06.963 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:06 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:06.588+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:07.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:06 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:06.805+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:07.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:06 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:06.805+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:07.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:06 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:06.805+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.820294+0000 front 2024-09-16T00:46:34.820452+0000 (oldest deadline 2024-09-16T00:47:00.120228+0000) 2024-09-16T00:47:07.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:06 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:06.638+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:07.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:06 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:06.638+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:07.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:06 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:06.638+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:07.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:06 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:06.638+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:07.451 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:07 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:07.172+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:07.451 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:07 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:07.172+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:07.451 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:07 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:07.172+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:07.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:07 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:07.596+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:07.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:07 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:07.596+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:07.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:07 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:07.596+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:07.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:07 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:07.788+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:07.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:07 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:07.788+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:07.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:07 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:07.788+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:07.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:07 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:07.788+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:08.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:07.815+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:08.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:07.815+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:08.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:07.815+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.820294+0000 front 2024-09-16T00:46:34.820452+0000 (oldest deadline 2024-09-16T00:47:00.120228+0000) 2024-09-16T00:47:08.108 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:07.815+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.721314+0000 front 2024-09-16T00:46:44.721353+0000 (oldest deadline 2024-09-16T00:47:07.621441+0000) 2024-09-16T00:47:08.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:07.660+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:08.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:07.660+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:08.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:07.660+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:08.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:07.660+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:08.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:07.660+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:08.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:08 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:08.215+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:08.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:08 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:08.215+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:08.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:08 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:08.215+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:08.876 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:08 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:08.622+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:08.877 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:08 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:08.622+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:08.877 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:08 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:08.622+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:08.877 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:08 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:08.622+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:08.877 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:08 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:08.835+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:08.877 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:08 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:08.835+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:08.877 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:08 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:08.835+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:08.877 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:08 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:08.835+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:09.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:08 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:08.766+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:19.216874+0000 front 2024-09-16T00:46:19.216904+0000 (oldest deadline 2024-09-16T00:46:47.417639+0000) 2024-09-16T00:47:09.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:08 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:08.766+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:27.918937+0000 front 2024-09-16T00:46:27.919182+0000 (oldest deadline 2024-09-16T00:46:52.018792+0000) 2024-09-16T00:47:09.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:08 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:08.766+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.820294+0000 front 2024-09-16T00:46:34.820452+0000 (oldest deadline 2024-09-16T00:47:00.120228+0000) 2024-09-16T00:47:09.107 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:08 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7[53157]: 2024-09-16T00:47:08.766+0000 7f9248941640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.721314+0000 front 2024-09-16T00:46:44.721353+0000 (oldest deadline 2024-09-16T00:47:07.621441+0000) 2024-09-16T00:47:09.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:08 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:08.695+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:09.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:08 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:08.695+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:09.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:08 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:08.695+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:09.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:08 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:08.695+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:09.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:08 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:08.695+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:09.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:09 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:09.247+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:09.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:09 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:09.247+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:09.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:09 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:09.247+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:09.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:09 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:09.247+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:09.961 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:09.796+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:09.961 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:09.796+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:09.961 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:09.796+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:09.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:09.796+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:09.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:09.646+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:09.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:09.646+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:09.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:09.646+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:09.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:09 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:09.646+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:10.085 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:09 smithi019 podman[71554]: 2024-09-16 00:47:09.797046902 +0000 UTC m=+5.463183528 container died cf3867af6ca64c6b6036e99ed90f3fd95ddf9ee5c99cfcebc482bc45dcc37d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:47:10.085 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:09 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:09.745+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:10.085 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:09 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:09.745+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:10.085 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:09 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:09.745+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:10.086 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:09 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:09.745+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:10.086 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:09 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:09.745+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:10.357 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:10 smithi019 podman[71554]: 2024-09-16 00:47:10.183774524 +0000 UTC m=+5.849911137 container cleanup cf3867af6ca64c6b6036e99ed90f3fd95ddf9ee5c99cfcebc482bc45dcc37d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-16T00:47:10.357 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:10 smithi019 bash[71554]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7 2024-09-16T00:47:10.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:10 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:10.227+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:10.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:10 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:10.227+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:10.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:10 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:10.227+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:10.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:10 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:10.227+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:10.758 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:10 smithi019 podman[71569]: 2024-09-16 00:47:10.443145535 +0000 UTC m=+0.641504887 container remove cf3867af6ca64c6b6036e99ed90f3fd95ddf9ee5c99cfcebc482bc45dcc37d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T00:47:10.961 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:10 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:10.785+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:10.961 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:10 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:10.785+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:10.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:10 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:10.785+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:10.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:10 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:10.785+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:10.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:10 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:10.600+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:10.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:10 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:10.600+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:10.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:10 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:10.600+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:10.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:10 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:10.600+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:11.030 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:10 smithi019 podman[71659]: 2024-09-16 00:47:10.661304902 +0000 UTC m=+0.015870465 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:47:11.030 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:10 smithi019 podman[71659]: 2024-09-16 00:47:10.794254734 +0000 UTC m=+0.148820281 container create df6c005857a59b1a6e53be413930587aa3bd7ba8d6962fb456b59b7b6f2ad8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:47:11.030 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:11 smithi019 podman[71659]: 2024-09-16 00:47:11.030184786 +0000 UTC m=+0.384750348 container init df6c005857a59b1a6e53be413930587aa3bd7ba8d6962fb456b59b7b6f2ad8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T00:47:11.030 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:10 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:10.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:11.030 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:10 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:10.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:11.030 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:10 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:10.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:11.030 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:10 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:10.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:11.031 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:10 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:10.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:11.357 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:11 smithi019 podman[71659]: 2024-09-16 00:47:11.033990741 +0000 UTC m=+0.388556289 container start df6c005857a59b1a6e53be413930587aa3bd7ba8d6962fb456b59b7b6f2ad8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T00:47:11.357 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:11 smithi019 podman[71659]: 2024-09-16 00:47:11.147218456 +0000 UTC m=+0.501784003 container attach df6c005857a59b1a6e53be413930587aa3bd7ba8d6962fb456b59b7b6f2ad8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T00:47:11.357 INFO:journalctl@ceph.osd.7.smithi019.stdout:Sep 16 00:47:11 smithi019 podman[71659]: 2024-09-16 00:47:11.224279561 +0000 UTC m=+0.578845119 container died df6c005857a59b1a6e53be413930587aa3bd7ba8d6962fb456b59b7b6f2ad8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:47:11.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:11 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:11.259+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:11.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:11 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:11.259+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:11.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:11 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:11.259+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:11.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:11 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:11.259+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:11.894 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T00:47:11.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:11.932 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T00:47:11.932 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-16T00:47:11.932 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.8 2024-09-16T00:47:11.960 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:11 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:11.754+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:11.960 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:11 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:11.754+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:11.960 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:11 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:11.754+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:11.960 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:11 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:11.754+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:11.960 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:11 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:11.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:11.960 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:11 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:11.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:11.960 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:11 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:11.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:11.960 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:11 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:11.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:12.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:11 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:11.781+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:12.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:11 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:11.781+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:12.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:11 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:11.781+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:12.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:11 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:11.781+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:12.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:11 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:11.781+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:12.211 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:11 smithi153 systemd[1]: Stopping Ceph osd.8 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:12.547 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:12.259+0000 7fe3c809a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:12.547 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:12.259+0000 7fe3c809a640 -1 osd.8 95 *** Got signal Terminated *** 2024-09-16T00:47:12.547 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:12.259+0000 7fe3c809a640 -1 osd.8 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:47:12.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:12 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:12.300+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:12.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:12 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:12.300+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:12.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:12 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:12.300+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:12.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:12 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:12.300+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:12.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:12 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:12.736+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:12.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:12 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:12.736+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:12.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:12 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:12.736+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:12.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:12 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:12.736+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:12.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:12 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:12.736+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:12.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:12.774+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:12.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:12.774+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:12.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:12.774+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:12.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:12.774+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:12.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:12.546+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:12.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:12.546+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:12.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:12.546+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:12.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:12 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:12.546+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:13.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:13 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:13.264+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:13.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:13 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:13.264+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:13.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:13 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:13.264+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:13.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:13 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:13.264+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:13.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:13 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:13.741+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:13.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:13 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:13.741+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:13.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:13 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:13.741+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:13.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:13 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:13.741+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:13.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:13 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:13.547+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:13.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:13 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:13.547+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:13.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:13 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:13.547+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:13.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:13 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:13.547+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:14.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:13 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:13.772+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:14.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:13 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:13.772+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:14.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:13 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:13.772+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:14.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:13 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:13.772+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:14.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:13 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:13.772+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:14.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:14 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:14.278+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:14.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:14 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:14.278+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:14.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:14 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:14.278+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:14.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:14 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:14.278+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:14.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:14 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:14.744+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:14.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:14 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:14.744+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:14.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:14 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:14.744+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:14.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:14 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:14.744+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:14.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:14 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:14.579+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:14.963 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:14 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:14.579+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:14.963 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:14 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:14.579+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:14.963 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:14 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:14.579+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:15.019 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:14 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:14.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:15.019 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:14 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:14.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:15.019 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:14 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:14.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:15.019 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:14 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:14.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:15.019 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:14 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:14.762+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:15.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:15 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:15.303+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:15.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:15 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:15.303+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:15.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:15 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:15.303+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:15.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:15 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:15.303+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:15.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:15.764+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:15.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:15.764+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:15.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:15.764+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:15.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:15.764+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:15.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:15.615+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:15.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:15.615+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:15.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:15.615+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:15.963 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:15.615+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:16.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:15 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:15.774+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:16.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:15 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:15.774+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:16.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:15 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:15.774+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:16.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:15 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:15.774+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:16.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:15 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:15.774+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:16.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:15 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:15.774+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:16.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:16 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:16.324+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:16.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:16 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:16.324+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:16.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:16 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:16.324+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:16.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:16 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:16.324+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:16.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:16 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:16.324+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:16.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:16 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:16.583+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:16.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:16 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:16.583+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:16.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:16 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:16.583+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:16.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:16 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:16.584+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:16.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:16 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:16.740+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.882251+0000 front 2024-09-16T00:46:12.882119+0000 (oldest deadline 2024-09-16T00:46:37.582101+0000) 2024-09-16T00:47:16.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:16 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:16.740+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:26.184601+0000 front 2024-09-16T00:46:26.184662+0000 (oldest deadline 2024-09-16T00:46:50.284802+0000) 2024-09-16T00:47:16.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:16 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:16.740+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:33.785838+0000 front 2024-09-16T00:46:33.785927+0000 (oldest deadline 2024-09-16T00:46:59.586388+0000) 2024-09-16T00:47:16.962 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:16 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:16.740+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.287634+0000 front 2024-09-16T00:46:44.287591+0000 (oldest deadline 2024-09-16T00:47:07.787599+0000) 2024-09-16T00:47:16.963 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:16 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8[53724]: 2024-09-16T00:47:16.740+0000 7fe3c06a8640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.488413+0000 front 2024-09-16T00:46:52.488387+0000 (oldest deadline 2024-09-16T00:47:15.988458+0000) 2024-09-16T00:47:17.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:16 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:16.739+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:17.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:16 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:16.739+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:17.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:16 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:16.739+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:17.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:16 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:16.739+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:17.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:16 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:16.739+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:17.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:16 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:16.739+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:17.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:17 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:17.331+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:17.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:17 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:17.331+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:17.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:17 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:17.331+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:17.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:17 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:17.331+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:17.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:17 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:17.331+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:17.961 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:17 smithi153 podman[72355]: 2024-09-16 00:47:17.75445397 +0000 UTC m=+5.626776056 container died 0ee8f6047d795a3560387fe0c8192be83ab9f131532ebe934e2f373eee54c92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2) 2024-09-16T00:47:17.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:17 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:17.569+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:17.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:17 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:17.569+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:17.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:17 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:17.569+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:17.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:17 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:17.569+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:18.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:17 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:17.701+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:18.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:17 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:17.701+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:18.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:17 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:17.701+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:18.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:17 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:17.701+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:18.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:17 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:17.701+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:18.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:17 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:17.701+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:18.385 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:18 smithi153 podman[72355]: 2024-09-16 00:47:18.100638377 +0000 UTC m=+5.972960457 container cleanup 0ee8f6047d795a3560387fe0c8192be83ab9f131532ebe934e2f373eee54c92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:47:18.385 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:18 smithi153 bash[72355]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8 2024-09-16T00:47:18.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:18 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:18.378+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:18.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:18 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:18.378+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:18.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:18 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:18.378+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:18.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:18 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:18.378+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:18.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:18 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:18.378+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:18.711 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:18 smithi153 podman[72370]: 2024-09-16 00:47:18.385136361 +0000 UTC m=+0.632308649 container remove 0ee8f6047d795a3560387fe0c8192be83ab9f131532ebe934e2f373eee54c92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T00:47:18.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:18 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:18.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:18.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:18 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:18.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:18.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:18 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:18.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:18.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:18 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:18.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:18.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:18 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:18.566+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:19.066 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:18 smithi153 podman[72455]: 2024-09-16 00:47:18.727739974 +0000 UTC m=+0.112686731 container create cd301b82c3f2e6222486152a5c37b3bb1016b18ea7e73cae882ae47f0961d4d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:47:19.066 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:18 smithi153 podman[72455]: 2024-09-16 00:47:18.63097593 +0000 UTC m=+0.015922678 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:47:19.066 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:18 smithi153 podman[72455]: 2024-09-16 00:47:18.965780686 +0000 UTC m=+0.350727438 container init cd301b82c3f2e6222486152a5c37b3bb1016b18ea7e73cae882ae47f0961d4d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:47:19.066 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:18 smithi153 podman[72455]: 2024-09-16 00:47:18.969057683 +0000 UTC m=+0.354004433 container start cd301b82c3f2e6222486152a5c37b3bb1016b18ea7e73cae882ae47f0961d4d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:47:19.066 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:19 smithi153 podman[72455]: 2024-09-16 00:47:19.065669808 +0000 UTC m=+0.450616568 container attach cd301b82c3f2e6222486152a5c37b3bb1016b18ea7e73cae882ae47f0961d4d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T00:47:19.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:18 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:18.676+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:19.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:18 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:18.676+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:19.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:18 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:18.676+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:19.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:18 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:18.676+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:19.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:18 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:18.676+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:19.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:18 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:18.676+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:19.462 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 16 00:47:19 smithi153 podman[72455]: 2024-09-16 00:47:19.157588613 +0000 UTC m=+0.542535361 container died cd301b82c3f2e6222486152a5c37b3bb1016b18ea7e73cae882ae47f0961d4d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T00:47:19.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:19.353+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:19.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:19.353+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:19.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:19.353+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:19.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:19.353+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:19.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:19 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:19.353+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:19.836 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:19 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:19.603+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:19.836 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:19 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:19.603+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:19.836 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:19 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:19.603+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:19.836 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:19 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:19.603+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:19.836 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:19 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:19.603+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:19.851 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.8.service' 2024-09-16T00:47:19.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:19.931 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-16T00:47:19.931 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-16T00:47:19.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.9 2024-09-16T00:47:20.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:19 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:19.629+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:20.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:19 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:19.629+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:20.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:19 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:19.629+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:20.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:19 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:19.629+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:20.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:19 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:19.629+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:20.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:19 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:19.629+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:20.268 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:19 smithi001 systemd[1]: Stopping Ceph osd.9 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:20.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:20.267+0000 7fe6ef96e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:20.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:20.267+0000 7fe6ef96e640 -1 osd.9 95 *** Got signal Terminated *** 2024-09-16T00:47:20.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:20.267+0000 7fe6ef96e640 -1 osd.9 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:47:20.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:20.320+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:20.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:20.320+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:20.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:20.320+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:20.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:20.320+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:20.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:20 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:20.320+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:20.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:20 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:20.597+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:20.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:20 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:20.597+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:20.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:20 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:20.597+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:20.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:20 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:20.597+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:20.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:20 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:20.597+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:20.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:20 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:20.597+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:20.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:20 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:20.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:20.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:20 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:20.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:20.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:20 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:20.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:20.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:20 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:20.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:20.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:20 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:20.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:21.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:21 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:21.327+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:21.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:21 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:21.327+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:21.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:21 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:21.327+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:21.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:21 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:21.327+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:21.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:21 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:21.327+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:21.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:21 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:21.562+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:21.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:21 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:21.562+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:21.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:21 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:21.562+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:21.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:21 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:21.562+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:21.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:21 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:21.562+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:21.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:21 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:21.562+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:21.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:21 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:21.620+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:21.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:21 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:21.620+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:21.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:21 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:21.620+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:21.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:21 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:21.620+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:21.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:21 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:21.620+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:22.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:22.300+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:22.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:22.300+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:22.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:22.300+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:22.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:22.300+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:22.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:22 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:22.300+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:22.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:22 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:22.530+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:22.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:22 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:22.530+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:22.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:22 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:22.530+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:22.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:22 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:22.530+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:22.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:22 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:22.530+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:22.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:22 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:22.530+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:22.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:22 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:22.652+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:22.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:22 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:22.652+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:22.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:22 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:22.652+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:22.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:22 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:22.652+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:22.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:22 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:22.652+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:23.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:23.344+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:23.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:23.344+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:23.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:23.344+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:23.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:23.344+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:23.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:23 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:23.344+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:23.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:23.497+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:23.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:23.497+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:23.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:23.497+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:23.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:23.497+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:23.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:23.497+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:23.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:23 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:23.497+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:23.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:23.689+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:23.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:23.689+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:23.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:23.689+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:23.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:23.689+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:23.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:23.689+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:23.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:23.689+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:24.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:24 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:24.309+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:24.701 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:24 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:24.309+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:24.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:24 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:24.309+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:24.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:24 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:24.309+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:24.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:24 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:24.309+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:24.702 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:24 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:24.309+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:59.958356+0000 front 2024-09-16T00:46:59.958408+0000 (oldest deadline 2024-09-16T00:47:23.458431+0000) 2024-09-16T00:47:24.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:24.535+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:24.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:24.535+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:24.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:24.535+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:24.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:24.535+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:24.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:24.535+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:24.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:24 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:24.535+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:24.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:24.644+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:24.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:24.644+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:24.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:24.644+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:24.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:24.644+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:24.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:24.644+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:24.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:24 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:24.644+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:25.589 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:25 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:25.266+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.551835+0000 front 2024-09-16T00:46:20.551713+0000 (oldest deadline 2024-09-16T00:46:45.851888+0000) 2024-09-16T00:47:25.589 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:25 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:25.266+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.852531+0000 front 2024-09-16T00:46:25.852426+0000 (oldest deadline 2024-09-16T00:46:50.453244+0000) 2024-09-16T00:47:25.589 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:25 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:25.266+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:34.355058+0000 front 2024-09-16T00:46:34.354998+0000 (oldest deadline 2024-09-16T00:47:03.155572+0000) 2024-09-16T00:47:25.589 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:25 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:25.266+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:43.656522+0000 front 2024-09-16T00:46:43.656733+0000 (oldest deadline 2024-09-16T00:47:08.956399+0000) 2024-09-16T00:47:25.589 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:25 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:25.266+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:50.657545+0000 front 2024-09-16T00:46:50.657618+0000 (oldest deadline 2024-09-16T00:47:15.357288+0000) 2024-09-16T00:47:25.589 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:25 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9[61904]: 2024-09-16T00:47:25.266+0000 7fe6ebf84640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:59.958356+0000 front 2024-09-16T00:46:59.958408+0000 (oldest deadline 2024-09-16T00:47:23.458431+0000) 2024-09-16T00:47:25.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:25 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:25.558+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:25.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:25 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:25.558+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:25.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:25 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:25.558+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:25.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:25 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:25.558+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:25.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:25 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:25.558+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:25.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:25 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:25.558+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:25.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:25 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:25.558+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:55.537908+0000 front 2024-09-16T00:46:55.538009+0000 (oldest deadline 2024-09-16T00:47:25.538256+0000) 2024-09-16T00:47:25.877 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:25 smithi001 podman[124129]: 2024-09-16 00:47:25.589232369 +0000 UTC m=+5.452501953 container died 2d6304517744c91e51bb522ae317fc521c949367509a3f5d60cbd953d6dd8964 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T00:47:25.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:25.657+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:25.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:25.657+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:25.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:25.657+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:25.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:25.657+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:25.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:25.657+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:25.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:25 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:25.657+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:26.201 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:26 smithi001 podman[124129]: 2024-09-16 00:47:26.050651766 +0000 UTC m=+5.913921340 container cleanup 2d6304517744c91e51bb522ae317fc521c949367509a3f5d60cbd953d6dd8964 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-16T00:47:26.201 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:26 smithi001 bash[124129]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9 2024-09-16T00:47:26.677 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:26 smithi001 podman[124145]: 2024-09-16 00:47:26.335357076 +0000 UTC m=+0.741940968 container remove 2d6304517744c91e51bb522ae317fc521c949367509a3f5d60cbd953d6dd8964 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T00:47:26.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:26.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:26.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:26.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:26.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:26.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:26.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:26.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:26.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:26.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:26.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:26.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:26.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:26 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:26.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:55.537908+0000 front 2024-09-16T00:46:55.538009+0000 (oldest deadline 2024-09-16T00:47:25.538256+0000) 2024-09-16T00:47:26.951 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:26 smithi001 podman[124234]: 2024-09-16 00:47:26.677753949 +0000 UTC m=+0.109808428 container create f03ac85c7343241129b37aa3742430350a38d788b13b9a53a947e610b56395d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T00:47:26.952 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:26 smithi001 podman[124234]: 2024-09-16 00:47:26.58444499 +0000 UTC m=+0.016499437 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:47:26.952 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:26 smithi001 podman[124234]: 2024-09-16 00:47:26.91352264 +0000 UTC m=+0.345577075 container init f03ac85c7343241129b37aa3742430350a38d788b13b9a53a947e610b56395d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T00:47:26.952 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:26 smithi001 podman[124234]: 2024-09-16 00:47:26.916922351 +0000 UTC m=+0.348976799 container start f03ac85c7343241129b37aa3742430350a38d788b13b9a53a947e610b56395d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T00:47:26.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:26 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:26.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:26.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:26 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:26.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:26.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:26 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:26.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:26.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:26 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:26.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:26.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:26 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:26.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:26.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:26 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:26.618+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:27.398 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:26 smithi001 podman[124234]: 2024-09-16 00:47:26.996964253 +0000 UTC m=+0.429018690 container attach f03ac85c7343241129b37aa3742430350a38d788b13b9a53a947e610b56395d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:47:27.399 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 16 00:47:27 smithi001 podman[124234]: 2024-09-16 00:47:27.112364848 +0000 UTC m=+0.544419293 container died f03ac85c7343241129b37aa3742430350a38d788b13b9a53a947e610b56395d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:47:27.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.9.service' 2024-09-16T00:47:27.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:27.799 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-16T00:47:27.799 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-16T00:47:27.799 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.10 2024-09-16T00:47:27.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:27 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:27.607+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:27.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:27 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:27.607+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:27.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:27 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:27.607+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:27.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:27 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:27.607+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:27.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:27 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:27.607+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:27.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:27 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:27.607+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:28.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:27.645+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:28.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:27.645+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:28.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:27.645+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:28.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:27.645+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:28.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:27.645+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:28.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:27.645+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:28.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:27 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:27.645+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:55.537908+0000 front 2024-09-16T00:46:55.538009+0000 (oldest deadline 2024-09-16T00:47:25.538256+0000) 2024-09-16T00:47:28.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:27 smithi019 systemd[1]: Stopping Ceph osd.10 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:28.607 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.119+0000 7f5c9f45d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:28.607 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.119+0000 7f5c9f45d640 -1 osd.10 95 *** Got signal Terminated *** 2024-09-16T00:47:28.607 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.119+0000 7f5c9f45d640 -1 osd.10 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:47:28.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:28 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:28.637+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:28.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:28 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:28.637+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:28.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:28 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:28.637+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:28.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:28 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:28.637+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:28.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:28 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:28.637+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:28.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:28 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:28.637+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:29.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.613+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:29.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.613+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:29.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.613+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:29.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.613+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:29.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.613+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:29.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.613+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:29.108 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:28 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:28.613+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:55.537908+0000 front 2024-09-16T00:46:55.538009+0000 (oldest deadline 2024-09-16T00:47:25.538256+0000) 2024-09-16T00:47:29.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:29 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:29.589+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:29.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:29 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:29.589+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:29.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:29 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:29.589+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:29.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:29 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:29.589+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:29.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:29 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:29.589+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:29.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:29 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:29.589+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:29.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:29 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:29.589+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:55.537908+0000 front 2024-09-16T00:46:55.538009+0000 (oldest deadline 2024-09-16T00:47:25.538256+0000) 2024-09-16T00:47:29.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:29 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:29.596+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:29.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:29 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:29.596+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:29.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:29 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:29.596+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:29.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:29 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:29.596+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:29.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:29 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:29.596+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:29.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:29 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:29.596+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:30.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:30 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:30.588+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:30.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:30 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:30.588+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:30.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:30 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:30.588+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:30.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:30 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:30.588+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:30.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:30 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:30.588+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:30.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:30 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:30.588+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:30.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:30 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:30.588+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:55.537908+0000 front 2024-09-16T00:46:55.538009+0000 (oldest deadline 2024-09-16T00:47:25.538256+0000) 2024-09-16T00:47:30.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:30 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:30.556+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:30.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:30 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:30.556+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:30.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:30 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:30.556+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:30.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:30 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:30.556+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:30.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:30 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:30.556+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:30.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:30 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:30.556+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:31.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:31 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:31.507+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:31.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:31 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:31.507+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:31.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:31 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:31.507+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:31.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:31 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:31.507+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:31.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:31 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:31.507+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:31.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:31 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:31.507+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:31.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:31 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:31.507+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:32.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:31 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:31.632+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:32.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:31 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:31.632+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:32.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:31 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:31.632+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:32.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:31 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:31.632+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:32.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:31 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:31.632+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:32.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:31 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:31.632+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:32.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:31 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:31.632+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:55.537908+0000 front 2024-09-16T00:46:55.538009+0000 (oldest deadline 2024-09-16T00:47:25.538256+0000) 2024-09-16T00:47:32.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:32 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:32.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:09.732802+0000 front 2024-09-16T00:46:09.732891+0000 (oldest deadline 2024-09-16T00:46:34.432913+0000) 2024-09-16T00:47:32.857 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:32 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:32.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.334004+0000 front 2024-09-16T00:46:20.333986+0000 (oldest deadline 2024-09-16T00:46:43.234145+0000) 2024-09-16T00:47:32.858 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:32 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:32.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:25.535123+0000 front 2024-09-16T00:46:25.535167+0000 (oldest deadline 2024-09-16T00:46:51.435044+0000) 2024-09-16T00:47:32.858 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:32 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:32.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-16T00:46:36.136303+0000 front 2024-09-16T00:46:36.136089+0000 (oldest deadline 2024-09-16T00:47:02.035980+0000) 2024-09-16T00:47:32.858 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:32 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:32.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:42.036304+0000 front 2024-09-16T00:46:42.036333+0000 (oldest deadline 2024-09-16T00:47:06.736371+0000) 2024-09-16T00:47:32.858 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:32 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:32.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:49.637363+0000 front 2024-09-16T00:46:49.637350+0000 (oldest deadline 2024-09-16T00:47:15.537241+0000) 2024-09-16T00:47:32.858 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:32 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10[60156]: 2024-09-16T00:47:32.598+0000 7f5c9ba73640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:55.537908+0000 front 2024-09-16T00:46:55.538009+0000 (oldest deadline 2024-09-16T00:47:25.538256+0000) 2024-09-16T00:47:32.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:32 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:32.516+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:32.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:32 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:32.516+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:32.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:32 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:32.516+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:32.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:32 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:32.516+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:32.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:32 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:32.516+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:32.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:32 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:32.516+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:32.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:32 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:32.516+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:33.788 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:33 smithi019 podman[71772]: 2024-09-16 00:47:33.514065876 +0000 UTC m=+5.509367170 container died c95447b661089b1aac57f2bd0096574a32f22d41495e95f19a7e8e591569b650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T00:47:33.788 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:33 smithi019 podman[71772]: 2024-09-16 00:47:33.788419785 +0000 UTC m=+5.783721075 container cleanup c95447b661089b1aac57f2bd0096574a32f22d41495e95f19a7e8e591569b650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:47:33.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:33 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:33.494+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:33.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:33 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:33.494+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:33.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:33 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:33.494+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:33.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:33 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:33.494+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:33.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:33 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:33.494+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:33.963 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:33 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:33.494+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:33.963 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:33 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:33.494+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:34.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:33 smithi019 bash[71772]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10 2024-09-16T00:47:34.107 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:34 smithi019 podman[71791]: 2024-09-16 00:47:34.020735399 +0000 UTC m=+0.499626174 container remove c95447b661089b1aac57f2bd0096574a32f22d41495e95f19a7e8e591569b650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:47:34.607 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:34 smithi019 podman[71880]: 2024-09-16 00:47:34.346363458 +0000 UTC m=+0.101976680 container create 5e35df78bee947565b563cac7c884a85bc5d835f72d63425d3593d84a82b3823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:47:34.607 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:34 smithi019 podman[71880]: 2024-09-16 00:47:34.260180863 +0000 UTC m=+0.015794026 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:47:34.607 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:34 smithi019 podman[71880]: 2024-09-16 00:47:34.573584101 +0000 UTC m=+0.329197264 container init 5e35df78bee947565b563cac7c884a85bc5d835f72d63425d3593d84a82b3823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e) 2024-09-16T00:47:34.607 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:34 smithi019 podman[71880]: 2024-09-16 00:47:34.576783961 +0000 UTC m=+0.332397114 container start 5e35df78bee947565b563cac7c884a85bc5d835f72d63425d3593d84a82b3823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:47:34.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:34 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:34.504+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:34.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:34 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:34.504+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:34.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:34 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:34.504+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:34.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:34 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:34.504+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:34.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:34 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:34.504+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:34.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:34 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:34.504+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:34.962 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:34 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:34.504+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:35.051 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:34 smithi019 podman[71880]: 2024-09-16 00:47:34.682165194 +0000 UTC m=+0.437778363 container attach 5e35df78bee947565b563cac7c884a85bc5d835f72d63425d3593d84a82b3823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T00:47:35.051 INFO:journalctl@ceph.osd.10.smithi019.stdout:Sep 16 00:47:34 smithi019 podman[71880]: 2024-09-16 00:47:34.765520787 +0000 UTC m=+0.521133941 container died 5e35df78bee947565b563cac7c884a85bc5d835f72d63425d3593d84a82b3823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T00:47:35.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.10.service' 2024-09-16T00:47:35.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:35.387 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-16T00:47:35.387 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-16T00:47:35.387 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.11 2024-09-16T00:47:35.671 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 systemd[1]: Stopping Ceph osd.11 for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:35.671 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.479+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:35.671 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.479+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:35.671 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.479+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:35.671 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.479+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:35.671 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.479+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:35.671 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.479+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:35.671 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.479+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:35.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.670+0000 7f7663d43640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:35.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.670+0000 7f7663d43640 -1 osd.11 95 *** Got signal Terminated *** 2024-09-16T00:47:35.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:35 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:35.670+0000 7f7663d43640 -1 osd.11 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:47:36.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:36 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:36.436+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:36.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:36 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:36.436+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:36.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:36 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:36.436+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:36.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:36 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:36.436+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:36.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:36 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:36.436+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:36.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:36 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:36.436+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:36.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:36 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:36.436+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:37.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:37.451+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:37.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:37.451+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:37.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:37.451+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:37.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:37.451+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:37.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:37.451+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:37.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:37.451+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:37.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:37 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:37.451+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:38.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:38.408+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:38.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:38.408+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:38.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:38.408+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:38.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:38.408+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:38.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:38.408+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:38.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:38.408+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:38.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:38 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:38.408+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:39.711 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:39.442+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:39.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:39.442+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:39.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:39.442+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:39.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:39.442+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:39.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:39.442+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:39.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:39.442+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:39.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:39.442+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:39.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:39 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:39.442+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6820 osd.8 since back 2024-09-16T00:47:16.977018+0000 front 2024-09-16T00:47:16.977053+0000 (oldest deadline 2024-09-16T00:47:38.677050+0000) 2024-09-16T00:47:40.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:40.397+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-16T00:46:12.668715+0000 front 2024-09-16T00:46:12.668821+0000 (oldest deadline 2024-09-16T00:46:37.968732+0000) 2024-09-16T00:47:40.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:40.397+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-09-16T00:46:20.869623+0000 front 2024-09-16T00:46:20.869849+0000 (oldest deadline 2024-09-16T00:46:44.369513+0000) 2024-09-16T00:47:40.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:40.397+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-16T00:46:24.369962+0000 front 2024-09-16T00:46:24.369987+0000 (oldest deadline 2024-09-16T00:46:50.270160+0000) 2024-09-16T00:47:40.712 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:40.397+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-09-16T00:46:44.872600+0000 front 2024-09-16T00:46:44.872705+0000 (oldest deadline 2024-09-16T00:47:07.672891+0000) 2024-09-16T00:47:40.713 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:40.397+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-16T00:46:52.973692+0000 front 2024-09-16T00:46:52.973710+0000 (oldest deadline 2024-09-16T00:47:18.273778+0000) 2024-09-16T00:47:40.713 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:40.397+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-16T00:46:58.274514+0000 front 2024-09-16T00:46:58.274559+0000 (oldest deadline 2024-09-16T00:47:22.974462+0000) 2024-09-16T00:47:40.713 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:40.397+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-09-16T00:47:06.375988+0000 front 2024-09-16T00:47:06.376039+0000 (oldest deadline 2024-09-16T00:47:31.076010+0000) 2024-09-16T00:47:40.713 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:40 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11[60287]: 2024-09-16T00:47:40.397+0000 7f765c351640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.153:6820 osd.8 since back 2024-09-16T00:47:16.977018+0000 front 2024-09-16T00:47:16.977053+0000 (oldest deadline 2024-09-16T00:47:38.677050+0000) 2024-09-16T00:47:41.304 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:41 smithi153 podman[72557]: 2024-09-16 00:47:41.025578704 +0000 UTC m=+5.447650837 container died 45f672136646d1a3360ddef2e04eb830855dbc97438cebc0d75fb448aa34389a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-16T00:47:41.647 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:41 smithi153 podman[72557]: 2024-09-16 00:47:41.395729232 +0000 UTC m=+5.817801353 container cleanup 45f672136646d1a3360ddef2e04eb830855dbc97438cebc0d75fb448aa34389a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:47:41.647 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:41 smithi153 bash[72557]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11 2024-09-16T00:47:41.961 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:41 smithi153 podman[72584]: 2024-09-16 00:47:41.646870723 +0000 UTC m=+0.617912048 container remove 45f672136646d1a3360ddef2e04eb830855dbc97438cebc0d75fb448aa34389a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T00:47:42.266 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:41 smithi153 podman[72672]: 2024-09-16 00:47:41.864585641 +0000 UTC m=+0.016577289 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:47:42.266 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:41 smithi153 podman[72672]: 2024-09-16 00:47:41.963927324 +0000 UTC m=+0.115918962 container create 9caaa48c774e6193cec71e66cb97e400e571539b79a83ed7b8929a5b04731039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T00:47:42.266 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:42 smithi153 podman[72672]: 2024-09-16 00:47:42.183213677 +0000 UTC m=+0.335205314 container init 9caaa48c774e6193cec71e66cb97e400e571539b79a83ed7b8929a5b04731039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:47:42.266 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:42 smithi153 podman[72672]: 2024-09-16 00:47:42.186517398 +0000 UTC m=+0.338509044 container start 9caaa48c774e6193cec71e66cb97e400e571539b79a83ed7b8929a5b04731039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:47:42.266 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:42 smithi153 podman[72672]: 2024-09-16 00:47:42.266225371 +0000 UTC m=+0.418217024 container attach 9caaa48c774e6193cec71e66cb97e400e571539b79a83ed7b8929a5b04731039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T00:47:42.685 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 16 00:47:42 smithi153 podman[72672]: 2024-09-16 00:47:42.371035617 +0000 UTC m=+0.523027265 container died 9caaa48c774e6193cec71e66cb97e400e571539b79a83ed7b8929a5b04731039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T00:47:42.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@osd.11.service' 2024-09-16T00:47:43.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:43.029 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-16T00:47:43.029 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-16T00:47:43.029 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.a 2024-09-16T00:47:43.336 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:47:43 smithi001 systemd[1]: Stopping Ceph mds.a for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:43.701 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:47:43 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-a[68550]: 2024-09-16T00:47:43.335+0000 7f517a2bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:43.701 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:47:43 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-a[68550]: 2024-09-16T00:47:43.335+0000 7f517a2bf640 -1 mds.a *** got signal Terminated *** 2024-09-16T00:47:45.701 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:47:45 smithi001 podman[124349]: 2024-09-16 00:47:45.393296003 +0000 UTC m=+2.160662055 container died ca4a96a20d49b34b74dc02f5a5295fdaa44cd15c89dd18bc46ddfd370c75a19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:47:46.161 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:47:45 smithi001 podman[124349]: 2024-09-16 00:47:45.841216758 +0000 UTC m=+2.608582852 container cleanup ca4a96a20d49b34b74dc02f5a5295fdaa44cd15c89dd18bc46ddfd370c75a19a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T00:47:46.161 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 16 00:47:45 smithi001 bash[124349]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-a 2024-09-16T00:47:46.338 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.a.service' 2024-09-16T00:47:46.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:46.373 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-16T00:47:46.373 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-16T00:47:46.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.d 2024-09-16T00:47:46.701 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:47:46 smithi001 systemd[1]: Stopping Ceph mds.d for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:47.201 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:47:46 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-d[67322]: 2024-09-16T00:47:46.705+0000 7fba1dc80640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:47.201 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:47:46 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-d[67322]: 2024-09-16T00:47:46.705+0000 7fba1dc80640 -1 mds.d *** got signal Terminated *** 2024-09-16T00:47:49.506 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:47:49 smithi001 podman[124470]: 2024-09-16 00:47:49.240769905 +0000 UTC m=+2.642723963 container died d05f3acf2a8b54aba002251e25b41e72a8372e9475d57043503e811ff519cf57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-d, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T00:47:49.873 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:47:49 smithi001 podman[124470]: 2024-09-16 00:47:49.597664893 +0000 UTC m=+2.999618950 container cleanup d05f3acf2a8b54aba002251e25b41e72a8372e9475d57043503e811ff519cf57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T00:47:49.874 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 16 00:47:49 smithi001 bash[124470]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-d 2024-09-16T00:47:50.028 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.d.service' 2024-09-16T00:47:50.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:50.062 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-16T00:47:50.062 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-16T00:47:50.062 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.g 2024-09-16T00:47:50.393 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:47:50 smithi001 systemd[1]: Stopping Ceph mds.g for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:50.701 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:47:50 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-g[68217]: 2024-09-16T00:47:50.391+0000 7f036841d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:50.701 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:47:50 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-g[68217]: 2024-09-16T00:47:50.391+0000 7f036841d640 -1 mds.g *** got signal Terminated *** 2024-09-16T00:47:54.701 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:47:54 smithi001 podman[124591]: 2024-09-16 00:47:54.406569538 +0000 UTC m=+4.117796898 container died 20f03c28ee4eccc64cceec799dfd9e83a194c1c40a891a2c00673b95de6c1cb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True) 2024-09-16T00:47:55.177 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:47:54 smithi001 podman[124591]: 2024-09-16 00:47:54.859452522 +0000 UTC m=+4.570679883 container cleanup 20f03c28ee4eccc64cceec799dfd9e83a194c1c40a891a2c00673b95de6c1cb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:47:55.177 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 16 00:47:54 smithi001 bash[124591]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-g 2024-09-16T00:47:55.352 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.g.service' 2024-09-16T00:47:55.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:55.386 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-16T00:47:55.386 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-16T00:47:55.386 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.j 2024-09-16T00:47:55.701 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:47:55 smithi001 systemd[1]: Stopping Ceph mds.j for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:56.201 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:47:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-j[66923]: 2024-09-16T00:47:55.703+0000 7f7e3bdb2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:56.201 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:47:55 smithi001 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-j[66923]: 2024-09-16T00:47:55.703+0000 7f7e3bdb2640 -1 mds.j *** got signal Terminated *** 2024-09-16T00:47:57.721 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:47:57 smithi001 podman[124713]: 2024-09-16 00:47:57.46971582 +0000 UTC m=+1.857609604 container died 5c3cdfa00a055977fae6e05b15119355a2f90829658651c1d7ef59b6f1504487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-j, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T00:47:57.721 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:47:57 smithi001 podman[124713]: 2024-09-16 00:47:57.721454712 +0000 UTC m=+2.109348496 container cleanup 5c3cdfa00a055977fae6e05b15119355a2f90829658651c1d7ef59b6f1504487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:47:57.980 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 16 00:47:57 smithi001 bash[124713]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-j 2024-09-16T00:47:58.162 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.j.service' 2024-09-16T00:47:58.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:47:58.196 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-16T00:47:58.196 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-16T00:47:58.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.b 2024-09-16T00:47:58.512 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:47:58 smithi019 systemd[1]: Stopping Ceph mds.b for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:47:58.780 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:47:58 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b[65807]: 2024-09-16T00:47:58.510+0000 7f153b32a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:47:58.780 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:47:58 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b[65807]: 2024-09-16T00:47:58.510+0000 7f153b32a640 -1 mds.b *** got signal Terminated *** 2024-09-16T00:48:02.820 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:48:02 smithi019 podman[71983]: 2024-09-16 00:48:02.52490167 +0000 UTC m=+4.130684166 container died 09e23b1983f32fbce3f38ef35b5bb98d7a4359b5226e50468e1bc78f04afe849 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T00:48:03.107 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:48:02 smithi019 podman[71983]: 2024-09-16 00:48:02.902710251 +0000 UTC m=+4.508492747 container cleanup 09e23b1983f32fbce3f38ef35b5bb98d7a4359b5226e50468e1bc78f04afe849 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-16T00:48:03.107 INFO:journalctl@ceph.mds.b.smithi019.stdout:Sep 16 00:48:02 smithi019 bash[71983]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-b 2024-09-16T00:48:03.304 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.b.service' 2024-09-16T00:48:03.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:48:03.340 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-16T00:48:03.340 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-16T00:48:03.340 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.e 2024-09-16T00:48:03.857 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:48:03 smithi019 systemd[1]: Stopping Ceph mds.e for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:48:03.857 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:48:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e[65194]: 2024-09-16T00:48:03.647+0000 7f975e06f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:48:03.857 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:48:03 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e[65194]: 2024-09-16T00:48:03.647+0000 7f975e06f640 -1 mds.e *** got signal Terminated *** 2024-09-16T00:48:06.857 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:48:06 smithi019 podman[72116]: 2024-09-16 00:48:06.601532872 +0000 UTC m=+3.042003854 container died c8d7c23eff5391ddd0be3c85cc8a0a8a0aab748fa89418d36ea061c129f63c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T00:48:07.107 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:48:06 smithi019 podman[72116]: 2024-09-16 00:48:06.947933351 +0000 UTC m=+3.388404332 container cleanup c8d7c23eff5391ddd0be3c85cc8a0a8a0aab748fa89418d36ea061c129f63c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T00:48:07.107 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:48:06 smithi019 bash[72116]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e 2024-09-16T00:48:07.376 INFO:journalctl@ceph.mds.e.smithi019.stdout:Sep 16 00:48:07 smithi019 podman[72130]: 2024-09-16 00:48:07.115641714 +0000 UTC m=+0.506296865 container remove c8d7c23eff5391ddd0be3c85cc8a0a8a0aab748fa89418d36ea061c129f63c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:48:07.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.e.service' 2024-09-16T00:48:07.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:48:07.414 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-16T00:48:07.414 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-16T00:48:07.414 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.h 2024-09-16T00:48:07.818 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:48:07 smithi019 systemd[1]: Stopping Ceph mds.h for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:48:08.107 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:48:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h[65495]: 2024-09-16T00:48:07.816+0000 7f9a42c7d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:48:08.107 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:48:07 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h[65495]: 2024-09-16T00:48:07.816+0000 7f9a42c7d640 -1 mds.h *** got signal Terminated *** 2024-09-16T00:48:10.852 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:48:10 smithi019 podman[72237]: 2024-09-16 00:48:10.591768136 +0000 UTC m=+2.960012557 container died ce175bee792691bc37017b7196f794a352253e9553d771ae90e83f166bbeec05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T00:48:11.107 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:48:10 smithi019 podman[72237]: 2024-09-16 00:48:10.951545437 +0000 UTC m=+3.319789840 container cleanup ce175bee792691bc37017b7196f794a352253e9553d771ae90e83f166bbeec05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T00:48:11.107 INFO:journalctl@ceph.mds.h.smithi019.stdout:Sep 16 00:48:10 smithi019 bash[72237]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-h 2024-09-16T00:48:11.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.h.service' 2024-09-16T00:48:11.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:48:11.421 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-16T00:48:11.421 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-16T00:48:11.421 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.k 2024-09-16T00:48:11.747 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:48:11 smithi019 systemd[1]: Stopping Ceph mds.k for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:48:12.107 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:48:11 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k[64879]: 2024-09-16T00:48:11.745+0000 7f7262cda640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:48:12.107 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:48:11 smithi019 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k[64879]: 2024-09-16T00:48:11.745+0000 7f7262cda640 -1 mds.k *** got signal Terminated *** 2024-09-16T00:48:14.446 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:48:14 smithi019 podman[72359]: 2024-09-16 00:48:14.135370414 +0000 UTC m=+2.493791975 container died c29345750f6f0a53bc4205db9ec63ef907eb4bfbce59742d88289e8e33e565e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T00:48:14.446 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:48:14 smithi019 podman[72359]: 2024-09-16 00:48:14.446524687 +0000 UTC m=+2.804946256 container cleanup c29345750f6f0a53bc4205db9ec63ef907eb4bfbce59742d88289e8e33e565e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T00:48:14.782 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:48:14 smithi019 bash[72359]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k 2024-09-16T00:48:14.783 INFO:journalctl@ceph.mds.k.smithi019.stdout:Sep 16 00:48:14 smithi019 podman[72371]: 2024-09-16 00:48:14.664294517 +0000 UTC m=+0.522177519 container remove c29345750f6f0a53bc4205db9ec63ef907eb4bfbce59742d88289e8e33e565e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:48:14.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.k.service' 2024-09-16T00:48:14.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:48:14.829 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-16T00:48:14.829 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-16T00:48:14.830 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.c 2024-09-16T00:48:15.135 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:14 smithi153 systemd[1]: Stopping Ceph mds.c for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:48:15.393 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c[65572]: 2024-09-16T00:48:15.134+0000 7fc8436d8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:48:15.393 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:15 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c[65572]: 2024-09-16T00:48:15.135+0000 7fc8436d8640 -1 mds.c *** got signal Terminated *** 2024-09-16T00:48:18.935 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:18 smithi153 podman[72775]: 2024-09-16 00:48:18.607600296 +0000 UTC m=+3.574011340 container died 3e37f8eed2c9fd4523be2e150069d7ceb8d772b11c5e285da8a64ebd33be783f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T00:48:19.211 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:19 smithi153 podman[72775]: 2024-09-16 00:48:19.026249287 +0000 UTC m=+3.992660325 container cleanup 3e37f8eed2c9fd4523be2e150069d7ceb8d772b11c5e285da8a64ebd33be783f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:48:19.212 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:19 smithi153 bash[72775]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c 2024-09-16T00:48:19.449 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.c.service' 2024-09-16T00:48:19.472 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:19 smithi153 podman[72798]: 2024-09-16 00:48:19.219044465 +0000 UTC m=+0.613109681 container remove 3e37f8eed2c9fd4523be2e150069d7ceb8d772b11c5e285da8a64ebd33be783f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:48:19.477 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:19 smithi153 systemd[1]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.c.service: Deactivated successfully. 2024-09-16T00:48:19.477 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:19 smithi153 systemd[1]: Stopped Ceph mds.c for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:48:19.477 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 16 00:48:19 smithi153 systemd[1]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.c.service: Consumed 3.526s CPU time. 2024-09-16T00:48:19.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:48:19.488 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-16T00:48:19.488 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-16T00:48:19.488 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.f 2024-09-16T00:48:19.813 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:19 smithi153 systemd[1]: Stopping Ceph mds.f for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:48:20.211 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:19 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f[65267]: 2024-09-16T00:48:19.813+0000 7f80dd5f5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:48:20.212 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:19 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f[65267]: 2024-09-16T00:48:19.813+0000 7f80dd5f5640 -1 mds.f *** got signal Terminated *** 2024-09-16T00:48:22.955 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:22 smithi153 podman[72910]: 2024-09-16 00:48:22.598237893 +0000 UTC m=+2.886946836 container died 6a5b145e9c1f0c66d6ef359ee6cee1ed7755fa9555725ff3783649b39bdcd086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:48:23.211 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:23 smithi153 podman[72910]: 2024-09-16 00:48:23.02994723 +0000 UTC m=+3.318656158 container cleanup 6a5b145e9c1f0c66d6ef359ee6cee1ed7755fa9555725ff3783649b39bdcd086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:48:23.212 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:23 smithi153 bash[72910]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f 2024-09-16T00:48:23.462 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.f.service' 2024-09-16T00:48:23.484 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:23 smithi153 podman[72921]: 2024-09-16 00:48:23.22268906 +0000 UTC m=+0.617197812 container remove 6a5b145e9c1f0c66d6ef359ee6cee1ed7755fa9555725ff3783649b39bdcd086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T00:48:23.484 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:23 smithi153 systemd[1]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.f.service: Deactivated successfully. 2024-09-16T00:48:23.484 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:23 smithi153 systemd[1]: Stopped Ceph mds.f for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:48:23.484 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 16 00:48:23 smithi153 systemd[1]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.f.service: Consumed 2.772s CPU time. 2024-09-16T00:48:23.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:48:23.495 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-16T00:48:23.495 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-16T00:48:23.495 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.i 2024-09-16T00:48:23.833 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:48:23 smithi153 systemd[1]: Stopping Ceph mds.i for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:48:24.211 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:48:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i[65875]: 2024-09-16T00:48:23.833+0000 7fd1539dd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:48:24.211 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:48:23 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i[65875]: 2024-09-16T00:48:23.833+0000 7fd1539dd640 -1 mds.i *** got signal Terminated *** 2024-09-16T00:48:26.711 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:48:26 smithi153 podman[73031]: 2024-09-16 00:48:26.593772294 +0000 UTC m=+2.869184131 container died f0b544629d6c2f509583e555f4132fe62dd23187ab98138ab5ac34df4a326316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:48:27.212 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:48:26 smithi153 podman[73031]: 2024-09-16 00:48:26.949803668 +0000 UTC m=+3.225215491 container cleanup f0b544629d6c2f509583e555f4132fe62dd23187ab98138ab5ac34df4a326316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:48:27.212 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 16 00:48:26 smithi153 bash[73031]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-i 2024-09-16T00:48:27.384 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.i.service' 2024-09-16T00:48:27.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:48:27.417 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-16T00:48:27.417 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-16T00:48:27.417 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.l 2024-09-16T00:48:27.728 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:27 smithi153 systemd[1]: Stopping Ceph mds.l for 1226ed86-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:48:28.212 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:27 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l[64967]: 2024-09-16T00:48:27.727+0000 7f54c8053640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:48:28.212 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:27 smithi153 ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l[64967]: 2024-09-16T00:48:27.727+0000 7f54c8053640 -1 mds.l *** got signal Terminated *** 2024-09-16T00:48:31.704 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:31 smithi153 podman[73152]: 2024-09-16 00:48:31.373543554 +0000 UTC m=+3.738554626 container died b0883f0ab6220ac596755f4de666a39b6fe389ef6f8051ac06e67df98f083a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:48:31.964 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:31 smithi153 podman[73152]: 2024-09-16 00:48:31.803236496 +0000 UTC m=+4.168247561 container cleanup b0883f0ab6220ac596755f4de666a39b6fe389ef6f8051ac06e67df98f083a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:48:31.964 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:31 smithi153 bash[73152]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l 2024-09-16T00:48:32.220 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.l.service' 2024-09-16T00:48:32.242 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:31 smithi153 podman[73164]: 2024-09-16 00:48:31.987293676 +0000 UTC m=+0.614654351 container remove b0883f0ab6220ac596755f4de666a39b6fe389ef6f8051ac06e67df98f083a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1226ed86-73c1-11ef-bceb-c7b262605968-mds-l, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T00:48:32.242 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:32 smithi153 systemd[1]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.l.service: Deactivated successfully. 2024-09-16T00:48:32.242 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:32 smithi153 systemd[1]: Stopped Ceph mds.l for 1226ed86-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:48:32.242 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 16 00:48:32 smithi153 systemd[1]: ceph-1226ed86-73c1-11ef-bceb-c7b262605968@mds.l.service: Consumed 1.166s CPU time. 2024-09-16T00:48:32.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:48:32.253 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-16T00:48:32.254 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:48:32.465 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:48:35.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:48:35.240 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:48:37.701 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:48:37.904 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:48:40.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:48:40.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:48:40.314 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:48:40.341 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T00:48:40.343 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386/remote/smithi001/crash 2024-09-16T00:48:40.344 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:48:40.389 INFO:teuthology.orchestra.run.smithi001.stderr:tar: /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:48:40.389 INFO:teuthology.orchestra.run.smithi001.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:48:40.391 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386/remote/smithi019/crash 2024-09-16T00:48:40.392 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:48:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:48:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:48:40.433 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386/remote/smithi153/crash 2024-09-16T00:48:40.434 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:48:40.475 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:48:40.475 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:48:40.476 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T00:48:40.477 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-16T00:48:40.517 INFO:tasks.cephadm:Compressing logs... 2024-09-16T00:48:40.517 DEBUG:teuthology.orchestra.run.smithi001:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:48:40.560 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:48:40.563 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:48:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-16T00:48:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:: No such file or directory 2024-09-16T00:48:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T00:48:40.593 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:48:40.594 INFO:teuthology.orchestra.run.smithi153.stderr:: No such file or directory 2024-09-16T00:48:40.594 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:48:40.594 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-16T00:48:40.594 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T00:48:40.594 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:48:40.595 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-16T00:48:40.596 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T00:48:40.596 INFO:teuthology.orchestra.run.smithi153.stderr: 83.4% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:48:40.596 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T00:48:40.596 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:48:40.596 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-16T00:48:40.596 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log: 90.5% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:48:40.607 INFO:teuthology.orchestra.run.smithi153.stderr: 87.9% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:48:40.607 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-16T00:48:40.607 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-16T00:48:40.607 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.8.log: 94.7% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-16T00:48:40.607 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-16T00:48:40.607 INFO:teuthology.orchestra.run.smithi153.stderr: 90.9% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:48:40.608 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-16T00:48:40.614 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.c.log: 93.3% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-16T00:48:40.614 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:48:40.615 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mon.a.log: 83.1% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.6.log: /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:48:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr: 94.7%/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.d.log: -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.j.log: /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mgr.x.log: 90.9% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.3.log: 94.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-16T00:48:40.617 INFO:teuthology.orchestra.run.smithi001.stderr: 94.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:48:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-16T00:48:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:: No such file or directory 2024-09-16T00:48:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T00:48:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T00:48:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose 94.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-16T00:48:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log: /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mgr.y.log: 87.9% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:48:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T00:48:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.h.log: 90.9% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:48:40.622 INFO:teuthology.orchestra.run.smithi019.stderr: 93.2% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T00:48:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-16T00:48:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:48:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:48:40.625 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:48:40.690 INFO:teuthology.orchestra.run.smithi019.stderr: 94.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-16T00:48:40.697 INFO:teuthology.orchestra.run.smithi153.stderr: 94.7% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-16T00:48:40.698 INFO:teuthology.orchestra.run.smithi153.stderr: 92.9% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-16T00:48:40.724 INFO:teuthology.orchestra.run.smithi019.stderr: 92.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-16T00:48:40.863 INFO:teuthology.orchestra.run.smithi001.stderr: 90.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T00:48:40.885 INFO:teuthology.orchestra.run.smithi153.stderr: 94.2% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-16T00:48:41.066 INFO:teuthology.orchestra.run.smithi153.stderr: 91.8% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T00:48:41.216 INFO:teuthology.orchestra.run.smithi019.stderr: 91.7% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T00:48:41.452 INFO:teuthology.orchestra.run.smithi001.stderr: 92.8% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T00:48:41.513 INFO:teuthology.orchestra.run.smithi001.stderr: 92.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T00:48:41.712 INFO:teuthology.orchestra.run.smithi001.stderr: 92.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-16T00:48:41.785 INFO:teuthology.orchestra.run.smithi019.stderr: 92.1% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T00:48:41.789 INFO:teuthology.orchestra.run.smithi001.stderr: 90.4% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T00:48:41.999 INFO:teuthology.orchestra.run.smithi153.stderr: 93.3% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-16T00:48:42.072 INFO:teuthology.orchestra.run.smithi153.stderr: 92.9% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-16T00:48:42.276 INFO:teuthology.orchestra.run.smithi019.stderr: 92.0% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T00:48:42.341 INFO:teuthology.orchestra.run.smithi019.stderr: 91.6% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-16T00:48:42.370 INFO:teuthology.orchestra.run.smithi001.stderr: 93.1% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T00:48:42.371 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-16T00:48:42.372 INFO:teuthology.orchestra.run.smithi001.stderr:real 0m1.797s 2024-09-16T00:48:42.372 INFO:teuthology.orchestra.run.smithi001.stderr:user 0m5.993s 2024-09-16T00:48:42.372 INFO:teuthology.orchestra.run.smithi001.stderr:sys 0m0.231s 2024-09-16T00:48:42.635 INFO:teuthology.orchestra.run.smithi019.stderr: 92.7% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T00:48:42.745 INFO:teuthology.orchestra.run.smithi019.stderr: 93.1% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-16T00:48:42.747 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-16T00:48:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m2.169s 2024-09-16T00:48:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m9.396s 2024-09-16T00:48:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m0.290s 2024-09-16T00:48:42.873 INFO:teuthology.orchestra.run.smithi153.stderr: 92.4% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T00:48:42.924 INFO:teuthology.orchestra.run.smithi153.stderr: 92.4% -- replaced with /var/log/ceph/1226ed86-73c1-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T00:48:42.926 INFO:teuthology.orchestra.run.smithi153.stderr: 2024-09-16T00:48:42.926 INFO:teuthology.orchestra.run.smithi153.stderr:real 0m2.348s 2024-09-16T00:48:42.926 INFO:teuthology.orchestra.run.smithi153.stderr:user 0m8.206s 2024-09-16T00:48:42.926 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m0.302s 2024-09-16T00:48:42.926 INFO:tasks.cephadm:Archiving logs... 2024-09-16T00:48:42.927 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386/remote/smithi001/log 2024-09-16T00:48:42.928 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:48:43.790 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386/remote/smithi019/log 2024-09-16T00:48:43.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:48:45.198 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386/remote/smithi153/log 2024-09-16T00:48:45.199 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:48:46.456 INFO:tasks.cephadm:Removing cluster... 2024-09-16T00:48:46.456 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 --force 2024-09-16T00:48:46.668 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:48:46.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 --force 2024-09-16T00:48:47.152 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:48:47.422 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1226ed86-73c1-11ef-bceb-c7b262605968 --force 2024-09-16T00:48:47.617 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 1226ed86-73c1-11ef-bceb-c7b262605968 2024-09-16T00:48:48.124 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T00:48:48.124 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:48:48.145 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:48:48.164 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:48:48.180 INFO:tasks.cephadm:Teardown complete 2024-09-16T00:48:48.181 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T00:48:48.191 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T00:48:48.192 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:48:48.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:48:48.207 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:48:48.323 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T00:48:48.334 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T00:48:48.334 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:48:48.337 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:48:48.339 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:48:48.352 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:48:48.353 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:48:48.354 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:48:48.360 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:48:48.360 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-16T00:48:48.360 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 -200us[ -195us] +/- 55ms 2024-09-16T00:48:48.360 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 31 -1216us[-1216us] +/- 42ms 2024-09-16T00:48:48.361 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +1341us[+1341us] +/- 72ms 2024-09-16T00:48:48.361 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:48:48.361 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:48:48.361 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-09-16T00:48:48.361 INFO:teuthology.orchestra.run.smithi001.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 -128us[ -129us] +/- 55ms 2024-09-16T00:48:48.361 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -1138us[-1138us] +/- 42ms 2024-09-16T00:48:48.362 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +1404us[+1404us] +/- 72ms 2024-09-16T00:48:48.362 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:48:48.363 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:48:48.363 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-09-16T00:48:48.363 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 -15us[ -127us] +/- 55ms 2024-09-16T00:48:48.364 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -1048us[-1161us] +/- 42ms 2024-09-16T00:48:48.364 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 32 +1535us[+1423us] +/- 72ms 2024-09-16T00:48:48.364 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:48:48.364 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T00:48:48.379 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T00:48:48.380 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T00:48:48.407 DEBUG:teuthology.orchestra.run.smithi001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:48:48.505 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:48:48.580 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:48:48.648 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:48:48.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:48:48.681 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:48:48.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:48:48.711 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:48:48.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:48:48.740 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T00:48:48.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T00:48:48.797 INFO:teuthology.task.internal:Duration was 2223.647805 seconds 2024-09-16T00:48:48.797 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T00:48:48.831 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T00:48:48.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:48:48.835 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:48:48.837 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:48:48.876 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:48:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:48:48.880 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:48:49.400 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T00:48:49.400 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-09-16T00:48:49.401 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:48:49.423 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-09-16T00:48:49.423 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:48:49.454 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2024-09-16T00:48:49.454 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:48:49.487 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T00:48:49.487 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:48:49.490 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:48:49.498 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:48:49.554 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T00:48:49.554 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:48:49.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:48:49.570 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:48:51.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T00:48:51.305 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T00:48:51.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:48:51.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:48:51.362 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:48:51.393 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T00:48:51.408 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:48:51.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:48:51.413 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:48:51.432 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-09-16T00:48:51.438 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-09-16T00:48:51.462 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2024-09-16T00:48:51.505 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:48:51.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:48:51.520 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:48:51.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:48:51.564 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:48:51.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:48:51.581 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T00:48:51.592 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T00:48:51.593 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386/remote/smithi001 2024-09-16T00:48:51.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:48:51.668 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386/remote/smithi019 2024-09-16T00:48:51.668 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:48:51.709 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906386/remote/smithi153 2024-09-16T00:48:51.709 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:48:51.750 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T00:48:51.750 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:48:51.753 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:48:51.756 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:48:51.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T00:48:51.817 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T00:48:51.817 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T00:48:51.841 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T00:48:51.841 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:48:51.843 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:48:51.846 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:48:51.857 INFO:teuthology.orchestra.run.smithi001.stdout: 265602 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:48 /home/ubuntu/cephtest 2024-09-16T00:48:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:48 /home/ubuntu/cephtest 2024-09-16T00:48:51.863 INFO:teuthology.orchestra.run.smithi153.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:48 /home/ubuntu/cephtest 2024-09-16T00:48:51.864 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T00:48:51.875 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T00:48:51.950 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2223.647805213928 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T00:48:51.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:48:52.029 INFO:teuthology.run:pass